A network anomaly detection method and device based on a double-branch model

Abstract

A network anomaly detection method and apparatus based on a dual-branch model is disclosed, relating to wireless communication, artificial intelligence, and big data analysis. First, performance time-series data is acquired from the communication terminal and preprocessed, including oversampling, undersampling, and feature selection. Then, an anomaly detection network is constructed and trained offline. Historical data is input into both the contrastive learning branch and the attention mechanism branch to learn internal correlations, prior correlations, and sequence correlations within the data structure, and to calculate contrastive loss and reconstruction loss. By jointly optimizing the multi-task loss function, the anomaly detection network is obtained. In the online detection phase, the anomaly detection network trained offline is used to detect network anomalies and periodically feedback is provided. Data at network anomaly moments are collected into a historical database, and the parameters of the anomaly detection network are periodically updated.

Description

Technical Field

[0001] This invention relates to the fields of wireless communication, artificial intelligence and big data analysis technology, and in particular to a network anomaly detection method and apparatus based on a dual-branch model. Background Technology

[0002] With the rapid development of 5G technology and the Internet, information and communication networks are becoming increasingly large-scale, and the number of access terminal devices is increasing dramatically. Ensuring the long-term stable operation of these networks has become an urgent problem to solve. Consequently, Artificial Intelligence for IT Operations (AIOps) has become a development trend and research hotspot both domestically and internationally, with many enterprises actively exploring and implementing AIOps. Simultaneously, national policies are promoting the digital and intelligent transformation of industries and fostering the development of the digital economy, providing strong support for the development of the intelligent operations and maintenance industry. AIOps includes functions such as network anomaly detection, root cause analysis, and fault early warning, with anomaly detection being a crucial component. Network management systems constantly collect key performance indicator (KPI) data from various terminals. By analyzing this KPI data, operations and maintenance personnel can determine the network's operational status. Therefore, ensuring the timely and accurate detection of network anomalies from massive amounts of KPI data has become a key challenge for network operations and maintenance.

[0003] Machine learning-based anomaly detection methods are now widely used. One major category uses supervised learning to train models, achieving a certain level of accuracy, but these methods have high requirements for data samples, rely excessively on data labels, and incur high training and detection costs, making them difficult to apply to large-scale internet services. Another major category overcomes the lack of labeled data through unsupervised tasks, learning information representations from complex temporal dynamics, saving labeling costs, and the learned feature representations typically have good generalization and versatility. Faced with the high cost of acquiring data labels in the era of big data, leading to significant operational difficulties, many anomaly detection methods based on unsupervised deep learning have been proposed. However, existing unsupervised deep learning methods struggle to extract useful features from complex network data, fail to consider the relationships between different dimensions of the data, and have relatively poor interpretability. Summary of the Invention

[0004] The primary objective of this invention is to leverage the advantages of artificial intelligence technology to construct a dual-branch anomaly detection network through comparative learning and self-attention mechanisms to solve the aforementioned problems in time series anomaly detection technology. This provides a network anomaly detection method based on a dual-branch model, which can detect network anomalies in a timely and accurate manner, providing assurance for operation and maintenance work and improving network operation and maintenance efficiency and user experience.

[0005] The second objective of this invention is to provide a network anomaly detection device based on a dual-branch model.

[0006] To achieve the above objectives, the present invention provides the following technical solutions.

[0007] A network anomaly detection method based on dual-branch time series includes the following steps:

[0008] 1) Data preprocessing: Obtain KPI data from the communication terminal and perform data preprocessing, including oversampling, undersampling, and feature selection;

[0009] 2) Construct and optimize the anomaly detection network: Input the preprocessed data from step 1) into the offline training module to construct the anomaly detection network and perform offline training. Input historical data into the contrastive learning branch and the attention mechanism branch respectively to learn the internal association, prior association and sequence association of the data structure and calculate the contrastive loss and reconstruction loss. By jointly optimizing the multi-task loss function, calculate the anomaly score to obtain the optimized anomaly detection network.

[0010] 3) Online detection and feedback: In the online detection phase, the anomaly detection network obtained in step 2) is used to perform real-time network anomaly detection. The detection results are periodically fed back, and the anomaly data is collected into the historical database. Based on the periodically fed-back data, the parameters of the anomaly detection network are updated regularly to optimize the detection results.

[0011] In step 1), the KPI data includes various network monitoring data, such as RRC connection establishment success rate, cell RLC layer downlink packet loss rate, same-frequency handover execution success rate, wireless disconnection rate, etc.

[0012] In step 1), the data preprocessing step addresses the data imbalance problem caused by the rarity of anomalies. Specifically, it includes: first, using oversampling techniques, such as Synthetic Minority Oversampling Technique (SMOTE), to insert existing anomaly class samples to create new synthetic anomaly class samples; then, using clustering algorithms, such as K-Means and hierarchical clustering, to cluster the data, and finally, selecting the cluster with the largest number of samples for undersampling to balance the number of anomaly and normal samples; finally, feature selection is performed by calculating the dependency of the class label on each feature and removing the features with the lowest dependency. Here, the commonly used feature selection technique mutual information is used to select features more relevant to the output class. Mutual information-based feature selection involves calculating the mutual information between each feature and the output class, selecting features with high mutual information scores, and discarding features with low mutual information scores. This helps identify and discard all features that have the least or no effect in classification, thereby improving model accuracy, reducing overfitting, and lowering computational costs.

[0013] In step 2), the specific steps for constructing the anomaly detection network include: inputting historical data into the contrastive learning branch and the attention mechanism branch respectively; learning the internal correlation of the data structure through the contrastive learning branch and calculating the contrastive loss; learning the prior correlation and sequence correlation of the data through the attention mechanism branch and calculating the reconstruction loss; and then jointly optimizing the multi-task loss function to calculate the anomaly score and obtain the anomaly detection network.

[0014] In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch, respectively. This includes: the contrastive learning branch contains two sub-networks with identical structures, both using a multilayer perceptron to map input samples to low-dimensional embeddings. The time-series data measured from a single device is defined as X = {x1, x2, ..., x...}. T}, where x i ∈R d Let d represent the observation value at time t, and d be the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each time point is sharded at different locations (the shard size k is determined according to the number of indicators, divided into two parts, a and b), and then combined. This is called data recombination, and the number of combinations is m = d - k + 1. and When these two complementary pairs are positive pairs, all other pairs are negative pairs. This represents the reorganization of data from input X. Through contrastive learning, a feature space is learned, maximizing the similarity between positive samples while minimizing the similarity between negative samples, thereby learning the internal relationships within the data structure. The contrastive loss L... c Calculated using the following cross-entropy loss function:

[0015]

[0016] in, represents the data segment at the i-th time point and the j-th index position, m is the number of combinations after segmentation and recombination, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent the two subnetworks of contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss.

[0017] In step 2), the preprocessed data is input into the contrastive learning branch and the attention mechanism branch, respectively. This includes using a multi-layer self-attention mechanism with a dual-branch structure, since a single-branch self-attention mechanism cannot simultaneously model prior association and sequence association. For prior association, a learnable Gaussian kernel is used to compute the prior with respect to relative temporal distance. A learnable scaling parameter σ is also used on the Gaussian kernel to allow the prior association to adapt to different time series patterns.

[0018] Initialize the attention mechanism parameters: query Q, key K, value V, and σ, where σ represents the learned scale. Generate prior associations based on the learned scale, with the i-th element corresponding to the i-th time point.

[0019] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0020]

[0021] in, Let Q, K, V, and σ represent the parameter matrices of the l-th layer, respectively. Then, rescaling is used to transform the association weights into a discrete distribution, defined as the prior association:

[0022]

[0023] The attention map is normalized along the last dimension using the Softmax function, and each row of Series forms a discrete distribution. Series Association is defined as follows:

[0024]

[0025] Using a self-attention mechanism, the association bias is defined as the symmetric KL divergence between prior association and sequence association, representing the information gain between these two distributions. The reconstruction loss L of the self-attention branch is defined as follows.rec Weighted reconstruction of differences and correlation biases:

[0026]

[0027] in, Describes the reconstruction of sequence x, ||·|| F and ||·|| K Let Frobenius and K norms be represented respectively; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s These represent prior association and sequential association, respectively; θ is used to weigh the loss term, and when θ>0, it amplifies the association bias.

[0028] In step 2), the joint optimization of the multi-task loss function specifically includes defining the joint loss Loss as:

[0029] Loss=λ×L rec +(1-λ)L c

[0030] This is used to balance the joint training process of the self-attention network and the contrastive learning network, to prevent either loss function from becoming too dominant. In the formula, λ represents the weight parameter; L... rec L represents the reconstruction loss of the attention branch; c The dual-branch anomaly detection network, designed with contrastive loss representing the contrastive learning branch, is trained by minimizing the loss function Loss. The weight parameter λ needs to be adjusted based on different datasets. To optimize the weight parameters using hyperparameter optimization frameworks (such as Hyperopt, Optuna, etc.), these frameworks offer rich optimization algorithms and search strategies, supporting parallel evaluation, early stopping, and result visualization, enabling efficient searching for optimal hyperparameter combinations.

[0031] Optionally, we first define the hyperparameter space, which is the range or list of possible values ​​for each hyperparameter. Here, we define a continuous space Λ = {λ1,…,λ1} in (0,1). n The `Study` object (in frameworks like Optuna) is initialized to store and manage all information during the optimization process, including historical trials, optimal parameters, etc.

[0032] Then define the objective function, take a set of hyperparameters as input, and return an F1 score of the model on the validation set:

[0033] f(Λ)→F1

[0034] Where f is the objective function and Λ is the hyperparameter configuration.

[0035] Next, optimization algorithms, such as random search or Bayesian optimization, are run to search for the optimal solution in the hyperparameter space. In each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation.

[0036] Λ t =optimize_algorithm(X,f,t)

[0037] Among them, Λ t `t` represents the hyperparameter configuration in the `t`-th iteration, and `optimize_algorithm` is the optimization algorithm. For each newly generated hyperparameter configuration, the framework runs an evaluation function. This typically involves training a machine learning model using the given hyperparameter configuration and evaluating its performance on a validation set. The results of each iteration (including hyperparameter configuration, scores, etc.) are stored in a `Study` object. The framework uses this information to guide subsequent search processes.

[0038] It should be noted that operations and maintenance personnel need to set iteration termination conditions, such as terminating the iteration if there is no significant performance improvement in n consecutive iterations.

[0039] After the optimization process is complete, the optimal hyperparameter configuration λ can be obtained from the Study object.

[0040] In step 2), calculating the anomaly score specifically includes: designing an anomaly score as an anomaly criterion. Normal data points are highly correlated; therefore, the distance between positive pairs obtained through contrastive learning is smaller than that between negative pairs. The anomaly score is defined as follows:

[0041]

[0042] Here, Softmax refers to the normalized exponential function, and ⊙ represents element-wise multiplication; Represents a sequence The reconstruction, ||·|| F Denotes the Frobenius norm; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s Representing prior association and sequence association respectively; L c This represents the contrastive loss, also known as the cross-entropy loss function. Based on the anomaly score, a hyperparameter threshold δ is used to determine whether a point is an anomaly (y=1) or a non-anomaly (y=0); if the score exceeds the threshold, the output y is considered an anomaly.

[0043]

[0044] Among them, y iThis represents the model output at time point i. This represents the model input at time point i. This represents the anomaly score at the i-th time point. The hyperparameter threshold δ is set by the operations and maintenance personnel based on different scenario data.

[0045] In step 3), the online detection phase includes real-time detection and periodic feedback, specifically: network KPI data is input into the trained anomaly detection network in real time; when a network anomaly is detected, an anomaly alarm is output; the operation and maintenance personnel determine whether the detected network anomaly is a real anomaly; if it is a real network anomaly, the KPI data at this moment is collected into the historical database, and the anomaly detection network parameters are periodically updated.

[0046] The aforementioned network anomaly detection device based on dual-branch time series is provided with a data acquisition module, an offline training module, a real-time detection module, and a feedback module in sequence.

[0047] The data acquisition module is used to acquire time series data of communication terminal performance and model feedback data;

[0048] The offline training module is used to build an anomaly detection model using historical data from the data acquisition module and to periodically update the anomaly detection model.

[0049] The real-time detection module is used for online anomaly detection;

[0050] The feedback module collects abnormal data detected by the real-time detection module, and after the operation and maintenance personnel confirm the abnormality, they input the data to the data acquisition module to expand the database.

[0051] Compared with the prior art, the present invention has the following outstanding technical effects and advantages:

[0052] This invention leverages the advantages of artificial intelligence technology to address the critical issue of network anomaly detection, proposing a network anomaly detection method and apparatus based on a dual-branch model. The core innovation of this invention lies in its unique dual-branch structure design. This design not only allows the model to simultaneously learn the internal correlations of KPI data and the correlations between sequences, but also significantly improves the efficiency of anomaly detection through parallel processing. The dual-branch architecture enables the model to more comprehensively capture anomaly patterns reflected in the data, without requiring complex learning strategies and reducing reliance on manual intervention, thus exhibiting higher reliability and stability in practical applications. In the dual-branch structure, this invention creatively combines the self-learning ability and robustness of contrastive learning with the characteristics of self-attention mechanism in capturing remote dependency information and strong generalization, improving the accuracy of anomaly detection and providing a solid foundation for network operation and maintenance. Furthermore, this invention introduces data augmentation technology based on contrastive learning and a hyperparameter joint optimization framework, effectively enhancing the model's generalization ability, enabling the model to adapt well to new network structures as LTE mobile networks evolve without significant modifications. This invention enables rapid and accurate identification of network anomalies, providing strong support for subsequent operation and maintenance work, improving network optimization efficiency, and enhancing network performance, user service quality, and user experience. Attached Figure Description

[0053] Figure 1 Here is a flowchart of a network anomaly detection method based on dual-branch time series.

[0054] Figure 2 Here is a network diagram of a network anomaly detection model based on a dual-branch time series;

[0055] Figure 3 This is a schematic diagram illustrating the contrastive learning principle in an anomaly detection model network based on a dual-branch time series.

[0056] Figure 4 This is a schematic diagram of a network anomaly detection method based on dual-branch time series.

[0057] Figure 5 This is a diagram of a network anomaly detection device based on a dual-branch time series. Detailed Implementation

[0058] The invention is described in detail below, with examples of the invention shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention.

[0059] One aspect of this invention provides a network anomaly detection method based on a two-branch time series, comprising: first, acquiring performance data from a communication terminal for data preprocessing, including oversampling, undersampling, and feature selection; then, constructing an anomaly detection network for offline training, inputting historical data into a contrastive learning branch and an attention mechanism branch respectively, learning internal correlations, prior correlations, and sequence correlations, and calculating contrastive loss and reconstruction loss, obtaining the anomaly detection network by jointly optimizing a multi-task loss function; in the online detection stage, using the anomaly detection network obtained from the offline stage for detection and periodic feedback, collecting anomaly data into a historical database, and periodically updating the anomaly detection network parameters.

[0060] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific embodiments.

[0061] Figure 1 This is a flowchart illustrating an embodiment of a network anomaly detection method based on dual-branch time series, which includes the following steps:

[0062] S101: Obtain network performance time-series data from the communication terminal. This performance time-series data includes various network monitoring data, such as RRC connection establishment success rate, downlink packet loss rate at the cell RLC layer, intra-frequency handover execution success rate, and wireless call drop rate in LTE networks. Data preprocessing is performed. First, oversampling techniques, such as Synthetic Minority Oversampling (SMOTE), are used to insert existing outlier class samples to create new synthetic outlier class samples. Then, clustering algorithms, such as K-Means and hierarchical clustering, are used to cluster the data. After clustering, the cluster with the largest number of samples is selected for undersampling to balance the number of outlier and normal samples. Finally, feature selection is performed by calculating the dependency of the class label on each feature and removing the features with the lowest dependency. Here, the commonly used feature selection technique mutual information is used to select features more relevant to the output class. Mutual information-based feature selection involves calculating the mutual information between each feature and the output class, selecting features with high mutual information scores, and discarding features with low mutual information scores. This helps identify and discard all features with minimal or no effect in classification, thereby improving model accuracy, reducing overfitting, and lowering computational costs.

[0063] S102, In the offline training phase, an anomaly detection network is constructed. Historical data is input into the contrastive learning branch and the attention mechanism branch respectively. The contrastive loss and reconstruction loss are calculated. By jointly optimizing the multi-task loss function, the final anomaly detection network is obtained for offline training.

[0064] Specifically, the process involves inputting historical data into the contrastive learning branch and the attention mechanism branch respectively. The contrastive learning branch learns the internal relationships within the data structure and calculates the contrastive loss. The attention mechanism branch learns the prior relationships and sequence relationships of the data and calculates the reconstruction loss. Then, the multi-task loss function is jointly optimized to obtain the anomaly detection network.

[0065] The step of inputting the preprocessed data into the contrastive learning branch specifically includes: the contrastive learning branch contains two sub-networks with the same structure, both using a multilayer perceptron to map the input samples to a low-dimensional embedding. The time-series data measured from a single device is defined as X = {x1, x2, ..., x...}. T}, where x i ∈R d Let d represent the observation value at time t, and d be the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each time point is sharded at different locations (the shard size k is determined according to the number of indicators, divided into two parts, a and b), and then combined. This is called data recombination, and the number of combinations is m = d - k + 1. and When these two complementary pairs are positive pairs, all other pairs are negative pairs. This represents the reorganization of data from input X. Through contrastive learning, a feature space is learned, maximizing the similarity between positive samples while minimizing the similarity between negative samples, thereby learning the internal relationships within the data structure. The contrastive loss L... c Calculated using the following cross-entropy loss function:

[0066]

[0067] in, Let represent the data segment at time point i and index position j, m be the number of combinations after segmentation and recombination, exp() refer to the exponential function with the natural constant e as the base, ln() refer to the logarithmic function with e as the base, M(·) and N(·) represent the two sub-networks of contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss. The preprocessed data attention mechanism branching specifically includes: since a single-branch self-attention mechanism cannot simultaneously model prior association and sequence association, a multi-layer self-attention mechanism with a dual-branch structure is used. For prior association, a learnable Gaussian kernel is used to compute the prior with respect to relative temporal distance. A learnable scaling parameter σ is also used on the Gaussian kernel, allowing the prior association to adapt to different time series patterns.

[0068] Initialize the attention mechanism parameters: query Q, key K, value V, and σ, where σ represents the learned scale. Generate prior associations based on the learned scale, with the i-th element corresponding to the i-th time point.

[0069] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0070]

[0071] in, Let Q, K, V, and σ represent the parameter matrices of the l-th layer, respectively. Then, rescaling is used to transform the association weights into a discrete distribution, defined as the prior association:

[0072]

[0073] The attention map is normalized along the last dimension using the Softmax function, and each row of Series forms a discrete distribution. Series Association is defined as follows:

[0074]

[0075] Through a self-attention mechanism, the association bias is defined as the symmetric KL divergence between prior association and sequential association, representing the information gain between these two distributions. The reconstruction loss of the self-attention branch is defined as a weighted average of the reconstruction difference and the association bias.

[0076]

[0077] in, Represents a sequence The reconstruction, ||·|| F and ||·|| K Let Frobenius and K norms be represented respectively; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s These represent prior association and sequential association, respectively; θ is used to weigh the loss term, and when θ>0, it amplifies the association bias.

[0078] The joint loss is defined as:

[0079] Loss=λ×L rec +(1-λ)L c

[0080] To balance the joint training process of the self-attention network and the contrastive learning network and prevent either loss function from becoming too dominant, the designed bi-branch anomaly detection network is trained by minimizing the loss function L. The weight parameters λ need to be adjusted according to different datasets. To optimize the weight parameters using hyperparameter optimization frameworks (such as Hyperopt, Optuna, etc.), these frameworks provide rich optimization algorithms and search strategies, supporting parallel evaluation, early stopping, and result visualization, enabling efficient searching for the optimal hyperparameter combination.

[0081] Optionally, we first define the hyperparameter space, which is the range or list of possible values ​​for each hyperparameter. Here, we define a continuous space Λ = {λ1,…,λ1} in (0,1). n The `Study` object (in frameworks like Optuna) is initialized to store and manage all information during the optimization process, including historical trials, optimal parameters, etc.

[0082] Then define the objective function, take a set of hyperparameters as input, and return an F1 score of the model on the validation set:

[0083] f(Λ)→F1

[0084] Where f is the objective function and Λ is the hyperparameter configuration.

[0085] Next, optimization algorithms, such as random search or Bayesian optimization, are run to search for the optimal solution in the hyperparameter space. In each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation.

[0086] Λ t =optimize_algorithm(Λ,f,t)

[0087] Among them, Λ t `t` represents the hyperparameter configuration in the `t`-th iteration, and `optimize_algorithm` is the optimization algorithm. For each newly generated hyperparameter configuration, the framework runs an evaluation function. This typically involves training a machine learning model using the given hyperparameter configuration and evaluating its performance on a validation set. The results of each iteration (including hyperparameter configuration, scores, etc.) are stored in a `Study` object. The framework uses this information to guide subsequent search processes.

[0088] Operations personnel need to set iteration termination conditions, such as terminating the iteration if there is no significant performance improvement in n consecutive iterations.

[0089] After the optimization process is complete, the optimal hyperparameter configuration λ can be obtained from the Study object.

[0090] After calculating the joint loss, an anomaly score is designed as the anomaly criterion. Normal data points are highly correlated, so the distance between positive pairs of representations obtained through contrastive learning is smaller than that between negative pairs. The anomaly score is defined as follows:

[0091]

[0092] Here, Softmax refers to the normalized exponential function, and ⊙ represents element-wise multiplication; Describes the reconstruction of sequence x, ||·|| F Denotes the Frobenius norm; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s Representing prior association and sequence association respectively; L c This represents the contrast loss, also known as the cross-entropy loss function.

[0093] Based on the anomaly score, a hyperparameter threshold δ is used to determine whether a point is an anomaly (y=1) or a non-anomaly (y=0); if the score exceeds the threshold, the output y is considered an anomaly, i.e.:

[0094]

[0095] Among them, y i This represents the model output at time point i. This represents the model input at time point i. This represents the anomaly score at time point i. The hyperparameter threshold δ is set by the operations and maintenance personnel based on different scenario data. S103: In the online phase, acquire the time series data of the performance to be tested, input it into the online detection module, and perform real-time anomaly detection using the anomaly detection model obtained in the offline phase.

[0096] S104: After real-time anomaly detection, the detected anomaly data is accumulated into the historical database for periodic model updates. Specifically, the operations and maintenance personnel determine whether the detected anomaly is a real anomaly and collect it into the historical database, periodically updating the anomaly detection network parameters.

[0097] like Figure 2 As shown, this embodiment of the invention provides a dual-branch network anomaly detection model including an attention network and a contrastive learning network:

[0098] Specifically, the contrastive learning network uses two encoders with the same structure but different dimensions. The time-series data measured from a single device is defined as X = {x1, x2, ..., x...}. T}, where x i ∈R dLet d represent the observation value at time t, and d be the time series dimension. The original sequence X is copied multiple times, and the multivariate time series data at each time point is sharded at different locations (the shard size k is determined according to the number of indicators, divided into two parts, a and b), and combined. The number of combinations is m = d - k + 1. and When these two complementary pairs are positive pairs, all other pairs are negative pairs. This represents the reorganization of data from input X. Through contrastive learning, a feature space is learned, maximizing the similarity between positive samples while minimizing the similarity between negative samples, thereby learning the internal relationships within the data structure. The contrastive loss is calculated using the following cross-entropy loss function:

[0099]

[0100] in, represents the data segment at the i-th time point and the j-th index position, m is the number of combinations after segmentation and recombination, exp() refers to the exponential function with the natural constant e as the base, ln() refers to the logarithmic function with e as the base, M(·) and N(·) represent the two subnetworks of contrastive learning, and the hyperparameter τ is the temperature constant of the contrastive loss.

[0101] Specifically, in attention networks, since single-branch self-attention mechanisms cannot simultaneously model prior associations and sequential associations, a multi-layer self-attention mechanism with a dual-branch structure is used. For prior associations, a learnable Gaussian kernel is employed to compute priors about relative temporal distances. A learnable scaling parameter σ is also used on the Gaussian kernel, allowing the prior associations to adapt to different time-series patterns.

[0102] Initialize the attention mechanism parameters: query Q, key K, value V, and σ, where σ represents the learned scale. Generate prior associations based on the learned scale, with the i-th element corresponding to the i-th time point.

[0103] Specifically, for the i-th time point, its association weight with the j-th time point is calculated by the Gaussian kernel:

[0104]

[0105] in, Let Q, K, V, and σ represent the parameter matrices of the l-th layer, respectively. Then, rescaling is used to transform the association weights into a discrete distribution, defined as the prior association:

[0106]

[0107] The attention map is normalized along the last dimension using the Softmax function, and each row of Series forms a discrete distribution. Series Association is defined as follows:

[0108]

[0109] Through a self-attention mechanism, the association bias is defined as the symmetric KL divergence between prior association and sequence association, representing the information gain between these two distributions. The reconstruction loss L of the self-attention branch is defined. rec Weighted reconstruction of differences and correlation biases:

[0110]

[0111] in, Describes the reconstruction of sequence x, ||·|| F and ||·|| K Let Frobenius and K norms be represented respectively; L represents the number of attention layers; KL(·||·) represents the KL divergence calculated between two discrete distributions; A p and A s These represent prior association and sequential association, respectively; θ is used to weigh the loss term, and when θ>0, it amplifies the association bias.

[0112] The joint loss is defined as:

[0113] Loss=λ×L rec +(1-λ)L c

[0114] In the formula, λ represents the weighting parameter; L rec L represents the reconstruction loss of the attention branch; c The contrastive loss represents the contrastive learning branch; the joint optimization of the multi-task loss function Loss is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid one loss becoming too large and having a dominant influence; the designed dual-branch anomaly detection network is trained by minimizing the loss function Loss; among them, the weight parameter λ needs to be adjusted according to different datasets.

[0115] like Figure 3 The diagram shown illustrates the contrastive learning principle in network anomaly detection based on dual-branch time series, as provided in this invention.

[0116] Specifically, to learn the internal correlations of multidimensional time series data x, a contrastive learning approach is used. The original sequence x is copied multiple times, and the multidimensional time series data at each time point is pieced at different locations (the piece size k is determined by the number of indicators, dividing it into two parts, a and b), and combined. The number of combinations is m = d - k + 1. As shown in the figure... and In this case, the two complementary pairs form a positive pair, while all other pairs are negative pairs. Through contrastive learning, the distance between positive pairs in the learned feature space is shortened, while the distance between negative pairs is widened. In practical applications, performance metrics are correlated, and different dimensions of the time series represent different metrics. For example, in access problems, wireless connection success rate is related to E-RAB establishment success rate. After segmentation and reassembly, the wireless connection success rate is spliced ​​together with other metric data, which no longer conforms to the original correlation. Through contrastive learning networks, the distance between negative pairs in the feature space is increased, thus learning the internal correlations within the data.

[0117] like Figure 4 The diagram illustrates a network anomaly detection method based on a dual-branch time series model provided in this invention: First, performance time series data is acquired from a communication terminal and preprocessed, including oversampling, undersampling, and feature selection. Then, an anomaly detection network is constructed and trained offline. Historical data is input into the contrastive learning branch and the attention mechanism branch respectively, learning internal correlations, prior correlations, and sequence correlations within the data structure, and calculating contrastive loss and reconstruction loss. The anomaly detection network is obtained by jointly optimizing the multi-task loss function. In the online detection phase, the anomaly detection network trained offline is used for detection and periodic feedback is provided. Anomaly data is collected into a historical database, and the parameters of the anomaly detection network are periodically updated.

[0118] This invention proposes a mobile handover performance optimization device based on a wireless access network intelligent controller, such as... Figure 5 As shown, the device includes: a data acquisition module, an offline training module, a real-time detection module, and a feedback module;

[0119] The data acquisition module is used to acquire time series data of communication terminal performance and model feedback data;

[0120] The offline training module is used to build an anomaly detection model using historical data from the data acquisition module and to periodically update the anomaly detection model.

[0121] The real-time detection module is used for online anomaly detection;

[0122] The feedback module collects abnormal data detected by the real-time detection module, and after the operation and maintenance personnel confirm the abnormality, they input the data to the data acquisition module to expand the database.

[0123] In summary, the network anomaly detection method based on a dual-branch time series according to embodiments of the present invention includes the following steps: First, performance time series data is acquired from the communication terminal and preprocessed, including oversampling, undersampling, and feature selection; then, an anomaly detection network is constructed and trained offline, with historical data input into the contrastive learning branch and the attention mechanism branch respectively, learning the internal correlation, prior correlation, and sequence correlation of the data structure, and calculating the contrastive loss and reconstruction loss. The anomaly detection network is obtained by jointly optimizing the multi-task loss function; in the online detection stage, the anomaly detection network obtained in the offline stage is used for detection and periodic feedback is performed, collecting anomaly data into the historical database, and periodically updating the anomaly detection network parameters. This invention can quickly and accurately identify network anomalies, providing strong support for subsequent operation and maintenance work, improving network optimization efficiency, and enhancing network performance, user service quality, and user experience. The proposed invention can improve the efficiency of anomaly detection in traditional operations and maintenance (O&M). It utilizes intelligent methods to analyze large amounts of network data. Compared to existing methods, it considers both the time dependency and internal structural relationships of the data, resulting in more comprehensive and reasonable feature extraction. This leads to improvements in network anomaly detection precision, accuracy, recall, and F1 score, and it is applicable to a wider range of scenarios. This demonstrates that the proposed method has a stronger ability to distinguish network anomalies and also exhibits good anomaly detection capabilities for imbalanced KPI data. It helps O&M personnel to discover and handle network anomalies more quickly and accurately, achieving efficient, accurate, and reliable network O&M goals.

[0124] The above embodiments are merely preferred embodiments of the present invention and should not be considered as limiting the scope of the present invention. All equivalent variations and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A network anomaly detection method based on a two-branch model, characterized by The method comprises the following steps: 1) data preprocessing: obtaining KPI data from a communication terminal for data preprocessing, including oversampling, undersampling and feature selection; 2) constructing and optimizing an anomaly detection network: inputting the preprocessed data of step 1) into an offline training module to construct an anomaly detection network and perform offline training, inputting the preprocessed data into a contrastive learning branch and an attention mechanism branch respectively, learning data structure internal association, prior association and sequence association and calculating contrastive loss and reconstruction loss, calculating an anomaly score by jointly optimizing a multi-task loss function, and obtaining an optimized anomaly detection network; The preprocessed data is respectively input into a contrast learning branch and an attention mechanism branch, wherein the contrast learning branch comprises two subnetworks with the same structure, both of which use a multilayer perceptron to map an input sample to a low-dimensional embedding; time series data measured from a single device is defined as wherein x i ∈R d represents an observation value at time t, d is a time series dimension; the original sequence is copied multiple times, and the multivariate time series data at each time is sliced and combined at different positions, that is, data reorganization; the slice size k is determined according to the number of indexes, and the slice is divided into s and b two parts, and the combination number is m=d-k+1; and When j=j', the two complementary combinations are a pair of positive examples, and the remaining combinations are negative example pairs, represents data reorganization of X; a feature space is learned through contrast learning, the similarity between positive samples is maximized, and the similarity between negative samples is minimized, so as to learn the internal association of the data structure; the contrast loss L c is calculated by the following cross-entropy loss function: wherein, represents the patch data at the j-th index position of the i-th time point, m is the combined number after patch reorganization, exp() represents the exponential function with the natural constant e as the base, ln() represents the logarithmic function with e as the base, M(·) and N(·) represent two sub-networks of contrast learning, and the hyperparameter τ is the temperature constant of the contrast loss; The preprocessed data is inputted into the contrastive learning branch and the attention mechanism branch, which comprises: using a multi-layer self-attention mechanism with a double-branch structure because a single-branch self-attention mechanism cannot model prior association and sequence association at the same time; for prior association, a learnable Gaussian kernel is used to calculate the prior with respect to the relative time distance; a learnable scale parameter sigma is also used for the Gaussian kernel, so that the prior association can adapt to different time sequence patterns; The attention mechanism parameters query Q, key K, value V and sigma are initialized, wherein sigma represents the learned scale, the prior association is generated based on the learned scale, and the i-th element corresponds to the i-th time point; Specifically, for the i-th time point, the association weight between the i-th time point and the j-th time point is calculated by the Gaussian kernel: where, respectively, denote the parameter matrices of Q, K, V, σ in the l-th layer. Then the relevance weights are transformed into a discrete distribution using re-scaling, defined as the prior relevance: The attention map is normalized along the last dimension using the Softmax function, and each row Series forms a discrete distribution, and the sequence association is defined as: By self-attention mechanism, define the correlation bias as the symmetric KL divergence between the prior correlation and the sequence correlation, and the correlation bias represents the information gain between the two distributions; define the reconstruction loss L of the self-attention branch rec Weighting of reconstruction difference and correlation bias: where, denotes the reconstruction of the sequence ‖·‖ F and ‖·‖ K denote the Frobenius and K-norms, respectively; L denotes the number of attention layers; KL(·||·) denotes the KL divergence computed between two discrete distributions; A p and A s represent the prior and sequence correlations, respectively; θ is used to weigh the loss term, which expands the correlation bias when θ>0; 3) online detection and feedback: in the online detection stage, the anomaly detection network obtained in step 2) is used for real-time network anomaly detection, and the detection result is periodically fed back, the abnormal data is collected to the historical database, and the anomaly detection network parameters are updated regularly according to the periodically fed back data, and the detection result is optimized.

2. The network anomaly detection method based on a two-branch model according to claim 1, wherein In step 1), the KPI data in step 1) includes RRC connection establishment success rate, cell RLC layer downlink packet loss rate, same frequency handover execution success rate and wireless drop rate.

3. The network anomaly detection method based on a two-branch model of claim 1, wherein In step 1), the data preprocessing in step 1) is used to solve the data imbalance problem caused by the rarity of anomalies, the data preprocessing balances the data set through oversampling and undersampling, optimizes the feature set through feature selection, and specifically includes oversampling, clustering and undersampling, feature selection, and feature selection based on mutual information; The oversampling inserts existing abnormal class samples to create new synthetic abnormal class samples; the clustering and undersampling first use a clustering algorithm to cluster the data, select the cluster containing the most samples after clustering for undersampling, and balance the number of abnormal and normal samples; the feature selection removes the feature with the smallest dependency degree by calculating the dependency of each feature on the class label; the feature selection based on mutual information uses mutual information to select features more related to the output class, selects features with high mutual information scores, and discards features with low mutual information scores; to improve the accuracy of the model, reduce overfitting and reduce the computational cost.

4. The network anomaly detection method based on a two-branch model of claim 1, wherein In step 2), the specific steps of constructing the anomaly detection network include: inputting the historical data into the contrast learning branch and the attention mechanism branch respectively, learning the internal association of the data structure through the contrast learning branch, and calculating the contrast loss; learning the prior association and sequence association of the data through the attention mechanism branch, and calculating the reconstruction loss; then jointly optimizing the multi-task loss function, calculating the anomaly score, and obtaining the anomaly detection network.

5. The network anomaly detection method based on a two-branch model of claim 1, wherein In step 2), the joint optimization of the multi-task loss function specifically includes: defining the joint loss Loss as: Loss = λ x L rec + (1 - λ) L c where λ represents a weight parameter; L rec represents the reconstruction loss of the attention branch; L c represents the contrastive loss of the contrastive learning branch; the joint optimization multi-task loss function Loss is used to balance the joint training process of the self-attention network and the contrastive learning network to avoid a certain loss being too large to have a dominant influence; the designed double-branch anomaly detection network is trained by minimizing the loss function Loss; wherein the weight parameter λ needs to be adjusted according to different data sets; in order to use the hyperparameter optimization framework to optimize the weight parameter, the hyperparameter optimization framework provides rich optimization algorithms and search strategies, supports parallel evaluation, early stopping, and result visualization functions, and can search for the optimal hyperparameter combination; (1) Define the hyperparameter space, i.e. the range or list of values each hyperparameter can take, define the continuous space Λ = {λ1,..., λ n}; Initialize the Study object for storing and managing all information during the optimization process, including historical trials, best parameters; (2) define the objective function, a set of hyperparameters as input, and return the F1 score of a model on the validation set: f(Λ)→F1 Where f is the objective function, and Λ is the hyperparameter configuration; (3) run the optimization algorithm to search for the optimal solution in the hyperparameter space; in each iteration, the optimization algorithm generates a new hyperparameter configuration and passes it to the objective function for evaluation; Λ t = optimize_algorithm(X, f, t) where Λ t is the hyperparameter configuration in the t-th iteration, optimize algorithm is the optimization algorithm; for each newly generated hyperparameter configuration, the framework runs the objective function for evaluation; this involves training a machine learning model using the given hyperparameter configuration and evaluating its performance on a validation set; the results of each iteration are stored into the Study object, including the hyperparameter configuration, the score; the framework uses this information to guide the subsequent search process; (4) after the optimization process is completed, obtain the best hyperparameter configuration λ from the Study object.

6. The network anomaly detection method based on a two-branch model of claim 1, wherein In step 2), the calculation of the anomaly score specifically includes: designing an anomaly score as an anomaly standard, and the normal data points have strong correlation, so the distance between positive examples obtained by contrast learning is smaller than that between negative examples, and the anomaly score is defined as follows: where Softmax refers to a normalized exponential function, and represents element-wise multiplication; L c represents the contrast loss, i.e., the cross-entropy loss function; Based on the anomaly score, use the hyperparameter threshold δ to determine whether a point is abnormal or not; if the score exceeds the threshold, output y as abnormal, that is: where y i represents the model output at the i-th time point, represents the anomaly score at the i-th time point, and the hyperparameter threshold value δ is set by the operation and maintenance personnel according to different scene data.

7. The network anomaly detection method based on a two-branch model of claim 1, wherein In step 3), the online detection phase, the anomaly detection network obtained in step 2) is used for real-time network anomaly detection, specifically including: Network KPI data is input into the trained anomaly detection network in real time, and abnormal alarm is output when network anomaly is detected, and the operation and maintenance personnel determine whether the detected network anomaly is a real anomaly, if it is a real network anomaly, collect the KPI data at this moment to the historical database, and periodically update the anomaly detection network parameters.

8. A dual-branch time series based network anomaly detection apparatus, characterized by The device adopts the network anomaly detection method based on the dual-branch model according to any one of claims 1-7, and is sequentially provided with a data acquisition module, an offline training module, a real-time detection module, and a feedback module. The data acquisition module is used to acquire communication terminal performance time series data and model feedback data. The offline training module is used to construct an anomaly detection model through historical data of the data acquisition module, and periodically update the anomaly detection model. The real-time detection module is used for online anomaly detection. The feedback module collects anomaly data detected by the real-time detection module, and inputs the data into the data acquisition module to expand the database after the operation and maintenance personnel confirm the anomaly.

Images