Intelligent network performance detection method and system

Abstract

The application discloses an intelligent network performance detection method and system, which comprises network data collection, network data set construction, network performance prediction model design, network anomaly judgment and network performance detection. The application belongs to the field of network detection, and specifically relates to an intelligent network performance detection method and system. The first-order difference term is used to constrain fluctuation speed, and the defect of neglecting change speed is compensated. The second-order difference term is used to quantify fluctuation intensity, distinguish sudden change and gradual change, and identify early fault precursors. The combination of deviation loss and fluctuation penalty term improves the identification ability of early risks. The index trend mutation is used to quantify the intensity of fluctuation, and a time-decay mutation upper limit is introduced. Abnormal fluctuation limit filtering is used to filter transient faults and reduce false alarms. Independent fluctuation penalty coefficients are allocated to links to realize high-load link sensitization, low-load link relaxation, adapt to different link performance requirements, and improve detection effect.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of network detection, and particularly relates to an intelligent network performance detection method and system. BACKGROUND

[0002] A network performance detection method is a technical means for monitoring the network operation state in real time by collecting and analyzing key indicators in network operation. However, the general network performance detection method has the problems of ignoring network fluctuation speed and intensity, not quantifying fluctuation difference, missing mutation risk, and thus leading to poor performance detection reliability. The general network performance detection method cannot distinguish between instantaneous failure and continuous anomaly, leading to the problems of ignoring real continuous failure, low link adaptability, and thus leading to poor performance detection effect. SUMMARY

[0003] In view of the above problems, the application provides an intelligent network performance detection method and system to overcome the defects of the prior art. The method can overcome the defects of ignoring the change speed and intensity, not quantifying the fluctuation difference, missing the mutation risk, and thus leading to poor performance detection reliability. The first-order difference term is used to constrain the fluctuation speed, and the second-order difference term is used to quantify the fluctuation intensity, distinguish between mutation and gradual change, and identify the failure precursor in advance. The combination of the deviation loss and the fluctuation penalty term can upgrade the network performance detection from static judgment to dynamic perception, improve the identification ability of early risk, and thus improve the performance detection reliability. The index trend mutation is used to quantify the fluctuation intensity, and the time-decay-type mutation upper limit is introduced to filter the instantaneous failure, reduce the false alarm, and assign independent fluctuation penalty coefficients to the links to realize the sensitization of high-load links, the relaxation of low-load links, adapt to different link performance requirements, and improve the detection effect.

[0004] The technical scheme adopted by the application is as follows: The application provides an intelligent network performance detection method, which comprises the following steps:

[0005] Step S1: network data collection;

[0006] Step S2: network data set construction;

[0007] Step S3: design of a network performance prediction model;

[0008] Step S4: network anomaly judgment;

[0009] Step S5: network performance detection.

[0010] Further, in step S1, the network data collection is to obtain historical network performance data; and the collected original indicators are smoothed by EWMA to obtain indicators of the network performance data after smoothing.

[0011] Further, in step S2, the network data set construction is to perform missing value processing, data screening, time series slicing and label annotation on the network performance data after smoothing; and a network data set is obtained.

[0012] Further, in step S3, the network performance prediction model is designed to take the network data set as input, and performance indicators are predicted through light-weight time series feature extraction; specifically, the step S3 includes:

[0013] Step S31: model architecture design; the model includes an indicator time series input layer, a light-weight time series feature extraction layer, an indicator prediction output layer and a loss calculation layer; the light-weight time series feature extraction layer extracts an indicator trend, and adopts a light-weight LSTM; the indicator prediction output layer maps a time series feature vector into a performance indicator prediction value in a future time window;

[0014] Step S32: network performance loss function design; a discriminative fluctuation penalty term is added in the loss function;

[0015] Step S33: abnormal fluctuation limit; an indicator trend mutation degree is set, when the indicator trend mutation degree is normal, the network performance loss function is unchanged; otherwise, an over-limit penalty factor is added in the fluctuation penalty term;

[0016] Step S34: multi-link self-tuning; different links have different performance requirements, and independent fluctuation penalty coefficients are allocated to different links, and the fluctuation penalty coefficients are dynamically adjusted according to link loads.

[0017] Further, in step S4, the network anomaly determination is to set an abnormal threshold and an instantaneous threshold, if a real-time calculated network performance loss value is higher than the abnormal threshold and a duration is higher than the instantaneous threshold, the network is determined to be abnormal; a model performance is evaluated based on a labeled label; and a network performance prediction model is established.

[0018] Further, in step S5, the network performance detection is to obtain network performance data in real time based on the established network performance prediction model, and the network performance data is input into the network performance prediction model after pre-processing and time series slicing processing, and the network performance detection is performed based on the network anomaly determination.

[0019] The intelligent network performance detection system provided by the application includes a network data collection module, a network data set construction module, a network performance prediction model design module, a network anomaly determination module and a network performance detection module.

[0020] The network data acquisition module acquires historical network performance data and performs smoothing processing;

[0021] The network data set construction module constructs a network data set based on the smoothed historical network performance data;

[0022] The network performance prediction model design module designs a network performance prediction model based on the network data set, with an LSTM feature extraction layer as the architecture, by designing a network performance loss function containing fluctuation penalty and abnormal limit domain;

[0023] The network anomaly judgment module evaluates the performance of the network performance prediction model;

[0024] The network performance detection module realizes real-time network performance detection based on the established network performance prediction model.

[0025] The beneficial effects achieved by the above-mentioned scheme are as follows:

[0026] (1) For the problem that general network performance detection methods ignore network fluctuation speed and severity, do not quantify fluctuation differences, miss sudden change risks, and thus lead to poor performance detection reliability, the scheme constrains fluctuation speed through a first-order difference term, compensates for the defect of ignoring change speed; quantifies fluctuation severity through a second-order difference term, distinguishes sudden change from gradual change, and identifies early fault precursors; through the combination of deviation loss and fluctuation penalty terms, the network performance detection is upgraded from static judgment to dynamic perception, the identification ability of early risks is improved, and thus the performance detection reliability is improved.

[0027] (2) For the problem that general network performance detection methods cannot distinguish between instantaneous faults and continuous anomalies, leading to neglect of real continuous faults, low link adaptability, and poor performance detection effect, the scheme quantifies the severity of fluctuation with index trend mutation, and introduces a time-decaying mutation upper limit, filters instantaneous faults through abnormal fluctuation limit domain, reduces false positives and false negatives; assigns independent fluctuation penalty coefficients to links, realizes sensitive link for high-load links and relaxed link for low-load links, adapts to different link performance requirements, and improves detection effect. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of an intelligent network performance detection method provided by the present application is shown in the figure;

[0029] Figure 2 A schematic diagram of an intelligent network performance detection system provided by the present application is shown in the figure;

[0030] Figure 3 A flowchart of step S3 is shown in the figure.

[0031] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and together with the description serve to explain the application, and do not limit the application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0033] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the system or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0034] Embodiment one, refer to Figure 1 The present application provides an intelligent network performance detection method, which comprises the following steps:

[0035] Step S1: network data acquisition; obtaining historical network performance data and performing smoothing processing;

[0036] Step S2: network data set construction; constructing a network data set based on the smoothed historical network performance data;

[0037] Step S3: design of network performance prediction model; based on the network data set, taking the LSTM feature extraction layer as the architecture, designing a network performance prediction model by designing a network performance loss function containing fluctuation penalty and abnormal limit domain;

[0038] Step S4: network anomaly determination; evaluating the performance of the network performance prediction model;

[0039] Step S5: network performance detection; realizing real-time network performance detection based on the established network performance prediction model.

[0040] Embodiment two, refer to Figure 1The embodiment is based on the above embodiment, in step S1, the network data collection is to acquire historical network performance data, the historical network performance data includes end-to-end delay, link packet loss rate, bandwidth utilization, link load and adjacent link delay; a multi-source collection architecture of probe + terminal + controller is adopted, a NetFlow probe is deployed on a core link to collect bandwidth utilization, link packet loss rate and adjacent link delay; a terminal periodically reports end-to-end delay; an SDN controller collects multi-source data in real time and unifies timestamps; the collected original indexes are smoothed by EWMA to obtain indexes of smoothed historical network performance data, denoted as: ; wherein, is an index value of the smoothed historical network performance data, t is a sampling time; is a smoothing coefficient; is an index value of the collected historical network performance data.

[0041] Embodiment three, refer to Figure 1 The embodiment is based on the above embodiment, in step S2, the network data set construction is to perform missing value processing, data screening, time sequence slicing and label annotation on the smoothed network performance data; the missing value processing is to fill in missing data by using a linear interpolation method; the data screening is to screen all-link data in a period to ensure that the data set contains normal-alarm-exception full-state data; the time sequence slicing is to divide continuous network performance data into samples of fixed length by using a sliding window method; the window includes an input window (historical data) and an output window (prediction target); the label annotation is to mark the link state in a future time window corresponding to the sample, including normal and abnormal; and a network data set is obtained.

[0042] Embodiment four, refer to Figure 1 and Figure 3 The embodiment is based on the above embodiment, in step S3, a network performance prediction model is designed to take the network data set as input, to perform performance index prediction through light-weight time sequence feature extraction; specifically including:

[0043] Step S31: model architecture design; the model includes an index time series input layer, a lightweight time series feature extraction layer, an index prediction output layer, and a loss calculation layer; the index time series input layer converts time series slice samples of a network dataset into tensors that can be processed by the model, eliminates the dimension effect through preprocessing, and outputs normalized time series tensors; the lightweight time series feature extraction layer extracts index change trends and multi-index correlations from the input time series data to support subsequent prediction, adopts lightweight LSTM, includes 2 layers of stacked LSTM, each layer contains a forget gate, an input gate, and an output gate, has no pooling layer, and outputs a time series feature vector; the index prediction output layer maps the time series feature vector into a performance index prediction value in a future time window; the network structure is 1 layer of a full connection layer without a hidden layer; the loss calculation layer calculates a loss based on a network performance loss function for the performance index prediction value and a normal state benchmark; and network parameters are updated based on a loss gradient;

[0044] Step S32: network performance loss function design; a discriminative fluctuation penalty term is added to the loss function to directly constrain the severity of index fluctuation and avoid focusing only on whether the index exceeds a threshold value but ignoring the index fluctuation speed; a severe fluctuation may be a precursor to failure; the network performance loss function is represented as: ; ; ; ; wherein, is a network performance loss; is a bias loss; is a sample in a network dataset; and are a prediction value and a benchmark value of the kth performance index, respectively; is a fluctuation penalty coefficient; n is the total number of performance indexes; is a time interval; is a first-order difference, which measures the index change; is a second-order difference, which quantifies the severity of fluctuation, and preferentially learns the pattern of mutation, i.e., risk, while not affecting the constraint on normal gradual change; is a mutation penalty coefficient.

[0045] By performing the above operations, the general network performance detection method ignores the network fluctuation speed and severity, does not quantify the fluctuation difference, and misses the mutation risk, thereby leading to poor performance detection reliability; the present scheme constrains the fluctuation speed through the first-order difference term to compensate for the defect of ignoring the change speed; quantifies the severity of fluctuation through the second-order difference term to distinguish between mutation and gradual change and identify the precursor to failure in advance; and through the combination of the bias loss and the fluctuation penalty term, the network performance detection is upgraded from static judgment to dynamic perception, the identification ability of early risks is improved, and the performance detection reliability is improved.

[0046] Embodiment five, refer to Figure 1 and Figure 3 , this embodiment is based on the above embodiment, in step S3, the design of network performance prediction model also includes:

[0047] Step S33: abnormal fluctuation limit; network burst failure will cause abnormal jump of index, but it may be instantaneous fault recovery, in order to avoid false alarm, carry out time attenuation type index trend mutation degree limit, let the change contribution of abnormal fluctuation controllable, avoid false alarm; set index trend mutation degree Satisfy When the index trend mutation degree is normal, that is , the network performance loss function is unchanged; otherwise, add an over limit penalty factor in the fluctuation penalty term, and the loss function is expressed as: ; wherein, is the upper limit of mutation degree; is the abnormal duration, which is counted from the first trigger of mutation degree over limit; is the basic mutation degree upper limit; is the attenuation coefficient; filter transient abnormal fluctuation, reduce false alarm, and improve detection accuracy;

[0048] Step S34: multi-link self-tuning ; different links have different performance requirements, so different links are allocated independent , which is adjusted dynamically according to link load; suppose the network has m links, adjust according to link load; the higher the load , the greater , which represents: ; wherein, is the fluctuation penalty coefficient of the ith link; is the dynamically updated fluctuation penalty coefficient; and are link load value and load threshold; is the penalty factor.

[0049] By performing the above operation, for the general network performance detection method, it cannot distinguish between transient fault and continuous abnormality, which leads to neglect of real continuous fault, low link adaptability, and poor performance detection effect. The scheme quantifies the degree of fluctuation of index trend mutation, introduces time attenuation type mutation upper limit, filters transient fault through abnormal fluctuation limit, reduces false alarm and false alarm; allocate independent fluctuation penalty coefficient for link, realize high load link sensitization, low load link relaxation, adapt to different link performance requirements, and improve detection effect.

[0050] Embodiment six, refer to Figure 1The embodiment is based on the above-mentioned embodiment, in step S4, the network anomaly determination is to set an anomaly threshold and a transient threshold, if the real-time calculated network performance loss value is higher than the anomaly threshold and the duration is higher than the transient threshold, the anomaly is determined; the model performance is evaluated based on the labeled label, the accuracy, recall rate and precision are used; the model parameters are adjusted based on the particle swarm search algorithm; and the network performance prediction model is obtained.

[0051] Embodiment seven, refer to Figure 1 The embodiment is based on the above-mentioned embodiment, in step S5, the network performance detection is based on the established network performance prediction model, the network performance data is obtained in real time, after pre-processing and time sequence slicing processing, it is input into the network performance prediction model, the network performance detection is carried out based on the network anomaly determination; if the network anomaly determination is abnormal, the management personnel is warned.

[0052] Embodiment eight, refer to Figure 2 The embodiment is based on the above-mentioned embodiment, the intelligent network performance detection system provided by the application comprises a network data acquisition module, a network data set construction module, a network performance prediction model design module, a network anomaly determination module and a network performance detection module.

[0053] The network data acquisition module acquires historical network performance data and carries out smoothing processing;

[0054] The network data set construction module constructs a network data set based on the smoothed historical network performance data;

[0055] The network performance prediction model design module is based on the network data set, takes the LSTM feature extraction layer as the architecture, designs the network performance prediction model by designing the network performance loss function containing the fluctuation penalty and the abnormal limit domain;

[0056] The network anomaly determination module evaluates the network performance prediction model performance;

[0057] The network performance detection module realizes real-time network performance detection based on the established network performance prediction model.

[0058] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes the elements inherent to such process, method, article or equipment.

[0059] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary and that changes can be made in the embodiments without departing from the spirit and scope of the application.

[0060] The above description of the application and its embodiments is not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the application.

Claims

1. An intelligent network performance testing method, characterized in that: The method includes the following steps: Step S1: Network data acquisition; Obtain historical network performance data and perform smoothing processing; Step S2: Network dataset construction; Construct a network dataset based on smoothed historical network performance data; Step S3: Design a network performance prediction model; Based on the network dataset, using the LSTM feature extraction layer as the architecture, design a network performance prediction model by designing a network performance loss function that includes fluctuation penalty and anomaly constraint. Step S4: Network anomaly detection; performance evaluation of the network performance prediction model; Step S5: Network performance detection; Real-time network performance detection is achieved based on the established network performance prediction model; In step S3, the network performance prediction model is designed using the network dataset as input and performs performance index prediction through lightweight temporal feature extraction; specifically, it includes: Step S31: Model architecture design; The model includes a time series input layer for indicators, a lightweight time series feature extraction layer, an indicator prediction output layer, and a loss calculation layer; The lightweight time series feature extraction layer extracts the trend of indicator changes and uses a lightweight LSTM; The indicator prediction output layer maps the time series feature vector to the predicted value of the performance indicator within the future time window; Step S32: Design of network performance loss function; Add a discriminative fluctuation penalty term to the loss function, expressed as: ; ; ;in, It is deviation loss; These are samples from a network dataset; and These are the predicted and baseline values ​​for the k-th performance metric, respectively; n is the total number of performance metrics. It is a time interval; It is a first-order difference; It is a second-order difference; Step S33: Abnormal fluctuation limit; set the index trend abrupt change degree. satisfy When the trend of the indicator changes normally, that is The network performance loss function remains unchanged; otherwise, an over-limit penalty factor is added to the fluctuation penalty term, and the loss function is expressed as: ;in, It is the upper limit of mutation degree; It is the duration of the anomaly, starting from the first trigger of the mutation degree exceeding the limit; It is the upper limit of the basic mutation degree; It is the attenuation coefficient; This is a loss of network performance; It is the fluctuation penalty coefficient; It is the mutation penalty coefficient; Step S34: Multi-link self-tuning, assigning independent fluctuation penalty coefficients to different links and dynamically adjusting them according to link load; In step S4, the network anomaly determination involves setting an anomaly threshold and an instantaneous threshold. If the real-time calculated network performance loss value is higher than the anomaly threshold and the duration is higher than the instantaneous threshold, it is determined to be an anomaly. The model performance is evaluated based on the labeled data, and the completed network performance prediction model is obtained.

2. The intelligent network performance testing method according to claim 1, characterized in that: In step S2, the network dataset construction involves processing the smoothed network performance data, performing missing value processing, data filtering, time-series slicing, and label annotation to obtain the network dataset.

3. The intelligent network performance testing method according to claim 2, characterized in that: In step S5, the network performance detection is based on the established network performance prediction model. The network performance data is acquired in real time, preprocessed and processed by time-series slicing, and then input into the network performance prediction model. Network performance detection is performed based on network anomaly determination.

4. An intelligent network performance testing system, used to implement the intelligent network performance testing method as described in any one of claims 1-3, characterized in that: It includes a network data acquisition module, a network dataset construction module, a network performance prediction model design module, a network anomaly detection module, and a network performance testing module; The network data acquisition module obtains historical network performance data and performs smoothing processing; The network dataset construction module constructs a network dataset based on smoothed historical network performance data. The network performance prediction model design module is based on the network dataset and uses the LSTM feature extraction layer as the architecture. It designs a network performance prediction model by designing a network performance loss function that includes fluctuation penalty and anomaly constraint. The network anomaly detection module evaluates the performance of the network performance prediction model; The network performance detection module performs real-time network performance detection based on the established network performance prediction model.

Images