Multi-mode network flow prediction system

Abstract

According to the multi-mode network flow prediction system, the flow detection module detects data parameters of a multi-mode sub-network, the data parameters enter the flow acquisition module after data cleaning, Z-score parameter anomaly detection and chi-square test, anomaly detection and classification are performed according to a threshold value, a change rate and duration of the flow parameters, when flow is abnormal, adjacent sub-network flow parameters are acquired, and the flow of the adjacent sub-network is predicted. Verifying the credibility of the abnormal traffic; the historical traffic acquisition module calls historical traffic data of the same sub-network and sets weights according to the time sequence and the space sequence; the traffic prediction module adopts a time sequence analysis method to predict the traffic trend of the sub-networks, adopts a data fusion algorithm to integrate and process the traffic of the plurality of sub-networks, predicts the total traffic, enters the resource optimization module, and adopts a congestion control algorithm to optimize the performance of the sub-networks and judge the used resources and residual resources of the adjacent sub-networks; and one sub-network link is selected to be distributed to an adjacent sub-network link for optimizing task configuration, so that sub-network flow balancing is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic detection, and in particular relates to a multi-modal network traffic prediction system. Background Art

[0002] A multimodal network is a deep learning model that can process multimodal data. It is usually composed of multiple sub-networks, each of which is responsible for processing different types of data. These sub-networks can be independent or interconnected.

[0003] In order to provide network services with good performance, it is necessary to predict the overall traffic generated in multiple modal networks and optimize the allocation of network resources in a timely manner according to the actual network conditions. The existing technology detects network traffic through network traffic monitoring software, and the server automatically adjusts the bandwidth according to the preset temporary bandwidth adjustment rules. For example, within a specified time period, according to the real-time network traffic situation, the bandwidth is automatically increased or decreased to meet the needs of users, which can avoid unnecessary waste of resources and improve resource utilization. However, it cannot predict network traffic, and dynamically link between sub-networks and optimize task configuration adjustments based on the predicted network traffic and the resource conditions of other sub-networks. Resource utilization needs to be improved. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a multimodal network traffic prediction system that can predict network traffic and dynamically adjust the links between sub-networks and optimize task configurations based on the predicted network traffic and the resource conditions of other sub-networks, thereby effectively improving resource utilization.

[0005] The technical solution is to include a traffic detection module, a traffic acquisition module, a historical traffic acquisition module, and a traffic prediction module. The traffic detection module detects the data output speed, data input speed, bandwidth, and bandwidth utilization of the multimodal sub-network, and enters the traffic acquisition module after data cleaning, Z-score parameter anomaly detection, and chi-square test;

[0006] The traffic acquisition module adopts the depth-first strategy DFS to actively acquire the data of the sub-network, performs anomaly detection and classification according to the threshold, change rate and duration of the traffic parameters, and divides it into normal traffic and abnormal traffic. When there is abnormal traffic, the traffic parameters of the adjacent sub-network are obtained to verify the credibility of the abnormal traffic;

[0007] The historical traffic acquisition module retrieves historical traffic data of the same sub-network and sets weights according to the time series and the space series respectively;

[0008] The traffic prediction module adopts a time series analysis method to predict the sub-network traffic trend, and adopts a data fusion algorithm to integrate and process the traffic of multiple sub-networks to predict the overall traffic.

[0009] Preferably, the traffic prediction module adopts a time series analysis method to predict the sub-network traffic trend, and adopts a data fusion algorithm to integrate and process the traffic of multiple sub-networks to predict the overall traffic. The specific steps are:

[0010] Step 1, obtain the traffic data and historical traffic data of the sub-network;

[0011] Step 2: Set the traffic data y of the subnetwork t The weight is 60%, historical traffic data y t -1 weight 20%, historical traffic data y t-2 The weight of yt-3 is 10%, and the weight of historical traffic data yt-3 is 10%;

[0012] Step 3, the traffic data weight of the sub-network is modified by the credibility;

[0013] Step 4: Calculate the weighted average to get the sub-network traffic prediction data y t+1 ;

[0014] Step 5, obtaining the degree of deviation between the historical predicted value and the measured value;

[0015] Step 6: Correct the sub-network traffic prediction data y t+1 ;

[0016] Step 7: Obtain the traffic prediction data y1 for each sub-network t+1 ,y2 t+1 ,y3 t+1 …yn t+1 ;

[0017] Step 8: Cluster the traffic types of each sub-network into regular traffic yc and burst traffic yz according to the threshold and bandwidth of the traffic prediction data of each sub-network;

[0018] The conventional traffic set adopts the equal weight model yc=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1 )*60%, the weights are all 60 / n%;

[0019] The burst traffic set uses the combined prediction model yz=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1)*40%+r, where r is the burst traffic duration and density bonus, r=burst traffic duration*density / 50%.

[0020] Step 9, regular traffic prediction model + burst traffic prediction model = total traffic prediction value;

[0021] Step 10, modifying the total flow forecast value according to the historical total flow forecast value, with weights of 25% and 75% respectively, and then calculating the weighted average;

[0022] Step 11, the revised total flow forecast value, first calculate the rate of change, and then use the exponential smoothing forecasting algorithm to obtain it.

[0023] Preferably, the resource optimization module uses a congestion control algorithm to adjust the subnetwork data output speed and data input speed according to the subnetwork traffic and change trends, overall traffic, and data classification, optimizes the subnetwork performance, determines the used resources and remaining resources of the adjacent subnetwork, and selects one to allocate to the adjacent subnetwork link for optimization task configuration to achieve subnetwork traffic balance.

[0024] Preferably, the sub-network link optimization task configuration is specifically as follows:

[0025] Step 1: Each sub-network has specific available resources: bandwidth, CPU speed, memory, hard disk speed, and 10% resource redundancy backup is guaranteed.

[0026] C4:

[0027] Among them, CPU and Memory are the CPU and memory of the subnetwork respectively;

[0028] Step 2: Determine the resources required for the task based on the traffic forecast of the current sub-network;

[0029] Step 3: predict the traffic based on the neighboring subnetworks and determine the remaining resources required for the task;

[0030] Step 4, according to the resources required by the task, the current sub-network resources, and the remaining resources of the adjacent sub-network, at least one required adjacent sub-network is obtained;

[0031] Step 5: After determining the adjacent sub-network, the link topology of the data task is completed.

[0032] The beneficial effects of the present invention are as follows: 1. The time series analysis method is used to weight the sub-network traffic and historical traffic. The weight of the sub-network traffic data is corrected by the credibility. The sub-network traffic is calculated by weighted average, and the historical change rate*sampling period is superimposed to obtain the predicted data. The predicted data is corrected by the historical deviation degree, so as to obtain the predicted data of each sub-network traffic, and the threshold and bandwidth of each sub-network traffic prediction data are clustered into regular traffic and burst traffic for each sub-network traffic type, and the equal weight model and the combined prediction model are used for calculation respectively. Finally, the regular traffic prediction model+burst traffic prediction model=total traffic prediction value, and the total traffic prediction value is corrected according to the historical total traffic prediction value, and the change rate is calculated, and then the exponential smoothing prediction algorithm is used to obtain it. The traffic data prediction is convenient for timely resource optimization according to the predicted overall traffic, and the resource optimization is realized by the resource optimization module;

[0033] 2. According to the sub-network traffic and its changing trend, overall traffic, and data classification, a congestion control algorithm is used to adjust the sub-network data output speed and data input speed to optimize the sub-network performance, determine the used resources and remaining resources of the adjacent sub-network, and select one to allocate to the adjacent sub-network link for optimized task configuration to achieve sub-network traffic balance. According to the predicted network traffic and the resource conditions of other sub-networks, the links between sub-networks can be dynamically adjusted to optimize task configuration, which can further improve resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a module diagram of the present invention.

[0035] Figure 2 It is a step diagram of the flow prediction module of the present invention.

[0036] Figure 3 It is a step diagram for inventing a resource optimization module. DETAILED DESCRIPTION

[0037] The technical solutions of various embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0038] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0039] A multimodal network traffic prediction system includes a traffic detection module, a traffic acquisition module, a historical traffic acquisition module, and a traffic prediction module. The traffic detection module detects the data output speed, data input speed, bandwidth, and bandwidth utilization of the multimodal sub-network, which can be obtained through software such as NetFlow analyzer and network bandwidth monitoring software, or obtained by detecting the sub-network computer CPU through a tester. After data cleaning, the missing value of the data output speed, data input speed, bandwidth, and bandwidth utilization of the reaction traffic parameters is interpolated and compensated nearby to ensure the integrity of the data.

[0040] Data cleaning includes the processing of redundant values, specifically filtering and deduplication, filtering out duplicate data and reducing the amount of data;

[0041] Then, the Z-score parameter anomaly detection is performed. Specifically, the deviation between the traffic parameter data (data outgoing speed, data incoming speed, bandwidth utilization) and the mean is calculated to determine whether it is abnormal. Z-score = 3 is set as the threshold to eliminate abnormal points.

[0042] Finally, the correlation between traffic flow, traffic parameters and weights is verified by the chi-square test. If it meets the requirements, it is allowed to enter the traffic acquisition module. Otherwise, the weight of the traffic parameters is adjusted. Specifically, the data output speed, data input speed and bandwidth utilization are set as independent variables, and the traffic flow is the dependent variable. The weight coefficients of the independent variables are set to 30%, 30% and 40% respectively, and the dependent variable is obtained to obtain the theoretical inference value R;

[0043] The degree of deviation is calculated by the chi-square test formula of the actual detection value C and the theoretical inference value R, df = (C-1)(R-1). The closer the degree of deviation is to 1, the higher the correlation between the independent variable and its weight coefficient and the dependent variable. When the correlation is lower than 0.7, the weight coefficient of the independent variable is adjusted, and finally the flow data is obtained by weighted calculation;

[0044] The traffic acquisition module adopts the depth-first strategy DFS to actively acquire the data of the sub-network, and sets the sub-network with abnormal historical traffic data as the root node. According to the spatial layout of the sub-network, priority is given to acquisition until more than two normal sub-networks are encountered, and backtracking is performed to first acquire and process the abnormal data to facilitate subsequent resource regulation. It can also be acquired according to the preset data sampling period, and anomaly detection and classification are performed according to the threshold, change rate, and duration of the traffic parameter, and divided into normal traffic and abnormal traffic. Specifically, first pre-classify by screening the traffic data; calculate the traffic prediction value for the next time by using the traffic data change rate in the previous one or several time slices; according to the duration of the prediction value, that is, after removing the disturbance; compare the traffic prediction value with the upper threshold and classify it; in case of abnormal traffic, obtain the traffic parameters of the adjacent sub-network. Since the resources of the adjacent sub-network will be allocated when the traffic data is abnormal, the traffic data will also increase and change. Therefore, the credibility of the abnormal traffic is verified by detecting the traffic parameters of the adjacent sub-network;

[0045] The historical traffic acquisition module retrieves the historical traffic data of the same sub-network for longitudinal prediction of the sub-network traffic trend, retrieves the adjacent sub-network data at the same time for horizontal comparison of related sub-networks, verifies the credibility of the predicted sub-network traffic trend, and sets weights according to the time series and space series, with larger weights for close time and space.

[0046] The traffic prediction module uses a time series analysis method to predict the sub-network traffic trend and obtain the predicted traffic of each sub-network. It uses a data fusion algorithm to integrate and process the predicted traffic of multiple sub-networks to obtain the predicted overall traffic, which is convenient for timely resource optimization based on the predicted overall traffic.

[0047] Furthermore, it also includes a resource optimization module, which uses a congestion control algorithm to adjust the subnetwork data output speed and data input speed according to the subnetwork traffic and changing trends, overall traffic, and data classification, optimizes the subnetwork performance, determines the used resources and remaining resources of the adjacent subnetwork, and selects one to be allocated to the adjacent subnetwork link for optimization task configuration, thereby achieving subnetwork traffic balance, and dynamically adjusting the links between subnetworks and optimizing task configuration according to the predicted network traffic and the resource conditions of other subnetworks, which can further improve resource utilization.

[0048] Furthermore, the sub-network link optimization task configuration is specifically as follows:

[0049] Step 1: Each sub-network has specific available resources: bandwidth, CPU speed, memory, hard disk speed, and 10% resource redundancy backup is guaranteed.

[0050] C4:

[0051] Among them, CPU and Memory are the CPU and memory of the subnetwork respectively, i represents the subnetwork, t represents the task being executed, available represents available resources, and required represents required resources;

[0052] Step 2: Determine the resources required for the task based on the traffic forecast of the current sub-network;

[0053] Step 3: predict the traffic based on the neighboring subnetworks and determine the remaining resources required for the task;

[0054] Step 4, according to the resources required by the task, the current sub-network resources, and the remaining resources of the adjacent sub-network, at least one required adjacent sub-network is obtained;

[0055] Step 5: After determining the adjacent sub-network, the link topology of the data task is completed.

[0056] Furthermore, when abnormal traffic is detected, a Softmax classifier is used to further confirm whether the abnormal traffic is normal data based on the temporal characteristics and spatial feature areas of the network traffic.

[0057] Furthermore, the traffic prediction module uses a time series analysis method to predict the sub-network traffic trend, and uses a data fusion algorithm to integrate and process the traffic of multiple sub-networks to predict the overall traffic. The specific steps are:

[0058] Step 1, obtain the traffic data and historical traffic data of the sub-network;

[0059] Step 2: Set the traffic data y of the subnetwork t The weight is 60%, historical traffic data y t -1 weight 20%, historical traffic data y t -2 weight 10%, historical traffic data y t -3 weight 10%;

[0060] Step 3, the traffic data weight of the sub-network is modified by the credibility;

[0061] Step 4: Calculate the sub-network traffic by weighted average and superimpose the historical change rate (y t -y t -1*) / y t -1*100%*sampling period, get the predicted data y t+1 ;

[0062] Step 5, obtaining the degree of deviation between the historical predicted value and the measured value;

[0063] Step 6: Correct the sub-network traffic prediction data y t+1 ;

[0064] Step 7: Obtain the traffic prediction data y1 for each sub-network t+1 ,y2 t+1 ,y3 t+1 …yn t+1 ;

[0065] Step 8: Cluster the traffic types of each sub-network into regular traffic yc and burst traffic yz according to the threshold and bandwidth of the traffic prediction data of each sub-network;

[0066] The conventional traffic set adopts the equal weight model yc=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1 )*60%, the weights are all 60 / n%;

[0067] The burst traffic set uses the combined prediction model yz=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1 )*40%+r, where r is the burst traffic duration and density bonus, r=burst traffic duration*density / 50%.

[0068] Step 9, regular traffic prediction model + burst traffic prediction model = total traffic prediction value;

[0069] Step 10, modifying the total flow forecast value according to the historical total flow forecast value, with weights of 25% and 75% respectively, and then calculating the weighted average;

[0070] Step 11, the revised total flow forecast value, first calculates the rate of change, and then uses the exponential smoothing forecasting algorithm to obtain it, so as to predict the flow data.

[0071] When the present invention is used specifically, the traffic detection module detects the data output speed, data input speed, bandwidth, and bandwidth utilization of the multimodal sub-network, performs data cleaning through the nearest interpolation compensation of missing values ​​and filtering of redundant values, and then performs Z-score parameter anomaly detection and removes abnormal points. Finally, the correlation between traffic and traffic parameters and weights is verified through the chi-square test. If it meets the requirements, it is allowed to enter the traffic acquisition module. Otherwise, the weight of the traffic parameter is adjusted. Finally, the traffic data is obtained by weighted calculation and enters the traffic acquisition module. The depth-first strategy DFS is adopted to actively obtain the data of the sub-network. Here, the sub-network with abnormal historical traffic data is set as the root node. According to the spatial layout of the sub-network, priority is obtained until more than two normal sub-networks are encountered. Backtracking is performed to first obtain and process the abnormal data to facilitate subsequent resource regulation. Anomaly detection and classification are performed according to the threshold, change rate, and duration of the traffic parameter, and it is divided into normal traffic and abnormal traffic. When there is abnormal traffic, the traffic parameters of the adjacent sub-network are obtained, and the credibility of the abnormal traffic is verified by detecting the traffic parameters of the adjacent sub-network. The consistency, stability and reliability of the results of the two are the higher the credibility, the closer the data is to reality and reliability;

[0072] The historical traffic acquisition module retrieves the historical traffic data of the same sub-network for vertical prediction of the sub-network traffic trend, retrieves the adjacent sub-network data at the same time for horizontal comparison of related sub-networks, verifies the credibility of the predicted sub-network traffic trend, and sets weights according to the time series and space series. The ones with close time and space have large weights and enter the traffic prediction module.

[0073] The traffic prediction module uses the time series analysis method to weight the sub-network traffic and historical traffic. The weight of the sub-network traffic data is modified by the credibility. The sub-network traffic is calculated by weighted average and the historical change rate (y t -y t-1* ) / y t-1*100%*sampling period, get the prediction data, the prediction data is corrected by the historical deviation degree, so as to get the traffic prediction data of each sub-network, and the threshold and bandwidth of each sub-network traffic prediction data, cluster the traffic type of each sub-network into regular traffic and burst traffic, respectively adopt the equal weight model and the combined prediction model to calculate, finally the regular traffic prediction model + burst traffic prediction model = total traffic prediction value, the total traffic prediction value is corrected according to the historical total traffic prediction value, and then the change rate is calculated, and then the exponential smoothing prediction algorithm is used to get it, so that the traffic data prediction is convenient for timely resource optimization according to the predicted overall traffic, and the resource optimization is realized by the resource optimization module, specifically, according to the sub-network traffic and change trend, overall traffic, data classification, the congestion control algorithm is used to adjust the sub-network data output speed and data input speed, optimize the sub-network performance, judge the used resources and remaining resources of the adjacent sub-network, select one to allocate to the adjacent sub-network link for optimization task configuration, realize the sub-network traffic balance, realize the dynamic optimization task configuration adjustment between the links of the sub-networks according to the predicted network traffic and the resource conditions of other sub-networks, and further improve the resource utilization rate.

[0074] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multimodal network traffic prediction system, comprising a traffic detection module, a traffic acquisition module, a historical traffic acquisition module, and a traffic prediction module, characterized in that: The traffic detection module detects the data output speed, data input speed, bandwidth, and bandwidth utilization of the multimodal sub-network, and enters the traffic acquisition module after data cleaning, Z-score parameter anomaly detection, and chi-square test; The traffic acquisition module adopts the depth-first strategy DFS to actively acquire the data of the sub-network, performs anomaly detection and classification according to the threshold, change rate and duration of the traffic parameters, and divides it into normal traffic and abnormal traffic. When there is abnormal traffic, the traffic parameters of the adjacent sub-network are obtained to verify the credibility of the abnormal traffic; The historical traffic acquisition module retrieves historical traffic data of the same sub-network and sets weights according to the time series and the space series respectively; The traffic prediction module adopts a time series analysis method to predict the sub-network traffic trend, and adopts a data fusion algorithm to integrate and process the traffic of multiple sub-networks to predict the overall traffic.

2. The multimodal network traffic prediction system according to claim 1, characterized in that: It also includes a resource optimization module, which uses a congestion control algorithm to adjust the subnetwork data output speed and data input speed according to the subnetwork traffic and changing trends, overall traffic, and data classification, optimizes the subnetwork performance, determines the used resources and remaining resources of the adjacent subnetwork, and selects one to allocate to the adjacent subnetwork link for optimized task configuration to achieve subnetwork traffic balance.

3. The multimodal network traffic prediction system according to claim 2, characterized in that: The sub-network link optimization task configuration is specifically as follows: Step 1: Each sub-network has specific available resources: bandwidth, CPU speed, memory, hard disk speed, and 10% resource redundancy backup is guaranteed. Among them, CPU and Memory are the CPU and memory of the subnetwork respectively; Step 2: Determine the resources required for the task based on the traffic forecast of the current sub-network; Step 3: predict the traffic based on the neighboring subnetworks and determine the remaining resources required for the task; Step 4, according to the resources required by the task, the current sub-network resources, and the remaining resources of the adjacent sub-network, at least one required adjacent sub-network is obtained; Step 5: After determining the adjacent sub-network, the link topology of the data task is completed.

4. The multimodal network traffic prediction system according to claim 1, characterized in that: The data cleaning includes interpolation compensation of missing values ​​for the data output speed, data input speed, bandwidth, and bandwidth utilization of the reaction flow parameters; The processing of the redundant values ​​is filtering and deduplication processing; The Z-score parameter anomaly detection determines whether it is abnormal by calculating the deviation between the data of the flow parameter and the mean, and sets Z-score=3 as the threshold to eliminate abnormal points; The chi-square test sets the data output speed, data input speed, and bandwidth utilization as independent variables, and the flow rate as the dependent variable. The weight coefficients of the independent variables are set to 30%, 30%, and 40% respectively, and the dependent variable is obtained to obtain the theoretical inference value R. The degree of deviation is calculated by using the chi-square test formula between the actual detection value C and the theoretical inference value R, df = (C-1)(R-1). The closer the degree of deviation is to 1, the higher the correlation between the independent variable and its weight coefficient and the dependent variable. When the correlation is lower than 0.7, the weight coefficient of the independent variable is adjusted, and finally the flow data is obtained by weighted calculation.

5. The multimodal network traffic prediction system according to claim 1, characterized in that: When abnormal traffic is detected, a Softmax classifier is used to further confirm whether the abnormal traffic is normal data based on the temporal characteristics and spatial characteristics of the network traffic.

6. The multimodal network traffic prediction system according to claim 1, characterized in that: The traffic prediction module uses a time series analysis method to predict the sub-network traffic trend, and uses a data fusion algorithm to integrate and process the traffic of multiple sub-networks to predict the overall traffic. The specific steps are as follows: Step 1, obtain the traffic data and historical traffic data of the sub-network; Step 2: Set the traffic data y of the subnetwork t The weight is 60%, historical traffic data y t-1 The weight is 20%, historical traffic data y t-2 The weight of yt-3 is 10%, and the weight of historical traffic data yt-3 is 10%; Step 3, the traffic data weight of the sub-network is modified by the credibility; Step 4: Calculate the weighted average to get the sub-network traffic prediction data y t+1 ; Step 5, obtaining the degree of deviation between the historical predicted value and the measured value; Step 6: Correct the sub-network traffic prediction data y t+1 ; Step 7: Obtain the traffic prediction data y1 for each sub-network t+1 ,y2 t+1 ,y3 t+1 …yn t+1 ; Step 8: Cluster the traffic types of each sub-network into regular traffic yc and burst traffic yz according to the threshold and bandwidth of the traffic prediction data of each sub-network; The conventional traffic set adopts the equal weight model yc=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1 )*60%, the weights are all 60 / n%; The burst traffic set uses the combined prediction model yz=(y1 t+1 +y2 t+1 +y3 t+1 …+yn t+1 )*40%+r, where r is the burst traffic duration and density bonus, r=burst traffic duration*density / 50%. Step 9, regular traffic prediction model + burst traffic prediction model = total traffic prediction value; Step 10, modifying the total flow forecast value according to the historical total flow forecast value, with weights of 25% and 75% respectively, and then calculating the weighted average; Step 11, the revised total flow forecast value, first calculate the rate of change, and then use the exponential smoothing forecasting algorithm to obtain the final total flow forecast value.

Images