Method and system for intelligent analysis of traffic between different regional level networks of an operator

Abstract

The application relates to the technical field of flow analysis, and particularly discloses a flow intelligent analysis method and system between different regional networks of an operator. The application acquires a structured flow record table by collecting original flow data of multiple devices and nodes; carries out IP geographical attribution, attribution credibility and service type identification on each flow, and records a flow label table; carries out four-dimensional flow aggregation on the original flow data, and acquires a four-dimensional flow portrait structure; carries out flow charging and optimization on the four-dimensional flow portrait structure according to a preset flow charging strategy, and acquires flow charging data. The application can carry out IP geographical attribution, attribution credibility and service type identification, acquire a four-dimensional flow portrait structure, carry out flow charging and optimization, thereby automatically and accurately distinguishing cross-regional flows of different service types, solving the address drift problem, and realizing efficient and accurate cross-regional flow charging.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of flow analysis, and particularly relates to a flow intelligent analysis method and system between networks of different regional levels of an operator. BACKGROUND

[0002] To cope with the change of billing rules between the regions of an operator, inter-regional settlement, the flow of each business in the region needs to be analyzed, and the flow of each business in the region is counted. Based on these data, the business party is communicated to reduce the out-of-region flow, and based on these data, additional billing is performed to reduce losses. In addition, the inter-regional settlement of the operator adopts a flow ladder pricing mode, and at the same time, a night idle time flow discount rule is adopted. The cross-regional flow needs to be counted by time period according to the business type.

[0003] In the prior art, the analysis of cross-regional flow usually has the following defects: 1. The existing tool cannot accurately distinguish the cross-regional flow of the business type (such as video / cloud service / ordinary Internet access); 2. Manual statistics has time lag (T+3 days or more), and cannot dynamically adjust the business strategy; 3. The traditional IP analysis does not consider address drift (such as misjudgment of the place of origin caused by dynamic switching of nodes); 4. Lack of multi-dimensional proportion analysis (business / time period / target region), which is difficult to support billing negotiation. SUMMARY

[0004] The purpose of the embodiment of the application is to provide a flow intelligent analysis method and system between networks of different regional levels of an operator, which aims to solve the problems proposed in the background art.

[0005] To achieve the above purpose, the technical scheme provided by the embodiment of the application is as follows:

[0006] The flow intelligent analysis method between networks of different regional levels of an operator specifically includes the following steps:

[0007] Collecting raw flow data of a plurality of devices and nodes, and standardizing, abnormally cleaning and recombining the raw flow data to obtain a structured flow record table;

[0008] Based on the structured flow record table, IP geographical attribution, attribution credibility and business category identification are performed on each flow to record a flow label table;

[0009] Based on the structured flow record table and the flow label table, four-dimensional flow aggregation is performed on the raw flow data to obtain a four-dimensional flow portrait structure;

[0010] According to a preset flow billing strategy, flow billing and optimization are performed on the four-dimensional flow portrait structure to obtain flow billing data.

[0011] As a further limitation of the technical scheme of the embodiment of the application, the collecting raw traffic data of the plurality of devices and nodes, and standardizing, abnormally cleaning and recombining the raw traffic data to obtain a structured traffic record table specifically comprises the following steps:

[0012] receiving a traffic analysis request;

[0013] identifying the traffic analysis request to determine a sampling frequency, a plurality of devices and nodes;

[0014] sampling the plurality of devices and nodes according to the sampling frequency to obtain raw traffic data;

[0015] standardizing protocols and unifying fields of the raw traffic data;

[0016] abnormally cleaning and recombining the raw traffic data;

[0017] obtaining a structured traffic record table.

[0018] As a further limitation of the technical scheme of the embodiment of the application, based on the structured traffic record table, IP geographical attribution, attribution credibility and service category identification are performed on each piece of traffic, and a traffic label table is recorded, specifically comprising the following steps:

[0019] based on a preset BGP / ASN database, an IP geographical mapping library is constructed, and a probability weight model is established;

[0020] through the IP geographical mapping library, IP geographical attribution identification is performed on each piece of traffic in the structured traffic record table, and an attribution identification result is recorded;

[0021] through the probability weight model, attribution credibility analysis is performed on each piece of traffic in the structured traffic record table to obtain an attribution credibility result;

[0022] based on the structured traffic record table, five-tuple and behavior features are extracted;

[0023] according to the five-tuple and the behavior features, service category identification is performed on each piece of traffic, and a service category identification result is recorded;

[0024] comprehensively, the attribution identification result, the attribution credibility result and the service category identification result are combined to generate a traffic label table.

[0025] As a further limitation of the technical scheme of the embodiment of the application, the service category identification result has a plurality of service types, specifically video service, cloud service and ordinary online service.

[0026] As a further limitation of the technical scheme of the embodiment of the present application, the four-dimensional traffic aggregation based on the structured traffic record table and the traffic label table to obtain the four-dimensional traffic portrait structure specifically comprises the following steps:

[0027] Based on the structured traffic record table and the traffic label table, the traffic is summarized according to the service type and the regional attribution, and a service-region-traffic matrix table is constructed;

[0028] Based on the service-region-traffic matrix table, a time period partition strategy is obtained;

[0029] According to the time period partition strategy, the original traffic data is divided, and four-dimensional traffic aggregation is performed based on the service-region-traffic matrix table to generate a four-dimensional traffic portrait structure.

[0030] As a further limitation of the technical scheme of the embodiment of the present application, the traffic charging and optimization of the four-dimensional traffic portrait structure according to the preset traffic charging strategy to obtain traffic charging data specifically comprises the following steps:

[0031] The preset traffic charging strategy is loaded, and the traffic charging strategy includes a cross-regional pricing strategy, a time period partition pricing strategy, and a ladder pricing strategy;

[0032] According to the cross-regional pricing strategy, cross-regional traffic charging is performed on the four-dimensional traffic portrait structure to obtain cross-regional charging data;

[0033] According to the time period partition pricing strategy, the four-dimensional traffic portrait structure is partitioned and priced based on the cross-regional charging data to obtain cross-regional charging data;

[0034] According to the ladder pricing strategy, the four-dimensional traffic portrait structure is ladder priced based on the cross-regional charging data to obtain traffic charging data.

[0035] A traffic intelligent analysis system between different regional level networks of an operator, the system comprising a traffic data acquisition unit, a traffic identification processing unit, a four-dimensional traffic aggregation unit, and a traffic charging optimization unit, wherein:

[0036] The traffic data acquisition unit is used to acquire original traffic data of a plurality of devices and nodes, and to standardize, abnormally clean, and recombine the original traffic data to obtain a structured traffic record table;

[0037] The traffic identification processing unit is used to identify IP geographical attribution, attribution credibility, and service category based on the structured traffic record table for each traffic to record a traffic label table;

[0038] A four-dimensional traffic aggregation unit is configured to perform four-dimensional traffic aggregation on the raw traffic data based on the structured traffic record table and the traffic label table, and obtain a four-dimensional traffic portrait structure;

[0039] A traffic billing optimization unit is configured to perform traffic billing and optimization on the four-dimensional traffic portrait structure according to a preset traffic billing strategy, and obtain traffic billing data.

[0040] As a further limitation of the technical scheme of the embodiment of the application, the traffic data acquisition unit specifically comprises:

[0041] A request receiving module is configured to receive a traffic analysis request;

[0042] A request identifying module is configured to identify the traffic analysis request, and determine a sampling frequency, a plurality of devices and nodes;

[0043] A traffic sampling module is configured to sample the plurality of devices and nodes according to the sampling frequency, and obtain raw traffic data;

[0044] A standardization and field unification module is configured to perform protocol standardization and field unification on the raw traffic data;

[0045] An abnormality cleaning and traffic recombination module is configured to perform abnormality cleaning and traffic recombination on the raw traffic data;

[0046] A structured traffic record table obtaining module is configured to obtain a structured traffic record table.

[0047] As a further limitation of the technical scheme of the embodiment of the application, the traffic identification processing unit specifically comprises:

[0048] An IP geographical mapping library constructing module is configured to construct an IP geographical mapping library based on a preset BGP / ASN database, and establish a probability weight model;

[0049] An IP geographical attribution identifying module is configured to perform IP geographical attribution identification on each piece of traffic in the structured traffic record table through the IP geographical mapping library, and record attribution identification results;

[0050] An attribution credibility analyzing module is configured to perform attribution credibility analysis on each piece of traffic in the structured traffic record table through the probability weight model, to obtain attribution credibility results;

[0051] A feature extracting module is configured to extract five-tuple and behavior features based on the structured traffic record table;

[0052] a service category identification module, configured to identify a service category of each flow according to the five-tuple and the behavior feature, and record a service category identification result;

[0053] a flow label table generation module, configured to generate a flow label table by comprehensively considering the home identification result, the home credibility result and the service category identification result.

[0054] As a further limitation of the technical scheme of the embodiment of the present application, the flow charging optimization unit specifically comprises:

[0055] a policy loading module, configured to load a preset flow charging policy, wherein the flow charging policy comprises a cross-region pricing policy, a time period partition pricing policy and a ladder pricing policy;

[0056] a cross-region flow charging module, configured to perform cross-region flow charging on the four-dimensional flow portrait structure according to the cross-region pricing policy, and obtain cross-region charging data;

[0057] a partition pricing optimization module, configured to perform partition pricing optimization on the four-dimensional flow portrait structure on the basis of the cross-region charging data according to the time period partition pricing policy, and obtain cross-region charging data;

[0058] a ladder pricing optimization module, configured to perform ladder pricing optimization on the four-dimensional flow portrait structure on the basis of the cross-region charging data according to the ladder pricing policy, and obtain flow charging data.

[0059] Compared with the prior art, the present application has the following advantages:

[0060] The embodiment of the present application can perform IP geographical home, home credibility and service category identification on each flow, obtain a four-dimensional flow portrait structure, and perform flow charging and optimization, thereby automatically and accurately distinguishing cross-region flows of different service types, solving the problem of address drift, and realizing efficient and accurate cross-region flow charging. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application.

[0062] Figure 1 A flow chart of the method provided by the embodiment of the application is shown.

[0063] Figure 2 A flow chart of acquiring a structured flow record table in the method provided by the embodiment of the application is shown.

[0064] Figure 3 An application architecture diagram of the system provided by the embodiment of the application is shown.

[0065] Figure 4 A structural block diagram of a flow identification processing unit in the system provided by the embodiment of the application is shown.

[0066] Figure 5 A structural block diagram of a flow billing optimization unit in the system provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0068] It can be understood that, in the prior art, the analysis of cross-regional flow usually has the following defects: 1. The existing tools cannot accurately distinguish the cross-regional flow of different business types (such as video / cloud service / ordinary Internet access); 2. Manual statistics have time lag (T+3 days or more), and cannot dynamically adjust business strategies; 3. The traditional IP analysis does not consider address drift (such as misjudgment of the place of origin caused by dynamic switching of nodes); 4. Lack of multi-dimensional proportion analysis (business / time period / target region), which is difficult to support billing negotiations.

[0069] To solve the above problems, the embodiment of the present application acquires raw flow data of a plurality of devices and nodes, and standardizes, cleans and reorganizes the raw flow data to acquire a structured flow record table; based on the structured flow record table, IP geographical attribution, attribution credibility and business type identification are performed on each flow to record a flow label table; based on the structured flow record table and the flow label table, four-dimensional flow aggregation is performed on the raw flow data to acquire a four-dimensional flow portrait structure; and according to a preset flow billing strategy, flow billing and optimization are performed on the four-dimensional flow portrait structure to acquire flow billing data. The IP geographical attribution, attribution credibility and business type identification are performed, the four-dimensional flow portrait structure is acquired, and the flow billing and optimization are performed, so that the cross-regional flow of different business types can be automatically and accurately distinguished, and the problem of address drift can be solved, thereby realizing efficient and accurate cross-regional flow billing.

[0070] Figure 1 A flow chart of the method provided by the embodiment of the application is shown.

[0071] Specifically, the traffic intelligent analysis method between the networks of different regions of an operator specifically comprises the following steps:

[0072] In step S101, original traffic data of a plurality of devices and nodes are collected, and the original traffic data are standardized, abnormally cleaned and recombined to obtain a structured traffic record table.

[0073] In the embodiment of the application, the traffic analysis request uploaded by a management personnel is received, the traffic analysis request is identified, the sampling frequency and the plurality of devices and nodes are determined, then the plurality of devices and nodes are sampled according to the sampling frequency to obtain the original traffic data, and then the original traffic data are standardized and unified in field (standard field mapping is performed to uniformly convert the fields of heterogeneous devices into internal common fields), and the original traffic data are abnormally cleaned and recombined (abnormal flows such as 0-byte flow, short connection and scanning behavior flow are filtered, and flow records are reconstructed into complete sessions to generate logical connections, which facilitates subsequent identification), to obtain the structured traffic record table.

[0074] Specifically, Figure 2 A flowchart for obtaining the structured traffic record table in the method provided by the embodiment of the application is shown.

[0075] In the preferred embodiment provided by the application, the collection of the original traffic data of a plurality of devices and nodes, the standardization, abnormal cleaning and recombination of the original traffic data, and the obtaining of the structured traffic record table specifically comprise the following steps:

[0076] In step S1011, a traffic analysis request is received.

[0077] In step S1012, the traffic analysis request is identified to determine the sampling frequency and the plurality of devices and nodes.

[0078] In step S1013, the plurality of devices and nodes are sampled according to the sampling frequency to obtain the original traffic data.

[0079] In step S1014, the original traffic data are standardized and unified in field.

[0080] In step S1015, the original traffic data are abnormally cleaned and recombined.

[0081] In step S1016, the structured traffic record table is obtained.

[0082] Further, the traffic intelligent analysis method between the networks of different regions of an operator further comprises the following steps:

[0083] Step S102, based on the structured flow record table, IP geographical attribution, attribution credibility and service type identification are performed on each flow, and a flow label table is recorded.

[0084] In the embodiment of the application, based on the preset BGP / ASN database, an IP geographical mapping library (summarizing BGP routing, ASN table and WHOIS registration information, establishing IP-region initial mapping) is constructed, and a probability weight model is established. Through the IP geographical mapping library, IP geographical attribution identification is performed on each flow in the structured flow record table, and the attribution identification result is recorded. Then, through the probability weight model, attribution credibility analysis is performed on each flow in the structured flow record table, and the attribution credibility result is obtained. Based on the structured flow record table, five-tuple (SRC_IP, DST_IP, PORT, protocol and time sequence) and behavior characteristics (such as packet interval, flow duration and jitter rate) are extracted. According to the five-tuple and the behavior characteristics, service type identification is performed on each flow, and the service type identification result is recorded. Then, the attribution identification result, the attribution credibility result and the service type identification result are comprehensively combined to generate a flow label table. The specific multiple service types include video service, cloud service and ordinary online service.

[0085] The SRC_IP in the five-tuple represents the source IP address of the flow, that is, the sender IP address of the packet, which is used to identify the network location of the initiating device or user of the flow;

[0086] The DST_IP in the five-tuple represents the destination IP address of the flow, that is, the receiver IP address of the packet, which is used to identify the network location of the target service or user of the flow;

[0087] The PORT in the five-tuple represents the destination port number, which is used to distinguish different network services or application programs;

[0088] The protocol in the five-tuple represents the network transmission layer or application layer protocol type;

[0089] The time sequence in the five-tuple represents the time-related characteristics, which is used to analyze the time behavior mode of the flow to assist in identifying real-time services and non-real-time services.

[0090] In the preferred embodiment provided by the application, the based on the structured flow record table, IP geographical attribution, attribution credibility and service type identification are performed on each flow, and a flow label table is recorded, which specifically includes the following steps:

[0091] Step S1021, based on the preset BGP / ASN database, an IP geographical mapping library is constructed, and a probability weight model is established;

[0092] Step S1022, IP geographical attribution recognition is performed on each flow in the structured flow record table through the IP geographical mapping library, and attribution recognition results are recorded;

[0093] Step S1023, attribution credibility analysis is performed on each flow in the structured flow record table through the probability weight model, and attribution credibility results are obtained;

[0094] Step S1024, five-tuple and behavior features are extracted based on the structured flow record table;

[0095] Step S1025, service category recognition is performed on each flow according to the five-tuple and the behavior features, and service category recognition results are recorded;

[0096] Step S1026, the attribution recognition results, the attribution credibility results and the service category recognition results are integrated to generate a flow label table.

[0097] Specifically, attribution credibility analysis is performed on each flow in the structured flow record table through the probability weight model to obtain attribution credibility results, and the specific steps are as follows:

[0098] Based on the BGP route update time stamp recorded in the IP geographical mapping library, a time decay function is constructed; based on the time decay function, the weight coefficient of historical route information exceeding a preset period is reduced, and the confidence weight of BGP update records is improved to obtain a dynamic weight model;

[0099] Through BGP data, topology change strength, time proximity and route stability indicators are obtained; based on the dynamic weight model, real-time network topology change time of an operator and BGP / ASN data are processed in space-time correlation to obtain a space-time correlation weight; based on the space-time correlation weight, IP geographical attribution recognition results are obtained; based on the IP geographical attribution recognition results, the topology change strength, the time proximity and the route stability indicators, a multi-dimensional verification matrix is generated;

[0100] Based on the multi-dimensional verification matrix and the IP geographical attribution recognition results, a three-level decision tree is constructed, and the attribution credibility of the flow is classified into three levels through the three-level decision tree to generate corresponding three-level credibility classification labels; wherein, the first-level decision of the three-level decision tree is that when the BGP route information and the topology change record exist in space-time overlap, the highest credibility label is directly assigned; the second-level decision is that for IP segments with single-dimensional conflict, historical router path backtracking analysis is started, and according to the backtracking analysis results, a medium credibility label is assigned; the third-level decision is that for IP segments across regional boundaries, flow behavior pattern matching is implemented, and the extracted five-tuple features are compared, if the matching is successful, the credibility label is improved, otherwise, a low credibility label is assigned;

[0101] The service attribute of the three-level credibility classification label and the traffic label table is associated to build an optimized probability weight model, and the attribution credibility result is obtained through the optimized probability weight model.

[0102] Further, the application solves the attribution misjudgment problem caused by the insufficient timeliness of BGP data by fusing the time decay function and the topology change record; by adopting the service type adaptive credibility grading mechanism, the calculation overhead is reduced while the analysis accuracy is ensured; by cross-verification of the five-tuple feature and the geographical credibility, the concealment of the abnormal traffic detection is enhanced.

[0103] Further, the traffic intelligent analysis method between different regional networks of the operator further includes the following steps:

[0104] In step S103, four-dimensional traffic aggregation is performed on the original traffic data based on the structured traffic record table and the traffic label table, and a four-dimensional traffic portrait structure is obtained.

[0105] In the embodiment of the application, based on the structured traffic record table and the traffic label table, the original traffic data is aggregated according to the service type and the regional attribution, a service-region-traffic matrix table is constructed, and based on the service-region-traffic matrix table, a time period partition strategy (for example, idle time is 0:00-6:00, busy time is 6:00-24:00) is obtained, then the original traffic data is divided according to the time period partition strategy, and four-dimensional traffic aggregation is performed based on the service-region-traffic matrix table to generate a four-dimensional traffic portrait structure.

[0106] Specifically, in the preferred embodiment provided by the application, the four-dimensional traffic aggregation on the original traffic data based on the structured traffic record table and the traffic label table to obtain the four-dimensional traffic portrait structure specifically includes the following steps:

[0107] In step S1031, the original traffic data is aggregated according to the service type and the regional attribution based on the structured traffic record table and the traffic label table, and a service-region-traffic matrix table is constructed.

[0108] In step S1032, a time period partition strategy is obtained based on the service-region-traffic matrix table.

[0109] In step S1033, the original traffic data is divided according to the time period partition strategy, and four-dimensional traffic aggregation is performed based on the service-region-traffic matrix table to generate a four-dimensional traffic portrait structure.

[0110] Specifically, based on the structured flow record table and the flow label table, the original flow data is aggregated according to the service type and the region attribution to construct a service-region-flow matrix table, and the specific steps are as follows:

[0111] Based on the service category identification result, the real-time level is extracted, and based on the real-time level, the transmission stability index is calculated through the attribution credibility; the inter-regional flow topology relationship graph is constructed through the IP geographical attribution identification result, and the historical transmission cost feature and the timestamp associated service flow fluctuation are obtained through the inter-regional flow topology relationship graph; based on the transmission stability index, the historical transmission cost feature and the timestamp associated service flow fluctuation, a multi-dimensional feature vector is obtained;

[0112] Based on the attribution credibility and the real-time level, a credible real-time comprehensive value is generated, and the credible real-time comprehensive value is used as a key factor; based on the key factor, a stability factor is calculated, and based on the stability factor and the historical cost distribution, a cost sensitivity factor is constructed; when the resource utilization rate exceeds the preset resource utilization rate threshold, the cost sensitivity factor weight is increased, and a service-region association weight matrix is generated;

[0113] Feature weighted clustering is adopted, and the original flow data similar in multi-dimensional features is aggregated to the service-region association weight matrix to generate an initial service-region-flow matrix table;

[0114] The historical distribution of the initial service-region-flow matrix table is compared with the real-time flow feature to identify an abnormal unit; the attribution credibility is used to verify the geographical attribution authenticity based on the abnormal unit, and after the verification, the multi-dimensional feature vector of the abnormal unit is reconstructed to obtain an optimized matrix table, and the optimized matrix table is used as the service-region-flow matrix table.

[0115] Further, the application improves the economy of cross-provincial flow scheduling through the dynamic association of weight factors and network states; through the combination of the abnormal detection mechanism and the attribution credibility verification, the accuracy of the flow matrix is ensured.

[0116] Specifically, based on the service-region-flow matrix table, a time period partition strategy is obtained, and the specific steps are as follows:

[0117] According to the time dimension of the historical flow data in the service-region-flow matrix table, the historical data is divided according to a preset period, the time period flow fluctuation coefficient is calculated based on the historical data to obtain a time period sensitivity coefficient; according to the spatial dimension of the historical flow data in the service-region-flow matrix table, an inter-regional flow transmission topology graph is established, the transmission strength of the flow between nodes in the inter-regional flow transmission topology graph is calculated to obtain a cross-regional transmission strength;

[0118] According to the service association dimension of the historical traffic data in the service-area-traffic matrix table, combined with the service category identification result, a mapping relationship between the service type and the space-time feature is established;Based on the time period sensitivity coefficient, the cross-region transmission intensity, and the mapping relationship between the service type and the space-time feature, a space-time distribution feature map is obtained;

[0119] Based on the space-time distribution feature map, a multi-task prediction model based on deep learning is constructed, and the multi-task prediction model based on deep learning is used as a dynamic network load prediction model;

[0120] According to the service category identification result, real-time services and non-real-time services are divided, and the cost distribution in the cross-region billing data is combined to calculate the sensitivity coefficient of the service type to the transmission delay;According to the sensitivity coefficient of the service type to the transmission delay, an association matrix of the service type and the time period partition is established to obtain a service time period sensitivity classification table;

[0121] Based on the prediction result of the dynamic network load prediction model and the priority data in the service time period sensitivity classification table, a multi-objective optimization model is constructed;Based on the multi-objective optimization model, a NSGA-II multi-objective optimization algorithm is used for strategy optimization to obtain an initial time period partition strategy;

[0122] The initial time period partition strategy is injected into the historical traffic to construct a simulation environment, and the core time period resource deviation degree is monitored in the simulation environment;When the core time period resource deviation degree exceeds the preset threshold, the dynamic network load prediction model and the NSGA-II multi-objective optimization algorithm are incrementally trained through the real-time traffic features in the service-area-traffic matrix table to obtain a time period partition strategy.

[0123] Further, the application has self-optimization capability through the deviation monitoring mechanism in the strategy verification stage;By deeply coupling the service features and the network load prediction, the accuracy of cross-province traffic scheduling is improved.

[0124] Further, the traffic intelligent analysis method between different regional networks of the operator further comprises the following steps:

[0125] Step S104, according to the preset traffic billing strategy, traffic billing and optimization are performed on the four-dimensional traffic portrait structure to obtain traffic billing data.

[0126] In the embodiment of the present application, the preset traffic charging strategy including the cross-region pricing strategy, the time period partition pricing strategy and the step pricing strategy is loaded, the cross-region traffic charging is performed on the four-dimensional traffic portrait structure according to the cross-region pricing strategy, the cross-region charging data is obtained, then the partition pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-region charging data according to the time period partition pricing strategy, the cross-region charging data is obtained, and then the step pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-region charging data according to the step pricing strategy, and the traffic charging data is obtained.

[0127] Specifically, in the preferred embodiment provided by the present application, the traffic charging and optimization of the four-dimensional traffic portrait structure according to the preset traffic charging strategy to obtain the traffic charging data specifically includes the following steps:

[0128] In step S1041, the preset traffic charging strategy is loaded, and the traffic charging strategy includes a cross-region pricing strategy, a time period partition pricing strategy and a step pricing strategy.

[0129] In step S1042, the cross-region traffic charging is performed on the four-dimensional traffic portrait structure according to the cross-region pricing strategy, and the cross-region charging data is obtained.

[0130] In step S1043, the partition pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-region charging data according to the time period partition pricing strategy, and the cross-region charging data is obtained.

[0131] In step S1044, the step pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-region charging data according to the step pricing strategy, and the traffic charging data is obtained.

[0132] Specifically, the step pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-region charging data according to the step pricing strategy to obtain the traffic charging data, and the specific steps are as follows:

[0133] Based on the cost distribution information of the cross-region charging data and the historical traffic characteristics of the service type and the region combination in the four-dimensional traffic portrait structure, a traffic baseline prediction model is constructed; based on the traffic baseline prediction model, the multi-dimensional deviation degree of the actual traffic of the current charging period and the predicted baseline is compared to construct a traffic threshold interval of step pricing, and based on the traffic threshold interval of step pricing, the threshold adaptive algorithm is used for parameter recombination processing for the region-service combination with abnormal deviation degree to generate a dynamic step threshold parameter group.

[0134] Based on the network resource load characteristics in the dynamic step threshold parameter set and the time period partition strategy, a broadband resource-step pricing correlation model is constructed; through the broadband resource-step pricing correlation model, a resource occupation coefficient is extracted in the four-dimensional traffic portrait structure, and the dynamic step threshold parameter set and the resource occupation coefficient are coupled and calculated to obtain a resource weight distribution matrix;

[0135] The resource weight factor in the resource weight distribution matrix is injected into the step pricing calculation engine, and a double-layer optimization algorithm is used for charging strategy iteration processing to obtain an optimized step charging scheme set;

[0136] Based on the original traffic characteristics in the four-dimensional traffic portrait structure, a charging effect evaluation model is constructed; the optimized step charging scheme set is used for cross-validation of the charging effect evaluation model to obtain a cross-validated charging effect evaluation model; and the cross-validated charging effect evaluation model is used to output traffic charging data.

[0137] Further, the application realizes dynamic adaptation of the step threshold parameter and the network resource state by deeply coupling the traffic prediction model and the real-time charging strategy, effectively balances the charging fairness and the resource utilization rate; by using a multi-level cross-validation architecture, the mathematical rigor and the engineering feasibility of the charging strategy are simultaneously ensured, forming a technical closed loop.

[0138] Further, Figure 3 A system application architecture diagram provided by an embodiment of the application is shown.

[0139] In another preferred embodiment provided by the application, a traffic intelligent analysis system between networks of different regional levels of an operator comprises:

[0140] A traffic data acquisition unit 101 is configured to acquire original traffic data of a plurality of devices and nodes, and to standardize, clean up anomalies and recombine traffic of the original traffic data to obtain a structured traffic record table.

[0141] In the embodiment of the application, the traffic data acquisition unit 101 receives a traffic analysis request uploaded by a management personnel, identifies the traffic analysis request, determines a sampling frequency and a plurality of devices and nodes, samples the plurality of devices and nodes according to the sampling frequency to obtain original traffic data, and then standardizes protocols and unifies fields (performs field standard mapping to uniformly convert heterogeneous device fields into internal general fields) of the original traffic data, cleans up anomalies and recombines traffic of the original traffic data (filters abnormal flows such as continuous 0-byte flows, short connection flows and scanning behavior flows, and reconstructs flow records into complete sessions to generate logical connections for facilitating subsequent identification), and obtains a structured traffic record table.

[0142] Specifically, in the preferred embodiments provided by the application, the flow data collection unit 101 specifically comprises:

[0143] The request receiving module is configured to receive a flow analysis request.

[0144] The request identifying module is configured to identify the flow analysis request, and determine a sampling frequency, a plurality of devices and nodes.

[0145] The flow sampling module is configured to sample the plurality of devices and nodes according to the sampling frequency, and obtain original flow data.

[0146] The standardization and field unification module is configured to perform protocol standardization and field unification on the original flow data.

[0147] The anomaly cleaning and flow recombination module is configured to perform anomaly cleaning and flow recombination on the original flow data.

[0148] The structured flow record table obtaining module is configured to obtain a structured flow record table.

[0149] Further, the flow intelligent analysis system between different regional networks of the operator further comprises:

[0150] The flow identification processing unit 102 is configured to perform IP geographical attribution, attribution credibility and service type identification on each flow based on the structured flow record table, and record a flow label table.

[0151] In the embodiments of the application, the flow identification processing unit 102 constructs an IP geographical mapping library (summarizes BGP routing, ASN table and WHOIS registration information, and establishes IP-region initial mapping) based on a preset BGP / ASN database, and establishes a probability weight model. Each flow in the structured flow record table is subjected to IP geographical attribution identification through the IP geographical mapping library, and the attribution identification result is recorded. Each flow in the structured flow record table is subjected to attribution credibility analysis through the probability weight model, and the attribution credibility result is obtained. Based on the structured flow record table, five-tuple (SRC_IP, DST_IP, PORT, protocol, time sequence) and behavior characteristics (such as packet interval, flow duration, jitter rate, etc.) are extracted, each flow is subjected to service type identification according to the five-tuple and the behavior characteristics, and the service type identification result is recorded. The attribution identification result, the attribution credibility result and the service type identification result are comprehensively combined to generate a flow label table. Specifically, the plurality of service types include video service, cloud service and ordinary online service.

[0152] Specifically, Figure 4 The structural block diagram of the flow identification processing unit 102 in the system provided by the embodiments of the application is shown.

[0153] In the preferred embodiments provided by the application, the traffic identification processing unit 102 specifically comprises:

[0154] An IP geographical mapping library construction module 1021, configured to construct an IP geographical mapping library based on a preset BGP / ASN database, and establish a probability weight model;

[0155] An IP geographical attribution identification module 1022, configured to perform IP geographical attribution identification on each piece of traffic in the structured traffic record table based on the IP geographical mapping library, and record attribution identification results;

[0156] An attribution credibility analysis module 1023, configured to perform attribution credibility analysis on each piece of traffic in the structured traffic record table based on the probability weight model, and obtain attribution credibility results;

[0157] A feature extraction module 1024, configured to extract five-tuple and behavior features based on the structured traffic record table;

[0158] A service category identification module 1025, configured to identify the service category of each piece of traffic based on the five-tuple and the behavior features, and record service category identification results;

[0159] A traffic label table generation module 1026, configured to generate a traffic label table by integrating the attribution identification results, the attribution credibility results, and the service category identification results.

[0160] Further, the traffic intelligent analysis system between different regional networks of the operator further comprises:

[0161] A four-dimensional traffic aggregation unit 103, configured to perform four-dimensional traffic aggregation on the original traffic data based on the structured traffic record table and the traffic label table, and obtain a four-dimensional traffic portrait structure.

[0162] In the embodiments of the application, the four-dimensional traffic aggregation unit 103 performs traffic aggregation on the original traffic data based on the structured traffic record table and the traffic label table according to the service type and the regional attribution, constructs a service-region-traffic matrix table, and obtains a time period partition strategy (for example, idle time is 0:00-6:00, busy time is 6:00-24:00) based on the service-region-traffic matrix table, then divides the original traffic data according to the time period partition strategy, and performs four-dimensional traffic aggregation based on the service-region-traffic matrix table to generate the four-dimensional traffic portrait structure.

[0163] A traffic charging optimization unit 104, configured to perform traffic charging and optimization on the four-dimensional traffic portrait structure according to a preset traffic charging strategy, and obtain traffic charging data.

[0164] In the embodiment of the present application, the traffic charging optimization unit 104 loads preset traffic charging strategies including a cross-region pricing strategy, a time period partition pricing strategy and a ladder pricing strategy, performs cross-region traffic charging on the four-dimensional traffic portrait structure according to the cross-region pricing strategy, obtains cross-region charging data, performs partition pricing optimization on the four-dimensional traffic portrait structure on the basis of the cross-region charging data according to the time period partition pricing strategy, obtains cross-region charging data, and then performs ladder pricing optimization on the four-dimensional traffic portrait structure on the basis of the cross-region charging data according to the ladder pricing strategy, and obtains traffic charging data.

[0165] Specifically, Figure 5 The structural block diagram of the traffic charging optimization unit 104 in the system provided by the embodiment of the present application is shown.

[0166] In the preferred embodiment provided by the present application, the traffic charging optimization unit 104 specifically includes:

[0167] The strategy loading module 1041 is configured to load preset traffic charging strategies, and the traffic charging strategies include a cross-region pricing strategy, a time period partition pricing strategy and a ladder pricing strategy.

[0168] The cross-region traffic charging module 1042 is configured to perform cross-region traffic charging on the four-dimensional traffic portrait structure according to the cross-region pricing strategy, and obtain cross-region charging data.

[0169] The partition pricing optimization module 1043 is configured to perform partition pricing optimization on the four-dimensional traffic portrait structure on the basis of the cross-region charging data according to the time period partition pricing strategy, and obtain cross-region charging data.

[0170] The ladder pricing optimization module 1044 is configured to perform ladder pricing optimization on the four-dimensional traffic portrait structure on the basis of the cross-region charging data according to the ladder pricing strategy, and obtain traffic charging data.

[0171] It should be understood that, although each step in the flowchart of each embodiment of the present application is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps has no strict sequence limitation, and these steps can be executed in other sequences. Moreover, at least a part of the steps in each embodiment can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least a part of other steps or sub-steps or stages of other steps.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0173] Any combination of the technical features of the above-mentioned embodiments can be combined. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0174] The above-mentioned embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

[0175] The above-mentioned embodiments are only the preferred embodiments of the present application, and are not used to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for intelligent analysis of traffic between different regional level networks of an operator, characterized in that, The method specifically comprises the following steps: Collecting raw traffic data of a plurality of devices and nodes, and standardizing, cleaning and reorganizing the raw traffic data to obtain a structured traffic record table; Based on the structured traffic record table, IP geographical attribution, attribution credibility and service category identification are performed on each traffic to record a traffic label table; Based on the structured traffic record table and the traffic label table, four-dimensional traffic aggregation is performed on the raw traffic data to obtain a four-dimensional traffic portrait structure; According to a preset traffic billing strategy, traffic billing and optimization are performed on the four-dimensional traffic portrait structure to obtain traffic billing data; The method specifically comprises the following steps: Based on a preset BGP / ASN database, an IP geographical mapping library is constructed, and a probability weight model is established; Through the IP geographical mapping library, IP geographical attribution identification is performed on each traffic in the structured traffic record table to record attribution identification results; Through the probability weight model, attribution credibility analysis is performed on each traffic in the structured traffic record table to obtain attribution credibility results; Based on the structured traffic record table, five-tuple and behavior features are extracted; According to the five-tuple and the behavior features, service category identification is performed on each traffic to record service category identification results; The attribution identification results, the attribution credibility results and the service category identification results are integrated to generate a traffic label table; The method specifically comprises the following steps: Based on the structured traffic record table and the traffic label table, traffic is summarized according to service types and regional attributions to construct a service-region-traffic matrix table; Based on the service-region-traffic matrix table, a time period partition strategy is obtained; According to the time period partition strategy, the raw traffic data is divided, and four-dimensional traffic aggregation is performed on the basis of the service-region-traffic matrix table to generate a four-dimensional traffic portrait structure.

2. The method for traffic intelligence analysis between different regional level networks of operators according to claim 1, characterized in that, The method specifically comprises the following steps: Receiving a traffic analysis request; Identifying the traffic analysis request to determine a sampling frequency, a plurality of devices and nodes; According to the sampling frequency, the plurality of devices and nodes are sampled to obtain raw traffic data; Protocol standardization and field unification are performed on the raw traffic data; Abnormal cleaning and traffic reorganization are performed on the raw traffic data; A structured traffic record table is obtained.

3. The method of intelligent traffic analysis between different regional level networks of operators as claimed in claim 1 wherein, Through the probability weight model, attribution credibility analysis is performed on each traffic in the structured traffic record table to obtain attribution credibility results, and the specific steps are as follows: Construct a time decay function based on the BGP route update timestamp recorded in the IP geographical mapping library; based on the time decay function, reduce the weight coefficient of historical route information exceeding the preset period, and improve the confidence weight of the BGP update record to obtain a dynamic weight model; Obtain BGP packets, and obtain the topology change intensity, time proximity and route stability indicators through the BGP packets; Based on the dynamic weight model, perform spatio-temporal correlation processing on the operator real-time network topology change time and BGP / ASN data to obtain a spatio-temporal correlation weight; Based on the spatio-temporal correlation weight, obtain an IP geographical attribution recognition result; Based on the IP geographical attribution recognition result, the topology change intensity, the time proximity and the route stability indicators, generate a multi-dimensional verification matrix; Based on the multi-dimensional verification matrix and the IP geographical attribution recognition result, construct a three-level decision tree, and perform three-level classification on the attribution credibility of the traffic through the three-level decision tree to generate a corresponding three-level credibility classification label; wherein the first-level decision of the three-level decision tree is: when the BGP route information and the topology change record overlap in space and time, directly assign the highest credibility label; the second-level decision is: for IP segments with single-dimensional conflicts, start historical router path backtracking analysis, and assign a medium credibility label according to the backtracking analysis result; the third-level decision is: for IP segments across regional boundaries, implement traffic behavior pattern matching and compare with the extracted five-tuple features, if the matching is successful, the credibility label is improved, otherwise, a low credibility label is assigned; Associate the three-level credibility classification label with the service attributes of the traffic label table to construct an optimized probability weight model, and obtain the attribution credibility result through the optimized probability weight model.

4. The method for traffic intelligence analysis between different regional level networks of operators according to claim 1, characterized in that, Based on the structured traffic record table and the traffic label table, perform traffic aggregation on the original traffic data according to the service type and regional attribution, and construct a service-region-traffic matrix table, the specific steps are as follows: Extract the real-time level based on the service category recognition result, and obtain the transmission stability indicator through the attribution credibility calculation based on the real-time level; construct an inter-regional traffic topology relationship graph through the IP geographical attribution recognition result, and obtain the historical transmission cost feature and timestamp associated business traffic fluctuation through the inter-regional traffic topology relationship graph; based on the transmission stability indicator, the historical transmission cost feature and the timestamp associated business traffic fluctuation, obtain a multi-dimensional feature vector; Generate a credible real-time comprehensive value based on the attribution credibility and the real-time level, and take the credible real-time comprehensive value as a key factor; Calculate the stability factor based on the key factor, and construct the cost sensitivity factor based on the stability factor and the historical cost distribution; when the resource utilization rate exceeds the preset resource utilization rate threshold, the cost sensitivity factor weight is improved, and a service-region association weight matrix is generated; Use feature weighted clustering to aggregate original traffic data with similar multi-dimensional features to the service-region association weight matrix to generate an initial service-region-traffic matrix table; Comparing the historical distribution of the initial service-area-traffic matrix table with real-time traffic characteristics to identify an abnormal unit; Verifying the geographical attribution authenticity of the abnormal unit based on attribution credibility, and after verification, reconstructing the multi-dimensional feature vector of the abnormal unit to obtain an optimized matrix table, which is taken as the service-area-traffic matrix table.

5. The method of intelligent traffic analysis between different regional level networks of operators according to claim 4, characterized in that, Based on the service-area-traffic matrix table, a time period partitioning strategy is obtained, and the specific steps are as follows: According to the time dimension of the historical traffic data in the service-area-traffic matrix table, the historical data is divided according to a preset period, and a time period traffic fluctuation coefficient is calculated based on the historical data to obtain a time period sensitivity coefficient; according to the spatial dimension of the historical traffic data in the service-area-traffic matrix table, a traffic transmission topology graph between regions is established, and the transmission strength of traffic between nodes in the traffic transmission topology graph between regions is calculated to obtain a cross-regional transmission strength; According to the service association dimension of the historical traffic data in the service-area-traffic matrix table, a mapping relationship between service types and spatio-temporal characteristics is established in combination with the service category recognition result; Based on the time period sensitivity coefficient, the cross-regional transmission strength, and the mapping relationship between the service types and the spatio-temporal characteristics, a spatio-temporal distribution characteristic map is obtained; Based on the spatio-temporal distribution characteristic map, a deep learning-based multi-task prediction model is constructed, which is taken as a dynamic network load prediction model; According to the service category recognition result, real-time services and non-real-time services are divided, and in combination with the cost distribution in the cross-regional billing data, a service type-transmission time delay sensitivity coefficient is calculated; according to the service type-transmission time delay sensitivity coefficient, an association matrix of service types and time period partitioning is established to obtain a service time period sensitivity classification table; Based on the prediction result of the dynamic network load prediction model and the priority data in the service time period sensitivity classification table, a multi-objective optimization model is constructed; Based on the multi-objective optimization model, a NSGA-II multi-objective optimization algorithm is used for strategy optimization to obtain an initial time period partitioning strategy; The initial time period partitioning strategy is injected into the historical traffic to construct a simulation environment, and the core time period resource deviation degree is monitored in the simulation environment. When the core time period resource deviation degree exceeds a preset threshold, the dynamic network load prediction model and the NSGA-II multi-objective optimization algorithm are incrementally trained through the real-time traffic characteristics in the service-area-traffic matrix table to obtain the time period partitioning strategy.

6. The method of intelligent traffic analysis between operator different regional level networks of claim 5, wherein, The traffic billing and optimization of the four-dimensional traffic portrait structure according to the preset traffic billing strategy includes the following steps: Load the preset traffic billing strategy, which includes a cross-regional pricing strategy, a time period partitioning pricing strategy, and a step pricing strategy; According to the cross-regional pricing strategy, cross-regional traffic billing is performed on the four-dimensional traffic portrait structure to obtain cross-regional billing data; According to the time period partitioning pricing strategy, partitioning pricing optimization is performed on the four-dimensional traffic portrait structure based on the cross-regional billing data to obtain cross-regional billing data; According to the step pricing strategy, the four-dimensional traffic portrait structure is optimized based on the cross-region billing data to obtain traffic billing data.

7. The method of intelligent traffic analysis between different regional level networks of operators according to claim 6, characterized in that, According to the step pricing strategy, the four-dimensional traffic portrait structure is optimized based on the cross-region billing data to obtain traffic billing data, and the specific steps are as follows: Based on the cost distribution information of the cross-region billing data and the historical traffic characteristics of the service type and region combination in the four-dimensional traffic portrait structure, a traffic baseline prediction model is constructed; based on the traffic baseline prediction model, the multi-dimensional deviation degree of the actual traffic of the day billing period and the predicted baseline is compared to construct the traffic threshold interval of the step pricing; Based on the traffic threshold interval of the step pricing, for the region-service combination with abnormal deviation degree, a threshold adaptive algorithm is used for parameter reorganization processing to generate a dynamic step threshold parameter set; based on the dynamic step threshold parameter set and the network resource load characteristics in the time period partition strategy, a broadband resource-step pricing correlation model is constructed; Through the broadband resource-step pricing correlation model, the resource occupation coefficient is extracted in the four-dimensional traffic portrait structure, and the dynamic step threshold parameter set and the resource occupation coefficient are coupled and calculated to obtain a resource weight distribution matrix; The resource weight factor in the resource weight distribution matrix is injected into the step pricing calculation engine, and a double-layer optimization algorithm is used for billing strategy iteration processing to obtain an optimized step billing scheme set; Based on the original traffic characteristics in the four-dimensional traffic portrait structure, a billing effect evaluation model is constructed; the optimized step billing scheme set is used to cross-validate the billing effect evaluation model to obtain a cross-validated billing effect evaluation model; and the cross-validated billing effect evaluation model is used to output traffic billing data.

8. A system for intelligent analysis of traffic between different regional level networks of an operator, characterized by, The system applies the traffic intelligent analysis method between different regional level networks of the operator as claimed in any one of claims 1 to 7, and the system comprises a traffic data acquisition unit, a traffic identification processing unit, a four-dimensional traffic aggregation unit and a traffic billing optimization unit, wherein: The traffic data acquisition unit is used to acquire original traffic data of a plurality of devices and nodes, and to standardize, abnormally clean and reorganize the original traffic data to obtain a structured traffic record table; The traffic identification processing unit is used to identify IP geographical attribution, attribution credibility and service category based on the structured traffic record table, and record a traffic label table; The four-dimensional traffic aggregation unit is used to aggregate the original traffic data based on the structured traffic record table and the traffic label table to obtain a four-dimensional traffic portrait structure; The traffic billing optimization unit is used to perform traffic billing and optimization on the four-dimensional traffic portrait structure according to a preset traffic billing strategy to obtain traffic billing data.

Images