Network analysis system and method supporting massive data real-time processing

Abstract

Provided are a network analytic system and method based on the mass data real-time processing technology. The system comprises a plurality of data adaptive nodes and data analytic clusters which are composed of a plurality of data analytic nodes and arranged in a network in a distribution mode, wherein the data analytic nodes support a P2P networking mode and a load balancing mechanism, so that the data analytic clusters have a telescopic function; the flow line type analytic processing process is completed among the data analytic nodes through an incident mechanism. According to the network analytic system and method supporting the real-time mass data processing, mass network data including operations of network fault monitoring, statistics, trouble shooting, diagnosis and the like can be analyzed and processed in real time, the network data can be analyzed in a fine mode, distribution type dynamic expansion is supported to expand system functions, and users can expand demands of themselves according to the analytic types to be detected in a self-defined manner. Furthermore, a distribution type structure is utilized by the system without excessively depending on single hardware performance, and the system can complete processing of complex logics of network data analysis and the like better. The system and method support multiple types of processing logics and lower the requirements for professional qualification of developers.

Description

technical field [0001] The invention relates to a network management technology, specifically, a network analysis system and method supporting real-time processing of massive data, and belongs to the technical field of network data analysis and network failure analysis. Background technique [0002] In recent years, China's Internet business has been developing continuously and rapidly, the scale of the network has continued to expand, network technologies have been introduced, network resources and business applications have become increasingly abundant, information exchanges have become faster and more efficient, and the influence of the Internet has increased dramatically. The Internet is playing an increasingly important role in promoting economic development, promoting social progress, and improving people's quality of life. [0003] After more than ten years of development, China's Internet development has shown great vitality. Great progress has been made in various ...

AI Technical Summary

Problems solved by technology

(2) Network service failure caused by unavailable network equipment

Moreover, traditional network monitoring software cannot realize the customization of network data analysis types

[0010] (2) Inaccurate analysis and positioning: For example, when Sniffer deals with the positioning of the monitored network data flow, it can only display the connection between the source data terminal and the destination data terminal

However, when monitoring information such as the CPU utilization rate and memory utilization rate of switches in the network, the highest can only count the average value in the last few minutes, which cannot reflect the real-time network status.

[0011] (3) Most of them cannot realize real-time data analysis: the current mainstream network monitoring software provides data capture function, but none of them can process the obtained data in real time; most of them cache the data and then perform offline data analysis

[0021] Because this technical solution needs to support a large flow of data, it is generally realized by mining the performance of the hardware itself, so it does not support complex processing logic well, and requires high skills for developers

In addition, DPI is mainly for traffic management, network security and other purposes, and it still lacks the ability to analyze network data.

Images