High availability system and method for network data analysis and computer readable storage medium

Abstract

The invention provides a high availability system and method for network data analysis. A cluster server group is built and started, a cluster external service is configured, and an analysis task provided by external equipment is received and sent to a data acquisition module; the analysis task is received, and network data is acquired, analyzed, subjected to data persistence and forwarded to a data analysis module; the network data is received and decoded, and the decoded network data is subjected to transaction association, and the transaction data obtained after transaction association is sent to a data consumption end. Various nodes in the cluster server group are subjected to heartbeat signal detection, and if detection results are abnormal, automatic fault switching or automatic expansion is executed. Data fragmentation and data redundancy are controlled by a cluster, the storage performance is improved under high-magnitude data, the processing performance is improved by innovating a data distribution algorithm, the data processing delay is shortened by adopting a stream computing architecture, high availability is achieved by means of the cluster mode, manual intervention isnot needed, and the switching time is effectively shortened.

Description

technical field [0001] The invention relates to the field of network performance monitoring, in particular to a highly available system and method for network data analysis and a computer-readable storage medium. Background technique [0002] Generally speaking, system high availability can be divided into two aspects: "storage high availability" and "computing high availability". Common architectures include "master-standby", "master-slave" and "cluster". [0003] The advantage of the active-standby architecture is that it is simple. There is no need for interaction between the active and standby machines, and the status judgment and switching operations are performed manually. The main disadvantages are: the manual switching operation is inefficient and error-prone; the standby machine is only used as a backup, and the hardware cost is wasted. The advantage of the master-slave architecture is that, compared with the master-slave scheme, the hardware performance of the sl...

AI Technical Summary

Problems solved by technology

The main disadvantages are: the manual switching operation is inefficient, and it is easy to make mistakes; the backup machine is only used as a backup, and there is a waste of hardware costs

The main disadvantages are: tasks need to be classified, and the task allocator will be more complicated; if the delay between the master and slave is relatively large, data inconsistency may occur

The above patent documents realize high-availability services, but their data sources are mainly webpage data or user behavior data. Using MapReduce of the Hadoop ecosystem for distributed computing requires loading data from HDFS and writing back the intermediate calculation results to HDFS. , there is a large time delay in the calculation process, which cannot meet the real-time requirements of the project

Images