High-availability system and method for network data analysis and computer-readable storage medium

Abstract

The present invention provides a highly available system and method for network data analysis, builds and starts a cluster server group, configures cluster external services, receives analysis tasks provided by external devices, and sends the analysis tasks to the data acquisition module; receives analysis tasks and collects network data , analyze the network data, persist the data, forward the network data to the data analysis module; receive the network data, decode the network data, perform transaction association on the decoded network data, and send the transaction-associated easy data to the data consumer . The heartbeat signal detection is performed on each node in the cluster server group, and if the detection result is abnormal, automatic failover or automatic capacity expansion is performed. Control data fragmentation and data redundancy through clusters, improve storage performance under high-level data, improve processing performance through innovative data distribution algorithms, use stream computing architecture to reduce data processing delay, and achieve high availability in cluster mode without manual intervention. Effectively shorten the switching time.

Description

technical field [0001] The invention relates to the field of network performance monitoring, in particular to a highly available system and method for network data analysis and a computer-readable storage medium. Background technique [0002] Generally speaking, system high availability can be divided into two aspects: "storage high availability" and "computing high availability". Common architectures include "master-standby", "master-slave" and "cluster". [0003] The advantage of the active-standby architecture is that it is simple. There is no need for interaction between the active and standby machines, and the status judgment and switching operations are performed manually. The main disadvantages are: the manual switching operation is inefficient and error-prone; the standby machine is only used as a backup, and the hardware cost is wasted. The advantage of the master-slave architecture is that, compared with the master-slave scheme, the hardware performance of the sl...

AI Technical Summary

Problems solved by technology

The main disadvantages are: the manual switching operation is inefficient, and it is easy to make mistakes; the backup machine is only used as a backup, and there is a waste of hardware costs

The main disadvantages are: tasks need to be classified, and the task allocator will be more complicated; if the delay between the master and slave is relatively large, data inconsistency may occur

The above patent documents realize high-availability services, but their data sources are mainly webpage data or user behavior data. Using MapReduce of the Hadoop ecosystem for distributed computing requires loading data from HDFS and writing back the intermediate calculation results to HDFS. , there is a large time delay in the calculation process, which cannot meet the real-time requirements of the project

Images