Method and device for log collection cluster load balancing

Abstract

The invention discloses a log collection cluster load balancing method and device. The method comprises: synchronizing the state information of the log server through zookeeper; determining an array of pointers to the data structure of the log server according to the state information of the log server; receiving the log sent by the log client; determining as the log by polling the array A pointer to an assigned log server; forwarding the log to the log server pointed to by the determined pointer to the log server assigned to the log. The present invention realizes the communication between log collection server and log server through zookeeper, because zookeeper is a cluster, has avoided the problem of single point of failure; According to the state information of log server, assign log server to store for the log received, realized load balanced.

Description

Technical field [0001] The embodiment of the present invention relates to network technology, and in particular to a method and device for load balancing of a log collection cluster. Background technique [0002] The log collection cluster is a service cluster composed of multiple log collection servers. The log sending client sends data to the cluster, and the data is stored in the cluster machines and provides an access interface for the data. In the prior art, the implementation method of the log collection cluster includes: the upload server receives the log upload notification message carrying the address information of at least one node collection server sent by the central controller, and saves the received address information of each node collection server; The upload server selects the address information of a node collection server to receive logs from the saved address information of the node collection server; the upload server sends the logs stored on the local disk ...

AI Technical Summary

Problems solved by technology

[0003] However, in the prior art, a central controller implements the communication between the upload server and the node collection server. If a problem occurs in the central controller, the entire system cannot continue to work, and there is a single point of failure problem; the prior art does not consider the central collection Server load problem. When the central collection server is a single machine, the flow it can withstand is limited. If the flow exceeds the limit value, packet loss will occur, resulting in log loss. When the central collection server is a cluster, the load is not considered. balance problem

Images