Method and device for log collection cluster load balance

Abstract

The invention discloses a method and a device for log collection cluster load balance. The method comprises the following steps: synchronizing state information of a log server through zookeeper, confirming data of a pointer of a log server data structure according to the state information of the log server, receiving a log transmitted from a log client, confirming the pointer of the log server distributed by the log in a data group polling mode, and forwarding the log to a confirmed server to which the pointer of the log server distributed by the log points. Communication between a log collection server and a log server is achieved through zookeeper, as the zookeeper is a cluster, the problem of single point of failure is avoided, the log server is allocated for the received log according to the state information of the log server so as to store the log, and load balance is achieved.

Description

technical field [0001] Embodiments of the present invention relate to network technologies, and in particular to a method and device for log collection cluster load balancing. Background technique [0002] The log collection cluster is a service cluster composed of multiple log collection servers. The log sending client sends data to the cluster, and the data will be saved in the cluster machines and provide data access interfaces. In the prior art, the implementation method of the log collection cluster includes: the upload server receives the log upload notification message carrying the address information of at least one node collection server sent by the central controller, and saves the received address information of each node collection server; The upload server selects the address information of a node collection server to receive logs from the saved address information of the node collection server; the upload server sends the log stored on the local disk to the nod...

AI Technical Summary

Problems solved by technology

[0003] However, in the prior art, a central controller implements the communication between the upload server and the node collection server. If a problem occurs in the central controller, the entire system cannot continue to work, and there is a single point of failure problem; the prior art does not consider the central collection Server load problem. When the central collection server is a single machine, the flow it can withstand is limited. If the flow exceeds the limit value, packet loss will occur, resulting in log loss. When the central collection server is a cluster, the load is not considered. balance problem

Images