High-performance network log message distribution method for supporting target cluster dynamic detection

Abstract

The invention provides a high-performance network log message distribution method for supporting target cluster dynamic detection. By adopting a keep-alive dynamic detection strategy of distributed detection and unified state release, consistency of receiving host status on each distribution node can be ensued, and at the same time, weighted load balancing is used in the log message distribution process to avoid a large amount of memory access. By adopting methods such as distributed dynamic detection and unified state distribution, the activity of the receiving host is ensured. The weighted load balancing algorithm is used to distribute the network log messages more evenly to the receiving host in the cluster environment. At the same time, the method of determining the ports of the receiving host packet by packet by hash values of the quintuple (source address, destination address, source port, destination port, and protocol) included in the session packet calculated by using the hashfunction effectively reduces the impact of the failure of the receiving host.

Description

technical field [0001] The invention belongs to the field of network management, and in particular relates to a high-performance network log message distribution method supporting dynamic detection of target clusters. Background technique [0002] The original meaning of session (Session) refers to a series of actions / messages from beginning to end. When a session is associated with a network protocol, it also includes two meanings of "connection-oriented" and "state-keeping". Network devices or software will generate session logs according to actual business needs. With the rapid development of network technology, it has become an important content of current research to process network traffic with high performance in real time and aggregate and distribute the generated log messages of the same session to the same specific cluster, especially in many applied research fields. . For example: network monitoring, e-commerce and other important fields. In the process of proc...

AI Technical Summary

Problems solved by technology

The existing technology mainly adopts the scheme that the message distribution nodes independently complete the state detection of the receiving host. However, when dealing with cross-regional message distribution, the same distribution node or multiple distribution nodes in different regions send messages to the same receiving host cluster. In some cases, if the distribution nodes make inconsistent judgments on the state of a receiving host in the cluster, it will directly cause different distribution nodes to distribute the messages of the same connection to different receiving hosts, resulting in the failure of log message distribution; at the same time, due to the In the process of message forwarding, it is necessary to access different memory spaces multiple times to obtain the address and port information of the receiving host, which consumes a lot of memory resources

Images