High speed filtering and stream dividing method for keeping connection features

Abstract

First, the invention sets up output port grouping rule and filtering tapping rule. Then, data filtering tapping process is carried out. IP data packets are picked up from original data packets received from network based on protocol analysis. Based on filtering tapping rule, data relevant to following processes are partitioned to group of each output port. Hash operation is carried out for address of data packet and port information in partitioned each group. Modular operation is carried our between the said Hash value and total number of port. The result is number of output port for the corresponding data packet in the group the data packet belongs to. Under high-speed backbone network, the invention meets need for clustering process and forwarding at high speed for mass data stream.

Description

technical field [0001] The invention relates to a high-flow data processing method of a broadband network, in particular to a high-speed filtering and splitting method suitable for maintaining connection characteristics of broadband network data, and is used in the field of network information technology. Background technique [0002] In the past two years, the construction of my country's backbone network and metropolitan area network has reached a considerable scale, and the bandwidth of the backbone network is usually above 2.5G. The performance of current network security products such as firewalls, intrusion detection systems, and network security audits on network data packets is the primary prerequisite for ensuring that the system can make correct and effective analysis and judgments on security events that occur in the network. Existing security products have great limitations for the massive data processing requirements of broadband networks above 2.5G and 10G: ...

AI Technical Summary

Problems solved by technology

However, the current load balancing processing mechanism is to save the address information of the TCP connection in the form of a connection table. With the expansion and complexity of the network scale, the connection table will grow exponentially with the expansion of the network scale.

In addition, there are still a large number of useless false connections on the network, and the continuous expansion of the data scale of the connection table will increase the resource consumption in the table lookup process. Even if a high-performance NPU is used, it is difficult to meet the processing requirements of super large connection data.

[0005] 3. The problem of quick lookup of the connection table limits the improvement of data processing capabilities

The disadvantage of this product is that in order to ensure the consistency of bidirectional data packet processing belonging to the same connection, it is necessary to record and process related information for each connection in this product. With the continuous expansion of the network scale, the increasingly large connection The storage and search of information cannot meet the data processing requirements of high-speed broadband networks

[0007] The broadband development of the Internet has led to rapid growth of network traffic. Faced with massive data on the core network and backbone network, network security products that can only meet the data processing capabilities of 100M networks and Gigabit networks cannot effectively monitor high-flow data. Packet collection and processing, resulting in missing data collection and false positives, has become a major bottleneck in the application of these security products in the field of telecommunications backbone networks

Images