Network Traffic Character Set Identification via FPGA Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network traffic analyzers face challenges in identifying character sets used in digital communication across different languages and encoding systems, as existing technologies struggle to decode and recognize keywords across various Unicode character sets, leading to incomplete or inaccurate analysis.
Innovation Solution
A network traffic analyzer system that includes a controller with a field-programmable gate array (FPGA) to scan network traffic, identify the character sets used in each data packet by comparing payload areas to known character sets, and store indications of these sets, allowing for real-time analysis without interrupting the network traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep packet inspection is performed to capture network traffic with specific keywords, then security analysis capability is improved, but the system fails to identify keywords when they are encoded in different character sets
Solution Approach 1:
The system performs preliminary identification of character sets used in network traffic packets before conducting keyword matching. By determining which character sets are present in the traffic flow in advance, the system can then translate keywords into the appropriate character sets for accurate matching, resolving the issue of missed detections across different encodings.
Solution Approach 2:
The system dynamically changes the encoding parameter (character set) of search keywords based on the detected character set of the network traffic. Instead of using a fixed character set for keyword matching, the system adapts the keyword encoding to match the traffic encoding, enabling accurate detection across multiple character sets including Unicode, UTF-8, and legacy encodings.
2Measurement precision
If the system identifies character sets for all network traffic packets, then analysis comprehensiveness is improved, but processing time and computational resources increase
Solution Approach 1:
The system applies character set identification selectively rather than uniformly to all packets. It focuses on identifying character sets for packets that contain potential keywords or are suspected of containing relevant information, while using faster methods for routine traffic. This partial application of the identification process reduces overall processing time while maintaining accuracy for critical packets.
Solution Approach 2:
The system segments the character set identification process into multiple stages: a fast initial scan for obvious encodings, followed by more detailed analysis only for packets that require it. This segmented approach allows the system to process most packets quickly while applying comprehensive analysis only where necessary, balancing accuracy with processing speed.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Apparatus, methods, and systems for use in analyzing a flow of network traffic between a first network and a second network are provided. One example method includes scanning the network traffic between the first and second networks. The network traffic includes a plurality of data packets. The method includes determining a character set included in each of the plurality of data packets, and storing an indication of each character set included in each scanned data packet.