Binary Protocol Field Extraction via Statistical Analysis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The rapid growth of proprietary and undocumented network protocols, such as those used in online games and social networks, hampers the ability to automatically reverse engineer protocol specifications, limiting network management, traffic analysis, and security operations.

Innovation Solution

A method and system for analyzing network protocols by extracting and identifying fields from conversations between servers and clients using a protocol field extractor, which calculates randomness and correlation measures to select candidate offsets and lengths, thereby defining the protocol fields.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual protocol analysis is used, then measurement precision is improved, but productivity deteriorates due to time-consuming manual reverse engineering

Engineering Contradiction:

Improveprotocol field identification accuracyVSAvoidprotocol analysis speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system performs automatic protocol field extraction by analyzing network traffic patterns itself, without requiring manual intervention. The automated field extractor processes conversations, calculates randomness measures, and identifies fields autonomously, transforming a manual process into a self-service system that maintains accuracy while dramatically improving productivity

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces manual mechanical analysis with automated computational processes. Instead of human analysts manually examining protocol data, the system uses computer processors to automatically extract fields, calculate statistical measures, and identify protocol structures through algorithmic analysis of network traffic

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If automatic field extraction is applied, then productivity is improved, but measurement precision deteriorates due to potential errors in automated detection

Engineering Contradiction:

Improveprotocol analysis speedVSAvoidprotocol field identification accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system incorporates feedback mechanisms where extracted fields are validated through statistical analysis. The randomness measure calculation provides feedback on whether extracted fields match expected protocol characteristics, allowing the system to adjust and refine its automated detection to maintain high precision while preserving improved productivity

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent uses sophisticated computational algorithms to replace manual analysis, ensuring that automated field extraction achieves both speed and accuracy. The system processes network traffic through multiple computational stages including conversation parsing, field candidate generation, and statistical validation, producing reliable results that match manual expert analysis

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If proprietary protocols are used, then adaptability is improved for specific applications, but ease of operation deteriorates due to lack of documentation

Engineering Contradiction:

Improveprotocol customization capabilityVSAvoidprotocol understanding and implementation

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system serves itself by automatically analyzing undocumented proprietary protocols and generating field specifications without requiring prior knowledge or documentation. The automated field extractor processes raw network traffic and produces structured protocol understanding, eliminating the need for operators to manually analyze complex proprietary protocols

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent introduces an automated intermediary system that bridges the gap between proprietary protocols and operational understanding. The field extractor acts as a mediator that translates undocumented binary protocols into structured field representations, making the protocols accessible and operable without requiring direct human interpretation of the proprietary formats

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10218598B2Automatic parsing of binary-based application protocols using network traffic

Publication Date: 2019.02.26 THE BOEING CO

AI summary

A method for analyzing a binary-based application protocol of a network. The method includes obtaining conversations from the network, extracting content of a candidate field from a message in each conversation, calculating a randomness measure of the content to represent a level of randomness of the content across all conversation, calculating a correlation measure of the content to represent a level of correlation, across all of conversations, between the content and an attribute of a corresponding conversation where the message containing the candidate field is located, and selecting, based on the randomness measure and the correlation measure, and using a pre-determined field selection criterion, the candidate offset from a set of candidate offsets as the offset defined by the protocol.