Network Anomaly Detection via Markov Chain Argument Modeling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current intrusion detection methods are inadequate in distinguishing between new good and bad behavior, leading to high false positive rates and inability to defend against previously unseen attacks, especially in web environments where polymorphism and dynamic attacks are common.

Innovation Solution

A probabilistic model is trained using a dataset of normal communication protocol messages to identify anomalies by analyzing the collocation of grams in argument strings, allowing for the detection of anomalous messages and alerting administrators to potential threats.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If signature-based detection approaches are used, then known exploits can be filtered out, but previously unseen attacks cannot be defended against

Engineering Contradiction:

Improvedetection accuracy for known attacksVSAvoidability to detect new attacks

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary action by training a probabilistic model on a dataset of normal communication protocol messages before deployment. This pre-trained model establishes a baseline of legitimate behavior patterns, enabling the system to detect anomalies without requiring pre-programmed signatures for specific attacks. The model is prepared in advance to adaptively recognize both known and unknown attack patterns.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention changes the detection parameter from fixed signature matching to dynamic probabilistic scoring. Instead of checking for exact pattern matches, the system calculates anomaly scores based on deviations from learned normal behavior patterns. This parameter transformation enables continuous adaptation to new attack types while maintaining detection of known exploits.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If anomaly detection approaches are used, then new behavior can be recognized, but distinction between good and bad behavior becomes difficult

Engineering Contradiction:

Improveability to recognize new behaviorVSAvoidaccuracy in distinguishing legitimate from malicious traffic

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system applies local quality by analyzing specific linguistic features of communication protocols rather than treating all traffic uniformly. The probabilistic model examines local patterns such as argument string structures, parameter names, and data formats within protocol messages. This localized analysis enables precise distinction between legitimate and malicious behavior in specific contexts while maintaining overall adaptability.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The invention implements feedback through anomaly scoring that provides continuous evaluation of message legitimacy. The system calculates probability scores based on deviations from normal behavior patterns and uses these scores to classify traffic. This feedback mechanism enables the system to learn from detected anomalies and improve its ability to distinguish between good and bad behavior over time.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If protocol-agnostic modeling is used, then network layer detection is simplified, but detection scope is constrained and vulnerable to fragmentation attacks

Engineering Contradiction:

Improvesimplicity of network layer detectionVSAvoidrobustness against fragmentation and blending attacks

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system introduces an intermediary layer that operates between the network layer and application layer. The probabilistic model serves as a mediator that processes reassembled protocol messages, analyzing both structural and semantic features. This intermediary approach maintains the simplicity of network layer detection while adding the reliability of application-layer semantic analysis to detect fragmented and blended attacks.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The invention adds another dimension to detection by moving from purely structural protocol analysis to semantic content analysis. The probabilistic model evaluates the meaning and context of protocol arguments, not just their format. This dimensional expansion enables detection of attacks that manipulate protocol structure while maintaining valid syntax, such as fragmentation and blending attacks.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS10819726B2Detecting network anomalies by probabilistic modeling of argument strings with markov chains

Publication Date: 2020.10.27 THE TRUSTEES OF COLUMBIA UNIV IN THE CITY OF NEW YORK

AI summary

Systems, methods, and media for detecting network anomalies are provided. In some embodiments, a training dataset of communication protocol messages having argument strings is received. The content and structure associated with each of the argument strings is determined and a probabilistic model is trained using the determined content and structure of each of the argument strings. A communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network is received. The received communication protocol message is compared to the probabilistic model and then it is determined whether the communication protocol message is anomalous.