Network Flow Threat Analysis Using Feature Selection and ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cyber threat intelligence methods lack effective machine learning-based analysis, particularly for network flow data, which is bulky and challenging for edge computing, and current rule-based systems are vulnerable to attacks like SYN DDoS, requiring expert knowledge and generating false positives.
Innovation Solution
An apparatus and method utilizing a flow detector, feature extraction, selection, classification, and threat computation models, combined with a cyber threat intelligence engine, to analyze and predict cyber threats by processing network flow data using machine learning techniques, including feature selection and deep neural networks for improved accuracy and reduced system loading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If network flow data is used for cyber threat intelligence analysis, then more information is available for machine learning-based analysis, but the data becomes bulky and challenging for edge computing
Solution Approach 1:
The patent extracts only the most relevant features from network flow data using a feature selection model. Instead of processing all network flow information, the system identifies and extracts key features that are most predictive of cyber threats, thereby reducing data volume while preserving essential information for analysis.
Solution Approach 2:
The patent segments the network flow data processing into distinct stages: feature extraction, feature selection, and classification. This segmentation allows the system to handle data in manageable portions, extracting only necessary features at each stage rather than processing the entire bulky dataset.
2Reliability
If rule-based classification methods like SIEM are used, then expert knowledge can be applied for security analysis, but the system becomes vulnerable to attacks and requires expert knowledge to deploy
Solution Approach 1:
The patent replaces rule-based classification with machine learning-based classification. Instead of relying on manually configured rules that require expert knowledge, the system uses trained machine learning models that automatically learn patterns from data, making the system easier to deploy while maintaining or improving security analysis capability.
Solution Approach 2:
The machine learning models automatically learn and adapt to new threat patterns without requiring continuous expert intervention. The system performs self-service by automatically updating its classification capabilities based on training data, reducing the need for expert knowledge in deployment and maintenance.
3Measurement precision
If machine learning-based analysis is implemented, then accuracy in detecting and predicting attacks is improved, but system complexity increases
Solution Approach 1:
The patent applies partial action by implementing only the necessary machine learning components needed for effective threat detection. Instead of implementing a complete complex ML pipeline, the system selectively applies feature extraction and classification only where needed, achieving good detection accuracy while limiting complexity growth.
Data Source
AI summary
An apparatus for analysing and monitoring a network flow for cyber threat intelligence is provided. The apparatus includes a feature selection model, a feature classification model, a threat computation model, a cyber threat intelligence engine, and a monitoring system. The feature selection model is configured to select flow-based features from a dataset in a network flow to generate network flow meta. The feature classification model is configured to analysis the flow-based features of the network flow meta to generate classified features. The threat computation model is configured to predict the threat features from the classified features using a swappable prediction model. The cyber threat intelligence engine is configured to collect and combine the flow-based features, the classified features, and the threat features to form one or more cyber threat joint features. The monitoring system is configured to output the cyber threat joint features.