Network Data Interpretation Pipeline for Security Threat Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network monitoring systems face challenges in interpreting network sensor events to identify machine operations and activities, leading to inefficiencies in resource usage and missed security threats due to the sheer volume of data and lack of contextual information.
Innovation Solution
Implementing a network data interpretation pipeline that recognizes machine operations and activities by classifying network sensor events into machine operations and higher-level activities, reducing data volume and enhancing security threat detection through contextual information and user-friendly interfaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If network sensors collect and upload all network traffic metadata to centralized monitoring platform, then complete network security monitoring is achieved, but data processing load and resource usage increase significantly
Solution Approach 1:
The patent segments the network monitoring system into distributed network sensors that collect metadata locally and a centralized monitoring platform that receives processed information. Each sensor operates semi-independently, filtering and selecting relevant network traffic metadata before transmission, which divides the overall processing load across multiple nodes rather than concentrating all data analysis at the central platform.
Solution Approach 2:
The patent implements preliminary filtering and selection of network traffic metadata at the network sensor level before data is uploaded to the centralized platform. This preliminary action reduces the volume of data that needs to be transmitted and processed centrally, addressing the contradiction by preprocessing data at the source to improve overall system efficiency while maintaining monitoring completeness.
2Reliability
If network sensors upload all raw network event data, then comprehensive security analysis is enabled, but resource usage and network bandwidth are wasted
Solution Approach 1:
The patent extracts only the essential and relevant features from raw network traffic metadata at the sensor level, selecting specific fields and events that are most indicative of security threats. This extraction process removes unnecessary data before transmission, reducing resource usage for data transfer and storage while retaining the information needed for comprehensive security analysis at the centralized platform.
3Measurement precision
If network monitoring systems analyze detailed network sensor events, then security threat detection accuracy is improved, but the complexity of data interpretation increases
Solution Approach 1:
The patent performs preliminary organization and contextualization of network sensor events at the distributed sensor level, structuring data in a standardized format with relevant metadata before transmission. This preliminary structuring reduces the complexity of data interpretation at the centralized platform while maintaining the detailed information needed for accurate security threat detection.
4Reliability
If all network sensor events are transmitted to centralized platform, then complete security monitoring is achieved, but network bandwidth and transmission resources are consumed
Solution Approach 1:
The patent extracts and transmits only the essential security-relevant information from network sensor events, filtering out redundant and less critical data before transmission to the centralized platform. This selective extraction maintains security monitoring completeness by focusing on threat-indicative events while significantly reducing network bandwidth consumption for data transmission.
Data Source
AI summary
Systems and methods are disclosed to implement a network data interpretation pipeline to recognize machine operations (MOs) and machine activities (MAs) from network traffic data observed in a monitored network. In embodiments, a MO recognition engine is implemented in the network to recognize MOs from network sensor events (NSEs) based on defined recognition patterns. The MOs and any unrecognized NSEs are uploaded to a network monitoring system, where they are further analyzed by a MA recognition engine to recognize higher-level machine activities performed by machines. The NSEs, MOs, and MAs are used by the network monitoring system to implement a variety of security threat detection processes. Advantageously, the pipeline may be used to add rich contextual information about the raw network data to facilitate security threat detection processes.