Network Event Threat Detection Through Cross-Network Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle to accurately identify network security threats across different networks due to variations in data distribution, leading to inefficiencies in resource utilization and missed threats or false positives.
Innovation Solution
The method employs covariate shift analysis and domain adaptation to align the feature distribution of a first dataset with a second dataset, generating a training dataset that is more applicable to the second network, thereby improving the performance of threat detection models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a machine learning model is trained using data from a first network, then the model can be applied to detect threats in a second network, but the model performance deteriorates due to differences in data distribution between the two networks
Solution Approach 1:
The patent applies domain adaptation techniques that transform the parameter distribution of the source dataset to match the target dataset's distribution characteristics. This involves adjusting statistical parameters such as mean, variance, and higher-order moments of the feature distributions through techniques like moment matching or neural network-based distribution alignment, thereby resolving the contradiction between model portability and detection accuracy.
Solution Approach 2:
The patent introduces an intermediary domain adaptation layer that acts as a bridge between the source network data and target network data. This intermediary component transforms the source data distribution to better align with the target distribution, enabling the model to maintain high accuracy when applied across different networks without requiring retraining on target data.
2Reliability
If traditional threat detection methods are used to police a large volume of network events, then all events can be reviewed, but the resource consumption and time required become prohibitively large
Solution Approach 1:
The patent segments the network events into different categories based on their threat likelihood characteristics. By dividing the large volume of events into high-priority suspicious events and normal events, the system can apply more rigorous analysis only to the segmented suspicious portion while quickly processing or filtering the remaining events, thereby maintaining detection completeness while improving overall processing efficiency.
Solution Approach 2:
The patent applies partial action by focusing computational resources on detecting the critical minority of threatening events rather than uniformly processing all events. The system performs comprehensive analysis only on events that exhibit suspicious characteristics, while applying lighter processing to normal events, thus achieving effective threat detection without the prohibitive resource cost of analyzing every single event in detail.
3Reliability
If a threat detection model flags network events as possible security threats, then potential threats can be identified, but false positives result in wasted resources and incorrect remedial actions
Solution Approach 1:
The patent implements feedback mechanisms where the outcomes of threat detections (true positives, false positives, true negatives, false negatives) are fed back into the system to continuously refine the detection model. This feedback loop allows the system to learn from its mistakes, adjust decision thresholds, and improve the balance between identifying real threats and minimizing false positives, thereby reducing resource waste from incorrect remedial actions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method is provided for training a machine learning model to identify one or more network events associated with a network and representing a network security threat. The method comprises: a) obtaining a first dataset comprising data representative of a plurality of network events in a first network; b) obtaining a second dataset comprising data representative of a plurality of network events in a second network; c) performing covariate shift analysis on the first dataset and the second dataset to identify and classify a plurality of differences between the first dataset and the second dataset; d) performing domain adaptation on the first dataset, based on a classified difference, to generate a training dataset; e) training a machine learning model using the training dataset to produce a trained threat detection model. In this way,