Neural Network Drift Detection Using Topological Persistence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art drift detection frameworks are inefficient in identifying adversarial attacks, as they fail to detect when neural networks are inefficient in detecting data, especially in cyber intrusion detection systems (IDSs), due to poor performance in identifying rare attack categories and data drift.
Innovation Solution
Utilizing topological persistence diagrams to quantify uncertainty in neural network (NN) classifications, leveraging features like barycenters to detect data drift and alert when NN predictions are unreliable.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If state-of-the-art drift detection frameworks are used, then the system can detect data drift, but they fail to identify rare attack categories and adversarial attacks effectively
Solution Approach 1:
The patent segments the detection task into two distinct components: (1) a drift detector that identifies when data distribution changes occur, and (2) a separate analysis module that examines the nature of detected drift to classify rare attack categories. This segmentation allows each component to be optimized for its specific function, improving overall detection accuracy for both common and rare attacks.
Solution Approach 2:
The patent introduces an intermediary drift detector module that sits between the neural network classifier and the final detection output. This intermediary analyzes the input data distribution and provides drift information that enhances the classifier's ability to identify rare attack categories, acting as a mediator that bridges general drift detection and specific attack identification.
2Productivity
If neural networks are trained on majority data (benign traffic and common attacks), then training efficiency is improved, but the system fails to detect rare attack categories
Solution Approach 1:
The patent applies preliminary action by training the neural network on the majority class data (benign traffic and common attacks) first to establish a strong baseline model efficiently. Then, a drift detector is trained separately on rare attack categories, and its output is integrated with the main classifier. This preliminary training approach maintains training efficiency while enabling rare attack detection through the combined system.
Solution Approach 2:
The patent implements partial action by focusing the rare attack detection capability specifically on the drift detector module rather than requiring the entire neural network to be retrained on all rare attack data. This allows the system to maintain high training efficiency on majority data while still achieving reliable rare attack detection through the specialized drift detector component.
3Measurement precision
If autoencoders are used for reconstruction, then in-distribution data reconstruction is accurate, but out-of-distribution data reconstruction errors are large
Solution Approach 1:
The patent uses feedback by monitoring the reconstruction error output from the autoencoder and feeding this information back to the drift detector. When reconstruction errors exceed a threshold, the drift detector is triggered to analyze the input data for potential rare attacks or distribution shifts. This feedback mechanism transforms the autoencoder's reconstruction performance into a useful signal for detecting out-of-distribution data.
Solution Approach 2:
The patent introduces the drift detector as an intermediary that processes the reconstruction error signals from the autoencoder. Rather than directly using reconstruction errors for classification, the drift detector mediates between the autoencoder's reconstruction performance and the final attack detection decision, improving the reliability of out-of-distribution detection.
Data Source
AI summary
Systems, devices, methods, and computer-readable media for detecting drifted data. A method includes generating, by a trained neural network (NN), a classification for an input cyber data packet, generating, based on a state of one or more layers of the NN responsive to the input, a topological persistence diagram, determining a distance between the topological persistence diagram and a topological feature associated with the classification, and issuing an alert responsive to the distance meeting one or more criterion.