Early Adaptive IO Stream Sampling for Ransomware Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems face challenges in accurately detecting security threats like ransomware attacks while minimizing processing and memory resource requirements.
Innovation Solution
Implementing early and adaptive input/output stream sampling techniques using machine learning models to analyze sub-slices of successive IO operations, generating probability scores, and comparing class signatures to detect potential threats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If interval-based sampling is used to sample slices of IO operations, then security threat detection accuracy is improved, but processing and memory resource requirements increase
Solution Approach 1:
The patent implements adaptive sampling that dynamically adjusts the sampling rate based on the detected IO patterns and threat likelihood. When threats are detected or suspected, the sampling rate increases to capture more detailed patterns for accurate classification. When no threats are present, the sampling rate decreases to conserve processing and memory resources, thus resolving the contradiction between detection accuracy and resource consumption
Solution Approach 2:
The system changes the sampling interval parameter adaptively based on system state. The sampling interval is shortened when threat patterns are detected and lengthened during normal operation. This parameter adjustment allows the system to maintain high detection accuracy when needed while reducing resource usage during normal operation
2Measurement precision
If more IO operation slices are sampled for analysis, then security threat detection accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies partial sampling by analyzing only a subset of IO operation slices rather than all slices. The sampling rate is adjusted to capture sufficient information for accurate threat detection while processing fewer data points than complete analysis would require, thus reducing processing time while maintaining detection accuracy
Solution Approach 2:
The system performs preliminary sampling of IO slices at adjusted rates before conducting full analysis. By pre-identifying suspicious patterns through adaptive sampling, the system can focus detailed analysis only on potentially threatening sequences, reducing overall processing time while maintaining detection accuracy
3Loss of time
If sampling rate is increased to detect threats earlier, then detection timing is improved, but resource consumption increases
Solution Approach 1:
The sampling rate is made dynamic and responsive to system conditions. The system monitors IO patterns and automatically increases sampling intensity when anomalies are detected, enabling earlier threat detection only when necessary. During normal operation, the sampling rate remains low to conserve resources, thus resolving the contradiction between early detection and resource consumption
Data Source
AI summary
Techniques for performing early and adaptive IO stream sampling for ML-based optimizations in a storage system. The techniques include obtaining a sub-slice of sampled data by performing early sampling of a slice of successive operations directed to a storage object. The techniques include generating features based on the sub-slice, processing the features using an ML model, and generating a probability score based on the ML model's output. The techniques include determining that the probability score falls within an overlap range of continuous variable distributions for benign and threat classes of data. The techniques include, in response to the probability score exceeding a specified threshold, comparing a class signature of the sub-slice with a target class signature of the threat class of data to determine a similarity between the class signatures, and, in response to the similarity exceeding a predetermined similarity level, assigning a “threat” class label to the probability score.