Fleet Storage Threat Detection Using Local and Cloud ML Stages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing and securing large fleets of storage systems, particularly in detecting and mitigating multi-dimensional threats across distributed networks.
Innovation Solution
Implementing a cloud-based monitoring system with a fleet-level analysis module using machine learning models to analyze threat probability scores and payload data from multiple storage systems, enabling comprehensive threat detection and management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional monitoring systems are used for storage fleets, then individual system monitoring is possible, but comprehensive fleet-wide threat detection and multi-dimensional analysis is insufficient
Solution Approach 1:
The monitoring system is divided into multiple independent modules: data collection module, threat detection module, analysis module, and response module. Each module handles specific aspects of monitoring, allowing comprehensive fleet-wide threat detection while maintaining modular architecture that manages complexity through functional separation.
Solution Approach 2:
The system transitions from traditional single-dimensional monitoring to multi-dimensional analysis by incorporating threat probability scores, payload data analysis, behavioral patterns, and contextual information. This dimensional expansion enables comprehensive security assessment across the entire storage fleet without overwhelming complexity in any single monitoring component.
2Measurement precision
If detailed threat analysis is performed on all storage systems, then detection precision is improved, but processing time and system response speed decrease
Solution Approach 1:
The system performs preliminary actions by continuously collecting and pre-processing data from storage systems, maintaining baseline behavioral patterns and threat probability scores. This allows the analysis module to make rapid assessments without performing exhaustive analysis from scratch, thus maintaining both precision and speed.
Solution Approach 2:
The system applies partial analysis by focusing computational resources on the most critical indicators and high-probability threats rather than uniformly analyzing all data points. This selective approach maintains detection precision for critical issues while reducing overall processing time through targeted analysis.
3Reliability
If real-time monitoring of all storage systems is implemented, then threat detection capability is improved, but data volume and information processing requirements increase
Solution Approach 1:
The system extracts only the most relevant information from the vast amount of data collected from storage systems. By filtering and selecting critical parameters such as threat probability scores, anomaly indicators, and high-value payload data, the system reduces information processing overhead while maintaining comprehensive security monitoring capability.
Solution Approach 2:
The system implements feedback mechanisms where threat detection results and analysis outcomes are fed back into the monitoring process. This allows continuous refinement of monitoring parameters and prioritization of high-value information, reducing the effective information processing load by focusing on the most relevant data streams.
Data Source
AI summary
A system includes a fleet of storage systems and a cloud-based monitoring system configured to monitor for security threats against the fleet. A first storage system in the fleet is configured to use a first local ML model trained on confirmed threat patterns to perform a first analysis of a first plurality of attributes associated with operations performed with respect to the first storage system during a first short-time window and determine, based on the first analysis, a first threat probability score. If the score meets a threshold, the first storage system sends the score and payload data to the cloud-based monitoring system, which performs, based on the received data, a fleet-level cloud-based analysis to determine a likelihood that the fleet of storage systems is being targeted by the security threat.