Structured Pattern Logging Protocol for Storage Data Mining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data mining systems face inefficiencies in deriving patterns from unstructured data sources, leading to increased computational load and noise in searching for relevant data, as they must sift through superfluous, non-related data.
Innovation Solution
Implementing a structured pattern logging protocol that assigns timestamps, event identifiers, states, types, triggers, and severity weights to events, allowing for efficient data parsing and storage, and enabling the identification of related events and sequences, thereby reducing the noise and size of relevant datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional data mining systems process unstructured data sources, then comprehensive data coverage is achieved, but computational load increases and noise in searching for relevant data increases
Solution Approach 1:
The patent segments data into structured events with specific categories (system events, user events, business events) and hierarchical levels (enterprise, department, team, individual). This segmentation allows the data mining system to process only relevant event types rather than all unstructured data, reducing computational load while maintaining comprehensive coverage of pertinent information.
Solution Approach 2:
The system performs preliminary structuring of data by assigning standardized schemas, tags, and metadata to events before they enter the data mining process. Event schemas define expected data formats and structures in advance, allowing the system to quickly filter and process relevant events without performing extensive analysis on raw unstructured data.
2Reliability
If conventional data mining systems search through unstructured data, then all potential patterns are captured, but time spent searching for relevant data increases
Solution Approach 1:
The patent applies local quality by assigning different levels of detail and structure to different types of events based on their relevance and importance. Critical events receive more detailed structuring and higher priority tags, allowing the system to quickly identify and analyze important patterns while using lighter processing for less critical events, thereby reducing overall search time without missing important patterns.
Solution Approach 2:
The system introduces event schemas and standardized data structures as intermediaries between raw data and pattern analysis. These schemas act as a filtering layer that pre-organizes data according to expected patterns, allowing the data mining system to quickly match events against known patterns without searching through all raw data, thus reducing search time while maintaining pattern detection completeness.
3Productivity
If structured pattern logging protocol is implemented, then data parsing efficiency improves, but system complexity increases
Solution Approach 1:
The patent implements universal event schemas that can accommodate multiple types of events (system, user, business) across different organizational levels through a single standardized framework. This multi-functional schema design allows the system to parse diverse event types using the same structural rules, improving parsing efficiency without requiring separate complex processing logic for each event type, thus managing system complexity.
4Loss of information
If all events are logged and parsed according to logging protocol, then complete event data is available, but computational load increases
Solution Approach 1:
The system implements partial action by selectively logging and parsing events based on their relevance, priority, and organizational context. Not all events are processed with the same level of detail - the system applies filtering rules that process only necessary events in full detail while using summarized or aggregated processing for less critical events, maintaining data completeness for important information while reducing overall computational load.
Data Source
AI summary
Systems and methods for improving data mining systems are described. In one embodiment, the systems and methods may include a storage drive and a hardware controller. In some embodiments, the hardware controller may be configured to detect a first event in the storage system, identify data associated with the first event, parse the data according to a logging protocol, and store the parsed data in a database.