Event View Selector for Flexible Data Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern data centers face challenges in analyzing massive quantities of heterogeneous performance data due to the discarding of raw data during pre-processing, limiting the ability to investigate unanticipated analysis needs, and the difficulty in applying semantic meaning to unstructured data.
Innovation Solution
An event view selector for a search user interface that allows transitions between different views of events, including raw, list, and table views, using a late binding schema to extract relevant field values at search time, enabling flexible analysis of minimally processed performance data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If raw performance data is discarded during pre-processing to reduce data size, then storage efficiency is improved, but the ability to perform unanticipated analysis is lost
Solution Approach 1:
The system performs preliminary indexing of raw performance data during data ingestion, creating a searchable structure that preserves the original data while enabling efficient retrieval. This preliminary action allows the system to maintain adaptability for unanticipated analysis without storing redundant copies of the entire raw dataset.
Solution Approach 2:
The patent introduces an intermediary layer between raw data ingestion and analysis that maintains the original performance data in an indexed format. This intermediary structure acts as a mediator that enables both space-efficient storage and flexible querying, allowing analysts to perform unanticipated analysis without requiring the original raw data to be preserved in its initial form.
2Productivity
If pre-specified data items are extracted and stored during data ingestion, then retrieval efficiency is improved, but the ability to analyze unanticipated data items is reduced
Solution Approach 1:
The system implements a universal indexed data structure that can serve multiple analysis purposes simultaneously. The indexed performance data is organized in a manner that enables both efficient retrieval for pre-specified analysis and flexible querying for unanticipated analysis, making the storage system multi-functional rather than specialized for a single purpose.
Solution Approach 2:
The patent employs dynamic schema evolution capabilities that allow the data structure to adapt to new analysis requirements over time. The indexed data can be reinterpreted and re-analyzed with different schemas and extraction rules without requiring re-ingestion, enabling the system to evolve from pre-specified to unanticipated analysis as needs change.
3Volume of stationary object
If data is pre-processed to reduce size, then storage requirements are reduced, but the capability to re-analyze original data is lost
Solution Approach 1:
The system creates an indexed copy of the raw performance data that preserves the essential information needed for analysis while occupying less storage space. This indexed representation maintains the fidelity and integrity of the original data for analytical purposes without requiring storage of the complete raw dataset, enabling re-analysis with reduced storage requirements.
4Adaptability or versatility
If extraction rules are applied at search time, then flexibility in analysis is improved, but processing time increases
Solution Approach 1:
The system performs preliminary indexing of performance data with extracted field values during data ingestion, creating a pre-processed structure that enables fast search-time queries. This preliminary extraction and indexing action reduces the computational burden during actual searches, allowing flexible extraction rules to be applied without excessive processing delays.
Data Source
AI summary
An event view selector for a search user interface is described. In one or more implementations, a service may operate to collect and store data as events and apply a late binding schema to extract events that match the search criteria and provide search results for display via the search user interface. The search user interface exposes an event view selector operable to enable transitions between multiple different views of the events associated with different levels of detail. The views may include at least a raw view, a list view, and a table view. Responsive to receiving an indication of a view selected via the event view selector, the selected view may be exposed via the search user interface.