Interactive Data Summary Views with Selectable Event Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern data centers face challenges in processing and indexing large volumes of heterogeneous performance data due to its unstructured nature, making it difficult to apply semantic meaning and perform efficient searches, especially when the initial data discard leads to loss of relevant information.
Innovation Solution
The SPLUNK® ENTERPRISE system employs a late-binding schema and event-based processing to store and analyze performance data, allowing flexible extraction of field values and indexing at search time, using techniques like map-reduce computations, keyword indices, and high-performance analytics stores to optimize data retrieval and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is pre-processed and stored in a structured form, then storage space is saved and retrieval efficiency is improved, but relevant information may be lost and the data cannot be re-analyzed later
Solution Approach 1:
The system performs preliminary indexing of unstructured data using a late-binding schema that preserves the original data structure while creating searchable indexes. This allows the data to be prepared for retrieval without imposing a rigid structured format that would lose information, enabling both efficient search and future re-analysis.
Solution Approach 2:
The patent changes the parameter of data structure from strictly structured to semi-structured with late-binding schema. This allows the system to maintain flexibility in data representation while still enabling efficient querying through indexes, resolving the contradiction between structure for efficiency and flexibility for information preservation.
2Loss of information
If unstructured data is maintained to preserve all information, then data completeness is improved, but indexing and searching operations become difficult
Solution Approach 1:
The system segments the data handling process into two parts: storing the complete unstructured data to preserve information, and creating separate indexes that capture semantic meaning. This segmentation allows the full data to be retained while indexes provide efficient search capabilities without requiring the entire system to be restructured.
Solution Approach 2:
The patent introduces an intermediary indexing layer that sits between the unstructured data storage and the search operations. This intermediary creates semantic representations of the unstructured data, enabling efficient searching without requiring the underlying data to be structured, thus reducing searching complexity while maintaining data completeness.
3Quantity of substance
If large volumes of search results are returned, then comprehensive information is provided, but user interpretation becomes difficult
Solution Approach 1:
The system extracts and highlights the most relevant portions of search results based on the query and semantic analysis. Instead of presenting all matching data uniformly, it extracts and prioritizes the most pertinent information, making large volumes of results more interpretable while still providing comprehensive coverage when needed.
Solution Approach 2:
The patent applies local quality by providing different levels of detail and organization for different portions of search results. Highly relevant results receive more prominent presentation and detailed annotation, while less relevant results are presented more concisely, making the overall large volume of results easier to interpret based on local importance.
Data Source
AI summary
In some embodiments, a method may include display of a data summary view of a set of events that correspond to query results of a query. Each event of the set of events may include data items of a plurality of event attributes. In embodiments, the data summary view can include various summary reports. Each summary report can include summary entries and a summary graph that each present a summary of data items of a selected event attribute, of the plurality of event attributes. At least one summary report can include summary entries that are selectable by a user. The method may further include filtering the set of event, in response to, and based on, selection of one or more of the selectable summary entries by the user and updating of at least the first and second summary graphs to correspond to the filtered set of events.