Unified Data Query System for IT Incident Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine data generated from diverse sources within IT environments is challenging due to the complexity and volume of data types and formats, with existing tools lacking efficient visual search and analysis capabilities.
Innovation Solution
An event-based data intake and query system with a flexible schema that allows late-binding extraction rules, enabling field-searchable events and pipelined search queries across disparate data sources, facilitating the identification of data subsets of interest through a user-friendly interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tools allow analysts to search data systems separately and collect results over a network, then data retrieval is possible, but the analysis process becomes piecemeal and inefficient
Solution Approach 1:
The patent combines multiple separate data systems into a unified search interface that allows analysts to search across all systems simultaneously. The system integrates data from diverse sources (databases, cloud services, machine data systems) into a single searchable repository, eliminating the need for separate searches and piecemeal analysis.
Solution Approach 2:
The search system is designed to handle multiple data types and formats universally through a single interface. It supports structured, semi-structured, and unstructured data from various sources using a flexible schema that can adapt to different data formats, providing a universal search capability across heterogeneous systems.
2Speed
If data systems pre-process data based on anticipated analysis needs, then retrieval efficiency improves, but flexibility to analyze all generated data is reduced
Solution Approach 1:
The system employs a dynamic schema that can adapt its structure based on the specific search needs. The flexible schema allows the system to optimize for speed when searching predefined subsets while maintaining the capability to analyze all raw data when needed. The schema can be dynamically adjusted to match the data being searched, providing both speed and flexibility.
Solution Approach 2:
The system performs preliminary indexing and structuring of data in a flexible format that preserves all original information. Rather than pre-processing data into fixed categories that limit analysis, the system prepares data in a way that enables both fast retrieval and comprehensive analysis, maintaining adaptability for future analysis needs.
3Adaptability or versatility
If massive quantities of raw data are stored for later retrieval, then analysis flexibility increases, but data management complexity increases
Solution Approach 1:
The system segments raw data into organized units with standardized schemas while preserving the ability to retrieve and analyze complete datasets. Data is divided into manageable chunks that can be independently processed and searched, reducing management complexity while maintaining analysis flexibility. The segmentation is performed in a way that preserves data relationships and context.
4Quantity of substance
If diverse data types and formats are collected from numerous devices, then data comprehensiveness improves, but search and analysis difficulty increases
Solution Approach 1:
The system changes the parameters of data storage by implementing a flexible schema that can represent diverse data types using a unified structure. Different data formats are converted into a standardized representation that preserves their unique characteristics while enabling consistent search operations. This parameter transformation allows comprehensive data collection without proportionally increasing search difficulty.
Data Source
AI summary
An information technology (IT) and security operations application enables the automatic assignment of incident events to analysts based on a variety of characteristics of the incident events to be assigned, the analysts and analyst teams, and other considerations. An IT and security operations application can perform the automatic assignment of incident events based at least in part on data indicating each analyst's knowledge of certain types of incidents, data indicating each analyst's efficiency at responding to certain types of incidents, and the like, where such data is automatically created and maintained by the application. In this manner, incident events can be efficiently assigned to analysts upon their receipt by the system without the need for a security team to constantly perform a cumbersome incident event assignment process based on a limited set of data, thereby improving analyst teams' ability to efficiently ensure the operation and security of IT environments for which the teams are responsible.