Time-Bounded Event Expediting for IT Security Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine data generated by modern computing environments is challenging due to the vast amount of data and varying formats, leading to inefficiencies in data analysis and insight generation.
Innovation Solution
A data intake and query system utilizing a late-binding schema that applies extraction rules during search time, enabling flexible schema development and field-searchable event storage, along with a metadata catalog for managing extraction rules and facilitating common information models across disparate data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If massive quantities of machine data are stored for later retrieval and analysis, then flexibility and depth of analysis are improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-processing machine data during ingestion to extract and store key fields and metadata before analysis is requested. This includes parsing data formats, extracting relevant fields, and organizing data into searchable structures in advance, so that when analysis is needed, the system can quickly retrieve and process only the necessary information rather than scanning all raw data.
Solution Approach 2:
The patent segments machine data into structured components with defined schemas, organizing data by fields, data types, and metadata categories. This segmentation allows the system to efficiently query and process specific portions of data based on analysis needs, rather than treating all data as a monolithic collection, thereby reducing processing time while maintaining analytical flexibility.
2Productivity
If pre-specified data items are extracted and stored during pre-processing, then retrieval efficiency is improved, but flexibility to analyze different aspects of data is reduced
Solution Approach 1:
The patent implements universality by creating a flexible schema system that can accommodate multiple data formats and field types within a unified structure. The schema is designed to be extensible and adaptable, allowing the same data storage system to support various analysis queries and data extraction patterns without requiring separate pre-processing for each analysis type.
Solution Approach 2:
The patent applies dynamics by making the data schema configurable and adaptable rather than fixed. The system can dynamically adjust to different data formats and analysis requirements, allowing schemas to be modified and extended as new analysis needs emerge, thus maintaining both retrieval efficiency and analytical flexibility.
3Quantity of substance
If extensive pre-processing of machine data is performed, then data volume is reduced and storage requirements decrease, but analysis flexibility and depth are limited
Solution Approach 1:
The patent applies extraction by selectively extracting and storing only the most critical fields and metadata during pre-processing, while retaining references to or capabilities of accessing the full raw data. This extraction approach reduces the volume of data that needs to be actively managed and processed, while still preserving the ability to perform deep analysis when needed by accessing the extracted key information.
Data Source
AI summary
Techniques are described that enable an IT and security operations application to prioritize the processing of selected events for a defined period of time. Data is obtained reflecting activity within an IT environment, wherein the data includes a plurality of events each representing an occurrence of activity within the IT environment. A severity level is assigned to each event of the plurality of events, where the events are processed by the IT and security operations application in an order that is based at least in part on the severity level assigned to each event. Input is received identifying at least one event of the plurality of events for expedited processing to obtain a set of expedited events, and the identified events are processed by the IT and security operations application before processing events that are not in the set of expedited events.