Augmented Process Model for Machine Data Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine-generated data from diverse sources is challenging due to the complexity and volume of data types and formats, requiring efficient data intake and query systems to facilitate real-time operational intelligence.
Innovation Solution
The implementation of an event-based data intake and query system, such as the SPLUNK® ENTERPRISE system, which collects, indexes, and searches machine-generated data using a late-binding schema, allowing flexible data modeling and extraction rules to handle various data formats and sources, enabling real-time analysis and reporting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored in disparate data formats across multiple systems, then each system can maintain its own data structure and processing logic, but it becomes difficult and time-consuming to search for desired data across systems
Solution Approach 1:
The patent introduces an intermediary layer (data lake, common data model, or data virtualization platform) that sits between disparate data sources and the search interface. This intermediary standardizes data access by translating various data formats into a unified structure, enabling efficient cross-system searches without requiring changes to the underlying systems. The intermediary maintains adaptability to different source formats while providing consistent access patterns.
Solution Approach 2:
The patent implements a universal data access layer that can handle multiple data formats and sources through a single interface. This universal layer provides common search capabilities across diverse systems by abstracting away format differences, allowing users to search all data regardless of its original source or structure without needing system-specific search mechanisms.
2Loss of information
If all machine-generated data from diverse sources is collected and stored for analysis, then comprehensive insights can be obtained, but the complexity and volume of data types and formats make analysis challenging
Solution Approach 1:
The patent segments the data analysis system into distinct modular components: data collection modules that handle specific source types, data standardization modules that normalize formats, analysis modules that process standardized data, and query modules that provide user interfaces. Each segment handles specific tasks independently, reducing overall system complexity while maintaining comprehensive data analysis capabilities.
Solution Approach 2:
The patent introduces intermediary components including data lakes for standardized storage, common data models for format normalization, and data virtualization layers that simplify access to diverse sources. These intermediaries absorb the complexity of handling diverse data formats, presenting a simplified interface to analysis tools while maintaining complete data representation.
3Speed
If pre-processing is applied to data before storage to facilitate searching, then search efficiency improves, but the flexibility to search all data types and formats is reduced
Solution Approach 1:
The patent implements dynamic data processing where the level and type of pre-processing applied to data depends on the specific query requirements. Data can be stored in both processed (indexed) and raw formats, with the system dynamically selecting the appropriate processing level based on the search request. This allows efficient searches for common patterns while maintaining the ability to perform comprehensive analysis on all data types when needed.
Solution Approach 2:
The patent applies preliminary actions selectively: essential metadata extraction and basic indexing are performed on all data during ingestion to enable fast searches, while more complex processing is deferred until needed. This preliminary processing provides baseline search efficiency without over-processing data that may not require extensive pre-analysis, preserving versatility for handling diverse data types.
Data Source
AI summary
Embodiments of the present invention are directed to managing process analytics across process components. In some embodiments, an indication of a state of a process instance associated with a process is determined by querying a process engine. In accordance with the indication of the state of the process instance, an augmented process model is used to search machine data for data corresponding with at least one component of the process. The data corresponding with the at least one component of the process is associated with a process instance identifier that uniquely identifies the process instance. Thereafter, the data along with the process instance identifier is provided to the process engine to update the state of the process instance.