Structured Log Record Enrichment Through Header-Detail Parsing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional log analytics tools struggle to efficiently scale and analyze large volumes of log data from numerous computing systems and applications due to inefficient per-host configuration, redundant processing, and the inability to associate structured and unstructured log messages, leading to challenges in data extraction and analysis.

Innovation Solution

A method is introduced to create a header-details relationship among structured log records using two parsers, one for header and one for detail information, enabling enrichment of detail records with extracted header fields, and allowing users to configure this relationship through a user interface or automated pattern detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional log analytics tools use per-host configuration approach, then they can collect and analyze log records from individual systems, but the system cannot efficiently scale when faced with massive systems involving large numbers of computing systems and applications

Engineering Contradiction:

ImprovescalabilityVSAvoidconfiguration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments log records into header records and detail records, where header records contain common information (host, application, service) and detail records contain specific event data. This segmentation enables centralized processing of common fields across all hosts while maintaining the ability to handle detailed host-specific information, thereby achieving scalability without increasing configuration complexity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent creates a universal log format where header records serve multiple purposes: they provide common context for all detail records, enable centralized analysis across heterogeneous systems, and eliminate the need for per-host configuration. The same header structure works across different computing systems, applications, and services, achieving multi-functionality that supports both scalability and simplicity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Productivity

If conventional systems work on a per-host basis, then they can handle individual systems, but redundant processing and resource usage increase significantly

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidredundant processing

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent merges common log fields (host, application, service) into a single header record that is shared across all detail records from the same source. Instead of processing these common fields separately for each host, the system processes them once at the header level and reuses the information for all associated detail records, thereby eliminating redundant processing and reducing resource consumption.

Inventive Principle:

Principle #5Merging (Combining)

3Ease of operation

If unstructured log messages are used, then humans can read and interpret logs easily, but extracting and searching for specific values becomes difficult and error-prone

Engineering Contradiction:

Improvelog readabilityVSAvoiddata extraction accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent segments log messages into structured header records and detail records, separating the common contextual information from the specific event data. This segmentation allows the system to maintain structured formatting for machine processing while preserving human-readable content in the detail records, thereby achieving both ease of operation and measurement precision.

Inventive Principle:

Principle #1Segmentation

4Measurement precision

If structured log messages with consistent format are used, then data extraction becomes reliable, but flexibility in handling different log formats is reduced

Engineering Contradiction:

Improvedata extraction reliabilityVSAvoidformat flexibility

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent segments the log format into a standardized header portion and a flexible detail portion. The header provides consistent structure for reliable data extraction, while the detail records can accommodate various log formats and structures from different sources. This segmentation enables the system to maintain extraction reliability while preserving format flexibility.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by making the header structure standardized and consistent across all logs, while allowing the detail structure to vary according to specific log sources. This enables reliable extraction of common fields through the consistent header format while maintaining flexibility in handling different detail formats from various systems.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20250315477A1System and techniques for enriching log records with fields from other log records in structured format

Publication Date: 2025.10.09 ORACLE INT CORP

AI summary

Embodiments relate to extracting various portions of log messages. Syntax, order and/or level information that differentiates header and detail message is identified. This syntax, order and/or level information is used to automatically associate enhance each of one or more detail log entries with information that provides context. This approach can facilitate efficient transmission of detail-information log entries that reduces redundant information, while still supporting flexible approaches for providing information that can enhance detail-information log entries.