Cross-Source Log Query Mapping Without Schema Normalization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing log data sources with different storage technologies, schemas, and formats require costly and error-prone normalization processes, leading to high computational and storage demands, as well as increased maintenance efforts.

Innovation Solution

Classify label fields across different log data source types using maps that store path information and field type mappings, generating instance-level queries without requiring schema conformance, thus reducing normalization needs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If log data from different sources with different storage technologies, schemas, and formats is collected and normalized, then data accessibility and relationships between log data are improved, but computational and storage demands increase significantly

Engineering Contradiction:

Improvedata accessibilityVSAvoidcomputational demand

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent segments the log data collection system into multiple independent components: different log data sources (firewall, IDS, HTTP server) with their own storage technologies and schemas can be queried independently through a unified interface, avoiding the need to normalize and store all data centrally. Each source maintains its own data structure while the query distribution system handles the integration logic.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a query distribution system as an intermediary between the user and multiple log data sources. This mediator receives unified queries, translates them into source-specific queries based on stored metadata about each source's schema and storage technology, and aggregates results without requiring data normalization or central storage.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

Engineering Contradiction:

Improvedata accessibilityVSAvoidmaintenance effort

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The system segments maintenance tasks by allowing each log data source to maintain its own data and schema independently. Changes to one source's schema or storage technology do not affect others, and the query distribution system adapts automatically using stored metadata, eliminating the need for centralized normalization maintenance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent uses parameter changes by storing metadata about each log data source's schema, storage technology, and format as configurable parameters. When sources change their data structures, only their metadata parameters need updating in the query distribution system, not the entire normalization framework, making maintenance simpler and more localized.

Inventive Principle:

Principle #35Parameter changes

3Manufacturing precision

If normalization processes are applied to log data from different sources, then schema conformance and data consistency are improved, but error rates increase

Engineering Contradiction:

Improveschema conformanceVSAvoiderror rate

Core Design Contradiction:

Manufacturing precisionVSReliability

Solution Approach 1:

Instead of normalizing data to achieve schema conformance (traditional approach), the patent inverts the approach by having the query distribution system translate queries to match each source's existing schema. This avoids normalization errors while maintaining effective data access and relationship querying across diverse sources.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS20260017322A1Querying across different log data source types

Publication Date: 2026.01.15 CROGL INC

AI summary

A system may attempt to classify, according to field types, label fields used by structures in instances of different log data source types, which use different structure types and different path information to access the structures of those structure types within the instances. Each of the structures includes at least one of the label fields and at least one data field associated with that label field. The system may store a result that includes the path information to access the structures and mappings of at least some of the field labels to the field types. The system may determine a field type involved in a system level query, and then generate a set of query graphs based on the field type and the result. The system may generate a set of instance level queries for a set of the instances according to the set of query graphs.