Log Information Compression and Correlation System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log information from various systems and sensors is often spread across multiple rows in databases, making it difficult to understand the context and requiring significant storage space, as extraneous information is not grouped together, hindering efficient analysis and storage.
Innovation Solution
A system that compresses and coalesces log information from multiple rows into a single row while preserving useful data, using compression rules to combine, discard, or select information, thereby grouping logs by entity and session, making it easier to understand and store.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If log information is stored in separate rows for each system and sensor, then complete information is preserved, but storage space is consumed and data is difficult to understand
Solution Approach 1:
The patent merges multiple log rows from different systems and sensors into a single consolidated log row. This is achieved by identifying common entities across logs and combining their information while removing redundant extraneous data, thereby reducing storage requirements while preserving essential information.
Solution Approach 2:
The patent extracts and removes extraneous information from individual log rows that is not useful for understanding the overall context. By filtering out redundant data and keeping only essential information, the system reduces storage consumption while maintaining data utility.
2Reliability
If log information from multiple systems is stored separately, then each system's data is complete, but understanding the overall context becomes difficult
Solution Approach 1:
The patent combines log information from multiple systems into a unified view by identifying common entities and correlating logs across systems. This consolidation presents a coherent narrative that is easier to understand while maintaining the completeness of individual system data through careful information selection.
Solution Approach 2:
The patent introduces an intermediary processing layer that correlates and contextualizes logs from different systems. This intermediary layer translates scattered system-specific logs into a unified contextualized view, making it easier to understand overall system behavior while preserving the reliability of source data.
3Loss of information
If detailed log information is maintained for each entity, then complete analysis data is available, but the complexity of data processing increases
Solution Approach 1:
The patent merges processing operations by consolidating multiple log rows into single representative rows. This reduces the volume of data that needs to be processed while maintaining analytical capability, thereby decreasing processing complexity without sacrificing analysis data availability.
Solution Approach 2:
The patent applies different processing treatments to different parts of the log data based on their importance. Essential information is preserved and correlated across systems, while extraneous information is filtered out, creating a differentiated processing approach that reduces overall complexity while maintaining analytical completeness.
Data Source
AI summary
Methods, systems and apparatus, including computer programs encoded on computer storage media for compressing sensor log information. One of the methods includes accessing log information maintained in one or more databases, the log information being generated in response to actions associated with entities, and the log information indicative of respective sessions for which one or more logs were generated, each log indicating an entity. Log information is grouped according to entity. One or more logs associated with respective sessions based on the grouped log information. Compressed logs are generated from logs associated with respective sessions based on compression rules.