Log Collection Using Shared Compression Dictionaries for Narrow Bands

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The existing log collection systems face performance degradation due to high communication resource consumption caused by large text log data sizes, especially when multiple log generation devices transmit data over narrow communication bands, limiting the number of devices that can be managed by a single log collection device.

Innovation Solution

A log collection system that generates a compressed dictionary based on received text logs and transmits it to log generation devices, allowing them to perform compression and decompression using this dictionary, thereby reducing the processing load and data size transmitted over the communication network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If text logs are transmitted in original format, then log data完整性 is maintained, but communication resource consumption increases

Engineering Contradiction:

Improvelog data完整性VSAvoidcommunication resource consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The log collection device performs preliminary analysis on received text logs to generate a compressed dictionary before actual compression transmission. This preliminary action enables subsequent log data to be compressed using the pre-generated dictionary, reducing communication resource consumption while maintaining data完整性

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

A compressed dictionary is introduced as an intermediary between the log generation devices and log collection device. The dictionary serves as a reference that enables compression of log data without losing information, acting as a mediator that allows efficient transmission while preserving data完整性

Inventive Principle:

Principle #24Intermediary (Mediator)

2Use of energy by moving object

If lexicographic compression is performed at log generation device, then communication resource consumption is reduced, but processing load of log generation device increases

Engineering Contradiction:

Improvecommunication resource consumptionVSAvoidprocessing load of log generation device

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

Instead of having log generation devices perform compression (which increases their processing load), the invention inverts the approach by having the log collection device generate the compressed dictionary and then use it to compress received logs. This shifts the compression burden from log generation devices to the log collection device, reducing communication resource consumption without increasing the processing load of log generation devices

Inventive Principle:

Principle #13The other way round (Inversion)

3Adaptability or versatility

If one log collection device manages many log generation devices, then system scalability is improved, but performance degradation occurs due to communication bandwidth limitations

Engineering Contradiction:

Improvesystem scalabilityVSAvoidsystem performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The invention changes the parameter of data transmission by compressing log data using a dictionary before transmission. This parameter change (from raw text to compressed format) significantly reduces the amount of data transmitted over the communication bandwidth, allowing one log collection device to manage more log generation devices without performance degradation, thus improving system scalability

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10469101B2Log collection device, log generation device, and log collection method

Publication Date: 2019.11.05 PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD

AI summary

There is provided a log collection device including: a log receiver that receives a text log from a log generation device; a dictionary generator that generates a compressed dictionary for performing text compression based on the received text log; a dictionary transmitter that transmits the generated compressed dictionary to the log generation device and instructs performing of the compression process using the compressed dictionary on the text log transmitted after the transmission of the compressed dictionary; and a decompression processor that performs a decompression process using the compressed dictionary on the text log received after transmission of the compressed dictionary.