Late Encoding Binding for Data Set Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data compression technologies face challenges in selecting the optimal encoding technique due to resource limitations, time constraints, and operational costs, which limits efficient data compression.

Innovation Solution

Implementing late encoding binding, which allows for the evaluation and selection of encoding techniques based on the data's characteristics and history, enabling better-informed encoding decisions at the time of data storage rather than at data creation, while maintaining compatibility with previous selections.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If encoding technique selection is performed at data creation time, then encoding decisions can be made immediately, but the selection is limited by resource constraints and time limitations

Engineering Contradiction:

Improveencoding decision speedVSAvoidencoding technique selection accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system performs preliminary encoding analysis and evaluation at data creation time without making final encoding decisions. Encoding history and metadata are captured in advance, allowing comprehensive evaluation later when resources are available, thus resolving the contradiction between immediate decision speed and selection accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The encoding selection process is segmented into two phases: (1) preliminary evaluation and history capture at data creation, and (2) final encoding technique selection at data storage time. This segmentation allows resource-intensive analysis to be performed when needed without delaying initial data ingestion.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If resource-intensive encoding analysis is performed, then optimal encoding techniques can be selected, but time consumption and operational costs increase

Engineering Contradiction:

Improveencoding technique selection accuracyVSAvoidencoding analysis time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Resource-intensive encoding analysis is performed in advance during data ingestion, with results stored as encoding history. The actual encoding technique selection occurs later during data storage operations, allowing comprehensive analysis without adding time pressure to the critical data ingestion path.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of performing repeated encoding analysis on the actual data, the system creates encoding history records and metadata that capture the essential characteristics needed for encoding selection. These copies enable efficient evaluation without requiring access to the original data during the selection process.

Inventive Principle:

Principle #26Copying

3Ease of operation

If encoding decisions are made at data creation, then the process is simple and fast, but the ability to select optimal encoding techniques is limited

Engineering Contradiction:

Improveencoding process simplicityVSAvoidencoding technique selection flexibility

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system maintains encoding history and metadata captured at data creation without requiring complex decision-making at that stage. This preliminary capture of information enables flexible and adaptive encoding technique selection later during data storage, while keeping the initial data creation process simple and fast.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Encoding history and metadata serve as intermediaries between data creation and encoding technique selection. These intermediaries preserve the simplicity of data creation while enabling sophisticated encoding decisions later, acting as a bridge that decouples the two processes.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Measurement precision

If comprehensive data analysis is performed for encoding selection, then optimal encoding can be achieved, but resource requirements increase

Engineering Contradiction:

Improveencoding selection accuracyVSAvoidcomputational resources for encoding analysis

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

Comprehensive data analysis for encoding selection is performed in advance during data ingestion, with results stored as encoding history. This preliminary action allows resource-intensive analysis to be completed when computational resources are available, without increasing resource requirements during the actual data storage operations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates compact encoding history records that capture the results of comprehensive data analysis without requiring the original data or computational resources. These copies enable accurate encoding selection with minimal resource consumption during the selection process.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS11281641B1Evaluating encoding history for late encoding binding of data

Publication Date: 2022.03.22 AMAZON TECH INC

AI summary

Late encoding binding can be applied to data added to a data set according to the encoding history for the data set. When a request to add data is received, an evaluation of the encoding history can determine whether a currently specified encoding for the data can be changed. If changeable, a different encoding scheme for the data may be identified and the data added to the data set according to the different encoding scheme.