Sparse Table Compaction via Virtual Page Mapping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for managing sparse tables in computer memory lead to wasteful storage and data loss due to inefficient handling of repeated data patterns, especially when interspersed with unique patterns.

Innovation Solution

A method where repeating virtual data pages are mapped to a single physical page on a many-to-one basis, while unique pages are mapped one-to-one, utilizing a translation lookaside buffer and real mapping facility to optimize storage by eliminating unnecessary data duplication.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If repeating virtual data pages are mapped to a single physical page on a many-to-one basis, then storage space is optimized and data duplication is eliminated, but data integrity may be compromised when repeated patterns are interspersed with unique patterns

Engineering Contradiction:

Improvestorage spaceVSAvoiddata integrity

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The patent segments the sparse table into distinct regions: a first portion containing repeating patterns that are compacted together, and a second portion containing unique patterns that are stored separately. This segmentation allows the system to apply different storage strategies to different data types, optimizing space for repeated patterns while preserving integrity for unique patterns.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by treating repeating patterns and unique patterns differently within the same data structure. Repeating patterns in the first portion are compacted using many-to-one mapping, while unique patterns in the second portion are preserved with one-to-one mapping, ensuring each type of data receives the appropriate storage treatment.

Inventive Principle:

Principle #3Local quality

2Quantity of substance

If all repeated pages prior to the last repeated page are skipped using an offset, then storage is reduced, but data is lost especially when repeated patterns are interspersed with non-repeated patterns

Engineering Contradiction:

Improvestorage spaceVSAvoiddata loss

Core Design Contradiction:

Quantity of substanceVSLoss of information

Solution Approach 1:

The patent divides the sparse table into a first portion for repeating patterns and a second portion for unique patterns. This segmentation prevents the loss of unique patterns that would occur if all pages prior to the last repeated page were simply skipped, as unique patterns in the second portion are explicitly preserved.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the storage parameter from uniform one-to-one mapping to a hybrid approach: many-to-one mapping for the first portion containing repeating patterns, and one-to-one mapping for the second portion containing unique patterns. This parameter change optimizes storage for repeated data while preserving unique data.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS7516298B2Sparse table compaction method

Publication Date: 2009.04.07 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method and system of sparse table compaction is disclosed. A repeating data pattern may be detected in a large data structure, identifying the large data structure as a sparse table. The large data structure is stored in a virtual memory as a series of virtual data pages. Multiple repeating virtual data pages may be mapped to a single physical data page on a multiple-to-one basis. Unique virtual data page may be mapped to a unique physical data page on a one-to-one basis.