Data Storage Method Using Intermediate Format Conversion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional Database Management Systems (DBMS) face performance issues with response times and data synchronization when handling large sets of static and read-only data, particularly for single-key lookups, due to internal mechanisms like concurrency control, access control, and logging.

Innovation Solution

A data processing method that stores data in an intermediate format, converts it into a production format, and merges two data volumes into one, utilizing cryptographic message digests for uniform hash distribution and delegating index search tasks to the file system, allowing for smaller index storage and efficient data retrieval.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional DBMS is used for storing and querying large datasets, then data management functionality is provided, but response time and synchronization time increase significantly

Engineering Contradiction:

Improvedata management functionalityVSAvoidresponse time and synchronization time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts the essential data storage and retrieval functionality from the traditional DBMS, removing unnecessary overhead components such as concurrency control, access control, transaction handling, and logging mechanisms. This creates a streamlined system that maintains core data management capabilities while eliminating time-consuming operations.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the data into fixed-length records with standardized formatting, organizing data into compact structures that enable efficient processing. By dividing large datasets into manageable fixed-size units, the system achieves faster access and reduced synchronization time while maintaining data integrity.

Inventive Principle:

Principle #1Segmentation

2Reliability

If standard DBMS internal mechanisms are implemented, then data integrity and access control are ensured, but performance is hindered

Engineering Contradiction:

Improvedata integrity and access controlVSAvoidquery performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements a self-service approach where the simplified data structure and format inherently provide data integrity without requiring complex transaction handling or logging mechanisms. The fixed-length record format and standardized organization automatically ensure data consistency, eliminating the performance overhead of traditional integrity checks.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If data is stored in traditional format, then compatibility with standard systems is maintained, but storage efficiency and retrieval speed are reduced

Engineering Contradiction:

Improvesystem compatibilityVSAvoidstorage efficiency and retrieval speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent changes the data storage parameters by implementing fixed-length records with standardized formatting and compact encoding. This transformation optimizes storage efficiency and retrieval speed while maintaining compatibility through standardized data organization principles that can interface with conventional systems.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11809728B2Data storage method for massive datasets

Publication Date: 2023.11.07 SNYK SWEDEN AB

AI summary

Systems and methods for storing data in an intermediate format for storing, converting the intermediate data format into a production data format of two data volumes, and merging the two data volumes into one data volume.