Database Sampling via Reverse Key Extraction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional database sampling methods often fail to preserve the semantics of production data, leading to inefficient resource consumption and complex processes due to random extraction and multiple iterations, which can result in semantically incorrect sample data.

Innovation Solution

A method that identifies query and associated tables based on filtering criteria and order lists, generates key value lists, and extracts sample data in a reverse order to efficiently extract a semantically correct portion of production data, reducing computational resources and time by traversing tables in two iterations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If random extraction of sample data is performed from production database, then extraction process is simple, but extracted sample data does not preserve semantics and may be semantically incorrect

Engineering Contradiction:

Improveextraction process simplicityVSAvoidsemantic correctness of sample data

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The system performs preliminary actions by identifying query tables and associated tables based on filtering criteria before extraction, and by creating order lists that define the extraction sequence. This preliminary organization ensures that when data is extracted, the semantic relationships are preserved while maintaining a systematic approach that doesn't require complex iterative processes.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system inverts the conventional approach by extracting data in reverse order of the order list rather than following the list sequentially. This inversion, combined with using key value lists generated in advance, ensures that foreign key relationships and semantic dependencies are maintained in the extracted sample data without requiring multiple iterative passes.

Inventive Principle:

Principle #13The other way round (Inversion)

2Reliability

If multiple iterations are used to extract semantically correct sample data, then semantic correctness is improved, but computational resources and time consumption increase

Engineering Contradiction:

Improvesemantic correctness of sample dataVSAvoidextraction speed and resource efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system performs all necessary preliminary actions in a single pass: identifying query and associated tables, creating order lists that define extraction sequence, and generating key value lists that capture relationships. This eliminates the need for multiple iterations while ensuring semantic correctness, thereby improving productivity without sacrificing reliability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces key value lists as intermediary structures that capture the semantic relationships and dependencies between tables. These key value lists serve as mediators that guide the extraction process, ensuring that sample data maintains semantic correctness while enabling single-pass extraction that improves computational efficiency.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If entire production data is copied to test environment, then testing reliability is improved, but space requirements and resource consumption increase

Engineering Contradiction:

Improvetesting reliabilityVSAvoidstorage space requirement

Core Design Contradiction:

ReliabilityVSVolume of stationary object

Solution Approach 1:

The system extracts only the necessary portion of production data that is required for specific test cases, rather than copying the entire production database. By identifying query tables and associated tables based on filtering criteria, the system extracts a minimal subset of data that maintains semantic correctness and satisfies testing requirements, thereby reducing storage space requirements while preserving testing reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system applies local quality by extracting data with specific characteristics required for testing rather than uniform random sampling. By using filtering criteria to identify relevant tables and data, the extracted sample data has the local quality needed for specific test cases, ensuring testing reliability while minimizing the volume of data stored in the test environment.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS9026548B2Database sampling

Publication Date: 2015.05.05 TATA CONSULTANCY SERVICES LTD

AI summary

The present subject matter relates to systems and methods for database sampling. The method comprises identifying at least one query table and one or more associated tables amongst a plurality of tables in a production database, based on filtering criteria. Further, the method comprises generating a key value list for the at least one query table and each of the one or more associated tables based on an order indicated by an order list. Based on the generated key value list, the sample data is extracted in a reverse order indicated by the order list, from the at least one query table and each of the one or more associated tables.