Synthetic Dataset Generation for Database Query Testing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The challenge is to generate a dataset against which a problematic query can be run, especially when no actual dataset is available, to recreate observed problems in database testing and troubleshooting, while avoiding security risks and resource inefficiencies.

Innovation Solution

A computer-implemented method that transforms queries with subqueries into transformed queries, generating query blocks with predicates, and uses this information to select data for a dataset, allowing the query to be executed and producing results.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If actual customer data is used for query testing and troubleshooting, then query results can be accurately reproduced, but data security risks increase and resource consumption increases

Engineering Contradiction:

Improvequery result accuracyVSAvoiddata security risk

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent creates synthetic datasets that copy the structural and relational characteristics of actual customer data without using the real data itself. The system generates test data that reproduces the schema, constraints, and relationships needed for accurate query testing while eliminating security risks associated with using actual customer information.

Inventive Principle:

Principle #26Copying

2Reliability

If actual customer data is used for query testing and troubleshooting, then query results can be accurately reproduced, but resource consumption increases

Engineering Contradiction:

Improvequery result accuracyVSAvoidresource consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

Instead of loading and processing large volumes of actual customer data, the system generates lightweight synthetic datasets that replicate only the necessary structural and relational properties. This copying approach significantly reduces resource consumption while maintaining the ability to accurately reproduce query results for testing and troubleshooting.

Inventive Principle:

Principle #26Copying

3Measurement precision

If subqueries are converted to outer query predicates, then query transformation accuracy improves, but processing complexity increases

Engineering Contradiction:

Improvequery transformation accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the query transformation process into distinct phases: identifying subqueries, extracting predicates, converting them to outer query equivalents, and validating the transformation. This segmentation manages processing complexity by breaking down the complex task into manageable steps while maintaining high transformation accuracy through systematic handling of each component.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11775517B2Query content-based data generation

Publication Date: 2023.10.03 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Query content-based data generation includes obtaining a query having an outer query and one or more subqueries, converting predicates of a subquery to be predicates of the outer query, and thereby obtain a transformed query, generate from the transformed query a query block having a list of predicate(s) selected from the transformed query, obtaining column information based on the list of predicate(s) and using the column information to select data for a dataset such that data records from the dataset are produced as results to executing the obtained query against the dataset.