Metadata Agent Query Routing for Distributed Database Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In large-scale database management systems, queries often need to be processed across multiple servers, leading to inefficiencies as all servers are typically queried, even if only a subset holds the required data, resulting in increased processing time and resource utilization.

Innovation Solution

A centralized database management system where a master device intercepts and interprets queries, identifying specific segment devices storing relevant data records using a metadata table, and sends queries only to those devices, leveraging a query execution engine to aggregate results and respond to clients.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If queries are sent to all servers in a distributed database system, then all servers can potentially return relevant results, but processing time and resource utilization increase significantly

Engineering Contradiction:

Improvecompleteness of query resultsVSAvoidquery processing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-computing and storing metadata information (minimum and maximum values of indexed columns) in metadata tables before queries arrive. This allows the query router to quickly determine which segment devices contain relevant data without querying all devices, thus reducing query processing time while maintaining result completeness

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a query router as an intermediary component between the client and segment devices. The router uses pre-stored metadata to filter and redirect queries only to relevant segment devices, eliminating the need for all devices to process every query. This intermediary layer maintains reliability by ensuring all relevant devices are queried while reducing time waste by excluding irrelevant devices

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If queries are sent to all servers in a distributed database system, then no relevant data is missed, but resource utilization across the system increases

Engineering Contradiction:

Improvecompleteness of query resultsVSAvoidsystem resource utilization

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary actions by pre-computing metadata (min/max values) and storing it in metadata tables before queries arrive. This allows the query router to efficiently determine which segment devices should receive queries, reducing unnecessary processing on irrelevant devices and lowering overall system resource utilization while maintaining complete result retrieval

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of querying all segment devices (excessive action), the system uses metadata-based routing to query only the necessary subset of devices (partial action). This selective approach reduces resource utilization by avoiding redundant query processing on devices that cannot possibly contain relevant data, while still ensuring all relevant data is retrieved

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If a centralized metadata table is implemented to route queries selectively, then query routing efficiency improves, but system complexity increases

Engineering Contradiction:

Improvequery routing efficiencyVSAvoidsystem architectural complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the database system into distinct functional components: a query router responsible for intelligent query routing, segment devices responsible for data storage and query execution, and metadata tables storing routing information. This segmentation allows each component to be optimized independently, improving query routing efficiency while managing complexity through clear separation of concerns

Inventive Principle:

Principle #1Segmentation

4Measurement precision

If metadata tables store minimum and maximum values for indexed columns, then query filtering accuracy improves, but storage requirements increase

Engineering Contradiction:

Improvequery filtering accuracyVSAvoidmetadata storage size

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent applies local quality by storing metadata (min/max values) only for columns that are defined as indexed columns in the table schema. Not all columns require metadata storage—only those that are frequently used for query filtering and have appropriate data types. This selective approach improves query filtering accuracy for indexed columns while minimizing the storage overhead of metadata tables

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10599728B1Metadata agent for query management

Publication Date: 2020.03.24 EMC IP HLDG CO LLC

AI summary

A method for processing queries includes a master device receiving a query from a client device, and interpreting the query to identify a data record identifier matching the query. The method further includes obtaining, using a metadata table on the master device, a segment device identifier storing a data record identified by the data record identifier for the query, sending the query to a segment device, the segment device being identified by the segment device identifier, and receiving a result of the query from the segment device. The method further includes transmitting the result to the client device.