Distributed Database Co-location via Greedy Algorithms

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

As network usage increases, managing and analyzing data streams in distributed databases leads to network latency and delayed operations due to inefficient data storage and query execution across geographically distributed data stores, resulting in excessive network resource consumption.

Innovation Solution

A data management application is implemented to optimize data storage by identifying and co-locating relevant data structures using a greedy algorithm, ensuring that data structures involved in queries are stored at the data store with the highest number of relevant data structures and sufficient storage space, thereby reducing the need for data movement and optimizing query execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is stored in geographically distributed data stores, then data storage capacity and availability are improved, but network latency and query execution time increase

Engineering Contradiction:

Improvedata storage capacityVSAvoidnetwork latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system performs preliminary actions by co-locating data structures that are likely to be queried together before queries are executed. The data management application analyzes query patterns and proactively moves data structures to the same data store, so that when queries are executed, the data is already in the optimal location, eliminating the need for time-consuming network transfers during query execution.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system segments the distributed database into multiple data stores and applies different co-location strategies to different data structures based on their query patterns. By dividing the data management approach into segments (different data structures, different query patterns), the system can optimize each segment independently, reducing overall network latency while maintaining distributed storage benefits.

Inventive Principle:

Principle #1Segmentation

2Productivity

If data structures are co-located at a single data store, then query execution efficiency is improved, but network bandwidth consumption increases due to data movement

Engineering Contradiction:

Improvequery execution efficiencyVSAvoidnetwork bandwidth consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system applies partial co-location by moving only the necessary data structures to the same data store based on query pattern analysis, rather than moving all data. The data management application selectively identifies data structures that are frequently queried together and co-locates only those, avoiding excessive data movement while still achieving query execution efficiency improvements.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system changes the parameter of data location by dynamically adjusting where data structures are stored based on query patterns. The data management application monitors query execution and modifies the location parameter of data structures over time, moving them to optimal locations only when beneficial, thus balancing query efficiency with network bandwidth consumption.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If data structures are frequently moved between data stores, then optimal query execution is achieved, but system complexity and operational overhead increase

Engineering Contradiction:

Improvequery execution performanceVSAvoiddata management complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system implements self-service by having the data management application automatically analyze query patterns and make co-location decisions without manual intervention. The application monitors query execution, identifies co-location opportunities, and moves data structures autonomously, reducing the operational overhead and complexity of managing a distributed database while maintaining optimal query performance.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system uses feedback from query execution patterns to drive co-location decisions. The data management application continuously monitors query performance and data access patterns, using this feedback to determine when and where to move data structures. This feedback mechanism automates the complexity of data management while ensuring that co-location actions are taken only when they will improve query execution.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9165018B2Managing a distributed database

Publication Date: 2015.10.20 AT&T INTELLECTUAL PROPERTY I L P

AI summary

Concepts and technologies are disclosed herein for managing a distributed database. A data management application can obtain a query. The data management application can analyze the query to determine a number of data structures relevant to the query. The data management application also can analyze data stores storing the data structures and move or assign data structures to other data stores within a distributed database. The movement of the data structures within the distributed database can be based upon greedy algorithms for moving data and/or executing queries.