Map-Reduce Document Integrating Queries in Distributed Database
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing map-reduce implementations require a common source format for data, which necessitates normalization and burden programmers with data management operations, including remote connectivity and coordination between nodes, often exceeding their expertise.
Innovation Solution
A computer-readable storage medium with executable instructions defines a map-reduce document that integrates map-reduce functions with queries in a query language, allowing operations to be executed in a distributed database, thereby simplifying data processing and eliminating the need for data normalization and managing remote connectivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a common source format is required for map-reduce operations, then data processing can be standardized, but data normalization operations are needed and data management complexity increases
Solution Approach 1:
The patent introduces a query language as an intermediary layer between the map-reduce framework and various data sources. This query language serves as a mediator that can directly access different data formats (relational databases, files, NoSQL databases) without requiring normalization to a common source format, thereby reducing data management complexity while maintaining format compatibility
Solution Approach 2:
The query language is designed to be universal and can interface with multiple data source types natively. Instead of requiring a single common source format, the system achieves multi-functionality by allowing the same map-reduce operations to work with diverse data formats through the query language abstraction, eliminating the need for data normalization
2Adaptability or versatility
If programmers manually specify data access routines and remote connectivity, then flexibility is achieved, but programming complexity increases and requires diverse skills
Solution Approach 1:
The query language acts as an intermediary that bridges the gap between map-reduce operations and data sources. Programmers write queries in this standardized language rather than manually specifying data access routines and remote connectivity details, reducing programming complexity while maintaining flexibility in accessing different data sources
Solution Approach 2:
The system provides self-service by automatically handling data access routines and remote connectivity coordination through the query language framework. The framework manages these complex operations transparently, allowing programmers to focus only on the core map-reduce logic without needing to specify data management details
Data Source
AI summary
A computer readable storage medium includes executable instructions to define a map-reduce document that coordinates processing of data in a distributed database. The map-reduce document complies with a map-reduce specification that integrates map-reduce functions with queries in a query language. The operations specified by the map-reduce document are executed in the distributed database.