Relation Valued Functions for Parallel Database Query Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing User Defined Functions (UDFs) in database management systems are insufficient for modeling complex applications and integrating them into query processing pipelines, as they are not relation-in, relation-out operators and require expertise in internal data structures and system calls, limiting their ability to handle complex queries and large-scale data processing.

Innovation Solution

The introduction of Relation Valued Functions (RVFs) that allow for tuple-set inputs and outputs, enabling complex computations and integration into query processing pipelines, with a separation of RVF-Shell for system interaction and user-function for application logic, and the use of MapReduce computation models within a parallel database engine to enhance UDF technology.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If User Defined Functions (UDFs) are used in database management systems, then application logic can be extended, but they are insufficient for modeling complex applications and require expertise in internal data structures and system calls

Engineering Contradiction:

Improveapplication logic extensionVSAvoiduser complexity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent introduces Relation Valued Functions (RVFs) as an intermediary layer between traditional UDFs and the database management system. RVFs provide a simplified interface that handles complex data structures and system calls internally, while presenting a cleaner abstraction to users. This mediator approach allows users to extend application logic without needing to understand internal DBMS data structures and system calls, thus resolving the contradiction between versatility and ease of operation.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments the UDF functionality into two distinct components: Relation Valued Functions (RVFs) for user-level operations and traditional UDFs for system-level operations. This segmentation allows users to work with simplified RVFs for complex applications while the system handles the complexity of internal data structures through the separated UDF layer, resolving the contradiction by dividing the functionality into appropriate abstraction levels.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If traditional UDFs are used, then simple extensions are possible, but they cannot handle complex queries and large-scale data processing efficiently

Engineering Contradiction:

Improvefunction extension capabilityVSAvoidcomplex query processing capability

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent makes Relation Valued Functions (RVFs) universal by designing them to handle multiple types of operations including complex queries, large-scale data processing, and various application logic extensions. RVFs can operate on relation-valued inputs and produce relation-valued outputs, making them applicable to a wide range of database operations. This multi-functionality resolves the contradiction by providing a single mechanism that can handle both simple extensions and complex processing tasks efficiently.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the parameters of function operations by allowing RVFs to accept and return relation-valued parameters rather than simple scalar values. This parameter change enables the functions to handle complex queries and large-scale data processing by operating on entire relations or sets of data, thus resolving the contradiction between function extension capability and complex query processing capability.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If MapReduce computation models are implemented in parallel database engine, then large-scale data processing performance improves, but system complexity increases

Engineering Contradiction:

Improvelarge-scale data processing performanceVSAvoidsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent implements MapReduce computation models within the parallel database engine in a self-service manner, where the database system automatically manages the complexity of MapReduce operations without requiring external coordination. The engine internally handles data distribution, parallel processing, and result aggregation, thus improving large-scale data processing performance while containing system complexity within the database boundary rather than increasing overall system architecture complexity.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent merges the MapReduce computation model with the parallel database engine architecture, combining two previously separate systems into a unified framework. This merging allows the database engine to natively support MapReduce operations, improving performance for large-scale data processing while avoiding the complexity of integrating separate MapReduce and database systems. The consolidation resolves the contradiction by integrating functionalities rather than adding layers of complexity.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS9268815B2Map-reduce and parallel processing in databases

Publication Date: 2016.02.23 HEWLETT PACKARD ENTERPRISE DEV LP

AI summary

One embodiment is a method that uses MapReduce and Relation Valued Functions (RVFs) with parallel processing to search a database and obtain search results.