Distributed Computing Nested Dataset Grouping Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed computing frameworks lack the concept of nested datasets, leading to complex expression of grouping logic and difficulty in code reusability, particularly with SQL-Like interfaces that fail to describe common computing requirements such as grouping data and retrieving specific records.
Innovation Solution
A method and apparatus for executing distributed computing tasks by parsing expressions to obtain operator keywords like grouping, operation, traversal, and key-value pair acquisition operators, enabling operations on nested datasets and supporting grouped data processing, thereby facilitating code reusability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SQL-Like interface is used to express distributed computing requirements, then the interface provides structured query capabilities, but it cannot describe common computing requirements such as grouping data and retrieving specific records from each group
Solution Approach 1:
The patent introduces nested dataset concepts where datasets can contain other datasets, enabling hierarchical data structures. This allows the interface to represent complex grouping operations by nesting grouped data within parent datasets, thereby expanding the capability to describe computing requirements while maintaining a unified interface structure.
Solution Approach 2:
The patent adds a hierarchical dimension to the data structure by introducing nested datasets with multiple levels. This dimensional expansion enables the representation of grouping operations and nested queries that cannot be expressed in flat SQL-like structures, thus improving adaptability without complicating the interface syntax.
2Adaptability or versatility
If native interfaces are used to configure and splice multiple different types of operators into a directed acyclic graph, then complete distributed computing requirements can be expressed, but the interface becomes inconvenient and codes are difficult to be reused
Solution Approach 1:
The patent creates a unified dataset interface that can handle multiple types of operations (grouping, filtering, sorting, etc.) through a common set of methods. This multi-functional approach allows the same interface to express various distributed computing requirements without requiring different operator splicing mechanisms, thereby improving both adaptability and ease of operation simultaneously.
Solution Approach 2:
The patent enables code reusability by allowing datasets to be referenced and reused multiple times in different contexts. Through the nested dataset structure, the same dataset can be copied or referenced in multiple parent datasets, eliminating the need to rewrite the same processing logic multiple times and improving code reusability while maintaining full expressiveness.
3Device complexity
If the concept of nested datasets is not provided, then the interface structure remains simple, but grouping logic becomes complex and cannot be properly expressed
Solution Approach 1:
The patent introduces nested dataset concepts where datasets can contain other datasets, enabling hierarchical data structures. This allows the interface to represent complex grouping operations by nesting grouped data within parent datasets, thereby expanding the capability to describe computing requirements while maintaining a unified interface structure.
Data Source
AI summary
A method and apparatus for executing a distributed computing task are provided. The method can include: parsing an expression of the distributed computing task to obtain an operator keyword; and executing, by using an operator corresponding to the operator keyword, the distributed computing task based on an input parameter of the operator, the input parameter of the operator including at least one of: a distributed stored distributed dataset including at least one data element, a distributed stored distributed key-value pair set including at least one key-value pair, and a value of the key-value pair in the distributed key-value pair set being a distributed dataset or a distributed key-value pair set.