Data Flow Graph Model Optimizer Hints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional optimizers require complex rules and heavy weighted patterns to optimize data flow graphs, especially when dealing with domain-specific language coding, necessitating additional processing resources and time, and fail to effectively apply optimizations based on business semantics without explicit logic.
Innovation Solution
A model optimizer with a rules engine uses hints associated with nodes in the data flow graph to override disqualification facts and specify rules for optimization, allowing for business semantic-based optimizations without complex logic implementation, and enabling user-generated metadata input via a graphical interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional optimizers use heavy weighted rules and complex detection patterns to optimize data flow graphs with domain-specific language coding, then optimization coverage is improved, but processing time and resource consumption increase
Solution Approach 1:
The patent applies preliminary action by having modelers annotate data flow graph nodes with hints during the modeling phase, before optimization occurs. These hints pre-specify optimization opportunities and business semantics, so the optimizer can directly apply rules without performing complex pattern detection or parsing during query optimization, thereby reducing processing time while maintaining optimization coverage.
2Reliability
If conventional optimizers implement complex business logic and pattern detection algorithms to handle domain-specific language, then business semantic optimization is improved, but device complexity increases
Solution Approach 1:
The patent extracts complex business logic and domain-specific language parsing requirements from the optimizer itself and places them in the data flow graph model definitions. By taking out the complexity from the optimization engine and embedding it in the model layer through hints and annotations, the optimizer becomes simpler while still achieving business semantic optimization.
Solution Approach 2:
The patent introduces hints as an intermediary mechanism between domain-specific language coding and optimizer rules. These hints act as a mediator that translates business semantics into optimization directives, allowing the optimizer to work with simple rule patterns rather than requiring complex domain-specific language parsing logic.
3Adaptability or versatility
If conventional optimizers require parsers for each domain-specific language to understand semantic, then language versatility is improved, but processing overhead increases
Solution Approach 1:
The patent applies preliminary action by having all language-specific semantic information and parsing requirements prepared in advance during model definition. Hints annotate nodes with optimization-relevant semantics before queries are processed, eliminating the need for runtime parsing of domain-specific language and reducing processing overhead while maintaining support for multiple languages.
Data Source
AI summary
A query is received, and in response, an initial data flow graph is generated that includes a plurality of nodes for executing the query with at least one of the nodes having at least one associated hint. The initial data flow graph is subsequently optimized using a model optimizer having a rules engine using a plurality rules to optimize the initial data flow graph. The at least one associated hint is used by the model optimizer to change how at least one of the plurality of rules is applied. Thereafter, execution of the query is initiated using the optimized data flow graph. Related apparatus, systems, techniques and articles are also described.