Code Generation from Directed Acyclic Graph Using Knowledge Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data integration systems face challenges in creating platform and technology-independent logical designs that can efficiently manage data flow between heterogeneous sources and targets, often requiring detailed attribute specification and relying on proprietary infrastructure for data transformation.
Innovation Solution
A data integration system that allows users to create platform and technology-independent logical designs, which are then analyzed to generate physical designs with executable code, using knowledge modules to adapt to various technologies and languages, thereby simplifying design and maintenance by avoiding the need for detailed attribute specification and leveraging existing infrastructure for data transformation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If proprietary infrastructure is used for data transformation, then data integration reliability is improved, but device complexity and cost increase
Solution Approach 1:
The patent introduces a code generation layer that acts as an intermediary between the logical design and the physical execution environment. This layer generates platform-specific code from abstract logical designs, eliminating the need for proprietary transform servers while maintaining reliability through standardized code generation processes.
Solution Approach 2:
The patent extracts the data transformation logic from proprietary infrastructure and relocates it to standard computing platforms. By separating the transformation logic from the execution platform, the system eliminates dependency on specialized hardware while maintaining transformation capabilities.
2Manufacturing precision
If detailed attribute specification is required for logical design, then manufacturing precision is improved, but ease of manufacture deteriorates
Solution Approach 1:
The patent segments the design process into two distinct phases: logical design (platform-independent, high-level data flow specification) and physical design (platform-specific code generation). This segmentation allows users to create simple logical designs without detailed attribute specification, while the system automatically handles the complex code generation phase.
Solution Approach 2:
The system performs preliminary code generation and platform-specific adaptation automatically during the design phase. By pre-generating the complex implementation details from simple logical designs, the system eliminates the need for users to manually specify detailed attributes, thereby improving ease of manufacture while maintaining design precision.
3Adaptability or versatility
If platform-specific code generation is implemented, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent creates a universal code generation framework that can adapt to multiple platforms through a standardized process. The system uses a single logical design representation that can be transformed into code for different platforms (SQL Server, Oracle, Hadoop, etc.) without requiring separate design processes, thereby achieving multi-functionality while managing complexity through standardization.
Data Source
AI summary
In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user's infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.