Metadata-Driven Data Integration Server for Cloud Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The migration of data from localized databases to centralized cloud-based storage systems often results in data being formatted differently, making it inaccessible across various teams within a company. Existing solutions require lengthy and computation-intensive processes for pre-emptive data transformation.
Innovation Solution
A data integration server method that automatically integrates data from a source database to a target database by determining the source and target file formats, generating an integrated script to transform and load the data, and executing this script to ensure compatibility and accessibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is migrated from localized databases to centralized cloud-based storage, then storage overhead and maintenance costs are decreased, but data accessibility and usability across teams are reduced due to format incompatibility
Solution Approach 1:
The patent introduces a data integration server as an intermediary component between the centralized cloud storage and various teams. This server automatically transforms data from different source formats into a standardized target format, enabling seamless accessibility across teams without requiring changes to the centralized storage architecture. The integration server acts as a mediator that resolves format incompatibility while maintaining the benefits of centralized storage.
Solution Approach 2:
The patent applies parameter changes by transforming the format parameter of stored data. The system automatically converts data from various source formats (different parameters) into a standardized target format, changing the format parameter to ensure compatibility and accessibility across different teams while maintaining the centralized storage structure.
2Adaptability or versatility
If data is pre-emptively transformed into a different format before storage, then data compatibility and accessibility are improved, but processing time and computational resources are increased
Solution Approach 1:
The patent implements preliminary action by pre-configuring transformation rules and scripts in the data integration server. These transformation parameters and formats are prepared in advance, allowing the system to quickly execute transformations when data is stored without requiring complex real-time analysis. The preliminary setup of transformation logic reduces the actual processing time during data ingestion.
3Adaptability or versatility
If manual data analysis and software development are performed to ensure data compatibility, then data accessibility is improved, but development time and resource consumption are increased
Solution Approach 1:
The patent implements self-service by enabling the data integration server to automatically perform data format transformation without requiring manual intervention. The system uses pre-configured transformation rules to autonomously convert data between formats, eliminating the need for manual data analysis and custom software development. This automated self-service approach improves productivity while maintaining data accessibility.
4Productivity
If automated data integration is implemented, then productivity and data accessibility are improved, but system complexity is increased
Solution Approach 1:
The patent applies universality by designing the data integration server with multi-functional capabilities. The single server performs multiple functions including data reception, format detection, automatic transformation, and loading into centralized storage. This universal approach consolidates what could be multiple separate systems into one integrated platform, improving productivity while managing system complexity through consolidation rather than proliferation of components.
Data Source
AI summary
A method comprises receiving, from a user equipment via a server-to-user equipment application programming interface, a request to store a source file in a target database as a target file compatible with the target database, wherein the request comprises metadata describing the source file, a source database storing the source file, and the target database, determining by an integrator application of the data integration server, a source file format of the source file based on the metadata, determining, determine, by the integrator application, a target file format of the target file based on the metadata, wherein the target file format is compatible with the target database, and automatically generating, by the integrator application, an integrated script based on a plurality of pre-defined scripts, the metadata, the source file format, and the target file format.