Data Conversion System Using Reusable Maps and Collision Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing data conversion processes are inefficient as they require customized solutions for each instance of format conversion, leading to increased costs due to the lack of reusable conversion maps, and often result in data collisions when merging information from different systems.
Innovation Solution
A conversion system that uses predefined conversion maps to automatically convert data from a source format to a destination format, including rules for field mapping and transformation, and detects collisions by assigning a new identifier to reconcile multiple entries of the same entity, allowing for seamless data merging across different systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If customized data conversion solutions are created for each instance, then data conversion accuracy is improved, but conversion cost and time increase
Solution Approach 1:
The system performs preliminary actions by pre-defining conversion maps for common data format pairs before actual conversion needs arise. These conversion maps are stored in a database and can be reused multiple times, eliminating the need to create customized conversion solutions for each instance while maintaining conversion accuracy.
Solution Approach 2:
The system creates copies of conversion maps that can be reused across multiple data conversion instances. Instead of creating unique conversion logic for each conversion task, the system copies and applies pre-defined conversion maps from a database, significantly reducing conversion time while maintaining consistency and accuracy.
2Manufacturing precision
If customized data conversion solutions are created for each instance, then data conversion accuracy is improved, but conversion cost increases
Solution Approach 1:
The system implements universality by creating conversion maps that serve multiple purposes and can be applied to numerous different data conversion tasks. A single conversion map can handle multiple instances of the same format conversion, making the system multi-functional and reducing the need for numerous customized solutions, thereby lowering conversion costs.
Solution Approach 2:
The system recovers value from previous conversion work by storing conversion maps in a database. Instead of discarding conversion logic after a single use, the system recovers and reuses these conversion maps for future conversions, reducing the need to recreate conversion solutions and lowering overall conversion costs.
3Productivity
If data merging is performed without collision detection, then processing speed is improved, but data quality deteriorates
Solution Approach 1:
The system performs preliminary collision detection and resolution before final data merging occurs. By identifying and resolving identifier collisions in advance, the system ensures data quality without requiring slow, manual intervention during the merging process, thus maintaining both processing speed and data quality.
4Manufacturing precision
If manual intervention is used for data conversion, then conversion accuracy is improved, but automation level decreases
Solution Approach 1:
The system implements self-service by automatically selecting appropriate conversion maps from the database and applying them to convert data without requiring manual intervention. The system autonomously handles the entire conversion process, from identifying the source and destination formats to selecting the appropriate conversion map and executing the conversion, thereby maintaining high automation levels while ensuring accuracy through pre-defined conversion rules.
Data Source
AI summary
Converting data from a first format to a second format may be used to merge two databases having different formatting. Conversion maps may be defined for pairs of formats and may be used to convert data between the pair of formats. Data may further be transformed from one information type or format into another information type or format. In one example, a first format may allow storage of a social security number while a second format might not. Thus, a social security number stored in the first format may be converted to an employee number for storage in the second format. Collisions between pieces of data may be detected and managed by generating an identifier for the pieces of data (e.g., data records) and merging the pieces of data into a single piece or portion of data (e.g., a single data record) corresponding to the assigned identifier.