Automated Data Modeling Using Fuzzy Reasoning Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of merging data from multiple applications into an enterprise data warehouse is time-consuming and resource-intensive due to manual efforts required for mapping operations across different formatting, field names, data types, and naming conventions, leading to inefficiencies and potential inconsistencies.
Innovation Solution
An automated data modeling system utilizing machine learning and fuzzy logic to scan databases, identify changes, perform data type conversions, and generate data models that comply with existing rules and constraints, while also suggesting new abbreviations and providing feedback to users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual mapping operations are performed to merge data from multiple applications, then data consistency and adherence to enterprise modeling standards can be maintained, but processing time and resource utilization increase significantly
Solution Approach 1:
The system enables automated self-service data modeling where the computer system autonomously scans databases, identifies changes, performs data type conversions, and generates data models without requiring manual intervention for each mapping operation, thereby reducing processing time while maintaining consistency through automated rule enforcement
Solution Approach 2:
Manual mechanical mapping operations are replaced with automated computational processes including machine learning models and fuzzy reasoning logic that automatically determine mapping relationships, data type conversions, and field name standardizations, eliminating the time-consuming manual effort while preserving data consistency through systematic rule application
2Reliability
If manual mapping operations are performed to account for differences in formatting, field names, and data types, then data quality can be maintained, but resource utilization increases
Solution Approach 1:
The system automatically performs data quality maintenance through self-service mechanisms including automated scanning of databases for changes, intelligent identification of data type conversion requirements, and autonomous generation of mapped data models, eliminating the need for manual resource-intensive operations while maintaining high data quality standards
Solution Approach 2:
Manual resource-intensive mapping operations are replaced with automated computational systems that use machine learning algorithms and fuzzy logic to efficiently determine mapping relationships, perform data type conversions, and standardize field names, significantly reducing resource utilization while maintaining data quality through systematic automated processes
3Productivity
If automated data modeling is implemented to reduce processing time, then productivity increases, but complexity of the system increases
Solution Approach 1:
The system introduces trained machine learning models and fuzzy reasoning logic as intermediaries that bridge the gap between raw database data and standardized data models, automating the complex mapping operations while presenting a simplified interface to users, thereby increasing productivity without requiring users to directly manage the underlying system complexity
Solution Approach 2:
The system performs preliminary actions by pre-training machine learning models on enterprise modeling standards and constraints before actual data modeling operations, so that when automated data modeling is executed, the complex rule sets and mapping logic are already prepared and optimized, enabling fast processing without requiring complex real-time decision-making
4Quantity of substance
If multiple data sources are merged into an enterprise data warehouse, then data comprehensiveness increases, but coordination effort and development time increase
Solution Approach 1:
The system enables automated self-service merging of multiple data sources by autonomously scanning each database for changes, intelligently identifying mapping relationships between different data sources, and automatically generating unified data models that comply with enterprise standards, thereby increasing data comprehensiveness without requiring manual coordination effort or extending development time
Data Source
AI summary
A system includes a network interface configured to access an enterprise data warehouse and an enterprise abbreviation list that maps enterprise terms to abbreviations. A processing device can communicate with the network interface and a memory device that stores instructions to analyze the enterprise data warehouse to determine name attribute scores based on occurrences of the enterprise terms and the abbreviations in the enterprise data warehouse. Execution of the instructions can also generate a scoring summary of phrases, apply fuzzy reasoning logic to identify one or more relationship patterns and weights for the phrases including at least one shared word to produce training data for a data model associated with the enterprise data warehouse, and update the data model with a new abbreviated field name associated with a new field name based on identifying a closest match of the new field name with the training data.