ML Attribute Mapping Standardizes Naming Conventions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed data systems, different database schemas with varying data representations and attribute names lead to difficulties in data transfer and communication, causing issues such as mismatched data and lack of understanding for system administrators.
Innovation Solution
A computer-implemented method that standardizes attribute names from a source data model to a destination data model using a machine learning network model, specifically an NLP-based algorithm, to predict mappings and create an attribute mapping table, with a custom dictionary for improving matching accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated machine learning methods are used for attribute name mapping, then productivity is improved, but manufacturing precision may worsen due to potential prediction errors
Solution Approach 1:
The system implements feedback through confidence score evaluation and manual review mechanisms. When the machine learning model's predicted mapping confidence falls below a threshold, the system flags these cases for manual review, allowing the automated process to handle high-confidence cases efficiently while maintaining accuracy through human verification of uncertain predictions.
Solution Approach 2:
The system performs preliminary standardization of attribute names before machine learning prediction, converting various naming conventions (camelCase, snake_case, PascalCase) to a unified format. This preliminary processing improves the accuracy of subsequent automated mapping by reducing variability in the input data.
2Manufacturing precision
If comprehensive attribute name standardization is performed, then manufacturing precision is improved, but device complexity increases
Solution Approach 1:
The system changes the parameter of attribute name representation by standardizing various naming conventions (camelCase, snake_case, PascalCase, hyphenated) into a unified format. This parameter transformation improves matching accuracy without requiring complex structural changes to the overall system architecture.
3Manufacturing precision
If manual review processes are implemented for unmapped attributes, then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The system applies partial manual review by only requiring human intervention for cases where the machine learning model's confidence score falls below a predetermined threshold. High-confidence predictions are accepted automatically, while only uncertain cases require manual review, thus minimizing time loss while maintaining precision for critical mappings.
Data Source
AI summary
A computer-implemented method of mapping attribute names of a source data model to a destination data model includes processing one or more source attribute names from the source data model and one or more destination attribute names of the destination data model to obtain standardized attribute names. The method includes determining whether any of the source attribute names have an existing entry in a custom dictionary. The method includes, for each source attribute name that has an existing entry in the custom dictionary, recording an output mapping for the source attribute name according to the existing entry in the custom dictionary. The method includes, for each source attribute name that does not have an existing entry in the custom dictionary, using a machine learning model to predict the mapping of each source attribute name to a corresponding one of the destination attribute names.