Data Meta-Model for Standardized Feature Engineering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning and deep learning solutions face challenges in feature engineering, particularly when dealing with similar datasets from different domains, as existing methods lack standardization, automation, and efficiency in reusing knowledge and effort, leading to inflexible code, significant development and testing overhead, and a lack of automation in feature generation.
Innovation Solution
The implementation of a data meta-model-based feature vector set generation method that maps datasets to a meta-model, generates functions, and trains machine learning models using these features, allowing for standardized and automated feature engineering across similar datasets, reducing the need for repetitive code development and enhancing productivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If feature engineering is done on a per dataset basis using conventional methods, then the feature engineering can be customized for each specific dataset, but the development and testing overhead increases significantly and code cannot be reused across datasets
Solution Approach 1:
The patent creates a universal data meta-model that can represent multiple datasets across different domains through common structural patterns. By identifying that datasets from similar domains (e.g., e-retail) share common data characteristics and relationships, the system enables a single feature engineering framework to serve multiple datasets, eliminating the need for separate custom development for each dataset while maintaining domain-specific adaptability
Solution Approach 2:
The patent segments the feature engineering process into distinct modular components: data meta-model definition, mapping function creation, and feature generation. This segmentation allows each component to be independently developed, tested, and reused across different datasets, reducing overall development and testing overhead while maintaining flexibility
2Manufacturing precision
If separate feature engineering is performed for each dataset, then the feature quality can be optimized for specific use cases, but the code development and testing time increases significantly
Solution Approach 1:
The patent performs preliminary action by creating a standardized data meta-model and pre-defining mapping functions that capture common data patterns across datasets. This preliminary framework is established once and then applied to multiple datasets, eliminating the need for repetitive feature engineering development and testing while maintaining the ability to optimize features for specific use cases through the predefined mappings
3Adaptability or versatility
If conventional feature engineering methods are used, then flexibility in handling different datasets is maintained, but automation is lacking and manual effort is required
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate feature engineering code through the data meta-model and mapping functions. Once the meta-model is defined and mappings are established, the system autonomously generates the necessary feature engineering implementations without requiring manual coding for each dataset, while still maintaining flexibility through the configurable meta-model structure
Data Source
AI summary
This disclosure relates generally to data meta model and meta file generation for feature engineering and training of machine learning models thereof. Conventional methods do not facilitate appropriate relevant data identification for feature engineering and also do not implement standardization for use of solution across domains. Embodiments of the present disclosure provide systems and methods wherein datasets from various sources/domains are utilized for meta file generation that is based on mapping of the dataset with a data meta model based on the domains, the meta file comprises meta data and information pertaining to action(s) being performed. Further functions are generated using the meta file and the functions are assigned to corresponding data characterized in the meta file. Further functions are invoked to generate feature vector set and machine learning model(s) are trained using the features vector set. Implementation of the generated data meta-model enables re-using of feature engineering code.