Machine Learning Model Training Structured Data Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models trained on unstructured data, such as BERT, are less effective when applied to structured data like CRM data, as they fail to capture the inherent structure and relationships within structured datasets, leading to suboptimal performance in AI services.
Innovation Solution
The proposed solution leverages the structured nature of CRM data by using word embedding techniques to capture topic and structural characteristics, generating attribute and entity embeddings, and employing a conditional language model architecture to pre-train machine learning models, allowing them to understand the inherent organization and interrelationships in CRM data, which can then be fine-tuned for specific tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If current machine learning models (e.g., BERT) trained on unstructured data are applied to structured data, then the models can be used with minimal modification, but they fail to capture the inherent structure and relationships within structured datasets, leading to suboptimal performance
Solution Approach 1:
The patent changes the fundamental parameters of model training by switching from unstructured text data to structured data formats (JSON, XML, CSV). It modifies training parameters including using attribute embeddings, entity embeddings, and relationship embeddings specific to structured data, thereby improving the model's ability to capture structural relationships while maintaining versatility across different structured data types
Solution Approach 2:
The patent segments structured data into distinct components: attributes, entities, and relationships. Each component is processed separately through dedicated embedding layers (attribute embedding layer, entity embedding layer, relationship embedding layer), allowing the model to capture specific structural characteristics of each component while maintaining overall model coherence
2Loss of information
If word embedding techniques are used to capture topic and structural characteristics in structured data, then the model can understand inherent organization and interrelationships, but the training process becomes more complex
Solution Approach 1:
The training process is segmented into distinct stages: data preprocessing to extract attributes/entities/relationships, separate embedding generation for each component type, and hierarchical assembly of embeddings. This segmentation manages complexity by breaking down the intricate task of structured data understanding into manageable, modular steps
Solution Approach 2:
The patent introduces intermediary embedding layers that act as mediators between raw structured data and the final model predictions. Attribute embeddings, entity embeddings, and relationship embeddings serve as intermediate representations that capture structural characteristics while simplifying the subsequent processing steps
Data Source
AI summary
A computing system may receive a corpus of training data including a plurality of data entity schemas. A first data entity of a first set of data entities corresponding to a first data entity schema is associated with a topic characteristic based on a first set of attributes defined by the first data entity schema, and a first attribute of the first set of attributes is associated with a structural characteristic that is common across each of the first set of data entities. The system may identify a respective attribute type identifier for each attribute of the first set, generate an attribute embedding for each attribute using the attribute value and the identifier, generate an entity embedding based on each attribute embedding and parameterize the topic characteristic for each data entity and the structural characteristic for each attribute.