Generative AI Embeddings for Machine Learning Input Accuracy

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current machine learning models lack accurate prediction of content representation and user intent, particularly in online networks, as they rely on separately trained embedding models and struggle to handle diverse content types and domains effectively.

Innovation Solution

The use of generative artificial intelligence (GAI) models to generate embeddings for content understanding, which can be used as inputs to separate machine learning models, eliminating the need for separately trained embedding models and enabling robust handling of diverse content types and domains.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If separately trained embedding models are used for content representation, then the machine learning model can process content data, but the prediction accuracy of content representation and user intent remains insufficient

Engineering Contradiction:
Improveprediction accuracyVSAvoidmodel complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent merges the embedding model and the machine learning model into a single unified model. The embedding layers are integrated directly within the neural network architecture, allowing the model to learn both content representation and prediction tasks simultaneously. This integration eliminates the need for separate embedding models while improving prediction accuracy through joint training and shared representations.

Inventive Principle:
Principle #5Merging (Combining)

2Adaptability or versatility

If traditional machine learning models are used, then the system can process content, but it struggles to handle diverse content types and domains effectively

Engineering Contradiction:
Improvecontent type handling capabilityVSAvoidcontent representation accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent implements a universal embedding layer that can process multiple content types including text, images, and videos through a single unified architecture. The embedding layers are designed to accept diverse input formats and convert them into a common representation space, enabling the model to handle various content types and domains effectively while maintaining high representation accuracy.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Measurement precision

If more complex embedding models are trained separately, then content representation may improve, but the training time and computational resources increase

Engineering Contradiction:
Improveembedding accuracyVSAvoidtraining time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary action by integrating the embedding functionality directly into the main model architecture during the initial model creation phase. Rather than training separate embedding models beforehand, the embedding layers are built-in and trained simultaneously with the prediction tasks during the main training process. This approach achieves accurate embeddings without the additional time cost of separate pre-training stages.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20240403623A1Generative artificial intelligence for embeddings used as inputs to machine learning models
Publication Date: 2024.12.05 MICROSOFT TECHNOLOGY LICENSING LLC
  • US20240403623A1 patent drawing
  • US20240403623A1 patent drawing
  • US20240403623A1 patent drawing

AI summary

In an example embodiment, a generative artificial intelligence (GAI) model is used to generate embeddings, eliminating the need for a separately trained embedding model or layer. These embeddings may then be used as input to another machine learning model. In some example embodiments, these embeddings are generated on interaction data regarding one or more interactions between a user and digital content presented on one or more online platforms.