Content Embedding Evaluation for Adaptive Media Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The efficient classification and processing of diverse visual media content, such as live action content with CGI elements, 3D animation, and 2D animation, is challenging due to the need for different treatment in pre- and post-production, and conventional manual classification methods often require trial and error to determine appropriate encoding workflows.
Innovation Solution
A machine learning model-based embedding system that utilizes contrastive learning and unsupervised clustering to automatically categorize and map content into a continuous vector space, enabling adaptive content evaluation for tasks like video encoding, colorization, and audio cleanup, without requiring human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional evaluation metrics (accuracy, precision, recall, F1-score) are used to evaluate NLP models, then the evaluation process is simple and fast, but the evaluation does not capture user perception and satisfaction
Solution Approach 1:
The patent introduces an intermediary embedding layer that translates traditional NLP model outputs into a format suitable for user perception evaluation. This intermediary layer bridges the gap between technical metrics and user perception by converting model predictions into embeddings that can be evaluated based on user satisfaction criteria.
Solution Approach 2:
The patent changes the evaluation parameters from traditional NLP metrics (accuracy, precision, recall, F1-score) to user-perception-based metrics. This involves transforming the evaluation space by mapping model outputs to user perception dimensions, allowing evaluation of whether changes improve actual user satisfaction rather than just technical performance.
2Adaptability or versatility
If deep learning models with multiple layers are used, then the model capacity increases, but the training time and computational resources required increase significantly
Solution Approach 1:
The patent segments the evaluation process into distinct components: model training, embedding generation, and user perception evaluation. This segmentation allows the heavy computational burden to be separated from the evaluation phase, where pre-computed embeddings can be efficiently processed to assess user perception without requiring re-training of the entire deep learning model.
3Productivity
If traditional evaluation metrics are used, then the evaluation process is fast and simple, but the evaluation cannot determine if changes actually improve user satisfaction
Solution Approach 1:
The patent replaces traditional mechanical evaluation metrics with a perception-based evaluation system. Instead of relying on statistical metrics that may not correlate with user satisfaction, the system uses embedding representations that directly model user perception, allowing evaluation of whether changes actually improve user satisfaction.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A system includes a computing platform having processing hardware, and a system memory storing software code and one or more machine learning (ML) model(s) trained using contrastive learning based on a similarity metric. The processing hardware is configured, to execute the software code to receive input data including a plurality of content segments, map, using the ML model(s), each of the plurality of content segments to a respective embedding in a. continuous vector space to provide a plurality of mapped embeddings, and perform one of a classification or a regression of the content segments using the plurality of mapped embeddings. The processing hardware is also configured to execute the software code to discover, based on the classification or the regression, at least one new label for characterizing the plurality of content segments.