Edge Language Model Reasoning for Personalized Content Ratings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks for generating personalized content ratings on edge devices face challenges due to computational intensity and power consumption, and the difficulty in providing accurate reasoning outputs for subjective user preferences without objective criteria for correctness.
Innovation Solution
A system that trains a smaller language model neural network using reasoning examples generated by a larger model, allowing it to generate both predicted ratings and natural language explanations, while optimizing for edge devices by leveraging machine learning co-processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large language model is used to generate personalized content ratings, then the accuracy of predicted ratings is improved, but the computational intensity and power consumption increase significantly
Solution Approach 1:
The patent segments the language model into two distinct components: a large language model (LLM) used only during the training phase to generate high-quality reasoning examples, and a smaller fine-tuned model deployed on edge devices for actual inference. This segmentation allows the system to benefit from the LLM's high accuracy while avoiding its computational burden during runtime, directly resolving the contradiction between rating accuracy and power consumption on edge devices.
2Reliability
If a large language model is deployed on an edge device, then the quality of reasoning outputs is improved, but the device's computational capacity and battery capacity are exceeded
Solution Approach 1:
The patent creates a copy of the reasoning capabilities by using the LLM to generate training examples that capture its reasoning quality. The smaller model is then fine-tuned on these examples, effectively copying the LLM's reasoning patterns into a lightweight model that can run on edge devices. This copying approach preserves reasoning quality while reducing computational complexity to fit device constraints.
3Adaptability or versatility
If reasoning outputs are generated to explain user ratings, then the personalization quality is improved, but the computational intensity increases
Solution Approach 1:
The patent applies preliminary action by generating comprehensive reasoning examples with the LLM during the offline training phase. These pre-generated reasoning outputs serve as training data that encode personalized reasoning patterns. During runtime, the small model leverages these pre-computed patterns to provide personalization without performing intensive reasoning generation, thus achieving adaptability while minimizing computational intensity during actual use.
Data Source
AI summary
Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for generating reasoning outputs and respective predicted ratings of content items using a language model neural network, training the language model neural network to further improve the quality of reasoning outputs, and generating high quality reasoning outputs for reasoning examples. By processing input sequences that include the interaction history of a particular user, the metadata of a current content item, and sometimes the rating of the current content item, the system can generate predicted ratings, generate candidate training reasoning outputs to train the language model neural network, and generate high quality example reasoning outputs.