On-device Prompt Generation for LLM Webpage Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models are highly dependent on the specific wording of user inputs, leading to inconsistent responses for similar queries, requiring users to repeatedly modify their phrasing to obtain desired results.
Innovation Solution
An on-device machine learning model generates prompts for a large language model based on information from webpages, including actions like summarization or key point identification, and updates based on user interaction, ensuring improved response relevance and privacy through on-device processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If users directly input queries to large language models, then the models can process and respond to requests, but the responses are inconsistent for similar queries requiring repeated phrasing modifications
Solution Approach 1:
The system performs preliminary actions by generating multiple prompt variations before the actual query is submitted to the large language model. The on-device machine learning model creates several rephrased versions of the user's input, and the system selects the best prompt to ensure consistent and reliable responses without requiring users to manually rephrase their queries multiple times.
2Productivity
If all webpage information is processed through cloud-based models, then comprehensive analysis can be performed, but sensitive information privacy cannot be guaranteed
Solution Approach 1:
The system segments the processing architecture into on-device and cloud-based components. Sensitive webpage information is processed locally by on-device machine learning models that run directly on the user's device, ensuring privacy protection. Non-sensitive information or aggregated data can be sent to cloud-based large language models for comprehensive analysis, thus balancing productivity enhancement with privacy protection.
Solution Approach 2:
The on-device machine learning model acts as an intermediary between the user's device and cloud-based large language models. It preprocesses webpage information locally, filtering and preparing data before transmission to cloud services. This intermediary layer ensures that sensitive information remains on the device while still enabling comprehensive webpage analysis through cloud-based processing of non-sensitive data.
3Reliability
If complex prompt engineering is implemented to improve response quality, then large language model effectiveness increases, but system complexity increases
Solution Approach 1:
The system implements self-service through automated prompt generation using on-device machine learning models. Instead of requiring manual prompt engineering or complex configuration by users, the system automatically generates optimized prompts based on the user's input and the specific large language model being used. This self-service approach maintains high response quality while minimizing system complexity from the user's perspective.
Data Source
AI summary
A method includes obtaining, using at least one processing device of an electronic device, information associated with a webpage presented to a user. The method also includes providing, using the at least one processing device, the information to an on-device machine learning model of the electronic device. The method further includes generating, using the on-device machine learning model, a prompt for a large language model based on the information. The prompt includes an action from a set of candidate actions that the large language model is able to perform and at least some of the information. The method also includes providing, using the at least one processing device, the prompt as input to the large language model and receiving, using the at least one processing device, a response from the large language model. In addition, the method includes presenting, using the at least one processing device, the response to the user.