Native On-Device Summarization to Reduce Latency and Processor Use
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for summarizing content in virtual spaces on communication platforms are inefficient, leading to increased latency and computational resource exhaustion, resulting in a disrupted user experience, especially for users accessing multiple virtual spaces with numerous posts and reactions.
Innovation Solution
Utilizing a machine-learning model native to the user device's operating system to generate summaries, determining whether to use on-device systems or backend servers based on device type, connectivity, battery level, and other factors, and prioritizing data to summarize, thereby reducing network traffic and processor cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional summarization techniques are used, then summaries can be generated, but latency increases and computational resources are exhausted
Solution Approach 1:
The summarization system segments the content into individual posts and processes them separately through the machine learning model, allowing parallel processing and reducing overall latency. The system processes posts in batches rather than sequentially, maintaining reliability while reducing time loss.
Solution Approach 2:
The system performs preliminary actions by pre-processing content data, filtering relevant information, and preparing it for the machine learning model. This includes identifying important posts based on engagement metrics and preparing summarized representations that reduce processing time while maintaining summary quality.
2Reliability
If conventional summarization techniques are used, then summaries can be generated, but computational resources are exhausted
Solution Approach 1:
The system applies partial action by processing only the most relevant and important posts rather than all content equally. It uses engagement metrics and post characteristics to identify which posts warrant summarization, reducing computational resource consumption while maintaining reliable summary generation for meaningful content.
Solution Approach 2:
The system changes parameters by adjusting the machine learning model's processing intensity based on post characteristics, engagement levels, and computational resource availability. This allows the system to maintain reliable summarization for high-value content while reducing computational effort for less important posts.
3Loss of information
If users access each virtual space to understand updates, then complete information is obtained, but time consumption increases
Solution Approach 1:
The system extracts key information from each virtual space by processing posts through the machine learning model and generating condensed summaries. This extraction maintains the essential information needed to understand updates while significantly reducing the time required to access and process content compared to reading all posts in full.
Solution Approach 2:
The machine learning model acts as an intermediary between the raw post data and the user, transforming detailed content into summarized representations. This intermediary process preserves information completeness while eliminating the need for users to access and read every individual post in each virtual space.
Data Source
AI summary
Techniques for generating a summary using a machine-learning model native to the operating system running on a user device are discussed herein. The communication platform may receive an instruction to generate a summary to be displayed to a user profile. In such cases, the communication platform may determine whether to generate the summary using on-device systems or using systems in a server of the communication platform (e.g., a device separate from the user device). Based on determining to generate the summary using the on-device systems, the communication platform may identify data to summarize. The communication platform may input the data into a machine-learning model (or large language model (LLM)) residing within the operating system of the user device and receive, as output, a summary. In such cases, the communication platform may cause the summary to be displayed via the user interface of the user device associated with the user profile.