In-App Screen Translation for Dynamic Text and Image Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Internet browser-based translation functions fail to translate text within images and do not support real-time text detection or translation, limiting their applicability to web-based services and restricting application translation when access to resources is not possible.
Innovation Solution
An electronic device and method that executes a first application, translates text from a first language to a second language using a second application, and displays the translation on a display, with real-time translation of dynamically changing content, allowing interaction between applications for enhanced user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Internet browser-based translation function is used, then text resource replacement in HTML is achieved, but text included in images cannot be translated and dynamic content cannot be translated in real-time
Solution Approach 1:
The translation system is segmented into multiple components: a translation application for static text resources, and an optical character recognition (OCR) module for dynamic text in images and real-time content. This segmentation allows each component to specialize in specific translation scenarios, achieving both web-based service translation and image/dynamic content translation simultaneously.
Solution Approach 2:
The patent merges the traditional browser-based translation function with OCR technology and real-time content detection capabilities. By combining these previously separate functions into an integrated system, the solution achieves comprehensive translation coverage including static text, image text, and dynamic content without requiring separate tools.
2Adaptability or versatility
If Internet browser-based translation function is used, then web-based service translation is enabled, but application translation is limited due to lack of resource access
Solution Approach 1:
The translation application is designed with universal functionality that works across multiple platforms and contexts. It can access and translate resources from both web-based services and packaged applications by implementing multiple resource access methods, including direct file access, package extraction, and network requests, making it universally applicable regardless of the source application type.
3Productivity
If real-time text detection and translation is implemented, then dynamic content translation is achieved, but system complexity increases
Solution Approach 1:
The system implements periodic action by detecting content changes at specific intervals or triggers rather than continuously monitoring all content. The translation application monitors for change events (such as user scrolling, content updates, or explicit translation requests) and only activates OCR and translation processes when changes are detected, reducing unnecessary system complexity while maintaining real-time translation capability.
Data Source
AI summary
An electronic device is provided that includes a display and a processor that executes a first application based on a first language, displays a first execution screen corresponding to the first application, wherein a content, which changes over time, is displayed in a first area of the first execution screen, executes a second application in response to receiving a first user input, translates a text included in the first execution screen from the first language to a second language using the second application and displays the translation, in a state in which the second application is executed and in response to the content in the first area being changed, extracts the text included in the changed content, translates the extracted text from the first language to the second language using the second application, and displays a second execution screen corresponding to the first execution screen based on the second language.