Automated Screen Image Translation via Text Extraction and Reassembly
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The labor-intensive process of translating software documentation, particularly screen shots, into multiple languages is complicated by the need to re-run the software application in each target language, requiring technical skills and incurring costs, and existing methods often result in clipped or mismatched text and images.
Innovation Solution
A method for automated translation of screen images involves capturing screen elements with metadata, allowing recreation in target languages without re-running the application, using serialized formats and language bundles to ensure high fidelity translation and correct layout.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If screen shots are included in multi-language documentation, then translation completeness is improved, but text layout accuracy deteriorates due to text length differences causing clipping and overflow
Solution Approach 1:
The patent applies preliminary action by extracting and translating text from screen shots before final layout assembly. The system identifies text regions, extracts them, translates them to target languages, and then reassembles the screen shots with translated text in a controlled sequence, preventing layout issues that would occur with post-translation adjustments.
Solution Approach 2:
The patent uses an intermediary approach by introducing a text extraction and reassembly system that acts as a mediator between the original screen shots and the final translated documentation. This intermediary process allows text to be separated, translated, and repositioned according to target language requirements without directly modifying the original image structures.
2Reliability
If translation experts manually translate text in screen shots, then translation quality is improved, but labor cost and time consumption worsen
Solution Approach 1:
The patent applies self-service by enabling automated extraction, translation, and reassembly of text from screen shots without requiring manual intervention from translation experts. The system performs these tasks autonomously, significantly improving productivity while maintaining consistent quality through standardized processes.
Solution Approach 2:
The patent replaces the mechanical system of manual text extraction and reinsertion with an automated computer-based system. This substitution eliminates the need for translation experts to manually work with screen shot images, replacing human labor with automated image processing and text manipulation algorithms.
3Stability of the object's composition
If the application is re-run in each target language to recreate screen shots, then text-image consistency is improved, but technical skill requirements and execution time worsen
Solution Approach 1:
The patent applies copying by creating translated versions of screen shots from the original captured images rather than requiring re-execution of the application. The system extracts text from the original screen shots, translates it, and generates copy versions with translated text, maintaining visual consistency without the complexity of re-running the application in different language environments.
Data Source
AI summary
An aspect includes receiving a description of a display screen generated by a software program executed in a first mode to interact with a user in first language. The description is in a serialized format that includes screen capture parameters and an offset location in a first dictionary corresponding to a screen element in the display screen. The first dictionary includes screen elements utilized by the software program when executing in the first mode to generate the display screen. A target language is identified, and a second dictionary of screen elements utilized by a deserialization program in the target language is accessed. The screen element is located at the offset location in the second dictionary. The description of the display screen is combined with the second dictionary, and the display screen is recreated in the target language based on the combining.