Mixed Text Extraction via OCR and Clipboard Mediation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cut-and-paste operations are inefficient for sharing mixed-type text data, such as rendered text and images with embedded text, between applications and the operating system, as they cannot select text presented as images, requiring manual re-entry and are laborious for simultaneous pasting across multiple applications.
Innovation Solution
Implementing a text extraction process that captures graphical data from applications, extracts text using zone type designators, and provides a text selection tool to designate and share text data among various applications, including those that use general-purpose graphics engines and integrate optical character recognition (OCR) for text detection within images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If text is rendered as an image in unregistered applications, then graphical output capability is improved, but text data sharing capability deteriorates
Solution Approach 1:
The patent introduces an intermediary text extraction process that acts as a mediator between the graphics engine and the operating system. This intermediary layer captures graphical data from the graphics engine, extracts text information from it, and makes the extracted text available to the operating system and other applications through the clipboard, thereby enabling text sharing without compromising graphical output capability
Solution Approach 2:
The patent replaces the manual mechanical process of selecting and copying text from images with an automated text extraction system. Instead of requiring users to manually select text from rendered images (which is impossible with traditional systems), the system automatically extracts text data from graphical output through OCR or other text detection methods, substituting automated processing for manual operations
2Ease of operation
If traditional cut-and-paste operations are used, then simplicity of operation is maintained, but productivity deteriorates
Solution Approach 1:
The patent merges the text extraction functionality with the existing clipboard operations. By integrating text extraction from graphical data with the standard copy-paste mechanism, the system allows users to select and paste text from images using the same simple interface and operations they already use for text, thereby maintaining ease of operation while dramatically improving productivity
Solution Approach 2:
The system enables self-service text extraction where the clipboard automatically extracts text from graphical data when image data is placed on the clipboard. The text extraction process operates automatically in the background, extracting text from the graphical representation without requiring additional user actions or manual intervention, thus maintaining simplicity while enhancing efficiency
3Measurement precision
If manual text entry is required for images with embedded text, then accuracy of text input is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary text extraction from graphical data automatically when the image is captured or placed on the clipboard, before the user needs to use the text. By extracting text in advance and making it available in the clipboard alongside or instead of the image data, the system eliminates the need for manual text entry while maintaining accuracy, thus resolving the time-accuracy tradeoff
Data Source
AI summary
Systems, methods, and devices for extracting and distributing text of mixed types from displayed graphical data from a display of an electronic device are disclosed. The text types can include rendered text and text represented in rendered images. Displayed graphical data can be captured from data being displayed by an application on a display device. Text data can be extracted from the captured graphical data as text data at the rendering tree level, or by an optical character recognition process. In response to the extracted text data, a text selection tool with visual representations of selectable text can be applied to the displayed text data. Using the text selection tool, a user can select a subset of the text. In response to the user selection, one or more other applications can be determined and the selected text can be passed to at least one of the other applications for execution.