Screen Scraping System with OCR Text Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices limit the ability to select and manipulate text within images and across the entire screen, failing to support simultaneous selection of text in SMS interfaces and text embedded within images.
Innovation Solution
An electronic device system that allows users to select any portion of the screen, recognizing and extracting text and images within the selected area using a combination of text extraction and image analysis methods, including OCR, to provide analysis results and related operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a scrape function is used to capture content, then both text and images can be selected, but text embedded within images cannot be extracted
Solution Approach 1:
The patent combines multiple extraction methods (text extraction for plain text areas and OCR for image areas) into a single unified scraping system. When a user selects a region containing both text and images, the system simultaneously applies both extraction methods to retrieve all content types including embedded text within images.
Solution Approach 2:
The patent introduces an image recognition/OCR component as an intermediary that processes image areas within the selected region. This intermediary component extracts text from images by converting image data into recognizable text, thereby enabling the system to retrieve embedded text that would otherwise be inaccessible.
2Device complexity
If text selection is limited to input fields, then selection functionality is simple to implement, but text in SMS interfaces and embedded text cannot be selected
Solution Approach 1:
The patent makes the selection function universal by enabling it to operate across multiple content types and interfaces. The same selection mechanism works for input fields, SMS message bodies, and images containing text, eliminating the need for separate selection implementations for different contexts.
Solution Approach 2:
The patent segments the screen into different content areas (text areas, image areas, SMS interfaces) and applies appropriate extraction methods to each segment. This segmentation allows the system to handle diverse content types while maintaining a unified selection interface for the user.
3Device complexity
If only edit operations are supported after text selection, then the system is simple to manage, but analysis and other operations are not available
Solution Approach 1:
The patent makes the operation menu dynamic by automatically adjusting available operations based on the selected content type. When text is selected, text-related operations (edit, copy, paste) are shown; when images or embedded text are selected, additional operations like image analysis and OCR are included in the menu.
Solution Approach 2:
The patent performs preliminary analysis of the selected content area to determine what operations are applicable before presenting the operation menu to the user. This preliminary action identifies content types (text, image, embedded text) and pre-configures the appropriate operation set, ensuring relevant operations are always available.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information providing method of an electronic device is provided. The information providing method includes determining a selected area based on a user input, determining an extraction method based on types of one or more objects included in the selected area, extracting information from the selected area according to the determined extraction method, and performing a specific function based on the information.