Screen Text Mining via OCR for Cross-Platform Security

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current text indexing software, such as Microsoft Active Accessibility (MSAA), has limitations including non-compliance from applications like Chrome, inability to detect text in images like PDFs, and platform specificity, which hampers effective monitoring and protection of sensitive information within enterprises.

Innovation Solution

A system and method for reactive text mining on computer screens that captures and indexes displayed text in real-time, regardless of the API or operating system, allowing for proactive enforcement of rules and policies to prevent unauthorized access to sensitive information, including text within images and graphics.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If MSAA API is used for text indexing, then text can be indexed from Windows applications, but it cannot detect text in images, does not work with Chrome or Java applications, and is platform-specific

Engineering Contradiction:

Improveplatform compatibilityVSAvoidtext detection capability

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent uses an intermediary approach by capturing screen images and applying OCR technology as a mediator between the display and the indexing system. This allows text extraction from any application or platform without relying on platform-specific APIs like MSAA, thereby resolving the contradiction between platform compatibility and text detection capability

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical API-based text extraction system with an optical character recognition system that works by capturing and analyzing visual display output. This substitution enables universal text detection across all platforms and applications, including those that do not support traditional accessibility APIs

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If real-time screen text mining is implemented, then unauthorized access can be detected and policies enforced, but system complexity and processing requirements increase

Engineering Contradiction:

Improvepolicy enforcementVSAvoidsystem architecture

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent implements preliminary action by pre-defining security policies, rules, and thresholds before deployment. The system is configured with predetermined criteria for what constitutes unauthorized access or policy violations, allowing automated real-time enforcement without complex decision-making logic during operation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent incorporates feedback mechanisms where the system continuously monitors screen content, compares it against predefined policies, and automatically responds to violations. This closed-loop feedback system simplifies the architecture by using rule-based automation rather than complex human-in-the-loop decision processes

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10853508B2Method and system for reactive mining of computer screens

Publication Date: 2020.12.01 TERAMIND INC

AI summary

Systems and methods provide for text mining, and in particular text mining of screens, of endpoint and other target computers and devices, of an enterprise or network. The text mining from the screen is based on the text that the end point user actually sees, as is actually displayed on the screen. As a result of the text mining, action can be taken against the endpoint computer.