Cross-Application Knowledge Base Using Screen Context and System Events

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Electronic terminals face challenges in providing a unified user experience across multiple applications due to heterogeneous user interfaces and increasing complexity, requiring a solution that allows transverse control independent of individual applications.

Innovation Solution

A method and device construct a knowledge base using system events, screen captures, and image analysis to establish a lookup table between application functions and system actions, enabling transverse control through an OTT application without relying on application-specific APIs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If each application offers its own user experience with unique interfaces, then application functionality is enhanced, but user interface homogeneity deteriorates

Engineering Contradiction:

Improveapplication functionalityVSAvoiduser interface homogeneity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent introduces a mediator application that acts as an intermediary between the user and multiple applications. This mediator captures system events, obtains screen captures, performs image analysis to identify application states, and translates user actions into application-specific commands. This allows unified control interface while maintaining access to diverse application functionalities.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The mediator application provides universal control capabilities across multiple applications through a single interface. By implementing general-purpose image analysis and event capture mechanisms, the system can control different applications without requiring application-specific interface implementations, thus achieving both versatility and ease of operation.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If applications change regularly to offer new functionalities via APIs, then application capabilities are enhanced, but user interface complexity increases

Engineering Contradiction:

Improveapplication capabilitiesVSAvoiduser interface complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The mediator application performs self-service by automatically capturing system events, obtaining screen captures, and analyzing application states through image processing. Instead of requiring manual configuration or complex API integrations for each application, the system autonomously adapts to new applications and functionalities through automated observation and learning.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system changes its approach from static API-based integration to dynamic image analysis-based control. By analyzing visual parameters of application interfaces through image processing, the mediator can adapt to changing application capabilities without requiring updates to control logic, thus managing complexity while maintaining versatility.

Inventive Principle:

Principle #35Parameter changes

3Ease of operation

If a unified control solution is implemented, then ease of operation improves, but implementation complexity increases

Engineering Contradiction:

Improveunified control capabilityVSAvoidimplementation complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent replaces complex mechanical or programmatic integration systems with optical/image-based control. Instead of implementing numerous application-specific API integrations (mechanical/system-level complexity), the system uses screen capture and image analysis (optical substitution) to achieve unified control, significantly reducing implementation complexity while maintaining ease of operation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Measurement precision

If application-specific APIs are used for control, then control precision improves, but adaptability to new applications deteriorates

Engineering Contradiction:

Improvecontrol precisionVSAvoidadaptability to new applications

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

Instead of using application-specific APIs to achieve control precision (traditional approach), the patent inverts the approach by using general-purpose image analysis to achieve both precision and adaptability. The system analyzes the visual state of any application interface and determines appropriate control actions based on image content, achieving precision without sacrificing adaptability to new applications.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS20250390766A1Method and device for constructing a knowledge base for the purpose of making cross-functional use of the application functions of a plurality of software items

Publication Date: 2025.12.25 ORANGE SA

AI summary

A method for constructing a knowledge base, implemented by a construction device during use of an electronic terminal. The method includes: when a system event on the electronic terminal is detected, a first step of obtaining at least one position datum in relation to a cursor, the cursor being associated with at least one pointing peripheral, and at least one digital image of a snapshot of at least part of the output of at least one screen of the terminal; and a second step of obtaining at least one system datum from the electronic terminal; a third step of obtaining at least one context datum based on analysis of all or part of the digital image; and a step of updating the knowledge base with the at least one position datum, system datum and context datum.