Remote Control UI Adaptation via Dynamic Image Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional techniques require maintenance of UI layout information on client-side applications whenever a new device is introduced, making it inefficient to support multiple devices through remote operation.

Innovation Solution

Implementing a remote operation system using VNC (Virtual Network Computing) and RFB (Remote Frame Buffer) protocols, allowing the client to dynamically adjust software keypads based on device information, eliminating the need for UI layout updates in the client application.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the client's-side application holds UI layout information on all devices, then multiple devices can be supported for remote operation, but maintenance becomes necessary whenever a new device is introduced

Engineering Contradiction:

Improvedevice compatibilityVSAvoidmaintenance burden

Core Design Contradiction:

Adaptability or versatilityVSEase of repair

Solution Approach 1:

The patent extracts the UI layout information from the client's-side application and relocates it to the image processing apparatus itself. The operation panel image generation unit in the image processing apparatus generates the UI layout information locally, eliminating the need for the client application to store and maintain device-specific UI layouts. This resolves the contradiction by maintaining device compatibility through local generation while eliminating the maintenance burden on the client side.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces an image generation unit as an intermediary component within the image processing apparatus that dynamically generates operation panel images based on device state and type. This intermediary eliminates the direct dependency between the client application and device-specific UI layouts, allowing the client to display remotely generated UI layouts without needing to maintain local copies for each device type.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If UI layout information is maintained in the client application, then remote operation can be performed, but the system requires updates whenever devices are updated

Engineering Contradiction:

Improveremote operation capabilityVSAvoidmaintenance time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The image processing apparatus performs self-service by generating its own operation panel UI layout information locally through the image generation unit. The apparatus autonomously creates and transmits the appropriate UI layout based on its device type and current state, eliminating the need for external client applications to maintain or update UI layout information. This ensures reliable remote operation capability while eliminating maintenance time requirements.

Inventive Principle:

Principle #25Self-service

3Ease of operation

If pre-stored UI layouts are used for each device, then device-specific operation can be achieved, but the client application becomes complex

Engineering Contradiction:

Improvedevice-specific controlVSAvoidclient application structure

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent extracts the UI layout generation functionality from the complex client application structure and relocates it to the image processing apparatus. The operation panel image generation unit resides in the device being controlled, not in the remote control application. This simplifies the client application while maintaining device-specific control capabilities through locally generated UI layouts.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent inverts the traditional architecture by making the image processing apparatus (the device being controlled) the active generator of UI layout information, rather than making the client application the generator. This inversion simplifies the client application structure while preserving ease of device-specific operation, as each device generates its own appropriate UI layout dynamically.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentEP2562637B1Information processing apparatus that carries out remote operation, control method therefor, and storage medium

Publication Date: 2018.10.24 CANON KK

Figure 1~2
Figure 3~4
Figure 5

AI summary

An information processing apparatus which is capable of, whenever a new image processing apparatus is put on sale, remotely controlling the image processing apparatus without performing maintenance on the information processing apparatus. A display unit displays a first screen corresponding to an operation screen displayed on at least one image processing apparatus and a second screen corresponding to the hardware key which the at least image processing apparatus has. A holding unit holds a plurality of image of hardware keys corresponding to the hardware key which the at least image processing apparatus has. A display control unit selects an image of a hardware key corresponding to a hardware key which an image processing apparatus performing communication via a communication unit has from among at least one image of a hardware key held in the holding unit, and displays the second screen using the selected image of the hardware key.