Automated Login via Virtual Keypad Image Decoding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional automated login systems for accessing password-protected Web sites are hindered by bot-limiting technologies such as virtual keypad interactive images, which prevent software agents from successfully logging in due to the complexity of deciphering distorted and randomly located characters.

Innovation Solution

A system that employs a software agent with an image processor, optical character recognizer, and image data encoder and decoder to navigate and decode virtual keypad images, allowing for automated login by capturing keystrokes and identifying character locations within the keypad image, thereby enabling efficient access to multiple client-subscribed Web sites without compromising security.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If virtual keypad interactive images with distorted and randomly located characters are used for authentication, then security against automated login is improved, but automated login capability deteriorates

Engineering Contradiction:
ImprovesecurityVSAvoidautomated login capability
Core Design Contradiction:
ReliabilityVSExtent of automation

Solution Approach 1:

The patent introduces an intermediary system consisting of image processing software and optical character recognition (OCR) technology that mediates between the virtual keypad interface and the automated login process. This intermediary captures the keypad image, processes it to locate and identify characters, and translates it into actionable keystroke information, thereby enabling automated login while preserving the security benefits of the virtual keypad interface

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the manual mechanical interaction required by virtual keypads (visual inspection and manual clicking) with an automated optical-mechanical system. The system uses image capture devices to photograph the keypad, optical character recognition to identify characters, and automated keystroke injection to input credentials, thereby substituting the manual mechanical process with an automated optical-based system that can decipher distorted and randomly located characters

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Object-affected harmful factors

If CAPTCHA and virtual keypad interfaces are implemented to prevent bot access, then protection against automated access is improved, but data aggregation efficiency deteriorates

Engineering Contradiction:
Improveprotection against bot accessVSAvoiddata aggregation efficiency
Core Design Contradiction:
Object-affected harmful factorsVSProductivity

Solution Approach 1:

The patent introduces an intermediary system consisting of image processing software and optical character recognition (OCR) technology that mediates between the virtual keypad interface and the automated login process. This intermediary captures the keypad image, processes it to locate and identify characters, and translates it into actionable keystroke information, thereby enabling automated login while preserving the security benefits of the virtual keypad interface

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the manual mechanical interaction required by virtual keypads (visual inspection and manual clicking) with an automated optical-mechanical system. The system uses image capture devices to photograph the keypad, optical character recognition to identify characters, and automated keystroke injection to input credentials, thereby substituting the manual mechanical process with an automated optical-based system that can decipher distorted and randomly located characters

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If clients must manually decipher and enter credentials on virtual keypads, then security validation is improved, but ease of operation deteriorates

Engineering Contradiction:
Improvesecurity validationVSAvoidease of login
Core Design Contradiction:
ReliabilityVSEase of operation

Solution Approach 1:

The patent implements self-service by enabling the system to automatically perform the credential entry process without human intervention. The image processing and OCR system autonomously captures the virtual keypad image, identifies the required characters, determines their locations, and inputs the credentials through automated keystroke injection, thereby eliminating the need for manual deciphering and entry while maintaining security validation

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces the manual mechanical interaction required by virtual keypads (visual inspection and manual clicking) with an automated optical-mechanical system. The system uses image capture devices to photograph the keypad, optical character recognition to identify characters, and automated keystroke injection to input credentials, thereby substituting the manual mechanical process with an automated optical-based system that can decipher distorted and randomly located characters

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables fully automated and efficient login to Web sites using bot-limiting technology, allowing clients to retrieve information from a single interface by accurately processing and entering credentials on virtual keypad images, thus overcoming the limitations of CAPTCHA and virtual keypad interfaces.

Implementation Method 1

an optical character recognizer, and an image data encoder and decoder

Methodology Applied
Scientific EffectOptical character recognition: Image Processing

Data Source

PatentEP2401839B1System and methods for automatically accessing a web site on behalf of client
Publication Date: 2019.09.04 YODLEE INC
  • EP2401839B1 patent drawingFigure 1
  • EP2401839B1 patent drawingFigure 2~3
  • EP2401839B1 patent drawingFigure 4

AI summary

A system for performing an automated network-based login procedure on an interactive keypad image includes a software agent executable from a digital medium connected to the network for navigating to a login page, accessing the keypad image, and performing an automated login, and an automated login support application executable from the same or a different digital medium connected to the network, the support application including at least an image processor, an optical character recognizer, and an image data encoder and decoder. The software agent performs a login at the virtual keypad image based on character image matching and location information acquisition for each character of a client' s specific set of credential characters included in the image of the keypad.