Screen Object Classification for Faster RPA Target Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional robotic process automation technologies face difficulties in efficiently detecting target objects on user screens due to varying detection criteria, leading to increased attempts and time required, low success probability, and unnecessary detection processes, which deteriorate efficiency and throughput.

Innovation Solution

A method and apparatus that utilize machine learning to generate an object classification model by encoding detection criteria information and extracting characteristics from bot executable files and execution logs, optimizing the identification of target objects on user screens.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional sequential detection rules are applied to detect target objects, then detection coverage is achieved, but detection time increases and efficiency deteriorates

Engineering Contradiction:

Improvedetection success probabilityVSAvoiddetection time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent changes the detection approach from sequential rule application to parallel machine learning-based classification. The object classification model processes multiple detection criteria simultaneously, outputting probabilities for different object types in parallel, thereby reducing detection time while maintaining or improving detection accuracy.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the mechanical sequential detection rule system with an artificial intelligence-based object classification model. This substitution enables the system to automatically learn and apply optimal detection strategies without manually configured sequential rules, improving both speed and accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If multiple detection rules are applied sequentially to ensure detection coverage, then detection thoroughness is improved, but the number of attempts increases and efficiency decreases

Engineering Contradiction:

Improvedetection thoroughnessVSAvoidautomation efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent performs preliminary classification by training an object classification model on historical detection data before actual detection tasks. The model learns optimal detection patterns in advance, enabling it to quickly and accurately identify target objects without requiring multiple sequential detection attempts during execution.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback from historical detection results and execution logs to continuously improve the object classification model. By analyzing past detection outcomes, the model learns from experience and refines its classification accuracy, reducing the need for multiple detection attempts.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If detection rules are customized for different object types and situations, then detection accuracy is improved, but system complexity increases

Engineering Contradiction:

Improvedetection accuracyVSAvoiddetection system complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent creates a universal object classification model that can handle multiple object types and detection scenarios through a single unified system. The model processes various detection criteria and adapts to different situations without requiring separate customized rule sets for each case, thereby maintaining detection accuracy while reducing system complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Ease of manufacture

If conventional detection methods are used without optimization, then implementation simplicity is maintained, but unnecessary detection processes occur and throughput decreases

Engineering Contradiction:

Improveimplementation simplicityVSAvoidtask throughput

Core Design Contradiction:

Ease of manufactureVSProductivity

Solution Approach 1:

The system implements self-service by automatically training and optimizing the object classification model using its own historical detection data and execution logs. This self-learning capability allows the system to improve its detection efficiency and eliminate unnecessary detection processes without requiring complex external optimization mechanisms.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11669775B2System and method for detecting a target object

Publication Date: 2023.06.06 SAMSUNG SDS CO LTD

AI summary

A target object detection method performed by a computing device includes detecting a first target object on a user screen and obtaining a bot executable file to perform a defined operation for the first target object, labelling object type classification criteria for the first target object by encoding detection criteria information of the first target object included in the bot executable file, extracting a characteristic of process execution information of the first target object included in the bot executable file, and generating an object classification model that outputs data on an object type corresponding to process execution information by performing machine learning using the object type classification criteria and the characteristic.