Unlock instant, AI-driven research and patent intelligence for your innovation.

Extraction, recognition and machine learning of user interface based on multi-anchor

A user interface and anchor point technology, applied in machine learning, user interface execution, character recognition, etc., can solve problems such as RPA errors

Active Publication Date: 2022-04-12
UIPATH INC
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

These errors may be more apparent in virtual or remote computer configurations
Also, button shapes or checkboxes in the UI can cause errors with RPA using CV

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Extraction, recognition and machine learning of user interface based on multi-anchor
  • Extraction, recognition and machine learning of user interface based on multi-anchor
  • Extraction, recognition and machine learning of user interface based on multi-anchor

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0014] For the methods and processes described below, the steps may be performed in any order out of sequence, and sub-steps not explicitly described or shown may be performed. Also, "coupled" or "operably coupled" may mean that objects are linked, but there may be zero or more intervening objects between the linked objects. Furthermore, any combination of the disclosed features / elements may be used in one or more embodiments. When reference to "A or B" is used, A, B, or A and B may be included, which may similarly be extended to longer lists. When the notation X / Y is used, it can include either X or Y. Optionally, when the notation X / Y is used, it can include both X and Y. X / Y notation can be similarly extended to longer lists with the same interpretation logic.

[0015] Figure 1A is an illustration of Robotic Process Automation (RPA) development, design, operation or execution 100 . Designer 102, sometimes referred to as a studio, development platform, development envir...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The extraction, recognition and machine learning of a multi-anchor-based user interface are disclosed. Multiple anchors are available for robotic process automation (RPA) of the user interface (UI). Multiple anchor points can be used to determine the relationship between elements in the captured image for the RPA's UI. Anchored results can be used for training or retraining of machine learning (ML) components.

Description

technical field [0001] Embodiments of the present disclosure relate generally to robotic process automation. Background technique [0002] Robotic Process Automation (RPA) can automate repetitive operations, functions or workflows in enterprise platforms, virtual machine (VM) configuration, remote desktop, cloud computing, desktop applications and more. For automated user interface (UI) capture, extraction, scraping, or execution, anchor or reference points can be used with computer vision (CV) or machine vision (MV) to identify one of the target areas of the image of the UI or multiple elements. However, configurations utilizing a single anchor or reference point may be susceptible to undesired duplication, errors, false positives, missing elements, and the like. [0003] Errors based on dots per inch (DPI), sizing, scaling, video frame rates, shadows, etc. may also be encountered in the RPA design or execution of workflows. These errors may be more apparent in virtual o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/451G06K9/62G06F8/34
CPCG06F9/451G06F9/452G06F8/34G06F18/214G06F3/0481G06N3/08G06N5/048G06F8/38G06T7/60G06F9/45512G06V10/46G06V30/153G06N20/00G06V30/10
Inventor C·沃伊库
Owner UIPATH INC