Text detection, insertion symbol tracking, and active element detection

A text detection and character technology, applied in character recognition, office automation, character and pattern recognition, etc., can solve the problem of unable to know which element of the user, unable to know the active element of the screen, etc.

Active Publication Date: 2021-06-08
UIPATH INC
View PDF6 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] When a user interacts with a computing system's input device, it may not be possible to know which element on the screen the user is typ

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text detection, insertion symbol tracking, and active element detection
  • Text detection, insertion symbol tracking, and active element detection
  • Text detection, insertion symbol tracking, and active element detection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Some embodiments relate to detection of typed and / or pasted text, caret tracking, and active element detection for computing systems. Some embodiments identify a location on a screen associated with a computing system where a user has been typing or pasting text, potentially including hotkeys or other keys that do not cause visible characters to appear, and based on the occurrence of one or more characters The current resolution (eg in coordinates) of the position of the cursor, the position where the cursor is blinking, or both to provide the physical position on the screen. The typing or pasting activity and / or the physical location of the caret may allow determination of which field(s) the user is typing or focusing on and an application or other application for process discovery.

[0044] Some embodiments are implemented in a feedback loop process that continuously or periodically compares the current screenshot with a previous screenshot to identify changes. A loc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Detection of typed and/or pasted text, insertion symbol tracking, and active element detection for computing systems are disclosed. In some embodiments, locations on a screen associated with a computing system at which a user always enters or pastes text may be identified, the text potentially including hot keys or other keys that do not cause the appearance of visible characters, and the physical location on the screen where the typing or pasting has occurred may be provided based on the location where the one or more characters have occurred, the location where the cursor is flickering, or the current resolution of both. This may be accomplished by identifying locations on the screen where changes have occurred and performing text recognition and/or inserted symbol detection on these locations. The physical location of the typing or pasting activity allows for the determination of active or focus elements in an application displayed on the screen.

Description

[0001] Cross References to Related Applications [0002] This application claims the benefit of U.S. Nonprovisional Patent Application Serial No. 16 / 864,629, filed May 1, 2020. The subject matter of this earlier filed application is hereby incorporated by reference in its entirety. technical field [0003] The present invention relates generally to computing system operation detection, and more particularly to detection of typed and / or pasted text, caret tracking, and active element detection for computing systems. Background technique [0004] When a user interacts with an input device of a computing system, it may not be possible to know which element on the screen the user is typing, although key presses can be determined through key hooking. Therefore, it may not be possible to know what the active element on the screen is. Accordingly, an improved approach for monitoring user interactions with computing systems could be beneficial. Contents of the invention [000...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/32G06F11/34G06F40/279G06V30/10
CPCG06F9/543G06F3/0233G06F9/542G06F9/546G06F2209/548G06F3/0481G06V2201/02G06V30/10G06V30/19013G06F3/0236G06F3/04812G06Q10/10G06F3/017G06V20/41
Inventor V·斯卡达
Owner UIPATH INC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap