Robotic Process Automation performed in a second session Automation of a process performed in a first session via a robot

The use of IPC facilitators and RPA drivers in separate sessions allows users to interact with their computing system while RPA robots perform tasks, addressing the limitation of traditional RPA technologies by enabling simultaneous use of multiple applications.

JP7594163B2Active Publication Date: 2024-12-04UIPATH INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2020553484
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-07-10
Filing Date
2020-09-22
Publication Date
2024-12-04
Estimated Expiration
2040-09-22

AI Technical Summary

Technical Problem

Existing robotic process automation (RPA) technologies restrict users from interacting with their computing systems while an RPA robot is performing tasks, as the robot controls the user interface, preventing simultaneous use of other applications.

Method used

Implementing inter-process communication (IPC) facilitators and RPA drivers to enable RPA robots to operate in a separate session, allowing users to interact with their computing system and applications independently.

Benefits of technology

Enables users to monitor and interact with their computing system while RPA robots perform automation, enhancing productivity by allowing simultaneous use of multiple applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594163000001
    Figure 0007594163000001
  • Figure 0007594163000002
    Figure 0007594163000002
  • Figure 0007594163000003
    Figure 0007594163000003
Patent Text Reader

Abstract

Automation of a process performed in a first session via a robotic process automation (RPA) robot executing in a second session is disclosed. In some embodiments, a form is displayed in a user session, while one or more attended RPA robots that retrieve and / or manipulate data for the application in the first session execute in one or more other sessions. In this manner, operation of the RPA robot may not prevent a user from using other applications or instances when the RPA robot is executing, but data changes made or facilitated by the RPA robot may be visible to the user in a window in the first session.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of, and claims the benefit of, U.S. Non-provisional Patent Application No. 16 / 924,910, filed July 9, 2020, which claims the benefit of, U.S. Non-provisional Patent Application No. 16 / 925,544, filed July 10, 2020. The subject matter of these earlier filed applications is incorporated herein by reference in its entirety.

[0002] The present invention relates generally to robotic process automation (RPA), and more particularly to the automation of a process running in a first session via an RPA robot running in a second session. [Background technology]

[0003] Attended automation robots, for example, typically run on a computing system operated by a user in the same session as the user. The RPA robot may collaborate with the user, for example, to perform certain tasks at the user's command. However, in attended automation scenarios, the RPA robot may "take over" the user's computing system. The user may want to perform other activities while the robot interacts with the computing system, but the user may not be able to do so. That is, the robot controls the application through a user interface (UI) in the same way that a user does (e.g., by simulating mouse clicks and keyboard input).

[0004] Various technologies exist to create complete or partial copies of an operating system or an application running on an operating system. Emulators have been around for decades and may provide developers with the ability to test and debug applications. For example, emulators provide developers with the ability to test and debug mobile applications that use operating systems that do not directly support running development tools. Both Android® and iOS® provide emulators that can be run from a development machine to test and debug Android® or iOS® applications because developer tools cannot be run natively on their mobile operating systems.

[0005] Simulators allow developers to host a window on their local machine where developers can test and debug application behavior that is difficult or impossible to run on the development machine. For example, simulators allow developers to click a button to rotate the simulator, which notifies the application running in the simulator that the device has been rotated for the purpose of testing and debugging the application's behavior in response to such events. Another common example is multi-touch. Since many developer machines do not support touch, simulators allow developers to test and debug how their application responds to multiple touch points. Android® and iOS® emulators also provide simulation capabilities. Additionally, Microsoft® provides simulators for Universal Windows Platform (UWP) applications.

[0006] A Virtual Machine hosts a second operating system on a machine and can be opened and monitored through a window. It runs a completely different operating system and shares hardware with the host machine. The "guest" machine must have its own copy of applications installed, and the "guest" machine does not share common resources or files with the user machine.

[0007] Docker® containers are conceptually a hybrid form of virtual machine: all the applications that need to be run are packaged into an immutable package that runs directly on the host operating system. The package is not an exact copy of another operating system, but by default it does not share or have access to any applications or resources on the host machine. So, from a user experience perspective, Docker® containers feel similar to virtual machines, but technically, containers are not running on a completely separate operating system.

[0008] However, traditional emulators, simulators, virtual machines (VMs), and hybrid VMs (e.g., Docker® containers, etc.) that provide operating system (OS) level virtualization do not address the issues that arise with attended automation robots running on the same computing system as the user. Thus, the user essentially becomes a spectator on their own computing system, unable to monitor the robot's work and interact with other applications on the machine that require user interaction. Thus, an improved approach may be beneficial. Summary of the Invention

[0009] Certain embodiments of the present invention may provide solutions to problems and needs in the art that have not yet been fully identified, recognized, or solved by current RPA technology. For example, some embodiments of the present invention relate to the automation of a process performed in a first session via an RPA robot running in a second session.

[0010] In one embodiment, a computer program is stored on a non-transitory computer-readable medium. The computer program is configured to cause at least one processor to execute an inter-process communication (IPC) facilitator and a robotic process automation (RPA) driver in a first session and to execute an RPA robot in a second session. The computer program is further configured to cause the at least one processor to receive, via the IPC facilitator, one or more messages from the RPA robot via the IPC, and to control the RPA driver to interact with an application or application object executed in the first session by the IPC facilitator based on the one or more messages received from the RPA robot.

[0011] In another embodiment, a computer-implemented method includes sending one or more messages by an RPA robot running in a second session via the IPC to an IPC facilitator running in a first session and receiving one or more messages from the RPA robot via the IPC facilitator, the computer-implemented method further including controlling an RPA driver running in the first session to interact with an application or application object running in the first session by the IPC facilitator based on the one or more messages received from the RPA robot.

[0012] In yet another embodiment, a computer-implemented method includes receiving, by an IPC facilitator running in a first session, one or more messages from an RPA robot running in a second session via IPC, the computer-implemented method further includes controlling an RPA driver running in the first session to interact with an application or application object running in the first session based on the one or more messages received from the RPA robot by the IPC facilitator. [Brief description of the drawings]

[0013] So that the advantages of certain embodiments of the invention may be readily understood, a more particular description of the invention, briefly described above, will now be made with reference to specific embodiments thereof which are illustrated in the accompanying drawings, which are to be understood as illustrating only typical embodiments of the invention and should not be considered as limiting its scope. The invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0014] [Figure 1] FIG. 1 is an architectural diagram illustrating a robotic process automation (RPA) system according to one embodiment of the present invention.

[0015] [Diagram 2] FIG. 1 is an architectural diagram illustrating a deployed RPA system according to one embodiment of the present invention.

[0016] [Diagram 3] FIG. 2 is an architecture diagram illustrating the relationship between designers, activities, and drivers according to one embodiment of the present invention.

[0017] [Figure 4] FIG. 1 is an architectural diagram illustrating an RPA system according to one embodiment of the present invention.

[0018] [Diagram 5] FIG. 1 is an architectural diagram illustrating a computing system configured to facilitate cross-session automation of an RPA robot, according to one embodiment of the present invention.

[0019] [Figure 6] 6A-6G illustrate an example of completing a form in a user session using an RPA robot, an IPC facilitator, and an RPA driver running in a robot session, according to one embodiment of the present invention.

[0020] [Figure 7] 7A-7G illustrate an example of completing a form in a user session using an RPA robot running in a robot session with direct variable modification according to one embodiment of the present invention.

[0021] [Figure 8] 1 is a flowchart illustrating a process for automation of a process performed in a user session via an RPA robot, an IPC facilitator, and an RPA driver running in a robotic session, according to an embodiment of the present invention.

[0022] [Figure 9] 1 is a flowchart illustrating a process for automation of a process executed in a user session via an RPA robot executed in a robot session with direct variable modification according to an embodiment of the present invention.

[0023] [Figure 10] 1 illustrates an example of multiple client session RPA robots interacting with a main session application according to an embodiment of the present invention.

[0024] [Figure 11] FIG. 1 is a flow diagram illustrating a process of running an attended automated RPA robot in a client session according to an embodiment of the present invention.

[0025] [Figure 12] FIG. 2 is a flow diagram illustrating the execution of a multi-robot collaborative workflow between a main session robot M1, a client session application A1, and a pair of client session robots C1, C2 according to an embodiment of the present invention.

[0026] [Figure 13] FIG. 1 is a flow diagram illustrating parallel execution of a main session RPA robot and a client session RPA robot according to an embodiment of the present invention.

[0027] [Figure 14] FIG. 1 is a flow diagram illustrating execution of a client session RPA robot based on a trigger of a main session application according to an embodiment of the present invention.

[0028] [Figure 15] 4 is a flowchart illustrating a process for performing cross-session automation according to one embodiment of the present invention.

[0029] Unless otherwise indicated, like reference numbers denote corresponding features consistently throughout the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Some embodiments relate to the automation of a process executed in a first session via an RPA robot executing in a second session. For example, a form (e.g., a web page, an email application (e.g., Outlook®, etc.), a spreadsheet application (e.g., Excel®, etc.), a customer relationship management (CRM) system application (e.g., Salesforce®, etc.), an enterprise resource management (ERM) system, a supply chain management system, a custom computer application, a mobile application, another application with fillable fields, etc.) may be displayed in the first session. In some embodiments, the first session may be a user session (also referred to herein as a main session or parent session). One or more RPA robots that obtain and / or interact with data for an application in the first session may be executed in one or more other sessions (also referred to herein as a robot session, second session, or secondary session). In certain embodiments, the robot session may be a child session of the first session.

[0031] In some embodiments, an RPA robot running in a client session sends messages to an RPA driver running in a main session via inter-process communication (IPC). Based on the IPC communication from the RPA robot, the RPA driver may implement various operating system (OS) and / or application programming interface (API) level interactions with the computing system. For example, the driver may implement mouse movements, button clicks, menu navigation, text entry into fields, window opening and closing, moving and / or resizing windows, or any other suitable interactions (operations) without departing from the scope of the present invention.

[0032] To implement such functionality, the driver may include, be controlled by, or be invoked by an IPC facilitator. In some embodiments, the IPC facilitator may be a separate application or process, or may be a sub-process of the driver. The IPC facilitator may listen for IPC messages from the RPA robot in the client session (e.g., listen for triggers), send and receive communications to and from the RPA robot, monitor the execution status of the RPA robot, combinations of these, and the like. The facilitator may also cause the driver to implement desired functionality in the parent session based on the IPC communication from the RPA robot. In certain embodiments, the IPC facilitator may be configured with scripts for a given RPA robot. In some embodiments, the IPC facilitator may be custom designed for the functionality to be performed in the parent session.

[0033] In some embodiments, an RPA robot in a client session may directly modify variables stored in memory on a computing system, and an application running in a parent session that has access to such variables may update the displayed values ​​when the display is refreshed or when the application reloads its UI. For example, a variable in an Excel® spreadsheet may be directly modified by an RPA robot running in a client session, which will also change the value displayed in the parent session.

[0034] In some embodiments, the operation of the RPA robot does not prevent a user from using other applications or instances while the RPA robot is running, but data changes made by the RPA robot may appear to the user as the application's display is updated in the user session window, but in certain embodiments, the RPA robot may interact with applications that do not have a user interface.

[0035] In some embodiments, the process may operate as follows: A user may click a button or otherwise launch an RPA robot in a child session of a user session (e.g., a parent or main session that is launched when a user boots up the user's computing system). In certain embodiments, for example, the robot session may already be running or may be launched separately by the user from the robot tray. The user may have the robot execute its workflow in the child session, and the robot may interact with one or more applications in the main session by communicating with an IPC facilitator running in the parent session. The IPC facilitator may then have the RPA driver execute the desired function in the parent session.

[0036] In some embodiments, an RPA robot running in a robot session may pause operation and provide a message if at least one application associated with the workflow is not launched, and an IPC facilitator may launch the application so that the robot can continue operation. Once the robot starts executing its workflow, changes to the data displayed in the application may be visible in the main session window if changes were made via the IPC facilitator and the RPA driver. For example, data may be displayed as the driver enters data based on data provided by the RPA robot in the client session (e.g., via IPC, saving the data to a flat file or database).

[0037] As a non-limiting example, in some embodiments, a shortcut may be created in Salesforce® to launch an RPA robot in a client session or to interact with an RPA robot already running in a client session. When a user clicks on the shortcut, the RPA robot starts executing the RPA process as designed in the RPA workflow. Based on the results of that execution (e.g., retrieving information from a database, invoking an artificial intelligence (AI) / machine learning (ML) model and retrieving the results, retrieving information from multiple legacy systems, etc.), the RPA robot interacts with an IPC facilitator, which interacts with the Salesforce® application instance running in the parent session accordingly. This may change some of the displayed information in the running Salesforce® application instance, and may appear (be visible) to the user as, for example, the RPA driver is executing the interaction.

[0038] In certain embodiments, an RPA robot running in a client session may open an application in a parent session via an IPC facilitator. The IPC facilitator may then access, add, modify, and / or delete data related to the application via an RPA driver. In this way, even if the RPA robot is not running in the parent session, the RPA robot can cause interactions with the parent session application in a manner that appears similar to, but is functionally different from, an RPA robot running in the parent session itself.

[0039] In some embodiments, a user may have a robot execute a workflow in a client session that accesses a website and collects some information. The RPA robot then provides this information to an IPC facilitator, which has an RPA driver, for example, enter the collected information into a spreadsheet that is visible in the main session. In certain embodiments, an application, such as Salesforce®, is opened in the main session. The user then executes an automation in the client session that reads the current client ID and accesses a website (such as HubSpot®) to collect information about the client's interaction with the website. However, the attended RPA robot may interact with any suitable application through the RPA facilitator and / or obtain data from any suitable source (e.g., a database, another application, etc.) without departing from the scope of the present invention.

[0040] Applications of some embodiments include, but are not limited to, emulators, simulators, VMs, hybrid VMs that provide OS-level virtualization (e.g., Docker® containers, etc.). Some embodiments create and host one or more robotic sessions as a window that contains the UI of the application being controlled by the attended automation process. In certain embodiments, only the interface of the application with which the robot interacts is shown. In some embodiments, no client session window is shown at all. As used herein, "window" may apply to a window that represents a UI shown in a main UI, a second screen of a second display of a computing system, a virtual desktop, an isolated environment (i.e., a window that draws the UI of all applications (referred to as "children") launched within that environment and runs them (in the context of a host session) (referred to as "host"), etc.) without departing from the scope of the invention. Running multiple sessions allows a robot to operate in a robotic session while a user is interacting with a first session (e.g., a parent session, etc.). Thus, a user may be able to interact with various applications while the robot is interacting with its data.

[0041] In some embodiments, any desired number of sessions of any number of robots may be created and used without departing from the scope of the invention. For example, a user may operate in a first session, a first robot may operate in a second session, a second robot may operate in a third session, etc. In certain embodiments, multiple robots may operate in a single session and may take turns to interact with one or more common applications via an RPA facilitator.

[0042] The ability to create a session may be implemented, for example, via a Windows Terminal Services child session, allowing a user to create a session on their own machine without the need for the user to log out. The newly created session appears as a child window and contains and launches the applications present in the user's session, meaning that the separation between the user and the robot occurs at the UI level. For example, if a file is deleted, this occurs across all sessions running on the computing system.

[0043] Certain embodiments may be used for Robotic Process Automation (RPA). FIG. 1 is an architectural diagram illustrating an RPA system 100, according to one embodiment of the present invention. The RPA system 100 includes a designer 110 that enables developers to design and implement workflows. The designer 110 may provide solutions for application integration and automation of third-party applications, management information technology (IT) tasks, and business IT processes. The designer 110 may facilitate the development of automation projects, which are graphical representations of business processes. In brief, the designer 110 facilitates the development and deployment of workflows and robots.

[0044] An automation project enables rule-based process automation by allowing developers to control the order of execution and relationships between a custom set of steps developed in a workflow, defined herein as "activities." One commercial example of an embodiment of the designer 110 is UiPath Studio™. Each activity may include an action, such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0045] Some types of workflows may include, but are not limited to, sequences, flowcharts, FSMs, and / or global exception handlers. Sequences may be particularly suitable for linear processes, allowing flow from one activity to another without perturbing the workflow. Flowcharts may be particularly suitable for more complex business logic, allowing for the integration of decisions and the connection of activities in more diverse ways with multiple branching logical operators. FSMs may be particularly suitable for large workflows. FSMs may use a finite number of states at run time, which are triggered by conditions (i.e., transitions) or activities. Global exception handlers may be particularly suitable for determining workflow behavior when an execution error occurs or for debugging the process.

[0046] Once a workflow is developed in Designer 110, the execution of the business process is orchestrated by Conductor 120, which orchestrates one or more Robots 130 that execute the workflow developed in Designer 110. One commercial example of an embodiment of Conductor 120 is UiPath Orchestrator™. Conductor 120 facilitates the creation, monitoring, and management of deployment of resources in the environment. Conductor 120 may also act as one of the integration or aggregation points with third party solutions and applications.

[0047] The conductor 120 may manage all the robots 130, connecting and running them from a central point. Types of robots 130 that can be managed include, but are not limited to, attended robots 132, unattended robots 134, development robots (similar to unattended robots 134, but used for development and testing purposes), and non-production robots (similar to attended robots 132, but used for development and testing purposes). Attended robots 132 work alongside humans on the same computing system, triggered by user events or scheduled to run automatically. Attended robots 132 may be used with the conductor 120 for a centralized process deployment and logging medium. Attended robots 132 help human users accomplish various tasks and may be triggered by user events. In some embodiments, processes cannot be started from the conductor 120 for this type of robot and / or cannot run under a locked screen. In certain embodiments, the attended robot 132 can only be started from the robot tray or from a command prompt. In some embodiments, the attended robot 132 should operate under human supervision.

[0048] Unattended Robots 134 run hands-free in a virtual environment and can automate many processes. Unattended Robots 134 may be responsible for providing remote execution, monitoring, scheduling, and work queue support. In some embodiments, debugging of all robot types may be performed from Designer 110. Both attended and unattended robots may automate a variety of systems and applications, including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those produced by SAP®, SalesForce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0049] The conductor 120 may have various functions including, but not limited to, provisioning, deployment, versioning, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between robots 130 and the conductor 120 (e.g., web applications, etc.). Deployment may include ensuring correct delivery of package versions to robots 130 assigned for execution. Versioning may include managing unique instances of some processes or configurations in some embodiments. Configuration may include maintaining and delivering robot environments and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot identification data and maintaining user permissions. Logging may include storing and indexing logs in a database (e.g., an SQL database, etc.) and / or other storage mechanism (e.g., ElasticSearch, etc., which provides the ability to store and quickly query large data sets). The conductor 120 may provide interconnectivity by serving as a centralized point of communication for third-party solutions and / or applications.

[0050] The robot 130 is an execution agent that executes workflows embedded in the designer 110. One commercial example of some embodiments of the robot 130 is UiPath Robots™. In some embodiments, the robot 130 installs a service that is managed by the Microsoft Windows Service Control Manager (SCM) by default. As a result, such a robot 130 can open an interactive Windows session under the local system account and have the privileges of a Windows service.

[0051] In some embodiments, the robot 130 may be installed in user mode. For such a robot 130, this means that the robot 130 is installed under the user and has the same rights as the user. This feature may be available for high density (HD) robots and may ensure maximum utilization of each machine. In some embodiments, any kind of robot 130 may be configured in an HD environment.

[0052] The Robot 130 in some embodiments is divided into several components, each dedicated to a specific automation task. The Robot components in some embodiments include, but are not limited to, an SCM-managed Robot Service, a user-mode Robot Service, an Executor, an Agent, and a Command Line. The SCM-managed Robot Service manages and monitors Windows sessions to act as a proxy between the Conductor 120 and the execution host (i.e., the computing system on which the Robot 130 executes). Such a service is entrusted with and manages the credentials of the Robot 130. A console application is launched by the SCM under the local system.

[0053] The user-mode robot service in some embodiments manages and monitors Windows sessions and acts as a proxy between the conductor 120 and the execution host. The user-mode robot service may be entrusted with and manage the credentials of the robot 130. If the SCM-managed robot service is not installed, a Windows application may be launched automatically.

[0054] An Executor may run a given job under a Windows session (i.e., an Executor may execute a workflow). An Executor may be aware of per-monitor dots per inch (DPI) settings. An Agent may be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. An Agent may be a client of this service. An Agent may request to start or stop jobs and change settings. A Command Line may be a client of the service. A Command Line is a console application that can request to start a job and wait for its output.

[0055] Splitting the robot 130 components as described above allows developers, support users, and computing systems to more easily execute, identify, and track what each component is doing. In this way, special behaviors may be configured for each component, for example, by setting different firewall rules for executors and services. In some embodiments, the executor may always be aware of per-monitor DPI settings. As a result, workflows may run at any DPI regardless of the configuration of the computing system on which they were created. In some embodiments, projects from the designer 110 may be independent of the browser zoom level. For applications that are not DPI aware or are intentionally marked as not aware, DPI may be disabled in some embodiments.

[0056] FIG. 2 is an architecture diagram illustrating a deployed RPA system 200 according to an embodiment of the present invention. In some embodiments, the RPA system 200 may be or may be part of the RPA system 100 of FIG. 1. It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, a robot application 210 includes an executor 212, an agent 214, and a designer 216. However, in some embodiments, the designer 216 may not be running on the computing system 210. The executor 212 executes the process. As shown in FIG. 2, multiple business projects may be running simultaneously. In such an embodiment, the agent 214 (e.g., a Windows service, etc.) is the single connection point for all the executors 212. All messages in such an embodiment are logged into the conductor 230, which further processes the messages via the database server 240, the indexer server 250, or both. As described above with respect to FIG. 1, the executor 212 may be a robotic component.

[0057] In some embodiments, a Robot represents an association between a machine name and a username. A Robot may manage multiple executors simultaneously. In a computing system that supports multiple interactive sessions running simultaneously (e.g., Windows Server 2012, etc.), multiple Robots may run simultaneously, each in a separate Windows session using a unique username. This is referred to as the HD Robot above.

[0058] Agent 214 is also responsible for sending the status of the robot (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading the necessary versions of packages to be executed. In some embodiments, communication between agent 214 and conductor 230 is always initiated by agent 214. In notification scenarios, agent 214 may open a WebSocket channel that is later used by conductor 230 to send commands to the robot (e.g., start, stop, etc.).

[0059] On the server side, the components include a presentation layer (web application 232, Open Data Protocol (OData) Representative State Transfer (REST) ​​Application Programming Interface (API) endpoints 234, notifications and monitoring 236), a service layer (API implementation / business logic 238), and a persistence layer (database server 240 and indexer server 250). Conductor 230 includes web application 232, OData REST API endpoints 234, notifications and monitoring 236, and API implementation / business logic 238. In some embodiments, most actions performed by a user on the conductor 230 interface (e.g., via browser 220) are performed by calling various APIs. Such actions may include, but are not limited to, starting a job on a robot, adding / removing data from a queue, scheduling a job to run without touching it, etc., without departing from the scope of the invention. Web application 232 is the visual layer of the server platform. In such an embodiment, the web application 232 uses HyperText Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the present invention. In such an embodiment, a user interacts with web pages from the web application 232 via the browser 220 to perform various actions to control the conductor 230. For example, a user may create robot groups, assign packages to robots, analyze logs per robot and / or per process, start and stop robots, etc.

[0060] In addition to the web application 232, the conductor 230 also includes a services layer that exposes an OData REST API endpoint 234. However, other endpoints may be included without departing from the scope of the invention. The REST API is used by both the web application 232 and the agent 214. In such an embodiment, the agent 214 is a supervisor of one or more robots on a client computer.

[0061] The REST API in such an embodiment covers configuration, logging, monitoring, and queuing functionality. In some embodiments, a configuration endpoint may be used to define and configure application users, permissions, robots, assets, releases, and environments. A logging REST endpoint may be used to log various information, such as errors, explicit messages sent by the robot, and other environment specific information. A deployment REST endpoint may be used by a robot to query the package version that needs to be executed when a start job command is used in the conductor 230. A queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to the queue, getting transactions from the queue, and setting the status of transactions.

[0062] The monitoring REST endpoint may monitor the web application 232 and the agents 214. The notification and monitoring API 236 may be a REST endpoint used to register the agents 214, deliver configuration settings to the agents 214, and send and receive notifications from the server and the agents 214. In some embodiments, the notification and monitoring API 236 may also use WebSocket communications.

[0063] The persistence layer includes a pair of servers in this embodiment: a database server 240 (e.g., a SQL server) and an indexer server 250. The database server 240 in this embodiment stores configurations of robots, robot groups, associated processes, users, roles, schedules, etc. Such information is managed in some embodiments via a web application 232. The database server 240 may manage queues and queue items. In some embodiments, the database server 240 may store messages logged by the robots (in addition to or instead of the indexer server 250).

[0064] Indexer server 250, which is optional in some embodiments, stores and indexes information logged by the robots. In certain embodiments, indexer server 250 may be disabled through a configuration setting. In some embodiments, indexer server 250 uses ElasticSearch®, a full-text search engine from an open source project. Messages logged by the robots (e.g., using activities such as log messages, line writes, etc.) may be sent via a logging REST endpoint to indexer server 250, where they may be indexed for future use.

[0065] FIG. 3 is an architecture diagram illustrating the relationship 300 between a designer 310, activities 320, 330, and a driver 340, according to an embodiment of the present invention. As described above, a developer uses the designer 310 to develop a workflow to be executed by the robot. The workflow may include user-defined activities 320 and UI automation activities 330. Some embodiments may identify non-textual visual components in an image, referred to herein as computer vision (CV). Some CV activities associated with such components may include, but are not limited to, click, type, get text, hover, element exists, refresh scope, highlight, and the like. In some embodiments, click identifies and clicks an element using, for example, CV, optical character recognition (OCR), fuzzy text matching, and multiple anchors. Type may identify an element using the above and type within the element. Get text identifies the location of a particular text and may be scanned using OCR. Hover may identify an element and hover over it. Element exists may use the techniques described above to check if an element is present on the screen. In some embodiments, there may be hundreds or thousands of activities that can be implemented in designer 310. However, any number and / or type of activities may be utilized without departing from the scope of the invention.

[0066] UI automation activities 330 are a subset of specialized low-level activities (e.g., CV activities, etc.) written in low-level code that facilitate interaction with the screen. UI automation activities 330 facilitate such interaction via drivers 340 that enable the robot to interact with desired software. For example, drivers 340 may include OS drivers 342, browser drivers 344, VM ​​drivers 346, enterprise application drivers 348, etc.

[0067] Driver 340 may interact with the OS at a low level to look for hooks, monitor keys, etc. Driver 340 may facilitate integration with Chrome®, IE®, Citrix®, SAP®, etc. For example, a "click" activity plays the same role in such different applications via driver 340.

[0068] FIG. 4 is an architecture diagram illustrating an RPA system 400 according to one embodiment of the present invention. In some embodiments, the RPA system 400 may be or may include the RPA systems 100 and / or 200 of FIG. 1 and / or FIG. 2. The RPA system 400 includes multiple client computing systems 410 that execute robots. The computing systems 410 can communicate with a conductor computing system 420 via web applications executed thereon. The conductor computing system 420 can in turn communicate with a database server 430 and an optional indexer server 440.

[0069] 1 and 3, it should be noted that although web applications are used in these embodiments, any suitable client / server software may be used without departing from the scope of the present invention. For example, the conductor may run a server-side application that communicates with a non-web-based client software application on a client computing system.

[0070] FIG. 5 is an architecture diagram illustrating a computing system 500 configured to facilitate session-to-session automation of an RPA robot, according to an embodiment of the present invention. In some embodiments, the computing system 500 may be one or more computing systems illustrated and / or described in this application. The computing system 500 includes a bus 505 or other communication mechanism for communicating information and a processor 510 connected to the bus 505 for processing information. The processor 510 may be any type of general-purpose or special-purpose processor, including a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), examples thereof, and / or any combination thereof. The processor 510 may also have multiple processing cores, at least some of which may be configured to perform a particular function. In some embodiments, multiple parallel processing may be used. In certain embodiments, at least one processor 510 may be a neuromorphic circuit that includes processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit may not require the typical components of a von Neumann computing architecture.

[0071] The computing system 500 further includes a memory 515 for storing information and instructions executed by the processor 510. The memory 515 may be comprised of any combination of random access memory (RAM), read-only memory (ROM), flash memory, cache, static storage devices such as magnetic or optical disks, or any other type of non-transitory computer readable medium, or any combination thereof. The non-transitory computer readable medium may be any available medium that can be accessed by the processor 510 and may include volatile media, non-volatile media, or both. The medium may be removable, non-removable, or both.

[0072] Additionally, the computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communication network via wireless and / or wired connections. In some embodiments, the communications device 520 may be configured to support any of a variety of wireless technologies, including Frequency Division Multiple Access (FDMA), Single Carrier FDMA (SC-FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), Orthogonal Frequency Division Multiplexing (OFDM), Orthogonal Frequency Division Multiple Access (OFDMA), Global System for Mobile Communications (GSM) communications, General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), cdma2000, Wideband-CDMA (W-CDMA), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), High Speed ​​Packet Access (HSPA), Long Term Evolution (LTE), LTE Advanced (LTE-A), 802.11x, Wi-Fi, Zigbee, Ultra Wideband (UWB), 802.16x, 802.15, Home Node-B (HnB), Bluetooth, Radio Frequency (RF), and other wireless technologies without departing from the scope of the present invention. The communication device 520 may be configured to use RFID, Infrared Data Association (IrDA), Near-Field Communications (NFC), Fifth Generation (5G), New Radio (NR), any combination thereof, and / or any other currently existing or future implemented communication standards and / or protocols. In some embodiments, the communication device 520 may include one or more antennas that are single, array, phased, switched, beamforming, beamsteering, combinations thereof, and / or any other antenna configuration without departing from the scope of the invention.

[0073] The processor 510 is further connected via bus 505 to a display 525, such as a plasma display, a liquid crystal display (LCD), a light emitting diode (LED) display, a field emission display (FED), an organic light emitting diode (OLED) display, a flexible OLED display, a flexible substrate display, a projection display, a 4K display, a high definition display, a Retina® display, an in-plane switching (IPS) display, or any other suitable display for displaying information to a user. The display 525 may be configured as a touch (haptic) display, a three-dimensional (3D) touch display, a multi-input touch display, a multi-touch display, or the like, using resistive, capacitive, surface acoustic wave (SAW) capacitive, infrared, optical imaging, dispersive signal techniques, acoustic pulse recognition, frustrated total internal reflection, or the like. Any suitable display device and haptic I / O may be used without departing from the scope of the invention.

[0074] A keyboard 530 and cursor control device 535, such as a computer mouse, touchpad, and the like, are further connected to bus 505 to allow a user to interface with computing system 500. However, in certain embodiments, a physical keyboard and mouse may not be present and the user may interface with the device solely through display 525 and / or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. In certain embodiments, no physical input devices and / or displays are present. For example, a user may interact with computing system 500 remotely via another computing system that communicates with computing system 500, or computing system 500 may operate autonomously.

[0075] The memory 515 stores software modules that provide functionality when executed by the processor 510. The modules include an operating system 540 for the computing system 500. The modules further include a cross-session automation module 545 that is configured to perform all or a portion of the processes described herein or derivatives thereof. The computing system 500 may also include one or more additional functionality modules 550 that include additional functionality.

[0076] Those skilled in the art will appreciate that the "system" may be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a mobile phone, a tablet computing device, a quantum computing system, any other suitable computing device, or a combination of devices, without departing from the scope of the present invention. The depiction of the above functions as being performed by a "system" is in no way intended to limit the scope of the present invention, but is intended to illustrate one example of many embodiments of the present invention. Indeed, the methods, systems, and apparatus disclosed herein may be implemented in a localized and distributed fashion consistent with computing technologies, including cloud computing systems.

[0077] It should be noted that some of the system functionality described herein has been illustrated as modules to further emphasize implementation independence. For example, a module may be implemented as hardware circuits including custom very large scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, and the like.

[0078] The modules may be implemented at least in part in software for execution by various types of processors. For example, an identified unit of executable code may comprise one or more physical or logical blocks of computer instructions, which may be organized as, for example, an object, a procedure, or a function. Notwithstanding this, the executable files of the identified modules need not be physically located together, but may comprise disparate instructions stored in various locations that, when logically combined, comprise the modules and achieve the above-mentioned purpose of the modules. Furthermore, the modules may be stored on a computer-readable medium without departing from the scope of the present invention, which may be, for example, a hard disk drive, a flash device, RAM, tape, and / or other such non-transitory computer-readable medium used to store data.

[0079] Indeed, a module of executable code may be a single instruction or many instructions, and may be distributed across several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and depicted herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be organized as a single data set, or may be distributed in different locations, including different storage devices, or may exist, at least in part, simply as electronic signals on a system or network.

[0080] Traditionally, in attended automation, a human user waits until an RPA robot running in the same session completes its task. However, in some embodiments, a human user creates one or more robotic sessions and hosts and runs the RPA robots in the sessions. Unlike traditional RPA systems, a user can benefit from the ability to interact with the user's computing system while the robot is executing the automation in the robotic session. In some embodiments, the user may monitor what the robot is doing and interact with the robot via a host automation window for the robotic session. In some embodiments, the user and the robotic session may be running on a remote machine controlled by the user's computing system.

[0081] However, in some embodiments, the RPA robot does not interact with the application that the user is using through an IPC facilitator. Instead, the RPA robot may interact with applications or other processes that are not visible to or used by the user through an RPA facilitator and an RPA driver. Such robots typically do not directly enhance the user's interaction with the application and may be considered unattended robots.

[0082] In certain embodiments, the RPA robot may be running on a user's computing system and drive the remote computing system through a remote runtime (e.g., through the UiPath Remote Runtime™). The UiPath Remote Runtime™ is a component that facilitates communication between remote applications or desktops (e.g., Citrix Virtual Apps and Desktops™, etc.) and dedicated UiPath extensions (e.g., UiPath Extension for Citrix™, UiPath Extension for Windows Remote Desktop, etc.). The UiPath Remote Runtime™ collects information related to targeted UI elements of the remote application and sends this information to the corresponding extensions, which allow selectors to be generated natively in the UI Explorer™.

[0083] 6A-6G illustrate an example of completing a form in a user session using an RPA robot, an IPC facilitator, and an RPA driver running in a robot session, according to an embodiment of the present invention. In FIG. 6A, a user session window 600 is shown where a user can interact with applications in the UI, and the robot is not currently running. A robot tray icon 610 is visible in the lower right portion of the user session window 600. In this embodiment, the RPA driver 640 includes an IPC facilitator 642 as a sub-process. However, in certain embodiments, the IPC facilitator 642 may be a separate application or process that communicates with the RPA driver 640 without departing from the scope of the present invention. In such an embodiment, the IPC facilitator 642 may communicate with the RPA driver 640 via IPC, API calls, etc. Indeed, in certain embodiments, IPC may not be used.

[0084] In FIG. 6B, the user launches a web browser and accesses the billing web page 620. In FIG. 6C, the user pulls out the robot tray 612 (e.g., by clicking on the robot tray icon 610) and selects the client session robot option 614 to run on the user's computing system. After selecting the robot to run, a robot session window 630 for the robot session appears as a child window on the screen, as shown in FIG. 6D. The RPA robot runs in the robot session. In this embodiment, a window 632 for the client session robot automatically launches within the robot session window 630, and the window 632 includes a button 634 for running a form filler workflow to retrieve data for the web page 620 in the user session.

[0085] In some embodiments, the robot session window may not be displayed and the robot may launch, run, and terminate automatically, invisible to the user. In certain embodiments, the robot may terminate the session after completing its workflow. In some embodiments, a robot session may be started, and the robot may launch and run, without using the robot tray 612 (e.g., by the user clicking a button in the main session application), rather than being launched from the robot tray.

[0086] Next, in FIG. 6E, after the user clicks on button 634, the robot starts retrieving data to fill in the form fields in the web page 620. Once the robot has retrieved some of the data, the data is sent to the IPC facilitator 642, which then has the RPA driver 640 enter the data into the web browser 620. For example, the RPA driver 640 may move the mouse over a given field and enter text from the data into the field. In FIG. 6E, the RPA driver 640 has already filled in the Invoice Number field, moved the mouse to the Invoice Amount field, clicked on the field, and entered the number 1. In FIG. 6E, the caret is displayed after the number 1. In certain embodiments, the RPA facilitator 642 may receive a complete set of data from the robot 632 before having the RPA driver 640 fill in the fields of the web page 620.

[0087] As described above, communication between a robot running in a robot session and an IPC facilitator 642 running in a user session is accomplished using an IPC protocol. The IPC protocol may facilitate communication over a network, pipes, Component Object Model (COM), remote procedure call (RPC), sockets, etc. Appropriate session creation mechanisms and IPC protocols may also be used for other supported operating systems. A robot may send status notifications (e.g., indicating that the robot is starting, running, paused, etc.), retrieved data, error messages, commands, or other communications to the IPC facilitator 642 via the IPC protocol.

[0088] Relevant text from the data captured by the robot is visible to the user as it is entered by the RPA driver 640 if the web page 620 is not covered by another window or minimized. Until the RPA driver 640 completes the data manipulation (interaction) for the form, the user can remain productive by interacting with other applications, or in some embodiments even interacting with the web page 620 itself, although the user's input may be overwritten if the RPA driver 640 changes data for the same field in which the user entered information.

[0089] Next, in Figure 6F, after the robot has completed its workflow, a message is displayed in window 632. As can be seen in the background, the robot has completed data retrieval for the form fields, and new input for the form fields is visible in web page 620 after input by RPA driver 640. The user may then close the robot session window 630, the robot may automatically close the robot session window 630 (and possibly the robot session itself), or the robot session window 630 may remain open. The user may then submit the completed form. See Figure 6G.

[0090] 7A-7G show an example of completing a form in a user session using an RPA robot running in a robot session with direct variable modification according to an embodiment of the present invention. In FIG. 7A, a user session window 700 is shown where the user can interact with the application in the UI and the robot is not currently running. A robot tray icon 710 is visible in the bottom right portion of the user session window 700.

[0091] In FIG. 7B, the user launches a web browser and accesses the billing web page 720. In FIG. 7C, the user pulls out the robot tray 712 (e.g., by clicking on the robot tray icon 710) and selects the client session robot option 714 to run on the user's computing system. After selecting the robot to run, a robot session window 730 for the robot session appears as a child window on the screen, as shown in FIG. 7D. The RPA robot runs in the robot session. In this embodiment, a window 732 for the client session robot automatically launches within the robot session window 730, and the window 732 includes a button 734 for running a form filler workflow for the web page 720 in the user session.

[0092] In some embodiments, the robot session window may not be displayed and the robot may launch, operate, and terminate automatically, invisible to the user. In certain embodiments, the robot may terminate its session after completing its workflow. In some embodiments, a robot session may be started, and the robot may launch and operate, without using the robot tray 712 (e.g., by the user clicking a button in the main session application), rather than being launched from the robot tray.

[0093] 7E, after the user clicks button 734, the robot begins filling out form fields in web page 720 by accessing and interacting with the form fields' variables in web page 720. For example, the robot may change memory values ​​of fields stored in RAM, permanent local storage, a database, or any other type of storage or location without departing from the scope of the invention. In certain embodiments, the variables may be assigned to the web application associated with web form 720 and stored as part of the memory used by that web application.

[0094] Each memory value modified by the robot causes the respective field of the web page 720 to be updated when the web page 720 is refreshed. Associated text modified by the robot due to data changes is visible to the user when the web page 720 is not covered by another window or minimized. Unlike the example of FIGS. 6A-6G where the RPA driver 640 manipulates the mouse to input text at the API level and / or through native mechanisms, the data appears in the form fields of the web page 720 without such interaction. Until the robot completes the data manipulation for the form, the user can interact with other applications to remain productive, or in some embodiments even interact with the web page 720 itself, although the user's input may be overwritten if the robot modifies data for the same field in which the user entered information.

[0095] Next, in Figure 7F, after the robot has completed its workflow, a message is displayed in window 732. As can be seen in the background, the robot completes the data changes for each memory variable associated with the form field, and after the web page 720 refreshes, the new inputs for the form fields are visible in the web page 720. The user may then close the robot session window 730, the robot may automatically close the robot session window 730 (and possibly the robot session itself), or the robot session window 730 may remain open. The user may then submit the completed form. See Figure 7G.

[0096] In some embodiments, the robot session may be a child session, and the child session may be created through a child session API of the operating system. In some embodiments, a second session may be created using a Windows Terminal Services child session or another child session API provided by the operating system without departing from the scope of the invention. A robot tray application (e.g., UiPath Robot Agent Desktop, etc.) or another application configured to launch a robot may then start a robot process in that child session using a process creation API in the operating system with appropriate arguments. The robot tray application or other suitable application may then communicate with the robot process using an appropriate protocol (e.g., a protocol built on named pipes, etc.).

[0097] 8 is a flow chart illustrating a process 800 for automation of processes performed in a user session via an RPA robot, an IPC facilitator, and an RPA driver running in a robotic session, according to one embodiment of the present invention. The process begins with launching a user session window at 810, which may be, for example, a main window associated with an operating system running on the user computing system. A robot session window is then launched at 820 as a child window of the user session window. In some embodiments, the robot session window may be launched, for example, in response to a robot being started or launched. A robot is then started in the robotic session at 830.

[0098] At 840, the robot executes and communicates with the IPC facilitator via the IPC. The IPC facilitator may accept status notifications, retrieved data, error messages, commands, or other communications from the robot via the IPC. The IPC facilitator may send messages to the robot (e.g., commands, status updates, error messages from the IPC facilitator or the RPA driver, etc.).

[0099] At 850, using the data obtained from the robot, the RPA facilitator controls or causes the RPA driver to perform application operations using the obtained data and / or other communications from the robot. For example, the RPA driver may move a mouse, click in fields, enter text, click buttons, navigate menus, open and close applications or windows, etc. After the robot execution is complete, at 860, the robot session may be ended and the robot session window may be closed, in some embodiments automatically.

[0100] FIG. 9 is a flow chart illustrating a process 900 for automation of a process executed in a user session via an RPA robot executed in a robot session with direct variable modification, according to an embodiment of the present invention. The process begins with launching a user session window at 910, which may be, for example, a main window associated with an operating system running on the user computing system. Next, at 920, a robot session window is launched as a child window of the user session window. In some embodiments, the robot session window may be launched, for example, in response to a robot being started or launched. Next, at 930, a robot is started in the robot session, and at 940, the robot manipulates data associated with an application (e.g., a web page, a spreadsheet, an ERP application, a sales application, etc.) running in the user session from the robot session. For example, the robot may create, modify, or delete information stored in the memory of the computing system to cause changes to common stored data accessed and used by applications in the user session. After the robot completes its execution, at 950, in some embodiments, the robot session may be terminated and the robot session window may be closed automatically.

[0101] In some embodiments, multiple running automation processes (e.g., UI automation processes, etc.) may be isolated in a client session. This may allow, for example, these processes to be invoked on demand from the main session via IPC. In this manner, multiple RPA robot processes may be available on demand and invoked from a main session application, a main session RPA robot, another client session RPA robot, etc. An RPA robot running in a client session may be able to collect data from the main session. In certain embodiments, an RPA robot running in a client session may wait for a specific trigger from the main session before executing its automation (automatic action) or part of an automation.

[0102] Figure 10 is an architecture diagram 1000 illustrating an example of multiple client session RPA robots interacting with a main session application, according to one embodiment of the present invention. In Figure 10, a main session 1010 and a client session 1020 are running. The client session 1020 includes a client session RPA robot 1 (1030) and a client session RPA robot 2 (1032).

[0103] The client session RPA robots 1030, 1032 communicate with a separate IPC facilitator application 1040 or an embedded RPA facilitator process or subroutine 1052 that is part of a driver 1050. In some embodiments, the driver 1050 may be, for example, the driver 340 of FIG. 3. The IPC facilitator 1040 or 1052 causes the driver 1050 to interact with the main session application and / or associated application objects 1060 (e.g., Component Object Model (COM) objects of a Microsoft Windows application) in a manner dictated by the respective workflow logic of the client RPA robot 1030 or 1032. For example, the driver 1050 may move the mouse, fill in form data, click one or more buttons, navigate menus, or any combination thereof, in the main session application 1060. In certain embodiments, the RPA facilitator application 1040 may communicate directly with the main session application and / or associated application objects 1060.

[0104] In certain embodiments, the RPA robot running in the client session may be an attended automation robot. For example, the RPA robot running in the client session may request input from a user in the main session. This may be done via IPC using the main session robot, a driver, etc.

[0105] FIG. 11 is a flow diagram 1100 illustrating a process of running an attended automated RPA robot in a client session, according to one embodiment of the present invention. In this embodiment, at 1120, a user utilizes a main session application 1110 that initiates a client session RPA robot process via an IPC. In some embodiments, the main session application 1110 may be an RPA assistant application that allows a user in the main session to interact with the attended RPA robot in the client session. For example, a dashboard of the RPA assistant application may display the current RPA processes supported by the robot, and the user may select a process to run. In certain embodiments, the RPA assistant application may display the steps of the client session workflow and / or the execution status while the client session RPA robot is running.

[0106] After the client session RPA robot starts at 1120, the client session RPA robot executes a first portion of its logic at 1130. After the client session RPA robot executes the first portion of the logic at 1130, it requests user input at 1140. This may be by a message in an application running in the main session, a message from the RPA robot running in the main session, displayed via an IPC facilitator and driver interacting with the main session application, etc. The user then provides the requested input, which is sent via IPC to the client session RPA robot, which executes a second portion of the logic at 1150 using the input. After the client session RPA robot process ends at 1160, the main session application 1110 may receive a notification that the RPA robot process has ended.

[0107] In certain embodiments, parts of the automation may be executed in the main session and other parts of the automation may be executed in the client session. For example, a main session application or RPA robot may execute a particular action in the main session and then have an RPA robot executing in a client session execute another part of the automation when the main session application or robot reaches a particular point in its execution, receives a trigger, a logical condition is met, etc. The client session RPA robot may execute parts of the automation sequentially or in parallel with the main session application or robot. The client session RPA robot may provide instructions, requested data, execution results, etc. to the main session application or robot.

[0108] FIG. 12 is a flow diagram 1200 illustrating the execution of a multi-robot collaborative workflow between a main session robot M1, a client session application A1, and a pair of client session robots C1, C2, according to an embodiment of the present invention. A1, C1, and / or C2 may be in the same client session or in different client sessions. M1 starts the execution of its workflow and reaches an activity that calls (requests) C1 to complete the workflow. In some embodiments, this may be performed by an "invoke workflow" activity with a flag set to execute an RPA process in the client session. M1 makes C1 execute its workflow via IPC, and the call activity waits. During this time, in some embodiments, M1 may execute other tasks.

[0109] After C1 completes the workflow, it notifies M1 via IPC and M1 resumes execution until it reaches an activity that calls A1, a process in the client session, to begin execution or complete a task. M1 causes A1 to perform and / or execute the task via IPC and the call activity (e.g., an "invoke process" activity, etc.) waits. In some embodiments, the invoke process activity may have a flag set to execute A1 in the client session. Again, in some embodiments, M1 may execute other tasks while waiting for A1.

[0110] After A1 completes its process or task, it notifies M1 via IPC and M1 resumes execution until it reaches the activity that calls C2 to complete the workflow. M1 then causes C2 to execute its workflow via IPC and the call activity waits. Again, in some embodiments, M1 may perform other tasks while waiting for C2.

[0111] Once C2 has completed its workflow, M1 is notified via IPC and resumes execution until it reaches an activity that calls C1 again to complete the workflow. This may be the same workflow activity previously executed or a different workflow or series of activities. M1 causes C1 to execute its workflow via IPC and the call activity waits. Furthermore, in some embodiments, M1 may execute other tasks while waiting for C1. Once C1 has completed its workflow, M1 is notified via IPC and resumes execution until M1's workflow is completed. In certain embodiments, C1 may remain running after it was originally called and may still be running when it is called again by M1.

[0112] As noted above, in some embodiments, M1 may include one or more "invoke workflow" activities and / or "invoke process" activities that may invoke RPA robots (e.g., C1 and / or C2) and / or other processes (e.g., A1) in the client session. Such activities may start the respective RPA robots and / or processes, communicate with already running RPA robots and / or processes, etc. Such activities may be created when an RPA developer develops a workflow for M1. In certain embodiments, an RPA robot may invoke an RPA robot and / or process from a client session to a main session via an RPA facilitator or other suitable functionality.

[0113] During execution, M1 may communicate with A1, C1, C2, and vice versa, via IPC. For example, M1 may send comments and requests to A1, C1, C2, M1 may receive status messages, results, and error messages from A1, C1, C2, etc. In this manner, M1 may act as a main RPA robot that controls client session RPA robots and / or other processes (e.g., applications, etc.) in the client session. In certain embodiments, M1, A1, C1, and / or C2 may operate in parallel.

[0114] In some embodiments, M1 may perform all interactions with applications running in the main session, which may eliminate the need for an IPC facilitator. Thus, C1, C2 may perform various data retrieval and processing tasks, and M1 may perform UI interaction tasks. In certain embodiments, at least some workflow activities of M1, C1, and / or C2 may be executed in parallel. One such example is shown in flow diagram 1300 of FIG. 13, where C1 executes based on M1's logic and execution is in parallel. During sections of C1, communication may be sent between M1 and C1. However, in certain embodiments, C1 may complete its execution independently and may complete its workflow execution after M1 has finished.

[0115] In some embodiments, all RPA robot activity is performed in a client session. The RPA robot may be called from the main session by a robot executor, such as a UiPath® robot tray, an IPC facilitator, or another application. Results of the RPA robot's execution and other communications may be provided to the robot executor or other application via IPC.

[0116] In certain embodiments, a standardized communication format may be used for IPC communication between the client session and the main session. For example, the main session process and the client session process may exchange XAML data. This helps ensure that the communication is in a format that each process can recognize.

[0117] In some embodiments, IPC may be used between RPA processes running in the same session for synchronization purposes. For example, multiple RPA processes may run in parallel in the background with a foreground process in the main session. IPC may provide a synchronization mechanism to pass information between the background process and the foreground or main process.

[0118] In certain embodiments, the driver (e.g., driver 340 in FIG. 3) will be loaded in the main RPA workflow process. Communication between the child RPA process and the main RPA workflow process may occur over IPC, so that the main RPA workflow process may utilize the driver to perform operations on the client session robot based on the IPC information.

[0119] In some embodiments, the client session RPA robot may handle applications that are open in the main session. For example, the client session RPA robot may handle an Excel spreadsheet that is open in the main session. This may be done, for example, via IPC between the client session RPA robot and the RPA facilitator. The RPA facilitator may have a driver execute interactions with the Excel spreadsheet (e.g., enter data into the spreadsheet, modify table values, etc.) based on information provided by the client session RPA robot. In certain embodiments, activities may be included in the RPA workflow that automatically retrieves the application's data between both sessions (e.g., Microsoft Office activity, web browser activity, etc.). In this way, the client session RPA robot may essentially handle applications that are running in the main session as if the applications were running in the client session RPA robot's own session. Indeed, in some embodiments, the RPA developer may not even be aware that the workflow or parts of it that the RPA developer is developing are executed in the client session.

[0120] In certain embodiments, for example, when designing an RPA robot using a designer application, an RPA developer may set a client session flag in an "invoke workflow" activity or an "invoke process" activity. A workflow invocation activity invokes an RPA workflow. A process invocation activity executes a process available on the local machine. By setting such a flag, an RPA workflow or process may be invoked in a client session rather than the main session.

[0121] In some embodiments, the execution of the client session RPA robot may be based on a trigger from the main session application and / or the RPA robot. FIG. 14 is a flow diagram 1400 illustrating the execution of a client session RPA robot based on a trigger from the main session application, according to an embodiment of the present invention. The main session application A1 (which may be an RPA robot in some embodiments) starts execution and waits for a trigger. After the trigger is received, A1 starts the execution of the client session RPA robot C1 via IPC. During the section of C1, communication may be sent between A1 and C1. Then, A1 waits for another trigger and C1 executes its workflow.

[0122] 15 is a flow chart illustrating a process 1500 for performing cross-session automation according to an embodiment of the present invention. The process begins with executing an RPA robot in a main session of a computing system at 1510. An RPA robot or application is called or launched in a client session by the main session RPA robot over IPC at 1520. Messages are communicated between the client session RPA robot or application and the main session RPA robot over IPC at 1530. In some embodiments, the main session RPA robot pauses its execution at 1540 until the client session RPA robot or application finishes executing.

[0123] At 1550, a result of the execution of the client session RPA robot or application is received by the main session RPA robot via the IPC. At 1560, the main session RPA robot uses the result to complete at least a portion of the main session RPA robot's workflow, interact with an application or application object running in the main session, or both. In some embodiments, at 1570, the process is repeated for the next client session RPA robot or client session application, and the process returns to step 1520.

[0124] In some embodiments, the client session RPA robot, the client session application, or both, are called or launched two or more times in the workflow of the main session RPA robot. In certain embodiments, the client session application is called or launched via a process invocation activity in the workflow of the main session RPA robot, or the client session RPA robot is called or launched via a workflow invocation activity in the workflow of the main session RPA robot, or both. In some embodiments, the process invocation activity includes a flag indicating that the client session application runs in the client session, or the workflow invocation activity includes a flag indicating that the client session RPA robot runs in the client session, or both. In certain embodiments, the client session application, the client session RPA robot, or both, continue to run after providing the results. In some embodiments, the IPC communication between the main session RPA robot and the client session RPA robot and / or the client session application is in a standardized communication format.

[0125] The process steps performed in Figures 8-15 may be executed by a computer program to encode instructions for a processor to perform at least a portion of the process described in Figures 8-15, according to one embodiment of the present invention. The computer program may be stored on a non-transitory computer readable medium. The computer readable medium may be, but is not limited to, a hard disk drive, a flash device, RAM, tape, and / or any other such medium or combination of media used to store data. The computer program may include encoded instructions for controlling a processor of a computing system (such as, for example, processor 510 of computing system 500 of Figure 5) to perform all or a portion of the process steps described in Figures 8-15, which may also be stored on a computer readable medium.

[0126] The computer program may be implemented in hardware, software, or a hybrid implementation. The computer program may be composed of modules designed to operatively communicate with each other and pass information or instructions to display. The computer program may be configured to run on a general purpose computer, an ASIC, or any other suitable device.

[0127] It will be readily understood that the components of the various embodiments of the present invention, as generally described and illustrated in the drawings herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention as illustrated in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention.

[0128] The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, references throughout this specification to "certain embodiments," "some embodiments," or similar language means that the particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Thus, appearances of the phrases "in certain embodiments," "in some embodiments," "in other embodiments," or similar language throughout this specification do not necessarily all refer to the same group of embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0129] It should be noted that references to features, advantages, or similar language throughout this specification do not imply that all of the features and advantages that may be realized in the present invention are or are included in any single embodiment of the present invention. Rather, language referring to features and advantages is understood to mean that the particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, descriptions of features, advantages, and similar language throughout this specification may, but do not necessarily, refer to the same embodiment.

[0130] Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize that the invention can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the invention.

[0131] Those skilled in the art will readily appreciate that the above invention may be implemented with steps in a different order and / or with hardware elements in a different configuration than that disclosed. Thus, while the present invention has been described based on such preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative constructions are apparent while remaining within the spirit and scope of the invention. Accordingly, reference should be made to the appended claims to determine the metes and bounds of the invention.

Claims

1. 1. A computer-implemented method comprising: running a robotic process automation (RPA) robot in a main session of a computing system; calling or invoking an RPA robot in a client session via inter-process communication (IPC) by the RPA robot in the main session; receiving, by the RPA robot of the main session via an IPC, one or more messages from the RPA robot of the client session; using results from the one or more messages received from the RPA robot of the client session to complete at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; The method according to claim 1, further comprising:

2. calling or invoking an application in the client session via IPC by the RPA robot of the main session; receiving, by the RPA robot of the main session via IPC, one or more messages from an application of the client session during and / or after execution of a task of the application; using results from the one or more messages received from an application of the client session, by the RPA robot of the main session to complete at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; The method of claim 1 further comprising:

3. Pausing execution of the RPA robot in the main session until an application in the client session completes execution. The method of claim 2 further comprising:

4. 3. The method of claim 2, wherein the application of the client session is called or launched two or more times in the workflow of the RPA robot of the main session.

5. calling or invoking additional applications in the client session via IPC by the RPA robot of the main session; receiving one or more messages from an additional application of the client session; using results from the one or more messages received from the additional application of the client session to complete, by the RPA robot of the main session, at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; The method of claim 2 further comprising:

6. 3. The method of claim 2, wherein an application of the client session is called or launched via a process invocation activity of a workflow of an RPA robot of the main session.

7. 7. The method of claim 6, wherein the process invocation activity includes a flag indicating that an application of the client session is to be executed in the client session.

8. 3. The method of claim 2, wherein after the task is completed, an application of the client session continues to run.

9. 3. The method of claim 2, wherein IPC communication between the RPA robot of the main session and the application of the client session is in a standardized communication format.

10. 2. The method of claim 1, further comprising pausing execution of an RPA robot of the main session until an RPA robot of the client session completes execution.

11. calling or initiating additional RPA robots in the client sessions via IPC by the RPA robots of the main session; receiving one or more messages from additional RPA robots of the client session via an IPC; using results from the one or more messages received from the additional RPA robot of the client session to complete at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; The method of claim 1 further comprising:

12. 2. The method of claim 1, wherein the RPA robot of the client session is called or invoked two or more times in the workflow of the RPA robot of the main session.

13. 2. The method of claim 1, wherein the RPA robot of the client session is called or initiated via a workflow invocation activity of a workflow of the RPA robot of the main session.

14. 14. The method of claim 13, wherein the workflow invocation activity includes a flag indicating that an RPA robot of the client session is to be executed in the client session.

15. 1. A computer-implemented method comprising: running a robotic process automation (RPA) robot in a main session of a computing system; calling or invoking an application in a client session via inter-process communication (IPC) by the RPA robot of the main session; receiving, by the RPA robot of the main session, via IPC, one or more messages from an application of the client session; using results from the one or more messages received from an application of the client session, by the RPA robot of the main session to complete at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; The method according to claim 1, further comprising:

16. calling or invoking additional applications in the client session via IPC by the RPA robot of the main session; receiving one or more messages from an additional application of the client session; using results from the one or more messages received from the additional application of the client session to complete, by the RPA robot of the main session, at least a portion of a workflow of the RPA robot of the main session, or to interact with an application or application object running in the main session, or both; 16. The method of claim 15 further comprising:

17. 16. The method of claim 15, wherein an application of the client session is called or launched via a process invocation activity of a workflow of an RPA robot of the main session.

18. 1. A computer-implemented method comprising: running a robotic process automation (RPA) robot in a main session of a computing system; calling or invoking an RPA robot in a client session via inter-process communication (IPC) by the RPA robot in the main session; calling or invoking an application in a client session via IPC by the RPA robot of the main session; receiving, by the RPA robot of the main session, one or more messages from the RPA robot of the client session and an application of the client session via an IPC; using results from the RPA robot of the client session and the one or more messages received from the application of the client session, to complete at least a portion of the workflow of the RPA robot of the main session, or to interact with one or more applications and / or application objects running in the main session, by the RPA robot of the main session; The method according to claim 1, further comprising:

19. calling or initiating additional applications, additional RPA robots, or both, in the client session via IPC by the RPA robot of the main session; receiving one or more messages from an additional application of the client session, an additional RPA robot of the client session, or both, via an IPC; completing, by the RPA robot of the main session, at least a portion of a workflow of the RPA robot of the main session, or interacting with applications and / or application objects running in the main session, or both, using results from the one or more messages received from an additional application of the client session, an additional RPA robot of the client session, or both; 20. The method of claim 18, further comprising:

20. 19. The method of claim 18, wherein an application of the client session is called or launched via a process invocation activity of a workflow of an RPA robot of the main session, or the RPA robot of the client session is called or launched via a workflow invocation activity of a workflow of an RPA robot of the main session, or both.

Citation Information

Patent Citations

  • Automation windows for robotic process automation

    US10654166B1

  • Robotic beacon device and system for remote triggering of events

    US20190122200A1

  • Integrated robotics and access management for target systems

    US9817967B1