Inserting probabilistic models and regulatory systems in deterministic workflows of robotic process automation

Abstract

Probabilistic models can be used in deterministic workflows for robotic process automation (RPA). Machine learning (ML) introduces a probabilistic framework where the outcome is uncertain, and thus the steps are uncertain. Deterministic workflows can be mixed with probabilistic workflows, or probabilistic activities can be inserted into deterministic workflows in order to create more dynamic workflows. A governance system can be used to monitor ML models and issue alerts, disable RPA robots, bypass RPA robots, or roll back to a previous version of the ML model when an error is detected by a data drift detector, a concept drift detector, or both.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of U.S. Non-Provisional Patent Application No. 16 / 708,083, filed December 9, 2019, and U.S. Provisional Patent Application No. 62 / 915,434, filed October 15, 2019. The subject matter of these earlier filed applications is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present invention relates generally to robotic process automation (RPA), and more particularly to inserting probabilistic models and regulatory systems in deterministic workflows for RPA. BACKGROUND

[0004] Current RPA systems work in a logical manner with decision points in sequential workflows. However, such workflows are based on set logic and are not dynamic. Accordingly, improved methods can be beneficial. SUMMARY

[0005] Certain embodiments of the present invention can provide solutions to problems and needs in the art that have not yet been fully identified, understood, or addressed by current RPA technology. For example, some embodiments of the present invention relate to inserting probabilistic models and regulatory systems in deterministic workflows for RPA.

[0006] In an embodiment, a computer-implemented method for implementing probabilistic models in deterministic workflows for robotic process automation (RPA) includes generating a deterministic RPA workflow for an RPA robot, generating a plurality of RPA robots configured to perform the deterministic RPA workflow, and deploying the plurality of RPA robots. The computer-implemented method further includes training a machine learning (ML) model associated with a probabilistic activity to replace a deterministic activity in the RPA workflow.

[0007] In another embodiment, a computer-implemented method for implementing probabilistic models in deterministic workflows for RPA includes generating a plurality of RPA robots configured to perform a deterministic RPA workflow, and deploying the plurality of RPA robots configured to perform the deterministic RPA workflow. The computer-implemented method further includes training a machine learning (ML) model associated with a probabilistic activity to replace a deterministic activity in the RPA workflow.

[0008] In yet another embodiment, a computer-implemented method includes generating an RPA robot configured to execute an RPA workflow that includes probabilistic activities, and deploying the RPA robot. The computer-implemented method further includes monitoring, by a governance system, a machine learning (ML) model invoked by a probabilistic activity in the RPA workflow to ensure that the ML model is running correctly on the input side via a data drift detector using a metric and on the output side using a concept drift detector. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to facilitate an understanding of the advantages of certain embodiments of the present application, reference is made to the detailed description of the application as shown in the Figures. While it should be understood that these drawings depict the typical embodiments of the application and are not therefore to be considered to be limiting of its scope, the application will be described and explained with additional specificity and detail by utilizing the accompanying drawings in which:

[0010] Figure 1 is an architectural diagram illustrating an RPA system according to an embodiment of the present application.

[0011] Figure 2 is an architectural diagram illustrating a deployed RPA system according to an embodiment of the present application.

[0012] Figure 3 is an architectural diagram illustrating the relationship between a designer, activities, and drivers according to an embodiment of the present application.

[0013] Figure 4 is an architectural diagram illustrating an RPA system according to an embodiment of the present application.

[0014] Figure 5 is an architectural diagram illustrating a computing system configured to use probabilistic models in deterministic workflows of RPA according to an embodiment of the present application.

[0015] Figure 6 is a flow diagram illustrating a process for implementing probabilistic models in deterministic workflows of RPA according to an embodiment of the present application.

[0016] Figure 7 is a flow diagram illustrating a process of a governance system according to an embodiment of the present application.

[0017] Figure 8 is an architectural diagram of a governance system according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] Some embodiments relate to using probabilistic models in deterministic workflows of RPA. For example, such embodiments can enable faster initial deployment. Machine learning introduces a probabilistic framework in which outcomes are uncertain, and thus steps are uncertain. Deterministic workflows can be mixed with probabilistic workflows, or probabilistic activities can be inserted into deterministic workflows in order to create more dynamic workflows. RPA workflows typically take the form of a logical flow with strong determinism, containing a fixed number of activities. However, in some embodiments, probabilistic activities can be inserted into the place of one or more deterministic activities in a workflow. In order to insert a probabilistic activity, certain factors can be considered, such as a confidence level of the probabilistic activity. If a machine learning (ML) model associated with the probabilistic activity is trained such that it becomes accurate enough to be used, the probabilistic activity can be used in the workflow in place of a deterministic activity. Thus, in some embodiments, a workflow can start out as deterministic in nature and be subsequently modified to be probabilistic.

[0019] Regulatory system

[0020] However, testing a probabilistic system can be difficult. Thus, in some embodiments, a regulatory system is used to monitor RPA robots executing probabilistic workflows to determine correct operation. For example, consider the case of an RPA implementation on an airplane. The robot can be trained to perform certain controls, such as causing the airplane to sacrifice altitude for speed if the robot determines that a stall is likely to occur. However, depending on the flight conditions, this can not always be the desired action. For example, performing this action shortly after takeoff can result in a crash.

[0021] The regulatory system can monitor actions taken by the robot, as well as other data from the system. In the example above, the data monitored can include pilot actions. If the regulatory system determines that the pilot pulls back on the yoke to stop the dive shortly after the robot causes the airplane to dive, then the regulatory system can disable the robot or the portion of the robot workflow, and send the collected data so that the robot can be retrained.

[0022] In some embodiments, the ML model can be monitored to ensure that it is operating correctly on both the input side and the output side with metrics. For example, if the ML model requires 16 data feeds, but one of the data feeds is corrupted, then it can not be appropriate to use the ML model. This is a monitoring for "data drift." Data drift detectors can be included on the input side.

[0023] On the output side, one can know the expected range. For example, if it is known that the rate of return visits to a certain hospital never exceeds 10%, but it is determined that 290% of the patients return, something is wrong. This is called concept drift, which can be monitored by a concept drift detector. Thus, a regulatory system can expect an ML model to perform in a certain way and to comply with certain parameters. If a data drift detector indicates an error, a concept drift detector indicates an error, or both, the regulatory system can issue an alert.

[0024] Consider the case of a regulatory system for a regulatory ML model implemented in an airplane. If a ML model that affects the control surfaces is run, when the autopilot is engaged the nose of the plane sharply sinks or rapidly descends, this can be flagged by the regulatory system, and the robot that invoked the ML model can be disabled or bypassed by manual control. If the robot is not mission critical, or a previous version of the ML model is known to work with high confidence in a mission critical system, the regulatory system can roll back the ML model to the previous version.

[0025] Figure 1 is an architectural diagram illustrating an RPA system 100 according to embodiments of the present application. The RPA system 100 includes a designer 110 that allows a developer to design and implement workflows. The designer 110 can provide a solution for application integration and automating third-party applications, managing information technology (IT) tasks, and business IT processes. The designer 110 can facilitate the development of automation projects, which are graphical representations of business processes. In short, the designer 110 facilitates the development and deployment of workflows and robots.

[0026] Automation projects enable the automation of rule-based processes by giving the developer control over the order of execution and the relationships between the custom step sets developed in a workflow, which is defined herein as an “activity.” One commercial example of an embodiment of the designer 110 is UiPath Studio TM Each activity can include actions such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows can be nested or embedded.

[0027] Some types of workflows can include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences can be particularly suitable for linear processes, enabling the flow from one activity to another without cluttering the workflow. Flowcharts can be particularly suitable for more complex business logic, enabling the integration of decisions and the connection of activities in a more diverse manner through multiple branching logic operators. FSMs can be particularly suitable for large workflows. FSMs can use a finite number of states in their execution, which are triggered by conditions (i.e., transitions) or activities. Global exception handlers can be particularly suitable for determining workflow behavior when an execution error is encountered and for debugging processes.

[0028] Once the workflows have been developed in the designer 110, the conductor 120 orchestrates the execution of the business processes, the conductor 120 orchestrates one or more robots 130 to execute the workflows that have been developed in the designer 110. One commercial example of an embodiment of the conductor 120 is UiPath Orchestrator TM The conductor 120 facilitates the creation, monitoring, and deployment of resources in a managed environment. The conductor 120 can act as an integration point with third-party solutions and applications.

[0029] The conductor 120 can manage fleets of robots 130, connecting and executing robots 130 from a centralized point. Types of robots 130 that can be managed include, but are not limited to, attended robots 132, unattended robots 134, development robots (similar to unattended robots 134, but for development and testing purposes), and non-production robots (similar to attended robots 132, but for development and testing purposes). Attended robots 132 are triggered by user events and work alongside humans on the same computing system. Attended robots 132 can be used with the conductor 120 for centralized process deployment and logging media. Attended robots 132 can assist human users in completing various tasks and can be triggered by user events. In some embodiments, processes cannot be started from the conductor 120 on this type of robot, and / or they cannot run under a locked screen. In certain embodiments, attended robots 132 can only be started from a robot tray or from a command prompt. In some embodiments, attended robots 132 should be run under the supervision of a human.

[0030] Unattended bots 134 run unattended in a virtual environment and can automate several processes. Unattended bots 134 can be responsible for remote execution, monitoring, scheduling, and support for work queues. In some embodiments, debugging for all bot types can be run in the designer 110. Both attended and unattended bots can automate various systems and applications, including but not limited to mainframes, web applications, VMs, enterprise applications (e.g., hosted by applications in production, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0031] The commander 120 may have various functions including, but not limited to, provisioning, deploying, configuring, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between the robots 130 and the commander 120 (e.g., a web application). Deployment may include ensuring that the correct version of a data package is delivered to the designated robot 130 for execution. Configuration may include maintenance and delivery of robot environment and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot identification data and maintaining user permissions. Logging may include storing and indexing records into a database (e.g., a SQL database) and / or another storage mechanism (e.g., a SQL database) that provides the ability to store and quickly query large data sets. ). The director 120 can provide interconnectivity by serving as a centralized communication point for third-party solutions and / or applications.

[0032] The robot 130 is an execution agent that runs the workflow built into the designer 110. One commercial example of some embodiments of the robot(s) 130 is UiPath Robots TM In some embodiments, the robot 130 is installed by default with Microsoft Services managed by the Service Control Manager (SCM). Therefore, such robots 130 can open interactive session, and has Permissions for services.

[0033] In some embodiments, robots 130 can be installed in user mode. This means that for such robots 130, they have the same rights as the user who installed the given robot 130. This feature can also be used for high-density (HD) robots, ensuring that each robot in the fleet is fully utilized to its full potential. In some embodiments, any type of robot 130 can be deployed in an HD environment.

[0034] Robots 130 in some embodiments are divided into components, each of which is dedicated to a particular automation task. Robot components in some embodiments include, but are not limited to, SCM-managed robot services, user-mode robot services, executors, agents, and command lines. SCM-managed robot services manage and monitor sessions and act as a proxy between director 120 and the execution host (i.e., the computing system on which robots 130 are executed). These services can be delegated and manage credentials for robots 130. SCM launches a console application under the local system.

[0035] User-mode robot services manage and monitor sessions and act as a proxy between director 120 and the execution host in some embodiments. User-mode robot services can be delegated and manage credentials for robots 130. If the SCM-managed robot services are not installed, then applications can automatically launch. Executors can run a given job under a session (i.e., they can execute workflows). Executors can know the dots-per-inch (DPI) settings of each monitor. Agents can be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. Agents can be clients of services. Agents can request to start or stop jobs and change settings. Command lines are clients of services. Command lines are console applications that can request to start a job and wait for its output.

[0036] Separating the components of robots 130 as described above helps developers, support users, and computing systems more easily run, identify, and track what each component is doing. Special behavior can be configured for each component in this way, such as setting different firewall rules for executors and services. In some embodiments, executors can always know the DPI settings of each monitor. Thus, workflows can be executed at any DPI, regardless of the configuration of the computing system that created the workflow. In some embodiments, projects from designer 110 can also be independent of browser zoom levels. For applications that do not know the DPI or are intentionally marked as not knowing, DPI can be disabled in some embodiments.

[0037]

[0038] Figure 2 is an architectural diagram illustrating a deployed RPA system 200 according to embodiments of the present invention. In some embodiments, RPA system 200 can be Figure 1 ​​​​RPA system 100, or can be part thereof. It should be noted that the client side, the server side, or both, can include any desired number of computing systems without departing from the scope of the invention. On the client side, the robot application 210 includes executors 212, agents 214, and designers 216. However, in some embodiments, the designers 216 can not run on the computing system 210. The executors 212 are running processes. Multiple business projects can run simultaneously, as shown in Figure 2 In this embodiment, the agents 214 (e.g., the service) are the single point of contact for all executors 212. All messages in this embodiment are logged into the director 230, which further processes them via the database server 240, the indexer server 250, or both. As noted above with respect to Figure 1 The executors 212 can be robot components.

[0039] In some embodiments, a robot represents an association between a machine name and a user name. A robot can manage multiple executors simultaneously. On a computing system that supports multiple interactive sessions running simultaneously (e.g., the server 2012), multiple robots can run simultaneously, each using a unique user name to run in a separate session. This is the HD robot mentioned above.

[0040] The agents 214 are also responsible for sending the status of the robot (e.g., sending "heartbeat" messages periodically indicating that the robot is still running) and downloading the required version of the data package to be executed. In some embodiments, the communication between the agents 214 and the director 230 is always initiated by the agents 214. In the notification scenario, the agents 214 can open a WebSocket channel that is subsequently used by the director 230 to send commands (e.g., start, stop, etc.) to the robot.

[0041] On the server side, there are included presentation layers (web application 232, Open Data Protocol (OData) Representational State Transfer (REST) Application Programming Interface (API) endpoints 234, and notifications and monitoring 236), service layers (API implementations / business logic 238), and persistence layers (database server 240 and indexer server 250). The conductor 230 includes the web application 232, OData REST API endpoints 234, notifications and monitoring 236, and API implementations / business logic 238. In some embodiments, most actions performed by a user in the interface of the conductor 220 (e.g., via the browser 220) are performed by invoking various APIs. Such actions can include, but are not limited to, starting jobs on robots, adding / removing data in queues, scheduling jobs for unattended runs, etc., without departing from the scope of the present invention. The web application 232 is the visual layer of the server platform. In the present embodiment, the web application 232 uses HyperText Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format can be used without departing from the scope of the present invention. In the present embodiment, a user interacts with web pages from the web application 232 via the browser 220 in order to perform various actions to control the conductor 230. For example, a user can create robot groups, assign data packages to robots, analyze records for each robot and / or for each process, start and stop robots, etc.

[0042] In addition to the web application 232, the conductor 230 includes a service layer that exposes OData REST API endpoints 234. However, other endpoints can be included without departing from the scope of the present invention. The REST APIs are consumed by both the web application 232 and the agent 214. In the present embodiment, the agent 214 is a supervisor of one or more robots on a client computer.

[0043] The REST APIs in the present embodiment encompass configuration, logging, monitoring, and queuing functionality. In some embodiments, configuration endpoints can be used to define and configure application users, permissions, robots, assets, releases, and environments. Logging REST endpoints can be used to log different information, such as errors, explicit messages sent by robots, and other environment-specific information. If a start job command is used in the conductor 230, robots can use a deployment REST endpoint to query for the data package version that should be executed. Queuing REST endpoints can be responsible for queue and queue item management, such as adding data to a queue, getting transactions from a queue, setting the status of a transaction, etc.

[0044] The monitoring REST endpoints can monitor the web application 232 and the agents 214. The notification and monitoring API 236 can be REST endpoints that are used to register agents 214, deliver configuration settings to the agents 214, and for sending / receiving notifications from the server and agents 214. In some embodiments, the notification and monitoring API 236 can also use WebSocket communication.

[0045] In this embodiment, the persistence layer includes a pair of servers - a database server 240 (e.g., a SQL server) and an indexer server 250. The database server 240 in this embodiment stores the configuration of robots, robot groups, related processes, users, roles, schedules, etc. In some embodiments, this information is managed through the web application 232. The database server 240 can manage queues and queue items. In some embodiments, the database server 240 can store messages recorded by robots (in addition to or instead of the indexer server 250).

[0046] The indexer server 250 (which is optional in some embodiments) stores and indexes information recorded by robots. In certain embodiments, the indexer server 250 can be disabled through a configuration setting. In some embodiments, the indexer server 250 uses Elasticsearch (which is an open source project full-text search engine). Messages recorded by robots (e.g., activity using record messages or writing rows, etc.) can be sent to the indexer server 250 through the record REST endpoint(s), where they are indexed for future use.

[0047] Figure 3is an architectural diagram illustrating the relationship 300 between the designer 310, activities 320, 330, and drivers 340 according to embodiments of the application. As described above, the developer uses the designer 310 to develop workflows that are executed by the robot. The workflows can include user-defined activities 320 and UI automation activities 330. Some embodiments are able to identify non-textual visual components in images, which are referred to herein as computer vision (CV). Some CV activities related to these components can include, but are not limited to, click, type, get text, hover, element exists, refresh scope, highlight, etc. In some embodiments, click uses, for example, CV, optical character recognition (OCR), fuzzy text matching, and multi-anchor to identify an element and click on it. Type can use the above and typing in elements to identify an element. Get text can identify the location of a particular text and scan it using OCR. Hover can identify an element and hover over it. Element exists can use the above techniques to check if an element exists on the screen. In some embodiments, there can be hundreds or even thousands of activities that can be implemented in the designer 310. However, any number and / or type of activities are available without departing from the scope of the application.

[0048] UI automation activities 330 are a subset of special low-level activities (e.g., CV activities) written in lower level code and are beneficial for interactions with the screen. The UI automation activities 330 facilitate these interactions via drivers 340, which allow the robot to interact with the desired software. For example, the drivers 340 can include OS drivers 342, browser drivers 344, VM drivers 346, enterprise application drivers 348, etc.

[0049] Drivers 340 can interact at a low level with the OS, look for hooks, monitor keys, etc. They can facilitate integration with different applications. For example, a "click" activity performs the same role in these different applications via the drivers 340.

[0050] Figure 4 is an architectural diagram illustrating an RPA system 400 according to embodiments of the application. In some embodiments, the RPA system 400 can be or can include the RPA system 100 and / or 200 of Figure 1 and / or Figure 2 The RPA system 400 includes a plurality of client computing systems 410 that run robots. The computing systems 410 are able to communicate with a conductor computing system 420 via a web application running thereon. The conductor computing system 420, in turn, is able to communicate with a database server 430 and an optional indexer server 440.

[0051] With respect to Figure 1 and Figure 3It should be noted that while web applications are used in these embodiments, any suitable client / server software can be used without departing from the scope of the application. For example, the director can run a server-side application that communicates with a non-web based client software application on the client computing system.

[0052] Figure 5 FIG. 5 is an architectural diagram illustrating a computing system 500 configured to use probabilistic models in deterministic workflows for RPA, according to embodiments of the present application. In some embodiments, the computing system 500 can be one or more of the computing systems depicted and / or described herein. The computing system 500 includes a bus 505 or other communication mechanism for communicating information, and a processor(s) 510 coupled to the bus 505 for processing information. The processor(s) 510 can be any type of general purpose processor, or a special purpose processor, including a central processing unit (CPU), a dedicated integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances of these, and / or any combination thereof. The processor(s) 510 can also have multiple processing cores, and at least some of the cores can be configured to perform specific functions. Multi-parallel processing can be used in some embodiments. In certain embodiments, at least one of the processor(s) 510 can be a neuromorphic circuit that includes processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit can not require typical components of a von Neumann computing architecture.

[0053] The computing system 500 also includes a memory 515 for storing information and instructions to be executed by the processor(s) 510. The memory 515 can be composed of any combination of random access memory (RAM), read only memory (ROM), flash memory, cache memory, static storage such as a magnetic or optical disk, or any other type of non-transitory computer readable medium, or combinations thereof. The non-transitory computer readable medium can be any available medium that is accessible by the processor(s) 510, and can include both volatile and non-volatile media, or both. The medium can also be removable or non-removable.

[0054] Additionally, computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communications network via wireless and / or wired connections. In some embodiments, communication device 520 can be configured to use frequency division multiple access (FDMA), single-carrier FDMA (SC-FDMA), time division multiple access (TDMA), code division multiple access (CDMA), orthogonal frequency division multiplexing (OFDM), orthogonal frequency division multiple access (OFDMA), global system for mobile communications (GSM), general packet radio service (GPRS), universal mobile telecommunications system (UMTS), cdma2000, wideband CDMA (W-CDMA), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), high-speed packet access (HSPA), long term evolution (LTE), LTE-Advanced (LTE-A), 802.1 lx, Wi-Fi, Zigbee, ultra-wideband (UWB), 802.16x, 802.15, home node-B (HnB), Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), near-field communication (NFC), fifth generation (5G), new radio (NR), any combination thereof, and / or any other currently existing or future implemented communication standard and / or protocol, without departing from the scope of the application. In some embodiments, communication device 520 can include one or more antennas that are singular, arrayed, phased, switched, beamformed, beamsteered, combinations thereof, and / or any other antenna configuration, without departing from the scope of the application.

[0055] Processor(s) 510 are also coupled via bus 505 to display 525, such as a plasma display, a liquid crystal display (LCD), a light-emitting diode (LED) display, a field emission display (FED), an organic light-emitting diode (OLED) display, a flexible OLED display, a flexible substrate display, a projection display, a 4K display, a high-definition display, Display 525 can be configured as a touch (haptic) display, a three-dimensional (3D) touch display, a multi-input touch display, a multi-point touch display, etc. Using resistive, capacitive, surface acoustic wave (SAW) capacitive, infrared, optical imaging, dispersive signal technology, acoustic pulse recognition, frustrated total-internal reflection, etc. Any suitable display device and haptic I / O can be used without departing from the scope of the application.

[0056] A keyboard 530 and a cursor control device 535, such as a computer mouse, touchpad, and so on, are further coupled to bus 505 to enable a user to interface with computing system. However, in some embodiments, a physical keyboard and mouse can not be present and a user can interact with the device only through the display 525 and / or a touchpad (not shown). Any type and combination of input device can be used, depending on design choices. In some embodiments, no physical input devices and / or display are present. For example, a user can interact with computing system 500 remotely via another computing system in communication with computing system 500, or computing system 500 can operate autonomously.

[0057] Memory 515 stores software modules that provide functionality when executed by processor(s) 510. The modules include an operating system 540 for computing system 500. The modules further include a probabilistic model module 545 configured to perform all or part of the processes described herein or derivatives thereof. Computing system 500 can include one or more additional functionality modules 550 that include additional functionality.

[0058] Those skilled in the art will appreciate that a“system” can be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a cellular telephone, a tablet computing device, a quantum computing system, or any other suitable computing device or combination of devices without departing from the scope of the application. The above-described functionality is represented as being performed by a“system” and is not intended to limit the scope of the application in any way but rather is intended to provide one example of many embodiments of the application. Indeed, the methods, systems, and devices disclosed herein can be implemented in localized and distributed forms consistent with computing technology, including cloud computing systems.

[0059] It should be noted that some of the system features described in this specification have been presented as modules, in order to more particularly emphasize their implementation independence. For example, a module can be implemented as a hardware circuit comprising custom very-large-scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, and the like.

[0060] Modules can also be implemented in, at least partially, software for execution by various types of processors. An identified unit of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, a procedure, or a function. Nevertheless, the executables of an identified module need not be physically located together, but can comprise disparate instructions stored in different locations that, when joined logically together, comprise the module and achieve the stated purpose for the module. Further, modules can be stored on a computer-readable medium, which can be, for example, a hard disk drive, flash device, RAM, tape, and / or any other such non-transitory computer-readable medium, without departing from the scope of the application.

[0061] Indeed, a module of executable code can be a single instruction, or many instructions, and can even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data can be identified and illustrated herein within modules, and can be embodied in any suitable form and organized within any suitable type of data structure. The operational data can be collected as a single data set, or can be distributed over different locations including over different storage devices, and can exist, at least partially, merely as electronic signals on a system or network.

[0062] Figure 6 is a flowchart illustrating a process 600 for implementing a probabilistic model in a deterministic workflow of RPA, according to embodiments of the application. The process begins with generating a deterministic RPA workflow for an RPA robot at 610. An RPA robot configured to execute the deterministic RPA workflow is generated at 620. The RPA robot is then deployed at 630.

[0063] An ML model associated with a probabilistic activity is trained to replace a deterministic activity in the deterministic RPA workflow at 640. Data is collected from the RPA robots implementing the deterministic workflow, their associated computing systems, or both, at 650. For example, the data can include, but is not limited to, operations performed by the robots, user interactions with the computing systems (e.g., buttons pressed, mouse movements, applications used, etc.), processes run by the computing systems, corrections made by the user, etc. The data is then used to periodically retrain the ML model until a desired threshold of confidence is reached at 660. A probabilistic workflow including probabilistic activities in place of the deterministic activities is then generated at 670, and an RPA robot implementing the workflow is generated and deployed at 680.

[0064] Figure 7 is a flowchart illustrating a process 700 for a regulatory system, according to embodiments of the application. In some embodiments, the process 700 can be implemented viaFigure 8 The supervisory system 810 can be implemented in accordance with the processes described in FIG. 7. The process begins (or continues from) at 710 where the supervisory system monitors the ML model invoked by the probabilistic activity in the RPA workflow to ensure that the ML model is running correctly on the input side via the data drift detector using the metrics and on the output side using the concept drift detector. In some embodiments, the supervisory system can monitor multiple ML models. In certain embodiments, the concept drift detector determines whether the output falls within a predetermined range. Figure 6 When no errors are detected at 720, the monitoring at 710 continues. However, when the data drift detector, the concept drift detector, or both indicate an error, an alert is issued at 730 (e.g., a message is sent to the computing system or the robot that an error has been detected). The supervisory system then takes remedial action at 740. The remedial action can include, but is not limited to, disabling the RPA robot, bypassing the RPA robot, rolling back to a previous version of the ML model, etc.

[0065]

[0066] Figure 8 is an architectural diagram 800 illustrating the operation of a supervisory system 810 in accordance with an embodiment of the present application. The supervisory system includes a data drift detector and a concept drift detector 814 that monitor the input to the ML model 820 and the output from the ML model 820, respectively. An RPA robot 830 invokes the ML model 820. When the supervisory system 810 receives an error from the data drift detector 812 and / or the concept drift detector 814, the supervisory system 810 can disable or bypass the RPA robot 830, generate an alert, etc.

[0067] In accordance with an embodiment of the present application, Figure 6 and Figure 7 The process steps performed in FIG. 7 can be performed by a computer program, encoded for a processor(s) to perform at least part of the processes described in FIG. 7. The computer program can be embodied on a non-transitory computer readable medium. The computer readable medium can be, but is not limited to, a hard disk drive, a flash memory device, a RAM, a magnetic tape, and / or any other such medium or combination of media used to store data. The computer program can include encoded instructions for controlling a processor(s) of a computing system (e.g., the processor(s) 510 of the computing system 500 of FIG. 6) to implement all or part of the process steps described in FIG. 7. The encoded instructions can also be stored on a computer readable medium. Figure 6 Figure 7 The process steps performed in FIG. 7 can be performed by a computer program, encoded for a processor(s) to perform at least part of the processes described in FIG. 7. The computer program can be embodied on a non-transitory computer readable medium. The computer readable medium can be, but is not limited to, a hard disk drive, a flash memory device, a RAM, a magnetic tape, and / or any other such medium or combination of media used to store data. The computer program can include encoded instructions for controlling a processor(s) of a computing system (e.g., the processor(s) 510 of the computing system 500 of FIG. 6) to implement all or part of the process steps described in FIG. 7. The encoded instructions can also be stored on a computer readable medium. Figure 5 Figure 6 Figure 7 The process steps performed in FIG. 7 can be performed by a computer program, encoded for a processor(s) to perform at least part of the processes described in FIG. 7. The computer program can be embodied on a non-transitory computer readable medium. The computer readable medium can be, but is not limited to, a hard disk drive, a flash memory device, a RAM, a magnetic tape, and / or any other such medium or combination of media used to store data. The computer program can include encoded instructions for controlling a processor(s) of a computing system (e.g., the processor(s) 510 of the computing system 500 of FIG. 6) to implement all or part of the process steps described in FIG. 7. The encoded instructions can also be stored on a computer readable medium.

[0068] ​​​​The computer program can be implemented in hardware, software, or a hybrid implementation. The computer program can consist of modules that are in operative communication with one another, and which are designed to pass information or instructions to display. The computer program can be configured to run on a general purpose computer, an ASIC, or any other suitable device.

[0069] It is readily understood that the components of the various embodiments of the present application, as generally described and illustrated in the Figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the application, as represented in the Figures, is not intended to limit the scope of the application, but is merely representative of selected embodiments of the application.

[0070] The features, structures, or characteristics of the application described in this specification can be combined in any suitable manner in one or more embodiments. For example, reference throughout this specification to "certain embodiments", "some embodiments", or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, appearances of the phrases "in certain embodiments", "in some embodiments", "in other embodiments", or similar language throughout this specification do not necessarily all refer to the same group of embodiments, and the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0071] It should be noted that the features, advantages, or similar language referred to throughout this specification and that does not mean that all of the features and advantages that can be achieved by the application should be present in any single embodiment of the application. Rather, language referring to the features and advantages is understood to mean that a particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. Thus, discussions throughout this specification referring to features and advantages that can be present in one embodiment does not mean that those features and advantages are necessarily present in a same embodiment, and that the described features, advantages, or characteristics can be combined in any suitable manner in one or more embodiments.

[0072] Furthermore, the described features, advantages, and characteristics of the application can be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the application can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages can be recognized in certain embodiments that can not be present in all embodiments of the application.

[0073] Those of ordinary skill in the art will readily understand that the application described above can be practiced with different orders of steps and / or with different hardware elements than those disclosed. Accordingly, although the present application has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions could be made to the present application w ithout departing from the spirit and scope of the application. In order to determine the metes and bounds of the present application, therefore, reference should be made to the appended claims.

[0074] In an embodiment, the computer-implemented method includes generating a deterministic RPA workflow and training an ML model associated with probabilistic activities to replace deterministic activities in the deterministic RPA workflow. The computer-implemented method further includes collecting data from RPA robots implementing the deterministic workflow and its related computing systems and periodically retraining the ML model using the collected data until a confidence threshold is reached. Then, the computer-implemented method includes generating a probabilistic workflow including probabilistic activities in place of the deterministic activities.

Claims

1. A computer-implemented method for implementing probabilistic models in deterministic workflows of robotic process automation (RPA), comprising: generating, by an RPA designer application, a deterministic RPA workflow for an RPA robot, the RPA workflow comprising deterministic activities; generating, by the RPA designer application, an RPA robot configured to execute the deterministic RPA workflow; training a machine learning (ML) model associated with a probabilistic activity to replace the deterministic activities in the RPA workflow, wherein the ML model is trained with collected data; and replacing, by the RPA designer application, the deterministic activities with the probabilistic activity.

2. The computer-implemented method of claim 1, further comprising: collecting data from a plurality of deployed RPA robots executing the deterministic RPA workflow, a computing system on which the RPA robots run, or both.

3. The computer-implemented method of claim 2, further comprising: periodically retraining the ML model using the collected data.

4. The computer-implemented method of claim 3, wherein replacing the deterministic activities in the RPA workflow with the probabilistic activity occurs after the ML model reaches a confidence threshold.

5. The computer-implemented method of claim 4, further comprising: generating a plurality of RPA robots configured to execute the RPA workflow comprising the probabilistic activity; and deploying the plurality of RPA robots configured to execute the RPA workflow comprising the probabilistic activity.

6. The computer-implemented method of claim 5, further comprising: monitoring the ML model to ensure the ML model is running correctly on the input side via a data drift detector using a metric and on the output side using a concept drift detector.

7. The computer-implemented method of claim 6, wherein the concept drift detector determines whether an output falls within a predetermined range.

8. The computer-implemented method of claim 6, further comprising: raising an alert when the data drift detector, the concept drift detector, or both indicate an error.

9. The computer-implemented method of claim 6, wherein when the data drift detector, the concept drift detector, or both indicate an error, the method further comprises: disabling or bypassing an RPA robot of the plurality of RPA robots.

10. The computer-implemented method of claim 6, wherein when the data drift detector, the concept drift detector, or both indicate an error, the method further comprises: rolling back to a previous version of the ML model.

11. A computer-implemented method for implementing probabilistic models in deterministic workflows of robotic process automation (RPA), comprising: generating, by an RPA designer application, a plurality of RPA robots configured to execute a deterministic RPA workflow, the RPA workflow comprising deterministic activities; deploying, by the RPA designer application, a plurality of RPA robots configured to execute the deterministic RPA workflow; training a machine learning, ML, model associated with the probabilistic activity to replace the deterministic activity in the RPA workflow, wherein the ML model is trained with collected data; and replacing, by the RPA designer application, the deterministic activity with the probabilistic activity.

12. The computer-implemented method of claim 11, further comprising: collecting data from the RPA robots executing the deterministic RPA workflow from multiple deployments, from computing systems on which the RPA robots run, or both.

13. The computer-implemented method of claim 12, further comprising: periodically retraining the ML model using the collected data; wherein the deterministic activity in the RPA workflow is replaced with the probabilistic activity after the ML model reaches a confidence threshold.

14. The computer-implemented method of claim 13, further comprising: generating a plurality of RPA robots configured to execute the RPA workflow including the probabilistic activity; and deploying the plurality of RPA robots configured to execute the RPA workflow including the probabilistic activity.

15. The computer-implemented method of claim 14, further comprising: monitoring the ML model to ensure that the ML model is running correctly on the input side via a data drift detector using a metric and on the output side using a concept drift detector.

16. The computer-implemented method of claim 15, wherein when the data drift detector, the concept drift detector, or both indicate an error, the method further comprises: disabling, bypassing, or rolling back to a previous version of the ML model, an RPA robot of the plurality of RPA robots.

17. A computer-implemented method comprising: generating, by an RPA designer application, a robotic process automation, RPA, robot configured to execute an RPA workflow including a probabilistic activity; deploying the RPA robot; and monitoring, by a governance system, a machine learning, ML, model called by the probabilistic activity in the RPA workflow, the governance system configured to ensure that the ML model is running correctly on the input side via a data drift detector using a metric and on the output side using a concept drift detector.

18. The computer-implemented method of claim 17, wherein the concept drift detector determines whether an output falls within a predetermined range.

19. The computer-implemented method of claim 17, further comprising: alerting, by the governance system, when the data drift detector, the concept drift detector, or both indicate an error.

20. The computer-implemented method of claim 17, wherein when the data drift detector, the concept drift detector, or both indicate an error, the method further comprises: disabling, bypassing, or rolling back to a previous version of the ML model by the governing system for an RPA robot of a plurality of RPA robots.

Images