Reconfigurable Workbench Pipeline for Robotic Process Automation Workflow

Abstract

A reconfigurable workbench pipeline for robotic process automation (RPA) workflows is disclosed. Different workbench pipelines can be built for different users. For example, a global workflow (e.g., a receipt extractor) can be initially built and used, but this workflow may not work optimally or at all for a particular user or a particular task. Machine learning (ML) models can be employed, potentially with a human-in-the-loop, to specialize the global workflow for a given task.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Non - Provisional Patent Application No. 16 / 707,977, filed on December 9, 2019, and U.S. Provisional Patent Application No. 62 / 915,413, filed on October 15, 2019. The subject matter of these prior - filed applications is hereby incorporated by reference in its entirety.

[0002] The present invention generally relates to robotic process automation (RPA), and more specifically, to a reconfigurable workbench pipeline for RPA workflows.

Background Art

[0003] Current RPA workflows are essentially deterministic and include a static pipeline of activities. In other words, a single logic flow is created and applied by a robot when performing the workflow. The "pipeline" refers to a series of steps for extracting data and / or specific actions taken based on the extracted data. However, such static pipelines may not be optimal for all situations, especially those that change over time or are targeted at specific users. Therefore, an improved solution can be beneficial.

Summary of the Invention

[0004] Certain embodiments of the present invention may provide solutions to problems and needs in the art that have not yet been fully identified, recognized, or solved by current RPA technologies. For example, some embodiments of the present invention relate to a reconfigurable workbench pipeline for RPA workflows.

[0005] In one embodiment, a computer-implemented method for providing a reconfigurable workbench pipeline for an RPA workflow includes performing a global workflow by an RPA robot. The computer-implemented method also includes determining that the pipeline of the global workflow is not working correctly for a scenario, and employing AI-driven testing to identify one or more ML models or other components that are applied to the pipeline and / or improved in the pipeline of the global workflow, to repair or specialize the global workflow with respect to the scenario. The computer-implemented method further includes implementing the identified ML models and / or other components in the pipeline of the local workflow.

[0006] In another embodiment, a computer-implemented method for providing a reconfigurable workbench pipeline for an RPA workflow includes determining that the pipeline of the global workflow is not functioning correctly for a scenario, and employing AI-driven testing to identify one or more ML models or other components that are applied to the pipeline and / or improved in the pipeline of the global workflow, to repair or specialize the global workflow with respect to the scenario. The computer-implemented method also includes implementing the identified ML models and / or other components in the pipeline of the local workflow.

[0007] In yet another embodiment, a computer-implemented method for providing a reconfigurable workbench pipeline for an RPA workflow employs AI-driven testing to identify one or more ML models or other components that are applied to the pipeline and / or improved in the pipeline of the global workflow, to repair or specialize the global workflow with respect to a scenario, and to implement the identified ML models and / or other components in the pipeline of the local workflow. The AI-driven testing includes checking the output of a first component of the pipeline and, if the output of the first component is incorrect, reconfiguring or replacing the first component. The AI-driven testing also includes checking the output of each subsequent component in the pipeline and reconfiguring or replacing each component that is operating incorrectly until the output of the pipeline is correct or the output of all components has been checked.

Brief Description of the Drawings

[0008] To facilitate an understanding of the advantages of specific embodiments of the present invention, a specific description of the present invention briefly described above is provided by reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings show only typical embodiments of the present invention and thus should not be considered as limiting its scope. The present invention will be described and explained with additional specificity and detail by using the accompanying drawings.

[0009]

Figure 1

[0010]

Figure 2

[0011]

Figure 3

[0012]

Figure 4

[0013]

Figure 5

[0014]

Figure 6

[0015]

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0016] Some embodiments relate to a reconfigurable workbench pipeline for an RPA workflow. In some embodiments, the reconfigurable workbench pipeline may apply a machine learning (ML) model that modifies or improves the operation of the pipeline. In certain embodiments, this may be a rule-based operation rather than an ML model. In some embodiments, the system may have control to propose models and / or transformation components to meet a given use case.

[0017] For example, consider the case of receiving an email that may have various attached files (e.g., claim documents, purchase order documents, etc.). Before sending the claim document flagged for claim processing to the ML model, it may be desirable to have components within a pipeline that operates as a gate function to identify and flag the claim document within the email. However, the pipeline may not operate correctly in all cases. Therefore, when executing the claim document through the pipeline, the system may monitor whether there is an output at the end of the pipeline. If there is no output, the system may apply a series of transformations to different pipeline stages to improve the pipeline. For example, the system may start from the first stage and check each subsequent stage to find the cause of the failure. The system may check components 1, 2, 3, etc. of the pipeline. If the pipeline cannot properly identify the document within the email, the system may attempt to apply different instances and / or configurations of the pipeline components in an attempt to find a solution. If no solution is found, the system may notify the RPA developer.

[0018] The reconfigurable pipelines of some embodiments may be applied to document processing, other data transformation pipelines, data extraction pipelines, etc. The reconfigurable pipeline may include components of a specialized model for human input and / or classification, extraction, etc. For example, a global workflow (e.g., invoice extractor) may be initially constructed and used, but this workflow may not work optimally or at all for a particular user or a particular task. The ML model may potentially be employed with a human-in-the-loop to specialize the global workflow for a given task. In some embodiments, as more human input is collected and used for retraining, the human input may be tracked and used to retrain the ML model so that the ML model becomes more intelligent.

[0019] In the context of document processing and other complex artificial intelligence (AI) applications, users tend to have different data, use cases, systems with which the user interacts, and outcomes. For example, both Walmart® and a small "mom and pop" retailer can benefit from an invoice processing solution, but their requirements are significantly different. An invoice processing solution for a small retailer may include retrieving invoices from incoming emails, performing text recognition on the invoices using optical character recognition (OCR) and / or image recognition (such as using computer vision (CV) techniques described in detail below), retrieving invoice information from the data recognized by the OCR / CV, and entering the retrieved invoice information into QuickBooks®.

[0020] However, an invoice processing solution for Walmart® can be substantially more complex. Such large retailers handle a vast number of different types of invoices that need to go to different systems and have additional items beyond receipts. For example, multiple document types can be included and it may be necessary to use classifiers to pick up the correct documents. Thus, large retailers may want the classifier to identify the types of invoices within this data and route the invoices to the correct downstream workflows based on those types. Such large retailers may also likely have incoming invoices written in different languages, so for example, a language processor may be beneficial to convert these invoices to English.

[0021] With the workbench pipeline of some embodiments, the "blocks" of the model or other logic can be applied to the workflow to customize and tailor the workflow from global requirements to local applications. These blocks and / or logic are stages (or components) of the pipeline. First, in some embodiments, the RPA developer creates an initial workflow as the "global" workflow of a particular application with built-in intelligence (e.g., via calls to an ML model). The ML model monitors the behavior of the running robot and can further automatically determine whether a particular activity should be included in the workflow, replaced with another activity, or improved. For example, if after the robot processes an invoice, the monitoring ML model determines that the user frequently accesses QuickBooks® to modify specific fields, the ML model may search for a different text recognition model or a CV model (e.g., where the text is standardized and placed within the company logo along with standardized words and images) and improve the text recognition activity to use this model instead.

[0022] In some embodiments, a base ML model may be used for the global workflow, and custom models may be added later. In this sense, the workflow may have a plug-and-play feature that allows for the addition, deletion, and / or modification of blocks after the initial implementation. For example, the global model may be an invoice data extraction model, but a Spanish translation model may be added as a local model to analyze Spanish invoices.

[0023] Figure 1 is an architecture diagram showing an RPA system 100 according to an embodiment of the present invention. The RPA system 100 includes a designer 110 that can enable a developer to design and implement a workflow. The designer 110 can provide solutions for application integration and automate third-party applications, management information technology (IT) tasks, and business IT processes. The designer 110 can facilitate the development of an automation project that is a graphical representation of a business process. Briefly speaking, the designer 110 facilitates the development and deployment of workflows and robots.

[0024] The automation project enables the automation of a rule-based process by giving the developer control over the execution order and relationships between a custom set of steps developed in the workflow, which are defined herein as "activities". One commercial example of an embodiment of the designer 110 is UiPath Studio (trademark). Each activity may include actions such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, the workflows may be nested or embedded.

[0025] Some types of workflows may include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. A sequence may be particularly suitable for a linear process that enables the flow from one activity to another without disrupting the workflow. A flowchart may be particularly suitable for complex business logic and enables the integration of decisions and the connection of activities in various ways through multiple branching logical operators. An FSM may be particularly suitable for large-scale workflows. An FSM may use a finite number of states in the execution of a workflow triggered by conditions (i.e., transitions) or activities. A global exception handler may be particularly suitable for determining the behavior of a workflow when an execution error is encountered and for debugging the process.

[0026] When a workflow is developed in the designer 110, the execution of the business process is orchestrated by the conductor 120, which orchestrates one or more robots 130 that execute the workflow developed in the designer 110. One commercially available example of an embodiment of the conductor 120 is UiPath Orchestrator (trademark). The conductor 120 facilitates the creation, monitoring, and management of resources within the environment. The conductor 120 may act as an integration point with third-party solutions and applications.

[0027] The conductor 120 can manage all the robots 130, connect the robots 130 from a central point, and execute. The types of robots 130 that can be managed include, but are not limited to, attended robots 132, unattended robots 134, development robots (similar to unattended robots 134 but used for development and testing purposes), and non-production robots (similar to attended robots 132 but used for development and testing purposes). The attended robot 132 is triggered by user events and operates together with humans on the same computing system. The attended robot 132 can be used with the conductor 120 for centralized process deployment and logging media. The attended robot 132 helps with various tasks achieved by human users and can be triggered by user events. In some embodiments, the process cannot be started from the conductor 120 of this type of robot and / or cannot be executed under a locked screen. In certain embodiments, the attended robot 132 can only be launched from a robot tray or command prompt. In some embodiments, the attended robot 132 should be executed under human supervision.

[0028] The unattended robot 134 can execute without a human operator in a virtual environment and automate many processes. The unattended robot 134 can be responsible for providing support for remote execution, monitoring, scheduling, and work queues. In some embodiments, debugging of all robot types may be performed by the designer 110. Both attended and unattended robots can automate various systems and applications including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those made by SAP®, SalesForce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0029] The conductor 120 can have various functions including, but not limited to, providing provisioning, deployment, configuration, queuing, monitoring, logging, and / or interconnectivity. Provisioning can include creating and maintaining a connection between the robot 130 and the conductor 120 (e.g., a web application). Deployment can include ensuring the correct delivery of the package version to the assigned robot 130 for execution. Configuration can include maintaining and delivering the robot environment and process configuration. Queuing can include providing management of queues and queue items. Monitoring can include continuously tracking robot identification data and maintaining user permissions. Logging can include storing and indexing logs in a database (e.g., an SQL database) and / or another storage mechanism (e.g., ElasticSearch® which provides the ability to store large datasets and query them quickly). The conductor 120 can provide interconnectivity by acting as a central point of communication for third-party solutions and / or applications.

[0030] Robot 130 is an execution agent that executes the workflow constructed by Designer 110. One commercial example of some embodiments of Robot 130 is UiPath Robots (trademark). In some embodiments, Robot 130, by default, installs the Microsoft Windows (registered trademark) Service Control Manager (SCM) management service. As a result, such a Robot 130 can open an interactive Windows (registered trademark) session under the local system account and have the rights of a Windows (registered trademark) service.

[0031] In some embodiments, Robot 130 may be installed in user mode. In the case of such a Robot 130, this means that it has the same rights as the user on whose machine a given Robot 130 is installed. This feature may also be available for high-density (HD) robots that guarantee full utilization of each machine to its maximum potential. In some embodiments, any type of Robot 130 may be configured in an HD environment.

[0032] Robot 130 in some embodiments is divided into several components, each of which is dedicated to a specific automation task. Robot components in some embodiments include, but are not limited to, SCM management robot service, user-mode robot service, executor, agent, and command line. The SCM management robot service manages and monitors the Windows (registered trademark) session and acts as a proxy between the conductor 120 and the execution host (i.e., the computing system on which Robot 130 is executed). These services are trusted with the credential information of Robot 130 and manage the credential information. The console application is launched by the SCM under the local system.

[0033] In some embodiments, the user-mode robot service manages and monitors the Windows® session and acts as a proxy between the conductor 120 and the execution host. The user-mode robot service may be trusted with and manage the credentials for the robot 130. If the SCM management robot service is not installed, the Windows® application may be automatically started.

[0034] The executor may execute a given job under the Windows® session (i.e., the executor may perform a workflow. The executor may recognize the per-monitor dots per inch (DPI) setting. The agent may be a Windows® Presentation Foundation (WPF) application that displays jobs available in the system tray window. The agent may be a client of the service. The agent may request the start or stop of a job and the change of settings. The command line is a client of the service. The command line is a console application, and the console application may request the start of a job and wait for its output.

[0035] As described above, splitting the components of the robot 130 facilitates easy execution, identification, and tracking by developers, support users, and computing systems as to what each component is performing. In this way, special behaviors can be configured for each component, such as setting different firewall rules for executors and services. In some embodiments, the executor can always recognize the DPI settings for each monitor. As a result, workflows can be performed at any DPI, regardless of the configuration of the computing system in which they were created. In some embodiments, projects from the designer 110 may be independent of the browser's zoom level. In the case of applications marked as not recognizing DPI or intentionally not recognizing DPI, in some embodiments, DPI can be disabled.

[0036] Figure 2 is an architectural diagram showing a deployed RPA system 200 according to an embodiment of the present invention. In some embodiments, the RPA system 200 may be the RPA system 100 of FIG. 1 or a part thereof. Note that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robot application 210 includes an executor 212, an agent 214, and a designer 216. However, in some embodiments, the designer 216 may not be executed on the computing system 210. The executor 212 is a running process. As shown in FIG. 2, several business projects may be executed simultaneously. The agent 214 (e.g., Windows® service) is a single connection point for all executors 212 in this embodiment. All messages in this embodiment are logged in the conductor 230, and the conductor 230 further processes the logged ones via the database server 240, the indexer server 250, or both. As described above with respect to FIG. 1, the executor 212 may be a robot component.

[0037] In some embodiments, the robot represents an association between a machine name and a user name. The robot may manage multiple executors simultaneously. In a computing system that supports multiple interactive sessions (e.g., Windows® Server 2012) running simultaneously, multiple robots may be executed simultaneously, each running in a separate Windows® session using a unique user name. This is referred to above as an HD robot.

[0038] Agent 214 also sends the status of the robot (e.g., periodically sends a "heartbeat" message indicating that the robot is still functioning) and is also responsible for downloading the required version of the package to be executed. The communication between Agent 214 and Conductor 230 is, in some embodiments, always initiated by Agent 214. In a notification scenario, Agent 214 may open a WebSocket channel that is later used by Conductor 230 to send commands (e.g., start, stop, etc.) to the robot.

[0039] On the server side, there are a presentation layer (web application 232, Open Data Protocol (OData) Representational State Transfer (REST) Application Programming Interface (API) endpoint 234, and notification monitoring 236), a service layer (API implementation / business logic 238), and a persistence layer (database server 240, indexer server 250). The conductor 230 includes the web application 232, the OData REST API endpoint 234, the notification monitoring 236, and the API implementation / business logic 238. In some embodiments, most of the actions that a user performs within the interface of the conductor 220 (e.g., via the browser 220) are performed by calling various APIs. Such actions may include, but are not limited to, starting a job on a robot, adding / removing data in a queue, scheduling a job for unattended execution, etc., without departing from the scope of the present invention. The web application 232 is the visual layer of the server platform. In this embodiment, the web application 232 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, script language, or any other format may be used without departing from the scope of the present invention. The user interacts with the web page from the web application 232 via the browser 220 in this embodiment to perform various actions to control the conductor 230. For example, the user may create a robot group, assign a package to a robot, analyze logs for each robot and / or for each process, start, and stop the robot, etc.

[0040] In addition to the web application 232, the conductor 230 also includes a service layer that exposes an OData REST API endpoint 234. However, other endpoints may be included without departing from the scope of the present invention. The REST API is consumed by both the web application 232 and the agent 214. The agent 214 is, in this embodiment, an administrator of one or more robots on a client computer.

[0041] The REST API in this embodiment covers configuration, logging, monitoring, and queuing functions. Configuration endpoints may be used, in some embodiments, to define and configure application users, permissions, robots, assets, releases, and environments. For example, REST endpoints may be used to log various information such as errors, explicit messages sent by robots, and other environment-specific information. Deployment REST endpoints may be used by robots to query the package version to be executed when a start job command is used within the conductor 230. Queuing REST endpoints may be responsible for queue and queue item management, such as adding data to a queue, retrieving transactions from a queue, and setting the status of a transaction.

[0042] Monitoring of the REST endpoints enables monitoring of the web application 232 and the agent 214. The notification monitoring API 236 may be a REST endpoint used for registration of the agent 214, delivery of configuration settings to the agent 214, and sending / receiving notifications from the server and the agent 214. The notification monitoring API 236 may also use WebSocket communication in some embodiments.

[0043] The persistent layer includes the pair of servers in this embodiment, namely the database server 240 (for example, an SQL server) and the indexer server 250. The database server 240 in this embodiment stores configurations such as robots, robot groups, related processes, users, roles, schedules, etc. In some embodiments, this information is managed via the web application 232. The database server 240 may manage queues and queue items. In some embodiments, the database server 240 may store (in addition to, or instead of, the indexer server 250) messages logged by robots.

[0044] The indexer server 250 is optional in some embodiments and stores and indexes information logged by robots. In certain embodiments, the indexer server 250 may be disabled via configuration settings. In some embodiments, the indexer server 250 uses ElasticSearch (registered trademark), a full-text search engine for open-source projects. Messages logged by robots (using activities such as log messages or row writes) may be sent to the indexer server 250 via the logging REST endpoint, where they are indexed for future use.

[0045] Figure 3 is an architecture diagram showing the relationship 300 between a designer 310, activities 320, 330, and a driver 340 according to an embodiment of the present invention. As described above, a developer uses the designer 310 to develop a workflow to be performed by a robot. The workflow may include user-defined activities 320 and UI automation activities 330. Some embodiments may identify non-text visual components in an image, which is referred to herein as computer vision (CV). Some CV activities related to such components may include, but are not limited to, click, type, get text, hover, element presence, refresh range, highlight, etc. In some embodiments, click may identify an element using, for example, CV, optical character recognition (OCR), fuzzy character matching, and multi-anchor, and then click on it. Type may identify an element using the above and types within the element. Getting text may identify the position of specific text using OCR and then scan it. Hover may identify an element and then hover over it. Element presence may check whether an element exists on the screen using the techniques described above. In some embodiments, the number of activities that may be implemented in the designer 310 may be hundreds or thousands. However, any number and / or type of activities are available without departing from the scope of the present invention.

[0046] The UI automation activity 330 is a subset of special low-level activities, which are written in low-level code (e.g., CV activities) to facilitate interaction with the screen. The UI automation activity 330 facilitates these interactions via a driver 340 through which the robot can interact with the desired software. For example, the driver 340 may include an OS driver 342, a browser driver 344, a VM driver 346, an enterprise application driver 348, etc.

[0047] Driver 340 can interact with the OS at a low level, such as searching for hooks and monitoring keys. They can facilitate integration with Chrome (registered trademark), IE (registered trademark), Citrix (registered trademark), SAP (registered trademark), etc. For example, the "click" activity performs the same role in these different applications via Driver 340.

[0048] FIG. 4 is an architectural diagram showing an RPA system 400 according to an embodiment of the present invention. In some embodiments, the RPA system 400 can be or include the RPA systems 100 and / or 200 of FIGS. 1 and / or 2. The RPA system 400 includes a plurality of client computing systems 410 that execute robots. The computing system 410 can communicate with a conductor computing system 420 via a web application executed thereon. Next, the conductor computing system 420 can communicate with a database server 430 and an optional indexer server 440.

[0049] Regarding FIGS. 1 and 3, although web applications are used in these embodiments, it should be noted that any suitable client / server software can be used without departing from the scope of the present invention. For example, the conductor may execute a server-side application that communicates with a non-web-based client software application on the client computing system.

[0050] FIG. 5 is an architectural diagram showing a computing system 500 configured to implement a reconfigurable workbench pipeline for an RPA workflow according to one embodiment of the present invention. In some embodiments, the computing system 500 may be one or more of the computing systems illustrated and / or described herein. The computing system 500 includes a bus 505 or other communication mechanism for communicating information, and a processor 510 coupled to the bus 505 for processing information. The processor 510 may be any type of general-purpose or dedicated processor, including a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. The processor 510 may also have multiple processing cores, and at least some of the cores may be configured to perform a specific function. In some embodiments, multiple parallel processing may be used. In certain embodiments, at least one of the processors 510 may be a neuromorphic circuit including processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit may not require the typical components of a von Neumann computing architecture.

[0051] The computing system 500 further includes a memory 515 for storing information and instructions executed by the processor 510. The memory 515 may be composed of any combination of random access memory (RAM), read-only memory (ROM), flash memory, cache, magnetic or optical disk such as static storage, or any other type of non-transitory computer-readable medium, or a combination thereof. The non-transitory computer-readable medium may be any available medium that can be accessed by the processor 510, and may include volatile media, non-volatile media, or both. The medium may also be removable, non-removable, or both.

[0052] Furthermore, computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communication network via a wireless and / or wired connection. In some embodiments, communication device 520 is configured to use frequency division multiple access (FDMA), single carrier FDMA (SC-FDMA), time division multiple access (TDMA), code division multiple access (CDMA), orthogonal frequency division multiplexing (OFDM), orthogonal frequency division multiple access (OFDMA), global system for mobile communications (GSM) for mobile, general packet radio service (GPRS), universal mobile telecommunications system (UMTS), cdma2000, wideband CDMA (W-CDMA), high speed downlink packet access (HSDPA), high speed uplink packet access (HSUPA), high speed packet access (HSPA), long term evolution (LTE), LTE-Advanced (LTE-A), 802.11x, Wi-Fi, ZigBee, ultra-wideband radio (UWB), 802.16x, 802.15, home node B (HnB), Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), near field communication (NFC), fifth generation (5G), New Radio (NR), any combination thereof, and / or any other currently existing or future implemented communication standard and / or protocol that does not depart from the scope of the present invention. In some embodiments, communication device 520 may include one or more antennas, which may be single, array, phase, switching, beamforming, beam steering, combinations thereof, and / or any other antenna configuration without departing from the scope of the present invention.

[0053] Processor 510 is further coupled to a display 525 via bus 505, which can be a plasma display, a liquid crystal display (LCD), a light emitting diode (LED) display, a field emission display (FED), an organic light emitting diode (OLED) display, a flexible OLED display, a flexible substrate display, a projection display, a 4K display, a high definition display, a Retina (registered trademark) display, an in-plane switching (IPS) display, or any other suitable display for presenting information to a user. Display 525 can be configured as a touch (haptic) display, a three-dimensional (3D) touch display, a multi-input touch display, a multi-touch display, etc., using resistive, capacitive, surface acoustic wave (SAW) capacitive, infrared, optical imaging, dispersive signal technology, acoustic pulse recognition, frustrated total internal reflection, etc. Without departing from the scope of the present invention, any suitable display device and haptic I / O can be used.

[0054] Keyboard 530 and a cursor control device 535, such as a computer mouse, a touchpad, etc., are further coupled to bus 505 to enable a user to interface with the computing system. However, in certain embodiments, a physical keyboard and mouse may not be present, and the user may interact with the device only via display 525 and / or a touchpad (not shown). Any type and combination of input devices can be used as a design choice. In certain embodiments, there is no physical input device and / or display. For example, a user may interact remotely with computing system 500 via another computing system that communicates with computing system 500, or computing system 500 may operate autonomously.

[0055] Memory 515 stores software modules that provide functionality when executed by processor 510. The modules include an operating system 540 for computing system 500. The modules further include a workbench benchmark pipeline module 545 configured to implement all or part of the processes described herein or derivatives thereof. Computing system 500 may include one or more additional functional modules 550 that include additional functionality.

[0056] One of ordinary skill in the art will understand that the "system" may be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a cellular phone, a tablet computing device, a quantum computing system, or any other suitable computing device, or a combination of devices, without departing from the scope of the present invention. Presenting the above functions as being performed by a "system" is in no way intended to limit the scope of the present invention, but rather is intended to provide an example of many embodiments of the present invention. In fact, the methods, systems, and devices disclosed herein may be implemented in localized and distributed forms consistent with computing technologies, including cloud computing systems.

[0057] Note that some of the system features described herein are presented as modules to more specifically emphasize their implementation independence. For example, a module may be implemented as a hardware circuit including custom very large scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, and the like.

[0058] The module may also be implemented at least in part by software for execution by various types of processors. The identified units of executable code may include, for example, one or more physical or logical blocks of computer instructions that may be organized as objects, routines, or functions. Nevertheless, the executable portions of the identified modules need not be physically co-located, but when logically coupled together, may be stored in different locations, include the modules, and include different instructions for achieving the stated purposes of the modules. Further, the module may be stored on a computer-readable medium that may be, for example, a hard disk drive, a flash device, a RAM, a tape, and / or any other such non-transitory computer-readable medium used for storing data without departing from the scope of the present invention.

[0059] In fact, the executable code of the module may be a single instruction or many instructions and may be distributed across several different code segments, in different programs, and between several memory devices. Similarly, the operating data is identified and illustrated herein within the module and may be embodied in any suitable form and organized within any suitable type of data structure. The operating data may be collected as a single data set or may be distributed across different locations including different storage devices and may exist only at least in part as electronic signals on a system or network.

[0060] Figure 6 shows a reconfigurable workbench pipeline 600 that is not generating (or is not generating correct) output, and operations for fixing its performance. In an initial state A, four components 1, 2, 3, and 4 are chained to produce output. However, as indicated by the "X" on the right, the output from component 2 is either not producing output or is producing incorrect output. The workbench pipeline first checks the inputs and outputs of component 1 in state B. These are determined to be correct, and thus the workbench pipeline proceeds to check the output of component 2 in state C. The output of component 2 is determined to be in error for this use case.

[0061] Next, the system attempts to find a solution to the problem by testing a new component (e.g., an ML model, conditional logic, etc.). In this example, the system determines that the insertion of component 5 provides the correct output for the pipeline and replaces component 2 with component 5 in step D. This improves the pipeline to produce the correct output, thus "fixing" the pipeline and making it reconfigurable.

[0062] Figure 7 is a flowchart showing a process 700 for implementing a reconfigurable workbench pipeline for an RPA workflow according to an embodiment of the present invention. The process begins at 710 with the execution of a global workflow by an RPA robot. Next, at 720, it is determined (e.g., by an RPA robot, another software application, a computing system, etc.) that the pipeline of the global workflow is not working correctly for a scenario. At 730, AI-driven tests are employed to identify one or more ML models or other components that are applied to the pipeline and / or improved in the pipeline of the global workflow to repair or specialize the global workflow with respect to the scenario.

[0063] In some embodiments, the identification of the model or other components applied to the pipeline may be similar to the process described above with respect to FIG. 6. For example, if the pipeline is not generating an output or is not generating the correct output, the inputs and outputs of the first component may be checked. If one or both of the inputs and outputs are incorrect, corrective actions may be taken (e.g., by analyzing the data source in the case of an input error or by reconfiguring or replacing the first component in the case of an output error). Analyzing the data source may include checking the logic in the software, system, robot, etc. that generates the input. For example, one or more SQL commands in a database operation may be incorrect. Reconfiguring the first component may include improving the parameters of the component, changing its logic, having the component call a different ML model, any combination thereof, etc. If the inputs and outputs of the first component are correct, the output of the second component is checked, then the third component is checked, and so on until all components that are not working properly within the pipeline are identified and addressed.

[0064] In some embodiments, at 730, if one or more ML models and / or other components cannot be determined for the pipeline, then at 732, the system notifies the RPA developer that the pipeline is not functioning correctly (e.g., by sending a communication including a workflow, pipeline components that are not functioning properly, which replacement components and / or whether improvements have been attempted on existing components, or any combination thereof), and at 734, receives guidance from the RPA developer. In some embodiments, the guidance from the RPA developer may include a repaired version of the pipeline, a local workflow, new pipeline components, any combination thereof, and the like. Next, at 740, the system implements the identified ML model and / or other components in the pipeline of the local workflow. Then, at 750, a new version of the RPA robot is generated to execute the local workflow. At 760, the new version of the RPA robot is deployed, and at 770, the local workflow is executed by the new version of the RPA robot.

[0065] The process steps implemented in FIG. 7 may be implemented by a computer program encoding instructions for a processor to implement at least a portion of the process described in FIG. 7, according to an embodiment of the present invention. The computer program may be embodied on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, RAM, a tape, and / or any other such medium or combination of media used to store data. The computer program may include encoded instructions for controlling a processor of a computing system (e.g., processor 510 of computing system 500 in FIG. 5) to implement all or a portion of the process steps described in FIG. 7, which may be stored on the computer-readable medium.

[0066] A computer program may be implemented in hardware, software, or a hybrid implementation. The computer program may be composed of modules designed to communicate operably with each other and pass information or instructions for display. The computer program may be configured to operate on a general-purpose computer, an ASIC, or any other suitable device.

[0067] It will be readily understood that the components of the various embodiments of the present invention may be arranged and designed in a wide variety of different configurations, as generally described and illustrated in the figures of this specification. Accordingly, the detailed description of the embodiments of the present invention, as represented in the accompanying drawings, is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the present invention.

[0068] The features, structures, or characteristics of the present invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, references throughout this specification to "certain embodiments," "some embodiments," or similar language mean that the particular features, structures, or characteristics described in connection with the embodiments are included in at least one embodiment of the present invention. Thus, appearances throughout this specification of "in certain embodiments," "in some embodiments," "in other embodiments," or similar language are not necessarily all referring to the same group of embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0069] Note that throughout this specification, references to features, advantages, or similar language do not imply that all features and advantages realizable by the present invention should be, or are, in any single embodiment of the present invention. Rather, the language referring to the features and advantages is to be understood as meaning that a particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Accordingly, throughout this specification, the descriptions of features and advantages, and similar language, may, but do not necessarily, refer to the same embodiment.

[0070] Furthermore, the described features, advantages, and characteristics of the present invention may be combined in any suitable manner in one or more embodiments. One skilled in the art will recognize that the present invention may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in particular embodiments that may not be present in all embodiments of the present invention.

[0071] One skilled in the art will readily understand that the present invention as described above may be practiced with steps in a different order and / or with hardware elements in a configuration different from that disclosed. Accordingly, while the present invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain improvements, modifications, and alternative constructions are apparent while remaining within the spirit and scope of the present invention. Accordingly, reference should be made to the appended claims to determine the scope of the present invention.

Claims

1. A computer-implemented method performed by a computing system for providing a reconfigurable workbench pipeline for a robotic process automation (RPA) workflow using machine learning (ML), the method comprising: performing, by an RPA robot executing on the computing system, a global workflow, wherein the global workflow is an initial workflow of tasks, and the global workflow includes a pipeline including a plurality of ML models; determining, by the RPA robot of the computing system, another RPA robot, or another software application, that at least one ML model of the plurality of ML models of the pipeline of the global workflow is not generating a correct output for an input to the at least one ML model by sequentially checking an input including an input query and an output including a result associated with the input query for each ML model of the plurality of ML models of the pipeline of the global workflow; exchanging, by the computing system, the at least one ML model that is not generating the correct output with at least one new ML model within the pipeline; generating, by the computing system, a local pipeline of a local workflow of the computing system; performing, by the RPA robot, the local workflow, the method further comprising: wherein the generated local pipeline includes the at least one new ML model that has been exchanged, a computer-implemented method.

2. notifying an RPA developer that the pipeline is not functioning correctly; receiving, from the RPA developer, guidance on how to correct the pipeline, the computer-implemented method according to claim 1, further comprising:

3. The computer-implemented method according to claim 2, comprising transmitting, to the RPA developer, a communication in which the notification includes the local workflow, log data indicating one or more pipeline ML models that are not functioning properly, which exchange ML model and / or whether an improvement to an existing ML model has been attempted, or any combination thereof.

4. The computer-implemented method according to claim 2, wherein the guidance from the RPA developer includes a repaired version of the pipeline, the local workflow, one or more new pipeline ML models, or any combination thereof.

5. Analyzing an input of a first ML model of the pipeline; The computer-implemented method according to claim 1, further comprising analyzing a data source of the input of the first ML model if the input of the first ML model is incorrect.

6. A computer-implemented method performed by a computing system for providing a reconfigurable workbench pipeline for a robotic process automation (RPA) workflow using machine learning (ML), the method comprising: Determining, by an RPA robot of the computing system, another RPA robot, or another software application, that at least one ML model of a plurality of ML models of a pipeline of a global workflow is not generating a correct output for an input to the at least one ML model by sequentially checking an input including an input query and an output including a result associated with the input query for each ML model of the plurality of ML models of the pipeline of the global workflow; Replacing, by the computing system, the at least one ML model that is not generating the correct output with at least one new ML model within the pipeline; Generating, by the computing system, a local pipeline of a local workflow of the computing system; Performing, by the RPA robot, the local workflow, wherein the generated local pipeline includes the at least one new ML model that has been replaced.

7. Notifying an RPA developer that the pipeline is not functioning correctly, The notification includes transmitting to the RPA developer a communication including the local workflow, one or more pipeline ML models that are not functioning properly, which exchange ML models and / or log data indicating whether an improvement to an existing ML model has been attempted, or any combination thereof, the computer-implemented method according to claim 6, further comprising notifying.

8. Receiving guidance from the RPA developer on how to modify the pipeline, The computer-implemented method according to claim 7, further comprising receiving the guidance from the RPA developer, the guidance including a repaired version of the pipeline, the local workflow, one or more new pipeline ML models, or any combination thereof.

9. Analyzing the input of the first ML model, When the input of the first ML model is incorrect, further analyzing the data source of the input of the first ML model, the computer-implemented method according to claim 6.

10. A computer-implemented method performed by a computing system for providing a reconfigurable workbench pipeline for a robotic process automation (RPA) workflow using machine learning (ML), By an RPA robot of the computing system, another RPA robot, or another software application, at least one of a plurality of ML models of a pipeline of a global workflow checks in sequence an input including an input query and an output including a result associated with the input query of each ML model of the plurality of ML models of the pipeline of the global workflow, and determines that a correct output is not being generated for the input to the at least one ML model; Exchanging, by the computing system, the at least one ML model for which the correct output is not being generated with at least one new ML model within the pipeline of the global workflow; generating, by the computing system, a local pipeline of a local workflow of the computing system; performing, by an RPA robot, the local workflow; and the generated local pipeline includes the at least one newly exchanged ML model, a computer-implemented method. **Claim 11** notifying an RPA developer that the pipeline of the global workflow is not functioning correctly, wherein the notification includes transmitting, to the RPA developer, a communication including the local workflow, one or more pipeline ML models that are not functioning properly, log data indicating which of the exchange ML models and / or whether an improvement to an existing ML model has been attempted, or any combination thereof, the computer-implemented method according to claim 10.

Images