Artificial intelligence / machine learning model training and recommendation engine for robotic process automation

Abstract

To provide a computing system, method and medium for realizing an artificial intelligence (AI) / machine learning (ML) recommendation engine for robotic process automation (RPA).SOLUTION: A designer 410 trains an AI / ML model 480 to provide recommendations for a next activity, a next sequence of activities, and / or modifications of parameters for one or more existing activities 420 to 450 to include during RPA workflow development. The recommendations are based on a context of where a user is in the RPA workflow. For user interface (UI) automation, the AI / ML model may be linked to an object repository and trained to make recommendations therefrom. The AI / ML model may also be trained to generate new UI descriptors for the object repository.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates generally to robotic process automation (RPA), and more specifically to artificial intelligence (AI) / machine learning (ML) model training and recommendation engines for RPA. [Background technology]

[0002] Traditionally, when developing a workflow for RPA automation, an RPA developer configures workflow steps to achieve a desired task. However, many steps are used in multiple automations. In addition, activities, descriptors, and other objects for automation may already exist. For RPA automation in a user interface (UI), developers usually build a descriptor (such as a selector) for each UI action, and default descriptors are usually generated. However, building reliable descriptors or other components of an RPA activity may take developer time for troubleshooting and trial-and-error development, which may not be efficient. It also lacks reusability. In addition, when systems and applications are changed or upgraded, the upgrade path of the process may not be straightforward. Therefore, an improved approach to developing RPA automation may be beneficial. Summary of the Invention

[0003] Certain embodiments of the present invention may provide solutions to problems and needs in the field that have not yet been fully identified, appreciated, or solved by current RPA technology. For example, some embodiments of the present invention relate to a design-time AI / ML recommendation engine for RPA.

[0004] In an embodiment, the one or more non-transitory computer-readable media store one or more computer programs. The one or more computer programs are configured for at least one processor to provide information about an RPA workflow developed in an RPA designer application, the RPA workflow including one or more activities, to one or more AI / ML models. The one or more AI / ML models are trained to suggest a next activity, suggest a sequence of next activities, suggest changes to at least one parameter of one or more activities in the RPA workflow, or any combination thereof, based on content of the one or more activities in the RPA workflow. The one or more computer programs are also configured for the at least one processor to receive output from the one or more AI / ML models including the proposed next activity, the proposed sequence of next activities, the changes to the proposed parameters, or any combination thereof. The one or more computer programs are further configured for the at least one processor to notify a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the proposed changes to the parameters, or a combination thereof, or to automatically modify the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to the parameters, or a combination thereof, into the RPA workflow.

[0005] In another embodiment, one or more computing systems include a memory storing computer program instructions and at least one processor configured to execute the computer program instructions. The computer program instructions are configured for the at least one processor to provide information about an RPA workflow developed in an RPA designer application, the RPA workflow including one or more activities, to one or more AI / ML models. The one or more AI / ML models are trained to suggest a next activity, suggest a sequence of next activities, suggest changes to at least one parameter of one or more activities in the RPA workflow, or any combination thereof, based on content of the one or more activities in the RPA workflow. The computer program instructions are also configured for the at least one processor to receive output from the one or more AI / ML models including the proposed next activity, the proposed sequence of next activities, the changes to the proposed parameters, or any combination thereof. The computer program instructions are further configured for the at least one processor to notify a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the proposed changes to the parameters, or a combination thereof, or to automatically modify the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to the parameters, or a combination thereof into the RPA workflow, wherein the proposed next activity, the proposed sequence of next activities, the proposed changes to the parameters, or a combination thereof is based on a context of the RPA workflow in a current development state.

[0006] In yet another embodiment, a computer-implemented method includes providing, by a computing system, information regarding an RPA workflow developed in an RPA designer application that includes one or more activities to one or more AI / ML models. The one or more AI / ML models are trained to suggest a next activity, suggest a sequence of next activities, suggest changes to at least one parameter of one or more activities in the RPA workflow, or any combination thereof based on content of the one or more activities in the RPA workflow. The computer-implemented method also includes receiving, by the computing system, output from the one or more AI / ML models that includes the proposed next activity, the proposed sequence of next activities, the changes to the proposed parameters, or any combination thereof. The computer-implemented method further includes notifying, by the computing system, a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the changes to the proposed parameters, or any combination thereof, or automatically modifying the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the changes to the proposed parameters, or any combination thereof into the RPA workflow. Further, the computer-implemented method includes suggesting, by the computing system, a next task to be executed for the RPA workflow based on output from at least one of the one or more AI / ML models when a confidence score associated with the proposed next task exceeds an automated performance threshold, or automatically executing, by the computing system, the proposed next task. [Brief description of the drawings]

[0007] So that the advantages of certain embodiments of the invention may be readily understood, a more particular description of the invention briefly described above will now be depicted with reference to specific embodiments which are illustrated in the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not to be considered as limiting its scope, but the invention will be depicted and explained with further specificity and detail through the use of the following accompanying drawings, in which:

[0008] [Figure 1] FIG. 1 is an architectural diagram illustrating a hyper-automation system, in accordance with an embodiment of the present invention.

[0009] [Diagram 2] FIG. 1 is an architectural diagram illustrating an RPA system according to an embodiment of the present invention.

[0010] [Diagram 3] FIG. 1 is an architectural diagram illustrating a deployed RPA system according to an embodiment of the present invention.

[0011] [Figure 4] FIG. 2 is an architecture diagram illustrating the relationships between designers, activities, and drivers according to an embodiment of the present invention.

[0012] [Diagram 5] FIG. 1 is an architectural diagram illustrating a computing system configured to provide a design-time AI / ML recommendation engine for RPA, in accordance with an embodiment of the present invention.

[0013] [Figure 6A] 1 illustrates an example of a neural network trained to provide suggestions for a next activity or sequence of activities, according to an embodiment of the present invention.

[0014] [Figure 6B] 1 illustrates an example of a neuron according to an embodiment of the present invention.

[0015] [Figure 7] 1 is a flowchart illustrating a process for training an AI / ML model(s) in accordance with an embodiment of the present invention.

[0016] [Figure 8] 1 is an example of a graph illustrating relationships between application elements, according to an embodiment of the present invention.

[0017] [Figure 9] 1 is a flowchart illustrating a process of a design-time AI / ML recommendation engine for RPA in accordance with an embodiment of the present invention.

[0018] [Figure 10] 1 is a flowchart illustrating a process for performing RPA error avoidance or self-repair of an RPA robot at runtime in accordance with an embodiment of the present invention.

[0019] Unless otherwise noted, like reference characters denote corresponding features consistently throughout the attached drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Detailed Description of the Embodiments Some embodiments relate to a design-time AI / ML recommendation engine for RPA. An AI / ML model may be trained to provide recommendations regarding next activities or sequences of activities to include during design-time RPA workflow development. In certain embodiments, a recommended next activity or sequence of activities may be automatically inserted into the RPA workflow if the confidence of the recommended next activity or sequence of activities exceeds an auto-insertion threshold (e.g., 80%, 90%, 99%, etc.). The recommendations may be based on the user's context in the development of the RPA workflow (i.e., the current state of the RPA workflow at that time). For example, if an RPA developer adds a send email activity to a workflow, an AI / ML model may use this context to recommend archiving the sent email as the next activity.

[0021] In some embodiments, the AI / ML model may be trained to suggest the next activity or sequence of activities using context based on a previously built RPA workflow. For example, if a user creates an activity to fill in some fields in a form based on a source, the next step after filling in the fields may often be to click a particular button (e.g., "Submit," "OK," etc.). The AI / ML model, having been trained on the existing RPA workflow, may learn that filling in all fields and then clicking such a button is likely the desired task, and may recommend an activity to the user that automatically fills in the remaining fields and clicks the appropriate button.

[0022] Note that in some embodiments, suggestions are not limited to activities and / or sequences of activities. For example, an AI / ML model may learn to configure parameters of existing activities (such as properties like inputs and outputs, improving the content of emails that the workflow generates, etc.). This can help ensure that the RPA workflow runs correctly by correcting user errors and / or providing missing information.

[0023] In certain embodiments, the AI / ML model may learn to suggest what to do next in the automation. For example, the AI / ML model may learn that similar RPA workflows tend to generate automation for developers to test the RPA workflow, publish the RPA workflow in an enterprise repository or directory, etc. If the confidence of the AI / ML model is high enough (e.g., above an automation performance threshold), the next step(s) may be executed automatically.

[0024] In some embodiments of UI automation, the AI / ML model is linked to the object repository and trained to make recommendations from there. For example, the AI / ML model may recognize in the UI that the RPA workflow is for a certain application, application version, app screen, etc., and recommend activities and / or their parameters (e.g., UI descriptors, inputs, outputs, etc.) for that particular application / version / screen by "seeing" the screen (e.g., using a computer vision (CV) model to identify the graphical elements and text on the screen). This may reduce or eliminate the need for the RPA developer to provide additional information in this regard when configuring an activity in the RPA workflow. Instead, the AI / ML model may recognize that the screen the user specified for the activity corresponds to a specific application / version / screen, and may search for the appropriate UI objects and / or UI descriptors for that activity, as well as the recommendation made by the AI / ML model for the next activity or sequence of activities to add.

[0025] In some embodiments, recommendations for next activities and / or parameters of such activities may be provided for non-UI automation. Previously developed RPA workflows may be used to train AI / ML models to recognize typical next activities or sequences thereof. Thus, suggestions for additional activities and / or sequences thereof may be provided for any type of RPA workflow, whether attended or unattended RPA robots, and UI or non-UI automation, without departing from the scope of the present invention.

[0026] In some embodiments, a graph of relationships may be constructed via the first AI / ML model. For example, the graph may link applications to screens, screens to versions, UI elements to screens and UI descriptors, etc. as nodes in the graph. In some embodiments, a connection between two nodes may be directed or bidirectional.

[0027] Once the graph is created, at least in its initial form, a second AI / ML model (or potentially a more complex version of the first AI / ML model) may be trained to use this graph to learn ontological relationships. For example, the AI / ML model may observe that when a user clicks a "Submit" button on one screen containing a form, the application displays another screen containing information about processing the submitted form. This may cause the AI / ML model to determine that there is an ontological relationship between the submit button and the next screen, and the AI / ML model may modify the graph to include a directed connection between the submit button and the next screen. Thus, the data itself may be organized into an ontology. Once the graph is built and the AI / ML model(s) are trained, they may be used to make suggestions and / or correct failures.

[0028] Thus, the graph may be used in some embodiments for both activity suggestions and self-healing purposes. Run-time corrections may be performed without manually modifying the respective RPA workflow. For example, if a graphical element on a screen is not identified, the AI / ML model may determine what went wrong (e.g., the position of the graphical element has changed and / or its design has changed) and automatically repair the automation to accommodate this change (e.g., the UI descriptor of the old graphical element design is replaced with the UI descriptor of the new graphical element design).

[0029] While automatic repair may be necessary for some errors (such as saving a document, locating a button that was moved, or auto-filling new form information required in a new version of an application), automatic repair may not be desirable for others. For example, if new compliance requirements are introduced that require approval by a manager before submission, the AI / ML model may not be able to add these additional features on its own. The AI / ML model may learn whether to automatically repair an error by training on examples where users accepted and rejected the suggested fixes. Over time, the AI / ML model learns what types of fixes are required and may automatically make the fixes or the RPA robot may fail.

[0030] In some embodiments, self-healing and / or preemptive repair capabilities may be provided at run time. Consider what happens if the interface of the application for which the RPA workflow was designed changes such that one or more activities no longer work as designed. For example, new fields may be added to forms that are required before submission, the names and / or positions of fields may be changed, the design and / or position of buttons may be changed, etc. In some embodiments, the AI / ML model or the RPA robot itself searches for changes in the UI object repository and / or another source (e.g., a web page that lists the changes in the new application version) and determines what modifications can be made to correct the issue. For example, for a new employee form, if a spouse field is now required (e.g., Name or N / A), the AI / ML model may be trained to determine that a "type into" activity is required in the RPA workflow and to add this activity to the workflow. An updated RPA automation may then be generated that replaces the previous automation in the original workflow, and this version may be used instead.

[0031] In some embodiments, workflow suggestions and / or repairs may be performed iteratively. For example, while a user is developing and testing an RPA workflow in an RPA designer application, an AI / ML model may iteratively test the workflow. If the workflow fails, the AI / ML model may provide suggestions and / or automatically apply repairs (e.g., changing activity parameters, replacing an activity with another activity appropriate for the task, adding one or more additional activities required to make a particular activity work, etc.) while the user is developing the workflow. The AI / ML model may be run periodically during workflow development to iteratively address issues in the RPA workflow, increasing the speed with which a user can develop the workflow and increasing the effectiveness and accuracy of the workflow.

[0032] In some embodiments, RPA automations may be automatically modified as the software application they are automating changes. For example, consider the case where a company switches from using SAP Concur® to Expensify®. Rather than having developers create new workflows for the new application, an AI / ML model may be trained to automatically search the Expensify® UI object repository of UI descriptors, activities, parameters, etc. that perform the same task(s). That is, the screens and processes may be slightly different, but the business logic of the task is the same or similar.

[0033] UI descriptors and other elements in one application's UI object repository may be mapped to those of other applications, possibly using semantic or ontological mapping. The AI / ML model semantically decomposes what is done in one application and another, dividing the tasks into screens. If the AI / ML model is not pre-trained for this purpose, the UI object repository may be constructed manually or automatically. For example, in the case of a change, a link to the target application's website may be provided, and a UI object repository may be constructed from the web application screens on the website. Reinforcement learning is used for this process, and the AI / ML model is given a reward function (e.g., submitting an invoice, filling out a form, copying information from a source to a destination, etc.). If the AI / ML model is not successful after a certain number of epochs, a human may continue and / or complete the training process by labeling the information, performing the actions the AI / ML model observes, revising the reward function, etc.

[0034] Some embodiments use a cloud implementation of an object repository accessible to RPA developers, potentially eliminating the need to package UI elements or use a package manager such as NuGet™. The AI / ML model may determine whether to request a better UI element. To do this, the RPA robot employing the AI / ML model may search for a previous version of the UI descriptor and / or UI descriptors of similar elements in other similar target applications developed when the UI descriptor fails. Additionally or instead, the RPA robot employing the AI / ML model may check for a new version of the UI descriptor in the object repository. This provides a self-healing capability that allows the RPA robot to find the element when the UI descriptor is not working, rather than throwing an error or skipping the activity.

[0035] Consider a case where the Japanese version of a website is loaded instead of the English version, but the functionality of the buttons on each page is the same. An RPA robot employing an AI / ML model may try UI descriptors of different language versions to try to find the button. Also, consider a case where a process is started in a different web browser than the one expected. The RPA robot may try UI descriptors of various web browsers to try to find a graphical element.

[0036] In some embodiments, AI / ML models may be used to generate correct UI descriptors. Over time, the object repository may become out of date if not updated regularly. AI / ML models can be trained to interpret how the application has changed and automatically update the object repository with working UI descriptors. This essentially provides automatic versioning. By complementing the object repository with AI / ML models that can generate UI descriptors, these can be used to find UI descriptors for activities in an RPA workflow.

[0037] To automatically update the UI repository with a working UI descriptor, some embodiments determine why a failure occurred. Telemetry data may be used to determine which element(s) failed and why, and this telemetry data is connected to the respective elements in the UI repository. Using this data, the AI / ML model may learn to extract what the failed process was trying to accomplish. For example, the AI / ML model may determine that the process was trying to use a UI descriptor with a mismatched version of the application, and then swap to a working UI descriptor. If this cannot be derived, other data points may be used to make this determination. For example, there may be a location in the UI that changes upon receipt of an update email that breaks the RPA process. Also, once the AI / ML model identifies the cause of the problem, an RPA Center of Excellence (COE) may be notified to modify the RPA process and point out the potential problem at runtime. Alternatively, the AI / ML model may make the correction in the object repository itself.

[0038] In some embodiments, semantic matching may be performed. For example, an AI / ML model may be trained to learn that certain text in a modified user interface matches text from a previous version. If an "Invoice Number" label is changed to "Bill No." and the location of an associated text field is moved on the screen, the AI / ML model may recognize that these labels match and update the UI descriptor to find the text field (e.g., using target anchor matching).

[0039] In some embodiments, multiple AI / ML models may be used. For example, there may be an AI / ML model for suggesting the next activity or sequence of activities, a CV model, a semantic matching model, a generative AI model, etc. The AI / ML model for suggesting the next activity or sequence of activities may be trained on previous RPA automations to learn how they were built, what similarities (patterns) exist in these automations, dependencies between activities in the RPA workflow (e.g., input from one activity comes from the output of another activity), output types, etc. The AI / ML model may learn to recognize these patterns, for example, if an automation has a certain number and / or type of activities, if the automation accomplishes a certain end task (e.g., write and send an email, fill out and send a form, etc.).

[0040] The object repository contains a library of UI objects. UI elements on a screen (e.g., text fields, buttons, labels, menus, checkboxes, etc.) may be grouped by application, application version, application screen, and collection of UI elements, with each screen having multiple UI elements. As used herein, a "screen" is an image of an application UI or a portion of an application UI at a point in time. An "application" or a version of a given application may be a combination of screens in this context. Each UI element may be described by one or more UI descriptors in some embodiments. UI elements, UI descriptors, applications, and application screens are UI objects. In some embodiments, UI elements and screens may be further differentiated into specific types of UI elements (e.g., buttons, checkboxes, text fields, etc.) and screens (e.g., top windows, modal windows, pop-up windows, etc.).

[0041] A selector is a type of UI descriptor that can be used to find UI elements in some embodiments. A selector, in some embodiments, has the following structure: <node_1 / ><node_2 / > ...<node_N / >

[0042] The final node represents the target GUI element, and all previous nodes represent the parents of that element.<node_1> is usually called the root node and represents the top window of the application.

[0043] Each node may have one or more attributes that aid in the correct identification of the particular level of the selected application. Each node, in some embodiments, has the following format: <ui_system attr_name_1=’attr_value_1’...attr_name_N=’attr_value_N’ / >

[0044] All attributes may have values ​​assigned to them, and attributes with constant values ​​may be chosen because changing the value of an attribute every time an application launches may cause the selector to not properly identify the associated element.

[0045] UI object library descriptors can be added directly to RPA workflow activities, saving developers time that might otherwise be required to create custom selectors for the activities. An object library is defined herein as a collection of UI descriptors that correspond to one or more screens from a particular version of an application. An example structure of an object library is shown below: ·SAP Version 1 Screen 1 Object 1 Object 2 ·… Screen 2… ·… Version 2… ·… ·Salesforce… ·…

[0046] It should be noted that the above hierarchy is provided as an example only. Any desired number of levels in the hierarchy and elements associated with each level may be used without departing from the scope of the present invention. In certain embodiments, a user may define the application map as he or she desires. Additionally, some nodes in the tree may, in some embodiments, only serve an administrative role and not a functional role. For example, UI elements may be grouped together in a container that does not have a UI descriptor. The container may, in some embodiments, exist solely for grouping purposes.

[0047] A UI descriptor is a set of instructions for finding UI elements. In some embodiments, the UI descriptor is an encapsulated data / structure format that includes UI element selector(s), anchor selector(s), CV descriptor(s), unified target descriptor(s), screen image capture (context), element image capture, other metadata (e.g., application and application version), combinations thereof, and the like. The encapsulated data / structure format may be extensible by future updates to the platform and is not limited to the above definition. Any suitable UI descriptor for identifying UI elements on the screen may be used without departing from the scope of the present invention. The unified target descriptor concatenates multiple types of UI descriptors. The unified target descriptor may function like a finite state machine (FSM), where in a first context, a first UI descriptor mechanism is applied, in a second context, a second UI descriptor is applied, and so on.

[0048] UI descriptors may be extracted from activities in an RPA workflow and added to a structured schema that groups UI descriptors by UI application, screen, and UI element. The UI descriptors may be part of one project for widespread reuse, part of a global repository for testing purposes, or part of a UI object library for global cross-project sharing in some embodiments. The object library in some embodiments is an encapsulation of UI descriptors grouped by application, application version, and screen. UI descriptors may be defined and added to the object library, which in some embodiments may be installed in other projects as a dependency after publishing. This allows the object library of UI descriptors to be reused. The UI application in some embodiments is a target application that has multiple versions, each version with multiple screens.

[0049] The UI Object Repository and reuse of UI descriptors can be beneficial for a variety of reasons. For example, if the location, appearance, and / or functionality of a UI element in an application changes, the UI descriptor can be modified and the changes can then be propagated to activities that use the UI descriptor. In this way, reusability can be provided at the specific level of the graphical elements in the UI.

[0050] The UI descriptor, in some embodiments, may work with a unified target that encompasses all UI detection mechanisms by which image detection and definition are performed. The unified target may merge multiple techniques for identifying and automating UI elements into a single cohesive approach. The unified target may prioritize selector-based and driver-based UI detection mechanisms and fall back to CV to find images if the first two mechanisms are not successful. In some embodiments, the selector editor and UI explorer may be configured to support the unified target.

[0051] FIG. 1 is an architecture diagram illustrating a hyperautomation system 100, according to an embodiment of the present invention. As used herein, “hyperautomation” refers to an automation system that brings together process automation components, integrated tools, and technologies that amplify the ability to automate work. For example, in some embodiments, RPA is used at the core of the hyperautomation system, and in certain embodiments, the automation capabilities may be extended by AI / ML, process mining, analytics, and / or other advanced tools. As the hyperautomation system learns processes, trains AI / ML models, and employs analytics, for example, more knowledge work may be automated, and computing systems within an organization, for example, both those used by individuals and those operating autonomously, may all be engaged as participants in the hyperautomation process. The hyperautomation system of some embodiments enables users and organizations to discover, understand, and extend automation efficiently and effectively.

[0052] The hyperautomation system 100 includes user computing systems such as a desktop computer 102, a tablet 104, and a smartphone 106. However, any desired user computing system may be used without departing from the scope of the present invention, including, but not limited to, a smart watch, a laptop computer, a server, an Internet of Things (IoT) device, and the like. Also, although three user computing systems are shown in FIG. 1, any suitable number of user computing systems may be used without departing from the scope of the present invention. For example, in some embodiments, tens, hundreds, thousands, or millions of user computing systems may be used. The user computing systems may be actively used by a user or may run automatically without much or any user input.

[0053] Each user computing system 102, 104, 106 has a respective automation process(es) 110, 112, 114 running thereon. In some embodiments, the automation processes are stored remotely (e.g., on a server 130 or on a database 140) (accessed via the network 120) and loaded by the RPA robot to implement the automation. The automation may exist as a script (e.g., XML, XAML, etc.) or may be compiled into machine-readable code (e.g., as a digital link library).

[0054] The automation process(es) 110, 112, 114 may include, but are not limited to, an RPA robot, a part of an operating system, a downloadable application(s) for the respective computing system, any other suitable software and / or hardware, or any combination thereof, without departing from the scope of the present invention. In some embodiments, one or more of the process(es) 110, 112, 114 may be a listener. The listener may be an RPA robot, a part of an operating system, a downloadable application for the respective computing system, or any other software and / or hardware, without departing from the scope of the present invention. Indeed, in some embodiments, the logic of the listener(s) is implemented partially or completely via physical hardware.

[0055] The listeners monitor and record data related to user interactions with their respective computing systems and / or the operation of unattended computing systems and transmit the data over a network (e.g., a local area network (LAN), a mobile communications network, a satellite communications network, the Internet, any combination thereof, etc.) to the core hyperautomation system 120. The data may include, but is not limited to, which button was clicked, where the mouse was moved, text entered into a field, one window was minimized and another opened, the application associated with the window, etc. In certain embodiments, the data from the listeners may be transmitted periodically as part of a heartbeat message. In some embodiments, the data may be transmitted to the core hyperautomation system 120 when a predetermined amount of data has been collected, after a predetermined period of time has passed, or both. One or more servers, such as server 130, receive the data from the listeners and store it in a database, such as database 140.

[0056] An automation process may execute logic developed in a workflow during design time. In the case of RPA, a workflow may include a set of steps, defined herein as "activities," that are executed in sequence or some other logic flow. Each activity may include an action such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0057] Long-running workflows for RPA in some embodiments are master projects that support service orchestration, human intervention, and long-running transactions in unattended environments. See, for example, U.S. Patent No. 10,860,905, which is incorporated by reference in its entirety. Human intervention comes into play when a particular process requires human input for exception handling, approval, or validation before proceeding to the next step of the activity. In this case, the process execution is paused, freeing up the RPA robot until the human task is completed.

[0058] Long-running workflows may support workflow fragmentation through persistence activities and may be combined with calling process and non-user interaction activities to orchestrate human tasks with RPA robot tasks. In some embodiments, multiple or multiple computing systems may participate in the execution of the logic of a long-running workflow. Long-running workflows may execute in sessions to facilitate rapid execution. In some embodiments, long-running workflows may orchestrate background processes that execute application programming interface (API) calls and may include activities that execute in the long-running workflow session. These activities may be invoked in some embodiments by a calling process activity. Processes with user interaction activities that execute in a user session may be invoked by starting a job from a conductor activity (conductors are described in more detail later in this specification). Users may interact through tasks that require completing a form in the conductor in some embodiments. An activity may be included that causes the RPA robot to wait for a form task to complete and then resume the long-running workflow.

[0059] One or more automation process(es) 110, 112, 114 are in communication with the core hyper-automation system 120. In some embodiments, the core hyper-automation system 120 may execute a conductor application on one or more servers, such as server 130. While one server 130 is shown for illustrative purposes, multiple or numerous servers in close proximity to one another or in a distributed architecture may be employed without departing from the scope of the present invention. For example, one or more servers may be provided for conductor functions, AI / ML model serving, certification, governance, and / or any other suitable functions without departing from the scope of the present invention. In some embodiments, the core hyper-automation system 120 may incorporate or be part of a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, and / or the like. In certain embodiments, the core hyper-automation system 120 may host multiple software-based servers on one or more computing systems, such as server 130. In some embodiments, one or more servers of the core hyper-automation system 120, such as server 130, may be implemented via one or more virtual machines (VMs).

[0060] In some embodiments, one or more automation process(es) 110, 112, 114 may invoke one or more AI / ML models 132 deployed on or accessible by the core hyper-automation system 120 and trained to accomplish various tasks. For example, the AI / ML models 132 may include models trained to look for different application versions, perform CV, generate UI descriptors, and provide suggestions for the next activity or sequence of activities in an RPA workflow. The AI / ML models may be trained using labeled data including previously created RPA workflows, screenshots of various application screens in different versions with corresponding UI elements, libraries of UI objects, etc. The AI / ML models 132 may be trained to achieve a desired confidence threshold without overfitting to a given set of training data.

[0061] The AI / ML model 132 may be trained for any suitable purpose without departing from the scope of the present invention, as discussed in more detail later herein. Two or more AI / ML models 132 may be chained (e.g., serially, in parallel, or a combination thereof) in some embodiments such that they collectively provide a cooperative output(s). The AI / ML models 132 may perform or assist in CV, OCR, document processing and / or understanding, semantic learning and / or analysis, analytical prediction, process discovery, task mining, testing, automated RPA workflow generation, sequence extraction, clustering detection, speech-to-text translation, any combination thereof, and the like. However, any desired number and / or type(s) of AI / ML models may be used without departing from the scope of the present invention. Using multiple AI / ML models may, for example, allow a system to develop a complete picture of what is going on on a given computing system. For example, one AI / ML model may perform OCR, another may detect buttons, another may compare sequences, and the like. The patterns may be determined by the AI / ML models individually or collectively by multiple AI / ML models. In particular embodiments, one or more AI / ML models are deployed locally on at least one of the computing systems 102, 104, 106.

[0062] In some embodiments, multiple AI / ML models 132 may be used. Each AI / ML model 132 is an algorithm (or model) that runs on data, and the AI / ML model itself may be, for example, a deep learning neural network (DLNN) of artificial "neurons" trained on training data. In some embodiments, the AI / ML model 132 may have multiple layers that perform various functions, such as statistical modeling (e.g., Hidden Markov Models (HMMs)), and may utilize deep learning techniques (e.g., long short-term memory (LSTM) deep learning, encoding of prior hidden states, etc.) to perform desired functions.

[0063] The hyper-automation system 100, in some embodiments, may provide four main groups of functions: (1) discovery, (2) build automation, (3) management, and (4) engagement. Automation (e.g., running on a user computing system, server, etc.) may be performed by software robots, such as RPA robots, in some embodiments. For example, attended robots, unattended robots, and / or test robots may be used. Attended robots collaborate with users to assist them with tasks (e.g., via UiPath Assistant™). Unattended robots operate independently of the user and may potentially run in the background without the user's knowledge. Test robots are unattended robots that run test cases against an application or RPA workflow. Test robots, in some embodiments, may run in parallel on multiple computing systems.

[0064] A discovery function may discover and provide automated recommendations for various business process automation opportunities. Such functions may be implemented by one or more servers, such as server 130. The discovery function may include providing an automation hub, process mining, task mining, and / or task capture, in some embodiments. An automation hub (e.g., UiPath Automation Hub™) may provide a mechanism to manage automation rollouts with visibility and control. Automation ideas may be crowdsourced from employees, for example, via a submission form. Calculations of feasibility and return on investment (ROI) for automating these ideas may be provided, documentation for future automations may be collected, and collaboration may be provided to expedite automation discovery-to-build.

[0065] Process mining (e.g., via UiPath Automation Cloud™ and / or UiPath AI Center™) refers to the process of collecting and analyzing data from applications (such as enterprise resource planning (ERP) applications, customer relationship management (CRM) applications, email applications, call center applications, etc.) to identify what end-to-end processes exist in an organization, how they can be effectively automated, and the impact that automation will have. This data may be obtained, for example, by listeners from user computing systems 102, 104, 106 and processed by a server, such as server 130. In some embodiments, one or more AI / ML models 132 may be employed for this purpose. This information may be exported to an automation hub to speed implementation and avoid manual information transfer. The goal of process mining may be to increase business value by automating processes in an organization. Some examples of process mining goals include, but are not limited to, increased profits, increased customer satisfaction, regulatory and / or contractual compliance, increased employee efficiency, etc.

[0066] Task mining (e.g., via UiPath Automation Cloud™ and / or UiPath AI Center™) identifies and aggregates workflows (e.g., employee workflows) and then applies AI to uncover patterns and variations in routine tasks and score such tasks for ease of automation and potential savings (e.g., time and / or cost savings). One or more AI / ML models 132 may be employed to uncover repetitive task patterns in the data. Repetitive tasks ripe for automation may then be identified. This information may be initially provided by the listener and, in some embodiments, analyzed on a server of the core hyper-automation system 120, such as server 130. Findings from task mining (e.g., Extensible Application Markup Language (XAML) process data) may be exported to process documentation or designer applications such as UiPath Studio™ to more quickly create and deploy automations.

[0067] Task mining in some embodiments may include taking screenshots with user actions (e.g., mouse click locations, keyboard input, application windows and graphical elements with which the user was interacting, timestamps for the interactions, etc.), collecting statistical data (e.g., performance time, number of actions, text input, etc.), editing and annotating the screenshots, specifying the types of actions to be recorded, etc.

[0068] Task capture (through UiPath Automation Cloud™ and / or UiPath AI Center™) automatically documents attended processes as users work on them or provides a framework for unattended processes. Such documentation may include process definition documents (PDDs), skeleton workflows, capturing actions for each part of the process, recording user actions and automatically generating comprehensive workflow diagrams with details about each step, tasks that are desirable to automate in formats such as Microsoft Word™ documents, XAML files, etc. Buildable workflows may, in some embodiments, be exported directly to designer applications such as UiPath Studio™. Task capture may simplify the requirements gathering process for both subject matter experts describing the process and CoE members providing production-grade automation.

[0069] Building automations may be accomplished through designer applications (UiPath Studio™, UiPath StudioX™, UiPath Web™, etc.). For example, an RPA developer at RPA development facility 150 may use RPA designer application 154 at computing system 152 to build and test automations for various applications and environments, such as web, mobile, SAP®, and virtual desktop. API integration may be provided for various applications, technologies, and platforms. Predefined activities, drag-and-drop modeling, and workflow recorders may facilitate automation with minimal coding. Document understanding capabilities may be provided through drag-and-drop AI skills for data extraction and interpretation that invoke one or more AI / ML models 132. Such automations may handle virtually any document type and format, including tables, checkboxes, signatures, and handwriting. When data is validated or exceptions are handled, this information may be used to retrain the respective AI / ML models, improving their accuracy over time.

[0070] RPA designer application 152 may be designed to invoke one or more of trained AI / ML models 132 on server 130 and / or generative AI models 172 in a cloud environment over network 120 (e.g., a local area network (LAN), a mobile communication network, a satellite communication network, the Internet, any combination thereof, etc.) to assist in the RPA automation development process. In some embodiments, one or more of the AI / ML models may be packaged with RPA designer application 152 or may otherwise be stored locally on computing system 150.

[0071] In some embodiments, the RPA designer application 152 and the one or more AI / ML models 132 may be configured to use an object repository stored in the database 140. The object repository may include a library of UI objects that may be used to develop RPA workflows via the RPA designer application 152. The object repository may be used to add UI descriptors to activities in the workflows of the RPA designer application 152 for UI automation. In some embodiments, one or more of the AI / ML models 132 may generate new UI descriptors and add them to the object repository in the database 140. Once an automation is completed in the designer application 152, the automation may be published on the server 130 and pushed out to the computing systems 102, 104, 106, etc.

[0072] The integration services allow developers to seamlessly combine, for example, user interface (UI) automation and API automation. Automations that require APIs or span both API and non-API applications and systems may be built. A repository (e.g., UiPath Object Repository™) or marketplace (e.g., UiPath Marketplace™) for pre-built RPA and AI templates and solutions may be provided to enable developers to more quickly automate a wide variety of processes. Thus, when building automations, the hyperautomation system 100 may provide user interfaces, development environments, API integrations, pre-built and / or custom-built AI / ML models, development templates, integrated development environments (IDEs), and advanced AI capabilities. The hyperautomation system 100, in some embodiments, may enable development, deployment, management, configuration, monitoring, debugging, and maintenance of RPA robots, which may provide automation for the hyperautomation system 100.

[0073] In some embodiments, components of the hyper-automation system 100, such as the designer application(s) and / or external rules engine, provide support for managing and enforcing governance policies to control various functions provided by the hyper-automation system 100. Governance is the ability for an organization to put policies in place to prevent users from developing automations (such as RPA robots) that can perform actions that may harm the organization, such as violating the EU General Data Protection Regulation (GDPR), the US Health Insurance Portability and Accountability Act (HIPAA), third-party application terms of use, etc. Because developers may otherwise create automations that violate privacy laws, terms of use, etc. during the execution of their automations, some embodiments implement access control and governance restrictions at the robot and / or robot design application level. This may provide an additional level of security and compliance to the automation process development pipeline in some embodiments by preventing developers from taking dependencies on unapproved software libraries that may pose security risks or operate in a manner that violates policies, regulations, privacy laws, and / or privacy policies. See, for example, U.S. Patent Application Publication No. 2022 / 0011732, which is incorporated by reference in its entirety into this specification.

[0074] The management functions may provide management, deployment, and optimization of automation across an organization. The management functions may include orchestration, test management, AI capabilities, and / or insights, in some embodiments. The management functions of the hyper-automation system 100 may also act as an integration point with third party solutions and applications for automation applications and / or RPA robots. The management functions of the hyper-automation system 100 may include, but are not limited to, facilitating provisioning, deployment, configuration, queuing, monitoring, logging, and interconnection of RPA robots, among others.

[0075] Conductor applications such as UiPath Orchestrator™ (which may be provided in some embodiments as part of UiPath Automation Cloud™ or may be provided on-premise, on a VM, on a private or public cloud, on a Linux™ VM, or as a cloud-native single container suite via UiPath Automation Suite™) provide orchestration capabilities to deploy, monitor, optimize, scale, and ensure the security of RPA robot deployments. Test suites (e.g., UiPath Test Suite™) may provide test management to monitor the quality of deployed automation. Test suites may facilitate test planning and execution, requirements fulfillment, and defect traceability. Test suites may include comprehensive test reports.

[0076] The analytics software (e.g., UiPath Insights™) can track, measure, and manage the performance of deployed automation. The analytics software can align automation actions with specific key performance indicators (KPIs) and strategic outcomes of the organization. The analytics software can present results in a dashboard format for better understanding by human users.

[0077] Data services (e.g., UiPath Data Service™) can bring data to a single, scalable, secure location, for example, stored in database 140, with a drag-and-drop storage interface. Some embodiments may provide low-code or no-code data modeling and storage to automation, while ensuring seamless access, enterprise-grade security, and scalability of data. AI capabilities may be provided by an AI center (e.g., UiPath AI Center™), which facilitates the incorporation of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options may make such capabilities accessible to non-data scientists. Deployed automation (e.g., RPA robots) may invoke AI / ML models from the AI ​​center, such as AI / ML models 132. Performance of the AI / ML models may be monitored and trained and improved using human-verified data, such as provided by data review center 160. Human reviewers may provide labeled data to the core hyper-automation system 120 via review applications 152 on computing systems 154. For example, the human reviewer may verify that the predictions by the AI / ML model 132 are accurate or may otherwise provide corrections. This dynamic input may then be saved as training data for retraining the AI / ML model 132 and stored in a database, such as database 140. The AI ​​center may then schedule and complete a training job to train a new version of the AI / ML model using the training data. Both positive and negative examples may be stored and used for retraining the AI / ML model 132.

[0078] Engagement capabilities engage humans and automation as one team for seamless collaboration around a desired process. Low-code applications can be built (e.g., via UiPath Apps™), even those lacking APIs in some embodiments, to connect browser tabs and legacy software. Applications can be created quickly using a web browser, for example, through a rich library of drag-and-drop controls. Applications can be connected to one automation or multiple automations.

[0079] Action Center (e.g., UiPath Action Center™) provides an easy and efficient mechanism to hand off processes from automation to humans and vice versa. Humans can provide approvals or escalations, handle exceptions, etc. Automation can then execute the automated functions of a given workflow.

[0080] A local assistant may be provided as a launch pad for users to launch automations (e.g., UiPath Assistant™). This functionality may be provided, for example, in a tray provided by the operating system, and may allow users to interact with RPA robots and RPA robot-powered applications on their computing systems. The interface may list automations approved for a given user and allow users to execute them. These may include ready-to-use automations from the automation marketplace, the automation hub's internal automation store, etc. When automations are running, they may run as local instances in parallel with other processes on the computing system so that the user can use the computing system while the automation performs its actions. In certain embodiments, the assistant is integrated with a task capture functionality so that users can document their soon-to-be-automated processes from the assistant's launch pad.

[0081] Chatbots (e.g., UiPath Chatbots™), social messaging applications, and / or voice commands may enable users to execute automations. This may simplify access to information, tools, and resources needed to interact with customers or conduct other activities. Human-to-human conversations may be automated as easily as other processes. Triggered RPA robots thus activated may be able to perform actions such as checking order status, posting data to a CRM, etc., using plain language commands.

[0082] End-to-end measurement and governance of automation programs at any scale may be provided by the hyper-automation system 100 in some embodiments. As a result of the above, analytics (e.g., via UiPath Insights™) may be employed to understand the performance of automations. Data modeling and analytics using any combination of available business metrics and operational insights may be used for various automation processes. Custom-designed and pre-built dashboards may visualize data across desired metrics, discover new analytical insights, track performance indicators, discover ROI for automations, perform telemetry monitoring on users' computing systems, detect errors and anomalies, and debug automations. An automation management console (e.g., UiPath Automation Ops™) may be provided to manage automations throughout the automation lifecycle. Organizations may govern how automations are built, what users can do with them, and which automations users have access to.

[0083] The hyper-automation system 100, in some embodiments, provides an iterative platform: processes can be discovered, automations can be built, tested, and deployed, performance can be measured, automation usage can be easily provided to users, feedback can be obtained, AI / ML models can be trained and retrained, and the process itself can be repeated, facilitating a more robust and effective set of automations.

[0084] In some embodiments, generative AI models are used. Generative AI may generate various types of content, such as text, images, audio, and synthetic data. Various types of generative AI models may be used, including but not limited to large-scale language models (LLMs), generative adversarial networks (GANs), variational autoencoders (VAEs), transformers, and the like. These models may be part of the AI / ML models 132 hosted on the server 130. For example, generative AI models may be trained on large corpora of text information to perform semantic understanding, understand the nature of what is present on the screen from the text, and the like. In certain embodiments, generative AI models 172 provided by existing cloud ML service providers, such as OpenAI®, Google®, Amazon®, Microsoft®, IBM®, Nvidia®, Facebook®, and the like, may be employed and trained to provide such capabilities. In generative AI embodiments where the generative AI model(s) 172 are hosted remotely, the server 130 may be configured to integrate with third-party APIs, such that the server 130 can send requests containing required input information to the generative AI model(s) 172 and receive the reply (e.g., semantic matching of fields between application versions, classification of application types on screens, etc.). Such embodiments can provide a more advanced and sophisticated user experience as well as access to state-of-the-art natural language processing (NLP) and other ML capabilities offered by these companies.

[0085] FIG. 2 is an architecture diagram illustrating an RPA system 200, according to an embodiment of the present invention. In some embodiments, the RPA system 200 is part of the hyper-automation system 100 of FIG. 1. The RPA system 200 includes a designer 210 that enables developers to design and implement workflows. The designer 210 provides solutions for application integration, as well as automating third-party applications, management information technology (IT) tasks, and business IT processes. The designer 210 may facilitate the development of automation projects, which are graphical representations of business processes. Simply put, the designer 210 facilitates the development and deployment of workflows and robots. In some embodiments, the designer 210 may be an application running on a user's desktop, an application running remotely in a VM, a web application, or the like.

[0086] An automation project enables rule-based process automation by giving developers control over the order of execution and relationships between a custom set of steps developed in a workflow, defined herein as "activities" as described above. One commercial example of an embodiment of the designer 210 is UiPath Studio™. Each activity may include an action such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0087] Some types of workflows may include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences may be particularly suitable for linear processes, allowing flow from one activity to another without cluttering the workflow. Flowcharts may be particularly suitable for more complex business logic, allowing for decision integration and connection of activities in more diverse ways via multiple branching logic operators. FSMs may be particularly suitable for large workflows. FSMs may use a finite number of states triggered by conditions (i.e., transitions) or activities during their execution. Global exception handlers may be particularly suitable for determining the behavior of a workflow when an execution error is encountered, or for debugging the process.

[0088] Once a workflow is developed in Designer 210, the execution of the business process is orchestrated by Conductor 220, which coordinates one or more Robots 230 that execute the workflow developed in Designer 210. One commercial example of an embodiment of Conductor 220 is UiPath Orchestrator™. Conductor 220 facilitates management of the creation, monitoring, and deployment of resources in an environment. Conductor 220 may act as an integration point with third party solutions and applications. Accordingly, in some embodiments, Conductor 220 may be part of the core hyper-automation system 120 of FIG. 1.

[0089] The conductor 220 may manage all the robots 230, connecting and executing them from a central point. Types of robots 230 that may be managed include, but are not limited to, attended robots 232, unattended robots 234, development robots (similar to unattended robots 234, but used for development and testing purposes), and non-production robots (similar to attended robots 232, but used for development and testing purposes). Attended robots 232 are triggered by user events and operate side-by-side with humans on the same computing system. Attended robots 232 may be used with the conductor 220 for a centralized process deployment and logging medium. Attended robots 232 may assist human users in accomplishing various tasks and may be triggered by user events. In some embodiments, processes cannot be initiated from the conductor 220 on this type of robot and / or they cannot be run under a locked screen. In certain embodiments, the attended robot 232 may only be launched from the robot tray or from a command prompt. The attended robot 232 preferably operates under human supervision in some embodiments.

[0090] Unattended robots 234 can operate unattended in a virtual environment and automate many processes. Unattended robots 234 can be responsible for providing remote execution, monitoring, scheduling, and work queue support. Debugging for all robot types can be performed in designer 210 in some embodiments. Both attended and unattended robots can automate a variety of systems and applications, including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those produced by SAP®, SalesForce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0091] Conductor 220 may have various capabilities including, but not limited to, provisioning, deployment, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between robots 230 and conductor 220 (e.g., web applications). Deployment may include ensuring correct delivery of package versions to robots 230 assigned for execution. Configuration may include maintaining and delivering robot environment and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot specific data and maintaining user permissions. Logging may include storing and indexing logs in a database (e.g., a Structured Query Language (SQL) database or a "not only" SQL (NoSQL) database) and / or another storage mechanism (e.g., ElasticSearch, which provides the ability to store large data sets and perform queries quickly). Conductor 220 may provide interconnectivity by operating as a centralized point of communication for third-party solutions and / or applications.

[0092] Robot 230 is an execution agent that implements the workflow built in designer 210. One commercial example of some embodiments of robot(s) 230 is UiPath Robots™. In some embodiments, robot 230 installs Microsoft Windows Service Control Manager (SCM) management services by default. As a result, such robot 230 can open interactive Windows sessions under the local system account and may have Windows service rights.

[0093] In some embodiments, a robot 230 may be installed in user mode, meaning that for such a robot 230, it has the same rights as the user in which the given robot 230 is installed. This feature may also be available for high density (HD) robots, ensuring maximum utilization of each machine. In some embodiments, either type of robot 230 may be configured in an HD environment.

[0094] The robot 230 in some embodiments is divided into multiple components, each specialized for a particular automation task. The robot components in some embodiments include, but are not limited to, an SCM managed robot service, a user mode robot service, an executor, an agent, and a command line. The SCM managed robot service manages and monitors the Windows session and acts as a proxy between the conductor 220 and the execution host (i.e., the computing system on which the robot 230 executes). These services are entrusted with and manage the credentials of the robot 230. The console application is launched by the SCM under the local system.

[0095] The user-mode robot service in some embodiments manages and monitors the Windows session and acts as a proxy between the conductor 220 and the execution host. The user-mode robot service may be delegated and manage the credentials of the robot 230. If the SCM management robot service is not installed, a Windows application may be launched automatically.

[0096] An Executor may execute a given job under a Windows session (i.e., execute a workflow). An Executor may be aware of per-monitor dots-per-inch (DPI) settings. An Agent may be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. An Agent may be a client of a service. An Agent may ask to start or stop a job or change settings. A Command Line is a client of a service. A Command Line is a console application that can request the start of a job and wait for its output.

[0097] As explained above, the separation of the robot 230 components helps developers, support users, and computing systems to more easily execute, identify, and track what each component is doing. In this way, special behaviors can be configured for each component, such as setting different firewall rules for executors and services. The executor may always be aware of per-monitor DPI settings in some embodiments. As a result, workflows may execute at any DPI regardless of the configuration of the computing system on which the workflow was created. Also, in some embodiments, projects from the designer 210 may be made independent of the browser zoom level. For applications that are not DPI aware or are marked as not DPI aware intentionally, in some embodiments, DPI may be disabled.

[0098] The RPA system 200 in this embodiment is part of a hyper-automation system. A developer can use the designer 210 to build and test RPA robots that utilize AI / ML models deployed to the core hyper-automation system 240 (e.g., as part of its AI center). Such RPA robots can send inputs for execution of the AI / ML model(s) and receive outputs therefrom via the core hyper-automation system 240.

[0099] One or more of the robots 230 may be listeners, as described above. These listeners may provide information to the core hyper-automation system 240 about what users are doing when they use their computing systems. This information may then be used by the core hyper-automation system for process mining, task mining, task capture, and the like.

[0100] An assistant / chatbot 250 may be provided on a user computing system to allow the user to launch an RPA local robot. The assistant may be located, for example, in the system tray. The chatbot may have a user interface so that the user can see the chatbot's text. Alternatively, the chatbot may not have a user interface, run in the background, and listen to the user's speech using the computing system's microphone.

[0101] In some embodiments, data labeling may be performed by a user of the computing system on which the robot is running, or on another computing system to which the robot provides information. For example, if the robot invokes an AI / ML model to perform CV on an image for a VM user, but the AI / ML model does not correctly identify a button on the screen, the user may draw a rectangle around the misidentified or unidentified component and potentially provide text with the correct identification. This information may be provided to the core hyperautomation system 240 and then later used to train a new version of the AI / ML model.

[0102] Figure 3 is an architecture diagram illustrating a deployed RPA system 300, according to an embodiment of the present invention. In some embodiments, the RPA system 300 may be part of the RPA system 200 of Figure 2 and / or the hyper-automation system 100 of Figure 1. The deployed RPA system 300 may be a cloud-based system, an on-premise system, a desktop-based system, etc., providing enterprise-level, user-level, or device-level automation solutions for automation of different computing processes.

[0103] It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robotic application 310 includes an executor 312, an agent 314, and a designer 316. However, in some embodiments, the designer 316 may not be running on the same computing system as the executor 312 and the agent 314. The executor 312 is running a process. As shown in FIG. 3, multiple business projects may be running simultaneously. The agent 314 (e.g., a Windows service) is a single connection point for all the executors 312 in this embodiment. All messages in this embodiment are logged into the conductor 340, which further processes them via the database server 350, the AI / ML server 360, the indexer server 370, or any combination thereof. As described above with respect to FIG. 2, the executor 312 may be a robotic component.

[0104] In some embodiments, a robot represents an association between a machine name and a username. A robot may manage multiple executors simultaneously. In computing systems that support multiple interactive sessions running simultaneously (such as Windows Server 2012), multiple robots may run simultaneously, each running in a separate Windows session using a unique username. This is referred to as the HD Robot above.

[0105] The agent 314 is also responsible for transmitting the robot's status (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading required versions of packages to be fulfilled. Communication between the agent 314 and the conductor 340 is, in some embodiments, always initiated by the agent 314. In a notification scenario, the agent 314 may open a WebSocket channel that is later used by the conductor 340 to send commands (e.g., start, stop, etc.) to the robot.

[0106] Listener 330 monitors and records data relating to user interactions with the operation of the attended and / or unattended computing systems on which listener 330 resides. Listener 330 may be an RPA robot, part of an operating system, a downloadable application for the respective computing system, or any other software and / or hardware without departing from the scope of the invention. Indeed, in some embodiments, the logic of the listener is implemented partially or completely via physical hardware.

[0107] The server side includes a presentation layer (web application 342, Open Data Protocol (OData) Representational State Transfer (REST) ​​Application Programming Interface (API) endpoints 344, notifications and monitoring 346), a service layer (API implementation / business logic 348), and a persistence layer (database server 350, AI / ML server 360, indexer server 370). Conductor 340 includes web application 342, OData REST API endpoints 344, notifications and monitoring 346, and API implementation / business logic 348. In some embodiments, most actions that a user performs in the Conductor 340 interface (e.g., via browser 320) are performed by calling various APIs. Such actions may include, but are not limited to, launching jobs on a robot, adding / removing data in a queue, scheduling jobs to run unattended, etc., without departing from the scope of the invention. Web application 342 is the visual layer of the server platform. In this embodiment, web application 342 uses HyperText Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the present invention. A user interacts with web pages from web application 342, in this embodiment via browser 320, to perform various operations to control conductor 340. For example, a user may create robot groups, assign packages to robots, analyze per-robot and / or per-process logs, start and stop robots, etc.

[0108] In addition to the web application 342, the conductor 340 also includes a services layer that exposes an OData REST API endpoint 344. However, other endpoints may be included without departing from the scope of the invention. The REST API is consumed by both the web application 342 and an agent 314, which in this embodiment is a supervisor of one or more robots on a client computer.

[0109] The REST API of the present embodiment covers configuration, logging, monitoring, and queuing functionality. The configuration endpoint may be used in some embodiments to define and configure users, permissions, robots, assets, releases, and environments of an application. The logging REST endpoint may be used to log various information, such as errors, explicit messages sent by the robot, and other environment specific information. The deployment REST endpoint may be used by the robot to query the version of the package that should be executed when a start job command is used in the conductor 340. The queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to the queue, getting transactions from the queue, and setting the status of transactions.

[0110] Monitoring REST endpoints may monitor the web application 342 and the agents 314. Notification and monitoring API 346 may be a REST endpoint used to register agents 314, deliver configuration settings to agents 314, and send and receive notifications from the server and agents 314. Notification and monitoring API 346 may use WebSocket communications in some embodiments.

[0111] The service layer APIs, in some embodiments, may be accessed through configuration of appropriate API access paths, for example, based on whether the conductor 340 and the overall hyperautomation system have an on-premise or cloud-based deployment type. The API for the conductor 340 may provide custom methods to query statistics about various entities registered with the conductor 340. Each logical resource may be an OData entity, in some embodiments. In such entities, components such as robots, processes, queues, etc. may have properties, relationships, and behaviors. The conductor 340 APIs, in some embodiments, may be consumed by web applications 342 and / or agents 314 in two ways: by getting API access information from the conductor 340 or by registering an external application to use the OAuth flow.

[0112] The persistence layer includes three servers in this embodiment: a database server 350 (e.g., a SQL server), an AI / ML server 360 (e.g., a server providing AI / ML model provision services such as an AI center function), and an indexer server 370. The database server 350 in this embodiment stores configurations of robots, robot groups, associated processes, users, roles, schedules, etc. This information is managed via a web application 342 in some embodiments. The database server 350 may manage queues and queue items. In some embodiments, the database server 350 may store messages logged by the robots (in addition to or instead of the indexer server 370). The database server 350 may also store process mining, task mining, and / or task capture related data received, for example, from a listener 330 installed on the client side. Although no arrow is shown between the listener 330 and the database 350, it should be understood that the listener 330 can communicate with the database 350 and vice versa in some embodiments. This data may be stored in the form of PDDs, images, XAML files, etc. Listener 330 may be configured to intercept user actions, processes, tasks, and performance metrics on the respective computing system on which listener 330 resides. For example, listener 330 may record user actions (e.g., clicks, characters typed, locations, applications, active elements, times, etc.) on its respective computing system and then convert these into a suitable format to be provided to and stored in database server 350.

[0113] AI / ML Server 360 facilitates the incorporation of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options may make such capabilities accessible to non-data scientists. Deployed automation (e.g., RPA robots) may invoke AI / ML models from AI / ML Server 360. Performance of AI / ML models may be trained and improved using monitored and human-validated data. AI / ML Server 360 may schedule and execute training jobs to train new versions of AI / ML models.

[0114] The AI / ML server 360 may store data related to AI / ML models and ML packages for configuring various ML skills for users at development time. An ML skill, as used herein, is a pre-built and trained ML model for a process that may be used, for example, by automation. The AI / ML server 360 may also store data related to document understanding techniques and frameworks, algorithms, and software packages for various AI / ML capabilities, including, but not limited to, intent analysis, natural language processing (NLP), speech analysis, different types of AI / ML models, etc.

[0115] Optionally in some embodiments, the indexer server 370 stores and indexes information logged by the robots. In certain embodiments, the indexer server 370 may be disabled via a configuration setting. In some embodiments, the indexer server 370 uses ElasticSearch®, a full-text search engine from an open source project. Messages logged by the robots (e.g., using activities such as log messages or line writes) may be sent via logging REST endpoint(s) to the indexer server 370, where they are indexed for future use.

[0116] FIG. 4 is an architecture diagram illustrating relationships 400 between a designer 410, activities 420, 430, 440, 450, drivers 460, APIs 470, and AI / ML models 480, according to an embodiment of the present invention. Thus, a developer uses the designer 410 to develop a workflow to be performed by a robot. Various types of activities may be displayed to the developer in some embodiments. The designer 410 may be local to the user's computing system or remote thereto (e.g., accessed via a local web browser that interacts with a VM or a remote web server). The workflow may include user-defined activities 420, API-driven activities 430, AI / ML activities 440, and / or UI automation activities 450. The user-defined activities 420 and API-driven activities 440 interact with the application via their APIs. The user-defined activities 420 and / or the AI / ML activities 440 may, in some embodiments, invoke one or more AI / ML models 480, which may be located locally and / or remotely to the computing system on which the robot is operating.

[0117] In some embodiments, non-text visual components in an image may be identified, referred to herein as CV. CV may be performed at least in part by AI / ML model(s) 480. Some CV activities related to such components may include, but are not limited to, extracting text from segmented label data using OCR, fuzzy text matching, cropping segmented label data using ML, comparing extracted text in label data to ground truth data, etc. In some embodiments, the number of activities that may be implemented in user-defined activities 420 may be hundreds or thousands. However, any number and / or type of activities may be used without departing from the scope of the present invention.

[0118] UI automation activities 450 are a subset of specialized low-level activities that are written in low-level code and facilitate interactions with the screen. UI automation activities 450 facilitate these interactions via drivers 460 that enable the robot to interact with desired software. For example, drivers 460 may include operating system (OS) drivers 462, browser drivers 464, VM ​​drivers 466, enterprise application drivers 468, etc. In some embodiments, one or more AI / ML models 480 may be used by UI automation activities 450 to perform interactions with the computing system. In certain embodiments, AI / ML models 480 may augment drivers 460 or replace them entirely. Indeed, in certain embodiments, drivers 460 are not included.

[0119] Driver 460 may interact with the OS at a low level, such as looking for hooks, monitoring keys, etc., via OS driver 462. Driver 460 may facilitate integration with Chrome®, IE®, Citrix®, SAP®, etc. For example, a "click" activity plays the same role in these different applications via driver 460.

[0120] FIG. 5 is an architecture diagram illustrating a computing system 500 configured to provide a design-time AI / ML recommendation engine for RPA, according to an embodiment of the present invention. In some embodiments, the computing system 500 may be one or more computing systems depicted and / or described herein. In certain embodiments, the computing system 500 may be part of a hyper-automation system such as that shown in FIG. 1 and FIG. 2. The computing system 500 includes a bus 505 or other communication mechanism for communicating information and a processor(s) 510 coupled to the bus 505 for processing information. The processor(s) 510 may be any type of general or application-specific processor, including a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. The processor(s) 510 may also have multiple processing cores, and at least some of the cores may be configured to perform specific functions. In some embodiments, multiple parallel processing may be used. In certain embodiments, at least one processor(s) 510 may be a neuromorphic circuit that includes processing elements that mimic biological neurons. In some embodiments, a neuromorphic circuit may not require typical components of a von Neumann computing architecture.

[0121] The computing system 500 further includes a memory 515 for storing information and instructions executed by the processor(s) 510. The memory 515 may be comprised of a random access memory (RAM), a read-only memory (ROM), a flash memory, a cache, a static storage device such as a magnetic or optical disk, or other types of non-transitory computer-readable media, or any combination thereof. The non-transitory computer-readable media may be any available media accessible by the processor(s) 510 and may include volatile media, non-volatile media, or both. Also, the media may be removable, non-removable, or both. The computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communication network via wireless and / or wired connections. In some embodiments, the communication device 520 may include one or more antennas that are a single antenna, an array of antennas, phased antennas, switched antennas, beamforming antennas, beam-steering antennas, combinations thereof, and / or any other antenna configuration without departing from the scope of the present invention.

[0122] The processor(s) 510 are further coupled to a display 525 via bus 505. Any suitable display device and haptic I / O may be used without departing from the scope of the invention.

[0123] A keyboard 530 and cursor control device 535, such as a computer mouse, touchpad, and the like, are further coupled to the bus 505 to allow a user to interface with the computing system 500. However, in certain embodiments, a physical keyboard and mouse may not be present and the user may interact with the device solely through the display 525 and / or touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. In certain embodiments, no physical input devices and / or displays are present. For example, a user may interact with the computing system 500 remotely via another computing system in communication with it, or the computing system 500 may operate autonomously.

[0124] The memory 515 stores software modules that provide functionality when executed by the processor(s) 510. The modules include an operating system 540 for the computing system 500. The modules further include an automated AI / ML recommendation engine model 545 configured to perform all or a portion of the AI / ML processes described herein or derivatives thereof. The computing system 500 may include one or more additional functional modules 550 that include additional functionality.

[0125] Those skilled in the art will appreciate that the "system" may be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a mobile phone, a tablet computing device, a quantum computing system, or any other suitable computing device or combination of devices without departing from the scope of the present invention. Presenting the above-described functions as being performed by a "system" is not intended to limit the scope of the present invention in any way, but rather to provide an example of many embodiments of the present invention. Indeed, the methods, systems, and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing techniques, including cloud computing systems. The computing system may be part of or otherwise accessible in a local area network (LAN), a mobile communications network, a satellite communications network, the Internet, a public or private cloud, a hybrid cloud, a server farm, any combination thereof, and the like. Any local or distributed architecture may be used without departing from the scope of the present invention.

[0126] It should be noted that some of the system features described herein are presented as modules to better emphasize implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integrated (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, and the like.

[0127] Modules may also be implemented at least in part in software for execution by various types of processors. For example, an identified unit of executable code may include one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, procedure, or function. Nevertheless, executable identified modules need not be physically located together, but may include separate instructions stored in different locations, which when logically combined, comprise the modules, to accomplish the purpose stated for the modules. Furthermore, modules may be stored on non-transitory computer readable media, such as, for example, hard disk drives, flash devices, RAM, tape, and / or any other non-transitory computer readable media used to store data without departing from the scope of the present invention.

[0128] Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed across several different code segments, different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within a module, and may be embodied in any suitable form and organized within any suitable type of data structure. Operational data may be collected as a single data set, or may be distributed in different locations across different storage devices, or may exist, at least in part, simply as electronic signals on a system or network.

[0129] Various types of AI / ML models may be trained and deployed without departing from the scope of the present invention. For example, Figure 6A illustrates an example of a neural network 600 trained to provide suggestions for a next activity or sequence of activities, according to an embodiment of the present invention.

[0130] Neural network 600 includes multiple hidden layers. Although both DLNNs and shallow learning neural networks (SLNNs) typically have multiple layers, SLNNs may have only one or two layers, typically fewer than DLNNs. Typically, the architecture of a neural network, as in neural network 600, includes an input layer, multiple hidden layers, and an output layer.

[0131] DLNNs often have many layers (10, 50, 200, etc.), with subsequent layers typically reusing features from previous layers to compute more complex and general functions. SLNNs, on the other hand, tend to have only a few layers and train relatively quickly, since expert features are pre-created from raw data samples; however, feature extraction is laborious. DLNNs, on the other hand, usually do not require expert features, but they take longer to train and tend to have more layers.

[0132] In both approaches, the layers are trained simultaneously on the training set and are typically checked for overfitting on a separate cross-validation set. Both techniques produce excellent results, and there is considerable enthusiasm for both approaches. The optimal size, shape, and number of individual layers depend on the problem being addressed by each neural network.

[0133] Returning to Figure 6A, the RPA workflow, one or more application screenshots(s), the graphical elements to be detected on the screen using CV, activity parameters, etc. are provided as the input layer and fed as inputs to the J neurons in hidden layer 1. In this example, all of these inputs are fed to each neuron, however, a variety of architectures are possible that may be used individually or in combination, including, but not limited to, feedforward networks, radial basis networks, deep feedforward networks, deep convolutional inverse graphics networks, convolutional neural networks, recurrent neural networks, artificial neural networks, long / short-term memory networks, gated recurrent unit networks, generative adversarial networks, liquid state machines, autoencoders, variational autoencoders, denoising autoencoders, sparse autoencoders, extreme learning machines, echo state networks, Markov chains, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, deep residual networks, Kohonen networks, deep belief networks, deep convolutional networks, support vector machines, neural Turing machines, or any other suitable type or combination of neural networks without departing from the scope of the invention.

[0134] Hidden layer 2 receives input from hidden layer 1, hidden layer 3 receives input from hidden layer 2, and so on for all hidden layers until the last hidden layer provides its output as an input for the output layer. Although multiple suggestions are shown herein as outputs, in some embodiments, only a single output suggestion is provided. In certain embodiments, the suggestions are ranked based on confidence scores.

[0135] It should be noted that the numbers of neurons I, J, K, and L are not necessarily equal. Thus, any desired number of layers may be used in a given layer of neural network 600 without departing from the scope of the present invention. Indeed, in certain embodiments, the types of neurons in a given layer may not all be the same.

[0136] The neural network 600 is trained to assign confidence score(s) to appropriate outputs. To reduce inaccurate predictions, in some embodiments, only those results with confidence scores equal to or greater than a confidence threshold may be provided. For example, if the confidence threshold is 80%, outputs with confidence scores above this amount may be used and the rest may be ignored.

[0137] Note that neural networks are probabilistic constructs that typically have a confidence score(s). This can be a score that the AI / ML model learned based on how often similar inputs were correctly identified during training. Some common types of confidence scores include a decimal number between 0 and 1 (which can also be interpreted as a confidence percentage), a numeric value between negative infinity and positive infinity, a set of formulas (e.g., "low", "medium", and "high"), etc. To obtain more accurate confidence scores, various post-processing calibration techniques such as temperature scaling, batch normalization, weight decay, and negative log-likelihood (NLL) can also be used.

[0138] The "neurons" of a neural network are algorithmically implemented as mathematical functions, usually based on the functionality of biological neurons. Neurons receive weighted inputs and have a sum and activation function that governs whether they pass on an output to the next layer. This activation function may be a nonlinear, thresholded activity function where nothing happens if the value is below a threshold, and responds linearly when the function exceeds the threshold (i.e., rectified linear unit (ReLU) nonlinearity). Sum and ReLU functions are used in deep learning because real neurons may have roughly similar activity functions. Through linear transformations, information may be subtracted, added, etc. In essence, neurons act as gating functions that pass on outputs to the next layer governed by their underlying mathematical function. In some embodiments, different functions may be used for at least some neurons.

[0139] JPEG2025010481000002.jpg84144

[0140] JPEG2025010481000003.jpg47144

[0141] JPEG2025010481000004.jpg30132

[0142] In this case, neuron 610 is a single-layer perceptron. However, any suitable neuron type or combination of neuron types may be used without departing from the scope of the present invention. It should also be noted that the range of values ​​of the activation function weights and / or output value(s) may vary in some embodiments without departing from the scope of the present invention.

[0143] A goal, or "reward function," is often employed. The reward function guides the search of the state space, exploring intermediate transitions and steps with both short-term and long-term rewards to attempt to achieve the goal (e.g., finding the most accurate answer to a user query based on relevant metrics). During training, various labeled data are fed through the neural network 600. Successful identification strengthens the weights of the inputs to the neurons, while failure to identify weakens those weights. A cost function such as mean squared error (MSE) or gradient descent may be used to ensure that slightly incorrect predictions are punished much less than those that are significantly incorrect. If the performance of the AI / ML model does not improve after a certain number of training iterations, the data scientist may change the reward function, correct the incorrect predictions, etc.

[0144] Backpropagation is a technique for optimizing synaptic weights in feedforward neural networks. Backpropagation can be used to "pop up" the hidden layers of a neural network to see how much loss each node is carrying, and then update the weights to minimize the loss, giving lower weights to nodes with higher error rates and vice versa. In other words, backpropagation allows data scientists to iteratively adjust the weights to minimize the difference between the actual output and the desired output.

[0145] Backpropagation algorithms are mathematically based on optimization theory. In supervised learning, training data with known outputs is passed through a neural network and an error is calculated from the known target output using a cost function, which gives the backpropagation error. An error is calculated at the output and this error is converted into a modification of the network weights that minimizes the error.

[0146] JPEG2025010481000005.jpg71144

[0147] JPEG2025010481000006.jpg23144

[0148] JPEG2025010481000007.jpg123144

[0149] JPEG2025010481000008.jpg106116

[0150] JPEG2025010481000009.jpg83144

[0151] The AI / ML model may be trained over multiple epochs until it reaches a good level of accuracy (e.g., 97% or higher using an F2 or F4 threshold for detection, approximately 2,000 epochs). This accuracy level may be determined in some embodiments using an F1 score, an F2 score, an F4 score, or any other suitable technique that does not depart from the scope of the present invention. Once trained on training data, the AI / ML model may be tested on a set of evaluation data that the AI / ML model has not encountered before. This helps to ensure that the AI / ML model does not "overfit" such that it performs well on the training data but not on other data.

[0152] In some embodiments, it may not be known what level of accuracy an AI / ML model is capable of achieving. Thus, if the accuracy of the AI / ML model starts to degrade when analyzing the evaluation data (i.e., the model performs well on the training data but starts to perform poorly on the evaluation data), the AI / ML model can undergo further epochs of training on the training data (and / or new training data). In some embodiments, the AI / ML model is deployed only when the accuracy reaches a certain level or when the accuracy of the trained AI / ML model is superior to the existing deployed AI / ML model. In certain embodiments, a collection of trained AI / ML models can be used to accomplish a task. For example, one model can be trained to suggest the next activity or sequence of activities, another model can perform CV, yet another model can attempt run-time repair if automation fails, etc.

[0153] In some embodiments, a transformer network such as SentenceTransformers™, a state-of-the-art Python™ framework for sentence, text, and image embedding, can be used. Such a transformer network learns associations of words and phrases with both high and low scores. This trains an AI / ML model to determine what is close to the input and what is not, respectively. Rather than using only word / phrase pairs, the transformer network may also use field lengths and field types.

[0154] In some embodiments, natural language processing (NLP) techniques such as word2vec, BERT, GPT-3, ChatGPT, etc., can be used to facilitate semantic understanding and provide more accurate and human-like answers. Other techniques, such as clustering algorithms, can be used to find similarities between groups of elements. Clustering algorithms can include, but are not limited to, density-based algorithms, distribution-based algorithms, centroid-based algorithms, and hierarchical-based algorithms, such as K-means clustering algorithms, DBSCAN clustering algorithms, Gaussian Mixture Models (GMM) algorithms, and Balanced Iterative Reduction and Clustering with Hierarchy (BIRCH) algorithms. Such techniques can also be useful for classification.

[0155] FIG. 7 is a flow chart illustrating a process 700 for training an AI / ML model(s), according to an embodiment of the present invention. In some embodiments, the AI / ML model(s) may be generative AI models, as described above. The neural network architecture of an AI / ML model typically includes multiple layers of neurons, including an input layer, an output layer, and a hidden layer. See, for example, FIGS. 6A and 6B. The input layer receives text-based input(s) and the output layer generates text-based response(s). Intermediate hidden layers process the input data and generate intermediate representations of the inputs that are used to generate the outputs. These hidden layers may include various types of neurons, such as convolutional neurons, recurrent neurons, and / or transformer neurons.

[0156] The training process begins at 710 by providing the RPA workflow, screenshots, graphical elements (if the automation is UI automation), and activity parameters (e.g., inputs and outputs), with or without labels. For UI automation, one or more graphs may be used that provide relationships between descriptors, graphical elements, screens, versions, applications, etc. The AI / ML model is then trained for multiple epochs at 720, and the results are reviewed at 730. While different types of AI / ML models may be used, LLMs and other generative AI models are typically trained using a process called "supervised learning," described above. Supervised learning involves providing the model with a large dataset, which the model uses to learn the relationships between inputs and outputs. During the training process, the model adjusts the weights and biases of neurons in the neural network to minimize the difference between the predicted outputs in the training dataset and the actual outputs.

[0157] One aspect of the model in some embodiments is the use of transfer learning. For example, transfer learning may utilize a pre-trained model, such as ChatGPT, that is fine-tuned for a particular task or domain in step 720. This allows the model to leverage already learned knowledge from the pre-training phase and adapt it to a particular application through the training phase in step 720.

[0158] The pre-training phase involves training the model based on an initial set of training data, which may be more general. In this phase, the model learns relationships in the data. The fine-tuning phase (e.g., in some embodiments, performed in addition to or instead of the initial training phase during step 720, when the pre-trained model is used as the initial basis for the final model) adapts the pre-trained model to a specific task or domain by training the model on a smaller data set specific to the task. For example, in some embodiments, the model may focus on UI automation and utilize an object repository. This may help the model to more accurately suggest the next activity or sequence of activities than a model trained on multiple or all types of automation. Fine-tuning may allow the model to learn the nuances of a task, such as specific vocabulary and syntax, specific graphical characteristics, specific data formats, etc., without requiring as much data as would be required to train the model from scratch. By leveraging the knowledge learned in the pre-training phase, the fine-tuned model may achieve state-of-the-art performance on a specific task, even with relatively little additional training data.

[0159] If the AI / ML model does not meet the desired confidence threshold at 740, training data is supplemented and / or the reward function is modified to help the AI / ML model better achieve its objective at 750, and the process returns to step 720. If the AI / ML model meets the confidence threshold at 740, the AI / ML model is tested against evaluation data at 760 to ensure that the AI / ML model generalizes well and that the AI / ML model does not over-fit with respect to the training data. The evaluation data includes information that the AI / ML model has not previously processed. If the confidence threshold is met at 770 for the evaluation data, the AI / ML model is deployed at 780. If not, the process returns to step 750 and the AI / ML model is further trained.

[0160] FIG. 8 is a basic example of a graph 800 illustrating relationships between application elements, according to an embodiment of the present invention. The graph 800 includes relationships between applications and their versions, versions and screens, screens and graphical elements (GEs), and graphical elements and their descriptors. In some embodiments, there may be different connections between these nodes. For example, the descriptors and / or graphical elements may be connected to the respective version(s) in which they appear. This information may help train AI / ML models to more accurately suggest UI automation activities and sequences of activities.

[0161] FIG. 9 is a flow chart illustrating a process 900 of a design-time AI / ML recommendation engine for RPA, according to an embodiment of the present invention. If transfer learning is used, the process may begin with pre-training one or more AI / ML models at 910. If not, or after step 910, the AI / ML model(s) are trained at 920 for the desired purpose. For example, the AI / ML model(s) may be trained to suggest a next activity based on the content of one or more activities in the RPA workflow, suggest a sequence of next activities, suggest changes to at least one parameter of one or more activities in the RPA workflow, or any combination thereof. Other models, such as CV models, models specialized for a particular type of automation, etc., may also be trained.

[0162] In some embodiments, at least one of the one or more AI / ML models is trained using a context based on a previously developed RPA workflow. In certain UI automation embodiments, at least one of the one or more AI / ML models is trained using information about applications, application versions, application screens, and screen graphical elements from an object repository. In some UI automation embodiments, at least one of the one or more AI / ML models is trained using one or more graphs that include relationships between applications, application versions, application screens, and screen graphical elements from an object repository. In certain embodiments, at least one of the one or more AI / ML models trained using the one or more graphs is trained to recognize ontological associations from the one or more graphs.

[0163] After training the AI / ML model(s), information about the RPA workflow under development is provided to at least one of the one or more AI / ML models at 930. The AI / ML model(s) then provide suggestions regarding the next activity, the sequence of next activities, changes to the proposed parameters, or any combination thereof, which are received at 940. In some embodiments, confidence score(s) for the suggestions are also provided as output. In certain embodiments, the proposed next activity, the proposed sequence of next activities, changes to the proposed parameters, or any combination thereof is based on the context of the RPA workflow in its current development state.

[0164] A user of the RPA designer application is then notified at 950 (1) of the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof, or (2) the RPA workflow is automatically modified to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof into the RPA workflow being developed. In some embodiments, the automatic modification of the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof is performed in response to a confidence score exceeding an automatic insertion threshold. In some embodiments, at 960, a next task to perform for the RPA workflow is suggested based on output from at least one of the one or more AI / ML models. In certain embodiments, the proposed next task is automatically performed when a confidence score associated with the task suggestion exceeds an automatic performance threshold.

[0165] In some embodiments, workflow suggestions and / or repairs may be performed iteratively. If the AI / ML model detects a problem with the workflow during development, at 970, the AI / ML model may automatically repair the workflow by modifying activity parameters, replacing the activity with another activity appropriate for the task, adding one or more additional activities required to make the given activity work, etc. If further development is required at 980, the process returns to step 930 (e.g., after the user creates a new activity, after the user creates a sequence of activities, after a predetermined amount of time has passed, after a certain number of activities have been created, after the user clicks a review button, etc.). In this way, problems with the RPA workflow can be addressed iteratively, the user can develop the workflow more quickly, and the effectiveness and accuracy of the workflow can be increased.

[0166] 10 is a flow chart illustrating a process 1000 for performing RPA error avoidance or self-healing of an RPA robot at runtime, according to an embodiment of the invention. The process begins at 1010 with executing an automation with the RPA robot. In some embodiments, the RPA robot or an AI / ML model invoked by the RPA robot may search for changes to an application associated with the automation. If the version of the application has changed or a different application is used for a task associated with the automation, this may indicate that the automation needs to be modified.

[0167] If the application or its version is modified in a way that affects the automation, the RPA robot either avoids the failure at 1020 or, alternatively, the RPA robot determines that an activity in the RPA workflow has failed at 1020. In the case of avoidance, the RPA robot invokes an AI / ML model to determine changes to be made to the workflow, makes the changes and generates a new automation to replace the previous one incorporating these changes. For example, if the RPA robot determines that an activity has failed, the failure may be detected by a graphical element associated with the activity not being found, the activity not completing and moving to the next activity within a period of time, an expected screen not being displayed, etc. If the issue is resolved by avoidance at 1030 or the type of error is such that repair should not be attempted, the RPA automation is performed in the case of avoidance at 1040 or, if repair should not be attempted, the RPA automation is terminated and a notification is provided to the user (e.g., via the RPA robot, robot tray application, etc.).

[0168] However, if an error occurs and the error is of a type that should be attempted to be repaired at 1030, information is provided at 1050 to one or more AI / ML models trained to perform self-repair of the RPA robot. This information may include information about the failed activity (e.g., its parameters, UI descriptors (if used), results of previous activities, etc.), screenshots, graphical element information from the CV model, etc. Proposed fix(es) are received at 1060 from the one or more AI / ML models based on the input information from step 1050.

[0169] The RPA robot, RPA designer application, or another application then attempts the fix(es) at 1070. If the fix(es) at 1080 are successful, the RPA workflow is repaired at 1090, and the process returns to performing the repaired RPA automation at 1010. In some embodiments, the RPA automation is resumed from the activity where the failure occurred. In certain embodiments, the RPA automation is re-performed from the beginning. However, if the attempted fix(es) at 1080 fail, the process proceeds to step 1040.

[0170] The process steps performed in Figures 9 and 10 may be performed by a computer program encoding instructions to a processor(s) to perform at least a portion of the process(es) described in Figures 9 and 10 according to an embodiment of the present invention. The computer program may be stored on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, a RAM, a tape, and / or any other such medium or combination of media used to store data. The computer program may include coded instructions for controlling a processor(s) of a computing system (e.g., processor(s) 510 of computing system 500 of Figure 5) to implement all or a portion of the process steps described in Figures 9 and 10, which may also be stored on a computer-readable medium.

[0171] The computer program may be implemented in hardware, software, or a hybrid implementation. The computer program may be composed of modules in operable communication with each other and designed to send information or instructions to a display. The computer program may be configured to run on a general purpose computer, an ASIC, or any other suitable device.

[0172] It will be readily understood that the components of the various embodiments of the present invention, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention, as represented in the accompanying figures, is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention.

[0173] The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, reference throughout this specification to "certain embodiments," "some embodiments," or similar language means that the particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Thus, the appearance of "certain embodiments," "some embodiments," "other embodiments," or similar language throughout this specification does not necessarily refer to the same group of all embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0174] It should be noted that references to features, advantages, or similar language throughout this specification do not imply that all of the features and advantages that may be realized in the present invention should be in any single embodiment of the present invention, or that any embodiment of the present invention. Rather, language referring to features and advantages is understood to mean that the particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussions of features and advantages throughout this specification, as well as similar language, may, but do not necessarily, refer to the same embodiment.

[0175] Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the invention may be practiced without the specific features or advantages of one or more particular embodiments. In other instances, additional features and advantages may be recognized in certain embodiments, although they may not be present in all embodiments of the invention.

[0176] Those of ordinary skill in the art will readily appreciate that the invention as described above can be practiced using steps in a different order and / or with hardware elements in different configurations than those disclosed. Thus, while the invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will become apparent while remaining within the spirit and scope of the invention. Accordingly, reference should be made to the appended claims to determine the scope of the invention.

Claims

1. One or more non-transitory computer readable media having stored thereon one or more computer programs, the one or more computer programs being configured to cause at least one processor to: providing information about an RPA workflow developed in a Robotic Process Automation (RPA) designer application, the RPA workflow including one or more activities, to one or more artificial intelligence (AI) / machine learning (ML) models, the one or more AI / ML models being trained to: suggest a next activity, suggest a sequence of next activities, suggest a change to at least one parameter of the one or more activities in the RPA workflow, or any combination thereof, based on content of the one or more activities in the RPA workflow; receiving output from the one or more AI / ML models including the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or any combination thereof; One or more non-transitory computer readable media configured to notify a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or combinations thereof, or automatically modify the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or combinations thereof into the RPA workflow.

2. 10. The one or more non-transitory computer-readable media of claim 1, wherein the one or more AI / ML models are configured to provide a confidence score for the proposed next sequence of activities, the proposed changes to parameters, or a combination thereof.

3. 10. The one or more non-transitory computer readable media of claim 1, wherein an automated modification of the RPA workflow to incorporate the proposed next activity, the sequence of the proposed next activities, a change to the proposed parameters, or a combination thereof is performed in response to a confidence score exceeding an automatic insertion threshold.

4. 10. The one or more non-transitory computer-readable media of claim 1, wherein the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or combinations thereof are based on a context of the RPA workflow in a current development state.

5. 10. The one or more non-transitory computer-readable media of claim 1, wherein at least one of the one or more AI / ML models is trained using a context based on a previously developed RPA workflow.

6. The one or more computer programs further comprise:

10. The one or more non-transitory computer-readable media of claim 1 , configured to suggest a next task to execute for the RPA workflow based on output from at least one of the one or more AI / ML models.

7. The one or more computer programs further comprise:

7. The one or more non-transitory computer-readable media of claim 6, configured to automatically execute the proposed next task if a confidence score associated with the proposed next task exceeds an automatic performance threshold.

8. The RPA workflow relates to UI automation, 10. The one or more non-transitory computer-readable media of claim 1, wherein at least one of the one or more AI / ML models is trained using information about an application, a version of the application, screens of the application, and graphical elements of the screens from an object repository.

9. The RPA workflow relates to UI automation, 10. The one or more non-transitory computer-readable media of claim 1, wherein at least one of the one or more AI / ML models is trained using one or more graphs including relationships between applications, versions of the applications, screens of the applications, and graphical elements of the screens from an object repository.

10. 10. The one or more non-transitory computer-readable media of claim 9, wherein at least one of the one or more AI / ML models trained using the one or more graphs is trained to recognize ontological associations from the one or more graphs.

11. The one or more computer programs further comprise: Detecting one or more problems in the RPA workflow; 10. The one or more non-transitory computer-readable media of claim 1 configured to automatically repair the RPA workflow.

12. 12. The one or more non-transitory computer-readable media of claim 11, wherein the automatic repair includes modifying activity parameters, replacing the activity with another activity that is more suitable for the task, adding one or more additional activities to the activity work, or any combination thereof.

13. 12. The one or more non-transitory computer-readable media of claim 11, wherein the automated repair of the RPA workflow is performed iteratively during development of the RPA workflow.

14. a memory for storing computer program instructions; and at least one processor configured to execute the computer program instructions, the computer program instructions comprising: providing information about an RPA workflow developed in a Robotic Process Automation (RPA) designer application, the RPA workflow including one or more activities, to one or more artificial intelligence (AI) / machine learning (ML) models, the one or more AI / ML models being trained to: suggest a next activity, suggest a sequence of next activities, suggest a change to at least one parameter of the one or more activities in the RPA workflow, or any combination thereof, based on content of the one or more activities in the RPA workflow; receiving output from the one or more AI / ML models including the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or any combination thereof; configured to notify a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof, or to automatically modify the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof into the RPA workflow; One or more computing systems, wherein the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof, is based on the context of the RPA workflow in its current development state.

15. 15. The one or more computing systems of claim 14, wherein automated modification of the RPA workflow to incorporate the proposed next activity, the sequence of the proposed next activities, the change to the proposed parameters, or a combination thereof is performed in response to a confidence score exceeding an automatic insertion threshold.

16. 15. The one or more computing systems of claim 14, wherein at least one of the one or more AI / ML models is trained using a context based on a previously developed RPA workflow.

17. The computer program instructions further include causing the at least one processor to:

15. The one or more computing systems of claim 14, configured to suggest a next task to execute for the RPA workflow based on output from at least one of the one or more AI / ML models.

18. The computer program instructions further include causing the at least one processor to:

20. The one or more computing systems of claim 17, configured to automatically execute the proposed next task if a confidence score associated with the proposed next task exceeds an automatic performance threshold.

19. The RPA workflow relates to UI automation, 15. The one or more computing systems of claim 14, wherein at least one of the one or more AI / ML models is trained using information about an application, a version of the application, a screen of the application, and graphical elements of the screens from an object repository.

20. The RPA workflow relates to UI automation, 15. The one or more computing systems of claim 14, wherein at least one of the one or more AI / ML models is trained using one or more graphs including relationships between applications, versions of the applications, screens of the applications, and graphical elements of the screens from an object repository.

21. 21. The one or more computing systems of claim 20, wherein at least one of the one or more AI / ML models trained using the one or more graphs is trained to recognize ontological associations from the one or more graphs.

22. The one or more computer programs further comprise: Detecting one or more problems in the RPA workflow; configured to automatically repair the RPA workflow; 15. The one or more computing systems of claim 14, wherein the automatic repair includes modifying activity parameters, replacing the activity with another activity that is more suitable for the task, adding one or more additional activities to the activity work, or any combination thereof.

23. providing, by a computing system, information about a robotic process automation (RPA) workflow developed in a RPA designer application, the RPA workflow including one or more activities, to one or more artificial intelligence (AI) / machine learning (ML) models, the one or more AI / ML models being trained to: suggest a next activity, suggest a sequence of next activities, suggest a change to at least one parameter of the one or more activities in the RPA workflow, or any combination thereof, based on content of the one or more activities in the RPA workflow; receiving, by the computing system, output from the one or more AI / ML models including the proposed next activity, a sequence of the proposed next activities, changes to the proposed parameters, or any combination thereof; notifying a user of the RPA designer application of the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof, by the computing system, or automatically modifying the RPA workflow to incorporate the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof into the RPA workflow; and when a confidence score associated with the proposed next task exceeds an automatic performance threshold, the computing system suggests, by the computing system, a next task to execute for the RPA workflow based on output from at least one of the one or more AI / ML models, or automatically executes, by the computing system, the proposed next task.

24. 24. The computer-implemented method of claim 23, wherein an automated modification of the RPA workflow to incorporate the proposed next activity, the sequence of the proposed next activities, a change to the proposed parameters, or a combination thereof is performed in response to the confidence score exceeding an auto-insertion threshold.

25. 24. The computer-implemented method of claim 23, wherein the proposed next activity, the proposed sequence of next activities, the proposed changes to parameters, or a combination thereof is based on a context of the RPA workflow in a current development state.

26. 24. The computer-implemented method of claim 23, wherein at least one of the one or more AI / ML models is trained using a context based on a previously developed RPA workflow.

27. The RPA workflow relates to UI automation, 24. The computer-implemented method of claim 23, wherein at least one of the one or more AI / ML models is trained using information about an application, a version of the application, screens of the application, and graphical elements of the screens from an object repository.

28. The RPA workflow relates to UI automation, 24. The computer-implemented method of claim 23, wherein at least one of the one or more AI / ML models is trained using one or more graphs including relationships between applications, versions of the applications, screens of the applications, and graphical elements of the screens from an object repository.

29. 30. The computer-implemented method of claim 28, wherein at least one of the one or more AI / ML models trained using the one or more graphs is trained to recognize ontological associations from the one or more graphs.

30. moreover, Detecting, by the computing system, one or more problems in the RPA workflow; automatically repairing, by the computing system, the RPA workflow; 24. The computer-implemented method of claim 23, wherein the automatic repair includes modifying activity parameters, replacing the activity with another activity that is more suitable for the task, adding one or more additional activities to the activity work, or any combination thereof.

Images