Artificial intelligence / machine learning model training and recommendation engine for robotic process automation

Abstract

The invention relates to an artificial intelligence / machine learning model training and recommendation engine for robotic process automation. An artificial intelligence (AI) / machine learning (ML) recommendation engine for robotic process automation (RPA) is disclosed. The AI / ML model may be trained to provide recommendations for a next activity, a sequence of next activities, and / or modifications to parameters of one or more existing activities included during RPA workflow development. These recommendations may be based on the context in which the user is in the RPA workflow. For user interface (UI) automation, an AI / ML model may be linked to an object repository and trained to make recommendations therefrom. The AI / ML model may also be trained to generate new UI descriptors for the object repository.

Description

Technical Field

[0001] The present invention relates generally to Robotic Process Automation (RPA), and more specifically to an Artificial Intelligence (AI) / Machine Learning (ML) model training and recommendation engine for RPA. Background Art

[0002] Traditionally, when developing a workflow for RPA automation, the RPA developer configures the steps of the workflow to achieve the intended task. However, many steps are used in multiple automations. In addition, activities, descriptors, and other objects used for automation may already exist. For RPA automation in a user interface (UI), developers typically build descriptors (e.g., selectors) for each UI action and typically generate default descriptors. However, building reliable descriptors or other components of RPA activities may consume developers' time for troubleshooting and trial-and-error development, which may not be efficient. There is also a lack of reusability. In addition, when systems and applications are changed or upgraded, the process upgrade path may not be simple. Therefore, it may be beneficial to develop improved methods for RPA automation. Summary of the invention

[0003] Certain embodiments of the present invention may provide solutions to problems and needs that have not yet been fully identified, understood, or addressed by current RPA technology in the field. For example, some embodiments of the present invention relate to a design-time AI / ML recommendation engine for RPA.

[0004] In an embodiment, one or more non-transitory computer-readable media store one or more computer programs. The one or more computer programs are configured to cause at least one processor to provide information related to an RPA workflow to one or more artificial intelligence AI / machine learning ML models, which is developed in an RPA designer application and includes one or more activities. The one or more AI / ML models are trained to suggest the next activity in the RPA workflow, the next activity sequence in the RPA workflow, the modification of the parameters of at least one of the one or more activities in the RPA workflow, or any combination thereof based on the content of one or more activities in the RPA workflow. The one or more computer programs are also configured to cause at least one processor to receive output from one or more AI / ML models, the output including the suggested next activity, the suggested next activity sequence, the suggested modification of parameters, or any combination thereof. The one or more computer programs are also configured to cause at least one processor to notify a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof, or to automatically modify the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof into the RPA workflow.

[0005] In another embodiment, one or more computing systems include a memory storing computer program instructions and at least one processor configured to execute the computer program instructions. The computer program instructions are configured to cause at least one processor to provide information related to an RPA workflow to one or more AI / ML models, the RPA workflow being developed in an RPA designer application and including one or more activities. The one or more AI / ML models are trained to suggest the next activity in the RPA workflow, suggest the next sequence of activities in the RPA workflow, suggest a modification of a parameter of at least one of the one or more activities in the RPA workflow, or any combination thereof based on the content of one or more activities in the RPA workflow. The computer program instructions are also configured to cause at least one processor to receive output from the one or more AI / ML models, the output comprising the suggested next activity, the suggested next sequence of activities, the suggested modification of parameters, or any combination thereof. The computer program instructions are further configured to cause at least one processor to notify a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof, or automatically modify the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof into the RPA workflow. The suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof is based on the context of the RPA workflow in a current state of development.

[0006] In another embodiment, a computer-implemented method includes providing, by a computing system, information related to an RPA workflow to one or more AI / ML models, the RPA workflow being developed in an RPA designer application and including one or more activities. The one or more AI / ML models are trained to suggest a next activity in the RPA workflow, a next sequence of activities in the RPA workflow, a modification of a parameter of at least one of the one or more activities in the RPA workflow, or any combination thereof based on the content of the one or more activities in the RPA workflow. The computer-implemented method also includes receiving, by a computing system, an output from the one or more AI / ML models, the output including the suggested next activity, the suggested next sequence of activities, the suggested modification of parameters, or any combination thereof. The computer-implemented method also includes notifying, by a computing system, a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification of parameters, or a combination thereof, or automatically modifying the RPA workflow to merge the suggested next activity, the suggested next sequence of activities, the suggested modification of parameters, or a combination thereof into the RPA workflow. In addition, the computer-implemented method includes suggesting, by the computing system, a next task to be performed for the RPA workflow based on an output from at least one of the one or more AI / ML models, or automatically performing, by the computing system, the suggested next task when a confidence score associated with the suggested next task exceeds an automatic performance threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to easily understand the advantages of certain embodiments of the present invention, the invention briefly described above will be described in more detail with reference to the specific embodiments illustrated in the accompanying drawings. Although it should be understood that these drawings depict only typical embodiments of the invention and are therefore not to be considered as limiting the scope thereof, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings. In which:

[0008] Figure 1 is an architectural diagram illustrating a hyper-automation system according to an embodiment of the present invention.

[0009] Figure 2 is an architectural diagram illustrating an RPA system according to an embodiment of the present invention.

[0010] Figure 3 is an architectural diagram illustrating an RPA system deployed according to an embodiment of the present invention.

[0011] Figure 4 is an architectural diagram illustrating the relationship between designers, activities, and drivers according to an embodiment of the present invention.

[0012] Figure 5is an architectural diagram illustrating a computing system configured to provide a design-time AI / ML recommendation engine for RPA according to an embodiment of the present invention.

[0013] Fig. 6A An example of a neural network that has been trained to provide suggestions for the next activity or sequence of activities in accordance with an embodiment of the present invention is illustrated.

[0014] Figure 6B An example of a neuron according to an embodiment of the present invention is illustrated.

[0015] Figure 7 is a flow chart illustrating a process for training (multiple) AI / ML models according to an embodiment of the present invention.

[0016] Figure 8 is an example of a diagram illustrating relationships between application elements according to an embodiment of the present invention.

[0017] Fig. 9 is a flow chart illustrating a process for a design-time AI / ML recommendation engine for RPA according to an embodiment of the present invention.

[0018] Fig.10 is a flow chart illustrating a process for performing RPA error preemption or self-healing for an RPA robot at runtime according to an embodiment of the present invention.

[0019] Unless otherwise indicated, like reference numerals denote corresponding features consistently throughout the drawings. DETAILED DESCRIPTION

[0020] Some embodiments relate to a design-time AI / ML recommendation engine for RPA. The AI / ML model can be trained to provide recommendations for the next activity or sequence of activities included during the development of a design-time RPA workflow. In certain embodiments, if the confidence of the recommended next activity or sequence of activities exceeds an automatic insertion threshold (e.g., 80%, 90%, 99%, etc.), the recommended next activity or sequence of activities can be automatically inserted into the RPA workflow. These recommendations can be based on the context in which the user is in the development of the RPA workflow (i.e., the current state of the RPA workflow at the time). For example, if an RPA developer adds a send email activity to a workflow, the AI / ML model can use this context to recommend archiving sent emails as the next activity.

[0021] In some embodiments, the AI / ML model can be trained to suggest the next activity or sequence of activities based on the context of the RPA workflow that has been built before. For example, if a user creates an activity based on a source to complete several fields in a form, the next step after completing the fields may frequently be to click a button (e.g., "Submit", "OK", etc.). The AI / ML model can learn by training on the existing RPA workflow that completing all fields and then clicking such a button may be the desired task, and recommend activities to the user that automatically complete the remaining fields and click the appropriate button.

[0022] It should be noted that in some embodiments, recommendations may not be limited to activities and / or activity sequences. For example, the AI / ML model may learn to configure parameters in existing activities (e.g., properties such as inputs and outputs, improvements to the content of emails being generated by the workflow, etc.). This may help ensure that the RPA workflow works properly by fixing user errors and / or providing missing information.

[0023] In some embodiments, the AI / ML model can learn to suggest what to do next using automation. For example, the AI / ML model can learn that for similar RPA workflows, developers tend to generate automation to test RPA workflows, publish RPA workflows in a company repository or catalog, etc. If the confidence from the AI / ML model is high enough (e.g., exceeds an automated performance threshold), the next step(s) can be taken automatically.

[0024] In some embodiments of UI automation, an AI / ML model is linked to an object repository and trained to make recommendations from it. For example, an AI / ML model may recognize that an RPA workflow is related to a specific application, version of an application, screen of an application, etc. in a UI, and make recommendations for its activities and / or its parameters (e.g., UI descriptors, inputs, outputs, etc.) for that specific application / version / screen by "looking at" the screen (e.g., using a computer vision (CV) model to recognize graphical elements and text on the screen). This may reduce or eliminate the need for RPA developers to provide further information in this regard when configuring activities in an RPA workflow. Instead, the AI / ML model may recognize that the screen on which the user indicates an activity corresponds to a given application / version / screen, and find the appropriate UI object and / or UI descriptor for that activity and the recommendation made by the AI / ML model for the next activity or sequence of activities to be added.

[0025] In some embodiments, recommendations for the next activity and / or parameters of such activities may also be provided for non-UI automation. Previously developed RPA workflows may be used to train AI / ML models to identify typical next activities or sequences thereof. Thus, recommendations for additional activities and / or sequences thereof may be provided for any type of RPA workflow, whether for attended or unattended RPA robots and whether for UI automation or non-UI automation, without departing from the scope of the present invention.

[0026] In some embodiments, a relationship graph may be constructed via the first AI / ML model. For example, the graph may link applications to screens, screens to versions, UI elements to screens, and UI descriptors, etc., as nodes in the graph. In some embodiments, the connection between two nodes may be directed or bidirectional.

[0027] Once the graph is created, at least in an initial form, a second AI / ML model (or perhaps a more complex version of the first AI / ML model) can be trained to use the graph and learn ontological associations. For example, the AI / ML model may observe that when a user clicks a "Submit" button on one screen that includes a form, this causes the application to display another screen that includes information related to the processing of the submitted form. This can cause the AI / ML model to determine that there is an ontological relationship between the submit button and the next screen, and the AI / ML model can modify the graph to include a directed connection between the submit button and the next screen. Thus, the data itself can be organized into entities. Once the graph is built and the AI / ML model(s) are trained, they can be used to make recommendations and / or correct failures.

[0028] Thus, in some embodiments, the graph can be used for activity suggestions as well as for self-healing purposes. Runtime corrections can be performed without manually modifying the corresponding RPA workflow. For example, if a graphical element on the screen is not identified, the AI / ML model can determine what went wrong (e.g., the graphical element changed position and / or its design changed) and the automation can automatically repair to accommodate this change (e.g., the UI descriptor designed for the old graphical element is replaced with the UI descriptor designed for the new graphical element).

[0029] While for some errors, automatic fixes are desirable (e.g., saving a document, finding a button that has been moved, automatically completing new form information required in a new version of an app, etc.), for other errors, automatic fixes may not be desirable. For example, if there is a new compliance requirement that requires a manager's approval before submission, the AI / ML model may not be able to add these additional features on its own. The AI / ML model can learn whether to automatically fix errors by training on examples of users accepting suggested corrections and rejecting them. Over time, the AI / ML model can learn the types of corrections expected and make them automatically, or allow the RPA robot to fail.

[0030] In some embodiments, self-healing and / or preemptive repair functionality may be provided at runtime. Consider a situation where the interface of an application for which an RPA workflow is designed changes so that one or more activities no longer function as designed. For example, new fields may have been added to a form that was required prior to submission, the name and / or location of a field may have changed, the design and / or location of a button may have changed, and so on. In some embodiments, the AI / ML model or the RPA robot itself searches the UI object repository and / or another source (e.g., a web page that lists changes to a new application version) for changes and determines what modifications can be made to correct the problem. For example, for a new employee form, if a spouse field (e.g., name or N / A) is now required, the AI / ML model can be trained so that it can determine that a "Type to" activity is required in the RPA workflow and add this activity to the workflow. An updated RPA automation can then be generated to replace the previous automation from the original workflow, and this version can be used instead.

[0031] In some embodiments, workflow suggestions and / or fixes can be performed iteratively. For example, as a user develops and tests an RPA workflow in an RPA designer application, the AI / ML model can iteratively test the workflow. If a workflow fails, the AI / ML model can provide suggestions and / or automatically apply fixes (e.g., modify activity parameters, replace one activity with another activity appropriate for the task, add one or more additional activities required to make a specific activity run, etc.) while the user is developing the workflow. The AI / ML model can be run periodically during workflow development to iteratively resolve issues in the RPA workflow, increase the speed at which users can develop workflows, and improve the effectiveness and accuracy of workflows.

[0032] In some embodiments, RPA automation can be automatically modified when the software application targeted by the automation changes. For example, consider a company switching from using SAP Switch to Instead of having developers create new workflows for new applications, AI / ML models can be trained to automatically search the UI object repository for objects that accomplish the same task(s). UI descriptors, activities, parameters, etc. In other words, although the screens and processes can be different, the business logic of the tasks is the same or similar.

[0033] UI descriptors and other elements in a UI object repository for one application can potentially be mapped to those UI descriptors and other elements for other applications using semantic or ontology mapping. The AI / ML model can semantically decompose what is done in one application and another application to decompose the task into screens. If the AI / ML model is not pre-trained for this purpose, the UI object repository can be built manually or automatically. In another case, for example, a link to the website of the target application can be provided, and the UI object repository can be built from the web application screen on the website. Reinforcement learning can be used for this process, in which the AI / ML model is given a reward function (e.g., submitting a checklist, completing a form, copying information from a source to a destination, etc.). If the AI / ML model does not succeed after a certain number of epochs, humans can mark the information, perform the operations observed by the AI / ML model, revise the reward function, etc. to continue and / or complete the training process.

[0034] Some embodiments use a cloud implementation of an object repository accessible to RPA developers and potentially eliminate the need to package UI elements, use libraries such as NuGet, or TM The AI / ML model can determine whether to seek a better UI element. To this end, the RPA robot using the AI / ML model can look for previous versions of the UI descriptor and / or UI descriptors for similar elements in other similar target applications that have been developed when the UI descriptor fails. Additionally or alternatively, the RPA robot using the AI / ML model can check whether there is a newer version of the UI descriptor in the object repository. This provides self-healing functionality, which can allow the RPA robot to find the element when the UI descriptor does not work, rather than throwing an error or skipping the activity.

[0035] Consider a case where the Japanese version of a website is loaded instead of the English version, but the buttons on each page have the same functionality. An RPA robot employing an AI / ML model can try the UI descriptors for different language versions to try to find the button. Additionally, consider a case where a process is launched in a different web browser than expected. An RPA robot can try the UI descriptors for different web browsers to try to find the graphical element.

[0036] In some embodiments, AI / ML models can be used to generate correct UI descriptors. Over time, and if not continuously updated, the object repository may become outdated. AI / ML models can be trained to interpret how applications change, and automatically update the object repository with working UI descriptors. This essentially provides automatic version control. By supplementing the object repository with AI / ML models that can generate UI descriptors, these models can be used to find UI descriptors for activities in the RPA workflow.

[0037] In order to automatically update the UI repository with working UI descriptors, some embodiments determine the reason why the failure occurred. Telemetry data can be used to determine which elements failed and why, and the telemetry data is connected to the corresponding elements in the UI repository. Using this data, the artificial intelligence / machine learning model can learn to extract the goals that the failed process is trying to achieve. For example, the AI / ML model can determine that the process is trying to use the UI descriptor on a mismatched version of the application and then exchange the working UI descriptor. If this cannot be derived, other data points can be used to make this determination. For example, when an update email is received, the location in the UI may have changed, thereby disrupting the RPA process. In addition, when the AI / ML model determines the root cause of the problem, the RPA Center of Excellence (COE) can be notified to make changes to the RPA process and point out potential problems at runtime. Alternatively, the AI / ML model can make corrections in the object repository itself.

[0038] In some embodiments, semantic matching can be performed. For example, an AI / ML model can be trained to learn that certain text in a changed user interface matches text from a previous version. If the label "invoice number" changes to "bill number" and the location of the associated text field moves on the screen, the AI / ML model can recognize these label matches and update the UI descriptor to find the text field (e.g., using target anchor matching).

[0039] In some embodiments, multiple AI / ML models may be used. For example, there may be an AI / ML model for suggesting the next activity or sequence of activities, a CV model, a semantic matching model, a generative AI model, and the like. The AI / ML model for suggesting the next activity or sequence of activities may be trained on previous RPA automations to learn how they are built, what similarities (patterns) exist in these automations, dependencies between activities in the RPA workflow (e.g., input from one activity comes from output from another activity), output types, and the like. For example, if there are a certain number and / or types of activities in the automation, the automation completes a specific final task (e.g., writing and sending an email, completing and submitting a form, etc.), etc., the AI / ML model may learn to recognize these patterns.

[0040] The object repository includes a library of UI objects. The UI elements on the screen (e.g., text fields, buttons, labels, menus, check boxes, etc.) can be grouped by application, application version, application screen, and UI element set, and each screen has multiple UI elements. As used herein, a "screen" is an image of an application UI or a part of an application UI at a certain point in time. In this context, an "application" or a version of a given application can be a union of screens. In some embodiments, each UI element can be described by one or more UI descriptors. UI elements, UI descriptors, applications, and application screens are all UI objects. In some embodiments, UI elements and screens can be further distinguished into specific types of UI elements (e.g., buttons, check boxes, text fields, etc.) and screens (e.g., top windows, modal windows, pop-up windows, etc.).

[0041] In some embodiments, a selector is a UI descriptor that can be used to detect UI elements. In some embodiments, a selector has the following structure:

[0042] <node_1 / ><node_2 / > ...<node_N / >

[0043] The last node represents the GUI element of interest, and all previous nodes represent the parent nodes of that element.<node_1> Often referred to as the root node, and represents the top window of an application.

[0044] Each node may have one or more attributes that help correctly identify the specific level of the selected application. In some embodiments, each node has the following format:

[0045] <ui_systemattr_name_1='attr_value_1'...attr_name_N='attr_value_N' / >

[0046] Each attribute can have an assigned value, and attributes can be selected to have constant values. This is because changing the value of an attribute each time the app is launched may cause the selector to not correctly identify the associated element.

[0047] UI object library descriptors can be added directly to RPA workflow activities, saving developers the time required to create custom selectors for activities. An object library is defined in this article as a collection of UI descriptors corresponding to one or more screens from a version of an application. An example of an object library structure is provided below.

[0048] ·SAP

[0049] Version 1

[0050] Screen 1

[0051] Subject 1

[0052] Object 2

[0053] ·…

[0054] ·Screen 2…

[0055] ·…

[0056] Version 2…

[0057] ·…

[0058] ·Saleforce…

[0059] ·…

[0060] It should be noted that the above hierarchy is provided only as an example. Any desired number of levels in the hierarchy and the elements associated with each level can be used without departing from the scope of the present invention. In certain embodiments, the user can define application mapping according to his or her expectations. In addition, in some embodiments, some nodes in the tree can only serve as management roles and not as functional roles. For example, UI elements can be grouped into containers without UI descriptors. In certain embodiments, containers can exist only for the purpose of grouping.

[0061] A UI descriptor is a set of instructions for finding UI elements. In some embodiments, a UI descriptor is an encapsulated data / structural format, which includes (multiple) UI element selectors, (multiple) anchor selectors, (multiple) CV descriptors, (multiple) unified target descriptors, screen image capture (context), element image capture, other metadata (e.g., applications and application versions), combinations thereof, etc. The encapsulated data / structural format can be extensible with future updates of the platform and is not limited to the above definition. Any suitable UI descriptor for identifying UI elements on the screen can be used without departing from the scope of the present invention. A unified target descriptor links multiple types of UI descriptors together. A unified target descriptor can work like a finite state machine (FSM), where a first UI descriptor mechanism is applied in a first context, a second UI descriptor is applied in a second context, and so on.

[0062] UI descriptors can be extracted from activities in RPA workflows and added to a structured pattern that groups UI descriptors by UI application, screen, and UI element. In some embodiments, the UI descriptor can be part of a project for extensive reuse, part of a global repository for testing purposes, or part of a UI object library for global cross-project sharing. In some embodiments, the object library is an encapsulation of UI descriptors grouped by application, application version, and screen. UI descriptors can be defined and added to an object library, which in some embodiments can be installed in other projects as a dependency after release. This makes the object library of UI descriptors available for reuse. In some embodiments, the UI application is a target application with multiple versions, each version having multiple screens.

[0063] UI object repository and UI descriptor reuse may be beneficial for a variety of reasons. For example, if the location, appearance, and / or functionality of a UI element in an application changes, the UI descriptor may be changed, and the change may then be propagated to activities using the UI descriptor. Thus, reusability may be provided at the level of identifying graphical elements in a UI.

[0064] In some embodiments, the UI descriptor can work with a unified target that contains all UI detection mechanisms through which image detection and definition are performed. The unified target can merge multiple techniques for identifying and automating UI elements into a single cohesive approach. The unified target can prioritize selector-based and driver-based UI detection mechanisms, and fall back to CV to find images if the first two mechanisms are unsuccessful. In some embodiments, the selector editor and UI browser can be configured to support the unified target.

[0065] Figure 1 is an architectural diagram illustrating a hyper-automation system 100 according to an embodiment of the present invention. "Hyper-automation" as used herein refers to an automation system that combines components of process automation, integrated tools, and technologies that enhance work automation capabilities. For example, in some embodiments, RPA can be used at the core of a hyper-automation system, and in certain embodiments, automation capabilities can be extended through AI / ML, process mining, analysis, and / or other advanced tools. For example, as the hyper-automation system learns processes, trains AI / ML models, and employs analysis, more and more knowledge work can be automated, and computing systems in an organization (e.g., systems used by individuals and systems that run autonomously) can all participate in becoming participants in hyper-automation processes. The hyper-automation system of some embodiments allows users and organizations to efficiently and effectively discover, understand, and extend automation.

[0066] The hyper-automation system 100 includes user computing systems such as a desktop computer 102, a tablet computer 104, and a smart phone 106. However, any desired user computing system may be used without departing from the scope of the present invention, including but not limited to smart watches, laptops, servers, Internet of Things (IoT) devices, etc. Figure 1 Three user computing systems are shown in FIG. 1 , but any suitable number of user computing systems may be used without departing from the scope of the present invention. For example, in some embodiments, tens, hundreds, thousands, or millions of user computing systems may be used. The user computing systems may be actively used by users or may run automatically without much or any user input.

[0067] Each user computing system 102, 104, 106 has a corresponding automation process (s) 110, 112, 114 running thereon. In some embodiments, the automation process is stored remotely (e.g., on a server 130 or in a database 140 and accessed via a network 120) and loaded by an RPA robot to implement automation. The automation may exist as a script (e.g., XML, XAML, etc.), or may be compiled into machine-readable code (e.g., as a digital link library).

[0068] The automated processes 110, 112, 114 may include, but are not limited to, an RPA robot, a portion of an operating system, a downloadable application for a corresponding computing system, any other suitable software and / or hardware, or any combination of these, without departing from the scope of the present invention. In some embodiments, one or more of the processes 110, 112, 114 may be a listener. The listener may be an RPA robot, a portion of an operating system, a downloadable application for a corresponding computing system, or any other software and / or hardware, without departing from the scope of the present invention. In fact, in some embodiments, the logic of the listener(s) is implemented partially or entirely via physical hardware.

[0069] The listener monitors and records data related to the user's interaction with the corresponding computing system and / or the operation of the unattended computing system, and sends the data to the core hyper-automation system 120 via a network (e.g., a local area network (LAN), a mobile communication network, a satellite communication network, the Internet, any combination thereof, etc.). The data may include, but is not limited to, which buttons were clicked, where the mouse was moved, the text entered in a field, one window was minimized and another window was open, the application associated with the window, etc. In some embodiments, the data from the listener may be sent periodically as part of a heartbeat message. In some embodiments, the data may be sent to the core hyper-automation system 120 once a predetermined amount of data has been collected, after a predetermined time period has elapsed, or both. One or more servers (such as server 130) receive the data from the listener and store it in a database (such as database 140).

[0070] An automated process can execute logic developed in a workflow during design time. In the case of RPA, a workflow can include a collection of steps, defined herein as "activities," that are executed in a sequence or some other logical flow. Each activity can include an action, such as clicking a button, reading a file, writing to a log panel, and so on. In some embodiments, workflows can be nested or embedded.

[0071] In some embodiments, RPA's long-running workflows are the main project that supports service orchestration, human intervention, and long-running transactions in an unattended environment. For example, see U.S. Patent No. 10,860,905, the entire contents of which are incorporated herein by reference. Human intervention comes into play when certain processes require human input to handle exceptions, approvals, or verifications before continuing to the next step of the activity. In this case, process execution is paused, freeing up the RPA robot until the human task is completed.

[0072] Long-running workflows can support workflow segmentation via persistent activities, and can be combined with call processes and non-user interactive activities to orchestrate manual tasks and RPA robot tasks. In some embodiments, multiple or many computing systems can participate in the logic of executing long-running workflows. Long-running workflows can run in sessions to facilitate fast execution. In some embodiments, long-running workflows can orchestrate background processes, which can include activities that execute application programming interface (API) calls and run in long-running workflow sessions. In some embodiments, these activities can be called by calling process activities. A process with user interactive activities running in a user session can be called by starting a job from an implementer (conductor) activity (an implementer described in more detail later in this article). In some embodiments, a user can interact through tasks of a form that needs to be completed in the implementer. Activities that cause the RPA robot to wait for the form task to complete and then resume the long-running workflow can be included.

[0073] One or more of the (multiple) automated processes 110, 112, 114 communicate with the core hyper-automation system 120. In some embodiments, the core hyper-automation system 120 may run an implementer application on one or more servers, such as server 130. Although one server 130 is shown for illustrative purposes, multiple or many servers in proximity to each other or in a distributed architecture may be employed without departing from the scope of the present invention. For example, one or more servers may be provided for implementer functionality, AI / ML model services, authentication, governance, and / or any other suitable functionality without departing from the scope of the present invention. In some embodiments, the core hyper-automation system 120 may incorporate or be part of a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, etc. In certain embodiments, the core hyper-automation system 120 may host multiple software-based servers, such as server 130, on one or more computing systems. In some embodiments, one or more servers (e.g., server 130) of the core hyper-automation system 120 may be implemented via one or more virtual machines (VMs).

[0074] In some embodiments, one or more of the automated processes 110, 112, 114 may call one or more AI / ML models 132 that are deployed on or accessible by the core hyper-automation system 120 and trained to complete various tasks. For example, the AI / ML models 132 may include models trained to find various application versions, perform CVs, generate UI descriptors, provide suggestions for the next activity or sequence of activities in an RPA workflow, and the like. The AI / ML models may be trained using labeled data, including but not limited to previously created RPA workflows, screenshots of various application screens and their corresponding UI elements for various versions, UI object libraries, and the like. The AI / ML models 132 may be trained to achieve a desired confidence threshold while not overfitting a given training data set.

[0075] The AI / ML model 132 can be trained for any suitable purpose without departing from the scope of the present invention, as will be discussed in more detail later herein. In some embodiments, two or more AI / ML models 132 can be linked (e.g., in series, in parallel, or a combination thereof) so that they jointly provide (multiple) collaborative outputs. The AI / ML model 132 can perform or assist in CV, OCR, document processing and / or understanding, semantic learning and / or analysis, analytical prediction, process discovery, task mining, testing, automatic RPA workflow generation, sequence extraction, cluster detection, audio to text translation, any combination thereof, etc. However, any desired number and / or (multiple) types of AI / ML models can be used without departing from the scope of the present invention. For example, using multiple AI / ML models can allow the system to develop a global picture of what is happening on a given computing system. For example, one AI / ML model can perform OCR, another can detect buttons, another can compare sequences, etc. The pattern can be determined individually by the AI / ML model, or it can be determined jointly by multiple AI / ML models. In certain embodiments, one or more AI / ML models are deployed locally on at least one of computing systems 102 , 104 , 106 .

[0076] In some embodiments, multiple AI / ML models 132 may be used. Each AI / ML model 132 is an algorithm (or model) that operates on data, and for example, the AI / ML model itself may be a deep learning neural network (DLNN) of trained artificial "neurons" trained on training data. In some embodiments, the AI / ML model 132 may have multiple layers that perform various functions, such as statistical modeling (e.g., hidden Markov models (HMMs)), and utilize deep learning techniques (e.g., long short-term memory (LSTM) deep learning, encoding of previous hidden states, etc.) to perform the desired functionality.

[0077] In some embodiments, the hyperautomation system 100 can provide four main groups of functionality: (1) discovery; (2) building automation; (3) management; and (4) engagement. In some embodiments, the automation (e.g., running on a user computing system, server, etc.) can be run by a software robot (such as an RPA robot). For example, attended robots, unattended robots, and / or testing robots can be used. Attended robots work with users to assist them in completing tasks (e.g., via UiPath Assistant). TM ). Unattended robots work independently of users and can potentially run in the background without the user's knowledge. Test robots are unattended robots that run test cases against applications or RPA workflows. In some embodiments, test robots can run in parallel on multiple computing systems.

[0078] The discovery functionality can discover different opportunities for automation of business processes and provide automatic recommendations. Such functionality can be implemented by one or more servers (such as server 130). In some embodiments, the discovery functionality can include providing an automation center, process mining, task mining, and / or task capture. The automation center (e.g., UiPathAutomation Hub) TM ) can provide a mechanism for managing automation rollouts with visibility and control. For example, automation ideas can be crowdsourced from employees via a submission form. Feasibility and return on investment (ROI) calculations for automating these ideas can be provided, documentation for future automations can be collected, and collaboration can be provided to get from automation discovery to build faster.

[0079] Process mining (e.g., via UiPath Automation Cloud TM and / or UiPath AI Center TM) refers to the process of collecting and analyzing data from applications (e.g., enterprise resource planning (ERP) applications, customer relationship management (CRM) applications, email applications, call center applications, etc.) to identify which end-to-end processes exist in an organization and how to effectively automate them and indicate what impact the automation will have. The data can be collected, for example, by a listener from user computing systems 102, 104, 106 and processed by a server such as server 130. In some embodiments, one or more AI / ML models 132 can be used for this purpose. This information can be exported to an automation center to speed up implementation and avoid manual information transfer. The goal of process mining can be to increase business value by automating processes within an organization. Some examples of process mining goals include, but are not limited to, increasing profits, improving customer satisfaction, regulatory and / or contractual compliance, improving employee efficiency, etc.

[0080] Task mining (e.g., via UiPath Automation Cloud TM and / or UiPath AI Center TM ) identifies and aggregates workflows (e.g., employee workflows), and then applies AI to expose patterns and variations in daily tasks, scoring such tasks for automation and potential savings (e.g., time and / or cost savings). One or more AI / ML models 132 can be employed to reveal recurring task patterns in the data. Repetitive tasks ripe for automation can then be identified. In some embodiments, this information can initially be provided by a listener and analyzed on a server (such as server 130) of the core hyper-automation system 120. The findings of task mining (e.g., Extensible Application Markup Language (XAML) process data) can be exported to a process document or designer application (such as UiPath Studio TM ) to create and deploy automations more quickly.

[0081] Task mining in some embodiments may include taking screenshots of user actions (e.g., mouse click locations, keyboard input, application windows and graphical elements with which the user is interacting, timestamps of interactions, etc.), collecting statistics (e.g., execution time, number of actions, text input, etc.), editing and annotating screenshots, specifying the types of actions to be recorded, etc.

[0082] Task capture (e.g., via UiPath Automation Cloud TM and / or UiPath AI Center TM) automatically documents attended processes as users work or provides a framework for unattended processes. This documentation can include capturing the actions of each part of the process in the form of a Process Definition Document (PDD), a skeleton workflow, recording user actions and automatically generating a comprehensive workflow diagram (including detailed information about each step), Microsoft In some embodiments, the ready-to-build workflow can be exported directly to a designer application such as UiPath Studio. TM Task capture can streamline the requirements gathering process for subject matter experts who interpret the process and CoE members who provide production-grade automation.

[0083] Build automation can be applied via a designer (e.g., UiPath Studio TM 、UiPath StudioX TM or UiPath Web TM ). For example, an RPA developer at the RPA development facility 150 may use an RPA designer application 154 of a computing system 152 to build and test RPAs for various applications and environments (such as web, mobile, and virtualized desktops). API integration can be provided for a variety of applications, technologies, and platforms. Predefined activities, drag-and-drop modeling, and a workflow recorder can make automation easier with minimal coding. Document understanding functionality can be provided via drag-and-drop AI skills for data extraction and interpretation, which call one or more AI / ML models 132. This automation can handle virtually any document type and format, including forms, checkboxes, signatures, and handwriting. When validating data or handling exceptions, this information can be used to retrain the corresponding AI / ML model, thereby improving its accuracy over time.

[0084] The RPA designer application 152 may be designed to call one or more of the trained AI / ML models 132 on the server 130 and / or the generative AI models 172 in the cloud environment via the network 120 (e.g., a local area network (LAN), a mobile communication network, a satellite communication network, the Internet, and any combination thereof, etc.) to assist in the RPA automation development process. In some embodiments, one or more of the AI / ML models may be packaged with the RPA designer application 152 or otherwise stored locally on the computing system 150.

[0085] In some embodiments, one or more of the AI / ML models 132 and the RPA designer application 152 may be configured to use an object repository stored in the database 140. The object repository may include a UI object library that may be used to develop RPA workflows via the RPA designer application 152. The object repository may be used to add UI descriptors to activities in the workflows of the RPA designer application 152 for UI automation. In some embodiments, one or more of the AI / ML models 132 may generate new UI descriptors and add them to the object repository in the database 140. Once automation is completed in the designer application 152, they may be published on the server 130, pushed to the computing systems 102, 104, 106, etc.

[0086] For example, integration services can allow developers to seamlessly combine user interface (UI) automation with API automation. Automations can be built that require APIs or traverse API and non-API applications and systems. Repositories for pre-built RPA and AI templates and solutions (such as the UiPath Object Repository) can be provided. TM ) or a marketplace (e.g. UiPathMarketplace TM ) to allow developers to automate various processes faster. Therefore, when building automation, the hyper-automation system 100 can provide a user interface, a development environment, API integration, pre-built and / or custom-built AI / ML models, development templates, an integrated development environment (IDE), and advanced AI capabilities. In some embodiments, the hyper-automation system 100 enables the development, deployment, management, configuration, monitoring, debugging, and maintenance of RPA robots, which can provide automation for the hyper-automation system 100.

[0087] In some embodiments, components of the hyper-automation system 100, such as (multiple) designer applications and / or external rule engines, provide support for managing and enforcing governance policies for controlling various functionality provided by the hyper-automation system 100. Governance is the ability of an organization to establish policies to prevent users from developing automation devices (e.g., RPA robots) that can take actions that may harm the organization (such as violating the EU General Data Protection Regulation (GDPR), the U.S. Health Insurance Portability and Accountability Act (HIPAA), third-party application terms of service, etc.). General Data Protection Regulation (GDPR), U.S. Health Insurance Portability and Accountability Act (HIPAA), third-party application terms of service, etc. Automation (e.g., RPA robots). Because developers may create automations that violate privacy laws, terms of service, etc. when executing their automations, some embodiments implement access control and governance restrictions at the robot and / or robot design application level. In some embodiments, this can provide an additional level of security and compliance to the automation process development pipeline by preventing developers from relying on unapproved software libraries that may introduce security risks or work in a manner that violates policies, regulations, privacy laws, and / or privacy policies. See, e.g., U.S. Patent Application Publication No. 2022 / 0011732, the entire contents of which are incorporated herein by reference.

[0088] Management functionality may provide management, deployment, and optimization of automation across an organization. In some embodiments, management functionality may include orchestration, test management, AI functionality, and / or insights. The management functionality of the hyper-automation system 100 may also serve as an integration point with third-party solutions and applications for automated applications and / or RPA robots. The management capabilities of the hyper-automation system 100 may include, but are not limited to, facilitating the provisioning, deployment, configuration, queuing, monitoring, logging, and interconnection of RPA robots, among others.

[0089] Implementer applications, such as UiPath Orchestrator TM (In some embodiments, this can be used as a UiPath Automation Cloud TM or locally, in a VM, in a private or public cloud, on a Linux TM in a VM or as a TM The UiPath Test Suite provides orchestration capabilities to deploy, monitor, optimize, scale, and secure RPA robot deployments. TM) can provide test management to monitor the quality of deployed automation. Test suites can facilitate test planning and execution, requirements fulfillment, and defect traceability. Test suites can include comprehensive test reporting.

[0090] Analytics software (e.g., UiPath Insights TM ) can track, measure, and manage the performance of deployed automation. Analytics software can align automation operations with specific key performance indicators (KPIs) and strategic outcomes for the organization. Analytics software can present results in a dashboard format for better understanding by human users.

[0091] Data Services (for example, UiPath Data Service TM ) can be stored in database 140, for example, and bring data into a single, scalable, secure place using a drag-and-drop storage interface. Some embodiments can provide low-code or no-code data modeling and storage for automation while ensuring seamless access to data, enterprise-grade security, and scalability. AI functionality can be provided by an AI center (e.g., UiPath AI Center TM ) is provided, which helps to incorporate AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options can make such functionality accessible even to those who are not data scientists. Deployed automation (e.g., RPA robots) can call AI / ML models, such as AI / ML model 132, from the AI ​​center. The performance of the AI / ML model can be monitored, trained, and improved using manually verified data, such as provided by the data review center 160. A human reviewer can provide labeled data to the core hyper-automation system 120 via the review application 152 on the computing system 154. For example, a human reviewer can verify that the prediction of the AI / ML model 132 is accurate or otherwise provide a correction. This dynamic input can then be saved as training data for retraining the AI / ML model 132 and can be stored in a database such as database 140. The AI ​​center can then schedule and execute training jobs to train a new version of the AI / ML model using the training data. Both positive and negative examples can be stored and used to retrain the AI / ML model 132.

[0092] Engagement functionality engages humans and automation as a team to collaborate seamlessly on the desired process. TM ) Build low-code applications to connect browser tabs and legacy software, even those that lack APIs in some embodiments. For example, you can use a web browser to quickly create applications using a rich library of drag-and-drop controls. Applications can be connected to a single automation or multiple automations.

[0093] Action Center (for example, UiPath Action Center TM ) provides a direct and efficient mechanism to hand over a process from automation to humans and vice versa. Humans can provide approvals or escalations, make exceptions, etc. Automation can then perform the automated functionality of a given workflow.

[0094] A local assistant can be provided as a launchpad for users to start automation (e.g., UiPath Assistant TM ). For example, this functionality may be provided in a tray provided by the operating system and may allow users to interact with RPA robots and RPA robot-driven applications on their computing systems. The interface may list automations approved for a given user and allow the user to run them. These may include off-the-shelf automations from the Automation Marketplace, internal automation storage in the Automation Center, and the like. When automations run, they may run as local instances in parallel with other processes on the computing system, so users can use the computing system while automating their actions. In some embodiments, the assistant is integrated with task capture functionality so that users can archive the processes they are about to automate from the Assistant Launchpad.

[0095] Chatbots (e.g., UiPath Chatbots TM ), social messaging applications, and / or voice commands can enable users to run automations. This can simplify access to information, tools, and resources that users need to interact with customers or perform other activities. Conversations between people can be easily automated, as can other processes. A triggered RPA robot launched in this way can perform actions such as checking order status, publishing data in a CRM, etc., possibly using simple language commands.

[0096] In some embodiments, end-to-end measurement and management of automation programs of any scale can be provided by the hyper-automation system 100. Based on the above, analytics can be used to understand the performance of automation (e.g., via UiPathInsights TM ). Data modeling and analysis using any combination of available business metrics and operational insights can be used for various automation processes. Custom designed and pre-built dashboards allow for visualization of data across desired metrics, discovery of new analytical insights, tracking of performance indicators, discovery of ROI for automation, telemetry monitoring of user computing systems, detection of errors and anomalies, and debugging of automation. An automation management console (e.g., UiPath AutomationOps TM) to manage automations throughout the automation lifecycle. Organizations can control how automations are built, what users can do with them, and which automations users can access.

[0097] In some embodiments, the hyperautomation system 100 provides an iterative platform. Processes can be discovered, automation can be built, tested, and deployed, performance can be measured, automation usage can be easily provided to users, feedback can be obtained, AI / ML models can be trained and retrained, and the process itself can be repeated. This facilitates a more robust, more effective automation suite.

[0098] In some embodiments, a generative AI model is used. Generative AI can generate various types of content, such as text, images, audio, and synthetic data. Various types of generative AI models can be used, including but not limited to large language models (LLMs), generative adversarial networks (GANs), variational autoencoders (VAEs), transformers, etc. These models can be part of the AI / ML model 132 hosted on the server 130. For example, a generative AI model can be trained on a large corpus of text information to perform semantic understanding, understand the essence of what is presented on the screen from the text, and so on. In some embodiments, a generative AI model provided by an existing cloud ML service provider (such as In a generative AI embodiment where (multiple) generative AI models 172 are remotely hosted, server 130 can be configured to integrate with a third-party API, which allows server 130 to send a request including the necessary input information to generative AI model 172 and receive a response back (e.g., semantic matching of fields between application versions, classification of application types on a screen, etc.). Such an embodiment can provide a more advanced and sophisticated user experience, as well as provide access to the most advanced natural language processing (NLP) and other ML capabilities provided by these companies.

[0099] Figure 2 2 is a diagram illustrating an architecture of an RPA system 200 according to an embodiment of the present invention. Figure 1The RPA system 200 includes a designer 210 that allows developers to design and implement workflows. The designer 210 can provide solutions for application integration and automation of third-party applications, management of information technology (IT) tasks, and business IT processes. The designer 210 can facilitate the development of automation projects, which are graphical representations of business processes. In short, the designer 210 facilitates the development and deployment of workflows and robots. In some embodiments, the designer 210 can be an application running on a user's desktop, an application running remotely in a VM, a web application, etc.

[0100] Automation projects enable automation of rule-based processes by giving developers control over the order of execution and the relationships between sets of custom steps developed in a workflow, which are defined herein as "activities" in accordance with the above. A commercial example of an embodiment of the designer 210 is UiPath Studio TM Each activity may include an action, such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0101] Some types of workflows may include, but are not limited to, sequences, flow charts, finite state machines (FSMs), and / or global exception handlers. Sequences may be particularly suitable for linear processes, allowing flows from one activity to another without cluttering the workflow. Flow charts may be particularly suitable for more complex business logic, enabling decision-making and connection activities to be integrated in a more diverse manner through multiple branching logic operators. FSMs may be particularly suitable for large workflows. FSMs may use a limited number of states in their execution, which are triggered by conditions (i.e., transitions) or activities. Global exception handlers may be particularly suitable for determining workflow behavior and debugging processes when execution errors are encountered.

[0102] Once a workflow is developed in the designer 210, the execution of the business process is orchestrated by the implementer 220, which orchestrates one or more robots 230 that execute the workflow developed in the designer 210. One commercial example of an embodiment of the implementer 220 is the UiPath Orchestrator TM The implementer 220 facilitates the management of the creation, monitoring, and deployment of resources in the environment. The implementer 220 may serve as an integration point with third-party solutions and applications. Based on the above, in some embodiments, the implementer 220 may be Figure 1 Part of the core hyper-automation system 120.

[0103] The implementer 220 can manage a queue of robots 230, connect and execute robots 230 from a centralized point. The types of robots 230 that can be managed include, but are not limited to, attended robots 232, unattended robots 234, development robots (similar to unattended robots 234, but used for development and testing purposes), and non-production robots (similar to attended robots 232, but used for development and testing purposes). Attended robots 232 are triggered by user events and operate with humans on the same computing system. Attended robots 232 can be used with implementers 220 for centralized process deployment and logging to media. Attended robots 232 can help human users complete various tasks and can be triggered by user events. In some embodiments, processes cannot be started from implementers 220 on this type of robot and / or they cannot run under a lock screen. In some embodiments, attended robots 232 can only be started from the robot tray or from a command prompt. In some embodiments, attended robots 232 should run under human supervision.

[0104] Unattended robots 234 run unattended in a virtual environment and can automate many processes. Unattended robots 234 can be responsible for remote execution, monitoring, scheduling, and support for work queues. In some embodiments, debugging of all robot types can be run in the designer 210. Both attended and unattended robots can automate various systems and applications, including but not limited to mainframes, web applications, VMs, enterprise applications (e.g., etc.) and computing system applications (e.g., desktop and notebook computer applications, mobile device applications, wearable computer applications, etc.).

[0105] The implementer 220 may have various capabilities including, but not limited to, provisioning, deploying, configuring, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining a connection between a robot 230 and the implementer 220 (e.g., a web application). Deployment may include ensuring that a packaged version is properly delivered to an assigned robot 230 for execution. Configuration may include maintenance and delivery of robot environment and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot identification data and maintaining user permissions. Logging may include storing and indexing logs into a database (e.g., a structured query language (SQL) database or a "not only" SQL (NoSQL) database) and / or other storage mechanisms (e.g., It provides the ability to store and quickly query large data sets.) The implementer 220 can provide interconnectivity by acting as a centralized communication point for third-party solutions and / or applications.

[0106] Robot 230 is an execution agent that implements the workflow constructed in designer 210. One commercial example of some embodiments of robot(s) 230 is UiPath Robots TM In some embodiments, the robot 230 installs Microsoft As a result, such a robot 230 can open interactive session, and has Permissions for services.

[0107] In some embodiments, robots 230 can be installed in user mode. For such robots 230, this means that they have the same permissions as the user who installed the given robot 230. This feature can also be used for high density (HD) robots, ensuring that each machine is fully utilized at its maximum potential. In some embodiments, any type of robot 230 can be configured in an HD environment.

[0108] In some embodiments, the robot 230 is split into several components, each dedicated to a specific automation task. The robot components in some embodiments include but are not limited to SCM-managed robot services, user mode robot services, executors, agents, and command lines. SCM-managed robot service management and monitoring The session acts as a proxy between the implementer 220 and the execution host (ie, the computing system on which the robot 230 executes). These services are trusted by the robot 230 and manage the credentials of the robot 230. The console application is launched by the SCM under the local system.

[0109] In some embodiments, user-mode robot services manage and monitor The user mode robot service can be trusted by the robot 230 and manage the credentials of the robot 230. If the robot service managed by the SCM is not installed, it can be automatically started application.

[0110] The actuator can be The executor can be aware of the dots per inch (DPI) setting of each monitor. The agent can be a window that displays available jobs in the system tray. Presentation Foundation (WPF) applications. Agents can be clients of services. Agents can request to start or stop jobs and change settings. Command lines are clients of services. Command lines are console applications that can request to start jobs and wait for their output.

[0111] As explained above, splitting the components of the robot 230 helps developers, support users, and computing systems to more easily run, identify, and track what each component is executing. Special behaviors can be configured for each component in this way, such as setting different firewall rules for executors and services. In some embodiments, the executor can always know the DPI setting of each monitor. As a result, workflows can be executed at any DPI, regardless of the configuration of the computing system on which the workflow is created. In some embodiments, projects from the designer 210 can also be independent of the browser zoom level. In some embodiments, DPI can be disabled for applications that are unaware of DPI or are intentionally marked as unaware.

[0112] The RPA system 200 in this embodiment is part of a hyper-automation system. Developers can use the designer 210 to build and test RPA robots that utilize AI / ML models deployed in the core hyper-automation system 240 (e.g., as part of its AI center). Such RPA robots can send inputs for executing (multiple) AI / ML models and receive outputs from them via the core hyper-automation system 240.

[0113] As described above, one or more of the robots 230 may be listeners. These listeners may provide information to the core hyper-automation system 240 about what users are doing when they use their computing systems. The core hyper-automation system may then use this information for process mining, task mining, task capture, and the like.

[0114] An assistant / chatbot 250 may be provided on the user computing system to allow the user to launch an RPA local robot. For example, the assistant may be located in the system tray. The chatbot may have a user interface so the user can see the text in the chatbot. Alternatively, the chatbot may lack a user interface and run in the background, using the computing system's microphone to listen to the user's voice.

[0115] In some embodiments, data labeling can be performed by a user of the computing system on which the robot is executing or a user of another computing system to which the robot provides information. For example, if the robot calls an AI / ML model that performs CV on an image of a VM user, but the AI / ML model fails to correctly identify a button on the screen, the user can draw a rectangle around the incorrectly identified or unidentified component and possibly provide the correct identification for the text. This information can be provided to the core hyperautomation system 240 and subsequently used to train a new version of the AI / ML model.

[0116] Figure 3is an architectural diagram illustrating an RPA system 300 deployed according to an embodiment of the present invention. In some embodiments, the RPA system 300 may be Figure 2 RPA system 200 and / or Figure 1 The deployed RPA system 300 may be a cloud-based system, a local system, a desktop-based system, etc. that provides enterprise-level, user-level, or device-level automation solutions for automation of different computing processes.

[0117] It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robot application 310 includes an executor 312, an agent 314, and a designer 316. However, in some embodiments, the designer 316 may not be running on the same computing system as the executor 312 and the agent 314. The executor 312 is running a process. Several business projects may be running simultaneously, such as Figure 3 In this embodiment, the agent 314 (e.g., The Executor 340 service is a single point of contact for all Executors 312. All messages in this embodiment are logged to the Executor 340, which further processes the messages via the database server 350, the AI / ML server 360, the indexer server 370, or any combination thereof. Figure 2 As discussed, the actuator 312 may be a robotic component.

[0118] In some embodiments, a robot represents an association between a machine name and a user name. A robot can manage multiple executors simultaneously. On a computing system that supports multiple interactive sessions running simultaneously (e.g., Server2012), multiple robots can run simultaneously, each in a separate A unique username is used in the session. This is what is referred to as an HD robot.

[0119] The agent 314 is also responsible for sending the status of the robot (e.g., periodically sending a "heartbeat" message indicating that the robot is still functioning) and downloading the required version of the package to be executed. In some embodiments, communication between the agent 314 and the implementer 340 is always initiated by the agent 314. In a notification scenario, the agent 314 can open a WebSocket channel that is later used by the implementer 340 to send commands to the robot (e.g., start, stop, etc.).

[0120] The listener 330 monitors and records data related to the user's interaction with the attended computing system and / or the operation of the unattended computing system on which the listener 330 resides. The listener 330 can be an RPA robot, part of an operating system, a downloadable application for a corresponding computing system, or any other software and / or hardware without departing from the scope of the present invention. In fact, in some embodiments, the logic of the listener is partially or completely implemented via physical hardware.

[0121] On the server side, it includes a presentation layer (web application 342, open data protocol (OData) representation state transfer (REST) ​​application programming interface (API) endpoint 344, and notification and monitoring 346), a service layer (API implementation includes / business logic 348), and a persistence layer (database server 350, AI / ML server 360, and indexer server 370). The implementer 340 includes a web application 342, an OData REST API endpoint 344, notification and monitoring 346, and an API implementation / business logic 348. In some embodiments, most actions performed by a user in the interface of the implementer 340 (e.g., via a browser 320) are performed by calling various APIs. These actions may include, but are not limited to, starting a job on a robot, adding / removing data in a queue, scheduling a job to run unattended, etc., without departing from the scope of the present invention. The web application 342 is the visual layer of the server platform. In this embodiment, the web application 342 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the present invention. In this embodiment, the user interacts with a web page from web application 342 via browser 320 to perform various actions to control implementer 340. For example, the user can create a group of robots, assign packages to robots, analyze logs per robot and / or per process, start and stop robots, etc.

[0122] In addition to the web application 342, the implementer 340 also includes a service layer that exposes an OData REST API endpoint 344. However, other endpoints may be included without departing from the scope of the present invention. The REST API is used by both the web application 342 and the agent 314. In this embodiment, the agent 314 is a manager of one or more robots on a client computer.

[0123] The REST API in this embodiment covers configuration, logging, monitoring and queuing functionality. In some embodiments, the configuration endpoint can be used to define and configure application users, permissions, robots, assets, versions and environments. For example, the logging REST endpoint can be used to log different information, such as errors, explicit messages sent by the robot, and other environment-specific information. The deployment REST endpoint can be used by the robot to query the package version that should be executed if the start job command is used in the implementer 340. The queue REST endpoint can be responsible for queue and queue item management, such as adding data to the queue, getting transactions from the queue, setting the status of transactions, etc.

[0124] The monitoring REST endpoint can monitor the web application 342 and the agent 314. The notification and monitoring API 346 can be a REST endpoint used to register the agent 314, deliver configuration settings to the agent 314, and send / receive notifications from the server and the agent 314. In some embodiments, the notification and monitoring API 346 can also use WebSocket communication.

[0125] In some embodiments, the API in the service layer can be accessed through the configuration of appropriate API access paths, for example, based on whether the implementer 340 and the entire hyper-automation system have a local deployment type or a cloud-based deployment type. The API for the implementer 340 can provide customized methods for querying statistics about various entities registered in the implementer 340. In some embodiments, each logical resource can be an OData entity. In such an entity, components such as robots, processes, queues, etc. can have attributes, relationships, and operations. In some embodiments, the API of the implementer 340 can be used by the web application 342 and / or the agent 314 in two ways: by obtaining API access information from the implementer 340, or by registering an external application to use the OAuth flow.

[0126] In this embodiment, the persistence layer includes three servers - a database server 350 (e.g., a SQL server), an AI / ML server 360 (e.g., a server providing AI / ML model service services, such as AI center functionality), and an indexer server 370. The database server 350 in this embodiment stores configurations of robots, robot groups, associated processes, users, roles, schedules, etc. In some embodiments, this information is managed by a web application 342. The database server 350 can manage queues and queue items. In some embodiments, the database server 350 can store messages logged by the robot (in addition to or in place of the indexer server 370). The database server 350 can also store process mining, task mining, and / or task capture related data, such as received from a listener 330 installed on the client side. In some embodiments, although no arrow is shown between the listener 330 and the database 350, it should be understood that the listener 330 is able to communicate with the database 350, and vice versa. The data can be stored in the form of a PDD, an image, a XAML file, etc. Listener 330 may be configured to intercept user actions, processes, tasks, and performance metrics on the corresponding computing system on which listener 330 resides. For example, listener 330 may record user actions (e.g., clicks, typed characters, location, application, active element, time, etc.) on its corresponding computing system and then transform these into a suitable format to be provided to and stored in database server 350.

[0127] The AI / ML server 360 facilitates incorporating AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options make such functionality accessible even to those who are not data scientists. Deployed automation (e.g., RPA robots) can call AI / ML models from the AI / ML server 360. Human-verified data can be used to monitor, train, and improve the performance of AI / ML models. The AI / ML server 360 can schedule and execute training jobs to train new versions of AI / ML models.

[0128] The AI / ML server 360 may store data related to AI / ML models and ML packages for configuring various ML skills for users at the time of development. As used herein, an ML skill is a pre-built and trained ML model for a process, which may be used, for example, by automation. The AI / ML server 360 may also store data related to document understanding techniques and frameworks, algorithms, and software packages for various AI / ML capabilities, including but not limited to intent analysis, natural language processing (NLP), speech analysis, different types of AI / ML models, and the like.

[0129] Indexer server 370, which is optional in some embodiments, stores and indexes information logged by the robot logs. In some embodiments, indexer server 370 can be disabled through configuration settings. In some embodiments, indexer server 370 uses It is an open source project full-text search engine. Messages logged by the robot (e.g., using activities such as logging messages or writing lines) can be sent to the indexer server 370 through the logging REST endpoint(s), where they are indexed for future use.

[0130] Figure 4 4 is an architectural diagram illustrating the relationship 400 between the designer 410, activities 420, 430, 440, 450, driver 460, API 470 and AI / ML model 480 according to an embodiment of the present invention. According to the above, the developer uses the designer 410 to develop a workflow executed by the robot. In some embodiments, various types of activities can be displayed to the developer. The designer 410 can be located locally or remotely on the user's computing system (for example, accessed via a local web browser or VM interacting with a remote web server). The workflow can include user-defined activities 420, API-driven activities 430, AI / ML activities 440 and / or UI automation activities 450. User-defined activities 420 and API-driven activities 440 interact with the application via their APIs. In some embodiments, user-defined activities 420 and / or AI / ML activities 440 can call one or more AI / ML models 480, which can be located locally and / or remotely on the computing system on which the robot is operating.

[0131] Some embodiments are capable of identifying non-textual visual components in images, which are referred to herein as CV. CV can be performed at least in part by (multiple) AI / ML models 480. Some CV activities related to such components can include, but are not limited to, extracting text from segmented label data using OCR, fuzzy text matching, cropping segmented label data using ML, comparing extracted text from label data with true value data, etc. In some embodiments, there can be hundreds or even thousands of activities that can be implemented in user-defined activities 420. However, any number and / or any type of activities can be used without departing from the scope of the present invention.

[0132] UI automation activities 450 are a subset of special lower-level activities written in lower-level code and facilitate interaction with the screen. UI automation activities 450 facilitate these interactions via drivers 460, which allow the robot to interact with the desired software. For example, drivers 460 may include operating system (OS) drivers 462, browser drivers 464, VM ​​drivers 466, enterprise application drivers 468, etc. In some embodiments, UI automation activities 450 can use one or more AI / ML models 480 to perform interactions with the computing system. In some embodiments, AI / ML models 480 can enhance drivers 460 or completely replace them. In fact, in some embodiments, drivers 460 are not included.

[0133] Driver 460 can interact with the OS at a low level via OS driver 462 to find hooks, monitor keys, etc. Driver 460 can facilitate communication with For example, the "click" activity performs the same role in these different applications via driver 460.

[0134] Figure 5 is an architectural diagram illustrating a computing system 500 configured to provide a design-time AI / ML recommendation engine for RPA according to an embodiment of the present invention. In some embodiments, computing system 500 may be one or more of the computing systems depicted and / or described herein. In some embodiments, computing system 500 may be part of a hyper-automation system, such as Figure 1 and 2 . The computing system 500 includes a bus 505 or other communication mechanism for transmitting information, and (multiple) processors 510 coupled to the bus 505 for processing information. The (multiple) processors 510 can be any type of general or special-purpose processor, including a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. The (multiple) processors 510 may also have multiple processing cores, and at least some of the cores may be configured to perform specific functions. In some embodiments, multi-parallel processing may be used. In some embodiments, at least one of the (multiple) processors 510 may be a neuromorphic circuit including a processing element that simulates a biological neuron. In some embodiments, the neuromorphic circuit may not require typical components of the von Neumann computing architecture.

[0135] The computing system 500 also includes a memory 515 for storing information and instructions to be executed by the (multiple) processors 510. The memory 515 may include any combination of random access memory (RAM), read-only memory (ROM), flash memory, cache, static memory such as a magnetic disk or optical disk, or any other type of non-transitory computer-readable medium or a combination thereof. The non-transitory computer-readable medium may be any available medium that can be accessed by the (multiple) processors 510 and may include volatile media, non-volatile media, or both. The medium may also be removable, non-removable, or both. The computing system 500 includes a communication device 520 such as a transceiver to provide access to a communication network via a wireless and / or wired connection. In some embodiments, the communication device 520 may include one or more antennas that are single, array, phased, switched, beamformed, beam-steering, combinations thereof, and / or any other antenna configurations without departing from the scope of the present invention.

[0136] Processor(s) 510 are also coupled to a display 525 via bus 505. Any suitable display device and haptic I / O may be used without departing from the scope of the present invention.

[0137] A keyboard 530 and a cursor control device 535, such as a computer mouse, touchpad, etc., are further coupled to bus 505 to enable a user to interact with computing system 500. However, in some embodiments, a physical keyboard and mouse may not be present, and a user may interact with the device only through display 525 and / or touchpad (not shown). Any type and combination of input devices may be used, depending on design choice. In some embodiments, there is no physical input device and / or display. For example, a user may interact with computing system 500 remotely via another computing system in communication with it, or computing system 500 may operate autonomously.

[0138] The memory 515 stores software modules that provide functionality when executed by the processor(s) 510. These modules include an operating system 540 for the computing system 500. These modules also include an AI / ML recommendation engine model 545 that is configured to perform all or a portion of the AI / ML processes described herein, or derivatives thereof. The computing system 500 may include one or more additional functional modules 550 that include additional functionality.

[0139] Those skilled in the art will appreciate that a "system" may be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a mobile phone, a tablet computing device, a quantum computing system, or any other suitable computing device, or a combination of devices, without departing from the scope of the present invention. Presenting the above functions as being performed by a "system" is not intended to limit the scope of the present invention in any way, but is intended to provide an example of many embodiments of the present invention. In fact, the methods, systems, and devices disclosed herein may be implemented in a localized and distributed form that conforms to computing technology (including cloud computing systems). The computing system may be a part of a local area network (LAN), a mobile communication network, a satellite communication network, the Internet, a public cloud or a private cloud, a hybrid cloud, a server farm, any combination thereof, or otherwise accessible thereto. Any localized or distributed architecture may be used without departing from the scope of the present invention.

[0140] It should be noted that some of the system features described in this specification have been presented as modules in order to more specifically emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors (such as logic chips, transistors), or other discrete components. A module may also be implemented in a programmable hardware device such as a field programmable gate array, programmable array logic, a programmable logic device, a graphics processing unit, etc.

[0141] Module can also be implemented at least in part with software for various types of processors to perform.Identified executable code unit can for example include one or more physical or logical blocks of computer instructions, which can for example be organized as objects, processes or functions.However, the executable file of the identified module does not need to be physically located together, but can include different instructions stored in different locations, which, when these instructions are logically connected together, form the module and realize the described purpose of the module.In addition, module can be stored on a computer-readable medium, which can be for example a hard disk drive, a flash memory device, a RAM, a magnetic tape and / or any other such non-transient computer-readable medium that is used to store data without departing from the scope of the present invention.

[0142] In fact, a module of executable code can be a single instruction or many instructions, and can even be distributed over several different code segments, between different programs, and across several memory devices. Similarly, operational data can be identified and illustrated herein within a module, and can be embodied in any suitable form and organized within any suitable type of data structure. The operational data can be collected as a single data set, or can be distributed over different locations, including over different storage devices, and can exist at least in part only as an electronic signal on a system or network.

[0143] Various types of AI / ML models can be trained and deployed without departing from the scope of the present invention. For example, Fig. 6A Illustrated is an example of a neural network 600 that has been trained to provide suggestions for the next activity or sequence of activities in accordance with an embodiment of the present invention.

[0144] Neural network 600 includes multiple hidden layers. DLNN and shallow learning neural network (SLNN) usually have multiple layers, but in some cases SLNN may only have one or two layers, and usually less than DLNN. Typically, a neural network architecture includes an input layer, multiple intermediate layers, and an output layer, as in the case of neural network 600.

[0145] DLNNs often have multiple layers (e.g., 10, 50, 200, etc.) and subsequent layers often reuse features from previous layers to compute more complex, general functions. On the other hand, SLNNs tend to have only a few layers and are relatively fast to train because expert features are created in advance from raw data samples. However, feature extraction is laborious. On the other hand, DLNNs typically do not require expert features, but tend to take longer to train and have more layers.

[0146] For both methods, the layers are trained simultaneously on the training set, usually checking for overfitting on an isolated cross-validation set. Both techniques can produce excellent results, and there is considerable enthusiasm for both methods. The optimal size, shape, and number of layers varies depending on the problem being solved by the respective neural network.

[0147] Return to Fig. 6A , RPA workflow, one or more application screenshots, graphical elements detected on the screen using CV, activity parameters, etc. are provided as input layers and fed as inputs to the J neurons of hidden layer 1. In this example, although a portion of these inputs are fed to each neuron, it is also possible to use various architectures alone or in combination, including but not limited to feedforward networks, radial basis networks, deep feedforward networks, deep convolutional inverse graph networks, convolutional neural networks, recurrent neural networks, artificial neural networks, long / short-term memory networks, gated recurrent unit networks, generative adversarial networks, liquid machines, autoencoders, variational autoencoders, denoising autoencoders, sparse autoencoders, extreme learning machines, echo state networks, Markov chains, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, deep residual networks, Kohonen networks, deep belief networks, deep convolutional networks, support vector machines, neural Turing machines, or any other suitable type or combination of neural networks without departing from the scope of the present invention.

[0148] Hidden layer 2 receives input from hidden layer 1, hidden layer 3 receives input from hidden layer 2, and so on for all hidden layers until the last hidden layer provides its output as input to the output layer. Although multiple suggestions are shown here as outputs, in some embodiments, only a single output suggestion is provided. In some embodiments, the suggestions are ranked based on confidence scores.

[0149] It should be noted that the number of neurons I, J, K, and L is not necessarily equal. Thus, any desired number of layers may be used for a given layer of neural network 600 without departing from the scope of the present invention. In fact, in some embodiments, the types of neurons in a given layer may not all be the same.

[0150] The neural network 600 is trained to assign confidence scores to the appropriate outputs. To reduce inaccurate predictions, in some embodiments only those results with confidence scores that meet or exceed a confidence threshold may be provided. For example, if the confidence threshold is 80%, then outputs with confidence scores exceeding this amount may be used, and the rest may be ignored.

[0151] It should be noted that neural networks are probabilistic constructs that typically have (multiple) confidence scores. This can be a score that the AI / ML model learns based on how often similar inputs are correctly identified during training. Some common types of confidence scores include a decimal between 0 and 1 (which can also be interpreted as a confidence percentage), a number between negative ∞ and positive ∞, a set of expressions (e.g., "low", "medium", and "high"), etc. Various post-processing calibration techniques can also be employed to try to obtain more accurate confidence scores, such as temperature scaling, batch normalization, weight decay, negative log-likelihood (NLL), etc.

[0152] "Neurons" in neural networks are implemented by algorithms as mathematical functions that are generally based on the functions of biological neurons. Neurons receive weighted inputs and have summation and activation functions that govern whether they pass output to the next layer. The activation function can be a nonlinear threshold activity function, where nothing happens if the value is below the threshold, but then the function responds linearly above the threshold (i.e., a modified linear unit (ReLU) nonlinearity). Sum functions and ReLU functions are used for deep learning because real neurons can have roughly similar activity functions. Via linear transformations, information can be subtracted, added, etc. In essence, neurons act as gating functions that pass output to the next layer based on the control of their underlying mathematical functions. In some embodiments, different functions can be used for at least some neurons.

[0153] Figure 6B An example of a neuron 610 is shown in FIG. Input x1, x2, ..., x from the previous layern are assigned corresponding weights w1, w2, ..., w n . Therefore, the collective input from the previous neuron 1 is w1x1. These weighted inputs are used in the neuron summation function modified by the bias, for example:

[0154]

[0155] This sum is compared to the activation function f(x) to determine whether the neuron "fires". For example, f(x) can be given by:

[0156]

[0157] Therefore, the output y of neuron 610 can be given by:

[0158]

[0159] In this case, neuron 610 is a single layer perceptron. However, any suitable neuron type or combination of neuron types may be used without departing from the scope of the present invention. It should also be noted that in some embodiments the range of weight values ​​and / or output values ​​of the activation function may be different without departing from the scope of the present invention.

[0160] An objective or "reward function" is often used. The reward function explores intermediate transitions and steps with short-term and long-term rewards to guide the search of the state space and attempt to achieve the goal (e.g., finding the most accurate answer to a user query based on an associated metric). During training, various labeled data are fed through the neural network 600. Successful identifications strengthen the weights of the inputs to the neurons, while unsuccessful identifications weaken them. Cost functions such as mean squared error (MSE) or gradient descent can be used to penalize slightly wrong predictions rather than severely wrong predictions. If the performance of the AI / ML model does not improve after a certain number of training iterations, the data scientist can modify the reward function, provide corrections for incorrect predictions, and so on.

[0161] Backpropagation is a technique used to optimize synaptic weights in feed-forward neural networks. Backpropagation can be used to "peel the lid" on the hidden layers of a neural network to see how much loss each node is responsible for, and then update the weights in such a way that the loss is minimized by giving lower weights to nodes with higher error rates and vice versa. In other words, backpropagation allows data scientists to repeatedly adjust weights to minimize the difference between actual output and desired output.

[0162] The back-propagation algorithm is mathematically built in optimization theory. In supervised learning, training data with known outputs is passed through a neural network and the error is calculated using a cost function from the known target output, giving the error for back-propagation. The error is calculated at the output and converted into a correction to the network weights that will minimize the error.

[0163] In the case of supervised learning, an example of backpropagation is provided below. Through a series of N nonlinear activity functions f between each layer i=1,...,N of the network i To process the column vector input x, the output of a given layer is first multiplied by the synaptic matrix W i , and add the bias vector b i The network output o is given by

[0164] o=f N (W N f N-1 (W N-1 f N-2 (...f1(W1x+b1)...)+b N-1 )+b N ) (4)

[0165] In some embodiments, o is compared to the target output t, resulting in an error It is desirable to minimize this.

[0166] Optimization in the form of a gradient descent procedure can be used to modify the synaptic weights W for each layer i to minimize the error. The gradient descent procedure requires computing an output o given an input x corresponding to a known target output t, and produces an error ot. This global error is then propagated backwards to give a local error for the weight update, which is similar to the calculation used for the forward propagation, but not exactly the same. In particular, the backpropagation step usually requires p j (n j )=f j ′(n j ) form of activity function, where n j is the network activity at layer j (i.e., n j =W j o j-1 +b j ) where o j =f j (n j ) and the prime symbol ′ denotes the derivative of the activity function f.

[0167] The weight update can be calculated as follows:

[0168]

[0169]

[0170]

[0171]

[0172]

[0173] in represents the Hadamard product (i.e., the element-wise product of two vectors), T represents the matrix transpose, and o j represents f j (W j o j-1 +b j ), where o0 = x. Here, the learning rate η is chosen with regard to machine learning considerations. Below, η is related to the neural Hebbian learning mechanism used in the neural implementation. Note that the synapses W and b can be combined into a large synaptic matrix, where the input vector is assumed to have additional synapses, and an extra column representing the b synapses is included in W.

[0174] The AI / ML model can be trained over multiple epochs until a good level of accuracy is achieved (e.g., 97% or better using an F2 or F4 threshold for detection and approximately 2,000 epochs). In some embodiments, this level of accuracy can be determined using an F1 score, an F2 score, an F4 score, or any other suitable technique without departing from the scope of the invention. Once trained on the training data, the AI / ML model can be tested on a set of evaluation data that the AI / ML model has not previously encountered. This helps ensure that the AI / ML model does not "overfit" and perform well on the training data but poorly on other data.

[0175] In some embodiments, it may not be known what level of accuracy the AI / ML model can achieve. Therefore, if the accuracy of the AI / ML model begins to decline when analyzing the evaluation data (i.e., the model performs well on the training data, but begins to perform poorly on the evaluation data), the AI / ML model may be trained for more periods of time on the training data (and / or new training data). In some embodiments, the AI / ML model is deployed only when the accuracy reaches a certain level or the accuracy of several trained AI / ML models is better than the currently deployed AI / ML model. In certain embodiments, a collection of trained AI / ML models can be used to complete a task. For example, one model can be trained to suggest the next activity or sequence of activities, another model can perform CV, and another model can attempt runtime repair when automation fails, and so on.

[0176] Some embodiments may use transformer networks such as SentenceTransformers TM , a Python library for state-of-the-art sentence, text, and image embeddings TM Framework. This transformer network learns associations between words and phrases with high and low scores. This trains the AI / ML model to determine what is close to the input and what is not close to the input respectively. The transformer network uses not only word / phrase pairs, but also field length and field type.

[0177] In some embodiments, natural language processing (NLP) technologies such as word2vec, BERT, GPT-3, ChatGPT, etc. can be used to promote semantic understanding and provide more accurate and human-like answers. Other technologies (such as clustering algorithms) can be used to find similarities between groups of elements. Clustering algorithms can include but are not limited to density-based algorithms, distribution-based algorithms, centroid-based algorithms, and hierarchy-based algorithms. K-means clustering algorithm, DBSCAN clustering algorithm, Gaussian mixture model (GMM) algorithm, balanced iterative reduction and clustering (BIRCH) algorithm using hierarchy, etc. These technologies can also help classification.

[0178] Figure 7 700 is a flowchart illustrating a process 700 for training (multiple) AI / ML models according to an embodiment of the present invention. In some embodiments, according to the above, (multiple) AI / ML models can be generative AI models. The neural network architecture of the AI / ML model typically includes multiple layers of neurons, including an input layer, an output layer, and a hidden layer. For example, see Fig. 6A and Figure 6B The input layer receives (multiple) text-based inputs, and the output layer generates (multiple) text-based responses. Intermediate hidden layers process the input data and generate intermediate representations of the inputs that are used to generate outputs. These hidden layers may include various types of neurons, such as convolutional neurons, recurrent neurons, and / or conversion neurons.

[0179] The training process begins by providing the RPA workflow, screenshots, graphical elements (if the automation is UI automation), and activity parameters (e.g., inputs and outputs), whether labeled or unlabeled, at 710. For UI automation, one or more graphs that provide relationships between descriptors, graphical elements, screens, versions, applications, etc. may be used. The AI / ML model is then trained for multiple epochs at 720, and the results are reviewed at 730. While various types of AI / ML models may be used, LLMs and other generative AI models are typically trained using a process known as "supervised learning," which is also discussed above. Supervised learning involves providing the model with a large data set, which the model uses to learn the relationship between inputs and outputs. During the training process, the model adjusts the weights and biases of the neurons in the neural network to minimize the difference between the predicted outputs and the actual outputs in the training data set.

[0180] One aspect of the model in some embodiments is the use of transfer learning. For example, transfer learning can utilize a pre-trained model (such as ChatGPT), which is fine-tuned for a specific task or domain in step 720. This allows the model to leverage knowledge already learned from the pre-training phase and adapt it to a specific application via the training phase of step 720.

[0181] The pre-training phase involves training the model on a possibly more general initial training data set. During this phase, the model learns relationships in the data. In the fine-tuning phase (e.g., if the pre-trained model is used as the initial basis for the final model, then in some embodiments, in addition to the initial training phase or in place of the initial training phase and performed during step 720), the pre-trained model is adapted to a specific task or domain by training the model on a smaller data set specific to the task. For example, in some embodiments, the model can focus on UI automation and utilize an object repository. This can help the model more accurately suggest the next activity or sequence of activities compared to models trained on multiple or all types of automation. Fine-tuning allows the model to learn the nuances of the task, such as specific vocabulary and grammar, certain graphic features, certain data formats, etc., without the need for the large amount of data required to train the model from scratch. By leveraging the knowledge learned in the pre-training phase, the fine-tuned model can achieve state-of-the-art performance on a specific task using relatively little additional training data.

[0182] If the AI / ML model fails to meet the desired confidence threshold at 740, the training data is supplemented and / or the reward function is modified at 750 to help the AI / ML model better achieve its goal, and the process returns to step 720. If the AI / ML model meets the confidence threshold at 740, the AI / ML model is tested on the evaluation data at 760 to ensure that the AI / ML model generalizes well and the AI / ML model is not overfit with respect to the training data. The evaluation data includes information that has not been processed by the AI / ML model before. If the confidence threshold for the evaluation data is met at 770, the AI / ML model is deployed at 780. If not, the process returns to step 750 and the AI / ML model is further trained.

[0183] Figure 8 800 is a basic example of a diagram illustrating the relationship between application elements according to an embodiment of the present invention. Diagram 800 includes the relationship between applications and their versions, versions and screens, screens and graphical elements (GEs), and graphical elements and their descriptors. In some embodiments, there may be different connections between these nodes. For example, descriptors and / or graphical elements may be connected to the corresponding version(s) in which they appear. This information may be helpful in training AI / ML models to more accurately suggest activities and activity sequences for UI automation.

[0184] Fig. 9 is a flow chart illustrating a process 900 of a design-time AI / ML recommendation engine for RPA according to an embodiment of the present invention. If transfer learning is used, the process may begin with pre-training one or more AI / ML models at 910. If not used, or after step 910, the (multiple) AI / ML model(s) are trained at 920 for the desired purpose. For example, (multiple) AI / ML models may be trained to suggest the next activity in an RPA workflow, suggest the next sequence of activities, suggest modifications to the parameters of at least one of the one or more activities, or any combination thereof based on the content of one or more activities in the RPA workflow. Other models may also be trained, such as CV models, focused models for specific types of automation, and the like.

[0185] In some embodiments, at least one of the one or more AI / ML models is trained using a context based on a previously developed RPA workflow. In some UI automation embodiments, at least one of the one or more AI / ML models is trained using information related to applications, versions of applications, screens of applications, and graphical elements of screens from an object repository. In some UI automation embodiments, at least one of the one or more AI / ML models is trained using one or more graphs that include relationships between applications, versions of applications, screens of applications, and graphical elements of screens from an object repository. In some embodiments, at least one of the one or more AI / ML models trained using one or more graphs is trained to recognize ontological associations from the one or more graphs.

[0186] After training the AI / ML model(s), information related to the RPA workflow being developed is provided to at least one of the one or more AI / ML models at 930. The AI / ML model(s) then provide a recommendation received at 940 regarding a next activity, a next sequence of activities, a suggested modification to a parameter, or any combination thereof. In some embodiments, a confidence score(s) for the recommendation is also provided as an output. In certain embodiments, the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof is based on the context of the RPA workflow in the current state of development.

[0187] The user of the RPA designer application is then: (1) notified of the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof; and (2) automatically modifies the RPA workflow at 950 to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof into the RPA workflow being developed. In some embodiments, the automatic modification of the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to a parameter, or a combination thereof is performed in response to a confidence score exceeding an automatic insertion threshold. In some embodiments, at 960, a next task to be performed by the RPA workflow is suggested based on an output from at least one of the one or more AI / ML models. In certain embodiments, the suggested next task is automatically performed when a confidence score associated with the suggestion for the task exceeds an automatic performance threshold.

[0188] In some embodiments, workflow suggestions and / or repairs can be performed iteratively. If the AI / ML model detects a problem in a workflow during development, the AI / ML model can automatically repair the workflow at 970 by modifying activity parameters, replacing one activity with another activity suitable for the task, adding one or more additional activities. If more development is required at 980, the process returns to step 930 (e.g., after a user creates a new activity, after a user creates a sequence of activities, after a predetermined amount of time, after a certain number of activities are created, after a user clicks a review button, etc.). In this way, problems in RPA workflows can be resolved in an iterative manner, users can develop workflows more quickly, and the effectiveness and accuracy of workflows can be improved.

[0189] Fig.10 1 is a flow chart illustrating a process 1000 for performing RPA error preemption or self-healing for an RPA robot at runtime according to an embodiment of the present invention. The process begins with the execution of automation by the RPA robot at 1010. In some embodiments, the RPA robot or an AI / ML model invoked by the RPA robot may search for changes to an application associated with the automation. If the version of the application changes or a different application is being used for a task associated with the automation, this may indicate that the automation should be changed.

[0190] If the application or its version has changed in a way that will affect the automation, the RPA robot preempts the failure at 1020, or alternatively, the RPA robot determines that an activity of the RPA workflow failed at 1020. In the case of preemption, the RPA robot calls the AI / ML model to determine the modifications that should be made to the workflow, makes the changes, and generates a new automation that incorporates the changes to replace the previous automation. For example, in the case where the RPA robot determines that the activity failed, the failure can be detected by not finding a graphical element associated with the activity, the activity not completing within a period of time and moving to the next activity, and the expected screen not appearing. If the preemption solves the problem or the error type is an error that should not be attempted to repair at 1030, then at 1040, the RPA automation is executed in the case of preemption, or the RPA automation is ended and the user is notified (e.g., via the RPA robot, the robot tray application, etc.) whether the repair should not be attempted.

[0191] However, if an error occurs and the error is of a type that should be attempted to be repaired at 1030, then information is provided to one or more AI / ML models that have been trained to perform self-healing for the RPA robot at 1050. This information may include information related to the failed activity (e.g., its parameters, UI descriptors (if used), results of previous activities, etc.), screenshots, graphical element information from the CV model, etc. At 1060, suggested correction(s) are received from the one or more AI / ML models based on the input information from step 1050.

[0192] The RPA robot, RPA designer application, or another application then attempts the correction(s) at 1070. If the correction is successful at 1080, the RPA workflow is repaired at 1090 and the process returns to execute the repaired RPA automation at 1010. In some embodiments, the RPA automation is resumed from the activity where the failure occurred. In some embodiments, the RPA automation is executed again from the beginning. However, if the correction(s) attempted at 1080 fail, the process proceeds to step 1040.

[0193] According to an embodiment of the present invention, Fig. 9 and Fig.10 The process steps performed in the method may be performed by a computer program encoding instructions for a processor(s) to perform Fig. 9 and Fig.10 The computer program may be embodied on a non-transitory computer readable medium. The computer readable medium may be, but is not limited to, a hard drive, a flash memory device, a RAM, a magnetic tape, and / or any other such medium or combination of media used to store data. The computer program may include coded instructions for controlling the processor(s) of a computing system (e.g., Figure 5 The processor(s) 510 of the computing system 500 are used to implement Fig. 9 and Fig.10 All or part of the process steps described in the present invention, the coded instructions may also be stored on a computer readable medium.

[0194] A computer program may be implemented in hardware, software or a hybrid implementation. A computer program may consist of modules that are operatively in communication with each other and are designed to pass information or instructions for display. A computer program may be configured to operate on a general purpose computer, an ASIC or any other suitable device.

[0195] It is easy to understand that, as generally described and illustrated in the accompanying drawings herein, the components of the various embodiments of the present invention can be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present invention as represented in the accompanying drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention.

[0196] In one or more embodiments, the features, structures or characteristics of the present invention described in the entire specification may be combined in any suitable manner. For example, references to "certain embodiments", "some embodiments" or similar language throughout the specification mean that specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the appearance of phrases "in certain embodiments", "in some embodiments", "in other embodiments" or similar language throughout this specification may not all refer to embodiments of the same group, and the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

[0197] It should be noted that references to features, advantages, or similar language throughout this specification do not mean that all features and advantages that can be achieved with the present invention should be or are in any single embodiment of the present invention. Instead, language referring to features and advantages should be understood to mean that a particular feature, advantage, or characteristic described in conjunction with an embodiment is included in at least one embodiment of the present invention. Therefore, discussion of features and advantages and similar language throughout this specification may, but does not necessarily, refer to the same embodiment.

[0198] In addition, the features, advantages and characteristics of the present invention described may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific features or advantages of a particular embodiment. In other examples, additional features and advantages that may not be present in all embodiments of the present invention may be recognized in certain embodiments.

[0199] Those skilled in the art will readily appreciate that the present invention as discussed above may be practiced with steps in a different order and / or with hardware elements that are different from the disclosed configurations. Therefore, although the present invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will be apparent while still being within the spirit and scope of the present invention. Therefore, in order to determine the scope and limits of the present invention, reference should be made to the appended claims.

Claims

1. One or more non-transitory computer-readable media storing one or more computer programs configured to cause at least one processor to: providing information related to a Robotic Process Automation (RPA) workflow to one or more artificial intelligence (AI) / machine learning (ML) models, the RPA workflow being developed in an RPA designer application and comprising one or more activities, the one or more AI / ML models being trained to suggest a next activity in the RPA workflow, suggest a next sequence of activities in the RPA workflow, suggest a modification to a parameter of at least one of the one or more activities in the RPA workflow, or any combination thereof based on content of the one or more activities in the RPA workflow; receiving output from the one or more AI / ML models, the output comprising the suggested next activity, the suggested sequence of next activities, the suggested modification to the parameter, or any combination thereof; and Notifying a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof, or automatically modifying the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof into the RPA workflow.

2. One or more non-transitory computer-readable media according to claim 1, wherein the one or more AI / ML models are configured to provide a confidence score for the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof.

3. One or more non-transitory computer-readable media according to claim 1, wherein the automatic modification of the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof is performed in response to the confidence score exceeding an automatic insertion threshold.

4. One or more non-transitory computer-readable media according to claim 1, wherein the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof is based on the context of the RPA workflow in a current development state.

5. The one or more non-transitory computer-readable media of claim 1, wherein at least one of the one or more AI / ML models is trained using a context based on an RPA workflow that has been previously developed.

6. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs are further configured to cause the at least one processor to: A next task to be performed for the RPA workflow is suggested based on an output from at least one of the one or more AI / ML models.

7. The one or more non-transitory computer-readable media of claim 6, wherein the one or more computer programs are further configured to cause the at least one processor to: The suggested next task is automatically performed when a confidence score associated with the suggested next task exceeds an automatic performance threshold.

8. The one or more non-transitory computer-readable media of claim 1, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using information about an application, a version of the application, a screen of the application, and graphical elements of the screen from an object repository.

9. The one or more non-transitory computer-readable media of claim 1, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using one or more graphs that include applications from an object repository, versions of the applications, screens of the applications, and relationships between graphical elements of the screens.

10. The one or more non-transitory computer-readable media of claim 9, wherein the at least one AI / ML model of the one or more AI / ML models trained using the one or more graphs is trained to identify ontology associations from the one or more graphs.

11. The one or more non-transitory computer-readable media of claim 1 , wherein the one or more computer programs are further configured to cause the at least one processor to: detecting one or more problems in the RPA workflow; and Automatically repair the RPA workflow.

12. The one or more non-transitory computer-readable media of claim 11, wherein the automatic repair comprises modifying activity parameters, replacing the activity with another activity suitable for the task, adding one or more additional activities that cause the activity to run, or any combination thereof.

13. The one or more non-transitory computer-readable media of claim 11, wherein the automatic repair of the RPA workflow is performed iteratively during development of the RPA workflow.

14. One or more computing systems comprising: Memory, which stores computer program instructions; as well as at least one processor configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor to: providing information related to a Robotic Process Automation (RPA) workflow to one or more artificial intelligence (AI) / machine learning (ML) models, the RPA workflow being developed in an RPA designer application and comprising one or more activities, the one or more AI / ML models being trained to suggest a next activity in the RPA workflow, suggest a next sequence of activities in the RPA workflow, suggest a modification to a parameter of at least one of the one or more activities in the RPA workflow, or any combination thereof based on content of the one or more activities in the RPA workflow; receiving output from the one or more AI / ML models, the output comprising the suggested next activity, the suggested sequence of next activities, the suggested modification to the parameter, or any combination thereof; as well as notifying a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof, or automatically modifying the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof into the RPA workflow, The suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof are based on the context of the RPA workflow in a current development state.

15. One or more computing systems according to claim 14, wherein the automatic modification of the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof is performed in response to the confidence score exceeding an automatic insertion threshold.

16. The one or more computing systems of claim 14, wherein at least one of the one or more AI / ML models is trained using a context based on an RPA workflow that has been previously developed.

17. The one or more computing systems of claim 14, wherein the computer program instructions are further configured to cause the at least one processor to: A next task to be performed for the RPA workflow is suggested based on an output from at least one of the one or more AI / ML models.

18. The one or more computing systems of claim 17, wherein the computer program instructions are further configured to cause the at least one processor to: The suggested next task is automatically performed when a confidence score associated with the suggested next task exceeds an automatic performance threshold.

19. The one or more computing systems of claim 14, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using information about an application, a version of the application, a screen of the application, and graphical elements of the screen from an object repository.

20. The one or more computing systems of claim 14, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using one or more graphs that include applications from an object repository, versions of the applications, screens of the applications, and relationships between graphical elements of the screens.

21. The one or more computing systems of claim 20, wherein the at least one AI / ML model of the one or more AI / ML models trained using the one or more graphs is trained to identify ontological associations from the one or more graphs.

22. The one or more computing systems of claim 14, wherein the one or more computer programs are further configured to cause the at least one processor to: detecting one or more problems in the RPA workflow; and Automatically repair the RPA workflow, where The automatic repair includes modifying activity parameters, replacing the activity with another activity suitable for the task, adding one or more additional activities that cause the activity to run, or any combination thereof.

23. A computer-implemented method comprising: providing, by a computing system, information related to a Robotic Process Automation (RPA) workflow comprising one or more activities to one or more Artificial Intelligence (AI) / Machine Learning (ML) models, the RPA workflow being developed in an RPA designer application, the one or more AI / ML models being trained to suggest a next activity in the RPA workflow, suggest a next sequence of activities in the RPA workflow, suggest a modification to a parameter of at least one of the one or more activities in the RPA workflow, or any combination thereof based on content of the one or more activities in the RPA workflow; receiving, by the computing system, output from the one or more AI / ML models, the output comprising the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or any combination thereof; notifying, by the computing system, a user of the RPA designer application of the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof, or automatically modifying the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof into the RPA workflow; as well as The computing system recommends a next task to be performed for the RPA workflow based on an output from at least one of the one or more AI / ML models, or automatically performs the recommended next task when a confidence score associated with the recommended next task exceeds an automatic performance threshold.

24. The computer-implemented method of claim 23, wherein the automatic modification of the RPA workflow to incorporate the suggested next activity, the suggested next sequence of activities, the suggested modification to the parameter, or a combination thereof is performed in response to the confidence score exceeding an automatic insertion threshold.

25. The computer-implemented method of claim 23, wherein the suggested next activity, the suggested sequence of next activities, the suggested modification to the parameter, or a combination thereof is based on the context of the RPA workflow in a current development state.

26. The computer-implemented method of claim 23, wherein at least one of the one or more AI / ML models is trained using a context based on an RPA workflow that has been previously developed.

27. The computer-implemented method of claim 23, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using information about an application, a version of the application, a screen of the application, and graphical elements of the screen from an object repository.

28. The computer-implemented method of claim 23, wherein: The RPA workflow is related to UI automation, and At least one of the one or more AI / ML models is trained using one or more graphs that include applications from an object repository, versions of the applications, screens of the applications, and relationships between graphical elements of the screens.

29. The computer-implemented method of claim 28, wherein the at least one of the one or more AI / ML models trained using the one or more graphs is trained to identify ontological associations from the one or more graphs.

30. The computer-implemented method of claim 23, further comprising: detecting, by the computing system, one or more problems in the RPA workflow; as well as automatically repairing the RPA workflow by the computing system, Wherein the automatic repair includes modifying activity parameters, replacing the activity with another activity suitable for the task, adding one or more additional activities that cause the activity to run, or any combination thereof.

Images