Action and / or process determination and recommendation for robotic process automation using semantic action graphs

Abstract

To provide a computer-implemented method for determining actions and / or processes for robotic process automation (RPA) using semantic action graphs, and multiple non-transitory computer-readable media.SOLUTION: The method derives metadata for deploying a semantic action graph, which is a graph that stores individual actions and potentially graphical elements and / or text associated with the actions as nodes and stores a relation between nodes as edges, from task mining applications that can monitor interactions of users with computing systems, workforce intelligence, etc. The metadata includes, for example, window titles and website uniform resource locators (URLs).SELECTED DRAWING: None

Description

[Technical Field]

[0001] The present invention relates generally to artificial intelligence (AI), and more particularly to determining and recommending actions and / or processes for robotic process automation (RPA) using semantic action graphs. [Background technology]

[0002] Users working with computing systems frequently perform a variety of tasks on the computing system. However, these tasks are frequently repeated, resulting in reduced efficiency. Therefore, improved and / or alternative approaches may be beneficial. Summary of the Invention

[0003] Certain embodiments of the present invention may provide solutions to problems and needs in the field that have not yet been fully identified, appreciated, or solved by current RPA technology. For example, some embodiments of the present invention relate to determining and recommending RPA actions and / or processes using semantic action graphs.

[0004] In an embodiment, one or more non-transitory computer-readable media store one or more computer programs. The one or more computer programs are configured to cause at least one processor to acquire task mining data from multiple user computing systems and apply a normalization technique to the task mining data to achieve normalization and reduce the task mining information to a range within one or more normalization curves. The one or more computer programs are also configured to cause the at least one processor to classify the normalized data into action groups that correlate with the event metrics data using one or more classification algorithms, one or more clustering algorithms, or both. The one or more computer programs are further configured to cause the at least one processor to index the action groups, search, connect, and correlate the action groups, and apply reinforcement learning in a supervised learning process based on the recorded action list. The one or more computer programs are further configured to cause the at least one processor to generate a semantic action graph using the indexed information.

[0005] In another embodiment, the computer-implemented method includes applying, by one or more computing systems, a normalization technique to the task mining data to achieve normalization and reduce the task mining information to a range within one or more normalization curves. The computer-implemented method also includes classifying, by one or more computing systems, the normalized data into action groups that correlate with the event metrics data using one or more classification algorithms, one or more clustering algorithms, or both. The computer-implemented method further includes indexing, by one or more computing systems, the action groups to search, connect, and correlate the action groups, and applying, by one or more computing systems, reinforcement learning in a supervised learning process based on the recorded action list. The computer-implemented method further includes generating, by the one or more computing systems, a semantic action graph using the indexed information.

[0006] In yet another embodiment, a computing system includes a memory storing computer program instructions and at least one processor configured to execute the computer program instructions. The computer program instructions are configured for the at least one processor to periodically obtain task mining data from a plurality of user computing systems and apply a normalization technique to the task mining data to achieve normalization and reduce the task mining information to ranges within one or more normalization curves. The computer program instructions are also configured for the at least one processor to classify the normalized data into action groups that correlate with the event metrics data using one or more classification algorithms, one or more clustering algorithms, or both. The computer program instructions are further configured for the at least one processor to index the action groups to search, connect, and correlate the action groups. The computer program instructions are further configured for the at least one processor to apply reinforcement learning in a supervised learning process based on the recorded action list and use the indexed information to generate a new semantic action graph or extend an existing semantic action graph. [Brief explanation of the drawings]

[0007] So that the advantages of particular embodiments of this invention may be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments which are illustrated in the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope, but the invention will be described and explained with additional specificity and detail through the use of the following accompanying drawings, in which:

[0008] [Figure 1] FIG. 1 is an architectural diagram illustrating a hyper-automation system, according to an embodiment of the present invention.

[0009] [Figure 2] FIG. 1 is an architectural diagram illustrating an RPA system according to an embodiment of the present invention.

[0010] [Figure 3] FIG. 1 is an architectural diagram illustrating a deployed RPA system according to an embodiment of the present invention.

[0011] [Figure 4] FIG. 2 is an architecture diagram illustrating the relationships between designers, activities, and drivers according to an embodiment of the present invention.

[0012] [Figure 5] FIG. 1 is an architectural diagram illustrating a computing system configured to perform RPA action decisions and recommendations using a semantic action graph, according to an embodiment of the present invention.

[0013] [Figure 6A] 1 illustrates an example of a neural network trained to augment RPA action decisions and recommendations using a semantic action graph, according to an embodiment of the present invention.

[0014] [Figure 6B] 1 illustrates an example neuron according to an embodiment of the present invention.

[0015] [Figure 7] 1 illustrates a portion of a semantic action graph representing a task according to an embodiment of the present invention.

[0016] [Figure 8] 1 is a flowchart illustrating a process for periodically processing task mining data and generating a semantic action graph according to an embodiment of the present invention.

[0017] [Figure 9]1 is a flowchart illustrating a process for monitoring a user of a computing system using a semantic action graph to provide recommendations for next actions and courses of action, according to an embodiment of the present invention.

[0018] Unless otherwise noted, like reference characters denote corresponding features consistently throughout the accompanying drawings. DETAILED DESCRIPTION OF THE INVENTION

[0019] Detailed Description of the Embodiments Some embodiments relate to determining and recommending RPA actions and / or processes using a semantic action graph. As used herein, a "semantic action graph" is a graph that stores individual actions and potential graphical elements and / or text associated with the actions as nodes (e.g., user interactions with a computing system) and stores relationships between the nodes as edges. In some embodiments, metadata for creating the semantic action graph may be derived from a task mining application that can monitor user interactions with a computing system. Data recorded may include, but is not limited to, buttons clicked, where the mouse was moved, text entered into a field and the field into which the text was entered, when one window was minimized and another was opened, the application associated with the window, etc. The data collected by the listener may then be transmitted to one or more servers and stored in a database that serves as a repository.

[0020] In some embodiments, other sources of data besides task mining may be used, such as workforce intelligence at the business level rather than the user level. These sources include, but are not limited to, benchmarking business processes, business process flows, etc. For example, a business analyst may want to see patterns in process flows to understand how their business processes compare to other users. Thus, semantic action graphs can be targeted to users (e.g., recommending next nested actions), organizations, industries, entire products, etc. At the lowest level of granularity, mouse clicks, key presses, application programming interface (API) calls, system events, etc. may be recommended. At a higher level of granularity, recommendations may include initiating an order, creating a lead, an approval process, etc. Process logs and / or task logs may be used for metadata.

[0021] This metadata may be used to generate a semantic action graph. In the case of task mining data, this data is provided to one or more AI / ML models, such as generative AI model(s) (e.g., large-scale language models (LLMs)). The AI / ML models may learn from which application various events originated (e.g., initiated by a user or computing system) and capture the context to provide semantic understanding. For example, a user may be interacting with a specific form within an application, a user may be searching for a specific article on a topic, or a user may be performing a currency conversion via a website. Such information from related business entities may also be useful; once the context is learned, a join operation may be performed in the database to retrieve data from other related systems for related entities. For example, a specific approval process for one business may be similar to one used in other businesses; this information may be taken into account to generate similar automation, but tailored to the unique characteristics of that user / business. This data can be used to further improve the scope, accuracy, and effectiveness of the AI / ML models. User action classes can be used to segment related actions into groups. For example, clicking a button or clicking a text field are both "click" activities. Thus, the relationship between actions can be ontological in nature.

[0022] In some embodiments, rather than training AI / ML models offline, AI / ML training occurs online while the system is operating (i.e., while the production environment is running). This training may occur periodically (e.g., every 5 minutes, every hour, once a day, etc.). As events, such as operating system events, browser events, application events, mouse clicks, key presses, etc., are captured and processed using the process described below, the semantic action graph becomes richer and more robust, improving the accuracy of the system.

[0023] In some embodiments, the entire business intelligence and AI process is deployed online in a live environment as a streaming processing pipeline. As described above, task mining is used to collect information from user computing systems ("client events"), which is sent to the business intelligence and AI system. The workforce intelligence system periodically processes the collected information by performing the following operations:

[0024] Applying normalization techniques

[0025] Normalization techniques are applied to reduce information to a specific range within the normalization curve(s), especially when large amounts of data (e.g., terabytes, petabytes, etc.) are collected. Otherwise, the amount of information collected may overwhelm the system. Normalization is achieved through the calculation of business metrics, data processing, screen data, metadata, and user-controlled data conversion. Data normalization means that data is transformed into the same format and similar scale within acceptable limits in order to optimize the processes that perform data processing.

[0026] Business metrics calculations may include hourly application usage aggregation, website usage aggregation, the number of context switches, and user computing system idle time. Daily metrics are aggregated based on hourly metrics. In terms of data processing, the system backend acts as a streaming data processing pipeline, to which various data analysis technologies such as business intelligence and AI / ML are applied. Screen data transformation means that screen images are processed to obtain graphical elements and text within them (e.g., via CV and OCR). Metadata may include window titles, website universal resource locators (URLs), etc. User control data may include mouse and keyboard input.

[0027] Classifying Datasets

[0028] The reduced and normalized dataset is then classified using classification and / or clustering algorithms into action group data that correlates with event metric data such as window events, browser events, application events, etc. "Action group" means a set of actions that apply to the same window, dialog, web page, etc. These classification and / or clustering algorithms include, but are not limited to, decision trees, ensemble trees, generalized additive models (GAMs), naive Bayes, k-nearest neighbors (kNNs), discriminant analysis, etc.

[0029] Index Classification Data

[0030] The classified / categorized information is then indexed to search, connect, and correlate the data. This is done using B-tree indexes, hash maps, etc.

[0031] Applying reinforcement learning

[0032] A reinforcement learning pattern matching algorithm is applied, which is then applied to a supervised learning process based on a customer-provided list of actions as feedback. Specifically, the user records a known list of actions (also called "tasks") as feedback / input to the system. The system uses the user's input to match the actions against indexed data.

[0033] Building or extending a semantic action graph

[0034] The semantic graph is constructed by the business intelligence and AI / ML models using the information indexed after a reinforcement learning run (or extended if it already exists, as the semantic action graph expands with new task-mining information and further training). The semantic action graph consists of nodes and edges and may be, for example, a directed acyclic graph. The AI / ML model(s) may be matched with a similar action group graph, where nodes represent action groups and paths are from one action group to another (e.g., from action group 1 to action group 2). In some embodiments, the semantic action graph may allow for some variation.

[0035] The above process provides a "task reference" where users provide tasks (recording a list of actions) as user feedback for reinforcement learning. The source data from task mining is applied in the above manner to output a semantic action graph after reinforcement learning. The sequence or order of actions is determined by pattern matching based on the tasks and timestamps recorded by the user. The generated index is used in graph matching to determine which path(s) are important and which paths are minor branches that can be trimmed (treated as acceptable variants).

[0036] Once a semantic action graph is created, an RPA robot may use it to recommend next action(s) to a user when using his or her computing system, to recommend next activities when a user is designing an RPA workflow, or to predict outcomes and recommend workaround paths for an RPA workflow. In some embodiments, a semantic action graph may be implemented at two hierarchical levels: the action level and the application level. A selected action node in the graph may provide the outcome, the affected path, and a recommended workaround path.

[0037]

[0038] FIG. 1 is an architectural diagram illustrating a hyperautomation system 100, according to an embodiment of the present invention. As used herein, “hyperautomation” refers to an automation system that brings together process automation components, integrated tools, and technologies that amplify the ability to automate work. For example, in some embodiments, RPA is used at the core of the hyperautomation system, and in certain embodiments, automation capabilities may be extended with AI / ML, process mining, analytics, and / or other advanced tools. As the hyperautomation system learns processes, trains AI / ML models, and employs analytics, for example, more knowledge work may be automated, and computing systems within an organization, e.g., both those used by individuals and those operating autonomously, may all be engaged as participants in the hyperautomation process. The hyperautomation system of some embodiments enables users and organizations to efficiently and effectively discover, understand, and extend automation.

[0039] The hyperautomation system 100 includes user computing systems such as a desktop computer 102, a tablet 104, and a smartphone 106. However, any desired user computing system, including, but not limited to, a smartwatch, a laptop computer, a server, an Internet of Things (IoT) device, etc., may be used without departing from the scope of the present invention. Also, while three user computing systems are shown in FIG. 1 , any suitable number of user computing systems may be used without departing from the scope of the present invention. For example, in some embodiments, tens, hundreds, thousands, or millions of user computing systems may be used. The user computing systems may be actively used by a user or may run automatically without much or any user input.

[0040] Each user computing system 102, 104, 106 has a respective automation process(es) 110, 112, 114 running thereon. In some embodiments, the automation processes are stored remotely (e.g., on a server 130 or on a database 140 accessed over a network 120) and loaded by an RPA robot to implement the automation. The automation may exist as a script (e.g., XML, XAML, etc.) or may be compiled into machine-readable code (e.g., as a digital link library).

[0041] The automation process(es) 110, 112, 114 may include, without limitation, an RPA robot, part of an operating system, downloadable application(s) for the respective computing system, any other suitable software and / or hardware, or any combination thereof, without departing from the scope of the present invention. In some embodiments, one or more of the process(es) 110, 112, 114 may be a listener. The listener may be an RPA robot, part of an operating system, a downloadable application for the respective computing system, or any other software and / or hardware without departing from the scope of the present invention. Indeed, in some embodiments, the logic of the listener(s) is implemented partially or fully via physical hardware.

[0042] The listeners monitor and record data related to user interactions with their respective computing systems and / or the operation of unattended computing systems and transmit the data over a network (e.g., a local area network (LAN), a mobile communications network, a satellite communications network, the Internet, any combination thereof, etc.) to the core hyperautomation system 120. The data may include, but is not limited to, which buttons were clicked, where the mouse was moved, text entered into a field, when one window was minimized and another was opened, the application associated with the window, etc. In certain embodiments, the data from the listeners may be transmitted periodically as part of a heartbeat message. In some embodiments, the data may be transmitted to the core hyperautomation system 120 when a predetermined amount of data has been collected, after a predetermined period of time has elapsed, or both. One or more servers, such as server 130, receive the data from the listeners and store it in a database, such as database 140.

[0043] An automation process may execute logic developed in a workflow during design time. In the case of RPA, a workflow may include a set of steps, defined herein as "activities," that are executed in sequence or some other logical flow. Each activity may include an action such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0044] In some embodiments, a long-running workflow for RPA is a master project that supports service orchestration, human intervention, and long-running transactions in an unattended environment. See, for example, U.S. Patent No. 10,860,905, which is incorporated herein by reference in its entirety. Human intervention occurs when a particular process requires human input for exception handling, approval, or validation before proceeding to the next step in the activity. In this case, process execution is paused, freeing up the RPA robot until the human task is completed.

[0045] Long-running workflows may support workflow fragmentation through persistence activities and may be combined with call process and non-user interaction activities to orchestrate human tasks with RPA robot tasks. In some embodiments, multiple or numerous computing systems may participate in the execution of a long-running workflow's logic. Long-running workflows may execute in sessions to facilitate rapid execution. In some embodiments, long-running workflows may orchestrate background processes that execute API calls and may include activities that execute in the long-running workflow session. These activities may, in some embodiments, be invoked by a call process activity. Processes with user interaction activities that execute in a user session may be invoked by starting a job from a conductor activity (conductors are described in more detail later in this specification). In some embodiments, users may interact through tasks that require the completion of a form in the conductor. An activity may be included that causes the RPA robot to wait for a form task to complete and then resume the long-running workflow.

[0046] One or more automation process(es) 110, 112, 114 are in communication with the core hyperautomation system 120. In some embodiments, the core hyperautomation system 120 may execute a conductor application on one or more servers, such as server 130. While one server 130 is shown for illustrative purposes, multiple or numerous servers in close proximity to one another or in a distributed architecture may be employed without departing from the scope of the present invention. For example, one or more servers may be provided for conductor functions, AI / ML model serving, certification, governance, and / or any other suitable functionality without departing from the scope of the present invention. In some embodiments, the core hyperautomation system 120 may incorporate or be part of a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, or the like. In certain embodiments, the core hyperautomation system 120 may host multiple software-based servers on one or more computing systems, such as server 130. In some embodiments, one or more servers of the core hyperautomation system 120, such as server 130, may be implemented via one or more virtual machines (VMs).

[0047] In some embodiments, one or more automation process(es) 110, 112, 114 may invoke one or more AI / ML models 132 deployed on or accessible by the core hyperautomation system 120 and trained to accomplish various tasks. For example, the AI / ML models 132 may include models trained to find different application versions, perform CV, perform OCR, generate UI descriptors, and provide suggestions for the next activity or sequence of activities in an RPA workflow. The AI / ML models may be trained using labeled data including elements of data sources (e.g., web pages, forms, scanned documents, application interfaces, screens, etc.), previously created RPA workflows, screenshots of various application screens in different versions with corresponding UI elements, libraries of UI objects, etc. The AI / ML models 132 may be trained to achieve a desired confidence threshold without overfitting to a given set of training data.

[0048] The AI / ML model 132 may be trained for any suitable purpose without departing from the scope of the present invention, as discussed in more detail later herein. Two or more AI / ML models 132 may be chained (e.g., serially, in parallel, or a combination thereof) in some embodiments so that they collectively provide collaborative output(s). The AI / ML model 132 may perform or assist in CV, OCR, document processing and / or understanding, semantic learning and / or analysis, analytical prediction, process discovery, task mining, testing, automated RPA workflow generation, sequence extraction, clustering detection, speech-to-text translation, any combination thereof, and the like. However, any desired number and / or type(s) of AI / ML models may be used without departing from the scope of the present invention. Using multiple AI / ML models, for example, allows a system to develop a holistic picture of what is happening on a given computing system. For example, one AI / ML model may perform OCR, another may detect buttons, another may compare sequences, and so on. Patterns may be determined individually by the AI / ML models or collectively by multiple AI / ML models. In particular embodiments, one or more AI / ML models are deployed locally on at least one computing system 102, 104, 106.

[0049] In some embodiments, multiple AI / ML models 132 may be used. Each AI / ML model 132 is an algorithm (or model) that runs on data, and the AI / ML model itself may be, for example, a deep learning neural network (DLNN) of artificial “neurons” trained on training data. In some embodiments, the AI / ML model 132 may have multiple layers that perform various functions, such as statistical modeling (e.g., hidden Markov models (HMMs)), and may utilize deep learning techniques (e.g., long short-term memory (LSTM) deep learning, encoding of prior hidden states, etc.) to perform desired functions.

[0050] The hyperautomation system 100, in some embodiments, may provide four main groups of functions: (1) discovery, (2) automation build, (3) management, and (4) engagement. Automation (e.g., running on a user computing system, server, etc.) may, in some embodiments, be performed by software robots such as RPA robots. For example, attended robots, unattended robots, and / or test robots may be used. Attended robots collaborate with users to assist them with tasks (e.g., via UiPath Assistant™). Unattended robots operate independently of users and may potentially run in the background without the user's knowledge. Test robots are unattended robots that run test cases against an application or RPA workflow. Test robots, in some embodiments, may run in parallel on multiple computing systems.

[0051] A discovery function may discover various business process automation opportunities and provide automated recommendations. Such functionality may be implemented by one or more servers, such as server 130. In some embodiments, the discovery function may include providing an automation hub, process mining, task mining, and / or task capture. An automation hub (e.g., UiPath Automation Hub™) may provide a mechanism for managing automation rollouts with visibility and control. Automation ideas may be crowdsourced from employees, for example, via a submission form. Calculations of the feasibility and ROI for automating these ideas may be provided, documentation for future automations may be collected, and collaboration may be provided to expedite automation discovery and creation.

[0052] Process mining (e.g., via UiPath Automation Cloud™ and / or UiPath AI Center™) refers to the process of collecting and analyzing data from applications (e.g., enterprise resource planning (ERP) applications, customer relationship management (CRM) applications, email applications, call center applications, etc.) to identify what end-to-end processes exist in an organization, how they can be effectively automated, and the impact of automation. This data may be obtained, for example, by listeners from user computing systems 102, 104, 106 and processed by a server, such as server 130. In some embodiments, one or more AI / ML models 132 may be employed for this purpose. This information may be exported to an automation hub to speed implementation and avoid manual information transfer. The goal of process mining may be to increase business value by automating processes within an organization. Some example goals of process mining include, but are not limited to, increased profits, improved customer satisfaction, regulatory and / or contractual compliance, improved employee efficiency, etc.

[0053] Task mining (e.g., via UiPath Automation Cloud™ and / or UiPath AI Center™) identifies and aggregates workflows (e.g., employee workflows) and then applies AI to uncover patterns and variations in routine tasks and score such tasks for ease of automation and potential savings (e.g., time and / or cost savings). One or more AI / ML models 132 may be employed to uncover repetitive task patterns within the data. Repetitive tasks ripe for automation may then be identified. This information may initially be provided by a listener and, in some embodiments, may be analyzed on a server of the core hyper-automation system 120, such as server 130. Findings from task mining (e.g., XAML process data) may be exported to process documentation or designer applications such as UiPath Studio™ to more quickly create and deploy automations. Task mining in some embodiments may include taking screenshots with user actions (e.g., mouse click locations, keyboard input, application windows and graphical elements with which the user was interacting, timestamps for the interactions, etc.), collecting statistical data (e.g., performance time, number of actions, text input, etc.), editing and annotating screenshots, specifying the types of actions to be recorded, etc.

[0054] Task capture (e.g., via the Task Mining Recorder) automatically records attended processes while users work, or provides a framework for unattended processes. Such documentation may include tasks that are desired to be automated in formats such as PDDs, skeleton workflows, capturing actions for each part of the process, recording user actions and automatically generating comprehensive workflow diagrams with details about each step, Microsoft Word® documents, XAML files, etc. Configurable workflows, in some embodiments, can be exported directly to designer applications such as UiPath Studio™. Task capture may simplify the requirements gathering process for both subject matter experts describing the process and Center of Excellence (CoE) members providing production-grade automation.

[0055] Building automations may be accomplished through a designer application (such as UiPath Studio™, UiPath StudioX™, or UiPath Studio Web™). For example, an RPA developer at RPA development facility 150 may use RPA designer application 154 on computing system 152 to build and test automations for various applications and environments, such as web, mobile, SAP®, and virtual desktop. API integration may be provided for various applications, technologies, and platforms. Predefined activities, drag-and-drop modeling, and a workflow recorder may facilitate automation with minimal coding. Document understanding functionality may be provided through drag-and-drop AI skills for data extraction and interpretation that invoke one or more AI / ML models 132. Such automations can handle virtually any document type and format, including tables, checkboxes, signatures, and handwriting. When data is validated or exceptions are handled, this information may be used to retrain the respective AI / ML models, improving their accuracy over time.

[0056] RPA designer application 152 may be designed to invoke one or more of trained AI / ML models 132 on server 130 and / or generative AI models 172 in a cloud environment over network 120 (e.g., a local area network (LAN), a mobile communication network, a satellite communication network, the Internet, any combination thereof, etc.) to assist in the RPA automation development process. In some embodiments, one or more of the AI / ML models may be packaged with RPA designer application 152 or may otherwise be stored locally on computing system 150.

[0057] In some embodiments, the RPA designer application 152 and one or more AI / ML models 132 can be configured to use an object repository stored in the database 140. See, for example, U.S. Pat. No. 11,748,069, incorporated herein by reference in its entirety. The object repository can include a library of UI objects that can be used to develop RPA workflows via the RPA designer application 152. The object repository can be used to add UI descriptors to activities in the workflows of the RPA designer application 152 for UI automation. In some embodiments, one or more of the AI / ML models 132 can generate new UI descriptors and add them to the object repository in the database 140. Once an automation is completed in the designer application 152, the automation can be published on the server 130 and pushed out to computing systems 102, 104, 106, etc.

[0058] Integration services allow developers to seamlessly combine UI automation and API automation, for example. Automations that require APIs or span both API and non-API applications and systems can be built. A repository (e.g., UiPath Object Repository™) or marketplace (e.g., UiPath Marketplace™) for pre-built RPA and AI templates and solutions may be provided to enable developers to more quickly automate a wide variety of processes. Thus, when building automations, hyperautomation system 100 may provide a user interface, development environment, API integration, pre-built and / or custom-built AI / ML models, development templates, an integrated development environment (IDE), and advanced AI capabilities. In some embodiments, hyperautomation system 100 enables the development, deployment, management, configuration, monitoring, debugging, and maintenance of RPA robots, which may provide automation for hyperautomation system 100.

[0059] In some embodiments, components of hyper-automation system 100, such as designer application(s) and / or external rules engines, provide support for managing and enforcing governance policies to control various functions provided by hyper-automation system 100. Governance is the ability for an organization to put policies in place to prevent users from developing automation (e.g., RPA robots) that can perform actions that could harm the organization, such as violating the EU General Data Protection Regulation (GDPR), the US Health Insurance Portability and Accountability Act (HIPAA), third-party application terms of use, etc. Because developers might otherwise create automations that violate privacy laws, terms of use, etc. during the execution of their automations, some embodiments implement access control and governance restrictions at the robot and / or robot design application level. This may provide an additional level of security and compliance to the automation process development pipeline in some embodiments by preventing developers from taking dependencies on unauthorized software libraries that may pose security risks or operate in a manner that violates policies, regulations, privacy laws, and / or privacy policies. See, for example, US Pat. No. 11,733,668, which is incorporated herein by reference in its entirety.

[0060] Management functions may provide management, deployment, and optimization of automation across an organization. Management functions may, in some embodiments, include orchestration, test management, AI capabilities, and / or insights. Management functions of hyperautomation system 100 may also act as an integration point with third-party solutions and applications for automation applications and / or RPA robots. Management functions of hyperautomation system 100 may include, but are not limited to, facilitating provisioning, deployment, configuration, queuing, monitoring, logging, and interconnection of RPA robots, among others.

[0061] Conductor applications, such as UiPath Orchestrator™ (which may be offered in some embodiments as part of UiPath Automation Cloud™, or may be offered on-premise, on a VM, in a private or public cloud, on a Linux™ VM, or as a cloud-native single-container suite via the UiPath Automation Suite™), provide orchestration capabilities to deploy, monitor, optimize, scale, and ensure the security of RPA robot deployments. Test suites (e.g., the UiPath Test Suite™) may provide test management to monitor the quality of deployed automation. Test suites may facilitate test planning and execution, requirements fulfillment, and defect traceability. Test suites may include comprehensive test reports.

[0062] Analytics software (e.g., UiPath Insights™) can track, measure, and manage the performance of deployed automation. Analytics software can align automation operations with specific key performance indicators (KPIs) and strategic outcomes for the organization. Analytics software can present results in a dashboard format that is more easily understood by human users.

[0063] Data services (e.g., UiPath Data Service™), stored in database 140, for example, can bring data to a single, scalable, and secure location with a drag-and-drop storage interface. Some embodiments may provide low-code or no-code data modeling and storage to automation while ensuring seamless access, enterprise-grade security, and scalability of data. AI capabilities may be provided by an AI center (e.g., UiPath AI Center™), which facilitates the incorporation of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options may make such capabilities accessible to non-data scientists. Deployed automation (e.g., RPA robots) may invoke AI / ML models from the AI ​​center, such as AI / ML model 132. The performance of AI / ML models may be monitored and trained and improved using human-verified data, such as provided by data review center 160. Human reviewers may provide labeled data to core hyperautomation system 120 via review application 152 on computing system 154. For example, the human reviewer may verify that predictions by the AI / ML model 132 and / or the generative AI model 172 are accurate or may otherwise provide corrections. This dynamic input may then be saved as training data for retraining the AI / ML model 132 and / or the generative AI model 172 and may be stored in a database, such as database 140. The AI ​​center may then schedule and execute training jobs to train new versions of the AI / ML model using the training data. Both positive and negative examples may be stored and used for retraining the AI / ML model 132 and / or the generative AI model 172.

[0064] Engagement capabilities engage humans and automation as one team for seamless collaboration around desired processes. Low-code applications can be built (e.g., via UiPath Apps™) to connect browser tabs with legacy software, even those lacking APIs in some embodiments. Applications can be quickly created using a web browser, for example, through a rich library of drag-and-drop controls. Applications can be connected to one automation or multiple automations.

[0065] Action Center (e.g., UiPath Action Center™) provides an easy and efficient mechanism for handing off processes from automation to humans and vice versa. Humans can provide approvals or escalations, handle exceptions, etc. Automation can then perform the automated functions of a given workflow.

[0066] A local assistant may be provided as a launch pad for users to launch automations (e.g., UiPath Assistant™). This functionality may be provided, for example, in a tray provided by the operating system and may allow users to interact with RPA robots and RPA robot-powered applications on their computing system. The interface may list automations approved for a given user and allow the user to run them. These may include ready-to-use automations from an automation marketplace, an automation hub's internal automation store, etc. When automations are running, they may run as local instances in parallel with other processes on the computing system so that the user can use the computing system while the automation performs its actions. In certain embodiments, the assistant is integrated with task capture functionality so that users can document their soon-to-be-automated processes from the assistant's launch pad.

[0067] Chatbots (e.g., UiPath Chatbots™), social messaging applications, and / or voice commands may enable users to execute automations, simplifying access to information, tools, and resources needed to interact with customers or perform other activities. Human-to-human conversations can be automated as easily as other processes. Triggered RPA robots activated in this way could potentially perform actions such as checking order status or posting data to a CRM using plain language commands.

[0068] End-to-end measurement and governance of automation programs at any scale may be provided by the hyper-automation system 100 in some embodiments. Accordingly, analytics (e.g., via UiPath Insights™) may be employed to understand automation performance. Data modeling and analytics using any combination of available business metrics and operational insights may be used for various automation processes. Custom-designed and pre-built dashboards may visualize data across desired metrics, discover new analytical insights, track performance indicators, discover ROI for automations, perform telemetry monitoring on user computing systems, detect errors and anomalies, and debug automations. An automation management console (e.g., UiPath Automation Ops™) may be provided to manage automations throughout their lifecycle. Organizations may govern how automations are built, what users can do with them, and which automations users have access to.

[0069] The hyper-automation system 100, in some embodiments, provides an iterative platform where processes can be discovered, automations can be built, tested, and deployed, performance can be measured, automation usage can be easily provided to users, feedback can be obtained, AI / ML models can be trained and retrained, and the process itself can be repeated, thereby facilitating a more robust and effective set of automations.

[0070] In some embodiments, generative AI models are used. Generative AI may generate various types of content, such as text, images, audio, and synthetic data. Various types of generative AI models may be used, including, but not limited to, LLMs, generative adversarial networks (GANs), variational autoencoders (VAEs), transformers, etc. These models may be part of the AI / ML models 132 hosted on the server 130. For example, generative AI models may be trained on large corpora of text information to perform semantic understanding, understand the nature of what is present on a screen from text, automatically generate code, etc. In particular embodiments, generative AI models 172 offered by existing cloud ML service providers, such as OpenAI®, Google®, Amazon®, Microsoft®, IBM®, Nvidia®, and Facebook®, may be employed and trained to provide such functionality. In generative AI embodiments in which the generative AI model(s) 172 are remotely hosted, the server 130 may be configured to integrate with third-party APIs, allowing the server 130 to send requests containing required input information to the generative AI model(s) 172 and receive their replies (e.g., semantic matching of fields between application versions, classification of application types on screens, etc.). Such embodiments can provide a more advanced and sophisticated user experience as well as access to cutting-edge natural language processing (NLP) and other ML capabilities offered by these companies.

[0071] One aspect of generative AI models in some embodiments is the use of transfer learning. In transfer learning, a pre-trained generative AI model, such as an LLM, is fine-tuned for a specific task or domain. This allows the LLM to leverage knowledge already learned during its initial training and adapt it to a specific application. In the case of an LLM, the pre-training phase involves training the LLM on a large text corpus, typically consisting of billions of words. During this phase, the LLM learns relationships between words and phrases, which enables the LLM to generate consistent, human-like responses to text-based input. The output of this pre-training phase is an LLM with a sophisticated understanding of the patterns underlying natural language.

[0072] The fine-tuning phase adapts a pre-trained LLM to a specific task or domain by training the LLM on a smaller, task-specific dataset. For example, in some embodiments, an LLM may be trained to analyze a specific type or types of data source to improve its accuracy with respect to that content. Such information may be provided as part of the training data, and the LLM may focus on these areas and learn to more accurately identify data elements therein. Fine-tuning allows an LLM to learn the subtle nuances of a task or domain, such as the specific vocabulary and syntax used in that domain, without requiring as much data as would be necessary to train an LLM from scratch. By leveraging the knowledge learned in the pre-training phase, a fine-tuned LLM may achieve state-of-the-art performance on a specific task with a relatively small amount of training data. In some embodiments, an LLM or other generative AI model may be trained using a vector database.

[0073] FIG. 2 is an architecture diagram illustrating an RPA system 200 according to an embodiment of the present invention. In some embodiments, the RPA system 200 is part of the hyper-automation system 100 of FIG. 1. The RPA system 200 includes a designer 210 that enables developers to design and implement workflows. The designer 210 provides solutions for application integration and automates third-party applications, management information technology (IT) tasks, and business IT processes. The designer 210 can facilitate the development of automation projects, which are graphical representations of business processes. Simply put, the designer 210 facilitates the development and deployment of workflows and robots. In some embodiments, the designer 210 can be an application running on a user's desktop, an application running remotely in a VM, a web application, or the like.

[0074] An automation project enables rule-based process automation by giving developers control over the order of execution and relationships between a custom set of steps developed in a workflow, defined herein as "activities" as described above. One commercial example of an embodiment of the designer 210 is UiPath Studio™. Each activity may include an action such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, workflows may be nested or embedded.

[0075] Some types of workflows may include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences may be particularly well-suited for linear processes, allowing the flow of one activity from another without cluttering the workflow. Flowcharts may be particularly well-suited for more complex business logic, allowing for the integration of decisions and the connection of activities in more diverse ways through multiple branching logic operators. FSMs may be particularly well-suited for large workflows. FSMs may use a finite number of states during their execution that are triggered by conditions (i.e., transitions) or activities. Global exception handlers may be particularly well-suited for determining workflow behavior when an execution error is encountered or for debugging the process.

[0076] Once a workflow is developed in designer 210, the execution of the business process is orchestrated by conductor 220, which coordinates one or more robots 230 that execute the workflow developed in designer 210. One commercial example of an embodiment of conductor 220 is UiPath Orchestrator™. Conductor 220 facilitates the management of the creation, monitoring, and deployment of resources in an environment. Conductor 220 may act as an integration point with third-party solutions and applications. Accordingly, in some embodiments, conductor 220 may be part of the core hyper-automation system 120 of FIG. 1 .

[0077] The conductor 220 may manage all robots 230, connecting and executing them from a centralized point. Types of robots 230 that may be managed include, but are not limited to, attended robots 232, unattended robots 234, development robots (similar to unattended robots 234 but used for development and testing purposes), and non-production robots (similar to attended robots 232 but used for development and testing purposes). Attended robots 232 are triggered by user events and operate in parallel with humans on the same computing system. Attended robots 232 may be used with the conductor 220 for centralized process deployment and logging media. Attended robots 232 may assist human users in accomplishing various tasks and may be triggered by user events. In some embodiments, processes cannot be initiated from the conductor 220 on this type of robot, and / or they cannot be run under a locked screen. In certain embodiments, the attended robot 232 can only be launched from the robot tray or from a command prompt. The attended robot 232 preferably operates under human supervision in some embodiments.

[0078] Unattended robots 234 operate unattended in virtual environments and can automate many processes. Unattended robots 234 can be responsible for providing remote execution, monitoring, scheduling, and work queue support. Debugging for all robot types can be performed in designer 210 in some embodiments. Both attended and unattended robots can automate a variety of systems and applications, including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those produced by SAP®, Salesforce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0079] The conductor 220 may have various capabilities, including, but not limited to, provisioning, deployment, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between robots 230 and the conductor 220 (e.g., a web application). Deployment may include ensuring the correct delivery of package versions to robots 230 assigned to perform. Configuration may include maintaining and delivering robot environment and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot-specific data and maintaining user permissions. Logging may include storing and indexing logs in a database (e.g., a Structured Query Language (SQL) database or a “not only” SQL (NoSQL) database) and / or another storage mechanism (e.g., ElasticSearch®, which provides the ability to store large datasets and quickly execute queries). The conductor 220 may provide interconnectivity by operating as a centralized point of communication for third-party solutions and / or applications.

[0080] Robots 230 are execution agents that implement workflows built in designer 210. One commercial example of some embodiments of robot(s) 230 is UiPath Robots™. In some embodiments, robots 230 install the Microsoft Windows Service Control Manager (SCM) management service by default. As a result, such robots 230 can open interactive Windows sessions under the local system account and may have Windows service rights.

[0081] In some embodiments, a robot 230 can be installed in user mode, meaning that such a robot 230 has the same rights as the user to whom the given robot 230 is installed. This feature can also be available for high-density (HD) robots, ensuring maximum utilization of each machine. In some embodiments, either type of robot 230 can be configured in an HD environment.

[0082] In some embodiments, the robot 230 is divided into multiple components, each specialized for a specific automation task. In some embodiments, the robot components include, but are not limited to, an SCM-managed robot service, a user-mode robot service, an executor, an agent, and a command line. The SCM-managed robot service manages and monitors Windows sessions and acts as a proxy between the conductor 220 and the execution host (i.e., the computing system on which the robot 230 executes). These services are responsible for managing the credentials of the robot 230. A console application is launched by the SCM under Local System.

[0083] The user-mode robot service in some embodiments manages and monitors Windows sessions and acts as a proxy between the conductor 220 and the execution host. The user-mode robot service may be delegated and manage the credentials of the robot 230. If the SCM management robot service is not installed, a Windows application may be launched automatically.

[0084] An Executor may execute a given job under a Windows session (i.e., execute a workflow). An Executor may be aware of per-monitor dots-per-inch (DPI) settings. An Agent may be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. An Agent may be a client of a service. An Agent may request to start or stop a job or change settings. A Command Line is a client of a service. A Command Line is a console application that can request the start of a job and wait for its output.

[0085] As described above, the separation of the robot 230 components helps developers, support users, and computing systems more easily implement, identify, and track what each component is doing. In this way, special behaviors can be configured for each component, such as setting different firewall rules for executors and services. The executor may always be aware of per-monitor DPI settings in some embodiments. As a result, workflows may execute at any DPI regardless of the configuration of the computing system on which the workflow was created. Also, in some embodiments, projects from the designer 210 may be made independent of the browser zoom level. For applications that are not DPI-aware or are intentionally marked as not-aware, some embodiments may disable DPI.

[0086] The RPA system 200 in this embodiment is part of a hyperautomation system. Developers can use the designer 210 to build and test RPA robots that utilize AI / ML models deployed to the core hyperautomation system 240 (e.g., as part of its AI center). Such RPA robots can send inputs for execution of the AI / ML model(s) and receive outputs therefrom via the core hyperautomation system 240.

[0087] One or more robots 230 may be listeners, as described above. These listeners may provide information to the core hyperautomation system 240 about what users are doing when they use their computing systems. This information may then be used by the core hyperautomation system for process mining, task mining, task capture, etc.

[0088] An assistant / chatbot 250 may be provided on a user computing system to allow the user to launch an RPA local robot. The assistant may be located, for example, in the system tray. The chatbot may have a user interface so that the user can view the chatbot's text. Alternatively, the chatbot may have no user interface, run in the background, and listen to the user's speech using the computing system's microphone.

[0089] In some embodiments, data labeling may be performed by a user of the computing system on which the robot is running, or on another computing system to which the robot provides information. For example, if the robot invokes an AI / ML model to perform CV on an image for a VM user, but the AI / ML model does not correctly identify a button on the screen, the user may draw a rectangle around the misidentified or unidentified component and potentially provide text with the correct identification. This information may be provided to the core hyperautomation system 240 and then later used to train a new version of the AI / ML model.

[0090] 3 is an architecture diagram illustrating a deployed RPA system 300, according to an embodiment of the present invention. In some embodiments, the RPA system 300 may be part of the RPA system 200 of FIG. 2 and / or the hyper-automation system 100 of FIG. 1. The deployed RPA system 300 may be a cloud-based system, an on-premise system, a desktop-based system, or the like, providing enterprise-level, user-level, or device-level automation solutions for the automation of different computing processes.

[0091] It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robot application 310 includes an executor 312, an agent 314, and a designer 316. However, in some embodiments, the designer 316 may not be running on the same computing system as the executor 312 and the agent 314. The executor 312 executes processes. As shown in FIG. 3, multiple business projects may be running simultaneously. In this embodiment, the agent 314 (e.g., a Windows service) is the single connection point for all executors 312. All messages in this embodiment are logged into the conductor 340, which further processes them via the database server 350, the AI / ML server 360, the indexer server 370, or any combination thereof. As described above with respect to FIG. 2, the executor 312 may be a robotic component.

[0092] In some embodiments, a Robot represents an association between a machine name and a username. A Robot may manage multiple executors simultaneously. In computing systems that support multiple interactive sessions running simultaneously (such as Windows Server 2012), multiple Robots may run simultaneously, each running in a separate Windows session using a unique username. This is referred to as an HD Robot above.

[0093] Agent 314 is also responsible for transmitting the robot's status (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading required versions of packages to be fulfilled. Communication between agent 314 and conductor 340 is, in some embodiments, always initiated by agent 314. In notification scenarios, agent 314 may open a WebSocket channel that is later used by conductor 340 to send commands (e.g., start, stop, etc.) to the robot.

[0094] Listener 330 monitors and records data related to user interactions with the operation of the attended and / or unattended computing systems on which listener 330 resides. Listener 330 may be an RPA robot, part of an operating system, a downloadable application for the respective computing system, or any other software and / or hardware without departing from the scope of the present invention. Indeed, in some embodiments, the listener's logic is implemented partially or fully via physical hardware.

[0095] The server side includes a presentation layer (web application 342, open data protocol (oData) representative state transfer (REST) ​​application programming interface (API) endpoint 344, notification and monitoring 346), a service layer (API implementation / business logic 348), and a persistence layer (database server 350, AI / ML server 360, indexer server 370). Conductor 340 includes web application 342, oData REST API endpoint 344, notification and monitoring 346, and API implementation / business logic 348. In some embodiments, most actions a user performs in the conductor 340 interface (e.g., via browser 320) are performed by calling various APIs. Such actions may include, but are not limited to, launching jobs on robots, adding / removing data from a queue, scheduling jobs to run unattended, etc., without departing from the scope of the present invention. Web application 342 is the visual layer of the server platform. In this embodiment, web application 342 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the present invention. A user interacts with web pages from web application 342, in this embodiment via browser 320, to perform various operations to control conductor 340. For example, a user may create robot groups, assign packages to robots, analyze per-robot and / or per-process logs, start and stop robots, etc.

[0096] In addition to the web application 342, the conductor 340 also includes a services layer that exposes an oData REST API endpoint 344. However, other endpoints may be included without departing from the scope of the present invention. The REST API is consumed by both the web application 342 and an agent 314, which in this embodiment is a supervisor of one or more robots on a client computer.

[0097] The REST API of this embodiment covers configuration, logging, monitoring, and queuing functionality. The configuration endpoint, in some embodiments, may be used to define and configure users, permissions, robots, assets, releases, and environments for an application. The logging REST endpoint may be used to log various information, such as errors, explicit messages sent by robots, and other environment-specific information. The deployment REST endpoint may be used by robots to query the version of a package that should be executed when a start job command is used in conductor 340. The queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to a queue, retrieving transactions from a queue, and setting the status of transactions.

[0098] Monitoring REST endpoints may monitor web applications 342 and agents 314. Notification and monitoring API 346 may be a REST endpoint used to register agents 314, deliver configuration settings to agents 314, and send and receive notifications from the server and agents 314. Notification and monitoring API 346 may use WebSocket communications in some embodiments.

[0099] The service layer APIs, in some embodiments, may be accessed through configuration of appropriate API access paths, for example, based on whether the conductor 340 and the overall hyperautomation system have an on-premise or cloud-based deployment type. The API for the conductor 340 may provide custom methods for querying statistics about various entities registered with the conductor 340. Each logical resource may, in some embodiments, be an oData entity. In such entities, components such as robots, processes, queues, etc. may have properties, relationships, and behaviours. The conductor 340 APIs, in some embodiments, may be consumed by web applications 342 and / or agents 314 in two ways: by obtaining API access information from the conductor 340 or by registering an external application to use the oAuth flow.

[0100] In this embodiment, the persistence layer includes three servers: a database server 350 (e.g., an SQL server), an AI / ML server 360 (e.g., a server that provides AI / ML model provisioning services such as an AI center function), and an indexer server 370. The database server 350 in this embodiment stores configurations of robots, robot groups, associated processes, users, roles, schedules, etc. This information is managed via a web application 342 in some embodiments. The database server 350 may also manage queues and queue items. In some embodiments, the database server 350 may store messages logged by robots (in addition to or instead of the indexer server 370). The database server 350 may also store process mining, task mining, and / or task capture related data received, for example, from a listener 330 installed on the client side. While no arrow is shown between the listener 330 and the database 350, it should be understood that in some embodiments, the listener 330 can communicate with the database 350, and vice versa. This data may be stored in the form of a PDF, image, XAML file, etc. Listener 330 may be configured to intercept user actions, processes, tasks, and performance metrics on each computing system on which listener 330 resides. For example, listener 330 may record user actions (e.g., clicks, typed characters, location, application, active element, time, etc.) on its respective computing system and then convert these into a format suitable for being provided to and stored in database server 350.

[0101] AI / ML server 360 facilitates the incorporation of AI / ML models into automation. Pre-built AI / ML models, model templates, and various deployment options may make such capabilities accessible to non-data scientists. Deployed automation (e.g., RPA robots) may invoke AI / ML models from AI / ML server 360. The performance of AI / ML models may be trained and improved using monitored and human-validated data. AI / ML server 360 may schedule and execute training jobs to train new versions of AI / ML models.

[0102] AI / ML server 360 may store data related to AI / ML models and ML packages for configuring various ML skills for users during development. As used herein, an ML skill is a pre-built and trained ML model for a process that can be used, for example, by automation. AI / ML server 360 may also store data related to document understanding techniques and frameworks, algorithms, and software packages for various AI / ML capabilities, including, but not limited to, intent analysis, NLP, speech analysis, different types of AI / ML models, etc.

[0103] Optionally in some embodiments, indexer server 370 stores and indexes information logged by the robots. In particular embodiments, indexer server 370 may be disabled via a configuration setting. In some embodiments, indexer server 370 uses ElasticSearch®, a full-text search engine from an open source project. Messages logged by robots (e.g., using activities such as log messages or line writes) may be sent via logging REST endpoint(s) to indexer server 370, where they are indexed for future use.

[0104] FIG. 4 is an architecture diagram illustrating relationships 400 between a designer 410, activities 420, 430, 440, 450, a driver 460, an API 470, and an AI / ML model 480, according to an embodiment of the present invention. As described above, a developer uses the designer 410 to develop a workflow to be performed by a robot. Various types of activities may be displayed to the developer in some embodiments. The designer 410 may be local to the user's computing system or remote thereto (e.g., accessed via a VM or a local web browser interacting with a remote web server). The workflow may include user-defined activities 420, API-driven activities 430, AI / ML activities 440, and / or UI automation activities 450. The user-defined activities 420 and API-driven activities 440 interact with the application via their APIs. The user-defined activities 420 and / or the AI / ML activities 440 may, in some embodiments, invoke one or more AI / ML models 480, which may be located locally and / or remotely relative to the computing system on which the robot is operating.

[0105] In some embodiments, non-text visual components in an image can be identified, referred to herein as CV. However, it should be noted that in some embodiments, CV incorporates OCR. CV may be performed at least in part by AI / ML model(s) 480. Some CV activities associated with such components may include, but are not limited to, extracting text from segmented label data using OCR, fuzzy text matching, cropping segmented label data using ML, comparing extracted text in label data to ground truth data, etc. In some embodiments, the number of activities that may be implemented in user-defined activities 420 may be in the hundreds or thousands. However, any number and / or type of activities may be used without departing from the scope of the present invention.

[0106] UI automation activities 450 are a subset of specialized low-level activities written in low-level code that facilitate interactions with the screen. UI automation activities 450 facilitate these interactions through drivers 460 that enable the robot to interact with desired software. For example, drivers 460 may include operating system (OS) drivers 462, browser drivers 464, VM ​​drivers 466, enterprise application drivers 468, etc. In some embodiments, one or more AI / ML models 480 may be used by UI automation activities 450 to perform interactions with the computing system. In particular embodiments, AI / ML models 480 may augment or completely replace drivers 460. Indeed, in certain embodiments, drivers 460 are not included.

[0107] Driver 460 may interact with the OS at a low level, such as by looking for hooks or monitoring keys, via OS driver 462. Driver 460 may also facilitate integration with Chrome®, IE®, Citrix®, SAP®, etc. For example, a "click" activity plays the same role in these different applications via driver 460.

[0108] FIG. 5 is an architecture diagram illustrating a computing system 500 configured to perform RPA action decisions and recommendations using a semantic action graph, according to embodiments of the present invention. In some embodiments, computing system 500 may be one or more computing systems depicted and / or described herein. In particular embodiments, computing system 500 may be part of a hyper-automation system such as those shown in FIGS. 1 and 2. Computing system 500 includes a bus 505 or other communication mechanism for communicating information and processor(s) 510 coupled to bus 505 for processing information. Processor(s) 510 may be any type of general or application-specific processor, including a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. Processor(s) 510 may also have multiple processing cores, at least some of which may be configured to perform specific functions. In some embodiments, multiple parallel processing may be used. In particular embodiments, at least one processor(s) 510 may be a neuromorphic circuit that includes processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit may not require the typical components of a von Neumann computing architecture.

[0109] The computing system 500 further includes a memory 515 for storing information and instructions executed by the processor(s) 510. The memory 515 may be comprised of random access memory (RAM), read-only memory (ROM), flash memory, cache, static storage such as a magnetic or optical disk, or other types of non-transitory computer-readable media, or any combination thereof. The non-transitory computer-readable media may be any available media accessible by the processor(s) 510 and may include volatile media, non-volatile media, or both. The media may also be removable, non-removable, or both. The computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communication network via wireless and / or wired connections. In some embodiments, the communication device 520 may include one or more antennas that are a single antenna, an array of antennas, a phased antenna, a switched antenna, a beam-forming antenna, a beam-steering antenna, a combination thereof, and / or any other antenna configuration without departing from the scope of the present invention.

[0110] The processor(s) 510 are further coupled to a display 525 via a bus 505. Any suitable display device and tactile I / O may be used without departing from the scope of the present invention. A keyboard 530 and cursor control device 535, such as a computer mouse, touchpad, or the like, are further coupled to the bus 505 to allow a user to interface with the computing system 500. However, in certain embodiments, a physical keyboard and mouse may not be present, and the user may interact with the device solely through the display 525 and / or touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. In certain embodiments, no physical input devices and / or displays are present. For example, a user may interact with the computing system 500 remotely via another computing system in communication with it, or the computing system 500 may operate autonomously.

[0111] Memory 515 stores software modules that provide functionality when executed by processor(s) 510. The modules include an operating system 540 for computing system 500. The modules further include a semantic action graph and embedding module 545 configured to perform all or a portion of the AI / ML processes described herein or derivatives thereof. Computing system 500 may include one or more additional functional modules 550 that include additional functionality.

[0112] Those skilled in the art will appreciate that a "system" may be embodied as a server, embedded computing system, personal computer, console, personal digital assistant (PDA), mobile phone, tablet computing device, smartwatch, quantum computing system, or any other suitable computing device or combination of devices without departing from the scope of the present invention. Presenting the above-described functions as being performed by a "system" is not intended to limit the scope of the present invention in any way, but rather to provide an example of many embodiments of the present invention. Indeed, the methods, systems, and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing techniques, including cloud computing systems. The computing system may be part of or otherwise accessible by a LAN, a mobile communications network, a satellite communications network, the Internet, a public or private cloud, a hybrid cloud, a server farm, any combination thereof, or the like. Any local or distributed architecture may be used without departing from the scope of the present invention.

[0113] It should be noted that some of the system features described herein are presented as modules to further emphasize implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integrated (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, etc.

[0114] Modules may also be implemented at least partially in software for execution by various types of processors. For example, an identified unit of executable code may include one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, a procedure, or a function. Nevertheless, executable identified modules need not be physically located together; they may include separate instructions stored in different locations that, when logically combined, comprise a module to achieve the purpose stated for the module. Furthermore, modules may be stored on non-transitory computer-readable media, such as, for example, a hard disk drive, a flash device, RAM, tape, and / or any other non-transitory computer-readable medium used to store data without departing from the scope of the present invention.

[0115] Indeed, a module of executable code may be a single instruction, many instructions, or even distributed across several different code segments, different programs, and multiple memory devices. Similarly, operational data may be identified and depicted herein within a module and may be embodied in any suitable form and organized within any suitable type of data structure. Operational data may be collected as a single data set or may be distributed in different locations across different storage devices, or may exist, at least in part, simply as electronic signals on a system or network.

[0116] Various types of AI / ML models may be trained and deployed without departing from the scope of the present invention. For example, FIG. 6A illustrates an example of a neural network 600 trained to supplement RPA action decisions and recommendations using a semantic action graph, according to an embodiment of the present invention. The neural network 600 includes multiple hidden layers. While both DLNNs and shallow learning neural networks (SLNNs) typically have multiple layers, SLNNs may sometimes have only one or two layers, typically fewer than DLNNs. Typically, the architecture of a neural network includes an input layer, multiple hidden layers, and an output layer, as in the case of the neural network 600.

[0117] DLNNs often have many layers (10, 50, 200, etc.), with subsequent layers typically reusing features from previous layers to compute more complex and general functions. SLNNs, on the other hand, tend to train relatively quickly because they have only a few layers and expert features are pre-created from raw data samples, although feature extraction is tedious. DLNNs, on the other hand, typically do not require expert features, but they take longer to train and tend to have more layers.

[0118] In both approaches, layers are trained simultaneously on a training set and are typically checked for overfitting on a separate cross-validation set. Both techniques produce excellent results, and there is considerable enthusiasm for both approaches. The optimal size, shape, and number of individual layers depend on the problem being addressed by each neural network.

[0119] Returning to FIG. 6A , user and computing system data (e.g., user and computing system actions and state information from task mining), CV and OCR data, process start and end data for ontology determination, business process data (e.g., business process flow), etc. are provided as input layers and fed as inputs to J neurons in hidden layer 1. A variety of other inputs are possible, including, but not limited to, computing system state information, published automations, business rules, information about what RPA workflows and / or tasks are related to, initial automation definitions, process automation documents, etc. In this example, all of these inputs are fed to each neuron, however a variety of architectures are possible that may be used individually or in combination, including but not limited to feedforward networks, radial basis networks, deep feedforward networks, deep convolutional inverse graphics networks, convolutional neural networks, recurrent neural networks, artificial neural networks, long / short-term memory networks, gated recurrent unit networks, generative adversarial networks, liquid state machines, autoencoders, variational autoencoders, denoising autoencoders, sparse autoencoders, extreme learning machines, echo state networks, Markov chains, Hopfield networks, Boltzmann machines, restricted Boltzmann machines, deep residual networks, Kohonen networks, deep belief networks, deep convolutional networks, support vector machines, neural Turing machines, or any other suitable type or combination of neural networks without departing from the scope of the invention.

[0120] Hidden layer 2 receives input from hidden layer 1, hidden layer 3 receives input from hidden layer 2, and so on for all hidden layers until the final hidden layer provides its output as input for the output layer. For example, neural network 600 may provide next action suggestions, next sequence of actions suggestions, existing process suggestions (e.g., recommended changes to a business process based on other related business processes in the training data), related business process suggestions, etc. Although multiple suggestions are shown as output herein, in some embodiments, only one output suggestion is provided. In particular embodiments, the suggestions are ranked based on a confidence score.

[0121] Note that the numbers of neurons I, J, K, and L are not necessarily equal. Thus, any desired number of layers may be used in a given layer of neural network 600 without departing from the scope of the present invention. In fact, in certain embodiments, the types of neurons in a given layer may not all be the same.

[0122] Neural network 600 is trained to assign confidence score(s) to appropriate outputs. To reduce inaccurate predictions, in some embodiments, only those results with confidence scores equal to or greater than a confidence threshold may be provided. For example, if the confidence threshold is 80%, outputs with confidence scores above this amount may be used, and the rest may be ignored.

[0123] Neural networks are probabilistic constructs that typically have confidence score(s). This can be a score that the AI / ML model learned based on how often similar inputs were correctly identified during training. Some common types of confidence scores include a decimal number between 0 and 1 (which can also be interpreted as a confidence percentage), a numeric value between negative infinity and positive infinity, a set of formulas (e.g., "low," "medium," and "high"), etc. To obtain more accurate confidence scores, various post-processing calibration techniques such as temperature scaling, batch normalization, weight decay, and negative log-likelihood (NLL) can also be used.

[0124] "Neurons" in neural networks are algorithmically implemented as mathematical functions, typically based on the functionality of biological neurons. Neurons receive weighted inputs and have sum and activation functions that govern whether they pass an output to the next layer. This activation function may be a nonlinear, thresholded activity function that does nothing if the value is below a threshold, and responds linearly when the function exceeds the threshold (i.e., rectified linear unit (ReLU) nonlinearity). Sum and ReLU functions are used in deep learning because real neurons may have roughly similar activity functions. Information may be subtracted, added, etc. via linear transformations. Essentially, neurons act as gating functions that pass their output to the next layer, governed by their underlying mathematical function. In some embodiments, different functions may be used for at least some neurons.

[0125] JPEG2025169870000001.jpg79136

[0126] JPEG2025169870000002.jpg44136

[0127] JPEG2025169870000003.jpg27122

[0128] In this case, neuron 610 is a single-layer perceptron. However, any suitable neuron type or combination of neuron types may be used without departing from the scope of the present invention. It should also be noted that the range of values ​​of the activation function weights and / or output value(s) may vary in some embodiments without departing from the scope of the present invention.

[0129] A goal, or "reward function," is often employed. The reward function guides the search of the state space, exploring intermediate transitions and steps with both short-term and long-term rewards in an attempt to achieve the goal (e.g., finding the most accurate answer to a user query based on relevant metrics). During training, various labeled data are fed through the neural network 600. Successful identification strengthens the input weights to neurons, while unsuccessful identification weakens those weights. Cost functions such as mean squared error (MSE) or gradient descent may be used to ensure that slightly incorrect predictions are punished much less than significantly incorrect predictions. If the performance of an AI / ML model does not improve after a certain number of training iterations, data scientists may change the reward function, correct incorrect predictions, etc.

[0130] Backpropagation is a technique for optimizing synaptic weights in feedforward neural networks. Backpropagation can be used to "pop the hood" of a neural network's hidden layers to see how much loss each node is carrying, and then update the weights to minimize loss, giving lower weights to nodes with higher error rates and vice versa. In other words, backpropagation allows data scientists to iteratively adjust the weights to minimize the difference between the actual output and the desired output.

[0131] The backpropagation algorithm is mathematically based on optimization theory. In supervised learning, training data with known outputs is passed through a neural network, and the error is calculated from the known target output using a cost function, which gives the backpropagation error. The error is calculated at the output, and this error is converted into a modification of the network weights that minimizes the error.

[0132] JPEG2025169870000004.jpg66136

[0133] JPEG2025169870000005.jpg20136

[0134] JPEG2025169870000006.jpg117136

[0135] JPEG2025169870000007.jpg111122

[0136] JPEG2025169870000008.jpg91136

[0137] An AI / ML model may be trained over multiple epochs until it reaches a good level of accuracy (e.g., 97% or higher using an F2 or F4 threshold for detection, approximately 2,000 epochs). This accuracy level may, in some embodiments, be determined using an F1 score, an F2 score, an F4 score, or any other suitable technique without departing from the scope of the present invention. Once trained on training data, the AI / ML model may be tested on a set of evaluation data that the AI / ML model has not previously encountered. This helps to ensure that the AI / ML model does not "overfit," performing well on the training data but poorly on other data.

[0138] In some embodiments, it may not be known what accuracy level an AI / ML model is capable of achieving. Thus, if the accuracy of an AI / ML model begins to degrade when analyzing evaluation data (i.e., the model performs well on the training data but begins to degrade on the evaluation data), the AI / ML model can undergo further epochs of training on the training data (and / or new training data). In some embodiments, an AI / ML model is deployed only when accuracy reaches a certain level or when the accuracy of the trained AI / ML model is superior to existing deployed AI / ML models. In certain embodiments, a collection of trained AI / ML models can be used to accomplish a task. For example, one model can be trained to recognize images, another model can recognize text, yet another model can recognize semantic and / or ontological associations, etc.

[0139] In some embodiments, a Transformer network such as SentenceTransformers™, a state-of-the-art Python™ framework for sentence, text, and image embedding, can be used. Such a Transformer network learns associations between words and phrases with both high and low scores. This trains an AI / ML model to determine what is close to the input and what is not, respectively. Rather than using only word / phrase pairs, the Transformer network may also use field lengths and field types.

[0140] In some embodiments, NLP techniques such as word2vec, BERT, GPT-3, ChatGPT, and other LLMs can be used to facilitate semantic understanding and provide more accurate and human-like answers, as described above. Other techniques, such as clustering algorithms, can be used to find similarities between groups of elements. Clustering algorithms can include, but are not limited to, density-based algorithms, distribution-based algorithms, centroid-based algorithms, and hierarchical-based algorithms, such as the K-means clustering algorithm, the DBSCAN clustering algorithm, the Gaussian Mixture Model (GMM) algorithm, and the Balanced Iterative Reduction and Clustering with Hierarchy (BIRCH) algorithm. Such techniques can also be useful for classification.

[0141] A semantic action graph may contain many actions for many different applications, with various paths representing tasks in it. An example of such a path / task is shown in semantic action graph portion 700 of FIG. 7. In this example, a web form is completed and an email confirming the completion of the web form is created and sent. The RPA robot executes field entry activities 702, 704, 706, 708, 710, 712, and 714 to enter information into seven fields in the browser. The robot clicks a submit button to submit form 716, causing the operating system to perform screen capture 718. The robot then switches the window to Outlook® 720 and enters the recipients 722, subject 724, and body 726 associated with the submitted form. The robot pastes the captured screen image 728 and then sends email 730.

[0142] FIG. 8 is a flowchart illustrating a process 800 for periodically processing task mining data and generating a semantic action graph according to an embodiment of the present invention. The process begins at 810 by periodically obtaining task mining data from a user computing system. Next, a normalization technique is applied to the task mining data to achieve normalization and reduce the task mining information to a specific range within the normalization curve(s) at 820. To achieve normalization, calculation of business metrics, data processing, conversion of screen data, metadata, user-controlled data, etc. are used. Data normalization means that data is converted to the same format and similar scale within an acceptable range in order to optimize the process of performing data processing.

[0143] The normalized data is sorted into action group data, which correlates with the event metric data, at 830, using classification and / or clustering algorithms. The action group data is then indexed at 840 to search, connect, and correlate the data. A reinforcement learning pattern matching algorithm is then applied to a supervised learning process at 850 based on the action list recorded as feedback / input to the system. The system uses the user's input to match actions with the indexed data. A semantic action graph is then initially constructed (or extended, if already present) by the business intelligence and AI / ML model(s) using the indexed information at 860. The process is repeated periodically. The semantic action graph can be accessed remotely by the RPA robot from the business intelligence system or deployed to the user computing system for local use. In the latter case, new versions of the semantic action graph can be periodically deployed to the user computing system after their creation.

[0144] FIG. 9 is a flowchart illustrating a process 900 for monitoring a user of a computing system using a semantic action graph and providing next action and sequence of action recommendations, according to an embodiment of the present invention. The process begins with an RPA robot monitoring a user of the computing system at 910. The RPA robot compares a user interaction or sequence of user interactions with the semantic action graph at 920. The RPA robot then finds one or more tasks that may match within an acceptable range (e.g., at least a certain confidence score) for the next step the user is expected to perform at 930. The RPA robot then proposes a task(s) to the user at 940. If the user accepts the task at 950, the RPA robot uses the actions and connections between them in the semantic action graph to execute and complete the task at 960. The process then returns to step 910.

[0145] However, if the user does not accept the proposed task at 950, the RPA robot performs task mining, monitoring subsequent user interactions with the computing system at 970 (e.g., key presses, mouse clicks, locations, interacted graphical elements, windows, applications in those windows, timestamps for each interaction, etc.), and sends the unaccepted task(s) and actions to the business and AI systems at 980. For example, the RPA robot may provide a unique identifier for each unaccepted task in the semantic action graph. This information may then be used for reinforcement learning, as in process 800 of FIG. 8.

[0146] The process steps performed in Figures 8 and 9 may be performed by a computer program encoding instructions to a processor(s) to perform at least a portion of the process(es) described in Figures 8 and 9 according to embodiments of the present invention. The computer program may be stored on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, RAM, tape, and / or any other such medium or combination of media used to store data. The computer program may include coded instructions for controlling a processor(s) of a computing system (e.g., processor(s) 510 of computing system 500 of Figure 5) to implement all or a portion of the process steps described in Figures 8 and 9, which may also be stored on a computer-readable medium.

[0147] The computer program may be implemented in hardware, software, or a hybrid implementation. The computer program may be composed of modules in operable communication with each other and designed to send information or instructions to a display. The computer program may be configured to run on a general-purpose computer, an ASIC, or any other suitable device.

[0148] It will be readily understood that the components of the various embodiments of the present invention, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention, as represented in the accompanying figures, is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention.

[0149] The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, references throughout this specification to "certain embodiments," "some embodiments," or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Thus, the appearances of "certain embodiments," "some embodiments," "other embodiments," or similar language throughout this specification do not necessarily refer to the same group of all embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0150] It should be noted that references to features, advantages, or similar language throughout this specification do not imply that all of the features and advantages that may be realized in the present invention are to be found in any single embodiment of the present invention, or in any embodiment of the present invention. Rather, language referring to features and advantages is understood to mean that the particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussions of features and advantages throughout this specification, and similar language, may, but do not necessarily, refer to the same embodiment.

[0151] Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the invention may be practiced without a particular feature or advantage of one or more particular embodiments. In other instances, additional features and advantages may be recognized in particular embodiments, although they may not be present in all embodiments of the invention.

[0152] Those of ordinary skill in the art will readily appreciate that the invention as described above can be implemented using steps in a different order and / or with hardware elements in different configurations than those disclosed. Thus, while the invention has been described in terms of these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will become apparent while remaining within the spirit and scope of the invention. Accordingly, reference should be made to the appended claims to determine the scope of the invention.

Claims

1. One or more non-transitory computer-readable media storing one or more computer programs, the one or more computer programs causing at least one processor to: Obtaining task mining data from multiple user computing systems; applying a normalization technique to the task mining data to achieve normalization and reduce the task mining information to a range within one or more normalization curves; classifying the normalized data into action groups that correlate with the event metric data using one or more classification algorithms, one or more clustering algorithms, or both; indexing said action groups to search, connect and correlate said action groups; Applying reinforcement learning to a supervised learning process based on a recorded action list, One or more non-transitory computer-readable media configured to generate a semantic action graph using the indexed information.

2. The one or more computer programs further include causing the at least one processor to: periodically repeating the steps of claim 1 using newly acquired task mining data; The one or more non-transitory computer-readable media of claim 1 configured to use the newly acquired task mining data to expand the semantic action graph.

3. 3. The one or more non-transitory computer-readable media of claim 2, wherein the one or more computer programs perform the steps of claim 1 in a live environment.

4. 10. The one or more non-transitory computer-readable media of claim 1, wherein applying the normalization techniques includes performing business metrics calculations, data processing, and transformations of screen data, metadata, and user control data to achieve normalization.

5. 10. The one or more non-transitory computer-readable media of claim 1, wherein data normalization comprises converting the task mining data to the same format and similar scale within a tolerance to optimize data processing.

6. 10. The one or more non-transitory computer-readable media of claim 1, wherein classifying the normalized data comprises applying a decision tree, an ensemble tree, a generalized additive model (GAM), a Naive Bayes algorithm, a k-nearest neighbor (kNN) algorithm, performing a discriminant analysis, or any combination thereof.

7. 10. The one or more non-transitory computer-readable media of claim 1, wherein indexing the categorized data is performed using a B-tree index, a hash map, or both.

8. 10. The one or more non-transitory computer-readable media of claim 1, wherein the reinforcement learning comprises applying a reinforcement learning pattern matching algorithm to match actions from the action list to the indexed data using user input.

9. The one or more non-transitory computer-readable media of claim 1 , wherein the semantic action graph includes nodes representing action groups and edges comprising relationships and ordering between the nodes.

10. The one or more non-transitory computer-readable media of claim 9 , wherein the semantic action graph is a directed acyclic graph.

11. 10. The one or more non-transitory computer-readable media of claim 9, wherein the semantic action graph allows for some variation of tasks within a tolerance range to distinguish between critical paths and trimmable minor branches.

12. The one or more computer programs further include causing the at least one processor to:

10. The one or more non-transitory computer-readable media of claim 1, configured to make the semantic action graph available to one or more RPA robots for use in monitoring one or more respective users on respective user computing systems.

13. The one or more computer programs further include causing the at least one processor to:

13. The one or more non-transitory computer-readable media of claim 12, configured to periodically deploy a new version of the semantic action graph to the user computing system after the new version is created.

14. applying, by one or more computing systems, a normalization technique to the task mining data to achieve normalization and reduce the task mining information to a range within one or more normalization curves; classifying, by the one or more computing systems, the normalized data into action groups that correlate with event metric data using one or more classification algorithms, one or more clustering algorithms, or both; indexing, searching, connecting, and correlating the action groups by the one or more computing systems; applying, by the one or more computing systems, reinforcement learning to a supervised learning process based on the recorded action list; A computer-implemented method comprising generating, by the one or more computing systems, a semantic action graph using the indexed information.

15. moreover, periodically repeating the steps of claim 14 using newly acquired task mining data by the one or more computing systems; The computer-implemented method of claim 14 , further comprising augmenting, by the one or more computing systems, the semantic action graph using the newly acquired task mining data.

16. The computer-implemented method of claim 14 , wherein data normalization comprises converting the task mining data to the same format and similar scale within a tolerance to optimize data processing.

17. The computer-implemented method of claim 14 , wherein the semantic action graph includes nodes representing action groups and edges containing relationships and ordering between the nodes.

18. 20. The computer-implemented method of claim 17, wherein the semantic action graph allows for some variation of tasks within a tolerance range to distinguish between critical paths and trimmable minor branches.

19. a memory for storing computer program instructions; and at least one processor configured to execute the computer program instructions, wherein the computer program instructions cause the at least one processor to: periodically obtain task mining data from multiple user computing systems; applying a normalization technique to the task mining data to achieve normalization and reduce the task mining information to a range within one or more normalization curves; classifying the normalized data into action groups that correlate with the event metric data using one or more classification algorithms, one or more clustering algorithms, or both; indexing said action groups to search, connect and correlate said action groups; Applying reinforcement learning to a supervised learning process based on a recorded action list, One or more computing systems using the indexed information to generate a new semantic action graph or extend an existing semantic action graph.

20. 20. The one or more computing systems of claim 19, wherein the semantic action graph is a directed acyclic graph including nodes representing action groups and edges containing relationships and ordering between the nodes.

21. 21. The one or more computing systems of claim 20, wherein the semantic action graph allows for some variation of tasks within a tolerance range to distinguish between critical paths and trimmable minor branches.

Images