Use artificial intelligence / machine learning models to determine interaction sequences, extract processes, and generate robots

Abstract

The use of artificial intelligence (AI) / machine learning (ML) models is disclosed to determine sequences of user interactions with a computing system, extract common processes, and generate robotic process automation (RPA) robots. The AI / ML models can be trained to recognize matching n-grams and / or useful end states of user interactions. Recorded actual user interactions can be analyzed, and matching sequences can be implemented as corresponding activities in an RPA workflow.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This is an international application claiming the benefit of and priority to U.S. Patent Application No. 17 / 070,168, filed October 14, 2020. The subject matter of this previously filed application is incorporated herein by reference in its entirety.

[0002] The present invention relates generally to user interface (UI) automation, and more specifically to using artificial intelligence (AI) / machine learning (ML) models to determine sequences of interactions between computing systems and users, extract common processes, and generate robotic process automation (RPA) robots. [Background technology]

[0003] To perform UI automation, RPA techniques may utilize driver- and / or application-level interactions to click buttons, enter text, and perform other interactions with the UI. However, keypress, mouse click, and other kernel hook information may not be available at the system level in some embodiments, or when building a new UI automation platform. Implementing such a UI automation platform typically requires extensive driver- and application-level functionality. Furthermore, it may be unclear how users interact with their computing systems. Therefore, alternative techniques for providing UI automation may be beneficial. Summary of the Invention

[0004] Certain embodiments of the present invention may provide solutions to problems and needs in the field that have not yet been fully identified, appreciated, or solved by current UI automation technologies. For example, some embodiments of the present invention relate to using AI / ML models to determine interaction sequences between computing systems and users, extract common processes, and generate RPA robots.

[0005] In an embodiment, a computer-implemented method includes providing, by a computing system, data including time-ordered interactions of a plurality of users with each of the user computing systems to an AI / ML model. The computer-implemented method also includes training, by the computing system, the AI / ML model to recognize relevant sequences of task-related user interactions in the time-ordered sequence of user interactions by comparing n-grams of the sequences of user interactions in the recorded data from the computing system with a sliding window to find relevant sequences. The n-grams include two or more values ​​of n. The computer-implemented method further includes deploying the trained AI / ML model.

[0006] In another embodiment, a non-transitory computer-readable medium stores a computer program configured to cause at least one processor to analyze recorded actual user interactions of a plurality of users with respective computing systems using an AI / ML model to determine a set of matching sequences in the recorded actual user interactions. The computer program is also configured to cause the at least one processor to generate one or more respective RPA workflows that include activities that implement the user interactions of the determined set of matching sequences.

[0007] In yet another embodiment, a system includes a memory storing computer program instructions and at least one processor configured to execute the computer program instructions. The computer program instructions are configured to cause the at least one processor to analyze recorded actual user interactions of a plurality of users with respective computing systems using an AI / ML model and determine matching sequences in the recorded actual user interactions. The computer program instructions are also configured to cause the at least one processor to determine a most efficient sequence of the matching sequences. The computer program instructions are further configured to cause the at least one processor to generate one or more respective RPA workflows that include activities implementing the user interactions of the determined most efficient matching sequence. [Brief explanation of the drawings]

[0008] So that the advantages of particular embodiments of this invention may be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments which are illustrated in the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope, but the invention will be described and explained with additional specificity and detail through the use of the following accompanying drawings, in which:

[0009] [Figure 1] FIG. 1 is an architectural diagram illustrating a robotic process automation (RPA) system, according to an embodiment of the present invention.

[0010] [Figure 2] FIG. 1 is an architectural diagram illustrating a deployed RPA system according to an embodiment of the present invention.

[0011] [Figure 3]FIG. 2 is an architecture diagram illustrating the relationships between designers, activities, and drivers according to an embodiment of the present invention.

[0012] [Figure 4] FIG. 1 is an architectural diagram illustrating an RPA system according to an embodiment of the present invention.

[0013] [Figure 5] FIG. 1 is an architecture diagram illustrating a computing system configured to use AI / ML models to determine sequences of user interactions with the computing system, extract common processes, and generate RPA robots, according to an embodiment of the invention.

[0014] [Figure 6] FIG. 1 is an architecture diagram illustrating a system configured to use AI / ML models to determine sequences of user interactions with a computing system, extract common processes, and generate RPA robots, according to an embodiment of the invention.

[0015] [Figure 7] 1 is a flowchart illustrating a process of using an AI / ML model to determine a sequence of user interactions with a computing system, extract common processes, and generate an RPA robot, according to an embodiment of the invention.

[0016] [Figure 8] 1 is a flowchart illustrating a process for performing process abstraction based on a desired end state using an AI / ML model and generating an RPA robot, according to an embodiment of the invention.

[0017] [Figure 9] FIG. 2 illustrates a comparison of n-grams representing sequences of user interactions according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] Unless otherwise noted, like reference characters denote corresponding features consistently throughout the accompanying drawings.

[0019] Detailed Description of the Embodiments Some embodiments relate to using AI / ML models to determine interaction sequences between a computing system and a user, extract common processes, and generate an RPA robot. User interactions can be determined by an AI / ML model trained to recognize applications, screens, and UI elements using computer vision (CV) and to recognize user interactions with the applications, screens, and UI elements. This AI / ML model can be the same AI / ML model that subsequently performs the sequence search, or a different AI / ML model. Individual user interactions include, but are not limited to, a user clicking a button, entering text in a field, interacting with a menu, closing a window, navigating to a different screen in an application, etc. An example of a user interaction sequence can include, for example, opening an application, entering information into multiple text fields, and clicking a submit button.

[0020] User interactions may be extracted from data collected from multiple computing systems. For example, the collected data may include video alone and / or other data collected from the computing systems, such as browser history, heat maps, file information, currently running applications and locations, system-level and / or application-level information, etc. The extracted user interactions may then be combined into a sequence of interactions related to a larger activity or task, such as writing and sending an email, entering information into a form and submitting it, or modifying a spreadsheet based on information obtained from a website.

[0021] In some embodiments, n-grams of multiple sizes may be used to search for matching sequences in the data. If the minimum value of n is too small, there may be too many matching sequences to identify meaningful sequences. Conversely, if the minimum value of n is too large, meaningful matching sequences smaller than n may be missed. During training, human reviewers may review identified sequences of some starting size n (e.g., 2, 3, 7, etc.). If this size produces too many useless sequences, the minimum sequence size may be incremented to n+1, then n+2, then n+3, etc., until matching sequences tend to be useful. After the minimum useful n is determined (e.g., the smallest n that produces at least 10% useful sequences, at least 30%, at least 50%, at least 90%, etc.), sequences below that n-gram size may be ignored or not determined / considered. In some embodiments, the AI / ML model may automatically select the minimum value of n based on the maximum number of matching sequences. In certain embodiments, a certain number of matches may be required for sequences of a certain n-gram size to be considered. For example, large organizations tend to have multiple or many users performing this process, so tens, hundreds, or even thousands of matches may need to exist in order for the sequence to be automated.

[0022] Recurrent neural networks (RNNs) may be particularly adept at determining useful ranges of n values. In some embodiments, RNNs can sweep n-grams of various sizes for useful sequences and determine the optimal windowing threshold (i.e., the useful range of n values) through a trial-and-error process that potentially includes sweeping all sequence sizes. The RNN's determination of the most optimal range may then eliminate the need for human input. Such systems can be used, for example, to automatically determine whether a user is likely to abandon a cloud computing service by looking for time-windowed sequences of user interactions that tend to indicate a high likelihood of the user abandoning the cloud computing service. Steps can then be taken to retain the customer (e.g., having a human representative contact the customer, offering a lower service price, etc.).

[0023] In some embodiments, an upper limit for n may also be determined. For example, if fewer than a certain number of sequences or no sequences are found above a certain value of n, the size of n may be limited to less than that value. Also, if n is too large, sequences may correspond to more than the given task and may contain other actions unrelated to those that multiple users happened to perform. This determination may be made by a human reviewer or automatically.

[0024] In certain embodiments, importance metric(s) may be used in addition to or instead of some or all of the sequence restriction techniques described above. For example, consideration may be given to whether a sequence of actions generates at least a certain amount of revenue, saves at least a certain amount of revenue, increases the efficiency / speed of a task by at least a certain amount, speeds up the customer acquisition process, reduces the number of communications required, or some combination thereof. Such importance metric(s) may assist in identifying sequences and processes that can most benefit from RPA that should be automated first.

[0025] To find the sequences, n-grams using a sliding window may be applied. See, for example, FIG. 8. For example, if the current value of n is 5, the first five interactions by a user may be compared to all time-ordered sequences of five interactions from other users, then interactions 2-6, 3-7, 4-8, etc., until all time-ordered sets of interactions for that size user have been compared with those of the other users under consideration. Alternatively, all user interactions of size n under consideration may be identified, retrieved from each user's time-ordered sequence of interactions, and stored in a database. For each n, matching sequences and the number of matches may be identified without using a sliding window. An importance metric(s) may also be applied to these matching sequences.

[0026] In some embodiments, the AI / ML model or a process that invokes the AI / ML model can associate interactions in the identified matching sequence with respective RPA activities. For example, a user clicking a “Submit” button can be associated with one or more RPA activities that identify the button on the screen, move the mouse to enter the button’s area, and click the button. Thus, an AI / ML model, or other application that uses an AI / ML model to identify a sequence or its output, can generate a “skeleton” RPA workflow that can be reviewed by an RPA developer, modified as needed, and implemented as an RPA robot. A draft RPA workflow can be created using task chains and exported to an RPA designer application. Alternatively, an RPA developer can be provided with a sequence and instructed to build a workflow that implements the sequence. Over time, as the AI / ML model becomes more accurate, a workflow can be created for the sequence, and an RPA robot implementing the workflow can be generated and deployed without human intervention.

[0027] Some sequences of interactions may be at least slightly different but accomplish the same task. To identify such sequences as functionally equivalent, some embodiments generate a probability graph containing loose relationships. Each possible or observed interaction, or a subset thereof, may be included as a node in the graph. The AI / ML model may calculate the probability that a user will "hop" from one node to another (i.e., the probability that the user will follow an edge between nodes). For example, if a spreadsheet is open, the probability that the user's next interaction will calculate column totals may be high, but the probability that the next interaction will open an email application may be relatively low. Edges may provide probabilities between nodes and, as a series of segments between them, may provide probabilities for sequences of nodes. Such sequences and their edges may provide the collective probability of starting at one node and arriving at another node via the sequence.

[0028] The probability graph can then be "pruned" to find related sequences after determining the edge probabilities. For example, edge probabilities less than 50%, 25%, 10%, etc. can be removed to reduce the number of edges and sequences. Levenshtein distance, fuzzy matching (e.g., where matching parameters are similar within a certain fuzziness threshold, such as within a 20% variation), and / or any other suitable technique can be used to determine related sequences without departing from the scope of the present invention. For example, two, three, or more sequences may lead to a particular end node with at least a certain probability, creating the possibility that a user could use these at least somewhat different sequences to accomplish the same task.

[0029] In some embodiments, the most efficient path among multiple related sequences may be desired, which may be determined based on the number of interactions / steps, the time associated with each step (e.g., obtained from timestamps in chronological data about user interactions), etc. An RPA workflow may then be generated based on the most efficient sequence that accomplishes the task.

[0030] In some embodiments, more complex interactions may be recognized and anticipated. For example, local control information may be used to determine interactions that are not visually obvious. Consider an example where a user wants to sync playlists between two applications (e.g., from iTunes® to Spotify®). The user might typically copy the song title from iTunes®, enter the song title into the Spotify® search box, search for the song, and repeat this process for the next song until the playlist is copied. The AI / ML model of some embodiments may recognize that the granular task of copying a song is repeated for every song in the playlist and prompt the user as to whether they want to copy all songs in the playlist.

[0031] In some embodiments, the AI / ML model may learn to automatically complete fields based on observed examples. For example, if a user creates an account from a data source that requires various information from that data source, the AI / ML model may observe that the user has entered information into some fields, generate a script to auto-fill all fields, and then ask the user if he or she wants the fields auto-filled. The AI / ML model may be deep learning neural network (DLNN) trained, shallow learning neural network (SLNN) trained, RNN trained, generative adversarial network (GAN) trained, combinations thereof, deterministic, etc. The advantage of deterministic rule-based systems for some applications is that they typically require fewer examples for training.

[0032] In some embodiments, AI / ML can be trained to recognize a desired outcome and determine the process(es) that lead to the desired outcome. For example, an AI / ML model can be trained to recognize when an account is created, a new user is added, revenue is generated, an issue is resolved, etc. The AI / ML model can then look back at data about user interactions or the user interactions themselves to recreate the sequence that led to the desired outcome. The AI / ML model or another process can then associate the interactions with activities and generate an RPA workflow that implements the sequence / process. This is called "programming by example"; by providing an example to the AI / ML model, it can then generate a program that can recreate the desired end state.

[0033] Modern networks are essentially large probability graphs. Once a desired outcome is determined, the AI / ML model can look back at the priors to evaluate what was most important to achieving the desired outcome. If it is determined that revenue increased as a result, the signal that led to the outcome can be decomposed into sub-signals to determine which sub-signal(s) led to the revenue increase.

[0034] In some embodiments, backpropagation may be used in neural networks. Much of modern deep learning is based on the backpropagation algorithm, which has mathematical foundations in optimization theory. Many recent modifications and improvements to deep learning owe much of their structure to what is known about how information is processed by the architecture of the human brain. That is, deep learning typically utilizes layered feedforward networks similar to the early layers of the visual cortex. The nonlinearity in each layer is typically a threshold nonlinearity, such as that found in the mean-field approximation of the integrate-and-fire model of neurons. Convolutional deep networks implement approximate translational symmetry, also found in the visual cortex. Various techniques can be used to backpropagate errors in a neural network, including, but not limited to: (1) providing a second network to backpropagate the error; (2) providing differences between basal and apical dendritic responses to spike inputs to justify backpropagation through the same layers as the forward propagation; (3) using global reinforcement learning; (4) using bidirectional recirculation in a recurrent network; and (5) using contrastive Hebbian learning.

[0035] Neural networks are generally pulse-gated, meaning that whether a "neuron" fires or not is based on the strength of the pulse and a mathematical function assigned to the neuron. Information comes in and goes out in the form of "spikes." A spike is an event, and a "synapse" governs how downstream neurons respond to that event. Thus, a spike from one neuron can propagate to others depending on the specific connections and logic of the neural network.

[0036] The nature of these connections and synapses controls behavior. A neuron may take the weights of the incoming synapses where a spike event occurred, sum them, and decide whether to generate a spike. Spikes may not change intensity in some embodiments, and are thus somewhat analogous to transmitting a single bit. A typical implementation may include neurons connected by a bus where communication is time-stamped; that is, spikes are emitted as events from one neuron at a certain time and then received by downstream neurons.

[0037] When information reaches a downstream neuron, a thresholded activity function (i.e., rectified linear unit (ReLU)) can be used such that if the value is below a threshold, nothing happens, but then if the value is above the threshold, the function responds linearly. ReLU functions are often used in deep learning because real neurons can have roughly similar activity functions. Linear transformations allow information to be subtracted, added, etc. In supervised learning, a known output is calculated at the network output, and an error is calculated from the known target output using a cost function, which can be the backpropagation error. An error can be calculated at the output, and this error can be converted into a modification to the network weights that minimizes the error.

[0038] A standard learning algorithm in some embodiments is Hebbian learning, which modifies weights by updating individual synapses based on the product of the firing rate of the presynaptic neuron and the firing rate of the receiving neuron (i.e., the postsynaptic neuron). For Hebbian learning (i.e., synaptic modification) to occur, the presynaptic and postsynaptic firing rates must be non-zero within a relatively short time; otherwise, the synapse will not be modified. In some embodiments, outside this time window, the synaptic weight may be decreased proportionally to its value.

[0039] Through pulse gating, firing rates can be propagated to pre- and postsynaptic neurons within the time frame in which Hebbian learning occurs. Thus, using pulse-gated learning, and thus synaptic modification, firing rates can control the onset and offset of learning and can be used to update synaptic weights to implement the backpropagation algorithm.

[0040] JPEG0007827709000001.jpg84152

[0041] JPEG0007827709000002.jpg20151

[0042] Backpropagation can be used to backpropagate errors to update synaptic weights. This allows for the strengthening of "neurons" that tend to lead to desirable outcomes. Somewhat similar to the process described above for probability graphs, nodes that appear to lead to desirable outcomes can be strengthened repeatedly until the desired outcome is reproduced. Due to the connections between nodes and their probabilistic weights, the behavior can be nearly analogous.

[0043] Accordingly, in some embodiments, the determined sequences of user interactions may be prioritized based on various metrics. For example, sequences that are more common, provide greater time savings to users through automation, etc. may be prioritized for RPA workflow generation. RPA robots that implement the sequences may be generated and deployed to user computing systems to accomplish tasks associated with the common sequences.

[0044] Typically, automating a sequence of a significant number of user interactions (e.g., 100) would require a significant amount of code in a programming language. However, some embodiments generate RPA robots that implement sequences of user interactions with much less or no coding. Indeed, task mining can be performed exclusively via AI or via AI with some human assistance.

[0045] In some embodiments, sequences may not need to be recognized at all. For example, associations between interactions and repetitions may be tracked. Neural networks may store associative data without memory overhead. Traditionally, a reverse index is used, where a user performs an action and then another action is performed using the reverse index. However, this reverse index may be prohibitively large in terms of resources in a real system. Alternatively, neural networks may be trained to achieve the same or similar results by reinforcing the neural network's nodes for an association graph without this memory overhead.

[0046] In some embodiments, AI may be used to execute the entire process end-to-end. For example, AI / ML model(s) may be used to determine the application, its graphical elements, and user interactions a user is using, determine common sequences of user interactions, and automatically build and deploy an RPA robot that implements the sequence of interactions at run time without design-time development. Training data may then be obtained from the user regarding scenarios in which the automatically developed robot operates or where anomalies occur, and this training data may be used to retrain and deploy the RPA robot. The training data may include areas in the image where misrecognition occurred or where elements were missed, label corrections, input corrections, etc. This process can be repeated to iteratively improve the robot.

[0047] Certain embodiments may be employed in robotic process automation (RPA). FIG. 1 is an architectural diagram illustrating an RPA system 100 according to an embodiment of the present invention. The RPA system 100 includes a designer 110 that enables developers to design and implement workflows. The designer 110 provides solutions for application integration and automates third-party applications, management information technology (IT) tasks, and business IT processes. The designer 110 can facilitate the development of automation projects, which are graphical representations of business processes. Simply put, the designer 110 facilitates the development and deployment of workflows and robots.

[0048] Automation projects enable rule-based process automation by giving developers control over the order of execution and relationships between custom sets of steps developed in workflows, defined herein as "activities." One commercial example of an embodiment of the designer 110 is UiPath Studio™. Each activity may include an action such as clicking a button, reading a file, or writing to a log panel. In some embodiments, workflows may be nested or embedded.

[0049] Workflow types may include, but are not limited to, sequences, flowcharts, FSMs, and / or global exception handlers. Sequences may be particularly well-suited for linear processes, allowing the flow of one activity from another without cluttering the workflow. Flowcharts may be particularly well-suited for more complex business logic, allowing for the integration of decisions and the connection of activities in more diverse ways through multiple branching logic operators. FSMs may be particularly well-suited for large workflows. FSMs may use a finite number of states during their execution that are triggered by conditions (i.e., transitions) or activities. Global exception handlers may be particularly well-suited for determining workflow behavior when an execution error is encountered or for debugging the process.

[0050] Once a workflow is developed in Designer 110, the execution of the business process is orchestrated by Conductor 120, which coordinates one or more Robots 130 that execute the workflow developed in Designer 110. One commercial example of an embodiment of Conductor 120 is UiPath Orchestrator™. Conductor 120 facilitates the management of the creation, monitoring, and deployment of resources in an environment. Conductor 120 may act as, or one of, an integration point with third-party solutions and applications.

[0051] The conductor 120 may manage all robots 130, connecting and executing them from a centralized point. Types of robots 130 that may be managed include, but are not limited to, attended robots 132, unattended robots 134, development robots (similar to unattended robots 134 but used for development and testing purposes), and non-production robots (similar to attended robots 132 but used for development and testing purposes). Attended robots 132 are triggered by user events and operate side by side with humans on the same computing system. Attended robots 132 may be used with the conductor 120 for centralized process deployment and logging media. Attended robots 132 may assist human users in accomplishing various tasks and may be triggered by user events. In some embodiments, processes cannot be initiated from the conductor 120 on this type of robot, and / or they cannot be run under a locked screen. In certain embodiments, the attended robot 132 can only be launched from the robot tray or from a command prompt. The attended robot 132 preferably operates under human supervision in some embodiments.

[0052] Unattended robots 134 operate unattended in virtual environments and can automate many processes. Unattended robots 134 can be responsible for providing remote execution, monitoring, scheduling, and work queue support. Debugging for all robot types can be performed in designer 110 in some embodiments. Both attended and unattended robots can automate a variety of systems and applications, including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those produced by SAP®, Salesforce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0053] The conductor 120 may have various capabilities, including, but not limited to, provisioning, deployment, versioning, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between robots 130 and the conductor 120 (e.g., web applications). Deployment may include ensuring the correct delivery of package versions to robots 130 assigned to perform. Versioning, in some embodiments, may include managing unique instances of some processes or configurations. Configuration may include maintaining and delivering robot environments and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot-specific data and maintaining user permissions. Logging may include storing and indexing logs in a database (e.g., an SQL database) and / or another storage mechanism (e.g., ElasticSearch®, which stores large data sets and provides the ability to quickly query them). The conductor 120 may provide interconnectivity by operating as a centralized point of communication for third-party solutions and / or applications.

[0054] Robots 130 are execution agents that execute workflows built by designer 110. One commercial example of some embodiments of robot(s) 130 is UiPath Robots™. In some embodiments, robots 130 install the Microsoft Windows Service Control Manager (SCM) management service by default. As a result, such robots 130 can open interactive Windows sessions under the local system account and may have Windows service rights.

[0055] In some embodiments, a robot 130 can be installed in user mode, meaning that for such a robot 130, the robot has the same rights as the user to whom it is installed. This feature can also be used for high-density (HD) robots, ensuring maximum utilization of each machine. In some embodiments, either type of robot 130 can be configured in an HD environment.

[0056] In some embodiments, the robot 130 is divided into multiple components, each specialized for a specific automation task. In some embodiments, the robot components include, but are not limited to, an SCM-managed robot service, a user-mode robot service, an executor, an agent, and a command line. The SCM-managed robot service manages and monitors Windows sessions and acts as a proxy between the conductor 120 and the execution host (i.e., the computing system on which the robot 130 executes). These services are responsible for managing credentials for the robot 130. A console application is launched by the SCM under Local System.

[0057] The user-mode robot service in some embodiments manages and monitors Windows sessions and acts as a proxy between the conductor 120 and the execution host. The user-mode robot service may be delegated and manage credentials for the robot 130. If the SCM management robot service is not installed, a Windows application may be launched automatically.

[0058] An Executor may execute a given job under a Windows session (i.e., execute a workflow). An Executor may be aware of dots per inch (DPI) settings per monitor. An Agent may be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. An Agent may be a client of a service. An Agent may ask to start or stop a job or change settings. A Command Line is a client of a service. A Command Line is a console application that can request the start of a job and wait for its output.

[0059] As described above, the separation of the robot 130 components helps developers, support users, and computing systems more easily implement, identify, and track what each component is doing. In this way, special behaviors can be configured for each component, such as setting different firewall rules for executors and services. Executors may always be aware of per-monitor DPI settings in some embodiments. As a result, workflows may execute at any DPI regardless of the configuration of the computing system on which the workflow was created. Also, in some embodiments, projects from the designer 110 may be made independent of the browser zoom level. For applications that are not DPI-aware or are intentionally marked as not-aware, some embodiments may disable DPI.

[0060] FIG. 2 is an architecture diagram illustrating a deployed RPA system 200 according to an embodiment of the present invention. In some embodiments, the RPA system 200 may be or be part of the RPA system 100 of FIG. 1. It should be noted that the client side, the server side, or both may include any desired number of computing systems without departing from the scope of the present invention. On the client side, the robot application 210 includes an executor 212, an agent 214, and a designer 216. However, in some embodiments, the designer 216 may not be running on the computing system 210. The executor 212 executes processes. As shown in FIG. 2, multiple business projects may be running simultaneously. The agent 214 (e.g., a Windows service) is the single connection point for all executors 212 in this embodiment. All messages in this embodiment are logged to the conductor 230, which further processes them via the database server 240, the indexer server 250, or both. As described above with respect to FIG. 1, the executor 212 may be a robotic component.

[0061] In some embodiments, a Robot represents an association between a machine name and a username. A Robot may manage multiple executors simultaneously. In computing systems that support multiple interactive sessions running simultaneously (such as Windows Server 2012), multiple Robots may run simultaneously, each running in a separate Windows session using a unique username. This is referred to as an HD Robot above.

[0062] The agent 214 is also responsible for transmitting the robot's status (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading required versions of packages to be fulfilled. Communication between the agent 214 and the conductor 230 is, in some embodiments, always initiated by the agent 214. In notification scenarios, the agent 214 may open a WebSocket channel that is later used by the conductor 230 to send commands (e.g., start, stop, etc.) to the robot.

[0063] The server side includes a presentation layer (web application 232, Open Data Protocol (OData) Representational State Transfer (REST) ​​Application Programming Interface (API) endpoint 234, notification and monitoring 236), a service layer (API implementation / business logic 238), and a persistence layer (database server 240, indexer server 250). Conductor 230 includes web application 232, OData REST API endpoint 234, notification and monitoring 236, and API implementation / business logic 238. In some embodiments, most actions a user performs in the conductor 230 interface (e.g., via browser 220) are performed by calling various APIs. Such actions may include, but are not limited to, launching jobs on a robot, adding / removing data from a queue, scheduling jobs to run unattended, etc., without departing from the scope of the present invention. Web application 232 is the visual layer of the server platform. In this embodiment, web application 232 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the present invention. A user interacts with web pages from web application 232, in this embodiment via browser 220, to perform various operations to control conductor 230. For example, a user may create robot groups, assign packages to robots, analyze per-robot and / or per-process logs, start and stop robots, etc.

[0064] In addition to the web application 232, the conductor 230 also includes a services layer that exposes an OData REST API endpoint 234. However, other endpoints may be included without departing from the scope of the present invention. The REST API is consumed by both the web application 232 and the agent 214, which in this embodiment is a supervisor of one or more robots on a client computer.

[0065] The REST API of this embodiment covers configuration, logging, monitoring, and queuing functionality. The configuration endpoint, in some embodiments, may be used to define and configure users, permissions, robots, assets, releases, and environments for an application. The logging REST endpoint may be used to log various information, such as errors, explicit messages sent by robots, and other environment-specific information. The deployment REST endpoint may be used by robots to query the version of the package that should be executed when a start job command is used in conductor 230. The queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to a queue, retrieving transactions from a queue, and setting the status of transactions.

[0066] Monitoring REST endpoints may monitor the web application 232 and the agents 214. The notification and monitoring API 236 may be a REST endpoint used to register the agents 214, deliver configuration settings to the agents 214, and send and receive notifications from the server and the agents 214. The notification and monitoring API 236 may use WebSocket communication in some embodiments.

[0067] The persistence layer, in this embodiment, includes a pair of servers—a database server 240 (e.g., SQL Server) and an indexer server 250. The database server 240 in this embodiment stores configurations for robots, robot groups, associated processes, users, roles, schedules, etc. This information is managed, in some embodiments, via a web application 232. The database server 240 may also manage queues and queue items. In some embodiments, the database server 240 may also store messages logged by robots (in addition to or instead of the indexer server 250).

[0068] Optionally in some embodiments, indexer server 250 stores and indexes information logged by the robots. In particular embodiments, indexer server 250 may be disabled via a configuration setting. In some embodiments, indexer server 250 uses ElasticSearch®, a full-text search engine from an open source project. Messages logged by the robots (e.g., using activities such as log messages or line writes) may be sent via logging REST endpoint(s) to indexer server 250, where they are indexed for future use.

[0069] FIG. 3 is an architecture diagram illustrating the relationships 300 between a designer 310, activities 320, 330, a driver 340, and an AI / ML model 350, according to an embodiment of the present invention. As can be seen, a developer uses the designer 310 to develop a workflow to be performed by the robot. The workflow may include user-defined activities 320 and UI automation activities 330. The user-defined activities 320 and / or UI automation activities 330, in some embodiments, may be located locally and / or remotely relative to the computing system on which the robot is operating and may invoke one or more AI / ML models 350. In some embodiments, non-text visual components in an image may be identified, referred to herein as computer vision (CV). Some CV activities associated with such components may include, but are not limited to, click, type, get text, hover, detect presence or absence of element, update scope, highlight, etc. In some embodiments, clicking identifies an element and clicks on it, for example, using CV, optical character recognition (OCR), fuzzy text matching, and multi-anchor. Type may identify an element using the above and types within elements. Get text may locate specific text and scan it using OCR. Hover may identify an element and hover over it. Detect element presence or absence may check to detect the presence or absence of an element on the screen using the techniques described above. In some embodiments, there may be hundreds or thousands of activities that can be implemented in designer 310. However, any number and / or types of activities may be utilized without departing from the scope of the present invention.

[0070] UI automation activities 330 are a subset of specialized low-level activities written in low-level code (e.g., CV activities) that facilitate interactions with a screen. UI automation activities 330 facilitate these interactions through drivers 340 and / or AI / ML models 350 that enable the robot to interact with desired software. For example, drivers 340 may include OS drivers 342, browser drivers 344, VM ​​drivers 346, enterprise application drivers 348, etc. One or more AI / ML models 350 may be used by UI automation activities 330 to determine the execution of interactions with the computing system. In some embodiments, AI / ML models 350 may augment or completely replace drivers 340. Indeed, in certain embodiments, drivers 340 are not included.

[0071] Drivers 340 may interact with the OS at a low level, such as by looking for hooks, monitoring keys, etc. They may facilitate integration with Chrome®, IE®, Citrix®, SAP®, etc. For example, a "click" activity plays the same role in these different applications via drivers 340.

[0072] FIG. 4 is an architecture diagram illustrating an RPA system 400, according to an embodiment of the present invention. In some embodiments, the RPA system 400 may be or include the RPA systems 100 and / or 200 of FIGS. 1 and / or 2. The RPA system 400 includes multiple client computing systems 410 that execute robots. The computing systems 410 can communicate with a conductor computing system 420 via web applications running thereon. The conductor computing system 420 can, in turn, communicate with a database server 430 and an optional indexer server 440.

[0073] 1 and 3, it should be noted that although web applications are used in these embodiments, any suitable client / server software may be used without departing from the scope of the present invention. For example, a conductor may run a server-side application on a client computing system that communicates with a non-web-based client software application.

[0074] FIG. 5 is an architecture diagram illustrating a computing system 500 configured to use AI / ML models to determine a sequence of user interactions with the computing system, extract common processes, and generate an RPA robot, according to an embodiment of the present invention. In some embodiments, computing system 500 may be one or more of the computing systems depicted and / or described herein. Computing system 500 includes a bus 505 or other communication mechanism for communicating information and processor(s) 510 coupled to bus 505 for processing information. Processor(s) 510 may be any type of general or application-specific processor, including a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), multiple instances thereof, and / or any combination thereof. Processor(s) 510 may also have multiple processing cores, at least some of which may be configured to perform specific functions. In some embodiments, multiple parallel processing may be used. In particular embodiments, at least one processor(s) 510 may be a neuromorphic circuit that includes processing elements that mimic biological neurons. In some embodiments, the neuromorphic circuit may not require the typical components of a von Neumann computing architecture.

[0075] The computing system 500 further includes memory 515 for storing information and instructions executed by the processor(s) 510. The memory 515 may be comprised of random access memory (RAM), read-only memory (ROM), flash memory, cache, static storage such as a magnetic or optical disk, or other types of non-transitory computer-readable media, or any combination thereof. The non-transitory computer-readable media may be any available media that can be accessed by the processor(s) 510 and may include volatile media, non-volatile media, or both. Also, the media may be removable, non-removable, or both.

[0076] Additionally, the computing system 500 includes a communication device 520, such as a transceiver, to provide access to a communication network via wireless and / or wired connections. In some embodiments, the communications device 520 may support any of the following radio technologies: Frequency Division Multiple Access (FDMA), Single Carrier FDMA (SC-FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), Orthogonal Frequency Division Multiplexing (OFDM), Orthogonal Frequency Division Multiple Access (OFDMA), Global System for Mobile (GSM) communications, General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), cdma2000, Wideband CDMA (W-CDMA), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), High-Speed ​​Packet Access (HSPA), Long Term Evolution (LTE), LTE Advanced (LTE-A), LTE-Advanced (LTE-B), LTE-Advanced (LTE-C), LTE-Advanced (LTE-B), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-C), LTE-Advanced (LTE-A), LTE-Advanced (LTE-C ... Advanced), 802.11x, Wi-Fi, Zigbee, Ultra-Wideband (UWB), 802.16x, 802.15, Home Node-B (HnB), Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Near-Field Communications (NFC), 5th Generation (5G), New Radio (NR), any combination thereof, and / or any other currently existing or future-implemented communication standard and / or protocol without departing from the scope of the present invention.In some embodiments, the communications device 520 may include one or more antennas that are a single antenna, an array of antennas, a phased antenna, a switched antenna, a beamforming antenna, a beamsteering antenna, a combination thereof, and / or any other antenna configuration without departing from the scope of the present invention.

[0077] The processor(s) 510 are further coupled via bus 505 to a display 525, such as a plasma display, a liquid crystal display (LCD), a light-emitting diode (LED) display, a field emission display (FED), an organic light-emitting diode (OLED) display, a flexible OLED display, a flexible substrate display, a projection display, a 4K display, a high-definition display, a Retina® display, an in-plane switching (IPS) display, or any other suitable display for displaying information to a user. The display 525 may be configured as a touch (haptic) display, a three-dimensional (3D) touch display, a multi-input touch display, a multi-touch display, or the like, using resistive, capacitive, surface acoustic wave (SAW) capacitive, infrared, optical imaging, dispersive signaling, acoustic pulse recognition, frustrated total internal reflection, or the like. Any suitable display device and haptic I / O may be used without departing from the scope of the invention.

[0078] A keyboard 530 and cursor control device 535, such as a computer mouse, touchpad, etc., are further coupled to bus 505 to allow a user to interface with computing system 500. However, in certain embodiments, a physical keyboard and mouse may not be present, and the user may interact with the device solely through display 525 and / or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. In certain embodiments, no physical input devices and / or displays are present. For example, a user may interact with computing system 500 remotely through another computing system in communication with it, or computing system 500 may operate autonomously.

[0079] The memory 515 stores software modules that, when executed by the processor(s) 510, provide functionality. The modules include an operating system 540 for the computing system 500. The modules further include an AI / ML model-based module 545 configured to perform all or a portion of the processes described herein, or derivatives thereof. The computing system 500 may include one or more additional functional modules 550 that include additional functionality.

[0080] Those skilled in the art will appreciate that a "system" may be embodied as a server, embedded computing system, personal computer, console, personal digital assistant (PDA), mobile phone, tablet computing device, quantum computing system, or any other suitable computing device or combination of devices without departing from the scope of the present invention. Presenting the above-described functions as being performed by a "system" is not intended to limit the scope of the present invention in any way, but rather to provide an example of many embodiments of the present invention. Indeed, the methods, systems, and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing techniques, including cloud computing systems. The computing system may be part of or otherwise accessible through a local area network (LAN), a mobile communications network, a satellite communications network, the Internet, a public or private cloud, a hybrid cloud, a server farm, any combination thereof, or the like. Any local or distributed architecture may be used without departing from the scope of the present invention.

[0081] It should be noted that some of the system features described herein are presented as modules to further emphasize implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integrated (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, etc.

[0082] Modules may also be implemented at least partially in software for execution by various types of processors. For example, an identified unit of executable code may include one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, a procedure, or a function. Nevertheless, executable identified modules need not be physically located together; they may include separate instructions stored in different locations that, when logically combined, comprise a module to achieve the purpose stated for the module. Furthermore, modules may be stored on non-transitory computer-readable media, such as, for example, a hard disk drive, a flash device, RAM, tape, and / or any other non-transitory computer-readable medium used to store data without departing from the scope of the present invention.

[0083] Indeed, a module of executable code may be a single instruction, many instructions, or even distributed across several different code segments, different programs, and multiple memory devices. Similarly, operational data may be identified and depicted herein within a module, and may be embodied and organized in any suitable form within any suitable type of data structure. Operational data may be collected as a single data set, or may be distributed in different locations across different storage devices, or may exist, at least in part, simply as electronic signals on a system or network.

[0084] FIG. 6 is an architecture diagram illustrating a system 600 configured to use AI / ML models to determine sequences of user interactions with computing systems, extract common processes, and generate RPA robots, according to an embodiment of the present invention. System 600 includes user computing systems such as a desktop computer 602, a tablet 604, a smartphone 606, etc. However, any desired computing system, including, but not limited to, a smartwatch, a laptop computer, etc., may be used without departing from the scope of the present invention. Also, while three user computing systems are shown in FIG. 6, any suitable number of computing systems may be used without departing from the scope of the present invention. For example, in some embodiments, tens, hundreds, thousands, or millions of computing systems may be used.

[0085] Each computing system 602, 604, 606 has a recorder process 610 running thereon that records screenshots and / or video of a user's screen or portions thereof. For example, a snippet of JavaScript can be embedded in a web browser as the recorder process 610 to track which components the user interacts with, what text the user types, which locations / components the user clicks with the mouse, what content the user scrolls past, how long the user pauses on a particular portion of content, etc. Scrolling past content may indicate that the content was somewhat close to what the user wanted, but didn't have exactly it. A click may indicate success.

[0086] The recorder process 610 need not be JavaScript and may be any suitable type of application and any desired programming language without departing from the scope of the present invention. This allows the recorder process 610 to be “generalizable” and track user interactions with multiple applications or any application with which a user is interacting. Using labeled training data from scratch can be challenging because it may enable an AI / ML model to recognize various controls but does not include information about which controls are commonly used and how. The recorder process 610 may be used to generate “heat maps” to help bootstrap the training process of the AI / ML model. Heat maps may include a variety of information, such as how often a user used an application, how often a user interacted with components of the application, the location of the components, and the content of the application / component. In some embodiments, heat maps may be derived from screen analysis, such as typed and / or pasted text detection, caret tracking, and active element detection of the computing systems 602, 604, 606. Some embodiments recognize where on a screen associated with computing system 602, 604, 606 a user types or pastes text, which may include hotkeys or other keys that do not display visible characters, and provide the physical location on the screen based on the current resolution (e.g., in coordinates) of where one or more characters were displayed, where the cursor was blinking, or both. The typing or pasting activity and / or the physical location of the caret may determine what field(s) the user is typing or focusing on and what application it is for process discovery or other applications.

[0087] Accordingly, in some embodiments, the recorder process 610 may record additional data to further assist in training the AI / ML model(s), such as web browser history, heat maps, key presses, mouse clicks, the location of mouse clicks and / or graphical elements on the screen with which the user is interacting, where the user was looking at the screen at different times, timestamps associated with screenshots / video frames, etc. This may be beneficial to provide key presses and / or other user actions that may not cause a screen change. For example, some applications may not provide a visual change when a user presses CTRL+S to save a file. However, in certain embodiments, the AI / ML model(s) may be trained solely based on captured screen images. The recorder process 610 may be a robot generated via an RPA designer application, part of an operating system, a downloadable application for a personal computer (PC) or smartphone, or any other software and / or hardware without departing from the scope of the present invention. Indeed, in some embodiments, the logic of one or more recorder processes 610 is implemented partially or fully via physical hardware.

[0088] Some embodiments are implemented in a feedback loop process that continuously or periodically compares a current screenshot with a previous screenshot to identify changes. The location on the screen where a visual change occurred may be identified and OCR may be performed on the location where the change occurred. The OCR results may then be compared with the contents of a keyboard cue (e.g., determined by key hooks) to determine whether a match exists. The location where the change occurred may be determined by comparing a box of pixels from the current screenshot with a box of pixels in the same location on the previous screenshot.

[0089] The images and / or other data recorded by recorder process 610 (e.g., web browser history, heat maps, key presses, mouse clicks, etc., the location of mouse clicks and / or graphical elements on the screen that the user is interacting with, the location of the user's view on the screen over a period of time, screenshots / video frames, voice input, gestures, emotions (e.g., whether the user is happy or annoyed), timestamps associated with biometrics (e.g., fingerprints, retinal scans, the user's pulse rate, etc.), information related to periods of user inactivity (e.g., "dead man's switch"), haptic information from a haptic display or touchpad, heat maps from multi-touch input, etc.) are sent via network 620 (e.g., a local area network (LAN), a mobile communications network, a satellite communications network, the Internet, any combination thereof, etc.) to server 630. In some embodiments, server 630 may be part of a public cloud architecture, a private cloud architecture, a hybrid cloud architecture, etc. In particular embodiments, server 630 may host multiple software-based servers on a single computing system 630. In some embodiments, server 630 may run a conductor application, and data from recorder process 610 may be transmitted periodically as part of a heartbeat message. In particular embodiments, data may be transmitted from recorder process 610 to server 630 when a predetermined amount of data has been collected, after a predetermined period of time has passed, or both. Server 630 stores the received data from recorder process 610 in database 640.

[0090] Server 630 includes AI / ML model 632. In some embodiments, AI / ML model 632 may have multiple layers performing various functions, such as statistical modeling (e.g., hidden Markov model (HMM)), and may utilize deep learning techniques (e.g., long short-term memory (LSTM) deep learning, encoding of prior hidden states, etc.) to identify sequences of user interactions. AI / ML model 632 may perform various tasks, such as extracting user interactions from recorder process 610, determining the sequence of the extracted user interactions, generating an RPA robot, etc. In some embodiments, the AI / ML model for determining the sequence of user interactions may be provided with interactions recognized by another AI / ML model along with the time the user interaction occurred, its location on the screen, etc.

[0091] In some embodiments, the AI / ML model 632 for recognizing a sequence of tasks may be provided with user interactions already recognized by another AI / ML model. However, in particular embodiments, the AI / ML model 632 undergoes a training phase using data from user interactions as input, training the AI / ML model 632 until it is sufficiently accurate but not overfitting to the training data. Acceptable accuracy may depend on the application. Identification errors may be highlighted by a human reviewer (e.g., by drawing a box around the misidentified element and including the correct identification), and this additional labeled data may be used to retrain the AI / ML model 632. Once sufficiently trained, the AI / ML model 632 may provide predictions as output, such as what application(s) and graphical element(s) will be recognized as present in the screen.

[0092] However, while this level of training provides information about what is present, more information may be needed to determine user interactions, such as comparing two or more consecutive screens to determine when typed characters appear from one to another, when a button is pressed, when a menu selection occurs, etc. Thus, after the AI / ML model 632 can recognize the graphical elements and applications on the screen, in some embodiments, the AI / ML model 632 is further trained to recognize labeled user interactions with the applications in the UI and understand such incremental actions taken by the user. Identification errors may be re-highlighted by a human reviewer (e.g., by drawing a box around the misidentified element and including the correct identification), and the AI / ML model 632 may be trained until its accuracy is high enough to be deployed to understand detailed user interactions with the UI.

[0093] In some embodiments, once trained to recognize user interactions, the trained AI / ML model 632 may be used to analyze video, pre-identified user interactions, and / or other information from the recorder process 610. This recorded information may include interactions that multiple / multiple users tend to perform. These interactions may then be analyzed for common sequences for subsequent automation.

[0094] The AI / ML model 632 (whether the same AI / ML model that identifies user interactions or a separate AI / ML model utilized for sequence recognition) in some embodiments needs to be trained to recognize sequences of interactions as its task. This may be accomplished by applying n-grams of multiple sizes to search for matching sequences in the data. The AI / ML model 632 may be trained to use a range of sizes of n that tend to identify meaningful sequences (e.g., based on frequency, importance metric(s), etc.). In some embodiments, n-grams are applied to sequences of user interactions over a sliding window to identify matching sequences.

[0095] The AI / ML model 632 can also be trained to associate user interactions in a sequence with RPA activities. This can be accomplished by having a human reviewer first identify the RPA activity for a given user interaction. The AI / ML model 632 can learn over time to predict activities for somewhat new interactions. For example, if previous activity for a menu item included finding a "Submit" button image on the screen, moving the mouse to position over the button's area, and clicking the button, the AI / ML model 632 can learn to perform a similar process for the "OK" button, the "Next" button, etc.

[0096] After the AI / ML model 632 can associate interactions with respective activities, the AI / ML model 632 or another process can be trained to generate an RPA workflow having a sequence of activities that accomplishes the task of the sequence of user interactions. The generated RPA workflow may then be used to generate and deploy an RPA robot, or the workflow may be sent to an RPA developer for review, such as via an application 652 (e.g., an RPA designer application) of the computing system 650 that allows the RPA developer to view and modify the RPA workflow. In some embodiments, the AI / ML model 632 may initially send the workflow to the RPA developer and, after the RPA developer accepts a certain number of RPA workflows without modification, switch to automatically generating and deploying RPA robots that implement the workflow logic without human interaction.

[0097] In some embodiments, the AI / ML model 632 may be trained to recognize sequences of user interactions that accomplish the same task but that are at least slightly different. The AI / ML model 632 may generate a probability graph that includes loose associations between user interactions. The probability graph may then be "pruned" to determine which sequences are related. For example, interactions may be removed, and the AI / ML model 632 may check whether the same task is accomplished. Levenshtein distance, fuzzy matching, and / or any other suitable techniques may be used to determine related sequences without departing from the scope of the present invention.

[0098] In some embodiments, the AI / ML model 632 may be trained to determine the most efficient path among multiple related sequences, which may be determined based on the number of interactions / steps, the time associated with each step (e.g., obtained from timestamps in chronological data about user interactions), etc. An RPA workflow may then be generated based on the most efficient sequence to accomplish the task.

[0099] In some embodiments, the AI / ML model 632 can be trained to recognize a desired outcome without first analyzing a sequence of user interactions. For example, the AI / ML model 632 can be trained to recognize that an account is created, a new user is added, revenue is generated, an issue is resolved, etc. The AI / ML model 632 can then look back at data about the user interactions or the user interactions themselves to recreate the sequence that led to the desired outcome. The AI / ML 632 model or another process can then associate the interactions with activities and generate an RPA workflow that implements the sequence.

[0100] Modern networks are essentially large probabilistic graphs. In some embodiments, once a desired outcome is determined, the AI / ML model 632 may look back at the priors to evaluate what was most important to achieving the desired outcome. If it is determined that revenue increased as a result, the signal that led to the outcome may be decomposed into sub-signals to determine which sub-signal(s) led to the increased revenue.

[0101] In some embodiments, sequences may not need to be recognized at all. For example, the AI / ML model 632 may be trained to recognize and track associations between interactions and their repetitions. If the AI / ML model 632 is a neural network, for example, based on an association graph, the AI / ML model 632 may strengthen the nodes of the neural network to tend to select interactions that lead to desirable outcomes.

[0102] Accordingly, in some embodiments, AI may be used to execute the entire process end-to-end. For example, AI / ML model(s) 632 may be used to determine the applications, their graphical elements, and user interactions used by users, determine common sequences of user interactions, and automatically build and deploy RPA robots that implement the sequences of interactions at run time without design-time development. Training data may then be obtained from users of computing systems 602, 604, 606 for scenarios in which the automatically developed robot performs tasks or in which anomalies occur, and this training data may be provided to server 630 and used to retrain AI / ML model 632. The training data may include areas in images where misrecognitions or overlooked elements occur, label corrections, input corrections, etc. This process may be repeated to iteratively improve the RPA robot.

[0103] AI layer

[0104] In some embodiments, multiple AI layers may be used. Each AI layer is an algorithm (or model) that runs on data, and the AI ​​model itself may be a deep learning neural network (DLNN) of artificial "neurons" trained on training data. Layers may run in serial, parallel, or a combination thereof.

[0105] AI layers may include, but are not limited to, a sequence extraction layer, a clustering detection layer, a visual component detection layer, a text recognition layer (e.g., OCR), a speech-to-text translation layer, or any combination thereof. However, any desired number and type(s) of layers may be used without departing from the scope of the present invention. Using multiple layers allows the system to develop a global picture of what is happening on the screen. For example, one AI layer may perform OCR, another may detect buttons, another may compare sequences, etc. Patterns may be determined individually by one AI layer or collectively by multiple AI layers.

[0106] FIG. 7 is a flowchart illustrating a process 700 for determining sequences of user interactions with a computing system, extracting common processes, and generating an RPA robot using an AI / ML model, according to an embodiment of the present invention. The process begins, at 710, by providing the AI / ML model with data including time-ordered interactions of multiple users with their respective computing systems. The AI / ML model is then trained, at 720, to recognize relevant sequences in the data. In some embodiments, the training includes training the AI / ML model to recognize relevant sequences of task-related user interactions in the time-ordered sequence of user interactions by comparing n-grams of the sequences of user interactions in the recorded data from the computing system with a sliding window to find relevant sequences. The n-grams may include more than one value of n.

[0107] In some embodiments, training includes training the AI / ML model to use one or more importance metrics when determining whether a task is found in a relevant sequence. In particular embodiments, the one or more importance metrics include whether the relevant course of action generates at least a certain amount of revenue, saves at least a certain amount of revenue, increases the efficiency and / or speed of a task by at least a certain amount, speeds up the customer acquisition process, reduces the number of communications required, or a combination thereof. In some embodiments, training includes training the AI / ML model to determine a minimum value of n such that at least a majority of sequences of size n are related to the task performed by the user. In particular embodiments, training includes training the AI / ML model to determine a maximum value of n such that n-grams beyond the maximum value of n are not considered. In some embodiments, training includes training the AI / ML model or an RPA designer application to associate user interactions with RPA activities. In particular embodiments, a minimum number of related sequences is required for the AI / ML model to determine that a sequence is related to a task.

[0108] The trained AI / ML model is then deployed at 730. In some embodiments, the deployed AI / ML model is configured to be invoked by one or more RPA robots. At 740, recorded actual user interactions of multiple users having respective computing systems are analyzed, and a set of matching sequences in the recorded actual user interactions is determined by the trained AI / ML model. In some embodiments, determining the set of matching sequences includes: generating, by the AI / ML model, a probability graph including associations between sequences of user interactions; pruning, by the AI / ML model, the probability graph to remove irrelevant user interactions and sequences; and determining, by the AI / ML model, matching sequences of user interactions as part of the set of matching sequences based on the pruned probability graph. In certain embodiments, determining whether sequences match is performed using Levenshtein distance, fuzzy matching, or a combination thereof.

[0109] The most efficient sequence of each set of matching sequences is determined at 750. In some embodiments, the determination of the most efficient sequence is based on the number of user interactions in the sequence, the time associated with the user interactions in the sequence, or a combination thereof. An RPA workflow implementing the most efficient sequence is then generated at 760. The RPA workflow includes activities that implement the user interactions of the sequence. In certain embodiments, the one or more generated RPA workflows are made accessible to one or more RPA designer applications at 770, allowing an RPA developer to edit the workflow as needed. Additionally or alternatively, a respective RPA robot is generated at 780 using the most efficient sequence. The one or more generated RPA robots may then be deployed to one or more user computing systems at 790.

[0110] FIG. 8 is a flowchart illustrating a process 800 for performing process extraction based on a desired end state using an AI / ML model and generating an RPA robot, according to an embodiment of the present invention. The process begins at 810 by providing the AI / ML model with data including time-ordered interactions of multiple users with their respective user computing systems. The AI / ML model is then trained to recognize desired outcomes at 820 and determine the process(es) that lead to the desired outcomes at 830. For example, the AI / ML model may be trained to recognize various desired states, such as an account created, a new user added, revenue generated, a problem resolved, etc. The AI / ML model may then look back at data about user interactions or the user interactions themselves to recreate the sequence that led to the desired outcome. In some embodiments, once the desired outcome is determined, the AI / ML model may look back at the priors to evaluate what was most important to achieving the desired outcome. The signals that led to the outcome may be decomposed into sub-signals, and which sub-signals led to the desired outcome may be determined. The AI / ML model is then deployed at 840.

[0111] Next, at 850, a desired outcome of the recorded actual user interaction data is determined by the AI / ML model. The AI / ML model then determines, at 860, the user interactions that lead to the desired outcome (e.g., by trying chronological interactions before the desired outcome is achieved to attempt to reproduce the desired outcome), or experiments by strengthening the nodes of the neural network player until the desired outcome is reproduced. An RPA workflow that leads to the desired outcome is then generated at 870 (e.g., by adding activities to the RPA workflow that are related to the user interactions, adding one or more activities that invoke one or more AI / ML models trained to perform the task via backpropagation, etc.). The one or more generated RPA workflows are made accessible to one or more RPA designer applications, respective RPA robots are generated / deployed, or both, at 880.

[0112] 9 is a diagram illustrating the comparison of n-grams representing sequences of user interactions according to an embodiment of the present invention. In this example, sequence 900 represents a chronological series of interactions of a first user, sequence 910 represents a chronological series of interactions of a second user, and sequence 920 represents a chronological series of interactions of a third user. Any number of users' interactions may be compared without departing from the scope of the present invention. Here, the n-gram size for the comparison is 5.

[0113] First, the 5-grams 902 of the first user's first five interactions (A1, A2, AA3, A4, A5) are compared to the time-ordered 5-grams of sequences 910 and 920. Any number of different interactions can be identified and compared without departing from the scope of the present invention. In this example, both sequences 910 and 920 contain matching 5-grams 912, 922, respectively, in the first 12 interactions. Because three matching 5-gram sequences 902, 912, 922 are found, this may indicate that this 5-gram is a potential candidate for process automation. Additionally, in some embodiments, the number of matches may be stored.

[0114] After a matching 5-gram sequence is identified for a first user, the 5-gram for a second use can be compared with a third user and any subsequent users to potentially find new matches with previously found matching sequences removed (e.g., by storing the interactions in a queue data structure and removing them from the queue as matches are found). This process can be repeated for all users until the penultimate user's interactions have been analyzed. If a range of n values ​​is considered, n can be incremented and the process can be repeated until all n-gram sizes within that range have been analyzed.

[0115] The process steps performed in Figures 7 and 8 may be performed by a computer program encoding instructions to a processor(s) to perform at least a portion of the process(es) described in Figures 7 and 8 according to an embodiment of the present invention. The computer program may be stored on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, RAM, tape, and / or any other such medium or combination of media used to store data. The computer program may include coded instructions for controlling a processor(s) of a computing system (e.g., processor(s) 510 of computing system 500 of Figure 5) to implement all or a portion of the process steps described in Figures 7 and 8, which may also be stored on a computer-readable medium.

[0116] The computer program may be implemented in hardware, software, or a hybrid implementation. The computer program may be composed of modules in operable communication with each other and designed to send information or instructions to a display. The computer program may be configured to run on a general-purpose computer, an ASIC, or any other suitable device.

[0117] It will be readily understood that the components of the various embodiments of the present invention, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention, as represented in the accompanying figures, is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention.

[0118] The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, references throughout this specification to "certain embodiments," "some embodiments," or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Thus, the appearances of "certain embodiments," "some embodiments," "other embodiments," or similar language throughout this specification do not necessarily refer to the same group of all embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0119] It should be noted that references to features, advantages, or similar language throughout this specification do not imply that all of the features and advantages that may be realized in the present invention are to be found in any single embodiment of the present invention, or in any embodiment of the present invention. Rather, language referring to features and advantages is understood to mean that the particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussions of features and advantages throughout this specification, and similar language, may, but do not necessarily, refer to the same embodiment.

[0120] Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. Those skilled in the relevant art will recognize that the invention may be practiced without a particular feature or advantage of one or more particular embodiments. In other instances, additional features and advantages may be recognized in particular embodiments, although they may not be present in all embodiments of the invention.

[0121] Those of ordinary skill in the art will readily appreciate that the invention as described above can be implemented using steps in a different order and / or with hardware elements in different configurations than those disclosed. Thus, while the invention has been described in terms of these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will become apparent while remaining within the spirit and scope of the invention. Accordingly, reference should be made to the appended claims to determine the scope of the invention.

Claims

1. providing, by a computing system, an artificial intelligence (AI) / machine learning (ML) model with data including time-ordered interactions of a plurality of users with each of the users' computing system; training, by the computing system, the AI / ML model to recognize relevant sequences of task-related user interactions in the time-ordered sequence of user interactions of the plurality of users by comparing n-grams of sequences of user interactions in recorded data from the computing system with a sliding window to find the relevant sequences of user interactions, wherein training the AI / ML model includes: determining a minimum value of n such that a majority of sequences of at least size n relate to tasks performed by said plurality of users; determining a highest value of n such that n-grams beyond said highest value of n are not considered, said n-grams including two or more values ​​of n; deploying the trained AI / ML model, wherein the deployed AI / ML model is configured to be invoked by one or more robotic process automation (RPA) robots; A computer-implemented method, wherein at least one of the related sequences of user interactions does not exactly match at least one of the related sequences of other user interactions.

2. The computer-implemented method of claim 1 , wherein a minimum number of related sequences is required for the AI / ML model to determine that the sequence is relevant to the task.

3. 10. The computer-implemented method of claim 1, further comprising: training, by the computing system, the AI / ML model to use one or more importance metrics in determining whether a task is found in the relevant sequence.

4. 4. The computer-implemented method of claim 3, wherein the one or more importance metrics include an associated course of action generating at least a certain amount of revenue, saving at least a certain amount of revenue, increasing the efficiency and / or speed of a task by at least a certain amount, speeding up the customer acquisition process, reducing the number of communications required, or a combination thereof.

5. 10. The computer-implemented method of claim 1, further comprising training the AI / ML model or robotic process automation (RPA) designer application to associate user interactions with RPA activities.

6. analyzing, with the AI / ML model, recorded actual user interactions of a plurality of users with their respective computing systems to determine a set of relevant sequences in the recorded actual user interactions, wherein the recorded actual user interactions include additional data to assist in training the AI / ML model, the additional data including web browser history, heat maps, key presses, mouse clicks, locations of mouse clicks and graphical elements on the screen with which the users are interacting, locations where the users were looking at the screen at different times, timestamps associated with screenshots or video frames; 6. The computer-implemented method of claim 5, further comprising: generating one or more respective robotic process automation (RPA) workflows that include activities that implement the determined set of related sequences of user interactions.

7. generating respective RPA robots that implement the one or more generated RPA workflows; 7. The computer-implemented method of claim 6, further comprising: deploying the one or more generated RPA robots to one or more user computing systems.

8. 10. The computer-implemented method of claim 6, further comprising making the generated one or more RPA workflows accessible to one or more RPA designer applications.

9. Determining the set of related sequences comprises: generating a probability graph containing associations between sequences of user interactions using the AI / ML model; pruning the probability graph to remove irrelevant user interactions and sequences according to the AI / ML model; and determining, by the AI / ML model, that a sequence of user interactions matches as part of the set of related sequences based on the pruned probability graph.

10. The computer-implemented method of claim 9 , wherein determining whether the sequences match is performed using Levenshtein distance, fuzzy matching, or a combination thereof.

11. Determining a most efficient sequence from a set of related sequences to generate an RPA workflow based on one or more efficiency metrics; 7. The computer-implemented method of claim 6, further comprising: generating each of the RPA workflows using the most efficient sequence.

12. 12. The computer-implemented method of claim 11, wherein one of the one or more efficiency metrics for determining the most efficient sequence is based on a number of user interactions in the sequence, a time associated with user interactions in the sequence, or a combination thereof.

13. A non-transitory computer-readable medium having stored thereon a computer program, the computer program causing at least one processor to: training an artificial intelligence (AI) / machine learning (ML) model to recognize relevant sequences of user interactions related to a task in the time-ordered sequences of user interactions of a plurality of users by comparing n-grams of the sequences of user interactions in the recorded data with a sliding window to find the relevant sequences of user interactions, and training the AI / ML model includes: determining a minimum value of n such that at least a majority of sequences of size n relate to tasks performed by said plurality of users; determining a highest value of n such that n-grams beyond said highest value of n are not considered, said n-grams including two or more values ​​of n; analyzing recorded actual user interactions of the plurality of users with their respective computing systems with the AI / ML model to determine the set of related sequences in the recorded actual user interactions of the plurality of users; configured to generate one or more respective robotic process automation (RPA) workflows including activities that implement the associated sequences of user interactions of the determined set of associated sequences; A non-transitory computer-readable medium, wherein at least one of the related sequences of user interactions does not exactly match at least one of the related sequences of other user interactions.

14. The computer program further comprises the at least one processor: generating respective RPA robots that implement the one or more generated RPA workflows; 14. The non-transitory computer-readable medium of claim 13, configured to deploy the one or more generated RPA robots to one or more user computing systems.

15. The computer program further comprises the at least one processor:

14. The non-transitory computer-readable medium of claim 13 configured to make the generated one or more RPA workflows accessible to one or more RPA designer applications.

16. Determining the set of related sequences comprises: generating a probability graph containing associations between sequences of user interactions using the AI / ML model; pruning the probability graph to remove irrelevant user interactions and sequences according to the AI / ML model; and determining, by the AI / ML model, that a sequence of user interactions matches as part of the set of related sequences based on the pruned probability graph.

17. 17. The non-transitory computer-readable medium of claim 16, wherein determining whether the sequences match is performed using Levenshtein distance, fuzzy matching, or a combination thereof.

18. The computer program further comprises the at least one processor: determining a most efficient sequence from the set of related sequences to generate an RPA workflow based on one or more efficiency metrics; 14. The non-transitory computer-readable medium of claim 13 configured to generate the respective RPA workflow using the most efficient sequence.

19. 20. The non-transitory computer-readable medium of claim 18, wherein one metric of the one or more efficiency metrics for determining the most efficient sequence is based on a number of user interactions in the sequence, a time associated with user interactions in the sequence, or a combination thereof.

20. determining the set of relevant sequences in the recorded actual user interactions includes using one or more importance metrics; 14. The non-transitory computer-readable medium of claim 13, wherein the one or more importance metrics include an associated course of action generating at least a certain amount of revenue, saving at least a certain amount of revenue, increasing the efficiency and / or speed of a task by at least a certain amount, speeding up the customer acquisition process, reducing the number of communications required, or a combination thereof.

21. 14. The non-transitory computer-readable medium of claim 13, wherein generation of each RPA workflow requires a minimum number of related sequences.

22. a memory for storing computer program instructions; and at least one processor configured to execute the computer program instructions, the computer program instructions causing the at least one processor to: training an artificial intelligence (AI) / machine learning (ML) model to recognize relevant sequences of user interactions related to a task in the time-ordered sequences of user interactions of a plurality of users by comparing n-grams of the sequences of user interactions in the recorded data with a sliding window to find the relevant sequences of user interactions, and training the AI / ML model includes: determining a minimum value of n such that at least a majority of sequences of size n relate to tasks performed by said plurality of users; determining a highest value of n such that n-grams beyond said highest value of n are not considered, said n-grams including two or more values ​​of n; analyzing recorded actual user interactions of the plurality of users with their respective computing systems with the AI / ML model to determine the relevant sequences in the recorded actual user interactions of the plurality of users; determining a most efficient sequence of said related sequences of user interactions; configured to generate one or more respective robotic process automation (RPA) workflows including activities that implement the determined most efficient associated sequence of user interactions; At least one of the related sequences of user interactions does not exactly match at least one of the related sequences of other user interactions.

23. The computer program instructions further include causing the at least one processor to: generating an RPA robot that implements the generated RPA workflow; 23. The system of claim 22, configured to deploy the generated RPA robot to one or more computing systems.

24. The computer program instructions further include causing the at least one processor to:

23. The system of claim 22, configured to make the generated RPA workflow accessible to one or more RPA designer applications.

25. 23. The system of claim 22, wherein the determination of the most efficient sequence is based on a number of user interactions in the sequence, a time associated with user interactions in the sequence, or a combination thereof.

26. determining the relevant sequences in the recorded actual user interactions includes using one or more importance metrics; 23. The system of claim 22, wherein the one or more importance metrics include an associated course of action generating at least a certain amount of revenue, saving at least a certain amount of revenue, increasing efficiency and / or speed of a task by at least a certain amount, speeding up the customer acquisition process, reducing the number of communications required, or a combination thereof.

27. 23. The system of claim 22, wherein the generation of each RPA workflow requires a minimum number of associated sequences.

Images