Methods, systems, and computer program products for processing communication workflows using machine learning techniques.

Abstract

This disclosure generally relates to evaluating communication workflows composed of tasks using machine learning techniques. More specifically, this disclosure relates to predicting task outcomes for generating communication workflows, generating recommendations for one or more tasks to add to portions of a communication workflow to complete the workflow, and systems and methods for generating vector representations of communication workflows.

Description

[0001] Cross-references to related applications

[0002] This disclosure claims priority to U.S. Patent Application Serial No. 16 / 800,869, filed February 25, 2020, entitled “Enhanced Processing For Communication Workflows Using Machine-Learning Techniques,” which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to using machine learning techniques to evaluate communication workflows composed of tasks. More specifically, this disclosure relates to systems and methods for predicting task outcomes for generating communication workflows using machine learning or artificial intelligence, generating recommendations for one or more tasks to be added to portions of the communication workflow to complete the workflow, and generating vector representations of the communication workflows. Background Technology

[0004] Workflows can be configured to trigger communications to users at specific times. For example, code can be executed to trigger one or more communications to a user's device via a communication channel at a given time. Multiple communications on different communication channels can be coordinated within a workflow to engage individual users. However, due to the complexity of workflows, evaluating the prospects of tasks that lead users to perform target actions is a technical challenge at the scale of big data. Determining tasks to be included in a workflow is often done manually, without fully utilizing previous workflows. Furthermore, given the complexity of task placement across various workflows, evaluating workflows relative to each other is inefficient and computationally intensive. Summary of the Invention

[0005] Certain aspects and features of this disclosure relate to using machine learning or artificial intelligence to process communication workflows. A communication workflow may include an ordered sequence of one or more tasks. Executing a communication workflow may include performing one or more tasks in an ordered sequence. For example, a task executing a communication workflow may include enabling a server to transmit communication (e.g., a digital message) to a target group of user devices via a communication channel. The communication may be configured to enable users operating user devices in the target group to perform one or more actions (e.g., selecting a link included in the communication). Instructions for the actions(one or more) performed at the user device may be received at a cloud-based application. Once the tasks(one or more) of the communication workflow have been executed, the cloud-based application may determine the task outcome of the communication workflow. The task outcome may represent an overall performance metric for the communication workflow. For example, the overall performance metric may include the percentage of user devices (in the target group of user devices) that performed the target action (e.g., selecting a link included in the communication) across all tasks in the communication workflow.

[0006] For illustrative purposes and as a non-limiting example only, a communication workflow may include a sequence of two tasks: a first task is an email communication task and a (subsequent) second task is a text messaging task. The email task may include transmitting email communications to a target group on a mobile device. The text messaging task may include later transmitting text messages (e.g., Short Message Service (SMS)) to a subset of the target group on the mobile device. When an email communication is received at the target group on the mobile device, the email communication may include an optional link. When a text message is received at a subset of the target group on the mobile device, the text message may also include the optional link. The cloud-based application may receive an indication, either via email communication or via text message, of whether the link was selected at the target group on the mobile device. The cloud-based application may determine the task outcome of the communication workflow (e.g., an overall performance metric for both tasks) as the percentage of users who selected the link via either email communication or text message. As another example, the cloud-based application may sum the partial transmission counts (e.g., the number of communications transmitted) for each of the email communication task and the text messaging task used in the communication workflow. The cloud-based application may also sum the partial link selection counts (e.g., the number of links selected) for each of the email communication task and the text messaging task. Then, cloud-based applications can determine the task outcome of the communication workflow by determining the ratio of the total link selection count to the total transmission count.

[0007] According to some aspects described herein, task outcomes of a communication workflow can be predicted by generating a composite feature vector representing the communication workflow and inputting the composite feature vector into a machine learning model. The machine learning model can be trained using previously executed communication workflows and their corresponding task outcomes. Furthermore, certain aspects and features may involve using machine learning techniques to generate recommendations for tasks to be added to a portion of the communication workflow to complete it. A set of tasks can be recommended to complete a portion of the communication workflow by inputting the composite feature vector of a portion of the communication workflow into a machine learning model to generate the output of a task representing certain values ​​(e.g., values ​​of task outcomes exceeding a threshold) that produce the task outcomes of previously executed communication workflows. Certain aspects and features may also involve using neural networks (e.g., Graph2Vec models) to generate vector representations of the communication workflow in a computationally efficient manner. These vector representations can be input into various machine learning models to perform various functions. According to some aspects of this disclosure, the vector representation can represent the entire communication workflow, including its structure, tasks, and task content.

[0008] In some implementations, cloud-based applications can generate composite feature vectors representing the tasks of previously executed communication workflows. Multidimensional machine learning feature vectors (e.g., singular value decomposition (SVD)) can be used to generate composite feature vectors. These composite feature vectors can be fed into a machine learning model to generate outputs predicting the task outcomes of new communication workflows (e.g., an overall performance metric or overall rate of the target actions being performed by the user receiving the communication) or recommendations for tasks to be included in new partial or incomplete communication workflows.

[0009] In some examples, a communication workflow can be a marketing campaign orchestration (MCO). A communication workflow can include an ordered sequence of tasks. Tasks can be configured to transmit digital communications to a set of target user devices (e.g., an audience). In some examples, a composite feature vector can be used as a normalized representation of the communication workflow, representing the tasks, parameters, target users, and content of the communication workflow. Multidimensional machine learning feature vectors can be generated to create the composite feature vector. Each task in the ordered sequence of tasks in the communication workflow can be represented by a task vector (e.g., a feature vector). The group of target user devices that receive the communication as a result of performing the tasks can be represented by a group vector. In some examples, the structure and component task vectors of the communication workflow can be combined to generate the composite feature vector representing the communication workflow.

[0010] An ordered sequence of tasks included in a communication workflow can represent the structure of the workflow. The structure of each communication workflow can include, for example, one or more nodes (e.g., start, split, and end nodes) or one or more stages (e.g., tasks, such as transmitting emails to a collection of user devices) connected by one or more branches. Branches can include a flow of a series of tasks (e.g., filtering tasks, transmission tasks, or delay tasks). For example, a node can represent a split task, where a target group of user devices is split into two or more subgroups. In some examples, the structure of a communication workflow can include a root node (e.g., a start task indicating the beginning of the communication workflow), intermediate nodes (e.g., communication tasks that transmit communications (such as emails) to user devices when executed), and leaf nodes (e.g., end nodes indicating the end of the communication workflow). Each tree branch can include a sequence of one or more tasks. Composite feature vectors can be generated to represent the overall structure of the communication workflow, the tasks, and the content of the communication for each task.

[0011] In some implementations, cloud-based applications can input composite feature vectors of a communication workflow into a machine learning model to generate outputs predicting the task outcomes of the communication workflow. Task outcomes can represent the performance of the communication workflow, including, for example, click-through rate or conversion rate for the target user group. In some implementations, given a new partial communication workflow (e.g., a workflow under development but not yet completed), a machine learning model can be used to evaluate the composite feature vectors of a portion of a previously executed workflow and the composite feature vector of the new partial communication workflow. According to these implementations, the machine learning model can generate recommended outputs representing one or more tasks added to the new partial communication workflow to complete it.

[0012] In some implementations, machine learning techniques (e.g., Graph2Vec) can be used to analyze the communication workflow as a graph to learn the rooted subgraph for each node. Machine learning techniques can generate a vector representation of the entire workflow without separating the structure of the communication workflow from its composite feature vectors. According to these implementations, the vector representation can generate an n-dimensional vector representing the entire communication workflow, including structure, tasks, and task content. As a non-limiting example, a neural network of a Graph2Vec model can be executed to learn the rooted subgraph embeddings of the communication workflow in an unsupervised manner. The resulting vector representation can be fed into a machine learning model to generate predictions of task outcomes for new communication workflows or to generate recommendations to be added to parts of the communication workflow to complete those parts, as described above.

[0013] In some implementations, a system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which, in operation, cause the system to perform actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause that device to perform actions. One general aspect includes a computer-implemented method comprising: accessing a workflow comprising one or more tasks coordinated in a sequential order, the workflow being configured to facilitate interaction with a set of user devices, each of the one or more tasks comprising executable code that, upon execution, performs a function associated with the set of user devices (e.g., splitting, filtering, transferring), and the workflow being associated with one or more parameters (e.g., a vocabulary for topic modeling, content images, etc.) representing each of the one or more tasks of the workflow; generating a composite feature vector representing the workflow, the composite feature vector being generated using feature vectors of each of the one or more tasks of the workflow, and the feature vectors of each of the one or more tasks being generated by performing one or more machine learning techniques using the one or more parameters representing the tasks; inputting the composite feature vector of the workflow into a trained machine learning model, the trained machine learning model having been trained to predict performance values ​​of the workflow, and the trained machine learning model having been trained using a training dataset representing one or more previously executed workflows and corresponding previous performance values ​​of each of the one or more previously executed workflows; and generating an output using the trained machine learning model that predicts the performance values ​​of the workflow. Other embodiments of this aspect include corresponding computer systems, apparatuses, and executable code stored on a non-transitory computer-readable storage medium, each configured to perform actions of a method.

[0014] The implementation may include one or more of the following features. The computer-implemented method further includes: determining the structure of a workflow, represented by a plurality of nodes in a tree structure, wherein two nodes in the plurality of nodes are connected by one or more stages, wherein each task in one or more tasks of the workflow corresponds to a node in the plurality of nodes or a stage in one or more stages; and evaluating a training dataset to determine whether the structure of the workflow matches the structure of at least one of one or more previously executed workflows. The computer-implemented method further includes: determining that the structure of the workflow matches (e.g., is the same as) the structure of a set of previously executed workflows in the training dataset. The computer-implemented method may also include comparing a composite feature vector of the workflow with a composite feature vector of each of the previously executed workflows in the set of previously executed workflows in a domain space. The computer-implemented method may also include selecting a subgroup of the set of previously executed workflows based on a threshold and the comparison result. The computer-implemented method may also include identifying a previous performance value for each of the previously executed workflows in the subgroup of previously executed workflows. The computer-implemented method may also include generating an output predicting the performance value of the workflow by determining a combination of previous performance values ​​associated with the subgroup of previously executed workflows. The computer-implemented method further includes: determining that the structure of the workflow matches the structure of one or more previously executed workflows in the training dataset, wherein during the generation of the training dataset: determining one or more subsets of the previously executed workflows, each subset corresponding to two or more previously executed workflows sharing a common structure. The computer-implemented method may also include performing a clustering operation (e.g., k-means clustering) on ​​each subset of the one or more subsets of the previously executed workflows, wherein each subset of the previously executed workflows is associated with one or more clusters of the previously executed workflows, wherein the one or more clusters are formed by performing a clustering operation on composite feature vectors of subsets of the previously executed workflows in a domain space, and wherein each previously executed workflow included in a cluster is associated with a composite feature vector that corresponds to the composite feature vectors of other previously executed workflows in that cluster; and generating an output predicting the performance values ​​of the workflow by identifying specific subsets of the training dataset whose corresponding structure corresponds to one or more subsets of the workflow structure. The computer-implemented method may further include assigning a workflow to a specific cluster corresponding to a specific subset of the one or more clusters based on a comparison of the workflow's composite feature vector with the composite feature vector of each of the one or more clusters. The computer-implemented method may also include determining a combination of prior performance values ​​associated with a previously executed workflow, which is associated with the specific cluster to which the workflow is assigned.The computer-implemented method further includes: determining that the structure of the workflow matches the structure of a set of previously executed workflows from one or more previously executed workflows in the training dataset, wherein during the generation of the training dataset: identifying one or more subsets of previously executed workflows, each subset corresponding to two or more previously executed workflows sharing a common structure. The computer-implemented method may also include training a supervised machine learning model for each subset of the one or more subsets of previously executed workflows; generating an output predicting the performance value of the workflow by: identifying a specific subset of the one or more subsets of the training dataset whose corresponding structure matches the structure of the workflow, inputting a composite feature vector of the workflow into the supervised machine learning model corresponding to the specific subset, and generating an output predicting the performance value of the workflow. The computer-implemented method further includes: determining that the structure of the workflow does not match the structure of any previously executed workflow from one or more previously executed workflows in the training dataset; and generating an output predicting the performance value of the workflow by: segmenting each previously executed workflow in one or more previously executed workflows in the training dataset into multiple previously executed sub-workflows. The computer-implemented method may further include comparing the structure of the workflow with each of a plurality of sub-workflows of each of one or more previously executed workflows in a training dataset. The computer-implemented method may further include identifying a set of previously executed sub-workflows that match the structure of the workflow among the plurality of previously executed sub-workflows. The computer-implemented method may further include comparing a composite feature vector of the workflow with a composite feature vector of each of the previously executed sub-workflows in a set of previously executed sub-workflows in a domain space. The computer-implemented method may further include selecting a subgroup of the set of previously executed sub-workflows based on a threshold and the comparison result. The computer-implemented method may further include determining a predicted performance value for the workflow based on previous performance values ​​associated with the selected subgroup of the previously executed sub-workflows. The computer-implemented method further includes a workflow defined by a list including metadata describing the structure of the workflow. The computer-implemented method may further include each task included in the workflow. The computer-implemented method may further include each feature vector representing a task among one or more tasks in the workflow. The computer-implemented method may further include a list being parsed to perform one or more functions. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0015] In some implementations, a system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which, in operation, cause the system to perform actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause that device to perform the actions. A general aspect includes a computer-implemented method comprising: providing an interface enabling a user to define a workflow, the workflow being configured to include a set of tasks facilitating interaction with a set of user devices, and each task in the set of tasks including executable code that, when executed, performs a function associated with the set of user devices (e.g., splitting, filtering, transferring); receiving at the interface input corresponding to selecting one or more tasks from the set of tasks, the one or more tasks defining a partial workflow, the partial workflow being associated with one or more parameters (e.g., a vocabulary for topic modeling, content images, etc.) for each of the one or more tasks characterizing the partial workflow; and identifying the structure of the partial workflow, the structure of the partial workflow consisting of one or more sections. The method includes: representing points and / or one or more stages, wherein each task in one or more tasks of a partial workflow corresponds to a node in one or more nodes or a stage in one or more stages; generating a composite feature vector representing the partial workflow, which is generated using the feature vectors of each task in the one or more tasks of the workflow, and the feature vectors of each task in the one or more tasks are generated by performing one or more machine learning techniques using one or more parameters characterizing the task (e.g., constructing machine learning feature vectors based on the parameters of the nodes and / or tasks of the new partial workflow); accessing a set of previously executed partial workflows, each of which is represented by a structure, a composite feature vector, and a performance value. The computer-implemented method further includes selecting a subset of the set of previously executed partial workflows that shares the same structure as the partial workflow.The computer-implemented method further includes determining one or more previously executed partial workflows similar to a subset of previously executed partial workflows, the similarity being based on a comparison between the composite feature vector of each previously executed partial workflow in the subset and the composite feature vector of the partial workflow; generating recommendations for completing the partial workflows, the recommendations including one or more recommended tasks for completing the partial workflows, the one or more recommended tasks being selected from one or more remaining tasks in one or more previously executed partial workflows that share the same structure as the partial workflows and are determined to be similar to the partial workflows, and the selection is based on previous performance values ​​of the one or more previously executed partial workflows that share the same structure as the partial workflows and are determined to be similar to the partial workflows; and displaying the recommendations on an interface. Other embodiments of this aspect include corresponding computer systems, apparatuses, and executable code or instructions (e.g., computer program products) stored on a non-transitory computer-readable storage medium, each configured to perform actions of the method.

[0016] Implementations may include one or more of the following features. A computer-implemented method, wherein comparing a composite feature vector of each previously executed partial workflow from a subset of previously executed partial workflows sharing the same structure as the partial workflow with the composite feature vector of the partial workflow includes: for each previously executed partial workflow, calculating a distance in a domain space between the composite feature vector of the previously executed partial workflow and the composite feature vector of the partial workflow; and comparing the distance with a threshold, wherein when the distance is equal to or less than the threshold, the previously executed partial workflow is determined to be similar to the partial workflow, and when the distance is greater than the threshold, the previously executed partial workflow is determined to be dissimilar to the partial workflow. The computer-implemented method further includes: sorting one or more previously executed partial workflows that share the same structure as the partial workflow and are determined to be similar to the partial workflow by arranging one or more previously executed partial workflows from highest performance value to lowest performance value. The computer-implemented method further includes: receiving additional input at an interface, the additional input corresponding to the selection of one or more recommended tasks to complete the partial workflow. The computer-implemented method may further include sequentially adding one or more recommended tasks to one or more tasks in response to receiving additional input to represent a complete workflow. The computer-implemented method also includes receiving additional input at an interface corresponding to the selection of at least one recommended task from one or more recommended tasks and at least one task not included in one or more recommended tasks, the selection of at least one recommended task and at least one task not included in one or more recommended tasks completing a partial workflow. The computer-implemented method may further include sequentially adding at least one recommended task and at least one task not included in one or more recommended tasks to one or more tasks in response to receiving additional input to represent a complete workflow. The computer-implemented method further includes a partial or complete workflow defined by a list including metadata describing: the structure of the workflow, each task included in the workflow, and each feature vector representing a task in a set of tasks included in the workflow. The computer-implemented method also includes parsing through the list to perform one or more functions. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0017] In some embodiments, a system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which, in operation, cause the system to perform actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause that device to perform actions. A general aspect includes a computer-implemented method comprising: accessing a workflow comprising one or more tasks coordinated in a sequential order, the workflow being configured to facilitate interaction with a set of user devices, each of the one or more tasks including executable code. The code, upon execution, performs functions associated with a set of user devices (e.g., splitting, filtering, transferring), and the workflow is associated with one or more parameters (e.g., a vocabulary for topic modeling, content images, etc.) for each of one or more tasks characterizing the workflow; defines a tree structure for the workflow, comprising multiple nodes and one or more stages, wherein two nodes of the multiple nodes of the tree structure are connected through at least one stage of one or more stages, and wherein each task of the one or more tasks of the workflow corresponds to a node of the multiple nodes or a stage of one or more stages; inputs the tree structure into a machine learning model (e.g., a graph-to-vector model), which results in sampling one or more portions of the tree structure of the workflow; and generates a vector representation of the tree structure of the workflow. Other embodiments of this aspect include corresponding computer systems, apparatuses, and executable code stored on a non-transitory computer-readable storage medium, each configured to perform actions of the method.

[0018] Implementations may include one or more of the following features. The computer-implemented method further includes: accessing a set of previously executed workflows; defining a tree structure for each previously executed workflow in the set of previously executed workflows; inputting the tree structure of each previously executed workflow in the set of previously executed workflows into a machine learning model; generating a vector representation of the tree structure of each previously executed workflow in the set of previously executed workflows; and storing the vector representation of each previously executed workflow in a training dataset. The computer-implemented method generates vector representations of the tree structure without evaluating vector representations of individual nodes among a plurality of nodes in the tree structure or individual stages among one or more stages. The computer-implemented method associates each node among a plurality of nodes in the tree structure and each stage among one or more stages with metadata used to characterize a task or part of a workflow. The computer-implemented method associates the metadata associated with each node or stage with a function performed by a task associated with that node or stage. The computer-implemented method uses a neural network configured to learn embeddings of the tree structure of the workflow. A computer-implemented method, wherein the evaluation of one or more nodes of a tree structure further includes: extracting one or more rooted partial tree structures for each of the plurality of nodes of the workflow's tree structure or for each of the one or more stages, each of the one or more rooted partial tree structures comprising a set of nodes one or more hops from the node or stage. The computer-implemented method may also include training a machine learning model using the one or more rooted partial tree structures. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium. Attached Figure Description

[0019] This specification refers to the following figures, wherein the same reference numerals are used in different figures to indicate the same or similar components.

[0020] Figure 1 This is a block diagram illustrating an example of a network environment according to some aspects of this disclosure.

[0021] Figure 2 This is a block diagram illustrating another example of a network environment according to some aspects of this disclosure.

[0022] Figure 3 An example of a process flow for predicting task results for generating a communication workflow, according to some aspects of this disclosure, is illustrated.

[0023] Figure 4 An example of a process flow for generating task recommendations to complete a portion of the communication workflow, according to some aspects of this disclosure, is illustrated.

[0024] Figure 5This is a flowchart illustrating an example process for predicting the results of a task in a communication workflow, according to some aspects of this disclosure.

[0025] Figure 6 This is a flowchart illustrating an example process for generating task recommendations to complete a portion of the communication workflow, based on some aspects of this disclosure.

[0026] Figure 7 This is a flowchart illustrating an example process for generating a vector representation of a communication workflow according to some aspects of this disclosure.

[0027] Figure 8 This is a simplified diagram illustrating a distributed system used to implement one of the embodiments.

[0028] Figure 9 It is a simplified block diagram illustrating one or more components of the system environment.

[0029] Figure 10 The illustration shows an exemplary computer system in which various embodiments of the present invention can be implemented. Detailed Implementation

[0030] Cloud-based applications may include a data management platform hosted on a cloud network. This platform can be configured to enable users to perform certain functions on data records. For example, a data record may be a user profile that includes one or more data fields. The data fields of a data record may include values ​​representing the user associated with the user profile. These data fields may be generated based on previous interactions, such as interactions between a user-operated browser and a web server.

[0031] Communication workflows can be executed by cloud-based applications (or, in some cases, by one or more servers outside the cloud network) to process certain data records. As a non-limiting example, a communication workflow can be configured as a coordinated sequence of one or more tasks (e.g., transmitting digital messages to attract users) performed on a target group of a user profile. Once a task is executed by the server, it enables the server to transmit communication to the user devices associated with the user profile via a communication channel. The content of the communication and the communication channel selected for transmission can be determined based on the configuration of the tasks and the communication workflow. Including a target group of user devices in a communication workflow allows tasks included in the workflow to transmit communication to each user device in the target group via a communication channel. The task configuration can define the content of the communication and the type of communication channel selected to facilitate the transmission of the communication. Non-limiting examples of communication channels include SMS messages, emails, telephone calls, pop-up messages while browsing web pages, and other suitable communication channels.

[0032] As the complexity of communication workflows increases, their configuration and execution present technical challenges. Furthermore, comparing two or more complex communication workflows is also a significant technical challenge. As a practical application, certain aspects and features of this disclosure generate vector representations of communication workflows to enhance comparisons of various new communication workflows with previously executed ones, predict the performance of new workflows, and recommend computationally efficient methods for completing parts of the communication workflow. Therefore, certain aspects and features of this disclosure improve cloud-based data management platform technology by enabling platforms to represent the structure and content of communication workflows using vectors (e.g., composite feature vectors or vectors generated using a graph-to-vector model) to compare and evaluate various communication workflows in a computationally efficient manner.

[0033] Some embodiments of this disclosure include cloud-based applications configured to generate vector representations of communication workflows. The vector representations can be generated by evaluating the tasks and content of the communication workflow using machine learning techniques such as Latent Dirichlet Assignment (LDA) for topic modeling and Word2Vec models. Feature vectors can be generated for each task included in the communication workflow. The feature vectors of the various tasks in the communication workflow can be concatenated to generate a composite feature vector. The composite feature vector can then be fed into a machine learning model to generate predictions of the task outcomes of the communication workflow. The composite feature vector can also be fed into a machine learning model to generate recommendations for including one or more tasks in the communication workflow to complete the workflow.

[0034] Figure 1 This is a block diagram illustrating an example of a network environment 100 used to process data records using a data management platform such as cloud-based application 120. Cloud network 110 can be any cloud-based network including one or more cloud-based servers and one or more cloud-based databases. The one or more cloud-based servers of cloud network 110 may include at least one application server hosting cloud-based application 120, and at least one database server. For example, cloud-based application 120 may be a data management platform configured to perform functions associated with one or more datasets of data records. In some implementations, a data record may represent a user profile including one or more data fields. Each of the one or more data fields may include a value generated in response to an interaction between a user device and a native application or web server during a previous interaction. For example, when a user device accesses a web server to load a web page on the user device, the web server may extract one or more user characteristics (e.g., the location where the request to load the web page was initiated) associated with the request from the user device to load the web page. Various user characteristics can be collected over time in the user profile.

[0035] Client system 130 may include a network of one or more servers and databases associated with a client (e.g., an entity such as a company). In some implementations, the network within client system 130 may include a web server hosting one or more web pages or an application server hosting native applications operated by or associated with the client. User device 140 may interact with client system 130 by transmitting communications to the web server of client system 130. For example, the communications may be a request to load a web page onto a browser running on user device 140. Similarly, user device 150 may transmit communications to client system 130 requesting that a web page be loaded onto the browser of user device 150. It will be appreciated that any number of user devices and any type of computing device (e.g., desktop computers, tablet computers, electronic kiosks, etc.) may communicate with client system 130.

[0036] In some implementations, client system 130 may define one or more workflows for communicating with user equipment. A communication workflow may be a sequence of orchestrated tasks, such as an MCO (Mean Control Order). Tasks may include executable code that, when executed by a server, causes client system 130 to transmit communication to user equipment via a communication channel. The content of the communication and the specific communication channel selected for transmission may be defined by the executable code of the task. For example, a task may be a digital message designed to attract certain target users. Workflows orchestrate a sequence of one or more tasks for an individual user profile within defined time intervals. Workflows can be effectively used to attract users associated with certain goods or services. As an illustrative example, client system 130 may store at least two workflows. A first workflow may include two tasks for user equipment 140, and a second workflow may include three tasks for user equipment 150. The first workflow may include a first task transmitting communication to user equipment 140 at a first time using a text messaging communication channel and a second task transmitting communication to user equipment 140 at a later second time using an email communication channel. The second workflow may include a first task of transmitting communication to user device 150 using an email communication channel at a first time, a second task of transmitting communication to user device 150 using a social media communication channel at a later second time (e.g., transmitting a message to a user's account using a social media platform), and a third task of transmitting communication to user device 150 using another email communication channel at a later third time. Communication received at the user device enables a user operating the user device to perform one or more actions. For example, communication may include an optional link. The user may choose to select the link or not. When the user performs an action, client system 130 may detect the action because a signal may have been transmitted back to client system 130 (or no signal may have been received within a defined time period). In some implementations, client system 130 may transmit a signal to cloud-based application 120 hosted on cloud network 110. The signal may include a response received from the user device in response to the received communication. Continuing the example above, client system 130 may transmit a signal to cloud-based application 120 when user device 140 responds to each of the first and second tasks, and similarly, when user device 150 responds to each of the first, second, and second tasks. In other implementations, cloud-based application 120 can perform workflow tasks instead of client system 130 performing task execution.

[0037] In some implementations, client system 130 can access cloud-based application 120 to create, design, or generate communication workflows. For example, cloud-based application 120 can generate interface data that causes an interface to be displayed on an end user associated with client system 130. The interface enables the user to create an ordered sequence of tasks and save this ordered sequence as a communication workflow. Each task can be configured to perform functions, such as dividing a target group of user devices into subgroups or transmitting communication to the target group of user devices. Cloud-based application 120 enables the end user associated with client system 130 to generate a composite feature vector representing the communication workflow. The composite feature vector can then be fed into a trained machine learning model to generate an output predicting the task outcomes of the communication workflow (e.g., the overall performance of all tasks in the communication workflow or a prediction of success). Cloud-based application 120 can also feed the composite feature vector into a trained machine learning model to generate recommendations for adding to or including in the communication workflow to complete the workflow (if the communication workflow is a partial workflow). The cloud-based application 120 can also execute one or more neural networks to learn subgraph embeddings of the communication workflow, thereby transforming the communication workflow into a vector representation.

[0038] Figure 2 This is a block diagram illustrating another example of a cloud network 110 according to some aspects of this disclosure. Cloud-based application 120 may include a network of cloud-based servers and databases. In some embodiments, the network of cloud-based application 120 may include a workflow structure generator 210, a feature vector generator 220, a workflow representation system 230, a workflow performance predictor 240, a partial workflow predictor 250, and a machine learning model 260. The machine learning model 260 may be trained using a training dataset 270.

[0039] Workflow structure generator 210 can be any server, processor, and / or database configured to store and potentially execute code that generates the structure of the communication workflow. Workflow structure generator 210 can receive input from users accessing cloud-based application 120. The input may represent the user selecting one or more tasks to include in a new communication workflow. The one or more tasks may be arranged in an ordered sequence determined by the user. The ordered sequence of tasks may represent the communication workflow. Workflow structure generator 210 can evaluate the ordered sequence of tasks upon receiving input to generate the structure of the communication workflow. The structure may be a tree structure including multiple nodes. Each node may represent a task. As an illustrative example, workflow structure generator 210 can generate a structure by searching for a starting task (e.g., the task that initiates the communication workflow) in the communication workflow. Workflow structure generator 210 can create a hierarchical data structure and use the starting task as the root node. Workflow structure generator 210 can continue searching for the next task in the communication workflow, which may be an email task (e.g., a task that delivers an email to a user's device). Workflow structure generator 210 can add stages (e.g., child nodes) representing email tasks to the root node of a hierarchical data structure. The next task could be a split task, where the target group of user devices is split into two or more subgroups. Workflow structure generator 210 can add another node to the hierarchical data structure to represent the split task. Workflow structure generator 210 can continue adding stages (e.g., nodes) to the hierarchical data structure until the communication workflow reaches a final task (e.g., a task to complete the communication workflow). The resulting hierarchical data structure can represent the structure of the communication workflow. The generated structure can be stored at workflow structure generator 210.

[0040] Feature vector generator 220 can be any server, processor, and / or database configured to generate task vectors for each task in a communication workflow. For example, U.S. Serial No. 16 / 286,297, filed February 26, 2019, describes a process for generating vectors representing tasks in a communication workflow, which is incorporated herein by reference in its entirety. Feature vector generator 220 can then, for example, cascade the task vectors of the communication workflow. The concatenation of the resulting task vectors can represent a composite feature vector of the communication workflow.

[0041] In some implementations, feature vector generator 220 can generate representations of the content and metadata of tasks included in the communication workflow. As an illustrative example, for an email task, feature vector generator 220 can generate representations of various parameters of the email communication. Parameters may include numerical and categorical variables of metadata associated with the email (e.g., product category, task type, etc.). In this example, the representation of email metadata may be preserved as is, scaled, dimensionality reduced, or processed in another way. Parameters may also include the email subject line and the email body. In this case, natural language processing (NLP), term frequency matrix, term frequency-inverse document frequency (TF-IDF), topic modeling, etc., can be performed to transform the text into a vector representation of the text. Parameters may also include email content (e.g., image data). In this case, feature vector generator 220 can use, for example, dimensionality reduction techniques (e.g., nonnegative matrix factorization (NMF), singular value decomposition (SVD), principal component analysis (PCA), and other suitable techniques) to transform raw image vectors into task vectors or as intermediate outputs of a neural network.

[0042] Feature vector generator 220 can also generate one or more vectors representing one or more target groups of user devices that will receive one or more communications in the communication workflow. As a non-limiting example, it is assumed that each user device included in a target group of user devices is represented by a feature vector (e.g., a vector representation of the characteristics of the user device), and the target group can be summarized by defining the target-group vector as the average (e.g., centroid) of the feature vectors of the constituent members of the target group. However, this disclosure is not limited to this technique. For illustration, the communication workflow may include an email communication task and a text messaging task. The email communication task can be configured to transmit emails to each user device in a first target group of user devices. The text messaging task can be configured to transmit text messages to each user device in a second target group of user devices. In this case, feature vector generator 220 can generate vectors for each user device in the first and second target groups of user devices.

[0043] Workflow representation system 230 can be any server, processor, and / or database configured to generate a vector representation of the communication workflow by learning graph embeddings of the communication workflow through the execution of a neural network. For example, workflow representation system 230 can input the structure of the communication workflow (e.g., determined by workflow structure generator 210) into a Graph2Vec model to generate an n-dimensional vector representation of the communication workflow.

[0044] Workflow performance predictor 240 can be any server, processor, and / or database configured to receive a vector representation of a communication workflow as input and generate a prediction of the task outcome of the communication workflow as output. In some embodiments, the vector representation is a composite feature vector generated by feature vector generator 220. In other embodiments, the vector representation is an n-dimensional vector representation of the communication workflow generated by workflow representation system 230. Workflow performance predictor 240 can input the vector representation of the communication workflow into a machine learning model (e.g., a supervised learning model trained with labeled data, such as known overall performance values ​​of previously executed workflows). The machine learning model can be a supervised, semi-supervised, or unsupervised model (e.g., k-means clustering). The output of the machine learning model can represent a prediction of the performance of the communication workflow. In some examples, the overall performance of the communication workflow can be represented by a task outcome that represents the rate at which a target user performs an action in response to receiving the communication (e.g., conversion rate, click-through rate of selecting a link included in a digital message, open rate of web pages opened from links included in the communication, etc.).

[0045] It will be recognized that an action is not necessarily associated with a link included in a communication. A user device may receive communication with a link, but a user may independently access a webpage to purchase goods or services. Even if the target action is the selection of a link, the purchase of goods or services can still be considered the target action, for example, if the link has already navigated the user to a webpage that enables the user to purchase goods or services.

[0046] Workflow performance predictor 240 can generate output predicting task outcomes, which may be a value representing the predicted rate of a target action performed by the user in response to receiving communication across all tasks in the communication workflow. Task outcomes can be predicted by evaluating known task outcomes of previously executed communication workflows (e.g., calculating the average of known task outcomes of previously executed communication workflows with the same structure as the new communication workflow and identified as similar in vector space). Previously executed communication workflows may be communication workflows for which tasks have previously been performed. Previously executed communication workflows may be target groups that have previously participated in or caused the transmission of communication to the user device. Previously executed communication workflows may be stored in training dataset 270. The cloud-based application 120 may have already stored task outcomes associated with each previously executed communication workflow.

[0047] In some implementations, the workflow performance predictor 240 can identify a set of previously executed communication workflows having the same structure as the communication workflow. Within the set of previously executed communication workflows, the workflow performance predictor 240 can identify one or more previously executed communication workflows similar to the communication workflow. Similarity can be measured in a multidimensional space of the composite feature vectors of the previously executed communication workflows. For example, a previously executed communication workflow with a given structure whose corresponding composite feature vectors are within a threshold distance from the composite feature vector of the communication workflow can be determined to be similar to the communication workflow. In some examples, the task outcome of the communication workflow can be predicted based on a combination of known task outcomes (e.g., average, weighted average, etc.) of the previously executed communication workflows determined to be similar to the communication workflow. In some examples, a confidence interval around the predicted task outcome of the communication workflow can be constructed based on the calculated variance of the known task outcomes of the previously executed communication workflows determined to be similar to the communication workflow and having the same structure as the new communication workflow.

[0048] In some implementations, previously executed communication workflows can be segmented by structure. For a given structure, machine learning techniques (e.g., k-means clustering) can be used to evaluate the composite feature vectors of the grouped previously executed communication workflows to form one or more clusters of previously executed communication workflows. Clustering can be performed based on the composite feature vectors of the previously executed communication workflows. Workflow performance predictor 240 can then assign communication workflows to clusters based on a comparison of the composite feature vectors of the communication workflows with the composite feature vectors associated with each cluster. After assigning communication workflows to clusters, the task outcomes of the communication workflows can be predicted based on a combination of known task outcomes of the previously executed communication workflows within the cluster (e.g., average, weighted average, etc.). In some examples, confidence intervals around the predicted task outcomes of the communication workflows can be constructed based on the variance calculated from the known task outcomes of the previously executed communication workflows within the cluster.

[0049] In some implementations, previously executed communication workflows can be segmented by structure. For each workflow structure, the workflow performance predictor 240 can train a machine learning model (e.g., supervised learning of classification or regression) using a composite feature vector of the previously executed communication workflows with that structure. The composite feature vector of the communication workflow can be input into a machine learning model associated with the structure of the communication workflow to generate a prediction of the task outcome. In some examples, confidence intervals around the predicted task outcome of the communication workflow can be determined based on a specific model type (e.g., linear regression, random forest regression, etc.).

[0050] In some examples, none of the previously executed communication workflows share the same structure as the communication workflow. Workflow performance predictor 240 can identify a set of portions of previously executed communication workflows that have the same structure as the communication workflow. Workflow performance predictor 240 can then use the techniques described above (e.g., identifying composite feature vectors within a threshold distance of the composite feature vectors of the communication workflow) to evaluate the composite feature vectors of the set of portions of the previously executed communication workflows to determine which portions are similar to the communication workflow. Similar to the techniques described above, known task results of the set of portions of the previously executed workflows (e.g., determining an average value) can be combined to determine a prediction of the task results for the communication workflow. In some implementations, the communication workflow can be segmented into partial workflows such that each partial workflow corresponds to the structure of a previously executed communication workflow. Using the techniques described above, the task results of each partial workflow can be determined based on known task results of previously executed communication workflows that share the same structure as the partial workflows. The predicted task results of each partial workflow can then be combined (e.g., averaged or summed) to determine the predicted task results of the complete communication workflow.

[0051] Partial workflow predictor 250 can be any server, processor, and / or database configured to generate recommendations for one or more tasks to complete a communication workflow. Cloud-based application 120 can generate an interface that enables users to create communication workflows. If a user has already created a partial communication workflow (e.g., the user has not yet set an end node), then partial workflow predictor 250 can be configured to generate recommendations to the user about which tasks to add to complete the partial communication workflow. Given a new partial communication workflow, partial workflow predictor 250 can generate a composite feature vector of the new partial communication workflow based on the techniques described above. Partial workflow predictor 250 can then identify a set of previously executed communication workflows with the same structure as the new partial communication workflow. As described above, partial workflow predictor 250 can determine the parts of previously executed communication workflows similar to the new partial communication workflow based on comparisons of composite feature vectors in the domain space. Partial workflow predictor 250 can rank the set of parts of previously executed communication workflows similar to the new partial communication workflow in descending order based on the known task results of the previously executed communication workflows. The partial workflow predictor 250 can then recommend the highest-ranking one or more partial segments of the previously executed communication workflow for the entire previously executed communication workflow. This recommendation may include the remaining tasks of the partial segments of the previously executed communication workflow.

[0052] The machine learning model 260 can be any server, processor, and / or database configured to generate, train, or execute machine learning or artificial intelligence models. For example, the machine learning model can be generated using one or more machine learning algorithms, such as an ensemble of multi-label classifiers (e.g., supervised or unsupervised learning), artificial neural networks (including backpropagation, Boltzmann machines, etc.), Bayesian statistics (e.g., Bayesian networks or knowledge bases), learn-to-rank techniques, logistic tree models, decision tree models, support vector machines, information fuzzy networks, hidden Markov models, hierarchical clustering (unsupervised), self-organizing maps, clustering techniques, and other suitable machine learning techniques (supervised, semi-supervised, or unsupervised).

[0053] Feature vector generator 220 can separate the structural representation of the communication workflow (e.g., a tree-based representation of the communication workflow including "nodes" and "branches") from the vector representation of the communication workflow (e.g., a concatenation of task vectors and vectors representing target groups of user devices). Additionally, machine learning model 260 can generate models for each structure of a previously executed communication workflow (e.g., a machine learning model can generate and train performance values ​​for each individual structure of the communication workflow, one pre-computed cluster per structure, and so on). The separated structural representations can be used to select a subset of the training dataset 270 of previously executed communication workflows that have the same structure as the new communication workflow (e.g., a communication workflow that has not yet been executed or is currently being executed). However, it will be appreciated that in some implementations, an alternative approach could be to generate one or more lists (e.g., a single list) of metadata describing the entire communication workflow. Cloud-based application 120 can execute code to determine a complete description of the communication workflow through list comprehension, including the structure of the communication workflow, the tasks included in the communication workflow, the task features of each task included in the communication workflow, the composite feature vector of the communication workflow, and other suitable information items. In some implementations, the list can be a data structure representing a communication workflow. For example, a list of previously executed communication workflows can be automatically parsed to identify previously executed communication workflows with the same structure as a given communication workflow. The list can also be automatically parsed to detect previously executed communication workflows similar to a given communication workflow (e.g., having (i) the same structure, and (ii) similarity in the multidimensional domain space of the task vectors that make up the task, etc.). The cloud-based application 120 can evaluate the list as a composite per-communication workflow vector that can be used as input to a tree-based machine learning model (e.g., training and applying a random forest model).

[0054] For illustrative purposes and as a non-limiting example only, a communication workflow may include four tasks. The structure of a communication workflow may include four “nodes” (e.g., “start”, “split”, “end branch 1”, and “end branch 2”).The list used for this communication workflow can be represented by the following data structure: [num_nodes = 4, node_type = “start”, num_tasks = 3, task_type = “filter”, filter_type, filter_input = [<target_group_vector for “targetgroup 0”> ], task_type = “email_communication”, communication_input = [<target_group_vector for “target group 1”> ], communication_task_vector = [<concatenated task content and context task vector of “task 1”> ], task_type = "delay", delay_input = [<target_group_vector for “target group 1”> ], delay_value = T1, node_type = “split”, split_type, split_input = [<target_group_vector for “target group 1”> ], num_split_outputs = 2, num_tasks_split_output_1 = 0, node_type = “end”, end_input = [<target_group_vector for “target group2”> ], num_tasks_split_output_2 = 1, task_type = “sms_communication”, communication_input = [<target_group_vector for “target group 3”> ],communication_task_vector = [<concatenated task content and context taskvector of “task 2”> ], task_type = “end”, end_input = [<target_group_vectorfor “target group 3”> ]).

[0055] Figure 3 This is a diagram illustrating an example of a process flow 300 for predicting task outcomes (e.g., overall performance) for generating a communication workflow according to some aspects of this disclosure. For example, process flow 300 may be at least partially composed of... Figures 1-2 Any of the components described herein may be used for execution. Additionally, process 300 may be executed to generate output predicting the results of a new communication workflow.

[0056] Process flow 300 may begin with user interaction with an interface provided by cloud-based application 120 to create a new communication workflow 310. As an illustrative example, the new communication workflow 310 may include an ordered sequence of tasks 1 through 5, where task 1 is the initiating task that initiates the workflow, task 2 is a split task subgroup that divides a target group of user devices into two subgroups, task 3 may be a communication task that sends an email to the subgroup of user devices associated with task 3, task 4 may be the ending node that terminates the workflow, task 5 may be a communication task that sends a text message to the subgroup of other user devices associated with task 5, and task 6 may be the ending node that terminates the workflow. Cloud-based application 120 may use workflow structure generator 210 to generate the structure of the new communication workflow 310.

[0057] The new communication workflow 310 can be input into the workflow performance predictor 240 to generate predicted performance values. The predicted performance values ​​can represent the predicted task outcome of the new communication workflow 310, such as the percentage of users who perform a target action in response to receiving communication triggered by a task (such as task 3 or 5) included in the workflow. The workflow performance predictor 240 can receive a composite feature vector of the new communication workflow 310 generated by the feature vector generator 220 and input the composite feature vector into a trained machine learning model stored at the machine learning model 260. In response to receiving the composite feature vector of the new communication workflow 310, the trained machine learning model can generate an output of the predicted performance of the new communication workflow 310.

[0058] Figure 4 This is a diagram illustrating an example of a process flow 400 for generating recommended tasks included in a partial communication workflow, according to some aspects of this disclosure. For example, process flow 400 may be at least partially derived from... Figures 1-2 Any of the components described herein may be used for execution. Additionally, process flow 400 may be executed to generate recommendations for one or more tasks to be included in a portion of the communication workflow.

[0059] Process flow 400 may begin with a user interacting with an interface provided by the cloud-based application 120 to create a new partial workflow 410. As an illustrative example, the new partial workflow 410 may include an ordered sequence of tasks 1 through 3, where task 1 is the initiating task that initiates the workflow, task 2 is a splitting task that divides a target group of user devices into two subgroups, and task 3 may be a communication task that sends emails to the subgroup of user devices associated with task 3. The cloud-based application 120 may use a workflow structure generator 210 to generate the structure of the new partial workflow 410.

[0060] The new partial workflow 410 can be input into the partial workflow predictor 250 to generate recommendations for one or more tasks to complete the new partial workflow 410. The partial workflow predictor 250 can receive a composite feature vector of the new partial workflow 410 generated by the feature vector generator 220. The partial workflow predictor 250 can identify a set of previously executed communication workflows with the same structure as the new partial workflow 410, stored in the training dataset 270. The partial workflow predictor 250 can then use the aforementioned similarity detection technique to determine one or more previously executed communication workflows that are similar to the composite feature vector of the new partial workflow 410. The partial workflow predictor 250 can rank the previously executed communication workflows that are determined to be similar in performance. For example, the best-performing part of the previously executed communication workflow can be selected. The remaining tasks of that selected part of the previously executed communication workflow can be recommended to complete the new partial workflow 410. As an example, tasks 4 and 5 can be recommended to be included in the new partial workflow 410 to generate the complete workflow 420.

[0061] Figure 5 This is a flowchart illustrating an example of a process 500 for generating task results for predicting communication workflows according to some aspects of this disclosure. In some embodiments, process 500 may be... Figure 1-2 The process 500 can be executed by any of the components described herein. For example, process 500 can be executed by cloud-based application 120 and any corresponding sub-components. Alternatively, as a non-limiting example, process 500 can be executed to generate predictions of task outcomes representing predictions of the communication workflow performed on a target group of user devices using machine learning techniques.

[0062] For example, process 500 may begin at block 510, where cloud-based application 120 can receive or access a communication workflow comprising an ordered sequence of one or more tasks. The ordered sequence of tasks can represent a structure, such as a hierarchical tree structure. Cloud-based application 120 can generate an interface that enables users to create communication workflows by selecting tasks and arranging them in an ordered order.

[0063] At box 520, feature vector generator 220 can evaluate selected tasks and ordered sequences of tasks to generate the composite feature vector representing the communication workflow. For example, feature vector generator 220 can use techniques such as Word2Vec, topic modeling, singular value decomposition (SVD), neural networks for image representation (e.g., in the example where the task is a message including an image), and other suitable techniques to generate vectors to represent each task. Feature vector generator 220 can then generate the composite feature vector based on the combination of vectors generated for each task.

[0064] At box 530, workflow performance predictor 240 can input the composite feature vector of the communication workflow into a trained machine learning model stored at machine learning model 260. In some examples, the trained machine learning model can be a supervised learning model, such as a linear regression model, support vector machine (SVM), logistic regression, Naive Bayes, decision tree model, k-nearest neighbor model, neural network, similarity learning, and other suitable models. Labeled data can be used to train the supervised learning model. For example, each previously executed communication workflow can be labeled with its corresponding performance value (e.g., click-through rate) based on the known task results of previously executed communication workflows. The supervised learning model can learn to classify or predict the performance value of the communication workflow. At box 540, workflow performance predictor 240 can generate an output corresponding to the prediction of the performance value of the communication workflow.

[0065] Figure 6 This is a flowchart illustrating an example of a process 600 for generating recommendations of tasks to be included in a new partial communication workflow, according to some aspects of this disclosure. In some embodiments, process 600 may be... Figures 1-2 Process 600 can be executed by any of the components described herein. For example, process 600 can be executed by cloud-based application 120 and any corresponding sub-components. Alternatively, as a non-limiting example, process 600 can be executed to generate recommendations for one or more tasks to be added to a new partial communication workflow for the user.

[0066] For example, process 600 may begin at block 610, where cloud-based application 120 can generate and facilitate the display of an interface that enables users to define communication workflows. At block 620, a user can navigate the interface to select tasks and arrange the selected tasks in an ordered sequence, representing an incomplete or partial communication workflow. The interface can receive user input corresponding to a partial communication workflow. A partial communication workflow can be associated with one or more parameters for each of the one or more tasks characterizing the partial communication workflow. For example, parameters may include vocabulary for topic modeling, a content image of the digital message, and other suitable characteristics of the task.

[0067] At box 630, workflow structure generator 210 can evaluate a partial communication workflow to identify or generate its structure. For example, this structure could be a hierarchical tree structure including a root node, one or more intermediate nodes, and one or more leaf nodes. The root node can represent the initiating task that initiates the partial communication workflow. Intermediate nodes can be any type of task, such as a splitting task that divides a target group of user devices into two or more subgroups, a filtering task that filters the target group of user devices, a communication task that transmits communication to the target group of user devices, and so on. Leaf nodes can be end nodes that represent the end of the communication workflow. A partial communication workflow may not yet include an end node because the workflow is partial or incomplete. In some examples, the structure of a partial communication workflow can be represented by one or more nodes and / or one or more stages. Each task in one or more tasks of the partial workflow can correspond to a node in one or more nodes or a stage in one or more stages.

[0068] At box 640, feature vector generator 220 can evaluate selected tasks and ordered sequences of tasks to generate composite feature vectors representing portions of the communication workflow. For example, feature vector generator 220 can generate vectors to represent each task using techniques such as Word2Vec, topic modeling, singular value decomposition (SVD), neural networks for image representation (e.g., in the example where the task is a message including an image), and other suitable techniques. Feature vector generator 220 can then generate composite feature vectors based on combinations of vectors generated for each task.

[0069] At box 650, the partial workflow predictor 250 can access a set of previously executed partial workflows from the training dataset 270. Each previously executed partial workflow in the set can be represented by a structure, a composite feature vector, and a performance value (e.g., task result). At box 660, the partial workflow predictor 250 can select a subset of the set of previously executed partial workflows. The selected subset of previously executed partial workflows can share the same structure as the partial workflow. The partial workflow predictor 250 can determine one or more previously executed partial workflows similar to the partial workflow from the subset of previously executed partial workflows. The comparison can be based on the composite feature vector of each previously executed partial workflow in the subset and the composite feature vector of the partial workflow, or the above. Figure 2 Any other similarity determination techniques described herein shall be used to determine similarity.

[0070] At box 670, the partial workflow predictor 250 can generate recommendations for one or more tasks to complete the partial workflow. The one or more recommended tasks can be selected from one or more remaining tasks of one or more previously executed partial workflows that share the same structure as the partial workflow, are determined to be similar to the partial workflow, and have the highest performance value. The selection of recommended tasks can be based on the previous performance values ​​of one or more previously executed partial workflows that share the same structure as the partial workflow and are determined to be similar to the partial workflow.

[0071] Figure 7 This is a flowchart illustrating an example of a process 700 for generating a vector representation of a communication workflow according to some aspects of this disclosure. In some embodiments, process 700 may be performed by... Figures 1-2 The process 700 can be executed by any of the components described herein. For example, process 700 can be executed by cloud-based application 120 and any corresponding sub-components. Alternatively, as a non-limiting example, process 700 can be executed to generate a vector representation of the communication workflow using a graph-based learning model (e.g., Graph2Vec). The graph-based learning model can use a neural network to generate a multi-dimensional vector representation of the structure and tasks of the communication workflow. The vector representation of the communication workflow generated by process 700 can be input into workflow performance predictor 240 and partial workflow predictor 250 to generate performance predictions and task recommendations, respectively.

[0072] For example, process 700 may begin at box 710, where cloud-based application 120 can generate a graph for each previously executed workflow of training dataset 270. Given a graph representing the structure of a communication workflow, workflow representation system 230 can sample multiple rooted subgraphs around different nodes of the graph. At box 720, workflow representation system 230 can use the sampled rooted subgraphs to train an n-dimensional neural network. In some examples, the rooted subgraphs may include a representation of the subgraph where a node has been selected as the root node, and one or more branches originating from that selected node can be learned. Nodes of the tree structure of the communication workflow can be iteratively selected as root nodes of the neural network to learn various branches originating from each root node. At box 730, training the neural network may include learning a subgraph embedding for each node of each previously executed communication workflow of training dataset 270. Furthermore, using a neural network to learn subgraph embeddings can transform the nodes and stages of the tree structure representing the communication workflow into a vector space while preserving the structure and metadata of the tree structure. A graph-based learning model can be used to generate vector representations for each previously executed communication workflow of training dataset 270. The vector representations stored in the training dataset 270 can then be evaluated against the vector representations of the new communication workflow to execute the methods and procedures described in this paper.

[0073] Figure 8 A simplified diagram is depicted for implementing a distributed system 800 of one embodiment. In the illustrated embodiment, the distributed system 800 includes one or more client computing devices 802, 804, 806, and 808, each client computing device executing and operating client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), via one or more networks 810. A server 812 may be communicatively coupled to remote client computing devices 802, 804, 806, and 808 via network 810.

[0074] In various embodiments, server 812 may be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services may be provided as web-based services or cloud services or under a Software as a Service (SaaS) model to users of client computing devices 802, 804, 806, and / or 808. Users operating client computing devices 802, 804, 806, and / or 808 may then use one or more client applications to interact with server 812 to utilize the services provided by these components.

[0075] In the configuration depicted in the figures, software components 818, 820, and 822 of system 800 are shown as implemented on server 812. In other embodiments, one or more components of system 800 and / or the services provided by these components may also be implemented by one or more of client computing devices 802, 804, 806, and / or 808. A user operating the client computing device can then utilize one or more client applications to use the services provided by these components. These components can be implemented in hardware, firmware, software, or a combination thereof. It should be recognized that various different system configurations are possible and may differ from the distributed system 800. The embodiments shown in the figures are therefore an example of a distributed system for implementing the system of the embodiments and are not intended to be limiting.

[0076] Client computing devices 802, 804, 806, and / or 808 can be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software such as Microsoft Windows Mobile® and / or various mobile operating systems (e.g., iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc.), and enabled for the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Client computing devices can be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®. Client computing devices can be workstation computers running any of the various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively or additionally, client computing devices 802, 804, 806, and 808 may be any other electronic device capable of communicating via one or more networks 810, such as thin client computers, internet-enabled gaming systems (e.g., Microsoft Xbox game consoles with or without Kinect® gesture input devices), and / or personal messaging devices.

[0077] Although an exemplary distributed system 800 with four client computing devices is shown, any number of client computing devices can be supported. Other devices, such as devices with sensors, can interact with the server 812.

[0078] One or more networks 810 in the distributed system 800 can be any type of network familiar to those skilled in the art, capable of supporting data communication using any of a variety of commercially available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, etc. By way of example only, one or more networks 810 can be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, etc. One or more networks 810 can be a wide area network and the Internet. It can include virtual networks, including but not limited to Virtual Private Networks (VPNs), intranets, extranets, Public Switched Telephone Networks (PSTN), infrared networks, wireless networks (e.g., networks operating under any protocol compliant with the IEEE 802.11 protocol suite, Bluetooth®, and / or any other wireless protocol); and / or any combination of these networks and / or other networks.

[0079] Server 812 may comprise one or more general-purpose computers, special-purpose server computers (as examples, including PC (personal computer) servers, UNIX® servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. In various embodiments, server 812 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, server 812 may correspond to a server used to perform the processes described above according to embodiments of this disclosure.

[0080] Server 812 can run any of the operating systems discussed above, as well as any commercially available server operating system. Server 812 can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA® servers, database servers, etc. Exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM, etc.

[0081] In some implementations, server 812 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 802, 804, 806, and 808. As an example, data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party information sources and continuous data streams. Real-time updates may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and the like. Server 812 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 802, 804, 806, and 808.

[0082] The distributed system 800 may also include one or more databases 814 and 816. Databases 814 and 816 may reside in various locations. As an example, one or more of databases 814 and 816 may reside on non-transitory storage media local to server 812 (and / or within server 812). Alternatively, databases 814 and 816 may be located remotely from server 812 and communicate with server 812 via a network-based connection or a dedicated connection. In one set of examples, databases 814 and 816 may reside in a storage area network (SAN). Similarly, any necessary files for performing the functions of server 812 may be appropriately stored locally on server 812 and / or remotely. In one set of embodiments, databases 814 and 816 may include relational databases suitable for storing, updating, and retrieving data in response to commands in SQL format, such as databases provided by Oracle.

[0083] Figure 9 This is a simplified block diagram of a system environment 900, according to some embodiments, whereby services provided by one or more components of the system of embodiments can be provided as cloud services. In the illustrated embodiments, system environment 900 includes one or more client computing devices 904, 906, and 908 that can be used by a user to interact with a cloud infrastructure system 902 providing cloud services. The client computing devices may be configured to operate client applications, such as web browsers, proprietary client applications (e.g., Oracle Forms), or some other application, which can be used by the user of the client computing devices to interact with the cloud infrastructure system 902 to use the services provided by the cloud infrastructure system 902.

[0084] It should be recognized that the cloud infrastructure system 902 depicted in the figures may have other components different from those depicted. Furthermore, the embodiment shown in the figures is merely one example of a cloud infrastructure system that can be incorporated into embodiments of the present invention. In some other embodiments, the cloud infrastructure system 902 may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or arrangements of components.

[0085] Client computing devices 904, 906, and 908 can be devices similar to those described above for 802, 804, 806, and 808.

[0086] While the exemplary system environment 900 is shown with three client computing devices, any number of client computing devices can be supported. Other devices, such as devices with sensors, can interact with the cloud infrastructure system 902.

[0087] One or more networks 910 can facilitate communication and exchange of data between clients 904, 906, and 908 and cloud infrastructure system 902. Each network can be any type of network familiar to those skilled in the art that supports data communication using any of a variety of commercially available protocols, including those described above for one or more networks 810.

[0088] The cloud infrastructure system 902 may include one or more computers and / or servers, which may include those computers and / or servers described above for server 812.

[0089] In some embodiments, services provided by a cloud infrastructure system may include a variety of services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. Services provided by a cloud infrastructure system can be dynamically scaled to meet the needs of users of the cloud infrastructure system. A specific instantiation of a service provided by a cloud infrastructure system is referred to herein as a "service instance." Generally, any service available to users from a cloud service provider's system via a communication network (such as the Internet) is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that constitute the cloud service provider's system differ from the customer's own on-premises servers and systems. For example, a cloud service provider's system may host applications, and users may subscribe to and use these applications on demand via a communication network such as the Internet.

[0090] In some examples, services within a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided to users by the cloud provider, or as otherwise known in the art. For example, services may include password-protected access to remote storage devices in the cloud via the Internet. As another example, services may include web-based hosted relational databases and scripting language middleware engines for private use by networked developers. As yet another example, services may include access to email software applications hosted on a cloud provider's website.

[0091] In some embodiments, cloud infrastructure system 902 may include a suite of application, middleware, and database service providers delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle public cloud provided by this assignee.

[0092] In various embodiments, cloud infrastructure system 902 can be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 902. Cloud infrastructure system 902 can provide cloud services via different deployment models. For example, services can be provided based on a public cloud model, where cloud infrastructure system 902 is owned by an organization selling cloud services (e.g., owned by Oracle), and the services are available to the general public or businesses in different industries. As another example, services can be provided based on a private cloud model, where cloud infrastructure system 902 operates only for a single organization and can provide services to one or more entities within that organization. Cloud services can also be provided based on a community cloud model, where cloud infrastructure system 902 and the services provided by cloud infrastructure system 902 are shared by several organizations in the relevant community. Cloud services can also be provided based on a hybrid cloud model, which is a combination of two or more different models.

[0093] In some embodiments, the services provided by cloud infrastructure system 802 may include one or more services offered under the Software as a Service (SaaS) category, Platform as a Service (PaaS) category, Infrastructure as a Service (IaaS) category, or other service categories that include hybrid services. A customer may subscribe to one or more services provided by cloud infrastructure system 902 via a subscription order. Cloud infrastructure system 902 then performs processing to deliver the services in the customer's subscription order.

[0094] In some embodiments, the services provided by the cloud infrastructure system 902 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, the SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, customers can leverage applications running on the cloud infrastructure system. Customers can obtain application services without having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services providing solutions for sales performance management, enterprise integration, and flexibility for large organizations.

[0095] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform can be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared, public architecture and to leverage the shared services provided by the platform to build new applications. The PaaS platform can manage and control the underlying software and infrastructure used to provide PaaS services. Customers can access PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0096] By leveraging services provided by a PaaS platform, customers can employ programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Service), and Java cloud services. In one embodiment, the database cloud service may support a shared services deployment model that enables organizations to aggregate database resources and provide database-as-a-service to customers in the form of a database cloud. The middleware cloud service provides customers with a platform for developing and deploying various cloud applications, and the Java cloud service provides customers with a platform for deploying Java applications within the cloud infrastructure system.

[0097] Various infrastructure services can be provided by IaaS platforms within cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources, such as storage devices, networks, and other basic computing resources, so that customers can utilize services provided by SaaS and PaaS platforms.

[0098] In some embodiments, the cloud infrastructure system 902 may also include infrastructure resources 930 for providing resources to customers of the cloud infrastructure system for delivering various services. In one embodiment, infrastructure resources 930 may include a combination of pre-integrated and optimized hardware, such as servers, storage devices, and networking resources, to perform services provided by PaaS and SaaS platforms.

[0099] In some embodiments, resources in cloud infrastructure system 902 can be shared by multiple users and dynamically reallocated as needed. Furthermore, resources can be allocated to users in different time zones. For example, cloud infrastructure system 930 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours, and then enable the same resources to be reallocated to another group of users located in a different time zone, thereby maximizing resource utilization.

[0100] In some embodiments, multiple internal shared services 932 may be provided, shared by different components or modules of the cloud infrastructure system 902 and by services provided by the cloud infrastructure system 902. These internal shared services may include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, etc.

[0101] In some embodiments, cloud infrastructure system 902 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, cloud management functionality may include the ability to provision, manage, and track customer subscriptions received by cloud infrastructure system 902.

[0102] In one embodiment, as depicted in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 920, an order orchestration module 922, an order supply module 924, an order management and monitoring module 926, and an identity management module 928. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0103] In exemplary operation 934, a customer using a client device (such as client device 904, 906, or 908) can interact with cloud infrastructure system 902 by requesting one or more services provided by cloud infrastructure system 902 and placing an order to subscribe to one or more services offered by cloud infrastructure system 902. In some examples, the customer can access cloud user interface (UI), cloud UI 912, cloud UI 914, and / or cloud UI 916 and place subscription orders through these UIs. Order information received by cloud infrastructure system 902 in response to a customer placing an order may include information identifying the customer and the one or more services offered by cloud infrastructure system 902 that the customer wishes to subscribe to.

[0104] After a customer places an order, the order information is received via the cloud UI, 912, 914 and / or 916.

[0105] At operation 936, orders are stored in order database 918. Order database 918 can be one of several databases operated by cloud infrastructure system 918 and operating in conjunction with other system components.

[0106] At operation 938, the order information is forwarded to the order management module 920. In some cases, the order management module 920 can be configured to perform order-related billing and accounting functions, such as verifying the order and, after verification, reserving the order.

[0107] At operation 940, order information is transmitted to order orchestration module 922. Order orchestration module 922 can use the order information to orchestrate services and supply resources for customer orders. In some cases, order orchestration module 922 can use the services of order supply module 924 to orchestrate resource supply to support subscribed services.

[0108] In some embodiments, the order orchestration module 922 enables the management of processes associated with each order and applies logic to determine whether an order should be provisioned. At operation 942, upon receiving a new subscription order, the order orchestration module 922 sends a request to the order provisioning module 924 to allocate resources and configure those resources required to fulfill the subscription order. The order provisioning module 924 enables the allocation of resources for services subscribed by customers. The order provisioning module 924 provides an abstraction layer between the cloud services provided by the cloud infrastructure system 900 and the physical implementation layer for provisioning the resources used to provide the requested services. Therefore, the order orchestration module 922 can be isolated from implementation details, such as whether services and resources are actually provisioned on demand or pre-provisioned and allocated / assigned only upon request.

[0109] At operation 944, once services and resources are supplied, notifications of the services provided can be sent to customers on client devices 904, 906, and / or 908 via the order supply module 924 of the cloud infrastructure system 902.

[0110] At operation 946, the order management and monitoring module 926 can manage and track customer subscription orders. In some cases, the order management and monitoring module 926 can be configured to collect service usage statistics from subscription orders, such as storage usage, data transfer volume, number of users, system uptime, and system downtime.

[0111] In some embodiments, the cloud infrastructure system 900 may include an identity management module 928. The identity management module 928 may be configured to provide identity services, such as access management and authorization services within the cloud infrastructure system 900. In some embodiments, the identity management module 928 may control information about customers who wish to utilize the services provided by the cloud infrastructure system 902. Such information may include information authenticating the identities of these customers and information describing what actions these customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 928 may also include management of descriptive information about each customer and how and by whom this descriptive information is accessed and modified.

[0112] Figure 10 An exemplary computer system 1000, in which various embodiments can be implemented, is illustrated. System 1000 can be used to implement any of the computer systems described above. As shown, computer system 1000 includes a processing unit 1004 that communicates with a plurality of peripheral subsystems via a bus subsystem 1002. These peripheral subsystems may include a processing acceleration unit 1006, an I / O subsystem 1008, a storage subsystem 1018, and a communication subsystem 1024. Storage subsystem 1018 includes a tangible computer-readable storage medium 1022 and system memory 1010.

[0113] Bus subsystem 1002 provides a mechanism for enabling various components and subsystems of computer system 1000 to communicate with each other as intended. While bus subsystem 1002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1002 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any architecture across various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0114] A processing unit 1004, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1000. One or more processors may be included in the processing unit 1004. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1004 may be implemented as one or more independent processing units 1032 and / or 1034, each including a single-core or multi-core processor. In other embodiments, the processing unit 1004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0115] In various embodiments, processing unit 1004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1004 and / or storage subsystem 1018. With appropriate programming, processor(s) 1004 can provide the various functions described above. Computer system 1000 may additionally include processing acceleration unit 1006, which may include digital signal processor (DSP), dedicated processor, etc.

[0116] I / O subsystem 1008 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox® 360 game controller using a natural user interface with gestures and voice commands. User interface input devices may also include eye posture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye posture into input in an input device (e.g., Google Glass®). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0117] User interface input devices may also include, but are not limited to: 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0118] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is used to encompass all possible types of devices and mechanisms for outputting information from computer system 1000 to a user or other computers. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0119] Computer system 1000 may include a storage subsystem 1018 containing software elements, shown as currently located within system memory 1010. System memory 1010 may store loadable and executable program instructions on processing unit 1004, as well as data generated during the execution of these programs.

[0120] Depending on the configuration and type of the computer system 1000, the system memory 1010 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by the processing unit 1004 and / or are currently being operated and executed by the processing unit 1004. In some embodiments, the system memory 1010 may include various different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some embodiments, such as during startup, a basic input / output system (BIOS) containing basic routines that facilitate the transfer of information between elements within the computer system 1000 may typically be stored in ROM. By way of example, but not limitation, the system memory 1010 also includes application programs 1012, program data 1014, and an operating system 1016, which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc. As an example, operating system 1016 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS and Palm® OS.

[0121] The storage subsystem 1018 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by a processor may be stored in the storage subsystem 1018. These software modules or instructions may be executed by the processing unit 1004. The storage subsystem 1018 may also provide a repository for storing data used according to the present invention.

[0122] Storage subsystem 1018 may also include a computer-readable storage medium reader 1020 that can be further connected to computer-readable storage medium 1022. Together with and optionally in conjunction with system memory 1010, computer-readable storage medium 1022 can generally represent a remote, local, fixed, and / or removable storage device plus storage medium for temporarily and / or more persistently containing, storing, transmitting, and retrieving computer-readable information.

[0123] The computer-readable storage medium 1022 containing code or a portion thereof may also include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media such as data signals, data transmissions, or any other medium that can be used to transmit desired information and can be accessed by the computing system 1000.

[0124] As an example, computer-readable storage medium 1022 may include a hard disk drive that reads or writes from a non-removable non-volatile magnetic medium to a non-removable non-volatile magnetic medium, a disk drive that reads or writes from a removable non-volatile disk to a removable non-volatile disk, and an optical disc drive that reads or writes from a removable non-volatile optical disc (such as a CD-ROM, DVD, and Blu-ray® disc or other optical media) to a removable non-volatile optical disc. Computer-readable storage medium 1022 may include, but is not limited to: Zip® drives, flash memory cards, Universal Serial Bus (USB) flash memory drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, etc. Computer-readable storage medium 1022 may also include: solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs), and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data for computer system 1000.

[0125] Communication subsystem 1024 provides interfaces to other computer systems and networks. Communication subsystem 1024 serves as an interface for receiving data from other systems and sending data from computer system 1000 to other systems. For example, communication subsystem 1024 enables computer system 1000 to connect to one or more devices via the Internet. In some embodiments, communication subsystem 1024 may include radio frequency (RF) transceiver components (e.g., using cellular phone technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 1202.11 series standards), or other mobile communication technologies, or any combination thereof), GPS receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, communication subsystem 1024 may provide wired network connectivity (e.g., Ethernet).

[0126] In some embodiments, the communication subsystem 1024 may also represent one or more users who can use the computer system 1000 to receive input communications in the form of structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc.

[0127] As an example, the communication subsystem 1024 can be configured to receive data feeds 1026 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0128] Furthermore, the communication subsystem 1024 can also be configured to receive data in the form of a continuous data stream, which may include an event stream 1028 and / or event updates 1030 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial price reporting systems, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, etc.

[0129] The communication subsystem 1024 can also be configured to output structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1000.

[0130] The computer system 1000 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0131] Due to the constantly evolving nature of computers and networks, the description of the computer system 1000 depicted in the figures is intended only as a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented in hardware, firmware, software (including applets), or combinations thereof. Additionally, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0132] In the foregoing description, various aspects of the invention have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects of the invention described above can be used individually or in combination. Furthermore, embodiments can be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A computer-implemented method, comprising: An interface is provided that enables users to define communication workflows, which can be configured to include a set of tasks that facilitate interaction with a set of user devices, and each task in the set of tasks includes executable code that, when executed, performs a function associated with the set of user devices. The interface receives input corresponding to selecting one or more tasks from the set of tasks, the one or more tasks defining a partial communication workflow, which is associated with one or more parameters of each of the one or more tasks characterizing the partial communication workflow; Identify the structure of the partial communication workflow, the structure of which is represented by one or more nodes and / or one or more stages, wherein each task in the one or more tasks of the partial communication workflow corresponds to a node in the one or more nodes or a stage in the one or more stages; Generate a composite feature vector representing the portion of the communication workflow, the composite feature vector being generated using the feature vector of each of the one or more tasks in the communication workflow, and the feature vector of each of the one or more tasks being generated by performing one or more machine learning techniques using the one or more parameters characterizing the task; Access a set of previously executed partial communication workflows, each of which is represented by a structure, a composite feature vector, and a task result; Select a subset of the previously executed partial communication workflows, which shares the same structure as the partial communication workflows; One or more previously executed partial communication workflows similar to the previously executed partial communication workflows are determined from a subset of the previously executed partial communication workflows, and the similarity is based on a comparison between the composite feature vector of each previously executed partial communication workflow in the subset and the composite feature vector of the partial communication workflow. Generate output for completing the partial communication workflow, the output including one or more recommended tasks for completing the partial communication workflow, the one or more recommended tasks being selected from one or more remaining tasks of the one or more previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow, and the selection is based on the task results of the one or more previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow; as well as The output is displayed on the interface.

2. The computer-implemented method of claim 1, wherein comparing the composite feature vector of each previously executed partial communication workflow from a subset of the previously executed partial communication workflows that share the same structure as the partial communication workflow with the composite feature vector of the partial communication workflow comprises: In the domain space, calculate the distance between the composite feature vector of the previously executed partial communication workflow and the composite feature vector of the partial communication workflow. as well as The distance is compared with a threshold, wherein when the distance is equal to or less than the threshold, it is determined that the previously executed partial communication workflow is similar to the partial communication workflow, and wherein when the distance is greater than the threshold, it is determined that the previously executed partial communication workflow is not similar to the partial communication workflow.

3. The computer-implemented method as described in claim 1, further comprising: The one or more previously executed partial communication workflows are sorted by arranging them from highest to lowest task result, sharing the same structure as the partial communication workflows and being identified as similar to the partial communication workflows.

4. The computer-implemented method as described in claim 1, further comprising: At the interface, additional input is received, which corresponds to selecting one or more of the recommended tasks to complete the portion of the communication workflow. as well as In response to receiving the additional input, the one or more recommended tasks are added sequentially to the one or more tasks to represent the complete communication workflow.

5. The computer-implemented method as described in claim 1, further comprising: Additional input is received at the interface, which corresponds to selecting at least one recommended task from the one or more recommended tasks and at least one task not included in the one or more recommended tasks. Selecting at least one recommended task and at least one task not included in the one or more recommended tasks completes the partial communication workflow. as well as In response to receiving the additional input, the at least one recommended task and the at least one task not included in the one or more recommended tasks are added sequentially to the one or more tasks to represent a complete communication workflow.

6. The computer-implemented method of claim 1, wherein part or all of the communication workflow is defined by a list including metadata, the metadata describing: The structure of the communication workflow; Each task included in the communication workflow; and Each feature vector represents a task in the set of tasks included in the communication workflow.

7. The computer-implemented method as described in claim 6, further comprising: The list of metadata is parsed to perform one or more functions.

8. A system for processing communication workflows using machine learning techniques, the system comprising: One or more processors; as well as A non-transitory computer-readable storage medium containing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations including: An interface is provided that enables users to define communication workflows, which can be configured to include a set of tasks that facilitate interaction with a set of user devices, and each task in the set of tasks includes executable code that, when executed, performs a function associated with the set of user devices. The interface receives input corresponding to selecting one or more tasks from the set of tasks, the one or more tasks defining a partial communication workflow, which is associated with one or more parameters of each of the one or more tasks characterizing the partial communication workflow; Identify the structure of the partial communication workflow, the structure of which is represented by one or more nodes and / or one or more stages, wherein each task in the one or more tasks of the partial communication workflow corresponds to a node in the one or more nodes or a stage in the one or more stages; Generate a composite feature vector representing the portion of the communication workflow, the composite feature vector being generated using the feature vector of each of the one or more tasks in the communication workflow, and the feature vector of each of the one or more tasks being generated by performing one or more machine learning techniques using the one or more parameters characterizing the task; Access a set of previously executed partial communication workflows, each of which is represented by a structure, a composite feature vector, and a task result; Select a subset of the previously executed partial communication workflows, which shares the same structure as the partial communication workflows; One or more previously executed partial communication workflows similar to the previously executed partial communication workflows are determined from a subset of the previously executed partial communication workflows, and the similarity is based on a comparison between the composite feature vector of each previously executed partial communication workflow in the subset and the composite feature vector of the partial communication workflow. Generate output for completing the partial communication workflow, the output including one or more recommended tasks for completing the partial communication workflow, the one or more recommended tasks being selected from one or more remaining tasks of previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow, and the selection is based on the task results of the one or more previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow; and The output is displayed on the interface.

9. The system of claim 8, wherein comparing the composite feature vector of each previously executed partial communication workflow from a subset of the previously executed partial communication workflows that share the same structure as the partial communication workflow with the composite feature vector of the partial communication workflow comprises: In the domain space, calculate the distance between the composite feature vector of the previously executed partial communication workflow and the composite feature vector of the partial communication workflow. as well as The distance is compared with a threshold, wherein when the distance is equal to or less than the threshold, it is determined that the previously executed partial communication workflow is similar to the partial communication workflow, and wherein when the distance is greater than the threshold, it is determined that the previously executed partial communication workflow is not similar to the partial communication workflow.

10. The system of claim 8, wherein the operation further comprises: The one or more previously executed partial communication workflows are sorted by arranging them from highest to lowest task result, sharing the same structure as the partial communication workflows and being identified as similar to the partial communication workflows.

11. The system of claim 8, wherein the operation further comprises: At the interface, additional input is received, which corresponds to selecting one or more of the recommended tasks to complete the portion of the communication workflow. as well as In response to receiving the additional input, the one or more recommended tasks are added sequentially to the one or more tasks to represent the complete communication workflow.

12. The system of claim 8, wherein the operation further comprises: Additional input is received at the interface, which corresponds to selecting at least one recommended task from the one or more recommended tasks and at least one task not included in the one or more recommended tasks. Selecting at least one recommended task and at least one task not included in the one or more recommended tasks completes the partial communication workflow. as well as In response to receiving the additional input, the at least one recommended task and the at least one task not included in the one or more recommended tasks are added sequentially to the one or more tasks to represent a complete communication workflow.

13. The system of claim 8, wherein a portion or all of the communication workflow is defined by a list including metadata, the metadata describing: The structure of the communication workflow; Each task included in the communication workflow; and Each feature vector represents a task in the set of tasks included in the communication workflow.

14. The system of claim 13, wherein the operation further comprises: The list of metadata is parsed to perform one or more functions.

15. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, comprising instructions configured to cause a processing means to perform operations, said operations including: An interface is provided that enables users to define communication workflows, which can be configured to include a set of tasks that facilitate interaction with a set of user devices, and each task in the set of tasks includes executable code that, when executed, performs a function associated with the set of user devices. The interface receives input corresponding to selecting one or more tasks from the set of tasks, the one or more tasks defining a partial communication workflow, which is associated with one or more parameters of each of the one or more tasks characterizing the partial communication workflow; Identify the structure of the partial communication workflow, the structure of which is represented by one or more nodes and / or one or more stages, wherein each task in the one or more tasks of the partial communication workflow corresponds to a node in the one or more nodes or a stage in the one or more stages; Generate a composite feature vector representing the portion of the communication workflow, the composite feature vector being generated using the feature vector of each of the one or more tasks in the communication workflow, and the feature vector of each of the one or more tasks being generated by performing one or more machine learning techniques using the one or more parameters characterizing the task; Access a set of previously executed partial communication workflows, each of which is represented by a structure, a composite feature vector, and a task result; Select a subset of the previously executed partial communication workflows, which shares the same structure as the partial communication workflows; One or more previously executed partial communication workflows similar to the previously executed partial communication workflows are determined from a subset of the previously executed partial communication workflows, and the similarity is based on a comparison between the composite feature vector of each previously executed partial communication workflow in the subset and the composite feature vector of the partial communication workflow. Generate output for completing the partial communication workflow, the output including one or more recommended tasks for completing the partial communication workflow, the one or more recommended tasks being selected from one or more remaining tasks of the one or more previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow, and the selection is based on the task results of the one or more previously executed partial communication workflows that share the same structure as the partial communication workflow and are identified as similar to the partial communication workflow; as well as The output is displayed on the interface.

16. The computer program product of claim 15, wherein comparing the composite feature vector of each previously executed partial communication workflow from a subset of the previously executed partial communication workflows that share the same structure as the partial communication workflow with the composite feature vector of the partial communication workflow comprises: In the domain space, calculate the distance between the composite feature vector of the previously executed partial communication workflow and the composite feature vector of the partial communication workflow. as well as The distance is compared with a threshold, wherein when the distance is equal to or less than the threshold, it is determined that the previously executed partial communication workflow is similar to the partial communication workflow, and wherein when the distance is greater than the threshold, it is determined that the previously executed partial communication workflow is not similar to the partial communication workflow.

17. The computer program product of claim 15, wherein the operation further comprises: The one or more previously executed partial communication workflows are sorted by arranging them from highest to lowest task result, sharing the same structure as the partial communication workflows and being identified as similar to the partial communication workflows.

18. The computer program product of claim 15, wherein the operation further comprises: At the interface, additional input is received, which corresponds to selecting one or more of the recommended tasks to complete the portion of the communication workflow. as well as In response to receiving the additional input, the one or more recommended tasks are added sequentially to the one or more tasks to represent the complete communication workflow.

19. The computer program product of claim 15, wherein the operation further comprises: Additional input is received at the interface, which corresponds to selecting at least one recommended task from the one or more recommended tasks and at least one task not included in the one or more recommended tasks. Selecting at least one recommended task and at least one task not included in the one or more recommended tasks completes the partial communication workflow. as well as In response to receiving the additional input, the at least one recommended task and the at least one task not included in the one or more recommended tasks are added sequentially to the one or more tasks to represent a complete communication workflow.

20. The computer program product of claim 15, wherein part or all of the communication workflow is defined by a list including metadata, the metadata describing: The structure of the communication workflow; Each task included in the communication workflow; and Each feature vector represents a task in the set of tasks included in the communication workflow.

Images