Generate Add-in Application Recipe Extension

Abstract

Techniques for generating plugin application recipe (PIAR) extensions are disclosed. A PIAR management application discovers one or more data values of a specific data type within a specific field of a plugin application, where the specific data type (a) is different from the data type of the specific field reported by the application and (b) is narrower than the data type of the specific field while conforming to the data type of the specific field. The PIAR management application identifies one or more mappings between (a) the specific data type and (b) one or more data types of the fields accepted by the actions of the plugin application. The PIAR management application provides a user interface including one or more candidate PIAR extensions based on the one or more mappings. Based on a user selection of a candidate PIAR extension, the PIAR management application executes a PIAR including the selected PIAR extension.

Description

Technical Field

[0001] The present disclosure relates to plug-in application recipes. Specifically, the present disclosure relates to generating and executing plug-in application recipes. Background Art

[0002] A plug-in application recipe (“PIAR”) is a collection that includes a triggering event (referred to herein as a “trigger” or “trigger condition”) and an action, which are logically arranged as an if-then formula. The “if” part of the formula corresponds to the PIAR trigger condition. The “then” part of the formula is conditional upon the “if” part being satisfied and corresponds to the action that can be triggered. A plug-in application can provide the action. The plug-in application providing the action can be the same as or different from the plug-in application providing the trigger.

[0003] A PIAR management application presents an interface that allows a user to define a PIAR. A PIAR definition indicates one or more actions to be performed by the PIAR management application. The PIAR definition also indicates the trigger of the plug-in application. When the PIAR management application detects that the trigger condition is satisfied, the PIAR performs the action(s) corresponding to the detected trigger.

[0004] The PIAR management application can be used for many purposes. For example, the PIAR management application can be used to automate repetitive tasks. Examples of PIARs include, but are not limited to: (a) opening the user's garage door (action) in response to detecting that the user's car is in the user's lane (trigger); (b) sending a notification to the user (action) in response to determining that the user's walking steps have not reached a specific target by 5:00 p.m. (trigger); (c) creating a new folder to store information about a sales contact (action) in response to detecting a new sales contact in the address book or email (trigger).

[0005] The term "plug-in application" refers to the fact that the trigger(s) and / or action(s) of an application are logically "plugged" into the PIAR management application and thus become part of the logic of the PIAR. For example, the PIAR management application can be organized according to a micro-service architecture such that several independent services are plugged into the PIAR management application. The plugged-in services can provide the monitoring service(s) specific to a particular application to support the trigger(s) for that particular application. Alternatively or additionally, the plugged-in services can provide the action service(s) specific to a particular application to support the execution of the action(s) for that particular application.

[0006] The methods described in this section are methods that can be implemented, but not necessarily methods that have been previously envisioned or implemented. Therefore, unless otherwise stated, no method described in this section should be assumed to be prior art solely because it is included in this section. Brief Description of the Drawings

[0007] Embodiments are illustrated in the figures of the drawings by way of example and not limitation. It should be noted that references to "an" or "one" embodiment in this disclosure do not necessarily refer to the same embodiment, and they mean at least one. In the figures:

[0008] Figure 1 illustrates a plug-in application recipe management system according to one or more embodiments;

[0009] Figure 2 illustrates a set of operations for generating a plug-in application recipe extension according to one or more embodiments;

[0010] Figure 3 illustrates an example of generating a plug-in application recipe extension according to one or more embodiments;

[0011] Figure 4 illustrates a set of operations for generating a plug-in application recipe based on semantic analysis according to one or more embodiments;

[0012] Figure 5 illustrates an example of generating a plug-in application recipe based on semantic analysis according to one or more embodiments;

[0013] Figure 6 illustrates a set of operations for user-assisted execution of a plug-in application recipe according to one or more embodiments;

[0014] Figures 7A - 7DIllustrates an example of executing a plug-in application recipe with user assistance according to one or more embodiments; and

[0015] Figure 8 Shows a block diagram of a computer system illustrating according to one or more embodiments. Detailed Description

[0016] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in different embodiments. In some examples, well-known structures and devices are described in reference to block diagram form in order to avoid unnecessarily obscuring the present invention.

[0017] 1 General Overview

[0018] 2 Architecture Overview

[0019] 2.1 Plug-in Application Recipe Management System

[0020] 2.2 Machine Learning Engine

[0021] 2.3 Semantic Analysis Engine

[0022] 3 Generating Plug-in Application Recipe Extensions

[0023] 3.1 Operations

[0024] 3.2 Illustrative Examples

[0025] 4 Plug-in Application Recipe Generation Based on Semantic Analysis

[0026] 4.1 Operations

[0027] 4.2 Illustrative Examples

[0028] 5 User-Assisted Execution of Plug-in Application Recipes

[0029] 5.1 Operations

[0030] 5.2 Illustrative Examples

[0031] 6 Miscellaneous; Extensions

[0032] 7 Hardware Overview

[0033] 8 Computer Networks and Cloud Networks

[0034] 1 General Overview

[0035] One or more embodiments include a Plug-in Application Recipe (PIAR) management system configured to generate and / or execute a PIAR using machine learning, semantic analysis, and / or other techniques to eliminate or minimize user input.

[0036] In an embodiment, the system is configured to generate a Plug-in Application Recipe (PIAR) extension. The PIAR management application discovers a specific data type within one or more data values of a specific field of the plug-in application, where the specific data type (a) is different from the data type of the specific field reported by the plug-in application and (b) is narrower than the data type of the specific field while conforming to the data type of the specific field. The PIAR management application identifies one or more mappings between (a) the specific data type and (b) one or more data types of the fields accepted by the actions of the plug-in application. The PIAR management application presents a user interface including one or more candidate PIAR extensions based on the (one or more) mappings. Based on the user's selection of a candidate PIAR extension, the PIAR management application executes a PIAR including the selected PIAR extension.

[0037] In an embodiment, the system is configured to generate a PIAR based on semantic analysis. In response to receiving a data item specifying (a) a desired genus of action and / or (b) a desired genus of trigger, the PIAR management application performs semantic analysis on the data item to identify one or more candidate PIARs. The candidate (one or more) PIARs are identified at least in part based on a mapping of actions and / or triggers to the desired genus of action and / or the desired genus of trigger. The mapping is at least in part based on metadata associated with a profile of the plug-in application corresponding to the actions and / or triggers. The PIAR management application stores, for each plug-in application, a corresponding profile for defining the plug-in application for use by one or more future PIARs. Based on user input approving a specific PIAR among the one or more candidate PIARs, the PIAR management application executes the specific PIAR.

[0038] In an embodiment, the system is configured to perform user-assisted execution of a Plug-in Application Recipe (PIAR). During the execution of the PIAR, the PIAR management application applies one or more data values of the plug-in application fields to a machine learning model to obtain:

[0039] (a) A candidate mapping between one or more sub - values found within the (one or more) data values and another field acceptable to the actions of another plug - in application, where the data type of the (one or more) data values is different from the reported data type of the other field, and (b) A confidence metric associated with the candidate mapping, which is at least partially based on whether the (one or more) sub - values are format - compatible with one or more stored formats mapped to the other data type. Based on determining that the confidence metric does not meet a threshold confidence criterion, the PIAR management application obtains user input for confirming or rejecting the candidate mapping and applies the user input to the execution of the PIAR.

[0040] One or more embodiments described in this specification and / or recited in the claims may not be included in this general overview section.

[0041] 2 Architecture Overview

[0042] 2.1 Plug - in Application Recipe Management System

[0043] Figure 1 Illustrated is a plug - in application recipe (PIAR) management system 100 according to one or more embodiments. As shown, the PIAR management system 100 includes a user interface 102, a PIAR management application 104, a data repository 112, one or more monitoring services 118, one or more execution services 120, one or more plug - in applications 122, and its various components. In one or more embodiments, the system 100 may include more or fewer components than Figure 1 the components shown. Figure 1 The components shown may be local or may be remote from each other. Figure 1 The components shown may be implemented in software and / or hardware. Each component may be distributed across multiple applications and / or machines. Multiple components may be combined into one application and / or machine. Operations described with respect to one component may alternatively be performed by another component. Additional embodiments and / or examples related to computer networks are described below.

[0044] In an embodiment, the PIAR management application 104 refers to hardware and / or software configured to perform the operations described herein for generating PIAR extensions. Alternatively or additionally, the PIAR management application 104 may be configured to perform the operations described herein for generating PIARs based on semantic analysis. Alternatively or additionally, the PIAR management application 104 may be configured to perform the operations described herein for user - assisted execution of PIARs. Examples of operations for generating PIAR extensions, semantic - analysis - based PIAR generation, and user - assisted PIAR execution are described below.

[0045] In an embodiment, the PIAR management system 100 includes one or more plug-in applications 122. The plug-in applications 122 are applications that are linked or "plugged into" the PIAR management application 104. Each PIAR is defined by reference to one or more plug-in applications 122. Specifically, the plug-in applications 122 include one or more trigger applications 124 and one or more action applications 126. The trigger applications 124 and the action applications 126 may be the same application or different applications. A particular plug-in application 122 may expose one or more triggers and also expose an API for performing an action. The same plug-in application 122 may be used as both a trigger application 124 and an action application 126 in the same PIAR or different PIARs. The PIAR management application 104 itself may be a trigger application 124 and / or an action application 126.

[0046] In an embodiment, one or more plug-in applications 122 are the following applications: (a) execute independently of the trigger detection process of the PIAR management application 104 (described in further detail below); (b) are not controlled by the trigger detection process of the PIAR management application 104; and / or (c) are remote from the PIAR management application 104. Thus, a plug-in application 122 that executes independently of the trigger detection process of the PIAR management application 104 may be referred to herein as an independently executing application.

[0047] In an embodiment, a particular plug-in application 122 is a "stand-alone" application relative to the PIAR management application 104. That is, the plug-in application 122 may provide its own user interface (e.g., a graphical user interface) through which a user accesses all of the functions of the plug-in application 122. The plug-in application 122 may provide functions that are completely independent of the PIAR management application 104 and do not depend on the PIAR management application 104 to function. The stand-alone plug-in application 122 is not a module or component of the PIAR management application 104. The one or more plug-in applications 122 and the PIAR management application 104 may be developed and supported by completely different business entities. As an example, the plug-in applications 122 may include a search application, an email application, and an audio player application, each of which is a stand-alone application that executes independently. Many different kinds of plug-in applications 122 may be used, and these examples should not be construed as limiting one or more embodiments.

[0048] In an embodiment, the trigger application 124 provides data that the PIAR management application 104 uses to determine whether a trigger condition is met. The trigger application 124 can provide the data to the PIAR management application 104 as a data stream and / or discrete data items. The trigger application 124 can use a "push" model to provide data to the PIAR management application 104, in which the trigger application 124 submits the data to the API of the PIAR management application 104. Alternatively or additionally, the trigger application 124 can use a "pull" model to provide data to the PIAR management application 104, in which the PIAR management application 104 queries the API of the trigger application 124 for data. Alternatively or additionally, the PIAR management application 104 can maintain a persistent connection to the trigger application 124 and continuously receive data from the trigger application 124 via the connection. The trigger application 124 can provide data to the PIAR management application 104 in many different ways and / or combinations thereof. In an embodiment, the trigger application 124 provides multiple data streams and / or data items corresponding to different data fields of the trigger application 124. As an example, an email application can provide data corresponding to the sender, recipient, subject, and email body fields. The trigger condition can be based on the (one or more) data values corresponding to one or more fields provided by one or more trigger applications 124. The PIAR management application 104 can transform, combine, and / or otherwise modify the data provided by one or more trigger applications 124 to determine whether the trigger condition is met.

[0049] In an embodiment, the PIAR management application 104 monitors the trigger application 124 to detect one or more triggers defined with respect to the trigger application 124. The PIAR management application 104 can use the monitoring service 118 to monitor the trigger application 124, and the monitoring service 118 reports to the PIAR management application 104 when a trigger is met. The monitoring service 118 can be implemented separately from the trigger application 124 or as a component (e.g., an installable module or plugin) of the trigger application 124. The monitoring service 118 can be configured to monitor a single trigger application 124. Alternatively or additionally, the monitoring service 118 can be configured to monitor multiple trigger applications. If a particular action application 126 (described in further detail below) also exposes one or more triggers, the monitoring service 118 can be configured to monitor the action application 126 to detect triggers defined with respect to the action application 126.

[0050] In an embodiment, the trigger application 124 includes an application-specific monitoring system that exposes (e.g., via a discovery mechanism implemented as a REST API) one or more predefined trigger conditions having predefined fields. One or more predefined fields associated with the one or more predefined trigger conditions may be customizable. The application-specific monitoring system may allow a limited amount of customization of the application-specific triggers, i.e., allow customization in some ways but not in other ways. In one example, the trigger application 124 is an email application. A trigger defined by reference to the email application is satisfied when the email application performs a specific keyword search. In this example, the application-specific trigger is customizable with respect to the trigger keyword, but not with respect to whether the keyword is within a specified number of words of another keyword or whether information about earlier emails having the keyword has been detected and stored in a table.

[0051] In one example, the trigger application 124 is a search application, and the trigger is defined as the search application performing a search for a recipient. In this example, the monitoring service 118 monitors the search application and notifies the PIAR management application 104 when and if the search application performs a search for a recipient. In another example, the trigger application 124 is an automotive sales application. A potential customer can search for a car from an advertised car database and request a meeting with a sales representative to purchase the car. The trigger condition can be defined as receiving a sales meeting request from the user that mentions an advertised car.

[0052] In an embodiment, the user interface 102 is configured to receive user input specifying one or more triggers that do not correspond natively to functions or data exposed by the trigger application 124. For example, the trigger application 124 may expose data values that change over time without natively exposing any functions for monitoring or comparing the data values as they change. Nevertheless, a trigger can still be defined to monitor the trigger application 124 for changes in the data values over time. As another example, the trigger may include conditions based on aggregated data values (e.g., summed or averaged over time, or summed or averaged over some consecutive data retention or change) or masked data values (e.g., data values indicating the presence or absence of data or changes in data, even if the data itself is masked from the PIAR management application 104 or the end user or is not available to the end user overall). Many different kinds of trigger conditions can be inferred, derived, or otherwise determined from the trigger application 124 without being natively supported by the trigger application 124.

[0053] In an embodiment, the trigger application 124 is not affected in any way by the execution of the PIAR management application 104. The PIAR management application 104 can determine that a trigger is satisfied when a change in one or more fields managed by the independently executing trigger application 124 meets one or more threshold criteria (such as a certain value or aggregate value). The PIAR management application 104 does not change or request a change to the one or more fields managed by the trigger application 124, but instead detects changes resulting from the independent operation of the trigger application 124. One or more threshold criteria specified for the PIAR management application 104 can be different from any threshold criteria supported by the one or more built-in monitoring processes (if any) of the trigger application 124, but such threshold criteria can be based on variables accessible from the trigger application 124, and such variables can change over time.

[0054] In one example, the trigger application 124 is a search application. The search application manages fields for a specified location. In response to a query, the search application displays search results associated with that location. A change in location from one city to another can meet the trigger's threshold criteria.

[0055] In an embodiment, the trigger application 124 (e.g., using the monitoring service 118 and / or an application-specific monitoring system of the trigger application 124) is continuously monitored to identify trigger events. That is, the monitoring process is started and continues to run until a termination condition is met. In response to a number of trigger events identified by the associated monitoring process, a particular PIAR can be evaluated and executed multiple times. In general, after the first instance of a selected trigger event occurs, the PIAR does not stop running. For example, a PIAR that opens a garage door in response to detecting a vehicle in the driveway can run permanently and cause the garage door to open any number of times. The monitoring process associated with a particular PIAR can run permanently until it is manually terminated by a user (e.g., via the user interface 102). Alternatively or additionally, the PIAR definition can include termination conditions such as a certain length of time of running and / or a maximum number of times of performing an action in response to a trigger. Typically, a PIAR is created with the intention that it runs in an execution mode independent of the user, i.e., as an automatic background service that continues to run permanently without further manual intervention.

[0056] In an embodiment, the PIAR management application 104 performs (either directly or by causing the execution of another process) one or more actions in response to detecting that a trigger condition is satisfied. The action application 126 is a plug-in application 122 that is instructed by the PIAR management application 104 to perform an action when the trigger condition is satisfied. Although the action application 126 performs the action, the PIAR management application 104 can also be said to "perform" the action in the sense that the PIAR management application 104 initiates the execution of the action. Similar to the trigger application 124, the action application 126 is linked or inserted into the PIAR management application 104. The action application 126 can be linked to the PIAR management application 110 via the execution service 120. The execution service 120 can be implemented separately from the action application 126 or as a component of the action application 126. According to PIAR, the execution service 120 can access the API corresponding to the action application 126 to use the action application 126 to perform an action.

[0057] In an embodiment, the action application 126 exposes (e.g., via a REST API and / or another interface or a combination thereof) a set of one or more supported actions. The set of supported action(s) can be discovered by the PIAR management application 104. The PIAR management application 104 can send a discovery request to the action application 126 and receive, in response to the discovery request, information describing the set of supported action(s).

[0058] In an embodiment, PIAR can involve multiple actions performed in response to detecting a trigger. Two or more actions can be performed concurrently or in a fixed order. Combinations of concurrent and / or sequential actions can be used. The output of one action can be used as the input of another action. In one example, when an email application receives an email from a prospective customer (e.g., received at an email address typically given to prospective customers), the trigger condition is satisfied. In this example, the series of actions performed in response to receiving the email includes (1) determining whether the prospect is already listed in a customer relationship management (CRM) application, and (2) if the prospect is not already listed in the CRM application, adding the prospect to the CRM application and sending a notification email to the sales manager. Adding the prospect to the CRM application and sending a notification email to the sales manager can be performed concurrently or in a specific order. For example, the notification email can be sent only after receiving confirmation from the CRM application that the prospect was successfully added. PIAR can include many different combinations of concurrent and / or sequential actions.

[0059] In an embodiment, one or more actions in a PIAR can be conditionally executed based on an evaluation of a trigger that initiated the action (or, if the action is part of a series of actions, based on an evaluation of the output from a previous action). For example, a PIAR can include conditional logic (e.g., if, then, else, etc.) that determines a specific order of actions (if any) to be performed in response to detecting a trigger. A PIAR that includes conditional logic can be referred to as a "conditional PIAR" or "branching PIAR". For different outcomes or permutations of evaluating the conditional logic, different branches of the branching PIAR can run as separate PIARs or sub-PIARs. The PIAR management application 104 can be configured to generate a sub-PIAR to handle a particular condition / branch of an existing PIAR in response to detecting that a condition is met. Evaluating a trigger and / or the output of a previous action may involve looping through multiple variable values. For example, a trigger or action can provide an array of values, and evaluating the trigger or action may involve iteratively evaluating (or "looping through") each of these values. Looping through such values themselves may be constrained by conditional logic. Many different combinations of conditional logic and / or looping through variables can be used to determine the specific operation(s) (if any) to be performed when a trigger is detected. The user interface 102 can include controls for defining the loops and / or conditional logic of a particular PIAR.

[0060] In an embodiment, the PIAR management application 104 itself is the action application 126. The PIAR management application 104 can perform actions of generating a new PIAR and / or performing another action or a combination thereof. For example, a PIAR can define a trigger as a customer's first purchase and define corresponding actions for generating a new PIAR for managing that customer's purchase. The new PIAR can define actions such as creating one or more database entries for storing customer purchase records or creating a product recommendation list based on the purchases completed by the customer.

[0061] In an embodiment, the PIAR management application 104 supports the definition of a PIAR involving a series of actions. For example, in response to a specific trigger, the PIAR can execute a specific action, "Action A". The output of Action A (i.e., the data generated by Action A, corresponding to one or more fields of the plug-in application that executes Action A) can then be used as input to another action, "Action B". The output of Action B (i.e., the data generated by Action B, corresponding to one or more fields of the plug-in application that executes Action B) can then be used as input to another action, "Action C", and so on. The PIAR management application 104 can transform the output of an action (e.g., by converting the data to another data type, and / or another transformation), and then use the transformed data as input to another action. The functions and / or data exposed by Action A can be used as input to Action B and / or subsequent actions (such as Action C). The functions and / or data exposed by a later action (such as Action C) may not be available as input to an earlier action (such as Action B or Action A). For example, if Action B is completed before Action C is completed, Action B cannot receive input from Action C.

[0062] In an embodiment, the set of functions of the plug-in application 122 (i.e., the trigger application 124 and / or the action application 126) accessible by the PIAR management application 104 is a subset of all the functions of the plug-in application 122. The plug-in application 122 can expose an application programming interface (API) that provides access to certain functions and / or data of the plug-in application 122, while not providing access to other functions and / or data of the plug-in application 122. The plug-in application 122 can expose functions to the PIAR management application 104 using a Representational State Transfer (REST) API and / or any other kind of API or a combination thereof. The functions exposed by the plug-in application 122 to the PIAR management application 104 can be a proper subset (also known as a strict subset) of all the functions of the plug-in application 122.

[0063] As an example, an email application can expose information about received emails (e.g., email date, time, sender, recipient, subject line, content, etc.) to the PIAR management application 104, without exposing the email sending function to the PIAR management application 104, even though the email application itself includes an email sending function. As another example, a calendar application can expose information about scheduled events (e.g., date, time, location, participants, etc.) and / or the function of scheduling new events to the PIAR management application 104, without exposing the function of canceling events to the PIAR management application 104, even though the calendar application itself includes a function for canceling events.

[0064] In an embodiment, the set of functions exposed by the plug-in application 122 to the PIAR management application 104 depends on one or more authorization policies. The PIAR management application 104 may store and use an authorization token to authenticate access to the API of the plug-in application 122. For example, the PIAR management application 104 may prompt the user via the user interface 102 to enter a username and password to access the API of the plug-in application 122. The PIAR management application 104 may store the username and password in an encrypted token, which it uses to request access to the exposed functions and / or data of the plug-in application 122. In an embodiment, authentication for accessing the plug-in application 122 uses an authorization proxy service. For example, the authentication may use one or more techniques described in U.S. Provisional Patent Application No. 62 / 748,105, entitled "Authorization Proxy Platform," which is incorporated herein by reference.

[0065] In an embodiment, the PIAR management application 104 is configured to generate a PIAR definition based on user input to the user interface 102. Specifically, the user may select one or more triggers and one or more corresponding actions to be performed when the (one or more) triggers are satisfied. As used herein, triggers and actions are collectively referred to as the "operations" of the PIAR. Alternatively or additionally, the user may specify: the name of the PIAR definition; the trigger application 124 used in the PIAR; and / or the action application 126 used in the PIAR. One or more of these operations may involve generating a new PIAR definition. The PIAR definition may also include information other than the (one or more) triggers and the (one or more) actions.

[0066] In an embodiment, the PIAR management application 104 stores the PIAR definition as a PIAR definition object 114 in the data repository 112. Alternatively or additionally, the PIAR management application 104 may store metadata 116 associated with the PIAR definition in the data repository 112 or other storage devices. For example, the metadata 116 may identify the user who created the PIAR definition, the creation time and date, the authorization level of the PIAR definition (e.g., whether (one or more) actions are allowed to receive personally identifiable information), the (one or more) plug-in applications associated with the (one or more) triggers and / or the (one or more) actions, and / or any other type of metadata describing the PIAR or associated therewith, or a combination thereof. If a PIAR definition is recursively generated during the execution of another PIAR, the recursively generated PIAR may be referred to as a "child" PIAR, and the PIAR that generated it may be referred to as the "parent" PIAR. The metadata associated with the child PIAR may include information about the parent PIAR directly in the metadata of the child PIAR and / or by reference to the metadata of the parent PIAR. The PIAR management application 108 may store the metadata 116 within the PIAR definition object 114 (i.e., as one or more of its logical components) or separately from the PIAR definition object 114. In an embodiment, the PIAR management application 104 stores the PIAR definition object 114 in JavaScript Object Notation (JSON) format, where the elements in the JSON structure correspond to one or more triggers, one or more actions, and / or the metadata 116.

[0067] In an embodiment, the data repository 112 is any type of storage unit and / or device for storing data (e.g., a file system, a database, a set of tables, or any other storage mechanism). Additionally, the data repository 112 may include multiple different storage units and / or devices. The multiple different storage units and / or devices may or may not belong to the same type or be located at the same physical site. Further, the data repository 112 may be implemented or executed on the same computing system as one or more other components of the PIAR management system 100. Alternatively or additionally, the data repository 112 may be implemented or executed on a computing system separate from one or more other components of the PIAR management system 100. The data repository 112 may be communicatively coupled to one or more other components of the PIAR management system 100 via a direct connection or via a network. Information describing the PIAR definition object 114 and / or the metadata 116 may be implemented across any component within the PIAR management system 100. However, for clarity and explanation purposes, this information is shown within the data repository 112.

[0068] In an embodiment, the user interface 102 includes one or more controls for modifying a PIAR (i.e., as described below, an existing PIAR and / or a candidate PIAR generated by the PIAR management application 104). The user interface 102 may include one or more controls for overriding the value(s) of one or more exposed fields of the trigger application 124 with one or more user-defined values in a particular PIAR. Alternatively or additionally, the user interface 102 may include one or more controls for overriding the value(s) of one or more expected fields of the action application 126 with one or more user-defined values in a particular PIAR. Alternatively or additionally, the user interface 102 may include one or more controls for designating a field as required (such that a particular instance of the PIAR execution will not complete if the value of the required field is missing). Alternatively or additionally, the user interface 102 may include one or more controls for designating a field as ignored (such that a particular instance of the PIAR execution will not incorporate the value of the ignored field even if the value of the field exists). Alternatively or additionally, the user interface 102 may include one or more controls for designating a field as an optional field (such that a particular instance of the PIAR execution will incorporate the value of the optional field if it exists and will ignore the optional field if it is missing).

[0069] In an embodiment, the user interface 102 includes one or more controls for specifying the value of a field that, if detected in a particular instance of the PIAR execution, is interpreted as an instruction for an action that guides the user-assisted PIAR execution (described in further detail below). For example, the user interface 102 may include controls for defining a particular tag (which may be a user-specific tag or shared by multiple users of the PIAR management application 104) that, if detected in a particular data value of a field provided by the trigger application 124 (e.g., in an email message), indicates to the PIAR management application 104: (a) to suppress prompting the user for input to confirm an action triggered by the PIAR, even if the execution independent of the user in this instance is associated with a low confidence indicator; (b) to prompt the user for input to confirm an action triggered by the PIAR, even if the execution independent of the user in this instance is associated with a high confidence indicator; or (c) to define a threshold confidence indicator in this instance that may deviate from the default confidence indicator required to proceed with the execution independent of the user, and above which the PIAR management application 104 must prompt the user for input to confirm an action triggered by the PIAR.

[0070] In an embodiment, one or more components of the PIAR management system 100 are implemented on one or more digital devices. The term "digital device" generally refers to any hardware device that includes a processor. A digital device can refer to a physical device that executes an application or a virtual machine. Examples of digital devices include computers, tablets, laptops, desktops, netbooks, servers, web servers, network policy servers, proxy servers, general-purpose machines, function-specific hardware devices, hardware routers, hardware switches, hardware firewalls, hardware network address converters (NATs), hardware load balancers, mainframes, televisions, content receivers, set-top boxes, printers, mobile phones, smartphones, personal digital assistants ("PDAs"), wireless receivers and / or transmitters, base stations, communication management devices, routers, switches, controllers, access points, and / or client devices.

[0071] 2.2 Machine Learning Engine

[0072] In an embodiment, the PIAR management application 104 includes a machine learning engine 108. Machine learning includes various techniques in the field of artificial intelligence that process computer-implemented, user-independent processes for solving problems with variable inputs. The PIAR management application 104 can be configured to use the machine learning engine 108 to perform one or more of the operations described herein in a user-independent execution mode.

[0073] In an embodiment, the machine learning engine 108 trains a machine learning model (not shown) to perform one or more operations. Training the machine learning model uses training data to generate a function that, given one or more inputs to the machine learning model, calculates a corresponding output. The output can correspond to a prediction based on previous machine learning. In an embodiment, the output includes a label, classification, and / or categorization assigned to the provided input(s). The machine learning model corresponds to a learned model for performing the desired operation(s) (e.g., labeling, classifying, and / or categorizing an input).

[0074] In an embodiment, the machine learning engine 108 may use supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, and / or another training method or a combination thereof. In supervised learning, the labeled training data includes input / output pairs, where each input is labeled with an expected output (e.g., a label, classification, and / or categorization), which is also referred to as a supervision signal. In semi-supervised learning, some inputs are associated with supervision signals, while other inputs are not. In unsupervised learning, the training data does not include supervision signals. Reinforcement learning uses a feedback system, where the machine learning engine 108 receives positive and / or negative reinforcement during the process of attempting to solve a particular problem (e.g., optimizing performance in a particular scenario according to one or more predefined performance criteria). In an embodiment, the machine learning engine 108 initially uses supervised learning to train a machine learning model and then uses unsupervised learning to update the machine learning model on an ongoing basis.

[0075] In an embodiment, the machine learning engine 108 may use many different techniques to label, classify, and / or categorize inputs. The machine learning engine 108 may transform an input into a feature vector that describes one or more attributes (“features”) of the input. The machine learning engine 108 may label, classify, and / or categorize the input based on these feature vectors. Alternatively or additionally, the machine learning engine 108 may use clustering (also known as cluster analysis) to identify commonalities in the inputs. The machine learning engine 108 may group (i.e., cluster) the inputs based on these commonalities. The machine learning engine 108 may use hierarchical clustering, k-means clustering, and / or another clustering method or a combination thereof. In an embodiment, the machine learning engine 108 includes an artificial neural network. The artificial neural network includes a plurality of nodes (also known as artificial neurons) and edges between the nodes. The edges may be associated with corresponding weights that represent the connection strength between the nodes, and the machine learning engine 108 adjusts these weights as machine learning progresses. Alternatively or additionally, the machine learning engine 108 may include a support vector machine. The support vector machine represents an input as a vector. The machine learning engine 108 may label, classify, and / or categorize the input based on the vector. Alternatively or additionally, the machine learning engine 108 may use a naive Bayes classifier to label, classify, and / or categorize the input. Alternatively or additionally, given a particular input, the machine learning model may apply a decision tree to predict the output for the given input. Alternatively or additionally, in cases where it is impossible or impractical to label, classify, and / or categorize an input within a fixed set of mutually exclusive options, the machine learning engine 108 may apply fuzzy logic. The above machine learning models and techniques are discussed for illustrative purposes only and should not be construed as limiting one or more embodiments.

[0076] In an embodiment, when the machine learning engine 108 applies different inputs to a machine learning model, the corresponding outputs are not always accurate. As an example, the machine learning engine 108 can use supervised learning to train the machine learning model. After training the machine learning model, if a subsequent input is identical to the input included in the labeled training data and the output is identical to the supervisory signal in the training data, the output is surely accurate. If the input is different from the input included in the labeled training data, the machine learning engine 108 may generate a corresponding output that is inaccurate or of uncertain accuracy. In addition to generating a specific output for a given input, the machine learning engine 108 can be configured to generate an indicator of the confidence in the accuracy of the output (or an indicator of the lack of confidence). The confidence indicator can include a numerical score, a boolean value, and / or any other type of metric corresponding to the confidence in the accuracy of the output (or lack of confidence).

[0077] 2.3 Semantic Analysis Engine

[0078] In an embodiment, the PIAR management application 104 includes a semantic analysis engine 108. The PIAR management application 104 can be configured to perform semantic analysis on one or more data items using the semantic analysis engine 110, as described below. Generally, as used herein, semantic analysis refers to programming techniques for determining the meaning associated with words in human language (i.e., individual words, sentences, paragraphs, etc.). As an example, the plug-in application 122 generates an output that includes words in human language (e.g., an email, a web page, a document, a video, an audio file, and / or any other type of output that includes words). Semantic analysis can determine the meaning associated with the output of the plug-in application 122. As another example, a user provides an input that includes words in the form of natural language instructions (e.g., text and / or audio input via the user interface 102). Semantic analysis can use natural language processing to determine the meaning of the natural language instructions. The meaning can correspond to the user's intent, i.e., the expectation of the user expressed for the PIAR management application 104 to perform a specific task. The meaning determined by semantic analysis can correspond to a predicted meaning based on an analysis of previous inputs. There are many different semantic analysis techniques. In an embodiment, the semantic analysis engine 110 uses the machine learning engine 108 to determine meaning by applying the (one or more) input words to a machine learning model. Alternatively or additionally, the semantic analysis engine 100 uses semantic analysis techniques that do not involve machine learning. For example, the semantic analysis engine 100 can use decision tree analysis and / or another form of semantic analysis or a combination thereof to process the input according to a fixed grammar and vocabulary.

[0079] 3. Generating Plug-in Application Recipe Extensions

[0080] 3.1 Operations

[0081] Figure 2 Illustrates an example set of operations for generating a plug-in application recipe extension according to one or more embodiments. Figure 2 One or more of the operations shown therein may be modified, rearranged, or omitted together. Thus, Figure 2 the operations in the specific order shown should not be construed as limiting the scope of one or more embodiments.

[0082] In an embodiment, unless otherwise specified, the operations described below may be performed in an execution mode independent of a user, i.e., as one or more processes performed without user input. Performing operations in an execution mode independent of a user can increase the speed and / or efficiency of the system by reducing or eliminating the need to obtain user input at various operation stages.

[0083] In an embodiment, a system (e.g., Figure 1 the PIAR management system 100) trains a machine learning model (operation 202). The system may train the machine learning model to identify one or more canonical data types in data items provided to the system. As used herein, a canonical data type is a specific data type supported by the system. The system may represent the canonical data type in a specific canonical format. For example, the system may support a "date" data type and store dates in a specific canonical format (e.g., "yyyy - mm - dd"). In this example, the system may train the machine learning model to identify dates in data items provided to the system that are not in the specific canonical format (e.g., dates in unstructured data such as natural language input).

[0084] Alternatively or additionally, the system may train the machine learning model to identify one or more data patterns in data items provided to the system. As used herein, a data pattern is a composite data type that combines two or more other data types that may be logically related to each other. For example, a "meeting" data pattern may combine date, time, location, and / or name data types arranged in a specific recognizable pattern. The data types included in a data pattern may themselves be data patterns. Alternatively or additionally, a data pattern may be based on the semantic content of a data item. For example, in data generated by a calendar application, one data pattern may be associated with a sports event, while another data pattern may be associated with a business lunch. A data pattern may also be a canonical data type of the system. In an embodiment, the system uses supervised learning to train the machine learning model based on a training data set (e.g., existing PIAR definitions or portions thereof) that identifies known data types (which may include one or more data patterns) in unstructured data.

[0085] In an embodiment, the system is configured to identify data types including one or more of the following: dates; email addresses; monetary amounts; physical addresses; user-defined data types (e.g., regular expressions that match user-defined alphanumeric character patterns); standardized data types (e.g., Institute of Electrical and Electronics Engineers (IEEE)-standardized data types, data types that conform to JavaScript Object Notation (JSON) schemas, and / or any other kind of standardized data type or combinations thereof); and / or another data type or combinations thereof. The system can identify data types using machine learning as described above and / or using non-machine learning techniques. For example, the system can store regular expressions that match known data types and apply the data item to the regular expressions without using a machine learning model to determine whether the data item corresponds to the data type.

[0086] In an embodiment, to identify the data type of a particular data item, the system is configured to examine the previous processing sequence associated with the data item. The previous processing sequence can provide insight into the data type of the data item. A PIAR can include a sequence of one or more actions, where the data type of the data item transforms from one action to the next, and the system can be aware of the previous (i.e., “upstream”) (one or more) data types of the data item in the sequence. As an example, a PIAR receives a data item representing a date in the system date format. The PIAR passes the data item to an email application that generates an email with the date in text format in the subject line. The system recognizes that the subject line contains a date because it accesses the “upstream” processing sequence of the PIAR.

[0087] In an embodiment, the system discovers one or more data types within one or more fields of a plug-in application (operation 204). The system can receive one or more data values as output of a trigger application (e.g., before and / or after evaluating a trigger) and / or as output of an action application (e.g., data generated as a result of performing an action). The data types discovered in the one or more data values can be canonical data types, data schemas, and / or any other kind of data types. For example, when performing a PIAR that receives data from an email application, the system can discover a data schema corresponding to an appointment within the email message body. The email application can expose the email content as a field that the email application reports as having a "string" data type. The content of a particular email (i.e., the data value corresponding to a particular instance of the field) can include a meeting invitation in natural language form. Based on the email content, the system can discover a "meeting invitation" data schema. If the system uses semantic analysis to discover data types, the system can use semantic context (i.e., words that are near and associated with one or more parts of the data) to infer the one or more data types. As another example, when performing a PIAR that generates a calendar appointment, the system can discover that one or more appointment instances correspond to a data schema associated with a sports event. In an embodiment, the system uses machine learning to discover one or more data types in one or more data values encountered when performing a PIAR. For example, the system can receive unstructured data from a trigger application and / or an action application and use machine learning to identify one or more data types in the unstructured data. Generally, in an embodiment, the discovered data type is different from the data type of the field that provides the one or more data values (i.e., the data type that the trigger application or action application reports for the field). The plug-in application can explicitly report the data type of a field. As an example, the plug-in application can expose a discoverable API that provides information about the one or more data types of the one or more fields accessible via the API. As another example, the plug-in application can provide the one or more data values in a data format that includes information about the one or more data types of the one or more fields (e.g., one or more tags in XML or JSON data that identify the one or more data types of the one or more fields included in the data). Alternatively or additionally, the plug-in application can implicitly report the data type of a field, i.e., by providing one or more data values of a particular data type without including additional data and / or metadata that explicitly identify the data type. The plug-in application can explicitly or implicitly report the data type of a field in many different ways.

[0088] In an embodiment, the discovered data type is narrower (i.e., more specific) than the data type(s) of the data value(s) in which it is discovered. For example, the data value may be of string type, while the data type discovered within the string is of the "email address" type. The data type of an email address complies with the string data type. In computer programming terms, an email address can be cast as a string data type. However, an email address is a narrower data type with a more specific meaning compared to the broader string data type. Generally, the data type discovered within the data value(s), even if narrower than the data value(s), complies with the data type reported for the data value(s). Continuing with the above example, there are no email addresses that do not comply with the string data type.

[0089] In an embodiment, the discovered data type is based on multiple data values. These data values can be for multiple instances of the same field and / or different fields. The system can obtain data values for multiple fields across different plug-in applications and discover a composite or "jumbo" data type based on the combination of data values for the multiple fields. For example, one field may provide an email address, another field may provide a phone number, and another field may provide a name. The system can discover a "contact" data type as a composite of these individual data values for different fields.

[0090] In an embodiment, if the system is unsure of the data type discovered as described above, the system can prompt the user for input to help identify the data type(s) discovered. For example, the system can identify multiple data types in unstructured data and present the identified data types as candidates to the user. The system can receive user input selecting one or more of the candidates.

[0091] In an embodiment, the system discovers one or more data types for which the currently executing PIAR has not defined any specific corresponding actions. For example, a PIAR can be configured to generate an email message based on new customer data received from a customer relationship management (CRM) application. In one or more instances, the data from the CRM application may include date information corresponding to an onboarding appointment for a new customer, but the PIAR does not include any specific corresponding actions for generating a calendar appointment.

[0092] In an embodiment, the system stores metadata (operation 206) that describes fields accepted by supported actions (i.e., one or more actions supported by one or more action applications). In this context, a field is an expected input of a particular data type for an action, supported by an action application. A field may also be referred to as a variable. The data type may be a canonical data type, a data schema, and / or any other kind of data type. For example, an email application may support a "send email" action, and for a given email, this operation expects: a sender; one or more recipients; a subject line; and email content. In an embodiment, the system uses machine learning to discover one or more fields of supported actions. Alternatively or additionally, the action application accepts fields for performing one or more actions via an API. The API may be discoverable such that the system can send a discovery request to the API and receive, in response to the discovery request, an object that describes the (one or more) actions supported by the action application and / or the (one or more) fields accepted by the (one or more) supported actions. Discovering the (one or more) actions supported by the action application and / or the (one or more) fields accepted by the (one or more) actions using a discovery request may require authentication.

[0093] In an embodiment, the system identifies one or more mappings between one or more discovered data types and one or more data types of one or more fields accepted by one or more actions (operation 208). Generally, a mapping between a discovered data type and the data type of a field accepted by an action indicates that the action accepts data of the same data type as the discovered data type as input for the field. In some embodiments, the mapping may be constrained by transforming one or more data types from an unstructured format to a canonical data type and / or a data schema, and / or transforming from one data type to another via a set of one or more predefined transformation functions (e.g., transforming a string to a date and / or any other kind of transformation between data types). The discovered data type "maps" to the data type of the action application field. In an embodiment, the system uses machine learning to identify one or more mappings between the discovered data type and the data type of the field accepted by the action. Alternatively or additionally, the system may identify branch mappings, i.e., mappings between: (a) a data type that varies between two or more data value instances encountered by the system during the execution of a PIAR and (b) the fields of two or more actions that each respectively accept inputs of one or more variable data types.

[0094] In an embodiment, the system generates one or more PIAR extensions based on the identified mapping(s). The PIAR extension defines one or more actions that can be taken regarding data received by the system when executing the currently-executed PIAR (i.e., when the received data includes the discovered data type(s)), but the currently-executed PIAR has not defined specific actions regarding that data. When incorporated into and executed by the system, the PIAR extension "extends" the currently-executed PIAR by providing PIAR functionality that the currently-executed PIAR does not provide. The PIAR extension can correspond to a separate PIAR, i.e., a PIAR that can be executed independently of the currently-executed PIAR. Alternatively, the PIAR extension can correspond to additional actions and / or logical branches to be included in the currently-executed PIAR. For example, a PIAR extension for generating an appointment based on unstructured data received from a CRM application can correspond to a separate PIAR and / or to additional actions in the currently-executed PIAR that has already generated an email based on data from the CRM application.

[0095] To generate a PIAR extension based on a mapping, the system can determine whether the specific mapping is a branch mapping (operation 210). If the mapping is a branch mapping, the system generates a branch PIAR extension based on the mapping (operation 212). If the mapping is not a branch mapping, the system can generate a linear PIAR extension based on the mapping (operation 214). A linear PIAR extension is a PIAR extension that does not include conditional logic for branching between different actions based on the received data type.

[0096] In an embodiment, the PIAR extension corresponds to a separate PIAR, i.e., a PIAR that can be executed independently of the currently-executed PIAR. The system can generate one or more PIAR definitions corresponding to the one or more PIAR extensions generated based on the identified mapping(s). If the PIAR extension modifies the currently-executed PIAR, the system may generate a corresponding modified PIAR definition. Alternatively or additionally, the system can generate a description of the PIAR extension (which is not a PIAR definition) for each PIAR extension generated. In an embodiment, the system determines whether another mapping for which the system has not generated a PIAR extension is identified (operation 216). If the other mapping is identified, the system can also generate a branch or linear PIAR extension based on the mapping (operation 212 or operation 214).

[0097] In an embodiment, the system presents the generated PIAR extension(s) in the user interface as one or more candidate PIAR extensions (operation 218). The user interface includes one or more controls for selecting a candidate PIAR extension, and based on the user selection, the system incorporates the candidate PIAR extension into the PIAR and begins execution. The user interface may present any kind of information about the generated PIAR extension(s), such as the trigger application(s) of the PIAR extension, the action application(s), the data type(s) used as the input and / or output field(s) in the PIAR extension, the data from the content item(s) used to discover one or more data types, and / or any other kind of information about the generated PIAR extension, or a combination thereof.

[0098] In an embodiment, the system detects the user's selection of a candidate PIAR extension (operation 220). As described above, the system may also detect user input that modifies the selected PIAR extension. In response to the selection of the modified or unmodified candidate PIAR extension, the system executes the PIAR that includes the PIAR extension (operation 222), which may be a new PIAR or a modification to the currently executing PIAR. If the system has not generated a PIAR definition that incorporates the selected PIAR extension, the system may generate the PIAR definition before executing the PIAR that incorporates the PIAR extension. In an embodiment, as described above, the system executes the PIAR in an execution mode independent of the user.

[0099] 3.2 Illustrative Examples

[0100] For clarity, detailed examples are described below. The components and / or operations described below should be understood as a specific example that may not be applicable to some embodiments. Therefore, the components and / or operations described below should not be construed as limiting the scope of any claims.

[0101] Figure 3Illustrated is a PIAR management application 340 linked to three plug-in applications: an email application 302, a calendar application 316, and a task management application 328. During operation of the PIAR, the PIAR management application 340 processes email messages 304 from the email application 302. The email messages 304 include a date 310, a physical address 312, and a name 314. The email application 302 does not directly expose fields corresponding to the date 310, the physical address 312, or the name 314, but rather indirectly exposes these fields within the content of the email message 304. The PIAR management application 340 performs semantic analysis on the email message 304 by applying the email message to a machine learning model that is trained to discover data types in unstructured data. The PIAR management application 340 discovers the date 310, the physical address 312, and the name 314 within the email message 304. Additionally, based in part on the semantic context 306 within the email message 304, the PIAR management application 340 predicts that the date 310, the physical address 312, and the name 314 may be logically related to each other and are thus components of a data pattern 308.

[0102] Additionally, in this example, the PIAR management application 340 sends a discovery request to the calendar application 316. In response to the discovery request, the PIAR management application 340 receives a discovery object 318 from the calendar application 316 that describes the data types of fields accepted by one or more actions of the calendar application 316. Specifically, the calendar application 316 supports a "create appointment" action (not shown) that accepts an appointment object 320. The appointment object 320 is a data pattern that includes a date 322, a physical address 324, and one or more participants 326.

[0103] Additionally, in this example, the PIAR management application 340 sends a discovery request to the task management application 328. In response to the discovery request, the PIAR management application 340 receives a discovery object 330 from the task management application 328 that describes the data types of fields accepted by one or more actions of the task management application 328. Specifically, the task management application 328 supports a "create task" action (not shown) that accepts a task object 332. The task object 332 is a data pattern that includes a date 334, a physical address 336, and an assignee 338.

[0104] Based on the discovered data types, the PIAR management application 340 identifies the mapping between the data schema 308 and the data types of the fields accepted by the supported actions. Specifically, in this example, the PIAR management application 340 identifies two possible mappings. For one mapping, the PIAR management application 340 predicts that the unknown data schema 308 exposed by the email application 302 can be mapped to the appointment object 320 accepted by the calendar application 316. For the other mapping, the PIAR management application 340 predicts that the unknown data schema 308 exposed by the email application 302 can be mapped to the task object 332 accepted by the task management application 328. Based on these two mappings, the PIAR management application generates two candidate PIAR extensions 342. In one candidate PIAR extension 334, detecting an instance of the data schema 308 in an email message triggers the creation of an appointment in the calendar application 316. In the other candidate PIAR extension 346, detecting an instance of the data schema 308 in an email message triggers the creation of a task in the task management application 328. The PIAR management application 340 presents the candidate PIAR extensions 342 in a user interface (not shown). If the user selects one of the candidate PIAR extensions 342, the PIAR management application 340 incorporates the PIAR extension into the currently executing PIAR, and the PIAR management application 340 executes the PIAR in an execution mode independent of the user.

[0105] 4 Plug-in Application Recipe Generation Based on Semantic Analysis

[0106] 4.1 Operations

[0107] Figure 4 An example set of operations for plug-in application recipe generation based on semantic analysis according to one or more embodiments is shown. Figure 4 One or more of the operations shown in can be modified, rearranged, or omitted together. Thus, Figure 4 the operations in the specific order shown in should not be construed as limiting the scope of one or more embodiments.

[0108] In an embodiment, unless otherwise specified, the operations described below can be executed in an execution mode independent of the user, i.e., as one or more processes executed without user input. Executing operations in an execution mode independent of the user can improve the speed and / or efficiency of the system by reducing or eliminating the need to obtain user input at various operation stages.

[0109] In an embodiment, the system (e.g., Figure 1The PIAR management system 100 trains a machine learning model (operation 402). The system can train a machine learning model to perform semantic analysis on structured and / or unstructured (e.g., natural language) inputs. In an embodiment, the system uses supervised learning to train the machine learning model based on a training data set that identifies a mapping between words (or combinations thereof) in a human language and instructions for performing PIAR.

[0110] In an embodiment, the system receives a data item that includes words in a human language (operation 404). The data item can take a variety of different forms. The data item can include natural language input from a user (e.g., text and / or audio input). Alternatively or additionally, the data item can include the output of a plug-in application (e.g., an email message, a text message, an audio file, a video file, a document, calendar data, and / or any other type of data or combination thereof that includes words in a human language).

[0111] In an embodiment, the system performs semantic analysis on the data item (operation 406). To perform semantic analysis, the system can apply the data item to the machine learning model. Alternatively or additionally, the system can use other semantic analysis techniques. Semantic analysis determines the meaning (or its prediction) associated with the word(s) in the data item.

[0112] In an embodiment, semantic analysis determines that the data item specifies a desired action category to be performed. For example, the data item can specify the action category "send an email". The system can determine that the phrase "send an email" specifies an action category because it maps to any number of actions supported by a plug-in application that can send emails. These supported actions belong to the same action category. As another example, the data item can specify the action category "remind me". The system can determine that the phrase "remind me" is an action category because it maps to any number of actions of a plug-in application that can generate an alert (e.g., an email alert, a text alert, a pop-up alert, etc.). These supported actions belong to the same action category. Semantic analysis can map the actions supported by the system to the desired action category. This mapping can be based on metadata stored by the system about the plug-in applications, which defines the action(s) and / or trigger(s) supported by each plug-in application. The metadata about each plug-in application corresponds to a profile of the plug-in application for current and / or future PIAR.

[0113] In an embodiment, semantic analysis determines that the data item specifies a desired trigger genre to be detected. For example, the data item can specify the trigger genre "when I receive an email". The system can determine that the phrase "when I receive an email" specifies a trigger genre because it maps to any number of trigger conditions associated with any number of plug-in applications that detect when an email has been received. These different trigger conditions belong to the same trigger genre. Semantic analysis can map the triggers supported by the system to the desired trigger genre. This mapping can be based on metadata stored by the system about the plug-in applications, which defines the action(s) and / or trigger(s) supported by each plug-in application. The metadata about each plug-in application corresponds to a profile of the plug-in application for current and / or future PIARs.

[0114] In an embodiment, at least partially based on the result(s) of semantic analysis (i.e., operation 406), the system determines whether the data item includes an instruction for performing a PIAR (operation 408). For example, the data item can include the words "whenever I receive an email from Mary, send me a text message", which the system recognizes as an instruction for performing a PIAR, where detecting the trigger ("whenever I receive an email from Mary") causes the action ("send me a text message"). As another example, the data item can include the words "turn off my morning alarm", which includes an instruction but not an instruction for performing a PIAR. In an embodiment, if the data item does not include an instruction for performing a PIAR, the system continues to process the data item without performing a PIAR based on the data item (operation 410).

[0115] In an embodiment, if the data item includes an instruction for performing a PIAR, the system determines whether an existing PIAR satisfies the instruction (operation 412). In the above example, the system may already include a PIAR definition that describes a trigger of "receiving an email from [email address]" and a corresponding action of "sending a text message to [phone number]". If the existing PIAR satisfies the instruction, the system can configure the existing PIAR based on the instruction (operation 415). Continuing with the above example, the system can replace [email address] in the trigger condition with Mary's email address from the user's address book and replace [phone number] in the action with the user's own phone number. If the system detects ambiguity when configuring the existing PIAR (e.g., if there are two contacts named Mary in the user's address book), the system may prompt the user to clarify the ambiguity.

[0116] In an embodiment, if no existing PIAR satisfies the instruction, the system can generate one or more new PIARs (operation 414). The system can generate multiple PIARs as candidates for satisfying the instruction. Alternatively or additionally, the system can generate one or more PIARs based on the mappings identified by the system as described above. The system can generate branched PIARs, linear PIARs, or a combination thereof. As an example, a data item can include the phrase "remind me when I receive an email from Mary". The system can determine (e.g., using machine learning as described above) that the phrase "remind me" is ambiguous and can mean "text me", "generate an app notification", or "email me". The system can generate PIARs corresponding to each possible option. If an existing PIAR satisfies one of the options, the system can configure the existing PIAR for that option and generate new PIARs for any option(s) that are not satisfied by any existing PIAR.

[0117] In an embodiment, the system presents one or more candidate PIARs in a user interface (i.e., existing and / or newly generated candidate PIARs identified by the system as satisfying the instruction) (operation 416). The user interface includes one or more controls for selecting a candidate PIAR, and based on the user selection, the system will begin to execute the candidate PIAR. The user interface can present any kind of information about the candidate PIAR(s), such as the trigger application(s) of the PIAR, the action application(s), the data item used to generate the PIAR, and / or any other kind of information about the candidate PIAR, or a combination thereof.

[0118] In an embodiment, the system detects the user's selection of a candidate PIAR (operation 418). As described above, the system can also detect user input that modifies the selected PIAR. In response to the selection of the modified or unmodified candidate PIAR, the system executes the selected PIAR (operation 420). If the system has not yet generated a PIAR definition corresponding to the selected PIAR, the system can generate the PIAR definition before executing the PIAR. In an embodiment, as described above, the system executes the PIAR in an execution mode independent of the user.

[0119] 4.2 Illustrative Examples

[0120] For clarity, detailed examples are described below. The components and / or operations described below should be understood as a specific example that may not apply to some embodiments. Therefore, the components and / or operations described below should not be construed as limiting the scope of any claims.

[0121] Figure 5Illustrated is a semantic analysis engine 504 running in a PIAR management application (not shown). The semantic analysis engine receives natural language input 502 from a user, which has an instruction of "Whenever I receive an email from a new potential customer, create an urgent task to follow up within 30 minutes and have a 5-minute reminder". The semantic analysis engine 504 performs semantic analysis on the natural language input 502. Based on the semantic analysis, the system identifies two candidate PIARs 506, 512 that satisfy the instruction in the natural language input 502. The two candidate PIARs 506, 516 have different trigger conditions 508, 514 and different actions 510, 516, and each action and condition has two components. Although Figure 5 only two candidate PIARs 506, 512 are illustrated, in an embodiment, the system generates additional candidate PIARs corresponding to each additional permutation of the components of the trigger condition and action(s) to provide the user with a full range of options.

[0122] Continuing with the above example, the PIAR management application presents the candidate PIARs 506, 512 in a user interface (not shown). When the user selects one of the candidate PIARs 506, 512 for execution, the PIAR management application continues to execute the selected PIAR in an execution mode independent of the user.

[0123] 5 User-assisted plug-in application recipe execution

[0124] 5.1 Operations

[0125] Figure 6 Illustrated is an example set of operations for user-assisted plug-in application recipe execution according to one or more embodiments. Figure 6 One or more of the operations shown therein may be modified, rearranged, or omitted together. Thus, Figure 6 the operations in the specific order shown should not be construed as limiting the scope of one or more embodiments.

[0126] In an embodiment, unless otherwise specified, the operations described below may be executed in an execution mode independent of the user, i.e., as one or more processes executed without user input. Executing operations in an execution mode independent of the user can improve the speed and / or efficiency of the system by reducing or eliminating the need to obtain user input at various operation stages.

[0127] In an embodiment, the system (e.g., Figure 1The PIAR management system 100 trains a machine learning model (operation 602). The system can train a machine learning model to identify data types (e.g., specification data types, data patterns, and / or another data type or a combination thereof). The machine learning model can be trained to identify data types in structured and / or unstructured data originating from one or more plug-in applications. In an embodiment, the system uses supervised learning to train the machine learning model based on a training data set that identifies the mapping between structured and / or unstructured data and corresponding data types. Techniques for identifying data types using machine learning and / or non-machine learning techniques are described in further detail above.

[0128] In an embodiment, the system executes PIAR in a user-independent execution mode (operation 604). Specifically, the system executes PIAR as an automatic background service that will continue to run permanently without further manual intervention in the absence of conditions that terminate or interrupt the execution of PIAR.

[0129] In an embodiment, when executing PIAR in a user-independent execution mode, the system receives one or more data values of one or more fields of one or more plug-in applications (i.e., data streams and / or discrete data items) (operation 605). The system can receive the (one or more) data values from a trigger application. As an example, PIAR can be configured to receive data values from an email application and evaluate the data values to determine whether a trigger condition is met. Alternatively or additionally, the system can receive the (one or more) data values from an action application. When executing an action that is part of PIAR, the plug-in application that executes the action can generate one or more data values. As an example, as an action, PIAR can use a calendar application to generate an appointment. The system can receive one or more data values representing the new appointment.

[0130] In an embodiment, the system applies the (one or more) data values received for the (one or more) plug-in application fields to a machine learning model (operation 606). Applying the (one or more) data values to the machine learning model can determine candidate mappings between (one or more) sub-values found in the (one or more) data values (e.g., using techniques for discovering data types as described herein) and the fields accepted by the actions of the plug-in application. In some embodiments, the candidate mappings can be constrained by transforming one or more data values or sub-values thereof from an unstructured format into a canonical data type and / or data schema, and / or transforming from one data type to another via a set of one or more predefined transformation functions (e.g., transforming a string into a date and / or any other kind of transformation between data types). Generally, the mapping between the sub-value and the field accepted by the action indicates that the action accepts data of the data type that exists (directly or via the data discovery process described herein) in at least a portion of the received data as input to the field. This data type is "mapped" to the data type of the action application field. As an example, the system can detect when an email body includes a new appointment request even if the email body only includes text data and does not include any (one or more) data values of type "appointment". The system can perform semantic analysis on the email message body by applying the email message to a machine learning model configured to perform semantic analysis to determine that the email message body includes a new appointment request. In one embodiment, the discovered data type (

[0131] As used herein, the term "candidate mapping" indicates that the mapping is determined using machine learning and may be incorrect. In an embodiment, applying data to the machine learning model generates a confidence metric representing the confidence that the mapping is correct (or representing the lack of such confidence). The confidence metric can be a number, letter, boolean value, or any other kind of metric for which two or more different values indicate different levels of confidence that the mapping is correct. The confidence metric can also be referred to as a confidence score.

[0132] In an embodiment, the confidence metric is based on whether the discovered sub-value(s) is / are adapted to one or more stored formats (i.e., data arrangement / configuration) of one or more data types accepted by one or more fields mapped to the plug-in application. As described above, the system may store the one or more formats in the plug-in application profile. The confidence metric may also be based on whether multiple sub-values of the data value are adapted to the stored format. If multiple candidate mappings (i.e., the adaptation between the sub-value and the supported action field format) are identified, the confidence score for a particular mapping may be low. Alternatively or additionally, the confidence score for a particular mapping may be based on the location of different discovered sub-values. Sub-values located earlier in the data value may receive a higher confidence metric compared to sub-values located later in the data value, and vice versa. Alternatively or additionally, different sub-values may be mapped to different field formats, and the system may apply different weights to different formats. The weights associated with each format may change over time, e.g., based on the absolute or relative frequency of different formats, and / or as the system learns (via the system-user feedback loop as described herein) which formats are most successful and / or most important as indicated by user input. Alternatively or additionally, the confidence metric may be based on context data surrounding the sub-value(s). The system may use semantic analysis to determine the semantic context of the sub-value and apply a corresponding weight to the confidence metric of the sub-value. The weights associated with each context may change over time, e.g., based on the absolute or relative frequency of different contexts, and / or as the system learns (via the system-user feedback loop as described herein) which contexts are most successful and / or most important as indicated by user input.

[0133] In an embodiment, the system determines whether the confidence metric meets a threshold confidence criterion (operation 608). The threshold confidence criterion indicates a value or range of values of the confidence metric outside of which the system lacks sufficient confidence that the mapping is correct. For example, when the system encounters an unfamiliar data structure for which the machine learning model has not been trained, or if the system determines that the received data includes multiple candidates (i.e., sub-values) for mapping to the same field accepted by the action application (e.g., multiple dates, email addresses, etc.), the confidence metric may not meet the threshold confidence criterion. The threshold confidence criterion may be a fixed value applicable to all confidence metrics. Alternatively, the threshold confidence criterion may be user-configurable, e.g., configurable at the user, group, or organizational level.

[0134] In an embodiment, if the confidence metric meets a threshold confidence criterion, the system continues to execute the PIAR in a user-independent execution mode (e.g., by performing the next action in the PIAR using the identified mapping). If the confidence metric does not meet the threshold confidence criterion, the system may generate a request for user input to confirm or reject the candidate mapping (operation 610). As an example, the PIAR is configured to send a text message when an email is urgent. The system receives an email that includes the sentence "Please process immediately." The system does not recognize the specific word "immediately," but predicts with 65% confidence that the word following "process" indicates urgency. The system generates a candidate mapping between the email message and the actions supported by the text messaging application. In this example, the confidence metric is a confidence score and the threshold confidence condition is a minimum confidence score of 90% confidence required to continue executing the PIAR in a user-independent execution mode. Since the threshold condition is not met, the system generates a request for user input to confirm or reject the candidate mapping.

[0135] In an embodiment, the system sends a request for user input to the user (operation 612). The user who receives the request may be the same user who created the PIAR or another user. The system may send the request to multiple users. The request may take many different forms. For example, the request may be an email, a text message, an application notification, and / or another form of request or a combination thereof. In an embodiment, the request notifies the user of the candidate mapping and requests user input for confirming or rejecting the candidate mapping. The request may include one or more of the following: received data that is a candidate for the mapping to the action; a confidence metric (e.g., a confidence score) associated with the candidate mapping; additional mappings (if the system identifies multiple candidate mappings); and / or other information associated with the candidate mapping.

[0136] In an embodiment, the system determines whether it has received user input in response to the request (operation 614). Depending on the form and content of the request, the user input may correspond to one or more of the following: confirm the candidate mapping; reject the candidate mapping; highlight and / or otherwise select a specific mapping among two or more candidate mappings; a suggested mapping other than any mapping identified by the system as a candidate mapping; and / or other forms of input, or a combination thereof.

[0137] In an embodiment, in addition to user input that confirms or rejects a candidate mapping, the system also receives user input indicating whether the system should generate requests more or less frequently in similar situations. The system can be configured to adjust its threshold confidence criteria based on the user input. If the user input indicates that the system should generate requests more frequently in similar situations, the system can adjust the threshold confidence criteria to require an even higher confidence before continuing to execute the PIAR in a user-independent execution mode. If the user input indicates that the system should generate requests more frequently in similar situations, the system can adjust the threshold confidence criteria to continue to execute the PIAR in a user-independent execution mode even when the confidence is low.

[0138] In an embodiment, the system applies the user input to the execution of the PIAR (operation 616) and continues to execute the PIAR in a user-independent execution mode (operation 604). Specifically, the system continues to execute the PIAR in the manner indicated in the user input. Based on the user input, the system can perform the actions indicated by the mapping, perform different actions specified by the user, take no action on the received data, and / or perform another action indicated by the user input, or a combination thereof.

[0139] In an embodiment, the system updates the machine learning model based on the user input (operation 618). Updating the machine learning model can improve the system's ability to generate higher-confidence results in the future in similar situations, thereby making it less likely for the system to request user input. In future instances of executing the PIAR, the system can receive similar data and generate candidate mappings with corresponding metrics that meet the threshold confidence criteria. Over time, based on the user input, the system can gradually train the machine learning model and request additional user input less and less frequently. The system can update the machine learning model based on the input of a specific user, only for that user. Alternatively, the system can update the machine learning model for multiple users using the same PIAR, such that the PIAR performs better for each of these users.

[0140] 5.2 Illustrative Examples

[0141] For clarity, detailed examples are described below. The components and / or operations described below should be understood as a specific example that may not apply to some embodiments. Therefore, the components and / or operations described below should not be construed as limiting the scope of any claims.

[0142] Figure 7AThe PIAR management application 704 executes the PIAR 706 in an execution mode independent of the user. The PIAR 706 is a branched PIAR, where the email application 702 is the trigger application, and there are two action applications: the calendar application 708 and the task management application 710. If the PIAR management application 704 identifies data corresponding to an event in an email message from the email application 702, it instructs the calendar application 708 to generate a new event. If the PIAR management application 704 identifies a task in the email message, it instructs the task management application 710 to generate a new task.

[0143] In Figure 7B , the PIAR management application 704 receives an email message 712 with the text "Handle this issue at 10:00 tomorrow morning". The PIAR management application 704 applies the email message 712 to a machine learning model 714 configured to perform semantic analysis on the unstructured text in the email message. Based on the email message 712, the machine learning model 714 generates candidate mappings 718 and corresponding confidence metrics 716. Specifically, the machine learning model 714 determines that a portion of the text of the email message may correspond to a calendar event or a task. None of the candidate mappings 718 have a confidence metric 716 high enough for the PIAR management application 704 to select that branch of the branched PIAR 706.

[0144] In Figure 7C , the PIAR management application 704 sends a request to the user interface 720 to present the candidate mappings 718 and request user input to confirm or reject one or more of the candidate mappings in the candidate mappings 718. The PIAR management application 704 receives a user response from the user interface 720 and selects (i.e., confirms) one of the candidate mappings 718. Based on the user response, the PIAR management application 704 updates the machine learning model 714 such that the branch of the PIAR 706 selected by the user is selected with a higher confidence in future similar situations.

[0145] In an embodiment, after resuming execution of PIAR 706 in an execution mode independent of the user, the PIAR management application 704 receives another email message 722 having similar text: "Process this issue at 2:00 on Tuesday". This time, when the PIAR management application 704 applies the email message 722 to the machine learning model 714, the machine learning model 724 selects the same action that the user previously selected for the earlier email message 712. The confidence metric 724 is high enough to allow the PIAR management application 704 to follow that branch of the branched PIAR 706 as an action 726 independent of the user, without requesting further user input.

[0146] 6. Miscellaneous; Extensions

[0147] An embodiment is directed to a system having one or more devices, the one or more devices including a hardware processor and configured to perform any of the operations described herein and / or recited in any of the following claims.

[0148] In an embodiment, a non-transitory computer-readable storage medium includes instructions that, when executed by one or more hardware processors, cause performance of any of the operations described herein and / or recited in any claim.

[0149] Any combination of the features and functions described herein may be used in accordance with one or more embodiments. In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary with implementation. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive. The sole and exclusive indicator of the scope of the present invention, and what the applicant intends to be the scope of the present invention, is the literal and equivalent scope of the set of claims issued in this application, in the specific form in which such claims are issued, including any subsequent corrections.

[0150] 7. Hardware Overview

[0151] According to an embodiment, the techniques described herein are implemented by one or more special-purpose computing devices (computing devices specifically configured to perform a certain function). The special-purpose computing device can be hard-wired to perform the techniques or can include digital electronic devices permanently programmed to perform the techniques, such as one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or network processing units (NPUs), or can include one or more general-purpose hardware processors programmed to perform the techniques according to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices can also combine custom hard-wired logic, ASICs, FPGAs, or NPUs with custom programming to complete the techniques. The special-purpose computing device can be a desktop computer system, a portable computer system, a handheld device, a network device, or any other device incorporating hard-wired and / or program logic to implement the techniques.

[0152] For example, Figure 8 is a block diagram of a computer system 800 on which embodiments of the present invention can be implemented. Computer system 800 includes a bus 802 or other communication mechanism for conveying information, and a hardware processor 804 coupled to bus 802 for processing information. The hardware processor 804 can be, for example, a general-purpose microprocessor.

[0153] Computer system 800 also includes a main memory 806 coupled to bus 802, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by processor 804. The main memory 806 can also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 804. When such instructions are stored in a non-transitory storage medium accessible to processor 804, computer system 800 is presented as a special-purpose machine customized to perform the operations specified in the instructions.

[0154] Computer system 800 also includes a read-only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A storage device 810, such as a magnetic disk or optical disk, is provided and coupled to bus 802 for storing information and instructions.

[0155] The computer system 800 can be coupled to a display 812 via a bus 802, such as a liquid crystal display (LCD), a plasma display, an electronic ink display, a cathode ray tube (CRT) monitor, or any other kind of device for displaying information to a computer user. An input device 814, including alphanumeric keys and other keys, is coupled to the bus 802 for transmitting information and command selections to the processor 804. Alternatively or additionally, the computer system 800 can receive user input via a cursor control 816, such as a mouse, a trackball, a trackpad, a touchscreen, or cursor direction keys for transmitting direction information and command selections to the processor 804 and for controlling the movement of a cursor on the display 812. This input device typically has two degrees of freedom in two axes (a first axis (e.g., x) and a second axis (e.g., y)), which allows the device to specify a position in a plane. The display 812 can be configured to receive user input via one or more pressure-sensitive sensors, multi-touch sensors, and / or gesture sensors. Alternatively or additionally, the computer system 800 can receive user input via a microphone, a camera, and / or some other kind of user input device (not shown).

[0156] The computer system 800 can implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that in combination with the computer system causes the computer system 800 to be a special-purpose machine or programs it to be a special-purpose machine. According to one embodiment, the techniques herein are performed by the computer system 800 in response to one or more sequences of one or more instructions contained in the main memory 806 being executed by the processor 804. Such instructions can be read into the main memory 806 from another storage medium, such as the storage device 810. Execution of the instruction sequence contained in the main memory 806 causes the processor 804 to perform the processing steps described herein. In alternative embodiments, hardwired circuitry can be used in place of or in combination with software instructions.

[0157] As used herein, the term “storage medium” refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such storage medium may include non-volatile media and / or volatile media. Non-volatile media includes, for example, optical discs or magnetic disks such as storage device 810. Volatile media includes dynamic memory such as main memory 806. Common forms of storage medium include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tape or any other magnetic data storage medium, CD-ROM, any other optical data storage medium, any physical medium with hole patterns, RAM, programmable read-only memory (PROM) and erasable PROM (EPROM), FLASH-EPROM, non-volatile random access memory (NVRAM), any other memory chip or cartridge, content addressable memory (CAM) and ternary content addressable memory (TCAM).

[0158] Storage medium is different from transmission medium but may be used in combination with transmission medium. Transmission medium participates in transferring information between storage media. For example, transmission medium includes coaxial cables, copper wires and optical fibers, including the wires that include bus 802. Transmission medium may also take the form of acoustic waves or light waves, such as those generated during radio wave and infrared data communications.

[0159] Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 804 for execution. For example, the instructions may initially be carried on a disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions via a network interface controller (NIC) (such as an Ethernet controller or a Wi-Fi controller) over a network. The NIC local to computer system 800 may receive the data from the network and place the data on bus 802. Bus 802 carries the data to main memory 806, and processor 804 retrieves and executes the instructions from main memory 806. The instructions received by main memory 806 may optionally be stored on storage device 810 before or after being executed by processor 804.

[0160] Computer system 800 also includes a communication interface 818 coupled to bus 802. Communication interface 818 provides a two-way data communication coupling to network link 820 that is connected to local network 822. For example, communication interface 818 may be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides a data communication connection to a corresponding type of telephone line. As another example, communication interface 818 may be a local area network (LAN) card that provides a data communication connection to a compatible LAN. A wireless link may also be implemented. In any such implementation, communication interface 818 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0161] Network link 820 generally provides data communication to other data devices via one or more networks. For example, network link 820 can provide a connection via local network 822 to host 824 or to a data device operated by an Internet service provider (ISP) 826. The ISP 826 in turn provides data communication services via the global packet data communication network now commonly referred to as the “Internet” 828. Both the local network 822 and the Internet 828 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through the various networks and on network link 820 and through communication interface 818, which carry digital data to and from computer system 800, are example forms of transmission media.

[0162] Computer system 800 can send messages and receive data, including program code, via the network(s), network link 820, and communication interface 818. In the Internet example, server 830 can send request code for an application via the Internet 828, ISP 826, local network 822, and communication interface 818.

[0163] The received code can be executed by processor 804 when it is received, and / or stored in storage device 810 or other non-volatile memory for later execution.

[0164] 8. Computer Networks and Cloud Networks

[0165] In one or more embodiments, a computer network provides a connection between a set of nodes that run software that utilizes the techniques described herein. The nodes can be local to each other and / or remote from each other. The nodes are connected by a set of links. Examples of links include coaxial cable, unshielded twisted pair, copper cable, fiber optic, and virtual links.

[0166] A subset of the nodes implements the computer network. Examples of such nodes include switches, routers, firewalls, and network address translators (NATs). Another subset of the nodes uses the computer network. Such nodes (also referred to as “hosts”) can execute client processes and / or server processes. A client process requests computing services (such as the execution of a particular application and / or the storage of a particular amount of data). A server process responds by executing the requested service and / or returning the corresponding data.

[0167] A computer network can be a physical network, including physical nodes connected by physical links. A physical node is any digital device. A physical node can be a hardware device with a specific function, such as a hardware switch, a hardware router, a hardware firewall, and a hardware NAT, etc. Additionally or alternatively, a physical node can be any physical resource that provides computing power to execute tasks, such as a physical resource configured to execute various virtual machines and / or applications that perform corresponding functions. A physical link is a physical medium that connects two or more physical nodes. Examples of links include coaxial cables, unshielded twisted pair cables, copper cables, and optical fibers.

[0168] A computer network can be an overlay network. An overlay network is a logical network implemented on top of another network (such as a physical network). Each node in the overlay network corresponds to a corresponding node in the underlying network. Therefore, each node in the overlay network is associated with an overlay address (for addressing the overlay node) and an underlying address (for addressing the underlying node that implements the overlay node). An overlay node can be a digital device and / or a software process (such as a virtual machine, an application instance, or a thread). The link connecting the overlay nodes is implemented as a tunnel through the underlying network. The overlay nodes at either end of the tunnel view the underlying multi-hop path between them as a single logical link. The tunnel is performed through encapsulation and decapsulation.

[0169] In an embodiment, a client can be local to and / or remote from a computer network. The client can access the computer network through other computer networks (such as a private network or the Internet). The client can use a communication protocol such as the Hypertext Transfer Protocol (HTTP) to send requests to the computer network. The requests are communicated through an interface (such as a client interface (such as a web browser), a program interface, or an Application Programming Interface (API)).

[0170] In an embodiment, a computer network provides connectivity between a client and network resources. Network resources include hardware and / or software configured to execute server processes. Examples of network resources include processors, data storage, virtual machines, containers, and / or software applications. Network resources are shared among multiple clients. The clients independently request computing services from the computer network. Network resources are dynamically assigned to requests and / or clients on a demand basis. The network resources assigned to each request and / or client can be scaled up or down based on, for example, (a) the computing services requested by a specific client, (b) the aggregated computing services requested by a specific tenant, and / or (c) the aggregated computing services requested by the computer network. Such a computer network can be referred to as a "cloud network".

[0171] In an embodiment, a service provider provides a cloud network to one or more end users. The cloud network can implement various service models, including but not limited to Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). In SaaS, the service provider provides the end user with the ability to use the service provider's applications executed on network resources. In PaaS, the service provider provides the end user with the ability to deploy custom applications onto network resources. Custom applications can be created using programming languages, libraries, services, and tools supported by the service provider. In IaaS, the service provider provides the end user with the ability to provision processing, storage, networking, and other basic computing resources provided by network resources. Any application, including an operating system, can be deployed on network resources.

[0172] In an embodiment, various deployment models can be implemented via a computer network, including but not limited to private clouds, public clouds, and hybrid clouds. In a private cloud, network resources are provided for exclusive use by a specific group of one or more entities (the term “entity” as used herein refers to a company, organization, individual, or other entity). The network resources can be located local to the premises of the specific group of entities and / or remote from the premises of the specific group of entities. In a public cloud, cloud resources are provided for multiple entities (also referred to as “tenants” or “customers”) that are independent of each other. The computer network and its network resources are accessed by clients corresponding to different tenants. Such a computer network can be referred to as a “multi-tenant computer network”. A number of tenants can use the same specific network resources at different times and / or simultaneously. The network resources can be located local to the premises of the tenants and / or remote from the premises of the tenants. In a hybrid cloud, the computer network includes a private cloud and a public cloud. The interface between the private cloud and the public cloud allows for the portability of data and applications. Data stored on the private cloud and data stored on the public cloud can be exchanged via the interface. Applications implemented on the private cloud and applications implemented on the public cloud may have dependencies on each other. Calls can be made from an application on the private cloud to an application on the public cloud (and vice versa) via the interface.

[0173] In an embodiment, the tenants of a multi-tenant computer network are independent of each other. For example, one tenant (by operation, tenant-specific practices, employees, and / or identity to the outside world) may be separate from another tenant. Different tenants may have different network requirements for the computer network. Examples of network requirements include processing speed, data storage volume, security requirements, performance requirements, throughput requirements, latency requirements, elasticity requirements, Quality of Service (QoS) requirements, tenant isolation, and / or consistency. The same computer network may need to implement different network requirements requested by different tenants.

[0174] In one or more embodiments, in a multi-tenant computer network, tenant isolation is implemented to ensure that applications and / or data of different tenants do not share with each other. Various tenant isolation methods can be used.

[0175] In an embodiment, each tenant is associated with a tenant ID. Each network resource of the multi-tenant computer network is marked with the tenant ID. A tenant is allowed to access a specific network resource only when the tenant and the specific network resource are associated with the same tenant ID.

[0176] In an embodiment, each tenant is associated with a tenant ID. Each application implemented by the computer network is marked with the tenant ID. Additionally or alternatively, each data structure and / or data set stored by the computer network is marked with the tenant ID. A tenant is allowed to access a specific application, data structure, and / or data set only when the tenant and the specific application, data structure, and / or data set are associated with the same tenant ID.

[0177] For example, each database implemented by the multi-tenant computer network can be marked with the tenant ID. Only the tenant associated with the corresponding tenant ID can access the data of the specific database. As another example, each entry in the database implemented by the multi-tenant computer network can be marked with the tenant ID. Only the tenant associated with the corresponding tenant ID can access the data of the specific entry. However, the database can be shared by multiple tenants.

[0178] In an embodiment, a subscription list indicates which tenants are authorized to access which applications. For each application, a list of tenant IDs of the tenants authorized to access the application is stored. A tenant is allowed to access a specific application only when the tenant ID of the tenant is included in the subscription list corresponding to the specific application.

[0179] In an embodiment, network resources corresponding to different tenants (such as digital devices, virtual machines, application instances, and threads) are isolated from tenant-specific overlay networks maintained by the multi-tenant computer network. As an example, packets from any source device in the tenant overlay network may be sent only to other devices within the same tenant overlay network. Encapsulation tunnels are used to prohibit any transmission from a source device on the tenant overlay network to a device in another tenant overlay network. Specifically, a packet received from the source device is encapsulated within an outer packet. The outer packet is sent from a first encapsulation tunnel endpoint (communicating with the source device in the tenant overlay network) to a second encapsulation tunnel endpoint (communicating with the destination device in the tenant overlay network). The second encapsulation tunnel endpoint decapsulates the outer layer packet to obtain the original packet sent by the source device. The original packet is sent from the second encapsulation tunnel endpoint to the destination device in the same specific overlay network.

[0180] In the foregoing specification, embodiments of the present invention have been described with reference to numerous specific details that may vary with implementation. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive. The sole and exclusive indicator of the scope of the present invention, and what the applicant intends to be the scope of the present invention, is the literal and equivalent scope of the set of claims issued in this application, in the specific form in which such claims are issued, including any subsequent corrections.

Claims

1. One or more non-transitory machine-readable media storing instructions that, when executed by one or more processors, cause: Managing an application through a Plug-in Application Recipe (PIAR), discovering a specific data type within one or more data values of a specific field of a first plug-in application, wherein the specific data type (a) is different from the data type of the specific field reported by the first plug-in application and (b) is narrower than the data type of the specific field while conforming to the data type of the specific field; Among them, The PIAR management application is configured to manage multiple PIAR definitions, each PIAR definition respectively identifying: (a) A trigger that corresponds to a condition that the PIAR management application evaluates on an ongoing basis at least in part based on data provided to the PIAR management application by one or more plug-in applications, and (b) An action to be performed when the condition is met; Through the PIAR management application, identifying one or more mappings between: (a) the specific data type discovered within the one or more data values of the specific field of the first plug-in application and (b) one or more data types among the multiple data types accepted by multiple fields respectively supported by multiple plug-in applications; Through the PIAR management application, presenting a user interface that includes one or more candidate PIAR extensions based on the one or more mappings; Based on a user selection of a specific PIAR extension from the one or more candidate PIAR extensions: Executing a PIAR that includes the specific PIAR extension.

2. The one or more media of claim 1, further storing instructions that, when executed by one or more processors, cause: Storing, through the PIAR management application, metadata describing the multiple fields accepted by the multiple actions; Among them, Identifying the one or more mappings includes accessing the metadata stored by the PIAR management application.

3. The one or more media as recited in claim 1, wherein, Discovering the specific data type within the one or more data values of the specific field of the first plug-in application includes identifying one or more canonical data types in unstructured data received from the first plug-in application.

4. The one or more media according to claim 3, wherein, Identifying the one or more canonical data types in the unstructured data includes applying the unstructured data to a machine learning model.

5. The one or more media of claim 3, wherein, Identifying the one or more canonical data types includes performing semantic analysis on the unstructured data.

6. The one or more media according to claim 3, wherein Identifying the one or more canonical data types includes: Identifying multiple candidate data types in the unstructured data; Presenting the multiple candidate data types in a user interface; Receiving a user selection of a specific data type from the multiple candidate data types.

7. The one or more media according to claim 3, wherein, The one or more canonical data types include one or more of the following: date; name; email address; monetary amount; or physical address.

8. The one or more media of claim 1, further storing instructions that, when executed by one or more processors, cause: Discover, via the PIAR management application, the plurality of fields accepted by the plurality of actions.

9. The one or more media of claim 8, wherein, Discovering the plurality of fields includes: Sending a discovery request to an application programming interface (API) of a second plug-in application among the plurality of plug-in applications; Receiving, from the second plug-in application in response to the discovery request, an object describing one or more fields accepted by one or more operations supported by the second plug-in application.

10. The one or more media of claim 1, further storing instructions that, when executed by one or more processors, cause: Authorize the PIAR management application to access the API of a specific plug-in application at least by: Obtaining an authorization code, via a proxy service running independently of the PIAR management application and the specific plug-in application, the authorization code indicating that a user of the PIAR management application is authorized to use functions of the specific plug-in application; Receiving, via the proxy service, from the PIAR management application a first request to execute the functions of the specific plug-in application for an account associated with the user; In response to the first request to access the functions of the specific plug-in application: Determining, via the proxy service, that the first request to execute the functions of the specific plug-in application is authorized, and Sending, via the proxy service, a second request to execute the functions of the specific plug-in application to the specific plug-in application; And Receiving, via the proxy service in response to the second request to execute the functions of the specific plug-in application, a confirmation that the second request has been authorized to execute the functions on the specific plug-in application.

11. The one or more media of claim 1, further storing instructions that, when executed by one or more processors, cause: Generate one or more PIAR definitions corresponding to the one or more candidate PIAR extensions.

12. The one or more media of claim 1, further storing instructions that, when executed by one or more processors, cause: Determine that the specific field provides a data value including an unreported data type that varies between a first data type and a second data type, Among them, The one or more mappings include a branch mapping that, depending on whether the specific data value of the specific field includes an unreported data type of the first date type or the second date type, maps to a first accepted data type of a first field among the plurality of fields or a second accepted data type of a second field among the plurality of fields.

13. The one or more media according to claim 12, wherein The first field among the plurality of fields is associated with a second plug-in application among the plurality of plug-in applications, and the second field among the plurality of fields is associated with a third plug-in application among the plurality of plug-in applications.

14. The one or more media according to claim 1, wherein, Identifying the one or more mappings includes: Determine that the specific data type corresponds to a specific data pattern; Determine that at least a subset of the multiple data types of the multiple fields corresponds to the specific data pattern.

15. The one or more media according to claim 1, wherein The multiple plugin applications include the first plugin application.

16. The one or more media according to claim 1, wherein Discover that the specific data type and identifying the one or more mappings are performed by the PIAR management application in an execution mode independent of the user.

17. The one or more media according to claim 1, wherein, The PIAR management application is a multi-tenant system.

18. The one or more media as recited in claim 1, wherein, Performing the specific PIAR extension includes performing user assistance for the specific PIAR extension.

19. A system for generating a plugin application recipe extension, comprising: At least one device including a hardware processor; The system is configured to perform operations including the following: Discover a specific data type within one or more data values of a specific field of a first plugin application via a plugin application recipe PIAR management application, wherein the specific data type (a) is different from the data type of the specific field reported by the first plugin application and (b) is narrower than the data type of the specific field while conforming to the data type of the specific field; Wherein the PIAR management application is configured to manage multiple PIAR definitions, each PIAR definition respectively identifying: (a) A trigger that corresponds to a condition that the PIAR management application evaluates on an ongoing basis at least in part based on data provided to the PIAR management application by one or more plugin applications, and (b) An action to be performed when the condition is met; Identify, via the PIAR management application, one or more mappings between: (a) the specific data type discovered within the one or more data values of the specific field of the first plugin application and (b) one or more data types of multiple fields respectively accepted by multiple actions supported by multiple plugin applications; Present, via the PIAR management application, a user interface that includes one or more candidate PIAR extensions based on the one or more mappings; Based on a user selection of a specific PIAR extension from the one or more candidate PIAR extensions: Execute a PIAR that includes the specific PIAR extension.

20. A method for generating a plugin application recipe extension, comprising: Discover a specific data type within one or more data values of a specific field of a first plugin application via a plugin application recipe PIAR management application, wherein the specific data type (a) is different from the data type of the specific field reported by the first plugin application and (b) is narrower than the data type of the specific field while conforming to the data type of the specific field; Wherein the PIAR management application is configured to manage multiple PIAR definitions, each PIAR definition respectively identifying: (a) A trigger, where the trigger corresponds to a condition that is evaluated on an ongoing basis by the PIAR management application at least in part based on data provided to the PIAR management application by one or more plug-in applications, and (b) An action to be performed when the condition is met; Via the PIAR management application, identify one or more mappings between: (a) the specific data type found within the one or more data values of the specific field of the first plug-in application and (b) one or more data types among the multiple data types accepted by the multiple fields respectively supported by the multiple plug-in applications; Via the PIAR management application, present a user interface that includes one or more candidate PIAR extensions based on the one or more mappings; Based on a user selection of a specific PIAR extension from the one or more candidate PIAR extensions: execute a PIAR that includes the specific PIAR extension; Wherein the method is executed by at least one device including a hardware processor.

Images