Agent orchestration on a computing device

Abstract

According to an aspect, a method includes receiving at least a portion of a query on a computing device, selecting an agent from a plurality of agents to perform an action related to at least the portion of the query, initiating display of an interface element as a suggestion for a subsequent action, and, in response to a selection of the interface element, initiating the action using the agent.

Description

Atty Docket No. 0120-1095W01AGENT ORCHESTRATION ON A COMPUTING DEVICE CROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 740,097, filed December 30, 2024, the disclosure of which is incorporated herein by reference in its entirety.BACKGROUND

[0002] A user may use a variety of agents (e.g., generative models, task assistants, etc.) to assist with research, organization, and / or computer actions. The agents may include general purpose agents, action-specific agents provided by software as a service (SaaS) applications, and / or organizational agents (e.g., in-house agents) provided by organizations (e.g., enterprises). For example, an application such as a customer relationship database application may include or be associated with an agent (e.g., an agent or artificial intelligence (Al) agent) that can help with actions associated with the customer relationship database application. A user may use other action-specific agents or general purpose agents for other tasks. These generative models operate independently of each other.SUMMARY

[0003] This disclosure relates to a technical solution of an agent orchestration engine (e.g., executable locally on a computing device or on a server computer) configured to select an agent among a plurality of agents using a query (e.g., a partial query, or an incomplete query) and / or usage data associated with the computing device, and initiate display of an interface element as a suggest for a subsequent action (e.g., an action to be performed by the selected agent) for the next computing action. In response to selection of the interface element, the agent orchestration engine initiates the action using the selected agent. For example, in response to at least a portion of a user query (e.g., “top customer issues for this quarter”), the agent orchestration engine may initiate display an action (e.g., summarize customer issues) of a relevant agent (e.g., an enterprise support agent) as a suggestion for the next computing action to be performed. In response to a user query (e.g., “show recent support cases”), the agent orchestration engine may initiate display anAtty Docket No. 0120-1095W01action (e.g., “generate case overview”) of a relevant agent (e.g., a case management agent) for the next computing action to be performed.

[0004] In some examples, a predictive model suggestion is displayed as an interface element. A predictive model suggestion may be an autosuggestion that is displayed along with other autosuggestions (e.g., predictive search queries, predictive computer resources such as webpages, applications, etc.) in response to submission at least a portion of user query (e.g., a partial user query or an incomplete user query). The examples discussed herein are not limited to browsers, where the agent orchestration engine may be a sub-component of an operating system configured to assist with identifying a relevant agent for the next computer action.

[0005] In some examples, the predictive model suggestion can be referred to as a model action suggestion, an action suggestion by an Al model, or a model output. In some examples, as a user is inputting a user query to an input area (and while the user query is incomplete), the agent orchestration engine may determine which agent (and which action to perform using a selected agent) that is relevant to the portion of the user query, and render an interface element (e.g., a selectable link) as a suggestion for a subsequent action. When selected, the agent orchestration engine prompts the agent to generate a model response (also referred to as an agent response), which is displayed in an interface (e.g., an agent interface) that enables the user to communicate (e.g., communicate directly) with the agent). In some examples, an input area (e.g., a search field of an interface such as a browser window or operating system interface) may operate as a point of entry (e.g., a single point of entry) for interacting with a plurality of agents.

[0006] In some aspects, the techniques described herein relate to a method including: receiving at least a portion of a query on a computing device; selecting an agent from a plurality of agents to perform an action related to at least the portion of the query; initiating display of an interface element as a suggestion for a subsequent action; and in response to a selection of the interface element, initiating the action using the agent.

[0007] In some aspects, the techniques described herein relate to a computing device including: at least one processor; and a non-transitory computer-readable medium storing executable instructions that cause the at least one processor to: receive at least a portion of a query on the computing device; select an agent from a plurality of agents toAtty Docket No. 0120-1095W01perform an action related to at least the portion of the query; initiate display of an interface element as a suggestion for a subsequent action; and in response to a selection of the interface element, initiate the action using the agent.

[0008] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations including: receiving at least a portion of a query on a computing device; selecting an agent from a plurality of agents to perform an action related to at least the portion of the query; initiating display of an interface element as a suggestion for a subsequent action; and in response to a selection of the interface element, initiating the action using the agent.

[0009] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 A illustrates an example of predictive suggestions for a user query, where the predictive suggestions include a predictive model suggestion for an action of an agent according to an aspect.

[0011] FIG. IB illustrates an example of a user interface that is rendered in response to a user selection of the predictive model suggestion according to an aspect.

[0012] FIG. 1C illustrates an example of an agent orchestration engine according to an aspect.

[0013] FIG. ID illustrates an example of manifest data according to an aspect.

[0014] FIG. IE illustrates an example of an agent orchestration engine according to another aspect.

[0015] FIG. IF illustrates an example of generating and displaying predictive suggestions, which includes a predictive model suggestion, according to an aspect.

[0016] FIG. 1G illustrates an example of an agent orchestration engine for generating a predictive model suggestion using usage data according to an aspect.

[0017] FIG. 1H illustrates an example of an agent orchestration engine for generating one or more application groups with one or more agents according to an aspect.Atty Docket No. 0120-1095W01

[0018] FIG. 2A illustrates an example of a management system that uses an agent orchestration engine according to an aspect.

[0019] FIG. 2B illustrates an example of a management system that uses an agent orchestration engine according to another aspect.

[0020] FIG. 3 illustrates an example of enabling an agent to operate with a browser application according to an aspect.

[0021] FIG. 4 illustrates an example of an interface of an application store platform for installing an application of an agent according to an aspect.

[0022] FIG. 5 illustrates an example of an interface of an administrative console application that depicts agent usage data for one or more agents for managed computer devices or managed browser profiles according to an aspect.

[0023] FIG. 6 illustrates an example of a management system that uses connector policies for enabling communicating and authentication with agents according to an aspect.

[0024] FIG. 7 illustrates a flowchart depicting example operations of agent orchestration according to an aspect.DETAILED DESCRIPTION

[0025] The disclosure provides a way to manage and coordinate multiple, different artificial intelligence (Al) assistants (also called agents or models) that a user might have access to on their computer, e.g., across installed applications, while enforcing controlled access, authentication, policy governance, and / or model-selection logic. For example, a user might have a general-purpose Al for answering questions, another Al assistant built into their customer database software, and yet another for helping with coding. These assistants typically do not work together. Existing systems do not provide a unified mechanism to register agents, supply descriptive capability metadata, authorize access through connector policies, route model requests according to user context, or select an appropriate model during real-time user interaction. This produces inconsistent execution behavior, fragmented integration patterns, and / or security exposure when untrusted models are invoked without device posture verification.

[0026] The technical problem may include a lack of a coordinated orchestration layer to manage and securely invoke a set of Al agents from different sources.Atty Docket No. 0120-1095W01Conventional systems lack the necessary architecture to discover, install, configure, and / or authenticate disparate Al agents, leading to technical deficiencies. For example, without such a layer, the computing device inefficiently expends processing cycles and network bandwidth by invoking models that are ill-suited for a particular task. Furthermore, conventional browsers, operating systems, and / or applications do not implement policy controls requirements and / or model capability descriptors before initiating model execution, which can create security vulnerabilities by allowing untrusted models to access data without verification of the device’s security posture. As a result, models may be invoked inefficiently, with incorrect permissions, or without awareness of task context, installed applications, browsing history, and / or device compliance requirements.

[0027] The technical solution may include a management framework that configures Al agents using manifest data, distributes agent or connector policies defining authentication and trust rules, and / or performs model-selection based on user input, browsing context, device posture, and historical usage signals. The management framework may establish a controlled communication channel between the device and a remote or local model endpoint, generate trust payloads containing device signals, verify those signals through an access engine, and may enable model invocation when access rules are satisfied. The technical solution may include infrastructure for policy-driven installation, capability discovery, proactive suggestion rendering, and / or model inference across grouped browser contexts or application collections.

[0028] The technical advantages of this architecture manifest as direct improvements to the functioning of the computing device. The policy-controlled authentication mechanism provides improved security and trust enforcement, preventing unauthorized access to local computer resources. The system reduces latency and the consumption of computational resources (e.g., CPU cycles, memory) by pre-emptively selecting the most appropriate and efficient model for a given task before invocation. The user interface of the computer is also improved by surfacing predictive suggestions that allow the user to bypass multiple manual steps, resulting in a more efficient and effective human-computer interaction. Additional technical advantages include a centralized and secure method for deploying and revoking models, which reduces system vulnerabilities. The architecture enables context-aware selection of the optimal available model andAtty Docket No. 0120-1095W01provides the technical capability to execute Al operations across multiple, disparate web pages or applications as a unified task domain - a function that conventional browsers and operating systems cannot perform. The disclosed system therefore provides a more reliable, scalable, and controllable method for integrating agents into a computing environment, thereby improving the computer's performance, reducing configuration overhead, and enhancing security.

[0029] This disclosure relates to a technical solution of an agent orchestration engine (e.g., executable locally on a computing device or on a server computer) configured to select an agent among a plurality of agents using a user query and / or usage data, and initiate display of a predictive model suggestion (e.g., an action using the selected agent) for the next computing action. For example, the agent orchestration engine may receive and / or store manifest data about a plurality of agents. The manifest data may include an agent description about the capabilities of a respective agent, sample queries, system prompts for different types of actions (e.g., actions, model actions), configuration parameters, permissions, and / or metadata about a respective agent.

[0030] In some examples, at least a portion of the computing device is managed by an organization using an enterprise management platform. In some examples, the computing device is a managed device, where a local agent is installed on the computing device, and the local agent is used to communicate with the enterprise management platform to receive and apply policy controls generated by an administrator of the organization. In some examples, the computing device includes a browser application with a managed profile, where browser activity under the managed profile is managed by the organization according to policy controls applied by the browser application.

[0031] In some examples, the computing device (e.g., a browser application, an operating system) receives, over a network, an agent policy from an enterprise management system. An agent policy may be an example of a policy control. An agent policy may include a list of agents selected by the organization’s administrator that can be used on the computing device or under a managed browser profile. In some examples, the agent policy includes the manifest data. In some examples, the agent policy includes configuration data, which, when executed by the application, causes the agents to be enabled on the computing device, which may include installing a program (e.g., anAtty Docket No. 0120-1095W01extension, web extension, a program, etc.) on the browser application or the operating system. In some examples, the manifest data is stored in the agent orchestration engine.

[0032] In some examples, an administrative console application, executing on an administrator’s computing device, may display a list of agents that can be used by managed devices or managed browser profiles. In some examples, the list of agents may be displayed along with agent usage data about the usage of the agents across computing devices. In response to a selection of one or more of the available agents, the administration console application may initiate the generation of an agent policy, which is transmitted from the enterprise management platform to the relevant computing devices. However, it is noted that the implementations are not limited to enterprise management platforms, where a computing device configured with the agent orchestration engine may be a non-managed computing device.

[0033] An agent may be a task assistant configured to respond to user queries. An agent may be a predictive model defining a neural network configured to receive prompts and generate model responses. An agent may be a component configured to receive user input (such as a query) and perform an action (e.g., generating text, providing data extraction, or initiating a workflow) on behalf of the user, thereby acting as a task assistant to augment a user's computing session. The agents may include generative models, artificial intelligence (Al) agents, Al bots, chat bots, Al chat bots, language models, large language models (LLMs), models, task assistants, or machine-learning (ML) models. The agents may include general -purpose generative models or application-specific models or action-specific models. In some examples, an agent has a browser extension, which, when enabled on the browser application, enables the user to communicate with the agent. In some examples, some of the agents may be associated with applications, where a first agent is an Al agent of a first application (e.g., a website, web resource, native application, etc.), and a second agent is an Al agent of a second application (e.g., a website, web resource, native application, etc.).

[0034] In some examples, the agent orchestration engine includes a model (e.g., an orchestration model) configured to generate a predictive model suggestion for an action to be performed by one of the agents using the manifest data and at least one of a user query (or a portion thereof), usage data (e.g., user activity on the computing device, browsingAtty Docket No. 0120-1095W01activity, etc.), session history, or context data as input(s) to the model. A predictive model suggestion may be displayed as an interface element for an action of an agent. The agent orchestration engine configures (e.g., applies, conditions, or prompts) the model using the manifest data so that the model can determine which agent to identify as a predictive model suggestion to perform an action related to a user query (e.g., received from an input field via typing or voice) and / or related to the user’s usage data. For example, a user may begin entering a user query into an input field, and the agent orchestration engine causes the computing device to display a plurality of predictive suggestions including a predictive model suggestion, which, when selected, is executed by the selected agent.

[0035] The agent orchestration engine may receive a portion of a user query as the user query is being entered in the input field, identify an agent among a plurality of agents for executing an action that is relevant to the user query, and may initiate the display of one or more predictive model suggestions among other predictive suggestions such as sample search queries and / or previously visited computer sources such as previously visited web pages and / or applications (e.g., native applications, web applications, etc.). A predictive model suggestion, generated by the model, identifies an action (e.g., summarize, arrange, generate, etc.), a predictive query such as a programmatically completed query (e.g., “top customer cases for project X”) that is generated using at least a portion of the user query and a particular agent (e.g., “Acme Agent”).

[0036] In some examples, the model generates a prompt that includes the action, the query, and a system prompt for the type of action to be performed by the selected agent. The predictive model suggestion may be an interface element such as a selectable resource locator with parameters, which, when selected, causes an application (e.g., a browser application, operating system, native application, etc.) to render a user interface object (e.g., a side panel) with an interface (e.g., an agent interface, a chat interface) that displays at least a portion of the prompt and a model response generated by the agent. In some examples, the predictive model suggestion, when selected, causes the application to render an application interface of an application (e.g., a web application) associated with the agent. For example, if the agent is an Al agent of a customer relationship database application, in response to a selection of the predictive model suggestion, the computing device may also render the application interface of the customer relationship databaseAtty Docket No. 0120-1095W01application, where the application interface is concurrently displayed with the agent interface.

[0037] FIGS. 1A through 1H illustrate an example of a system 100 with an agent orchestration engine 138 that selects an agent 142 among a plurality of agents 142 using at least a portion of a user query 114 and / or usage data 108 associated with a computing device 102, and initiates display of a predictive model suggestion 116 for the next computing action. The predictive model suggestion 116 may identify an action 122 to be performed using the selected agent 142. In some examples, the predictive model suggestion 116 is an autosuggestion that is displayed along with other predictive suggestions 174 (e.g., autosuggestions such as predictive search queries 176, predictive computer resources 178, etc.). In some examples, the predictive model suggestion 116 is displayed in a tab 110 (e g., a browser tab) of a browser application 106.

[0038] In some examples, as the user is inputting a user query 114 to an input field 112, the agent orchestration engine 138 may determine which agent 142 (and which action 122 to execute using a selected agent 142) that is relevant to at least a portion of the user query 114, and render an interface element (e.g., a selectable link) as a predictive model suggestion 116. In some examples, the user query 114 is an incomplete query or a partial query. In response to selection of the interface element, the agent orchestration engine 138 prompts the agent 142 to perform the action 122 and generate a model response 134. The model response 134 may be displayed in an agent interface 130 associated with the agent orchestration engine 138. The agent interface 130 may be an interface that enables a user to communicate (e.g., communicate directly) with the agent 142. The input field 112 may operate as a point of entry (e.g., a single point of entry) for interacting with a plurality of agents 142.

[0039] The agent orchestration engine 138 may include error handling and fallback logic. For example, if a selected agent 142 fails to return a model response 134 within a predefined time limit or returns an error code, the agent orchestration engine 138 may automatically select a secondary, or fallback, agent from the plurality of agents 142 to perform the action 122. The manifest data 144 may specify one or more fallback models for each primary agent. In some examples, the agent interface 130 may display an errorAtty Docket No. 0120-1095W01message and provide an option for the user to report the low-quality response, which can be used as a negative signal for updating the model 148.

[0040] The agent orchestration engine 138 may be a component that is part of the browser application 106. In some examples, the agent orchestration engine 138 is a component that is part of the operating system 105. The agent orchestration engine 138 includes a model 148. The model 148 may be a neural network-based model configured to identify an agent 142 and an action 122 that is relevant to a user query 114 (including a user query 114 that is at least partially incomplete). The model 148 is a generative model such as a language model, a large language model, or a machine-learning model. In some examples, the model 148 is referred to as an orchestration model configured to identify an action 122 of an agent 142 that is related to at least a portion of a user query 114 and / or usage data 108 on the computing device 102.

[0041] The model 148 may be trained using machine learning techniques on a corpus of user queries paired with corresponding optimal generative models and actions. In some examples, the model 148 is updated on the computing device 102 using reinforcement learning, where the selection history 170 and subsequent user engagement signals with the model response 134 serve as feedback to refine the model's weights 162. For instance, a positive engagement signal (e.g., the user copying text from a model response) may increase the likelihood that the model 148 will select the same agent 142 for a similar query in the future.

[0042] In some examples, the agent orchestration engine 138, including the model 148, is stored locally on the computing device 102 (e.g., in one or more memory devices 103 of the computing device 102). In some examples, the model 148 is stored on a server computer that is remote from the computing device 102. In some examples, the agent orchestration engine 138 receives and / or generates signals representing at least a portion of the user query 114, the usage data 108, and / or context data 140 and transmits or provides the signals to the model 148.

[0043] In some examples, the signal generation, prompt generation and execution (e.g., inference computation) of the model 148 are performed locally on the computing device 102. In some examples, the agent orchestration engine 138 stores selection history 170. The selection history 170 includes combinations of a user query 114 and a predictiveAtty Docket No. 0120-1095W01model suggestion 116. In some examples, the selection history 170 includes an indication on whether the predictive model suggestion 116 is selected by the user. In some examples, the selection history 170 includes one or more engagement signals about the model response(s) 134 generated by the agent(s) 142, such as user interactions with the model response 134, follow-up queries, or the time spent viewing the response. In some examples, the model 148 is updated based on the actual selections made by the user and / or engagement signals about the model responses 134.

[0044] The agent orchestration engine 138 may store selection history 170. The selection history 170 includes combinations of a user query 114 and a predictive model suggestion 116. In some examples, the selection history 170 includes an indication on whether the predictive model suggestion 116 is selected by the user. In some examples, the selection history 170 includes one or more engagement signals about the model response(s) 134 generated by the agent(s) 142. In some examples, the model 148 is updated based on the actual selections made by the user and / or engagement signals about the model responses 134.

[0045] It is noted that a user of the computi ng device 102 and / or the browser application 106 may be provided with controls allowing the user to make an election as to both if and when the system may enable the collection of information representing the usage data 108, the user selection history 170, and / or other signals relating to the user’s use of the computing device 102 and / or the browser application 106. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user of the computing device 102 and / or the browser application 106 may have control over what information is collected about the user, how that information is used, and what information is provided to the user and / or to the server computer.

[0046] Referring to FIG. 1A, the browser application 106 may render a tab 110 (e.g., a browser tab) on the computing device 102. Although some examples depict generating and displaying predictive model suggestions 116 as omnibox suggestions in aAtty Docket No. 0120-1095W01tab 110, the examples discussed herein are not limited to browser applications 106. For example, the input field 112 may be a field on any type of interface such as a callout affordance, an operating system interface, a user interface object, or generally any interface rendered by a computing device 102, including a browser application 106, an operating system 105, or another application 104 executing on the operating system 105. In some examples, the input field 112 is referred to as an input area.

[0047] The tab 110 may include an input field 112-1. In some examples, the input field 112-1 is an address bar of the tab 110. The tab 110 may include an input field 112-2. In some examples, the input field 112-2 is a search field for receiving search criteria to search the Internet. A user may enter a user query 114 in the input field 112-1 or the input field 112-2 to search the Internet and retrieve search results. While the user is submitting the user query 114 (e.g., via text or voice) in the input field 112-1 or the input field 112-2, a predictive input engine 172 may generate, rank, and display predictive suggestions 174 in a user interface object or as a list that is proximate to (e.g., adjacent or below) the input field 112-1 or the input field 112-1. At least a portion of a query 114 may be input received by a computing device 102, which may be a character, a string, a voice segment, a complete query, or an incomplete query. An incomplete query is defined as a user input submitted to the input area where the user has not yet completed the intended full query, and the input is used to generate predictive suggestions 174 before submission.

[0048] In some examples, the predictive input engine 172 is a component of the browser application 106. In some examples, the predictive input engine 172 is a component of the operating system 105. In some examples, the predictive input engine 172 is a component of a non-browser application. The predictive input engine 172 receives one or more predictive model suggestions 116 from the agent orchestration engine 138. The predictive input engine 172 may operate as a standalone module or may include the agent orchestration engine 138 as an integrated component configured to evaluate input signals, select one or more agents 142, and present predictive suggestions 174.

[0049] In some examples, the predictive suggestions 174 may include multiple categories of suggestion types that are programmatically displayed while a user provides input in the input field 112 (e.g., omnibox text entry shown in FIG. 1 A). A predictive suggestion 174 may be an autosuggested completion or recommendation generated basedAtty Docket No. 0120-1095W01on at least a portion of a user query 114 and / or usage data 108. A predictive suggestion 174 may be displayed as soon as one or more characters are typed into the input field 112, and may represent a query completion, a navigation target, a task pathway, or other system-generated recommendation intended to accelerate user input or navigation workflow. A predictive suggestion 174 may be displayed as an interface element (e.g., a selectable interface element), which, when selected, initiates an action. Each predictive suggestion 174 may include a label, identifier, ranking value, resource reference, or other information configured to guide the user toward executing an intended task such as completing a predictive search query 176, or selecting a predictive computer resource 178, or selecting a predictive model suggestion 116.

[0050] The predictive suggestions 174 may include predictive model suggestions 116 generated by the agent orchestration engine 138. A predictive model suggestion 116 may be displayed as an interface element (e.g., a selectable interface element) with information about the corresponding agent 142 and / or action 122. The predictive model suggestions 116 may be inserted into the list of predictive suggestions 174 identified by the predictive input engine 172. A predictive model suggestion 116 may identify an action 122 to be performed by an agent 142 that is related to at least a portion of the user query 114 or the usage data 108. In some examples, the action 122 is referred to as an agent action. In some examples, the action 122 is a single task or a multi-task operation. The action 122 is an action to be performed by at least one of the agents 142. In some examples, the action 122 includes executable instructions that cause the computing device 102 to perform an action. In some examples, the action 122 is a multi-task operation that involves the use of the agent 142 (or multiple agents 142) and the computing device 102.

[0051] A predictive model suggestion 116 is a form of autosuggestion that indicates a candidate task executable by an agent 142. The predictive model suggestion 116 differs from a predictive search query 176 and a predictive computer resource 178 in that the predictive model suggestion 116 presents a query 118 paired with an action 122 selected for execution by one or more agents 142. A predictive model suggestion 116 may be displayed as an interface element. An interface element may be a visual control that presents information about the predictive suggestion 116 and enables the user to select the interface element. The predictive model suggestion 116 may include an applicationAtty Docket No. 0120-1095W01element 111, information identifying a query 118, a model identifier 120, and / or information identifying an action 122.

[0052] The application element 111 is a visual indicator associated with an application interface 128 or agent interface 130, and may include an icon, label, thumbnail, or other graphical element identifying the downstream system that will execute the action 122. The query 118 is model -generated natural language representing the intended semantic meaning of the user query 114. The query 118 may be expanded or reformulated from partial input typed into the input field 112 and may incorporate contextual signals from the usage data 108. In some examples, the query 118 is referred to as a completed query. For example, the user query 114 may be an incomplete query, and the query 118 may represent a predictive query or completed query.

[0053] The action 122 is the specific operation to be performed by the selected agent 142 using the query 118. The action 122 may include generating a summary, extracting structured information, initiating a workflow request, or performing an automated analysis relevant to the user’s task. The query 118 and the action 122 may be ultimately combined to form a prompt 132 that is sent to the agent 142 upon user selection. The model identifier 120 is the identifier that names and distinguishes the agent 142, and may include the model’s name, version, capability class, or configuration reference used to route execution.

[0054] The predictive suggestions 174 may also include predictive search queries 176. A predictive search query 176 may represent an autosuggested search-query proposal derived from the character(s) entered into the input field 112. The predictive suggestions 174 may further include predictive computer resources 178. A predictive computer resource 178 may represent an autosuggested navigation target or a resource endpoint previously accessed or determined relevant to the user query 114 or the usage data 108. Examples of predictive computer resources 178 include prior webpages, web applications, enterprise dashboards, documents, emails, or any resource identifiable by a resource locator.

[0055] In response to selecting the predictive model suggestion 116, the agent orchestration engine 138 may generate a prompt 132 and transmit or provide the prompt 132 to the agent 142 identified in the predictive model suggestion 116. The prompt 132Atty Docket No. 0120-1095W01may include the action 122 and the query 118 and may include a system prompt 144c. The system prompt 144c may be instructions that assist the agent 142 to perform the action 122. The system prompt 144c may be a predefined set of instructions that correspond to the selected agent 142 and / or the action 122. The system prompt 144c may define formatting, tone, safety conditions, reasoning depth, or operational constraints for the agent 142. The system prompt 144c may not be visible to the user but may still influence the resulting output.

[0056] In some examples, the predictive model suggestion 116 includes a resource locator associated with the selected agent 142 or an execution endpoint capable of processing the prompt 132. The agent orchestration engine 138 may use the resource locator to route the prompt 132 to a remote or local model endpoint and receive a corresponding model response 134. The model response 134 may include generated natural language, structured values, task execution results, recommended follow-up steps, or other output, and may be returned in a serialized form such as JSON.

[0057] As shown in FIG. IB, selecting the predictive model suggestion 116 may cause the computing device 102 to render an agent interface 130. The agent interface 130 may display the prompt 132 and the model response 134 produced by the agent 142 and may remain open as a persistent workspace for multi-step interactions, task refinement, or conversational exchanges with the agent 142. The agent interface 130 may be a UI component dedicated to interaction with the agent 142. The agent interface 130 may support conversational dialogue, follow-on queries, inline actions, file or context inputs, and may be updated as additional model responses 134 are received. In some examples, the agent interface 130 is a panel within the browser application 106 that remains visible alongside the current webpage or application 104, allowing the user to work with model output without navigating away.

[0058] In response to a selection of the predictive model suggestion 116, the computing device 102 may also render an application interface 128 of an application 104 associated with the agent 142. The application interface 128 is the interface that enables the user to communicate (e.g., communicate directly) with the application 104. The application interface 128 may be displayed concurrently with the agent interface 130, and, in some examples, both interfaces remain visible at the same time, such as when the agentAtty Docket No. 0120-1095W01interface 130 occupies a side panel and the application interface 128 occupies the primary content region of the tab 110. In some examples, the user may interact with the agent 142 through the agent interface 130 while simultaneously viewing or operating within the application interface 128, enabling the user to execute actions, verify model -generated output, or supply additional content to the predictive model suggestion 116.

[0059] The application interface 128 is the execution environment or content viewport through which the application 104 is accessed. The application interface 128 may include structured data views, navigable pages, form fields, embedded dashboards, selectable UI controls, hyperlinks, text regions, and / or application-specific UI components. In some examples, the application interface 128 displays the results of the action 122, which may include modification of application content based on the query 118, submission of workflow commands to the application 104, or retrieval of additional predictive computer resources 178.

[0060] The application 104 may be awebpage or SaaS application, and the agent 142 may be an Al agent capable of operating on or assisting with tasks related to that application 104. The predictive model suggestion 116 may include a resource locator (such as a URL, API endpoint, or deep-link reference) that connects the query 118 with the location (e.g., appropriate location) in the application 104. The computing device 102 may use the resource locator to load the appropriate view into the application interface 128 and allow direct interaction with relevant data, content, or user workflows.

[0061] The applications 104 may include any type of executable software accessible via the computing device 102. The applications 104 may include one or more browser applications 106 configured to access and display remote information via tabs 110, one or more web applications (including progressive web applications) delivered through the browser application 106, or non-web native applications that execute directly on the operating system 105. A web application may be server-hosted, server-backed, or implemented as a progressive web application with offline capabilities. A non-web application may be locally installed, partially cached, or executed using system-level application programming interfaces (APIs) of the operating system 105.

[0062] The applications 104 may also include native applications developed for specific device platforms, general -purpose desktop applications, or mobile applicationsAtty Docket No. 0120-1095W01designed for smartphone or tablet environments. In some examples, a mobile application is executed on a desktop or laptop computer using a compatible runtime environment. The applications 104 may additionally include Linux applications executed in a containerized or virtualized user environment.

[0063] As shown in FIG. ID, the agent orchestration engine 138 may receive, maintain, and update manifest data 144 for one or more agents 142. The manifest data 144 may define how an agent 142 is contacted, invoked, prompted, or displayed within the browser application 106 and / or the operating system 105. The manifest data 144 may originate from one or multiple sources, including a local configuration file, a downloaded manifest, model metadata packaged with an application 104, an enterprise management platform, and / or an application store platform. The manifest data 144 may persist across user sessions and may be modified over time as new models are enabled or as model behavior evolves.

[0064] The manifest data 144 includes information used by the agent orchestration engine 138 to evaluate and select an agent 142 for a given action 122 and / or query 118. The manifest data 144 may define the actions 122 and queries 118 capable by an agent 142 that is enabled on the computing device 102. These values may be stored as records, files, encoded structures, or runtime profile objects. The agent orchestration engine 138 may reference one or more of these fields when determining how to execute an action 122 or query 118. The manifest data 144 may influence how predictive model suggestions 116 are generated, ranked, or displayed. The manifest data 144 may also be used to determine whether a model is permitted to run, how it should respond, or how its output is formatted into a model response 134. In some examples, different models may include different manifest data 144, allowing the agent orchestration engine 138 to route interactions to the most appropriate model based on context, capability, and / or user preference.

[0065] The manifest data 144 is analogous to an agent manifest and includes details required for a browser application or an operating system to support a third-party agent. The manifest data 144 may include several proposed sections defining the agent's capabilities. Specifically, the agent description 144a explains the agent's capabilities and usage conditions. The configuration parameters 144e are variables that adjust parameters such as endpoints or URLs for the agent 142. The sample queries 144b provide examplesAtty Docket No. 0120-1095W01of common user interactions. The permissions 144f define the required access, such as page content access or access to a browser or an operating system’s surfaces. The system prompt 144c contains model -level prompts designed to maintain response quality, and the agent loops 144d define repeatable workflows. The metadata 144g may include quality checks, model dependencies, tunings, and compliance links.

[0066] In further detail, the agent description 144a includes capability information defining the functional scope of an agent 142. The agent description 144a may identify supported domains (e.g., finance, healthcare, human resources, project management, customer support), supported subject matters (e.g., contract drafting, budgeting, troubleshooting, scheduling, software debugging), accepted input formats (e.g., naturallanguage text, JSON structures, tables, uploaded documents, webpage content), and supported output modalities (e.g., plain-text responses, lists, action commands, structured data, formatted tables, report-style summaries).

[0067] The agent description 144a may include values indicating the suitability of the agent 142 for particular operations. The suitability may be expressed as confidence scores, numeric ratings, Boolean flags, weighted capability rankings, or tags indicating relative strength (e.g., “strong at summarization,” “moderate at code synthesis,” “not recommended for multi-file reasoning”). The agent description 144a may further define performance attributes such as latency expectations, response length limits, mediahandling ability, fact-retrieval capacity, or multi-step reasoning support. The agent description 144a may also include preferred usage contexts and contexts where execution is not recommended.

[0068] The sample queries 144b include example input strings or prompt templates illustrating how an agent 142 is intended to be invoked. Sample queries 144b may demonstrate phrasing, structure, or input style expected by the agent 142, and may include example tasks, example instructions, or representative request patterns. The sample queries 144b may be used by the predictive input engine 172 to form or refine query 118 and may also assist the agent orchestration engine 138 in producing predictive model suggestions 116 that align with the capabilities described in the agent description 144a.

[0069] The system prompts 144c are predefined instruction sets that shape how an agent 142 behaves when processing a prompt 132. A system prompt 144c may establishAtty Docket No. 0120-1095W01constraints on response style (e.g., concise, bullet-formatted, step-by-step), define required output structure (e.g., JSON object, table, action command), specify safety or correctness conditions, or set limits on reasoning depth or content scope. A system prompt 144c may also instruct the agent 142 to follow task-specific behavior, such as providing summaries, extracting key fields, generating step sequences, or producing follow-up recommendations. In some examples, a system prompt 144c may include preference weights, template wording, or control directives that ensure consistent and predictable responses across multiple interactions.

[0070] The agent loops 144d define repeatable instruction sequences that an agent 142 may execute over multiple steps without requiring new user input. An agent loop 144d may specify how a task is broken into stages, how intermediate results are evaluated, and what conditions trigger continuation, refinement, or termination of the loop. In some examples, an agent loop 144d includes retry logic, iterative improvement rules, follow-up prompt generation, or stepwise execution required to complete a complex workflow. An agent loop 144d may support tasks such as repeatedly summarizing new content as it appears, extracting updated values from a changing document, monitoring a data source, or refining a generated response until a threshold quality or confidence level is met.

[0071] The configuration parameters 144e define operational values used to control the behavior of an agent 142 during execution. The configuration parameters 144e may include maximum input size, maximum output length, execution time limits, retry counts, temperature or randomness settings, output formatting templates, and / or rate-limit thresholds. The configuration parameters 144e may define routing address information, authentication material, content-type negotiation, and / or model-selection rules. In some examples, configuration parameters 144e enable dynamic adjustment of model behavior based on device capability, network responsiveness, user preference, or instruction quality.

[0072] The permissions 144f define what data, surfaces, and system capabilities an agent 142 is allowed to access. Permissions 144f may authorize or restrict access to webpage content, tabs 110, clipboard data, local files, user identity information, application data stores, or system-level functions. Permissions 144f may further specify allowable action categories, such as read-only retrieval, content modification, command execution, information insertion, or task automation. In some examples, permissions 144fAtty Docket No. 0120-1095W01may include scope boundaries, expiration conditions, revocation triggers, and / or escalation requirements before an agent 142 can perform higher-risk actions.

[0073] The metadata 144g includes supplemental information describing attributes of an agent 142. The metadata 144g may include version identifiers, update timestamps, supported response formats, supported languages, safety tiering, observed accuracy, model size, evaluation benchmarks, latency expectations, and / or historical performance indicators. The metadata 144g may also include compatibility tags indicating whether the agent 142 supports structured output, multi -turn reasoning, rich media handling, and / or domain-specific task execution.

[0074] The agents 142 may be referred to as artificial intelligence (Al) agents, agents, Al bots, Al chat bots, language models, large language models (LLMs), models, or machine-learning (ML) models. The agents 142 may include general-purpose generative models or application-specific models or action-specific models. In some examples, an agent 142 has a browser extension, which, when enabled on the computing device 102 (e.g., the browser application 106, the operating system 105), enables the computing device 102 to communicate with the agent 142. In some examples, some of the agents 142 may be associated with applications 104, where a first agent (e.g., agent 142-1) is an Al agent of a first application (e.g., application 104-1) (e.g., a website, web resource, native application, etc.), and a second agent (e.g., an agent 142-2) is an Al agent of a second application (e g., application 104-2) (e.g., a website, web resource, native application, etc ). In some examples, the agents 142 include one or more general-purpose agents (e.g., agent 142-3).

[0075] The agent orchestration engine 138 includes a model 148 configured to generate a predictive model suggestion 116 identifying an action 122 to be performed by one of the agents 142. The model 148 may receive manifest data 144 together with input data that includes a user query 114, usage data 108, session history 168, and / or context data 140.

[0076] The model 148 that performs the agent selection may be implemented as a local agent orchestrator model configured to run on the computing device 102 itself, potentially using on-device technology such as an advanced local language model. This local model is able to receive a typed text query 114 and the list of agents 142 provided byAtty Docket No. 0120-1095W01the organization. Alternatively, the orchestration engine 138 may have a more powerful remote orchestration option (e.g., an online model) that augments or replaces the local model. In both cases, the model 148 refines the input and selects the correct agent 142 to route the request to, based on the context available from the query and previous user session information.

[0077] The model 148 is a selection model that determines which agent 142 is suitable for an action 122 or query 118. The model 148 may classify the user query 114, compare manifest data 144 for multiple agents 142, and output a predictive model suggestion 116 identifying a target agent 142. In some examples, the model 148 operates as a trained machine-learning model, a rules-based evaluator, a ranking engine, or a combined decision system. The model 148 is defined by model configuration data 161. The model configuration data 161 includes the weights 162 of the model 148. The weights 162 are parameter values stored in the model configuration data 161 that influence how the model 148 produces predictive model suggestions 116. The agent orchestration engine 138 may receive the model configuration data 161 as part of an installation package, a downloadable model update, a runtime-generated configuration profile, or another provisioning mechanism.

[0078] The agent orchestration engine 138 configures the model 148 using the manifest data 144 so that the model 148 can select an agent 142 to execute an action 122 associated with a user query 114 or inferred from the usage data 108. The manifest data 144 may inform suitability fortask types, required permissions 144f, supported input formats, expected output modality, or operational constraints. Configuration of the model 148 may include storing the manifest data 144 as model configuration data 161, conditioning the model 148 using information derived from the agent description 144a, or updating the weights 162 based on new agents 142 added to or removed from the computing device 102. The model 148 may be reconfigured dynamically when manifest data 144 changes, allowing predictive model suggestions 116 to adapt as model availability, capability, or context evolves.

[0079] The agent orchestration engine 138 receives portions of the user query 114 as the user query 114 is entered into the input field 112. As input is received, the agent orchestration engine 138 evaluates the user query 114, determines a tentative task type, andAtty Docket No. 0120-1095W01identifies an agent 142 from among a plurality of agents 142 for performing an action 122 relevant to the user query 114. Based on this evaluation, the agent orchestration engine 138 initiates display of one or more predictive model suggestions 116. The predictive model suggestions 116 may be displayed concurrently with predictive search queries 176 and predictive computer resources 178 within the predictive suggestion region associated with the input field 112. A predictive model suggestion 116 generated by the model 148 identifies an action 122, a query 118 derived from the user query 114, and a selected agent 142 determined to be capable of performing the action 122. The predictive model suggestion 116 may update dynamically as additional characters are entered or context data 140 changes.

[0080] In some examples, the model 148 generates a prompt 132 that includes the action 122, the query 118, and a system prompt 144c for the action 122. The predictive model suggestion 116 may be displayed as a selectable interface element, which, when selected, causes the browser application 106 or the operating system 105 (or a non-browser application) to render a user interface element such as a side panel containing the agent interface 130. The agent interface 130 may display a portion of the prompt 132 and a model response 134 generated by the agent 142. In some examples, when the agent 142 is associated with an application 104, selection of the predictive model suggestion 116 causes the application interface 128 of the application 104 to be displayed, while the agent interface 130 remains concurrently visible, allowing interaction with the agent 142 while viewing or manipulating the content associated with the query 118, as illustrated in FIG. IB.

[0081] In some examples, the agent orchestration engine 138 may generate and display a predictive model suggestion 116 based on the usage data 108 of the computing device 102, without receiving a user query 114 from the input field 112.

[0082] The usage data 108 includes information describing prior interactions on the computing device 102. The usage data 108 may include browsing activity, visited webpages, opened files, application usage sequences, timestamps, recurring user tasks, previously entered queries, user selections of predictive suggestions 174, and / or actions executed within applications 104. The usage data 108 may be generated by detecting user interactions during active sessions, by recording task history through the browserAtty Docket No. 0120-1095W01application 106 or operating system 105, or by receiving structured activity data from applications 104 installed on the computing device 102. The usage data 108 may be stored locally, stored remotely, or maintained as a rolling record of past activity.

[0083] Based on the usage data 108, the agent orchestration engine 138 may proactively initiate generation and display of a predictive model suggestion 116. A predictive model suggestion 116 may be triggered when the usage data 108 indicates that an agent 142 can perform an action 122 relevant to a recent or ongoing task, even if the user has not typed input into the input field 112.

[0084] As shown in FIG. 1G, the agent orchestration engine 138 may receive one or more signals derived from the usage data 108. The signals may cause the agent orchestration engine 138 to generate a predictive model suggestion 116 and to display the suggestion within a callout affordance 190. The callout affordance 190 may be a user interface element such as a menu entry, a clickable icon, a hover-activated element, or a pop-out interface component. The agent orchestration engine 138 may instruct the browser application 106 or operating system 105 to display the callout affordance 190 in a visible region of a browser window or other interface element. In some examples, the callout affordance 190 is shown within a tab strip. In other examples, the callout affordance 190 is overlaid on top of existing content within the browser window so that the predictive model suggestion 116 is viewable and discoverable during user activity.

[0085] Referring to FIG. 1H, the agent orchestration engine 138 may generate one or more application groups 192 based on detected relationships among two or more tabs, windows, files, or applications 104 that are active on the computing device 102. An application group 192 may be formed when resources share a common topic, entity, task objective, or workflow pattern. The agent orchestration engine 138 may evaluate content signals such as page text, tab titles, metadata, URL paths, search terms, file names, or application-specific context to identify related resources. In some examples, the application group 192 is generated dynamically as additional tabs or applications are opened, closed, or modified, and the composition of the application group 192 may change in real time as subject matter relevance shifts.

[0086] In some examples, FIG. 1H depicts multiple browser tabs that are grouped within an application group 192 because the subject matter displayed in each tab 110 isAtty Docket No. 0120-1095W01semantically related. The agent orchestration engine 138 may use the application group 192 as a context signal to generate a predictive model suggestion 116. The predictive model suggestion 116 may be generated by the model 148 to perform an action 122 across the application group 192, such as summarizing content across multiple tabs, extracting entities appearing in more than one tab, generating a task list based on information displayed in separate resources, or drafting output that references data pulled from multiple related sources. The predictive model suggestion 116 may be displayed as a callout affordance 190 or within an agent interface 130, and selection of the predictive model suggestion 116 may cause the agent 142 to operate on data associated with at least two resources within the application group 192.

[0087] The computing device 102 may be any type of computing device that includes one or more processors 101, one or more memory devices 103, and an operating system 105 configured to execute (or assist with executing) one or more applications 104. In some examples, the computing device 102 includes a display 160. In some examples, the computing device 102 does not include a display 160. In some examples, the computing device 102 is a laptop or desktop computer. In some examples, the computing device 102 is a tablet computer. In some examples, the computing device 102 is a smartphone. In some examples, the computing device 102 is awearable device. The display 160 is the display of the computing device 102. The display 160 may also include one or more external monitors that are connected to the computing device 102.

[0088] The operating system 105 is a system software that manages computer hardware, software resources, and provides common services for computing programs. In some examples, the operating system 105 is an operating system designed for a larger display 160 such as a laptop or desktop (e.g., sometimes referred to as a desktop operating system). In some examples, the operating system 105 is an operating system for a smaller display 160 such as a tablet or a smartphone (e g., sometimes referred to as a mobile operating system).

[0089] The processor(s) 101 may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof. The processor(s) 101 can be semiconductor-based - that is, the processors can include semiconductor material that can perform digital logic. The memoryAtty Docket No. 0120-1095W01device(s) 103 may include a main memory that stores information in a format that can be read and / or executed by the processor(s) 101. The memory device(s) 103 may include one or more random-access memory (RAM) devices and / or one or more read-only memory (ROM) devices. The memory device(s) 103 may store applications (e.g., the operating system 105, applications 104, etc.) and modules (e.g., agent orchestration engine 138) that, when executed by the processors 101, perform certain operations.

[0090] An agent 142 and / or a model 148 is a predictive model. In some examples, an agent 142 and / or a model 148 includes a neural network. The agent 142 and / or a model 148 may be an interconnected group of nodes, each node representing an artificial neuron. The nodes are connected to each other in layers, with the output of one layer becoming the input of a next layer. The agent 142 and / or a model 148 transforms an input, received by the input layer, transforms it through a series of hidden layers, and produces an output via the output layer. Each layer is made up of a subset of the set of nodes. The nodes in hidden layers are fully connected to all nodes in the previous layer and provide their output to all nodes in the next layer. The nodes in a single layer function independently of each other (e.g., do not share connections). Nodes in the output provide the transformed input to the requesting process. In some examples, an agent 142 and / or a model 148 is a convolutional neural network, which is a neural network that is not fully connected.Convolutional neural networks therefore have less complexity than fully connected neural networks. Convolutional neural networks can also make use of pooling or max pooling to reduce the dimensionality (and hence complexity) of the data that flows through the neural network and thus this can reduce the level of computation required. This makes computation of the output in a convolutional neural network faster than in neural networks.

[0091] In some examples, an agent 142 and / or a model 148 is a deep neural network (DNN). For example, a deep neural network (DNN) may have one or more hidden layers disposed between the input layer and the output layer. However, an agent 142 and / or a model 148 may be any type of artificial neural network (ANN) including a convolution neural network (CNN). The neurons in one layer are connected to the neurons in another layer via synapses. Each synapse is associated with a weight 162. A weight 162 is a parameter within an agent 142 and / or a model 148 that transforms input data within the hidden layers. As an input enters the neuron, the input is multiplied by a weightAtty Docket No. 0120-1095W01value and the resulting output is either observed or passed to the next layer in an agent 142 and / or a model 148. For example, each neuron has a value corresponding to the neuron’s activity (e.g., activation value). The activation value can be, for example, a value between 0 and 1 or a value between -1 and +1. The value for each neuron is determined by the collection of synapses that couple each neuron to other neurons in a previous layer. The value for a given neuron is related to an accumulated, weighted sum of all neurons in a previous layer. In other words, the value of each neuron in a first layer is multiplied by a corresponding weight and these values are summed together to compute the activation value of a neuron in a second layer. Additionally, a bias may be added to the sum to adjust an overall activity of a neuron. Further, the sum including the bias may be applied to an activation function, which maps the sum to a range (e.g., zero to 1). Possible activation functions may include (but are not limited to) rectified linear unit (ReLu), sigmoid, or hyperbolic tangent (TanH).

[0092] While many examples describe the agent orchestration engine 138 operating within a browser or desktop operating system, the techniques described herein are applicable to other computing environments. For example, the agent orchestration engine may be implemented on a mobile device (e.g., a smartphone or tablet), where it suggests actions based on user activity within mobile applications. In other examples, the engine may operate within a voice-controlled assistant, selecting an agent to answer a spoken query. Further implementations include in-vehicle infotainment systems, augmented reality displays, or other smart devices where multiple agents may be available to assist a user.

[0093] FIGS. 2A and 2B illustrates an example of a management system 200 that uses an agent orchestration engine 238 according to an aspect. The management system 200 may be an example of the system of FIGS. 1A to 1H and may include any of the details discussed with reference to those figures.

[0094] In some examples, as the user is inputting a user query 214 to an input field 212, the agent orchestration engine 238 may determine which agent 242 (and which action to execute using a selected agent 242) that is relevant to at least a portion of the user query 214, and render a selectable element (e.g., a selectable link) as a predictive model suggestion 216. When selected, the agent orchestration engine 238 prompts the agent 242 to perform the action and generates a model response. In some examples, an input fieldAtty Docket No. 0120-1095W01212 (e g., a search field of a tab 210 or an input field of an interface of an operating system 205) may operate as a point of entry (e.g., a single point of entry) for interacting with a plurality of agents 242.

[0095] In some examples, the predictive model suggestions 216 may be included as part of predictive suggestions 274 while the user is submitting the user query 214 (e.g., via text or voice) in the input field 212. For example, the browser application 206 or the operating system 205 may include a predictive input engine 272 that generates, ranks, and displays predictive suggestions 274 as the user submits the user query 214 via the input field 212. The predictive suggestions 274 may include one or more predictive model suggestions 216, as well as predictive search queries and / or predictive computer resources.

[0096] In some examples, at least a portion of a computing device 202 is managed by an organization using an enterprise management platform 254. In some examples, the computing device 202 is a managed device. A policy manager 219 (e.g., a local agent) may be installed on the computing device 202. The policy manager 219 is used to communicate with the enterprise management platform 254 to receive (over a network 250) and apply policy controls 294 from an enterprise management platform 254. In some examples, the policy controls 294 are generated by an administrator of the organization using a computing device 233. In some examples, the computing device 202 includes a browser application 206 with a managed profile 225a, where browser activity under the managed profile 225a is managed by the organization according to policy controls 294 applied by the browser application 206. In some examples, the policy manager 219 is a component of the browser application 206.

[0097] The policy controls 294 may include an allow list identifying applications 204 permitted for installation on the computing device 202. The allow list may specify application identifiers, version ranges, required permissions, or compatibility constraints. Some of the applications 204 may be obtained from an application store platform 223. The application store platform 223 may be a platform (e.g., a distribution platform) that provides access to downloadable software packages. The application store platform 223 may host applications 204, agents 242, browser extensions, configuration bundles, or installation artifacts. The application store platform 223 may publish metadata describing each application 204, including version information, publisher information, supportedAtty Docket No. 0120-1095W01device types, required permissions, and installation size. The computing device 202 may retrieve installation packages, manifests, or registration entries from the application store platform 223 using a network connection or a local repository synchronized from the platform.

[0098] In some examples, the policy controls 294 include an installation list of applications 204, including one or more agents 242, that instructs the computing device 202 to install applications 204 programmatically without requiring user interaction. The agent policy 296 may include the installation list within the configuration data 256, enabling the policy manager 219 to automatically install the agents 242 or to install client code on the computing device 202 that enables communication with the agents 242.Installation of an agent 242 may include installation of an agent component, such as a browser extension, web application, or executable integration module, which allows the browser application 206 or operating system 205 to communicate with the agent 242. In some examples, the agent 242 executes locally on the computing device 202. In other examples, the agent 242 executes on a remote server system, and the installed agent component enables message exchange between the computing device 202 and the agent 242.

[0099] An agent policy 296 may be an example of a policy control 294. An agent policy 296 may include a list 217 of agents 242 selected by the organization’s administrator that can be used on the computing device 202 or under the managed profde. The agent policy 296 includes the manifest data 244. In some examples, the agent policy 296 includes configuration data 256, which, when executed by the browser application 206 or the operating system 205, causes the agents 242 to be enabled on the computing device 202, which may include installing a program (e.g., an extension, web extension, a program, etc.) or a native application on the browser application 206 or the operating system 205.

[0100] In some examples, the enterprise management platform 254 receives the manifest data 244 from a computing device 213 associated with a provider or an organization. A provider may upload the manifest data 244 to the enterprise management platform 254, and the published manifest data 244 may subsequently be included within an agent policy 296 distributed to computing devices 202 associated with the organization. InAtty Docket No. 0120-1095W01some examples, the manifest data 244 is stored within or synchronized to the agent orchestration engine 238 on the computing device 202.

[0101] In some examples, the manifest data 244 is received from the application store platform 223 when the agent 242 is installed on the computing device 202.Installation of the agent 242 may include installation of an agent component, such as a browser extension, web application, or integration module, within the browser application 206 or the operating system 205. The agent component may enable communication between the computing device 202 and the agent 242. In some examples, the agent 242 executes locally on the computing device 202. In other examples, the agent 242 executes on a remote server system, and the installed agent component enables transmission of prompts and receipt of model responses.

[0102] In some examples, one or more agents 242 are associated with corresponding applications 204. An agent 242-1 may operate as an Al agent for an application 204-1, and an agent 242-2 may operate as an Al agent for an application 204-2. The associated applications 204 may include websites, web resources, native applications, or other executable programs that provide an interaction surface for a user task. In addition to application-specific agents, the computing device 202 may store or access one or more general-purpose agents 242-3 configured to perform tasks across multiple applications 204 or without reference to a specific application context.

[0103] The application 204-1 may include or install an agent component configured to communicate with the agent 242-1, and the application 204-2 may include or install an agent component configured to communicate with the agent 242-2.Downloading or installation of an application 204-1 may include receiving the agent component for the corresponding agent 242-1 so that communication with the agent 242-1 is enabled through the browser application 206 or the operating system 205. In some examples, installation of the application 204-1 further includes receiving manifest data 244 defining the capabilities of the agent 242-1.

[0104] The agent orchestration engine 238 may configure a model 248 using the manifest data 244. The manifest data 244 may identify supported task types, input modalities, output forms, required permissions, and / or performance characteristics for each agent 242. The model 248 may use the manifest data 244 to evaluate a user query, extractAtty Docket No. 0120-1095W01relevant features from the user query, and generate a generative-model selection result identifying one of the agents 242 for execution.

[0105] The model 248 may be configured by storing the manifest data 244 as model configuration data, conditioning the model 248 using descriptor attributes, or updating parameter weights to bias selection outcomes toward agents 242 that align with the task type, application context, or historical success rate for similar inputs. During operation, the agent orchestration engine 238 may execute the model 248 to determine an agent 242 capable of performing an action associated with a user query 214 or inferred from computer activity patterns recorded on the computing device 202.

[0106] In some examples, an administrative console application 235 executing on a computing device 233 may render a list 217 of agents 242 that are available for use by managed computing devices 202 or managed profiles 225a on a user interface 237. The list 217 of agents 242 may be displayed together with corresponding agent usage data 258, which reflects operational activity associated with the agents 242 across the computing devices 202 within an organization. An administrator may select one or more agents 242 from the list 217, and in response to the selection, the administrative console application 235 may cause an agent policy 296 to be generated and transmitted through the enterprise management platform 254 to one or more computing devices 202.

[0107] The agent usage data 258 is information describing use of the agents 242 across computing devices 202. The agent usage data 258 may indicate how frequently an agent 242 has been invoked, how many predictive model suggestions 116 have been selected by users, which applications 204 the agent 242 was used with, and the types of actions performed by the agent 242. The agent usage data 258 may also include temporal information, performance measurements, error counts, completion success rates, and / or model interaction durations. In some examples, the agent usage data 258 includes aggregated or anonymized summaries of user interactions with the agents 242, and the agent usage data 258 may be provided to the administrative console application 235 in real time or as scheduled synchronization updates.

[0108] The browser application 206 may be associated with one or more profdes 225. A profde 225 is associated with a user account 241. A profde 225 may define settings 227 for a person associated with the user account 241. The settings 227 may include aAtty Docket No. 0120-1095W01wide variety of settings such as privacy options, security settings, search engine preferences, and / or autofill and autocomplete behavior. Also, the settings 227 may include information such as the user’s bookmarks, passwords, search history, favorites, and / or which applications 204 (e.g., extensions, web applications, etc.) are enabled or installed. In some examples, multiple profiles 225 can exist for a single user account 241, where each user can create a new profile 225 that stores the user’s preferences and settings and may store information about the user’s use of the browser application 206.

[0109] A user may have multiple profiles 225, including a managed profile 225a and a non-managed profile 225b. A window 229-1 operating under the managed profile 225a may include one or more tabs 210 associated with settings 227a. A window 229-2 operating under the non-managed profile 225b may include one or more tabs 210 associated with settings 227b. Each profile 225 maintains its own configuration state, browsing data, installed applications 204, and local storage objects separately from other profiles on the same computing device 202.

[0110] The non-managed profile 225b may correspond to a user profile associated with a non-managed user account 241b. The non-managed profile 225b includes settings 227b that are controlled by the user (e.g., entirely controlled by the user). When the browser application 206 is signed into using the non-managed user account 241b, the user may install applications 204, select bookmarks, save passwords, synchronize browsing history, or access content associated with external file storage services. The resulting data may be stored as part of the settings 227b of the non-managed profile 225b. The nonmanaged profile 225b operates without policy -based restriction, and agents 242 available to the non-managed profile 225b may be selectable or installable without approval from an administrative entity.

[0111] The managed profile 225a may correspond to a user profile associated with a managed user account 241a. The managed profile 225a includes settings 227a, which may differ from settings 227b in access controls, installation privileges, or permitted applications 204. When the browser application 206 is operating under the managed user account 241a, the policy controls 294 govern which applications 204, extensions, or agents 242 are enabled. The policy controls 294 are defined by an administrator of an organization and retrieved by the policy manager 219 on the computing device 202. TheAtty Docket No. 0120-1095W01policy controls 294 include the agent policy 296, which may define installation permissions, model availability, and communication access to agents 242.

[0112] When the browser application 206 is operating under the managed profile 225a, the agent orchestration engine 238 may enable or restrict use of agents 242 according to the agent policy 296. The agent orchestration engine 238 may reference the manifest data 244 and the policy controls 294 to determine whether an agent 242 is permitted to process a user query, appear as a predictive model suggestion 216, or be displayed within a predictive suggestion region. The agent orchestration engine 238 may suppress predictive model suggestions 216 for agents 242 that are not authorized by the agent policy 296 and may select or prioritize agents 242 that are explicitly allowed for the managed profile 225a. In some examples, the agent orchestration engine 238 applies different model-selection logic, access rules, or ranking parameters for the managed profile 225a than for the nonmanaged profile 225b.

[0113] The computing device 202 may communicate with the server computer 252 over the network 250. The server computer 252 may be computing devices that take the form of a number of different devices, for example a standard server, a group of such servers, or a rack server system. In some examples, the server computer 252 may be a single system sharing components such as processors and memories. The network 250 may include the Internet and / or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, satellite network, or other types of data networks. The network 250 may also include any number of computing devices (e.g., computer, servers, routers, network switches, etc.) that are configured to receive and / or transmit data within network 250. Network 250 may further include any number of hardwired and / or wireless connections.

[0114] The server computer 252 may include one or more processors formed in a substrate, an operating system and one or more memory devices. The memory devices may represent any kind of (or multiple kinds of) memory (e.g., RAM, flash, cache, disk, tape, etc.). In some examples (not shown), the memory devices may include external storage, e.g., memory physically remote from but accessible by the server computer 252. The server computer 252 may include one or more modules or engines representing specially programmed software.Atty Docket No. 0120-1095W01

[0115] The computing device 233 may be an example of the computing device 202 and may include any of the features discussed with reference to the computing device 202. For example, the computing device 233 may be a laptop or a desktop computer. In some examples, the computing device 233 may be a tablet or a smartphone. The computing device 233 may include one or more processors and one or more memory devices. In some examples, the computing device 233 is associated with an administrator of an organization. For example, the administrator may be associated with an organization that owns or manages the computing device 202. For example, the computing device 202 (and the computing device 233) may be an enterprise-owned computing device such as a work computer owned or managed by the user’s company or a school computer owned or managed by the user’s school.

[0116] FIG. 3 illustrates an example of enabling an agent 342 to operate with a browser application. As shown in FIG. 3, a tab 310 includes an extension element 363 associated with the agent 342. The extension element 363 may appear in a toolbar, tab header, omnibox region, or other browser-resident UI region, and may be selectable to invoke one or more capabilities provided by the agent 342. In some examples, the browser application renders a permission interface 321 responsive to installation of the agent 342 or selection of the extension element 363, where the permission interface 321 defines one or more permissions to access browsing context data.

[0117] The permissions may include read-access to content of a web page rendered in the tab 310, write-access to modify elements of the page, access to user input submitted into an input field, access to network request metadata, or access to browser-stored state such as cookies, local storage entries, history items, and / or or cached resources. In some examples, granting the permission interface 321 enables the agent 342 to receive page context data that includes document object model (DOM) structure, hyperlink URLs, visible text tokens, image metadata, form values, and programmatic event streams associated with user interactions. The agent 342 may process the page context data to generate a model output that includes an action or suggested command capable of operating on the browser application, such as filling form fields, rewriting selected text, generating a summary of page content, composing an email draft, initiating a search query, or providing a contextually-grounded model response window.Atty Docket No. 0120-1095W01

[0118] In some examples, the agent 342 may execute remotely while a client-side agent executes within the browser process to mediate communication with a remote model endpoint. The client-side agent may serialize the page context data, transmit the serialized context data to a remote inference system, receive a generated inference output from the agent 342, and cause rendering of an interface element associated with the generated inference output within the tab 310. In some examples, upon receiving user selection of the extension element 363 or an agent suggestion, the browser transmits a model-invocation request containing at least a portion of the page context data, a request type (e.g., summarization, rewrite, query-assist), and configuration metadata that identifies one or more model settings. In some examples, the permission interface 321 is persistent after installation so that subsequent invocations of the agent 342 operate without re-prompting for new permissions, unless revoked or reconfigured by a policy manager.

[0119] In some examples, the extension element 363 is displayed when (e.g., only when) a policy manager 219 has enabled the agent 342 under a managed profile and has not disabled execution privileges for the agent within the tab 310. In this way, FIG. 3 demonstrates an integrated gateway through which an agent may read and modify browser-hosted content and perform actions within the context of a rendering surface controlled by the browser application.

[0120] FIG. 4 illustrates an example of an interface 465 of an application store platform 223 for installing an application 404 associated with an agent 442. The interface 465 may present a catalog of applications that are searchable, filterable, and categorized according to functional classification such as productivity, communication, planning, education, development, entertainment, social media, or other categories as depicted. The interface 465 may expose installation affordances for selecting the application 404, viewing application metadata, and / or initiating installation of an agent component that enables the browser application 206 or the operating system 205 to communicate with the agent 442. The application 404 may include an extension or executable module that, once installed, registers a capability set, a permission requirement set, and a communication endpoint corresponding to a resource locator of the agent 442.

[0121] In some examples, selection of the application 404 via the interface 465 triggers transmission of an installation instruction to the computing device 202, such thatAtty Docket No. 0120-1095W01the computing device 202 retrieves installation content from the application store platform 223, writes the installation content to local memory, and initializes the agent component. The agent component may include an interface layer configured to send queries to the agent 442, receive model responses, negotiate authentication events, and manage permission scopes requested by the agent 442. When the application 404 is installed, the computing device 202 may generate local registration records indicating installation state, version information, capability declarations, and an identifier of the agent 442, which can be consumed by a policy manager 219 or an agent orchestration engine 238.

[0122] In some examples, installation of the application 404 also results in automatic receipt of manifest data 244, system prompts, model identity data, or configuration data 256 from the application store platform 223. The manifest data 244 may define operational characteristics of the agent 442 including input format requirements, supported action types, resource locators, or execution constraints. The application store platform 223 may publish updated descriptor data 244 for retrieval by managed devices or distributed agent policies 296, enabling the computing device 202 to dynamically adapt agent selection behavior, permission enforcement, policy control rules, and routing logic for request execution.

[0123] In further examples, the interface 465 may indicate whether the application 404 is permitted for installation under an enterprise management configuration. For instance, an enterprise-managed browser profile may only surface applications permitted by an agent policy 296, and the interface 465 may visually denote applications authorized for enterprise deployment. A user operating outside a managed profile may select the application 404 independently, in which case installation may proceed without enterprise mediation, while a managed profile may cause automatic installation of the application 404 or automatic provisioning of the agent component according to the configuration data 256. The interface 465 may additionally store ratings, usage counts, domain approvals, or organization-specific availability indicators, providing contextual data to guide deployment and policy evaluation.

[0124] FIG. 5 illustrates an example of an interface 537 of an administrative console application that depicts agent usage data 558 for one or more agents 542. The interface 537 may present a dashboard view displaying each agent 542 as an entry row orAtty Docket No. 0120-1095W01card, along with corresponding usage metrics, historical activity summaries, policy status indicators, and availability information. The agent usage data 558 may include model invocation frequency, number of predictive model suggestions surfaced to managed devices, number of user selections of predictive model suggestions, average response latency, completion success rate, and / or distribution of action types executed by a respective agent 542. In some examples, the interface 537 displays usage statistics over selectable time ranges and may differentiate usage by device class, profile type, or installed application 404.

[0125] In some examples, the interface 537 enables an administrator to enable, disable, prioritize, or revoke access to an agent 542 based on the agent usage data 558. The interface 537 may also identify agents 542 that exceed usage thresholds, display low-usage models that may be candidates for de-provisioning, or highlight newly installed models exhibiting rapid adoption across managed devices. The administrative console application may update the interface 537 continuously or at synchronization intervals as new agent usage data 558 is received from devices enrolled under policy controls.Selection of an agent 542 within the interface 537 may open a detailed view including model identifier data, descriptor values, configuration state, version history, recent model responses, and links to modify agent policy for that model.

[0126] In some examples, the administrative console application triggers distribution of an updated agent policy 296 in response to modifications made through the interface 537, such as changing a permission scope, allowing installation of an agent 542 for a new group of devices, or demoting a low-performing agent 542 from automatic suggestion eligibility. The interface 537 may additionally provide search and filtering controls, allowing an administrator to locate a targeted agent 542 by name, capability tag, action domain, application association, or usage pattern reflected in the agent usage data 558. FIG. 5 therefore illustrates a management interface through which enterprise-wide deployment, monitoring, and optimization of agents 542 can be curated based on operational telemetry collected across devices 202.

[0127] FIG. 6 illustrates a management system 600 that uses connector policies 696 to enable communication, authentication, and secure model invocation between a computing device 602 and one or more agents 642. The management system 600 mayAtty Docket No. 0120-1095W01operate as an example of the management system 200 described in FIGS. 2A and 2B and may include any of the details discussed with reference to the previous figures.

[0128] When agents 642 are registered or onboarded into an enterprise management platform 654, the enterprise management platform 654 may distribute a connector policy 696 to one or more computing devices 602. The enterprise management platform 654 may execute on one or more server computers 652 remote from the computing device 602. The computing device 602 may be fully device-managed or may operate with a managed browser profile under the authority of the enterprise management platform 654. Receipt of the connector policy 696 authorizes the computing device 602 to communicate with the associated agent 642 under defined access conditions.

[0129] The enterprise management platform 654 may maintain a set of policy controls 694. The policy controls 694 may be transmitted to computing devices 602 under management control or transmitted to computing devices 602 operating with browser profiles under management control. The policy controls 694 may include connector policies 696, where each connector policy 696 establishes whether communication with an agent 642 is permitted, restricted, or denied. The connector policy 696 may operate as a type of agent policy 296 referenced in FIGS. 2A and 2B.

[0130] The enterprise management platform 654 may extend the enterprise connectors framework with an additional connector type to support enterprise agents that are certified by a program, such as an enterprise recommended program. This connector (e.g., the connector policy 696) allows for collecting configuration parameters specific to a certain agent, company, or team. This may enable administrators to configure an agent once on behalf of all their users. In some examples, the agent policy may be specifically termed a connector policy 696 which defines rules, routing parameters, security conditions, and authentication requirements for connecting to the agent 642. The computing device 602 may use the connector policy 696 to generate a payload 669 including device signals 677 to authenticate access to the agent 642 via an access engine 673.

[0131] In further detail, a connector policy 696 defines rules, routing parameters, security conditions, and authentication requirements for connecting to an agent 642. The connector policy 696 may specify how the computing device 602 establishes a session with the agent 642, which communication channels may be used, and what device attributes areAtty Docket No. 0120-1095W01evaluated before granting access. The connector policy 696 may include a model identifier, one or more resource locators, a permission scope, an authentication sequence, and trust requirements based on device posture or account classification.

[0132] The connector policy 696 includes a resource locator 615 that identifies an endpoint associated with an agent 642. The resource locator 615 may identify a URL, domain pattern, protocol, or network address that the computing device 602 may contact when invoking the agent 642.

[0133] The computing device 602 includes a policy manager 619 configured to receive the policy controls 694 and the connector policies 696. The policy manager 619 may store, evaluate, and update the connector policies 696, and may enforce access rights during communication attempts with agents 642.

[0134] When a tab 610 accesses a resource locator 615a corresponding to the resource locator 615 specified in the connector policy 696, the policy manager 619 may generate a payload 669 including device signals 677. The payload 669 may be formatted as a transmission object that can be authenticated by a remote system before communication with the agent 642 is permitted.

[0135] The payload 669 includes device signals 677. Device signals 677 may represent capability, integrity, and / or trust-relevant information associated with the computing device 602. In some examples, a device signal 677 includes information associated with a computing device 602 that represents at least one of a capability, integrity, or trust-relevant attribute of the computing device 602, where the information is used as authentication evidence to determine whether access to a resource is permitted. In some examples, a device signal 677 is a set of data compiled at runtime from a computing device 602, wherein the data describes a security posture and compliance status of the computing device 602 to enable policy-based access decisions. In some examples, a device signal 677 is one or more attributes of a computing device 602 that are used to authenticate the computing device 602, where the one or more attributes include at least one of a hardware attribute, a software attribute, a security attribute, or a user-account attribute.

[0136] The device signals 677 may be compiled at runtime by the policy manager 619 and used as authentication evidence for determining whether access to the agent 642 isAtty Docket No. 0120-1095W01permitted. A device signal 677 may include one or more attributes about the computing device 602 that can be used to authenticate the computing device 602. The device signals may include attributes such as an operating system version, serial number, MAC address, installed security software, encryption state, application inventory, security patch level, memory configuration, or CPU architecture of the computing device 602. The device signals 677 may also reflect user-account classification, model-usage history, past access attempts, or the existence of previously verified credentials. The device signals 677 enable policy -based decision-making based on device security posture and enterprise compliance requirements.

[0137] The policy manager 619 may transmit the payload 669 over a network 650 to an access engine 673 operated by an access provider 671. The access engine 673 may authenticate or authorize the payload 669 before permitting communication with the agent 642. In some examples, the access engine 673 communicates with a verified access server 665 to validate user authentication credentials, enterprise enrollment state, or identity assertions included in the payload 669. The access engine 673 may then apply an access rule 675 to the device signals 677 within the payload 669 to determine whether the agent 642 may be accessed.

[0138] The access engine 673 may be a validation system that evaluates device signals 677 against the access rule 675. The access provider 671 may be a remote authorization entity that governs entry to agent endpoints. The verified access server 665 may be a credential and trust attestation system that confirms authentication state before access is granted. The access rule 675 may define threshold conditions, trust requirements, time-based constraints, or other logic for allowing or denying access to the agent 642.

[0139] The policy manager 619 may receive a connector policy 696 for each agent 642 authorized for use on the computing device 602. The connector policy 696 may be retrieved from the enterprise management platform 654 dynamically or pre-installed as part of a device configuration package.

[0140] In some examples, the connector policy 696 may include manifest data and configuration data, enabling the computing device 602 to make model-selection decisions before attempting authentication or connection. The configuration data may influence theAtty Docket No. 0120-1095W01model -sei ection behavior of the agent orchestration engine or determine whether predictive model suggestions may be surfaced to a user.

[0141] An administrative console application 635 executing on an administrator computing device 633 may display a user interface 637 that allows an administrator to define an access rule 675 or create a connector policy 696. The user interface 637 may enable customization of trust requirements, authentication flow, device compliance standards, or availability of agents 642 across managed devices.

[0142] FIG. 7 illustrates a flowchart 700 depicting example operations of agent orchestration according to an aspect. Although the flowchart 700 of FIG. 7 illustrates the operations in sequential order, it will be appreciated that this is merely an example, and that additional or alternative operations may be included. Further, operations of FIG. 7 and related operations may be executed in a different order than that shown, or in a parallel or overlapping fashion. The operations may define a computer-implemented method.

[0143] Operation 702 includes receiving at least a portion of a query on a computing device. Operation 704 includes selecting an agent from a plurality of agents to perform an action related to at least the portion of the query. Operation 706 includes initiating display of an interface element as a suggestion for a subsequent action.Operation 708 includes, in response to a selection of the interface element, initiating the action using the agent.

[0144] Clause 1. A method comprising: receiving at least a portion of a query on a computing device; selecting an agent from a plurality of agents to perform an action related to at least the portion of the query; initiating display of an interface element as a suggestion for a subsequent action; and in response to a selection of the interface element, initiating the action using the agent.

[0145] Clause 2. The method of clause 1, wherein the portion of the query is an incomplete query, the interface element being a first interface element, the method comprising: rendering an interface having an input area; receiving, via the input area, the incomplete query; and in response to the incomplete query, initiating display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.Atty Docket No. 0120-1095W01

[0146] Clause 3. The method of clause 1 or 2, further comprising: in response to the selection of the interface element, generating a plurality of signals about attributes of the computing device; and authenticating access to the agent using the plurality of signals.

[0147] Clause 4. The method of any of clauses 1 to 3, further comprising: receiving manifest data describing one of more functions of at least one of the plurality of agents; and selecting the agent from the plurality of agents using the manifest data.

[0148] Clause 5. The method of clause 4, further comprising: in response to the selection of the interface element, generating a prompt based on at least the portion of the query and a portion of the manifest data corresponding to the agent; providing the prompt to the agent to perform the action; receiving a model response generated by the agent; and initiating display of an interface associated with the agent, the interface displaying at least a portion of the prompt and at least a portion of the model response.

[0149] Clause 6. The method of clause 5, wherein the portion of the query is an incomplete query, the method comprising: generating a predictive query based on the incomplete query; obtaining, from the manifest data, agent instructions to assist the agent to perform the action; and generating the prompt to include the predictive query and the agent instructions.

[0150] Clause 7. The method of any one of clauses 1 to 6, further comprising: in response to the selection of the interface element: initiating display of a first interface associated with an application for the agent, the first interface configured to enable a user to communicate with the application; and initiating display of a second interface associated with the agent, the second interface configured to enable the user to communicate with the agent, the second interface being displayed concurrently with the first interface on a display of the computing device.

[0151] Clause 8. The method of any one of clauses 1 to 7, wherein the action is a first action, and the agent is a first agent, the method further comprising: receiving usage data representing activities performed by a user with the computing device; selecting a second agent from the plurality of agents to perform a second action related to the usage data; and initiating display of an interface object on the computing device, the interface object identifying the second agent and the second action as a proactive suggestion.Atty Docket No. 0120-1095W01

[0152] Clause 9. The method of any one of clauses 1 to 8, further comprising: updating history data based on selections of interface elements associated with the plurality of agents; and selecting the agent from the plurality of agents using the history data.

[0153] Clause 10. A computing device comprising: at least one processor; and a non-transitory computer-readable medium storing executable instructions that cause the at least one processor to: receive at least a portion of a query on the computing device; select an agent from a plurality of agents to perform an action related to at least the portion of the query; initiate display of an interface element as a suggestion for a subsequent action; and in response to a selection of the interface element, initiate the action using the agent.

[0154] Clause 11. The computing device of clause 10, wherein the portion of the query is an incomplete query, the interface element being a first interface element, wherein the executable instructions include instructions that cause the at least one processor to: render an interface having an input area; receive, via the input area, the incomplete query; and in response to the incomplete query, initiate display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.

[0155] Clause 12. The computing device of clause 10 or 11, wherein the executable instructions include instructions that cause the at least one processor to: receive manifest data describing one of more functions of at least one of the plurality of agents; configure an orchestration model using the manifest data; and select the agent from the plurality of agents using at least the portion of the query as an input to the orchestration model.

[0156] Clause 13. The computing device of clause 12, wherein the executable instructions include instructions that cause the at least one processor to: in response to the selection of the interface element, generate a prompt for the agent based on at least the portion of the query and a portion of the manifest data corresponding to the agent selected by the orchestration model; provide the prompt to the agent to perform the action; receive a model response generated by the agent; and initiate display of an interface associated with the agent, the interface displaying at least a portion of the prompt and at least a portion of the model response.

[0157] Clause 14. The computing device of clause 13, wherein the portion of the query is an incomplete query, wherein the executable instructions include instructions thatAtty Docket No. 0120-1095W01cause the at least one processor to: generate a predictive query based on the incomplete query; obtain, from the manifest data, agent instructions configured to assist the agent to perform the action; and generate the prompt to include the predictive query, the agent instructions, and a resource locator portion associated with the agent.

[0158] Clause 15. The computing device of any one of clauses 10 to 14, wherein the interface element is associated with a first resource locator portion of an application associated with the agent and a second resource locator portion associated with the agent, wherein the executable instructions include instructions that cause the at least one processor to: in response to the selection of the interface element: initiate display of a first interface associated with the application based on the first resource locator portion, the first interface configured to enable a user to communicate with the application; and initiate display of a second interface associated with the agent based on the second resource locator portion, the second interface configured to enable the user to communicate with the agent, the second interface being displayed concurrently with the first interface on a display of the computing device.

[0159] Clause 16. The computing device of any one of clauses 10 to 15, wherein the action is a first action, and the agent is a first agent, wherein the executable instructions include instructions that cause the at least one processor to: receive usage data representing activities performed by a user with the computing device; select a second agent from the plurality of agents to perform a second action related to the usage data; and initiate display of an interface object on the computing device, the interface object identifying the second agent and the second action as a proactive suggestion.

[0160] Clause 17. The computing device of any one of clauses 10 to 16, wherein the interface element includes a visual element associated with the agent, information identifying a predictive query that was generated based on the portion of the query, information identifying the action, and information identifying the agent.

[0161] Clause 18. Anon-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprising: receiving at least a portion of a query on a computing device; selecting an agent from a plurality of agents to perform an action related to at least the portion of the query; initiating display of an interface element as a suggestionAtty Docket No. 0120-1095W01for a subsequent action; and in response to a selection of the interface element, initiating the action using the agent.

[0162] Clause 19. The non-transitory computer-readable medium of clause 18, wherein the interface element is a first interface element, wherein the portion of the query is an incomplete query, wherein the operations further comprise: rendering an interface having an input area; receiving, via the input area, the incomplete query; and in response to the incomplete query, initiating display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.

[0163] Clause 20. The non-transitory computer-readable medium of clause 18 or 19, wherein the operations further comprise: receiving manifest data describing one of more functions of at least one of the plurality of agents; and selecting the agent from the plurality of agents using the manifest data.

[0164] Clause 21. A method comprising: receiving information that identifies a plurality of agents; generating an agent policy for at least one computing device associated with an enterprise management platform, the agent policy including manifest data describing one or more functions of the plurality of agents; and transmitting the agent policy to the at least one computing device, the manifest data configured to be used to select an agent from the plurality of agents to perform an action related to a query.

[0165] Clause 22. The method of clause 21, further comprising: transmitting a list of agents and usage data about the list of agents for display on a computing device associated with an administrator; and in response to a selection of the plurality of agents from the list of agents, receiving the information that identifies the plurality of agents.

[0166] Clause 23. The method of clause 22, wherein the usage data includes at least one of agent invocation frequency, number of actions selected, or duration of agent interaction across a plurality of computing devices associated with the enterprise platform.

[0167] Clause 24. The method of any one of clauses 21 to 23, wherein the agent policy includes a connector policy defining at least one access rule for communicating and authenticating with one or more of the plurality of agents.

[0168] Clause 25. The method of clause 24, further comprising: receiving, from the at least one computing device, a plurality of signals about attributes of the computingAtty Docket No. 0120-1095W01device; and authenticating the at least one computing device to access the agent based on the at least one access rule and the plurality of signals.

[0169] Clause 26. The method of any one of clauses 21 to 25, wherein the agent policy is configured to cause the at least one computing device to install an agent component to enable communication with the agent selected from the plurality of agents.

[0170] Clause 27. A computer program product storing executable instructions that cause at least one processor to execute any one of clauses 21 to 26.

[0171] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0172] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a non-transitory computer-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0173] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensoryAtty Docket No. 0120-1095W01feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0174] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or non-transitory medium of digital data communication (e.g., a communication network).Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet.

[0175] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship with each other.

[0176] In this specification and the appended claims, the singular forms "a," "an" and "the" do not exclude the plural reference unless the context clearly dictates otherwise. Further, conjunctions such as “and,” “or,” and “and / or” are inclusive unless the context clearly dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B. Further, connecting lines or connectors shown in the various figures presented are intended to represent example functional relationships and / or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device.Moreover, no item or component is essential to the practice of the examples disclosed herein unless the element is specifically described as “essential” or “critical”.

[0177] Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary skill in the art.Atty Docket No. 0120-1095W01

[0178] Moreover, use of terms such as up, down, top, bottom, side, end, front, back, etc. herein are used with reference to a currently considered or illustrated orientation. If they are considered with respect to another orientation, it should be understood that such terms must be correspondingly modified.

[0179] Although certain example methods, apparatuses and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that terminology employed herein is for the purpose of describing particular aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.

Claims

1. Atty Docket No. 0120-1095W01WHAT IS CLAIMED IS:

1. A method comprising:receiving at least a portion of a query on a computing device;selecting an agent from a plurality of agents to perform an action related to at least the portion of the query;initiating display of an interface element as a suggestion for a subsequent action; andin response to a selection of the interface element, initiating the action using the agent.

2. The method of claim 1, wherein the portion of the query is an incomplete query, the interface element being a first interface element, the method comprising:rendering an interface having an input area;receiving, via the input area, the incomplete query; andin response to the incomplete query, initiating display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.

3. The method of claim 1 or 2, further comprising:in response to the selection of the interface element, generating a plurality of signals about attributes of the computing device; andauthenticating access to the agent using the plurality of signals.

4. The method of any of claims 1 to 3, further comprising:receiving manifest data describing one of more functions of at least one of the plurality of agents; andselecting the agent from the plurality of agents using the manifest data.

5. The method of claim 4, further comprising:Atty Docket No. 0120-1095W01in response to the selection of the interface element, generating a prompt based on at least the portion of the query and a portion of the manifest data corresponding to the agent;providing the prompt to the agent to perform the action;receiving a model response generated by the agent; andinitiating display of an interface associated with the agent, the interface displaying at least a portion of the prompt and at least a portion of the model response.

6. The method of claim 5, wherein the portion of the query is an incomplete query, the method comprising:generating a predictive query based on the incomplete query;obtaining, from the manifest data, agent instructions to assist the agent to perform the action; andgenerating the prompt to include the predictive query and the agent instructions.

7. The method of any one of claims 1 to 6, further comprising:in response to the selection of the interface element:initiating display of a first interface associated with an application for the agent, the first interface configured to enable a user to communicate with the application; andinitiating display of a second interface associated with the agent, the second interface configured to enable the user to communicate with the agent, the second interface being displayed concurrently with the first interface on a display of the computing device.

8. The method of any one of claims 1 to 7, wherein the action is a first action, and the agent is a first agent, the method further comprising:receiving usage data representing activities performed by a user with the computing device;selecting a second agent from the plurality of agents to perform a second action related to the usage data; andAtty Docket No. 0120-1095W01initiating display of an interface object on the computing device, the interface object identifying the second agent and the second action as a proactive suggestion.

9. The method of any one of claims 1 to 8, further comprising:updating history data based on selections of interface elements associated with the plurality of agents; andselecting the agent from the plurality of agents using the history data.

10. A computing device comprising:at least one processor; anda non-transitory computer-readable medium storing executable instructions that cause the at least one processor to:receive at least a portion of a query on the computing device; select an agent from a plurality of agents to perform an action related to at least the portion of the query;initiate display of an interface element as a suggestion for a subsequent action; andin response to a selection of the interface element, initiate the action using the agent.

11. The computing device of claim 10, wherein the portion of the query is an incomplete query, the interface element being a first interface element, wherein the executable instructions include instructions that cause the at least one processor to:render an interface having an input area;receive, via the input area, the incomplete query; andin response to the incomplete query, initiate display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.

12. The computing device of claim 10 or 11, wherein the executable instructions include instructions that cause the at least one processor to:Atty Docket No. 0120-1095W01receive manifest data describing one of more functions of at least one of the plurality of agents;configure an orchestration model using the manifest data; andselect the agent from the plurality of agents using at least the portion of the query as an input to the orchestration model.

13. The computing device of claim 12, wherein the executable instructions include instructions that cause the at least one processor to:in response to the selection of the interface element, generate a prompt for the agent based on at least the portion of the query and a portion of the manifest data corresponding to the agent selected by the orchestration model;provide the prompt to the agent to perform the action;receive a model response generated by the agent; andinitiate display of an interface associated with the agent, the interface displaying at least a portion of the prompt and at least a portion of the model response.

14. The computing device of claim 13, wherein the portion of the query is an incomplete query, wherein the executable instructions include instructions that cause the at least one processor to:generate a predictive query based on the incomplete query;obtain, from the manifest data, agent instructions configured to assist the agent to perform the action; andgenerate the prompt to include the predictive query, the agent instructions, and a resource locator portion associated with the agent.

15. The computing device of any one of claims 10 to 14, wherein the interface element is associated with a first resource locator portion of an application associated with the agent and a second resource locator portion associated with the agent, wherein the executable instructions include instructions that cause the at least one processor to:in response to the selection of the interface element:Atty Docket No. 0120-1095W01initiate display of a first interface associated with the application based on the first resource locator portion, the first interface configured to enable a user to communicate with the application; andinitiate display of a second interface associated with the agent based on the second resource locator portion, the second interface configured to enable the user to communicate with the agent, the second interface being displayed concurrently with the first interface on a display of the computing device.

16. The computing device of any one of claims 10 to 15, wherein the action is a first action, and the agent is a first agent, wherein the executable instructions include instructions that cause the at least one processor to:receive usage data representing activities performed by a user with the computing device;select a second agent from the plurality of agents to perform a second action related to the usage data; andinitiate display of an interface object on the computing device, the interface object identifying the second agent and the second action as a proactive suggestion.

17. The computing device of any one of claims 10 to 16, wherein the interface element includes a visual element associated with the agent, information identifying a predictive query that was generated based on the portion of the query, information identifying the action, and information identifying the agent.

18. A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprising:receiving at least a portion of a query on a computing device;selecting an agent from a plurality of agents to perform an action related to at least the portion of the query;initiating display of an interface element as a suggestion for a subsequent action; andAtty Docket No. 0120-1095W01in response to a selection of the interface element, initiating the action using the agent.

19. The non-transitory computer-readable medium of claim 18, wherein the interface element is a first interface element, wherein the portion of the query is an incomplete query, wherein the operations further comprise:rendering an interface having an input area;receiving, via the input area, the incomplete query; andin response to the incomplete query, initiating display of the first interface element and a second interface element as suggestions for the subsequent action, the second interface element representing a suggested search query related to the incomplete query.

20. The non-transitory computer-readable medium of claim 18 or 19, wherein the operations further comprise:receiving manifest data describing one of more functions of at least one of the plurality of agents; andselecting the agent from the plurality of agents using the manifest data.

Images