Context-aware collaboration using a language model

Abstract

According to an aspect, a method includes rendering a shared tab group on a user device, the shared tab group including a plurality of tabs, detecting a user query from an input to an interactive interface associated with the shared tab group, generating contextual data about the plurality of tabs in the shared tab group, transmitting a prompt to a language model, the prompt including the user query and the contextual data, receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data, and initiating display of the model response in the interactive interface.

Description

Atty Docket No. 0120-920W01CONTEXT-AWARE COLLABORATION USING A LANGUAGE MODELCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority- to U.S. Provisional Patent Application No. 63 / 718,377, filed on November 8, 2024, entitled “CONTEXT- AWARE COLLABORATION USING A LANGUAGE MODEL”, the disclosure of which is incorporated by reference herein in its entirety-.BACKGROUND

[0002] A conventional browser can conduct searches and display websites and other information. For example, a user may sign-in under their user account (and, in some examples, under their profile) and use the browser to research a wide variety of topics, including vacation planning such as where to stay, the weather, and / or activities to do, etc. Furthermore, a user may interact with a language model to ask questions about a variety of topics. To collaborate on a topic, users may have to use other applications (e.g., email, chat, phone, text) to share their browser findings and information from the model responses.SUMMARY

[0003] This disclosure relates to integrating a language model into a shared computing session (e.g., a shared tab group) between multiple user devices (e.g., collaborators) in a manner that maintains security and user privacy. The language model can be invoked to answer inputs provided to (e.g.. messages posted to) an interactive interface associated with the shared computing session. For example, in response to detection of a message posted to the interactive interface, an application (e.g., a browser application or an operating system) generates contextual data (e.g., common contextual data) about the shared computing session and includes the contextual data in a prompt to a language model, which causes the language model to generate a model response that responds to the input posted to the interactive interface by a collaborator. In other words, in some examples, in response to a message posted to the interactive interface by a collaborator, the language model can be invoked to generate a model response that responds to the message.Atty Docket No. 0120-920W01

[0004] In some aspects, the techniques described herein relate to a method including: rendering a shared tab group on a user device, the shared tab group including a plurality of tabs; detecting a user query from an input to an interactive interface associated with the shared tab group; generating contextual data about the plurality of tabs in the shared tab group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of the model response in the interactive interface.

[0005] In some aspects, the techniques described herein relate to a computing device including: at least one processor; and a non-transitory computer-readable medium storing executable instructions that cause the at least one processor to: render a shared tab group on the computing device, the shared tab group including a plurality of tabs; detect a user query from an input to an interactive interface associated with the shared tab group; generate contextual data about the plurality of tabs in the shared tab group; transmit a prompt to a language model, the prompt including the user query and the contextual data; receive a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiate display of the model response in the interactive interface.

[0006] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium storing executable instructions that cause at least one processor to execute operations, the operations including: rendering a shared tab group on a user device, the shared tab group including a plurality of tabs; detecting a user query from an input to an interactive interface associated with the shared tab group; generating contextual data about the plurality of tabs in the shared tab group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query7and the contextual data; and initiating display of the model response in the interactive interface.

[0007] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.Atty Docket No. 0120-920W01BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 A illustrates a user interface with an interactive interface that displays a model response that was generated based on contextual data associated with a shared computing session according to an aspect.

[0009] FIG. IB illustrates a system that uses a language model in a shared computing session according to an aspect.

[0010] FIG. 2 illustrates a user interface with an interactive interface that displays a model response that was generated based on contextual data associated with the shared computing session according to another aspect.

[0011] FIG. 3 illustrates a flowchart depicting example operations of using a language model in a shared computing session according to an aspect.DETAILED DESCRIPTION

[0012] In an example, imagine you and your friends are working together on a project, like planning a vacation or completing a school assignment. You can create a shared group of browser tabs that everyone in the group can see and add to in real-time. Integrated into this shared space is an intelligent assistant (e.g., a language model) that is aware of the content in the shared tabs (e.g., all the shared tabs). This assistant acts like a knowledgeable team member who has read everything the group has collected.

[0013] For example, if a group of friends is planning a trip, they can have tabs open for flights, hotels, and local attractions. A user can ask the assistant in an interactive interface (e.g., a shared chat), “Based on the hotel we chose, what are some good restaurants within walking distance?’' The assistant understands which hotel is being discussed by looking at the relevant tabs and can provide a helpful, context-aware answer. In another example, a group of students researching a project can ask the assistant to “summarize the key findings from all the open research articles” or “create a list of action items for our group based on our chat history and the project requirements document we have open.” The assistant can then generate the summary or task list and share it with the group, streamlining their workflow.

[0014] This disclosure relates to an application (e.g., a browser application, an operating system, or a non-browser application) that integrates a language model in a shared computing session (e.g., a shared tab group) between multiple user devices (e.g., collaborators) in a manner that maintains security and user privacy. For example, the application generates contextual data (e.g., also referred to as common contextual data or shared contextual data) about the shared computing session and includes the contextual dataAtty Docket No. 0120-920W01 in a prompt to a language model, which causes the language model to generate a model response that responds to a user query posted to an interactive interface by a collaborator. The application renders the interactive interface, which displays content that is synchronized among the collaborators. Put another way, the interactive interface is accessible by, e.g., displayed to, the collaborators on their respective devices, each collaborator viewing the synchronized content in the interactive interface. In some examples, the interactive interface may be an interface that enables communication amongst the collaborators relating to the shared computing session. In other words, in response to an input (e.g., a message) posted to the interactive interface by a collaborator, the language model can be invoked to generate a model response that responds to a user query' included in the message. The response is then displayed in the interactive interface, giving all collaborators access to the response content.

[0015] The shared tab group is a tab group that is shared among multiple users. Collaborators can add tab(s) to a shared tab group (or more generally a window to a shared window group), where the shared tab group is rendered on the collaborator’s display devices so that each collaborator can view the same information. In some examples, the tab(s) include browser tabs. In some examples, the tab(s) may include interfaces from one or more non-browser applications. For example, a collaborator may add a tab to the tab group on their user device, and the newly added tab may be displayed on other collaborators’ user devices such that the same information is shared among the collaborators. In some examples, a user may create a new shared tab group within their user account and invite other members. The members (also referred to as collaborators) can add tabs to the shared tab group, and these tabs will be visible to the members. Changes made to the tabs yvithin the shared tab group are reflected in real time for the members. The techniques discussed herein is not limited to a shared tab group. For example, the system may enable a user to share other computer resources (e.g., besides browser content) in a shared computing session. The shared computing session may be a broader term representing a shared content group that can include tabs, windows, files, or other computer resources. For example, although a tab (e.g., browser tab) is used as an example of a computing resource that is shared among collaborators, a shared computing session may share other interfaces (e.g., windows) associated with other applications besides a browser application. For example, browser tabs, applications, interfaces, and / or other windows can be added to a shared group, and the items that are part of the shared group are synced across the collaborators. The terms shared tab group and shared computing session may be used interchangeably.Atty Docket No. 0120-920W01

[0016] The application renders an interactive interface that enables the collaborators to interact with each other and with a language model. Thus, the language model may be considered a collaborator. An input (e.g., messages such as user-generated messages, modelgenerated messages) provided to the interactive interface are visible to each collaborator in the shared computing session. The interactive interface may be a real-time interface, which may be a digital conversation space that allows collaborators to exchange information with minimal delay (e.g., messages are delivered and shown right after they are sent, so one can chat back and forth quickly). The input provided to the interactive interface may be text, video, audio, or fdes (or any combination thereof). The interactive interface enables the collaborators to post (e.g., enter) input, which are visible to the members of the shared tab group. The interactive interface includes an input field (e.g., where a user can enter input), a display area (e.g., where input or messages from the collaborators are posted), and one or more controls such as sending input, attaching files, providing engagement signals (e.g., like, dislike, favoritize, etc.), and / or change settings.

[0017] In further detail, the application includes a collaborative session manager configured to detect a user query from an input to an interactive interface. The interactive interface can be associated with the shared computing session. In some examples, the interactive interface is a chat interface. In some examples, the interactive interface is an interface of the application or the browser application. In some examples, the interactive interface is an interface of the operating system of the user device. In some examples, the interactive interface is the interface of a w eb application, a native application, or generally any computer resource that is part of a shared computing session. In some examples, the application is a browser application. In some examples, the application is an operating system. In some examples, the application is a non-browser application configured to render different interfaces or windows.

[0018] For example, a user may enter input such as "‘What’s the average temperature in Kihei’', and the collaborative session manager may detect that the textual data in the input corresponds to a question or action that can be addressed by a language model. In some examples, while the user is drafting content in a document (e.g., a shared document), the collaborative session manager may generate a user query based on the text, where the user query’ may be a prompt for a drafting suggestion or a request for a helpful webpage or application. In some examples, the language model is invoked by including a language model identifier (e.g., “@LLM”) in the message. For example, the collaborative session manager may determine that the input includes a language model identifier, and, in responseAtty Docket No. 0120-920W01 to the message being determined as including the language model identifier, identify at least some content of the input as the user query.

[0019] In response to the detection of the user query, the collaborative session manager generates contextual data (also referred to as common contextual data or shared contextual data) associated with the shared computing session. The contextual data is information about computing activity (e.g., browsing activity) of collaborators during the shared computing session. In some examples, the contextual data includes information about tabs that are added to the tab group (and shared) by the collaborators. When the shared computing session comprises a shared tab group including a plurality of tabs, the contextual data can comprise information about the plurality of tabs in the shared tab group. In some examples, the contextual data may include resource locators (e.g., uniform resource locators) in the shared tab group, page content, a history associated with the interactive interface (e.g., collaborator input to the interactive interface such as messages posted to the interactive interface), index information about the resource locators of the shared tabs, and / or engagements (e.g., reactions such as hearts, likes, etc.) with the input posted to the interactive interface. The collaborative session manager generates a prompt based on the user query and the contextual data. For example, the prompt includes the user query and the contextual data. In some examples, the prompt includes one or more system prompts (e.g., predefined instructions) that are inserted into the prompt. The collaborative session manager may transmit the prompt to a language model. In some examples, the language model is stored on one or more server computers. In some examples, the language model is stored on a user device. The language model may generate a model response that responds to the user query using the contextual data.

[0020] Because the collaborative session manager generates the contextual data for information that is shared to the shared computing session by the collaborators and the language model does not store the contextual data in memory, the application provides one or more technical benefits of maintaining security and / or privacy. Moreover, because the language model is provided with the contextual data (rather than all the information that is shared to the shared computing session by the collaborators) the application can provide one or more technical benefits of providing a model response in a resource efficient manner. The system may enable collaborators to interact together in real time with the language model, which is aware of the shared context (e.g., the contextual data) to execute a task in an application in an integrated manner.Atty Docket No. 0120-920W01

[0021] The collaborative session manager provides the model response in the interactive interface, where the model response is viewable by the collaborators of the shared computing session. For example, in response to a user query (e.g., ‘‘What’s the average temperature in Kihei”) posted to the interactive interface, the language model uses the query and the contextual data to generate a model response that is responsive to the query (e.g., “The average temperature in Kihei in June is. .. .”). In some examples, the model response identifies a particular resource locator (e.g., a URL) that is included in the contextual data and used to generate the model response. In some examples, the model response includes a selectable link to the resource locator, which, when selected, renders the computer resource (e.g., the website) on the user device. In some examples, the model response includes executable code (e.g., structured instructions) to perform a computer task.

[0022] In some examples, the language model may be used to evaluate and answer questions about a current set of resource locators in the shared tab group. In some examples, the language model may be used to summarize content across the current set of resource locators in the shared tab group and provide one or more recommendations in the form of new resources identified by the language model or proposed user queries for actions of the language model (e.g., summarize content, create a task list, etc.). In some examples, the language model may be used to generate a shared document with information from the shared computing session. In some examples, the language model may be used to generate a set of tasks for the collaboration group to complete. These and other features are further described with reference to the figures.

[0023] FIGS. 1 A to IB illustrates a system 100 that uses a language model 154 in a shared computing session 120 involving multiple user devices 102 (e.g., user device 102-1, user device 102-2) to generate a model response 132 based on contextual data 118 and render at least a portion of the model response 132 in an interactive interface 128. In some examples, the model response 132 is used to programmatically execute a computer task 173.

[0024] FIG. 1 A illustrates a user interface depicting a shared computing session 120 with a shared tab group 125 and an interactive interface 128 with a plurality of messages 130 (e.g., message 130-1, message 130-2) posted by users (e.g., user A, user B) of a collaboration group. A message 130 may be a type of input provided to the interactive interface 128. The user interface of FIG. 1A may be an interface provided by the system 100 of FIG. IB. The shared tab group 125 includes a tab 126 in the foreground, and the interactive interface 128 may be located adjacent to the tab 126. In some examples, the tab 126 is a browser tab. The interactive interface 128 can be associated with the shared tab group 125. In some examples.Atty Docket No. 0120-920W01 the interactive interface 128 is a UI object that is rendered within a browser window 127 that includes the shared tab group 125. A user can bring any one of the tabs 126 that are part of the shared tab group 125 into the foreground, and the interactive interface 128 continues to be displayed in the browser window 127. The shared tab group 125 includes a label 137 that is positioned in a tab strip of the browser window 127. The tab strip is a region of a browser window 127 that includes a visible row of one or more tabs 126. In some examples, the label 137 includes a name of the shared tab group 125. The name may be created by the creator of the shared tab group 125. In some examples, the label 137 is generated by the language model 154.

[0025] A shared tab group 125 may also be referred to more generally as a shared group, where the shared group is not limited to a tab-based browser interface. In some examples, the shared group may include a plurality of web resources rendered within brow ser tabs, browser windows, or other application-specific containers. In some examples, the shared group may include resources outside of a browser environment, such as files, documents, native application interfaces, or instances of non-browser applications. The items included in the shared group may be synchronized across collaborators in the same manner as the tabs 126 of the shared tab group 125.

[0026] Although tabs 126 are used as illustrative examples of how web resources may be represented and synchronized, a tab 126 may be one container type for a web resource. In some examples, a web resource may be displayed within a standalone browser window, a split-view panel, a native application surface that embeds web content, or another graphical container. Accordingly, references to tabs 126 herein may, in some examples, be interpreted as references to web resources rendered in any suitable container. In some examples, the shared group includes a plurality of web resources, where each web resource may correspond to a tab 126, a window 124, or another graphical or programmatic container capable of rendering the resource. Thus, the shared group may represent a collection of w eb resources rather than a collection limited strictly to brow ser tabs.

[0027] A web resource is not limited to a traditional web page or HTML document. A web resource may include a webpage, a web application, a progressive web application (PWA), a single-page application, a server-rendered document, an interactive web tool, or other network-retrievable content accessible via a resource locator 135 (e.g., a URL or URI). In some examples, a web resource may also include content hosted within hybrid or embedded browser views, sandboxed application views, or other resource containers rendered by a browser application or operating system.Atty Docket No. 0120-920W01

[0028] Referring to FIGS. 1A and IB, in response to user A posting a message 130-1 to the interactive interface 128 via an input field 168, a collaborative session manager 110 may detect a submission of a user query 116 in the message 130-1, and the collaborative session manager 110 may generate a prompt 114. Message 130-1 can be an example of input to the interactive interface 128 from which the user query 116 is detected. The prompt 114 includes the textual data of the message 130-1 and contextual data 118. The contextual data 118 may be referred to as common contextual data or shared contextual data. The inclusion of the contextual data 118 may provide context to a language model 154 and may assist the language model 154 in generating an accurate response to the message 130-1.

[0029] During a collaborative session, the language model 154 can be invoked to generate a model response 132 that responds to a user query 116 included in the message 130-1 posted to the interactive interface 128. Because the system 100 discussed herein generates the contextual data 118 for information that is shared to the shared computing session 120 by the collaborators and the language model 154 does not store the contextual data 118 in memory, the technical solution provides one or more technical benefits of maintaining security and / or privacy. The contextual data 118 may include information about a shared tab group 125 (or more generally shared windows 124). In some examples, the shared tab group 125 may be a shared interface group, where the shared interface group includes interfaces shared by the collaborators, and an interface may be a tab, a window, or generally any user interface element that is added to the shared computing session 120.

[0030] In some examples, the contextual data 118 may include resource locators 135 for tabs 126 added to the shared tab group 125 by the collaborators. In some examples, the contextual data 118 may include a chat history 136 associated with the interactive interface 128, index information 138 about the resource locators 135 in the shared tab group 125 that are obtained from an index structure 164 (also referred to as an index), and / or engagement data 140 with the input (e.g., messages 130) posted to the interactive interface 128 by the collaborators and / or language model 154. The contextual data 118 may include other information about the content rendered in the tabs 126 and / or information about non-browser resources used by the participants of the shared computing session 120.

[0031] In response to the prompt 114, a language model 154 generates a model response 132-1 that responds to the collaborator’s message 130-1 that w as posted to the interactive interface 128. and the model response 132-1 is displayed in the interactive interface 128. In response to user B posting a message 130-2 to the interactive interface 128 via the input field 168, the collaborative session manager 110 may detect a submission of aAtty Docket No. 0120-920W01 user query 116 in the message 130-2, and the collaborative session manager 110 may generate another prompt 114. The prompt 114 includes the textual data of the message 130-2 and the contextual data 1 18. In response to the prompt 114, the language model 154 may generate a model response 132-2 that responds to the collaborator’s message 130-2 that was posted to the interactive interface 128, and the model response 132-2 is displayed in the interactive interface 128. In some examples, the model response 132-2 may identify a resource 180 that assists with answering the message 130-2. In some examples, the resource 180 is any computer resource identifiable by a resource locator 135. The resource 180 may be a webpage, an application, a web application, a document, etc. (e.g., a URL). In some examples, the resource 180 may be referred to as a web resource.

[0032] Referring to FIG. 1A, the system 100 implements a technical solution involving an application 106 (e.g., a browser application 108, an operating system 105, or non-browser application) that integrates a language model 154 in a shared computing session 120 between multiple user devices 102 (e.g., user device 102-1, user device 102-2) (e.g., collaborators) in a manner that maintains security and / or privacy. For example, the application 106 generates the contextual data 118 about the shared computing session 120 and includes the contextual data 118 in a prompt 114 to a language model 154, which causes the language model 154 to generate a model response 132 that responds to a user query 116 posted to an interactive interface 128 by a collaborator. In other words, in response to a message 130 posted to the interactive interface 128 by a collaborator, the language model 154 can be invoked to generate a model response 132 that responds to a user query 1 16 included in the message 130. Because the system 100 discussed herein generates the contextual data 118 for information that is shared to the shared computing session 120 by the collaborators and the language model 154 does not store the contextual data 118 in memory, the technical solution provides one or more technical benefits of maintaining security and / or privacy.

[0033] The contextual data 118 includes information about the shared context among the multiple user devices 102. The contextual data 118 may be information that characterizes the state or content of a shared computing session 120 and is available to multiple participants of the shared computing session 120. In some examples, the contextual data 1 18 represents shared knowledge of the collaboration environment and may be generated, updated, or maintained by a collaborative session manager 110. The contextual data 118 provides context for interpreting user inputs and for guiding generation of model responses 132, without being limited to any particular format, structure, or type of information.Atty Docket No. 0120-920W01

[0034] In some examples, the model response 132 (or a portion thereof) is displayed in an interactive interface 128 of the shared computing session 120. In some examples, the model response 132 is used to perform a computer task 173 with respect to the shared computing session 120. Performing a computer task 173 may include creating and sharing a computing resource (e.g., a document, a web document, a website, etc.) and / or modifying or arranging a tab group 125. In some examples, the computer task 173 includes opening a computer resource such as creating a document (and, in some examples, populating the document with information from the model response 132), rendering a new brow ser tab in the shared tab group 125 with a new w ebpage recommended or suggested by the language model 154. In some examples, the model response 132 includes structured instructions. The structured instructions may include system instructions, structured data, JavaScript Object Notation (JSON), and / or source code to implement the computer task 173.

[0035] In some examples, the model response 132 may include structured instructions, generated by the language model 154, that are configured to cause the user device 102 to launch or instantiate a web application within the browser application 108 or another rendering environment. A web application may include any browser-rendered application surface, such as a collaborative editing tool, an interactive web document, a productivity application, or another browser-based interface that accepts user-generated or programmatically generated input. Upon launching the web application, the task executor 172 may use the structured instructions of the model response 132 to add content generated by the language model 154 to the web application, such as inserting text, structured sections, form entries, or other programmatically generated information. In some examples, the task executor 172 may further add the web application to the shared group (e.g.. the shared tab group 125 or another group of synchronized resources), thereby making the launched and populated web application available to all collaborators within the shared computing session 120.

[0036] In some examples, the model response 132 may cause the user device 102 to open or create a web document (e.g., an online document editor or collaborative web application), populate the document with a series of model-generated content items such as a list of tasks or action items, and then add the web document to the shared group so that collaborators may review , edit, or interact with the generated content. For instance, the language model 154 may generate a sequence of to-do items extracted from contextual data 118 or chat history 136, and the task executor 172 may insert these items into the opened web document and synchronize the resulting document across the shared group.Atty Docket No. 0120-920W01

[0037] In some examples, the model response 132 may cause the user device 102 to launch a spreadsheet web application, populate a sheet with generated values, table rows, or structured data derived from resource locators 135, and then add the populated spreadsheet to the shared group. In some examples, the model response 132 may cause the user device 102 to open a presentation-style web application, insert model -generated title slides or summary7sections, and add the resulting presentation resource to the shared group. In some examples, the web application may be a form-editing tool, a dashboard, a code editor, or another browser-rendered interface, and the structured instructions may cause the task executor 172 to insert generated content, configure application settings, or trigger other application-specific actions before adding the web resource to the shared group.

[0038] The system 100 may include a user device 102-1 and a user device 102-2. The user device 102-1 can be an example of a first user device and user device 102-2 can be an example of a second user device. The user device 102-1 or the user device 102-2 can be an example of a computing device. Although two user devices 102 are depicted as collaborators, a shared computing session 120 may include any numbers of user devices 102 that are part of the shared computing session 120. The user device 102-1 includes a collaborative session manager 110 configured to manage a shared computing session 120. In some examples, the collaborative session manager 110 is a component of a browser application 108. In some examples, the collaborative session manager 110 is a component of an operating system 105. In some examples, the collaborative session manager 110 is a component of a non-browser application (e g., a particular instance of an application 106) such as a native application installed on the operating system 105. The collaborative session manager 110 may initiate a display of one or more interfaces (with one or more user controls) to enable a user to initiate a shared computing session 120 with one or more collaborators. For example, the user may provide one or more user identifiers (e.g., email addresses, usernames, etc.), and the collaborative session manager 110 may invite the identified collaborators to join the shared computing session 120.

[0039] The shared computing session 120 may include a shared tab group 125. The collaborative session manager 110 may provide the shared tab group 125 on a display 122 of each of the collaborator’s devices. The shared tab group 125 may be identifiable by one or more interface elements that visually identify the tabs 126 as part of the shared tab group 125. The collaborative session manager 110 may add tabs 126 to the shared tab group 125 and / or remove tabs 126 from the shared tab group 125 based on the collaborator's user interactions. In some examples, a user may drag a tab 126 to the shared tab group 125 to add the tab 126 toAtty Docket No. 0120-920W01 the shared tab group 125. In some examples, a user may drag away a tab 126 from the shared tab group 125 to remove the tab 126 from the shared tab group 125. Although some examples use tabs 126 (e.g., browser tabs) that are added to a shared tab group 125, it is understood that the collaborative session manager 110 may apply to interfaces or windows (referred to as shared windows 124) outside of a browser application 108. For example, the collaborative session manager 110 may enable collaborators to add windows or interfaces (e.g., windows 124) of non-browser applications to a shared window (or interface) group. For example, the shared group may include non-web or non-browser elements, such as locally executed applications, documents, or UI surfaces from native applications. These items may be represented as windows 124 or other resource containers and managed within the same synchronization framework as the browser-based web resources

[0040] In some examples, the browser application 108 implements the tab group 125 by associating a tab group data structure with the corresponding tab objects, serializing the metadata for the tab group 125 into a persistent state, and modifying the user interface of the tab strip based on the membership and attributes of the tab group 125. The synchronizer 162 may provide synchronization of the tab group 125 either across multiple user devices 102 associated with a single user account (e.g., profile synchronization) or across multiple user accounts that participate in a shared computing session 120 (e.g., collaborative synchronization).

[0041] In some examples, the browser application 108 implements tab groups 125 using browser-managed data structures that associate a collection of tabs 126 with additional metadata. The browser application 108 may instantiate a tab group object in a memory device 103. The tab group object may include references to the identifiers of the tabs 126 that are members of the tab group 125. a title string for the tab group 125. a color attribute, and / or a state attribute that specifies whether the tab group 125 is collapsed or expanded.

[0042] The browser application 108 may further store the tab group object in a session state service. The session state service may be a subsystem of the browser application 108 that serializes runtime state information, including tab data, browsing history’, and / or window layouts, into a persistent storage medium (e.g., a local database, configuration file, or key-value store in the file system). The session state service may also deserialize the stored state data when the browser application 108 is restarted. As such, when the browser application 108 is closed and reopened, the session state service may reload the tab group object, thereby reconstituting the membership of the tab group 125 and its associatedAtty Docket No. 0120-920W01 metadata. This may enable the tab group 125 to persist across restarts of the browser application 108.

[0043] The browser application 108 may also render a bounding indicator in the user interface of the tab strip for the tab group 125. The bounding indicator may be a graphical container with the title and color attributes of the tab group 125. When the membership of the tab group 125 changes (e.g., when a tab 126 is added or removed), the browser application 108 may update the tab group object in the memory device 103 and in the session state service and may further redraw the bounding indicator in the tab strip to reflect the current membership.

[0044] In some examples, the browser application 108 may also expose the tab group object through an application programming interface (API). For example, the browser application 108 may support a tab groups API that allows an extension to query, create, modify, or delete tab groups 125. The browser application 108 may also integrate tab groups 125 with a profile synchronization sendee (e.g., a server-based synchronization subsystem that serializes tab group objects and exchanges them via a network 150) that synchronizes the tab group objects across multiple user devices 102 associated with a user account. In some examples, the profile synchronization service may transmit serialized representations of tab group objects to a server computer 160 and may receive updates from the serv er computer 160 to replicate the tab group objects on other user devices 102. In some examples, the profile synchronization service may be considered an example of the synchronizer 162, where the synchronizer 162 provides synchronization of the tab group 125 across devices associated with the same account.

[0045] In some examples, the synchronizer 162 may be extended to provide collaborative synchronization of the tab group 125 across multiple user accounts participating in a shared computing session 120. In some examples, operations performed by one collaborator (e.g., add-tab, remove-tab, or set-title) may be published to the synchronizer 162, which may then sequence and propagate the operations to other collaborators’ user devices 102. Each receiving browser application 108 may apply the operations to its local state and may redraw the tab strip to reflect the shared state of the tab group 125.

[0046] In some examples, the browser application 108 implements tab groups 125 by storing group metadata in a session state database. The browser application 108 may instantiate a tab group object that includes references to the identifiers of the tabs 126 that are part of the tab group 125, a title string, a color attribute, and / or a state attribute indicating whether the tab group 125 is collapsed or expanded. The tab group object may be serializedAtty Docket No. 0120-920W01 into the session state database, which may be a structured storage service of the browser application 108 that records tab data, browsing history, and / or layout information. When the browser application 108 is restarted, the session state database may be queried, and the tab group object may be deserialized to restore the tab group 125 with its original membership and metadata.

[0047] In some examples, the browser application 108 may further integrate the tab group object with a collections subsystem. The collections subsystem may convert a tab group object into a collection record that can be persisted, retrieved, and / or restored independently of the browser’s active session state. This may allow the tab group 125 to be reconstituted even if the active browsing session has been closed without a saved state.

[0048] In some examples, the browser application 108 may also integrate the tab group 125 with a profile synchronization service. The profile synchronization service may serialize the tab group object and may transmit the serialized object to one or more server computers 160 via a network 150. The server computer(s) 160 may store the serialized object in association with a user account identifier. When another user device 102 associated with the same user account executes the browser application 108, the profile synchronization service may retrieve the serialized object from the server computers 160 and may deserialize the object to replicate the tab group 125 on the other user device 102. In some examples, the profile synchronization service may be considered an example of the synchronizer 162, where the synchronizer 162 provides synchronization of the tab group 125 across user devices 102 associated with the same account.

[0049] In some examples, the synchronizer 162 may be further configured to provide collaborative synchronization of the tab group 125 across multiple user accounts that participate in a shared computing session 120. For example, the synchronizer 162 may allocate a session record for the tab group 125 on the server computers 160, authenticate the collaborators, and propagate updates to the collaborators’ user devices 102 in real time or near real time. Each operation performed on the tab group 125 (e.g., adding a tab 126, removing a tab 126, reordering the tabs 126, or changing a visual attribute of the tab group 125) may be published to the synchronizer 162, which may then sequence the operations and deliver them to the other collaborators via a channel (e g., a publish / subscribe channel). The browser application 108 on each collaborator’s device may apply the operations to its local state of the tab group 125 and may redraw the tab strip accordingly.

[0050] In some examples, to maintain consistency, the synchronizer 162 may assign operation identifiers, enforce ordering policies, and provide conflict resolution (e.g., last-Atty Docket No. 0120-920W01 writer-wins for scalar attributes or conflict-free replicated data types for ordering). The synchronizer 162 may also generate compact snapshots of the tab group 125 that can be transmitted to collaborators who join the shared computing session 120 after it has been established.

[0051] In some examples, the synchronizer 162 may also expose a collaboration invitation workflow, wherein the browser application 108 may generate an invitation token for the tab group 125 and the shared computing session 120 and may transmit the token to prospective collaborators using a messaging service of the operating system 105. Upon acceptance, the synchronizer 162 may add the collaborator’s user account to the session and begin delivering operation streams for the tab group 125 to the collaborator’s user devices 102.

[0052] In some examples, the browser application 108 represents a tab group 125 by assigning an internal grouping identifier to each tab 126 that belongs to the tab group 125. The grouping identifier, along with attributes such as the title of the tab group 125 and its collapsed or expanded state, may be written to a JSON-based session store. The session store may be a persistent file maintained by the browser application 108 that includes structured records for active tabs 126, windows 124, and / or tab groups 125. When the browser application 108 is restarted, the JSON-based session store may be read and the grouping identifiers may be applied to the restored tab objects, thereby reconstructing the tab group 125. In some examples, the browser application 108 may integrate the tab group 125 with a sync service. The sync service may be an example of an aspect of the synchronizer 162. The sync service is configured to serialize and replicate tab group objects across user devices 102 tied to the same account. In some examples, the synchronizer 162 may be extended to propagate updates to multiple accounts in a shared computing session 120, enabling live collaborative tab groups 125 consistent with the operations described above.

[0053] In some examples, the browser application 108 implements '‘tab islands” as an example of tab groups 125. The browser application 108 may establish adjacency links between consecutive tabs 126 that are members of the same tab island. The adjacency links may be recorded in a tab model data structure of the browser application 108 that tracks relationships among the tabs 126. The brow ser application 108 may further generate a visual separation indicator in the tab strip for each tab island and may update the indicator when the set of adjacent tabs 126 in the tab island changes. In some examples, the browser application 108 may integrate tab islands with a synchronization service, which may be implemented by the synchronizer 162 to serialize and replicate the tab islands across user devices 102Atty Docket No. 0120-920W01 associated with the same account. In some examples, the synchronizer 162 may be extended to support inter-account collaborative synchronization of tab islands, thereby allowing multiple collaborators to view and edit the same tab group 125 in real time.

[0054] In some examples, the browser application 108 implements tab groups 125 by serializing a set of resource locators 135 and associated metadata to a preference fde on the user device 102. The preference fde may include the identifiers of the tabs 126 that belong to the tab group 125, the ordering of the tabs 126, the title string, the color attribute, and / or the collapsed or expanded state of the tab group 125. The preference file may be stored in a local persistent storage medium (e.g., a property list file or structured database) that is read by the browser application 108 when the tab group 125 is reconstructed at startup.

[0055] In some examples, the browser application 108 may also integrate tab groups 125 with a profile synchronization service that synchronizes the preference file across user devices 102 associated with the same account. In some examples, the profile synchronization sendee may be implemented by the synchronizer 162, which may serialize the preference file for the tab group 125, encrypt the serialized data, and transmit it via the network 150 to one or more server computers 160. The sen7er computers 160 may store the serialized representation in association with a user account identifier and may propagate the serialized representation to other user devices 102 signed into the same account. Each receiving user device 102 may deserialize the representation, update its local preference file, and reconstitute the tab group 125 in its browser application 108. Thus, the synchronizer 162 may provide persistence and replication of tab groups 125 across devices of a single user.

[0056] In some examples, the browser application 108 may support shared tab groups 125, which may be synchronized across multiple user accounts participating in a shared computing session 120. In some examples, the synchronizer 162 may allocate a session record for the shared tab group 125 on the server computer(s) 160, which may include an access control list (ACL) that identifies the collaborators who are authorized to join the session. Each operation performed on the shared tab group 125 (e.g., adding a new tab 126 with a resource locator 135, removing a tab 126, or reordering the tabs 126) may be captured by the browser application 108, published to the synchronizer 162, and propagated by the synchronizer 162 to the other collaborators’ devices. Each receiving browser application 108 may then apply the operation to its local tab group object and redraw the tab strip accordingly.

[0057] In some examples, the synchronizer 162 may sequence operations performed by collaborators and may resolve conflicts using predetermined policies. For example, theAtty Docket No. 0120-920W01 synchronizer 162 may apply a last-writer- wins policy for title changes and may apply a stable ordering policy for tab indices. The synchronizer 162 may also generate snapshots of the shared tab group 125, which may be transmitted to collaborators that newly join the shared computing session 120 to bootstrap their local state before subsequent operations are streamed.

[0058] In some examples, the browser application 108 may also support collaborator attribution for shared tab groups 125. For example, when a collaborator adds a tab 126, the synchronizer 162 may associate the operation with a collaborator identifier and may propagate this attribution to the other user devices 102. The browser application 108 may then display a visual cue indicating which collaborator contributed the tab 126. In some examples, the synchronizer 162 may also support collaborator management workflows, such as generating invitation tokens, transmitting the tokens via the operating system 105 messaging service, and updating the ACL when a collaborator joins or leaves.

[0059] In some examples, the synchronizer 162 may also apply privacy and security policies in connection with shared tab groups 125. For example, the synchronizer 162 may encrypt serialized tab group objects, may scope distribution of updates only to collaborators identified in the ACL, and may invalidate stored snapshots of the tab group 125 when a collaborator is removed. The synchronizer 162 may further persist encry pted backups of the shared tab group 125 to the server computer(s) 160 to enable recovery in the event of data loss on a user device 102.

[0060] In some examples, the browser application 108 may provide one or more user interface features in connection with a shared tab group 125, where the synchronizer 162 propagates collaborator state information to support these features. For example, the synchronizer 162 may associate collaborator identifiers with presence information (e.g., which tab 126 a collaborator is actively viewing) and may transmit this information to the other user devices 102. The browser application 108 may then render a corresponding visual indicator (e.g., an avatar, an initial, or a color badge) in the interactive interface 128 adjacent to the tab 126. In this way. the interactive interface 128 visually identifies which collaborator is currently viewing or interacting with each tab 126.

[0061] In some examples, the synchronizer 162 may propagate notifications of updates to the shared tab group 125, including operations such as adding, removing, or reordering a tab 126. The browser application 108 may then update the interactive interface 128 by generating a temporary visual effect (e.g., highlighting, bolding, or animating a tab label) or by dimming or removing the graphical element corresponding to a removed tab 126.Atty Docket No. 0120-920W01In some examples, the browser application 108 may also display activity notifications in an associated conversation thread (e.g.. a messages thread), where each notification corresponds to an operation performed on the shared tab group 125 and is selectable to cause navigation into the shared tab group 125.

[0062] In some examples, the browser application 108 may provide collaborator management and integrated communication controls for a shared tab group 125. For example, the interactive interface 128 may render a menu, button, or icon labeled “Collaborate,” which may provide selectable options such as “Add Collaborator,” “Remove Collaborator,” or “Stop Sharing.” The synchronizer 162 may generate an invitation token identifying the shared computing session 120 and may transmit the token to prospective collaborators via a messaging service of the operating system 105. In some examples, the interactive interface 128 may also render communication controls (e.g., labeled “Message,” “Audio,” or “Video”), which, when selected, may invoke a messaging or conferencing application of the operating system 105. The synchronizer 162 may propagate collaborator identifiers to the operating system 105 so that the communication application can generate invitations and initiate a session with the identified collaborators.

[0063] In some examples, the synchronizer 162 may be configured to replicate navigation state across collaborators’ devices. For example, the synchronizer 162 may capture an indication of an active tab selection or a scroll position of a tab 126 on a first user device 102, may transmit the navigation state to the other user devices 102. and may cause the browser application 108 on the other devices to synchronize the interactive interface 128 accordingly. The browser application 108 may then redraw the tab strip or content viewport to reflect the navigation state, thereby enabling guided navigation in which one collaborator's navigation actions are mirrored on the displays 122 of the other collaborators.

[0064] As indicated above, in a shared computing session 120, a tab group 125 with one or more tabs 126 are synchronized and shared with collaborators (e.g., user device 102-1, user device 102-2) in real-time or near real-time. In the example of FIG. IB, user device 102-1 and user device 102-2 are part of the shared computing session 120. As such, any tabs 126 that are added by a user device to the shared tab group 125 are visible to the other user devices in the shared computing session 120.

[0065] In some examples, a user may use the user device 102-1 to open a new tab (e.g., tab 126-1) and add the tab 126-1 to the shared tab group 125. In some examples, the tab 126-1 may display a web resource (e.g., a webpage, application, a web document), which is identifiable by a resource locator 135. The synchronizer 162 may cause the tab 126-1 to beAtty Docket No. 0120-920W01 populated in the shared tab group 125 on other collaborators such as user device 102-2. For example, the synchronizer 162 may identify user accounts that are part of the shared computing session 120, and the user devices may retrieve information associated with the shared computing session 120 such as tabs 126 and / or windows 124 that are added to the shared computing session 120 by other collaborators. In some examples, a user may use the user device 102-2 to open a new tab (e.g., tab 126-2) and add the tab 126-2 to the shared tab group 125. The synchronizer 162 may cause the tab 126-2 to be populated in the shared tab group 125 on other collaborators such as user device 102-1.

[0066] When a collaborator adds a tab 126 to the shared tab group 125, this may be understood more generally as adding a web resource to the shared group. In some examples, the web resource may be rendered as a tab. In some examples, the web resource may be displayed within a window, pane, or other container. The synchronizer 162 may propagate the underlying resource locator 135 and associated metadata regardless of the container type.

[0067] The collaborative session manager 110 may render an interactive interface 128 in the shared computing session 120 that enables collaborators to exchange communications with each other and with a language model 154. In some examples, the interactive interface 128 may be a digital conversation space that displays messages 130, where the messages 130 include user-generated text, model-generated responses, and / or system notifications. The interactive interface 128 may be configured as a real-time interface, such that messages 130 are delivered and displayed with minimal delay after transmission, thereby enabling the collaborators to converse in a chat-like manner.

[0068] In some examples, the interactive interface 128 may include one or more input modalities. For example, the interactive interface 128 may include a text input field for receiving typed input from a collaborator, an audio capture control that activates a microphone of the user device 102 to record speech, or a video capture control that activates a camera of the user device 102 to capture live video. The collaborative session manager 110 may process the audio or video to generate corresponding message objects, which may be transmitted to the synchronizer 162 for propagation to the other user devices 102.

[0069] In some examples, the interactive interface 128 may support multimodal output in addition to text. For example, the interactive interface 128 may render an audio waveform or a video window in association with a message 130 when the message includes audio or video content. In some examples, the interactive interface 128 may render rich media content such as hyperlinks, inline previews of webpages identified by resource locators 135, document thumbnails, images, or interactive widgets (e g., buttons or sliders).Atty Docket No. 0120-920W01

[0070] In some examples, the interactive interface 128 may include controls that enable collaborators to interact with the shared tab group 125 and / or shared windows 124. For example, the interactive interface 128 may render a selectable message that, when activated, may open a corresponding tab 126 in the shared tab group 125. In some examples, the interactive interface 128 may include drag-and-drop support, wherein a collaborator maydrag a tab 126, a resource locator 135, or a file object into the interactive interface 128 to create a corresponding message 130.

[0071] In some examples, the interactive interface 128 further includes one or more settings that enable collaborators to tailor how the language model 154 and contextual data 118 are used in connection with the shared computing session 120. For example, a setting may allow a collaborator to specify whether the language model 154 is programmatically invoked for all user queries 116 or only when explicitly referenced by a language model identifier. For example, the language model 154 can be invoked when it is determined that the user input to the interactive interface 128 includes a language model identifier. Another setting may define the scope of contextual data 118 to be provided in prompts 114 to the language model 154, such as restricting the contextual data 118 to content from a currently active tab 126, to metadata about resource locators 135 in the shared tab group 125, or to engagement data 140 generated during the session. Additional settings may enable collaborators to control visibility of model responses 132. such as whether the response is rendered to all participants in the shared tab group 125 or privately to a requesting user device 102. In some examples, the interactive interface 128 includes settings that govern execution of model-generated tasks, such as whether the task executor 172 is authorized to launch a new application 106, generate a new tab 126, or populate user interface fields with structured data. The interactive interface 128 may also include privacy and security- settings, such as anonymizing personal identifiers before contextual data 118 is transmitted or programmatically clearing cached data following each invocation of the language model 154.

[0072] In some examples, the interactive interface 128 may also support collaborator status indicators. For example, the interactive interface 128 may render a ‘“typing” indicator when another collaborator is composing a message 130 or may render an activity marker when another collaborator is recording audio or video. In some examples, the interactive interface 128 may display participant identifiers alongside each message 130, enabling attribution of the communication to a particular collaborator.

[0073] The collaborative session manager 110 detects a user query 116 from a message 130 posted to the interactive interface 128. For example, a user may post a messageAtty Docket No. 0120-920W01130 such as “What’s the average temperature in Kihei”, and the collaborative session manager 110 may detect that the textual data in the message 130 corresponds to a question or action that can be determined by a language model 154. In some examples, the language model 154 is invoked by including a language model identifier (e.g., “@LLM”) in the message 130. A language model identifier may be a string of characters that identifies a language model 154. The collaborative session manager 110 may determine that the messages 130 include a language model identifier, and, in response to the message 130 being determined as including the language model identifier, identify the content of the message as the user query 116. In some examples, the collaborative session manager 110 may parse the textual content, apply natural language preprocessing (e.g., tokenization, entity detection), and identify the resulting processed content as the user query 116.

[0074] In some examples, invocation of the language model 154 may occur without an explicit identifier being included in the user input. In some examples, the collaborative session manager 110 may analyze textual input posted to the interactive interface 128 and determine that the input resembles a query, command, or instruction suitable for handling by the language model 154. In some examples, the collaborative session manager 1 10 may programmatically identify at least a portion of the input as the user query 116 and generate the prompt 114 accordingly. In some examples, the collaborative session manager 110 may apply classification models or heuristic rules to distinguish between conversational dialogue and actionable queries, thereby reducing false positives.

[0075] In further detail, the collaborative session manager 110 determines whether a message 130 corresponds to a user query 116 suitable for processing by the language model 154 by applying one or more query-detection techniques. For example, the collaborative session manager 110 may use a classification model (e.g.. a natural language processing classifier) trained to distinguish between conversational messages and actionable queries, such as requests for factual information, recommendations, summaries, or execution of a task. The classification model may analyze linguistic features including interrogative phrasing, intent verbs (e.g., “summarize,” “compare,” “plan,” “recommend”), sentence structure, and / or token patterns to determine whether the input represents a user query 116.

[0076] In some examples, the collaborative session manager 110 applies one or more heuristic rules to identify user queries 116, such as detecting question marks, trigger keywords, or imperative voice patterns, and may combine such heuristics with conversational context signals (e.g.. whether a prior model response 132 occurred within a threshold number of messages). In some examples, explicit invocation mechanisms may be used, such asAtty Docket No. 0120-920W01 detecting the presence of a language model identifier (e.g., “@LLM”), detecting activation of a dedicated query input control in the user interface, or detecting a gesture request associated with a selected portion of shared content. These mechanisms may operate independently or in combination, and confidence thresholds may be applied to reduce false positives, such that only messages determined to be intentional queries are used to generate prompt 114 for the language model 154.

[0077] In some examples, the collaborative session manager 110 may support multimodal invocation mechanisms. For example, a collaborator may provide an audio input via a microphone, and the system 100 may detect a spoken query as the user query7116. In some examples, a collaborator may provide gesture-based input, such as selecting or highlighting text within a shared resource (e.g., a shared web resource), and the system may generate a corresponding user query 1 16 for the language model 154. In some examples, the interactive interface 128 may further include a selectable icon or voice-command keyword that explicitly signals that the subsequent input should be processed as a query to the language model 154.

[0078] In some examples, invocation of the language model 154 may be triggered by contextual suggestions. For example, when the collaborative session manager 110 determines that a newly added resource (e.g., web resource) to the shared computing session 120 contains information relevant to a prior discussion thread, the collaborative session manager 110 may prompt 114 the language model 154 to generate a summarization or recommendation without requiring an explicit user request. In some examples, the collaborative session manager 110 may provide selectable interface elements, such as a context-sensitive button or icon presented alongside a shared tab or message, that when activated invoke the language model 154 to process the associated content. In some examples, the collaborative session manager 110 may pre-generate one or more candidate queries based on the shared context (e.g., “Summarize this article,” “Compare this document to prior discussions”) and may present these candidate queries as one-click options in the interactive interface 128.

[0079] In response to detection of the user query 116, the collaborative session manager 110 generates or retrieves contextual data 118 associated with the shared computing session 120. In some examples, the contextual data 118 is generated or updated in response to detection of a user query7116. In some examples, the contextual data 118 may be generated, retrieved, or updated independently of a user query7116. such as when the state ofAtty Docket No. 0120-920W01 the shared computing session 120 changes (e.g., when a collaborator adds a new resource, modifies a tab, or posts a message).

[0080] The contextual data 118 may include window state data 134. The window state data 134 refers to information that characterizes the operational status or configuration of a window or application during the shared computing session 120. In some examples, the window state data 134 identifies resources (e.g.. web resources) currently active within the shared tab group 125, including one or more resource locators 135. A resource locator 135 refers to an identifier or pointer that enables access to a web resource or a non-web resource. In some examples, a resource locator 135 may be a uniform resource locator (URL), a uniform resource identifier (URI), a file path, an application identifier, or another reference string. A resource locator 135 may correspond to content stored locally on a device, content stored on a network-accessible server, or content provided through a server-based application.

[0081] In some examples, the contextual data 118 may include information representing the content exchanged or displayed during the shared computing session 120. Such information may encompass digital content rendered in shared resources (e.g., web resources) as well as conversational data exchanged among collaborators. In some examples, the contextual data 118 may include page content from a browser tab or other resource (e.g., web resource) shared in the shared computing session 120. In some examples, the contextual data 118 includes a chat history 136 associated with the interactive interface 128. The chat history 136 may record messages generated by collaborators and / or responses generated by the language model 154, thereby providing context for subsequent queries and model responses. The contextual data 118 can include a portion of the history (e.g., the chat history 136) associated with the interactive interface 128.

[0082] In some examples, the contextual data 118 includes index information 138 associated wdth one or more resource locators 135 in the shared tab group 125. The index information 138 refers to data that characterizes or describes a computer resource (e.g., a web resource) in a manner that facilitates efficient retrieval or analysis. The index information 138 may be derived from or stored within an index structure 164. The index structure 164 may be any data structure configured to organize information about computer resources (e.g., web resources), such as webpages, applications, extensions, web applications (including progressive web applications), or documents, to support searching, classification, or correlation. In some examples, an index structure 164 is referred to as an index file or index maintained by a search engine, although other implementations may employ embeddingAtty Docket No. 0120-920W01 stores, classification tables, or other indexing mechanisms. The index information 138 may include identifiers, tokens, terms, keywords, or other descriptive data that enable the collaborative session manager 110 and / or the language model 154 to identify, rank, or retrieve relevant computer resources (e.g., web resources).

[0083] In some examples, the contextual data 118 includes engagement data 140 (e.g., reactions such as hearts, likes, etc.) with the messages 130 posted to the interactive interface 128 and / or the content shared in the shared computing session 120. In some examples, the engagement data 140 includes user interactions with content or messages within a shared computing session 120. The engagement data 140 may include explicit reactions such as likes, dislikes, favorites, votes, comments, and / or tags. The engagement data 140 may also include implicit interactions such as cursor hovers, dwell time, scroll depth, selection activity, or click-through behavior. The engagement data 140 provide signals of collaborator interest or relevance that may be incorporated into the contextual data 118 used to generate model responses 132.

[0084] In some examples, the contextual data 118 may include metadata or descriptive information associated with resources (e.g., web resources) in the shared computing session 120. For example, when a tab 126 is added to a shared tab group 125, the collaborative session manager 110 may extract and include metadata such as a page title, description, keywords, author information, and / or schema tags (e.g., structured markup embedded in the webpage). In some examples, the contextual data 118 may include a representation of the document object model (DOM) or other data structures underlying a web resource, thereby providing additional detail that can be used by the language model 154 to generate a contextually relevant response.

[0085] In some examples, as part of generating the contextual data 118 and preparing the prompt 1 14, the collaborative session manager 110 and prompt generator 1 12 may transform data collected from the shared tab group 125 into a structured format suitable for processing by the language model 154. The browser application 108 may extract page content from each tab 126 by traversing a rendered DOM associated with the tab, capturing visible text nodes, metadata tags, and / or selected attributes of interactive elements. In some examples, a summarization or visibility filter may be applied so that semantically relevant or user- visible portions of the content are retained. The extracted text may be serialized into a structured representation (e.g., JSON or labeled key-value pairs) and linked to the corresponding resource locator 135.Atty Docket No. 0120-920W01

[0086] The index information 138 may be retrieved or generated from an index structure 164 maintained by the browser application 108 or the synchronizer 162. In some examples, the index structure 164 may be a transient, session-specific index constructed from the resource locators 135 of the shared tab group 125 and enriched with metadata from local history, bookmarks, or cached search indices. Each index entry can include fields such as title, summary text, timestamp, and / or collaborator attribution. Engagement data 140, including likes, favorites, or dwell-time measures, may be converted into textual or numerical descriptors (e.g., a text entry such as “User A liked message 5” or a weighted scalar reflecting relative interest). These structured data elements may be combined to form a cohesive contextual data object that the prompt generator 112 inserts into the prompt 114 transmitted to the language model 154.

[0087] In some examples, the contextual data 118 may include information about resources beyond browser tabs (e.g., information about non-web resources included in the shared group). For example, the shared computing session 120 may also include files accessed through a server-based storage application, local documents opened in a productivity application, or interfaces to other applications 106. In some examples, the contextual data 118 may include identifiers or metadata of these resources (e.g., file names, file types, access permissions, timestamps of creation or modification, and / or unique application identifiers).

[0088] In some examples, the contextual data 118 may also include state information about user devices participating in the shared computing session. For example, the contextual data 118 may include device identifiers, operating system information, or timing information (e.g., a system clock value, time zone data, and / or elapsed time since a last activity). This device-related state information may assist the language model 154 in tailoring its output to the circumstances of the collaboration. In further examples, the contextual data 118 may include engagement data that extends beyond explicit reactions in the interactive interface 128. In some examples, the contextual data 118 may include cursor movements, scrolling activity, dwell time on particular web resources, and / or patterns of user interaction that indicate interest levels in certain web resources within the shared computing session 120. Incorporating such implicit engagement signals may allow the language model 154 to weigh web resources more heavily when generating responses.

[0089] Further to the descriptions above, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs or features described herein may enable collection of user information, and if the user is sent content orAtty Docket No. 0120-920W01 communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user. The collaborative session manager 110 includes a prompt generator 112 configured to generate a prompt 114 with the user query 116 and the contextual data 118.

[0090] In some examples in which the collaborative session manager 110 is implemented within the browser application 108, the collaborative session manager 110 generates the contextual data 118 by accessing (e.g., directly accessing) one or more subsystems of the browser application 108 that store information about the shared tab group 125. The browser application 108 may store internal representations of all active browser tabs, including uniform resource locators (e.g., resource locators 135), page titles, navigation history, and document object model (DOM) structures associated with the respective browser tabs. The collaborative session manager 110 can query these internal data structures to identity' the tabs 126 in the shared tab group 125 and extract page content, metadata (e.g., meta description fields, schema markup, keywords), and / or structural descriptors of each page. The collaborative session manager 110 may also obtain window state data 134 maintained by the browser application 108, such as whether a tab 126 is currently in the foreground, a last active timestamp, and whether the tab 126 is pinned, muted, or otherwise modified. The collaborative session manager 110 then includes this information in the contextual data 118 to capture a representation of the shared computing session 120 as maintained by the browser application 108.

[0091] In some examples, the collaborative session manager 110 implemented within the browser application 108 further enhances the contextual data 118 by integrating index information 138 from an index structure 164 provided by the browser application 108. The index structure 164 may include tokens, terms, and identifiers corresponding to resource locators 135 that are currently loaded in the shared tab group 125. For example, the browser application 108 may maintain a history' index, a cache index, or a search index used internally for features such as autocomplete and browsing history search. The collaborative session manager 110 can retrieve descriptors from the index structure 164 and incorporate these descriptors into the contextual data 118 so that the language model 154 can evaluate andAtty Docket No. 0120-920W01 resolve references to the resource locators 135. The collaborative session manager 110 may also record a portion of the chat history 136 and engagement data 140 from the interactive interface 128, thereby integrating conversational context maintained by the browser application 108 with page-1 ev el descriptors for the shared tab group 125.

[0092] In some examples, the collaborative session manager 110 is implemented within the operating system 105 and generates the contextual data 118 by accessing one or more components of the operating system 105 that manage application processes, windows, and file system activity. For example, the collaborative session manager 110 can interface with a window manager of the operating system 105 to identify one or more windows 124 associated with the shared computing session 120. The window7manager of the operating system 105 may store per- window attributes such as application identifiers, window titles, focus state, z-order, and visibility, and the collaborative session manager 110 includes these attributes in the window state data 134 of the contextual data 118. When the windows 124 correspond to brow ser application instances, the collaborative session manager 110 can query7the browser application 108 for resource locators 135 associated with the visible tabs 126. When the shared windows 124 correspond to non-browser applications 106, the collaborative session manager 110 may instead record file paths, application identifiers, or document metadata provided by the operating system 105 as resource locators 135 in the contextual data 118.

[0093] In some examples, the collaborative session manager 110 uses one or more services associated with the operating system 105 services beyond the window manager to generate the contextual data 118. For example, the collaborative session manager 110 may access search or indexing services of the operating system 105, which store an index structure 164 of local files, documents, and / or application resources. From the service(s), the collaborative session manager 110 can retrieve index information 138 (e.g., keywords, identifiers, timestamps, file metadata) corresponding to resources identified as part of the shared computing session 120. The collaborative session manager 110 can also obtain accessibility data or content descriptors exposed by the operating system 105 for windows 124, such as control labels, text fields, and / or semantic role information. These descriptors can be incorporated into the contextual data 118 to provide additional detail to the language model 154.

[0094] In some examples, when implemented within the operating system 105, the collaborative session manager 110 includes device-level state information in the contextual data 118. Such information may include a device identifier, system clock values, time zoneAtty Docket No. 0120-920W01 data, or last interaction timestamps that are retrieved from kernel services or system daemons of the operating system 105. The collaborative session manager 110 may further incorporate engagement data 140 such as input events, click counts, cursor focus changes, or touch interactions recorded by input subsystems of the operating system 105. By combining window state data 134, index information 138, chat history 136, and engagement data 140, the collaborative session manager 110 constructs the contextual data 118 as a comprehensive representation of the shared computing session 120 that reflects both browser application 108 activity and non-browser application 106 activity coordinated by the operating system 105.

[0095] The collaborative session manager 110 may transmit the prompt 114 to a language model 154 for processing. The prompt can include the user query 116 and the contextual data 118. The prompt 114 may be an input representation provided to the language model 154 that specifies a task, request, or query to be performed. In some examples, the prompt 114 may include textual, symbolic, or structured information formatted so that the language model 154 can generate a model response 132. For example, the prompt 114 may be a natural language string, a serialized object in a structured markup language (e.g., JSON, XML), or a multimodal record including textual fields, audio waveforms, image data, or video metadata.

[0096] In some examples, the collaborative session manager 110 may generate the prompt 114 based on the user query 116, the contextual data 118, system instructions, or any combination thereof. The collaborative session manager 110 may further apply promptconstruction rules or templates, such that the user query 1 16 is embedded into a larger instruction block that specifies the expected format or constraints of the model response 132. For example, the collaborative session manager 110 may prepend system instructions that define role behavior of the language model 154 (e.g., “act as a summarize!." “return results in tabular form,’’ or “cite resources with identifiers”).

[0097] In some examples, the collaborative session manager 110 may also augment the prompt 114 with session-specific information. For example, the contextual data 118 may include identifiers of the tabs 126 or shared windows 124 that were active at the time of the query, excerpts of content from those web resources, metadata associated with the collaborators, or historical dialogue messages 130 exchanged through the interactive interface 128. In some examples, the collaborative session manager 110 may normalize or tokenize this information, merge it into a unified prompt 114, and encode the result into a representation consumable by the language model 154 (e.g., embeddings, key -value pairs, or feature vectors).Atty Docket No. 0120-920W01

[0098] In some examples, the prompt generator 112 may construct the prompt 114 by combining the user query 116 with the contextual data 118 using a structured schema that enables parsing (e.g., consistent parsing) by the language model 154. For example, the prompt 114 may be generated in a serialized format such as JavaScript Object Notation (JSON), XML, or other key-value encoding. Each section of the prompt 114 may be labeled to indicate its source or type of information. A representative example may include fields such as a "query” field containing the textual data entered by the user, a “chat history’7field including excerpts of the conversation from the interactive interface 128, and one or more “shared_tab” fields including identifiers, titles, and summarized content extracted from tabs 126 of the shared tab group 125. In some examples, the prompt generator 112 may include metadata describing the origin of each element (e.g., user identifier, timestamp, or tab index) to preserve context continuity between collaborators.

[0099] In some examples, the prompt generator 112 may generate the prompt 114 as an unstructured or semi-structured text block that concatenates the user query' 116 with labeled sections of contextual data 118, each preceded by a human-readable header (e.g.. “Query: ... ”, “Shared Tabs: ... ”, “Engagement Signals: . .. ”). The format used may depend on the capabilities of the language model 154 or a configuration setting of the collaborative session manager 110. Regardless of structure, the prompt 114 provides a unified representation of the user’s intent and the relevant shared context so that the language model 154 can generate the model response 132 based on both the immediate message and the broader collaborative environment.

[0100] In some examples, the collaborative session manager 110 may perform validation or transformation of the prompt 114 before transmitting it to the language model 154. For example, the collaborative session manager 110 may enforce a maximum length constraint, truncate less relevant dialogue history, or compress structured data to ensure the prompt 114 satisfies interface requirements of the language model 154. In some examples, the collaborative session manager 110 may generate multiple alternative prompt candidates, rank them based on relevance scores derived from contextual data 118, and select the highest- scoring candidate as the prompt 114 to transmit.

[0101] In some examples, the prompt 114 may also include machine-interpretable tags or control tokens that influence the operation of the language model 154. For example, the collaborative session manager 110 may insert a tag to indicate the desired temperature or creativity level of the model response 132 or may include a token that specifies a particular domain (e g., medical, legal, or technical) to bias the output of the language model 154. InAtty Docket No. 0120-920W01 some examples, the collaborative session manager 110 may further include resource locators 135 or pointers to external data sources in the prompt 114, thereby enabling the language model 154 or an associated retrieval module to incorporate external information into the response generation process.

[0102] In some examples, the collaborative session manager 110 may generate the prompt 114 in a manner that is specific to the collaborative computing environment. For example, the collaborative session manager 110 may augment the prompt 114 with collaborator identifiers or roles, such that the language model 154 receives not only the textual query, the contextual data 118, but also information indicating which collaborator initiated the query and the identities of other collaborators in the shared computing session 120. In some examples, the collaborative session manager 110 may include metadata identifying the active tab 126, the shared tab group 125, or the shared window 124 at the time of the query. For instance, when a collaborator issues a uery such as “summarize this,” the collaborative session manager 110 may extract content associated with the currently active resource locator 135 and embed the extracted content into the prompt 114.

[0103] In some examples, the collaborative session manager 110 may also generate the prompt 114 in response to non-textual user interactions. For example, when a collaborator highlights text within a shared tab 126 or drags a resource locator 135 into the interactive interface 128, the collaborative session manager 110 may construct a corresponding user query 116 and incorporate it into the prompt 114. In some examples, the collaborative session manager 1 10 may generate the prompt 114 from conversation threads within the interactive interface 128, selectively incorporating only messages 130 that are relevant to the user query 116 rather than the entire session history.

[0104] In some examples, the collaborative session manager 110 may further construct hybrid prompts that combine user-generated content with system-generated candidate queries. For example, when a collaborator posts a user query 116 requesting a summary on the interactive interface 128, the collaborative session manager 110 may also generate candidate queries (e.g., suggestive queries) such as “compare with tab 126-2” or “provide action items” based on the contextual data 118 and may merge the user query 1 16 with the candidate queries to form the prompt 1 14. In some examples, the collaborative session manager 110 may also include role-based instructions (e.g., “format results for presentation to a manager”) derived from collaborator metadata, thereby customizing the behavior of the language model 154 for different participants.Atty Docket No. 0120-920W01

[0105] In some examples, the collaborative session manager 110 may assemble the prompt 114 as a multimodal bundle of structured data. For example, the prompt 114 may include a text field containing the user query 116, an image field containing a screenshot of the shared tab 126, and a structured field encoding relevant contextual data 1 18 such as timestamps, collaborator identifiers, or resource locators 135. The collaborative session manager 110 may then transmit this multimodal prompt 114 to the language model 154 or to an associated retrieval module, enabling the language model 154 to incorporate the multimodal context into the generated model response 132.

[0106] The language model 154 may generate a model response 132 that responds to the user query 116 using the contextual data 118. In this way, the model response 132 can be generated by the language model 154 based on the user query 116 and the contextual data 118. Since the collaborative session manager 110 generates the contextual data 118 for information that is shared to the shared computing session 120 by the collaborators and the language model 154 does not store the contextual data 118 in memory, the collaborative session manager 110 provides one or more technical benefits of maintaining security and / or privacy.

[0107] In some examples, the system may incorporate mechanisms to preserve privacy and data security' when transmitting contextual data 118 to the language model 154. For instance, prompts 114 containing contextual data 118 may be transmitted over an encrypted communication channel, such as a transport layer security (TLS) session or other end-to-end encryption protocol. In some examples, the collaborative session manager 110 may apply message-level encryption before transmission, using a session key established between the browser application 108 and a language-model sendee executing on the server computer(s) 160. In some examples, the server computer(s) 160 may process the prompt within an isolated execution environment — such as a containerized process, virtual machine, or secure enclave — configured to discard transient data after (e.g., immediately after) the model response 132 is generated.

[0108] In some examples, the synchronizer 162 or language-model service may enforce a stateless inference policy to minimize (e.g., prevent) storage of user-supplied content beyond the duration of model inference. Logging of prompts or contextual data 118 may be disabled or replaced with anonymized operational metrics (e.g., token counts, latency statistics). The browser application 108 may further implement client-side controls that redact or obfuscate personally identifiable information before generating the prompt 114. These technical measures, in combination with the transient-memory behavior of theAtty Docket No. 0120-920W01 language model 154, may help ensure that contextual data 118 associated with a shared computing session 120 remains ephemeral and inaccessible after generation of the corresponding model response 132.

[0109] The system 100 may enable collaborators to interact together in real time with the language model 154, which is aware of the shared context (e.g., the contextual data 118). The model response 132 may be an output generated by a language model 154 in response to a prompt 114. A model response 132 may include natural language text, structured data, executable instructions, or other machine-interpretable content. The model response 132 reflects the language model’s processing of the prompt 114 and may be displayed to collaborators, used to perform a computer task 173, or incorporated into the shared computing session 120.

[0110] The collaborative session manager 1 10 provides the model response 132 in the interactive interface 128, and at least a portion of the model response 132 is viewable by the collaborators of the shared computing session 120. Display of the model response 132 can be initiated in the interactive interface 128. For example, in response to a user query 116 (e.g., ■‘What’s the average temperature in Kihei”) posted to the interactive interface 128, the language model 154 uses the user query 116 and the contextual data 118 to generate a model response 132 that is responsive to the user query 116 (e.g., “The average temperature in Kihei in June is... .”).

[0111] In some examples, the model response 132 identifies a particular resource locator 135 that is included in the contextual data 118 and used to generate the model response 132. In some examples, the model response 132 includes a selectable link, which, when selected, causes an application 106 (e.g., the browser application 108) to render web content that was used to formulate the model response 132. In some examples, if the resource locator 135 corresponds to a resource that is already associated with an existing tab 126 within the shared tab group 125, selection of the link may cause the browser application 108 to switch focus to that tab, thereby bringing the corresponding content to the foreground. If the resource locator 135 corresponds to a resource not currently represented within the shared tab group 125, selection of the link may cause the browser application 108 to create and render a new tab 126 including the resource, where the new tab 126 becomes part of the shared tab group 125.

[0112] In some examples, the language model 154 may be used to evaluate and answer questions about a current set of resource locators 135 in the shared tab group 125. In some examples, the language model 154 may be used to summarize content across theAtty Docket No. 0120-920W01 current set of resource locators 135 in the shared tab group 125 and provide one or more recommendations (e.g., resources identified by the language model 154 or one or more proposed user queries for actions available to be taken by the language model 154). In some examples, the language model 154 may be used to generate a shared document with information from the shared computing session 120. In some examples, the language model 154 may be used to generate a set of tasks for the collaboration group to complete.

[0113] In some examples, the model response 132 is used to perform a computer task 173 with respect to the shared computing session 120. In some examples, the collaborative session manager 110 includes a task executor 172 configured to perform a computer task 173 using at least a portion of the model response 132. Performing a computer task 173 may include creating and sharing a computing resource (e.g., a document, web application, a web document, a website, etc.) and / or modifying or arranging a tab group 125. In some examples, the computer task 173 includes opening a computer resource (e.g., web application) such as creating a web document (and, in some examples, populating the web document with information from the model response 132), rendering a new browser tab in the shared tab group 125 with a new webpage recommended or suggested by the language model 154. In some examples, the model response 132 includes instructions generated by the language model 154 to implement the computer task 173. The instructions may include system instructions, structured data, JavaScript Object Notation (JSON), and / or source code (e.g., JavaScript), generated by the language model 154, to implement the computer task 173.

[0114] In some examples, the model response 132 may cause the task executor 172 to launch a software application, which may include a web application, native application, hybrid application, or embedded application interface. The task executor 172 may launch such an application in response to structured instructions provided by the language model 154 and may populate the application with generated content, configure settings, or trigger application-specific actions. In some examples, the model response 132 may instruct creation of a new instance of a web application or collaborative interface. For example, the task executor 172 may instantiate a new document editor, spreadsheet application, form builder, code editor, or dashboard interface and populate the interface with model-generated content before adding the instance to the shared group. Other examples of web applications launched using structured instructions include online whiteboards, diagramming tools, task management platforms, calendar interfaces, dashboards, and cloud-hosted development environments. The model response 132 may populate these interfaces with generated content or other structured information before the resource is added to the shared group.Atty Docket No. 0120-920W01

[0115] In some examples, the model response 132 may include recommendations that are programmatically derived from the contextual data 118 to assist a collaboration group. The recommendations may identify one or more resource 280 to add to the shared tab group 125, propose actions to resolve open questions in the chat history 136, or surface conflicts detected across tabs 126 (e.g., inconsistent dates, prices, or policies). In some examples, the recommendations are prioritized using engagement data 140 (e.g., dwell time, reactions) and index information 138. and, in some examples, may include rationales that reference specific resource locators 135 and highlighted passages. The interactive interface 128 may render a recommendation (e.g., each recommendation) as a selectable element that, when activated, causes the browser application 108 to switch to a relevant tab 126, render a new tab 126 containing the resource 280, or submit another user query 116 to the language model 154.

[0116] In some examples, the recommendation includes a proposed user query, which, when selected by a collaborator, prompts the language model 154 to perform an action (e.g., a computer task 173). In some examples, the collaborative session manager 110 or prompt generator 112 may programmatically generate a proposed user query based on the contextual data 1 18 or the state of the shared computing session 120. For instance, the system may analyze recent collaborator messages 130, updates to the shared tab group 125, or changes in engagement data 140 to determine that additional information may be relevant. The system may then construct a candidate query (e.g., “summarize these updates,'’ “compare current drafts,” or “find related resources”) as a proposed user query. In some examples, the language model 154 may also generate a proposed user query based on its prior responses or understanding of the shared context and transmit the proposed user query' to the user device for display. The proposed user query may be displayed within the interactive interface 128 as a selectable element, such as a button, chip, or text link. In some examples, the interactive interface 128 may include a selectable control (e.g., “identify available actions”) that, when selected, causes the language model 154 to identify and return a list of actions that the language model can perform given the current shared context. When a collaborator selects a proposed user query or action from the interactive interface 128, the system transmits the selected user query to the language model 154 as the user query 116 to obtain a corresponding model response 132.

[0117] In some examples, the model response 132 may specify generation of a task list for the collaborators. The task executor 172 may create a structured checklist object in the interactive interface 128 or within a shared document resource 280. where each task entry includes a label, an optional assignee suggestion, a due date extracted from the chat history'Atty Docket No. 0120-920W01136 or page content, and / or links to one or more resource locators 135. The checklist may be synchronized across the shared computing session 120 so that state changes (e.g., completion, reassignment) are propagated to all user devices 102. In some examples, the task executor 172 updates the checklist in response to subsequent messages 130 (e.g., marking a task complete when a collaborator confirms completion), thereby maintaining a live, session- scoped action plan.

[0118] In some examples, the model response 132 may instruct creation or opening of a shared document and population of content generated by the language model 154. Upon receiving such instructions, the task executor 172 may open an existing document resource associated with the shared computing session 120 or create a new document resource and add it to the shared tab group 125. The task executor 172 may then insert structured sections (e.g., title, summary, bullet lists, tables, etc.) and embed citations to resource locators 135 and quoted excerpts identified in the recommendations. In some examples, the task executor 172 enforces collaborator permissions before modifying the document, records version metadata (e.g., timestamp, collaborator identifier), and applies formatting that highlights portions originating from specific resources (e.g., the parking discussion on a webpage), thereby enabling collaborators to review or revert inserted content.

[0119] In some examples, the model response 132 may be encoded as a structured command that the task executor 172 validates and executes. A command may include an action field (e.g.. create task list, add task, open or create document, insert section. add_reference, open_or_focus_tab), a target field identifying a resource 280 or tab 126, and parameters specifying content (e.g., section text, assignee, due-date, links to resource locators 135). The task executor 172 may verify that an action (e.g., each action) is authorized and, upon validation, perform the corresponding computer task 173, including switching to an existing tab 126, rendering anew tab 126 within the shared tab group 125, or populating the shared document with the generated content.

[0120] A computer task 173 may be a step or sequence of steps executable by the user device 102-1. For example, the operating system 105 and / or the browser application 108 may include individual commands (e.g., shortcut keys) or user controls that enable the user to perform some actions with respect to settings, applications, UI interactions, etc. However, these preexisting commands or controls may be limited, which can cause the user to perform multiple actions to accomplish a particular computer task. A computer task 173 may include a sequence of multiple operations such as opening an application or a file, programmatically performing a clicking operation, inserting data into a text box, and saving the file. TheAtty Docket No. 0120-920W01 system 100 may enable the generation of a wide variety of computer tasks 173 that were previously not defined by the operating system 105.

[0121] The computer tasks 173 may extend beyond rendering or arranging tabs 126 and may include creating or modifying resources across multiple applications or system contexts. In some examples, the task executor 172 may initiate generation of a document, spreadsheet, presentation, or other digital artifact based on structured data contained in the model response 132. In some examples, the task executor 172 may perform actions in native operating system environments such as launching a productivity application, creating a calendar entry, adjusting application or system settings, or populating user interface fields with information generated by the language model 154.

[0122] In some examples, the task executor 172 may be configured to process structured outputs provided by the language model 154. In some examples, the model response 132 may include JavaScript Object Notation (JSON), Extensible Markup Language (XML), or other structured representations that identify the actions to be performed. The task executor 172 may parse such instructions and implement corresponding system operations. In some examples, the model response 132 may include source code, scripts, or programmatic instructions, which the task executor 172 may interpret to perform a defined sequence of steps, such as interacting with a remote application programming interface (API) to retrieve or post data on behalf of the collaboration group.

[0123] In some examples, the task executor 172 may support automation of multi-step workflows. In some examples, a model response 132 may specify that a relevant webpage be retrieved, summarized, and appended to a shared document, followed by notifying collaborators of the update. In some examples, the task executor 172 may orchestrate a sequence of actions across different applications and services to complete the workflow without requiring manual intervention by the collaborators. By enabling execution of model- driven tasks that span multiple contexts, the collaborative session manager 110 provides technical benefits including reduction of redundant user operations, consistent application of group instructions, and improved efficiency in collaborative computing sessions.

[0124] In some examples, the task executor 172 may implement a restricted process (e g., a sandboxed execution environment) configured to interpret structured instructions returned in the model response 132 while preventing unauthorized access to system resources of the user device 102. The restricted process may isolate execution within a restricted process context that is limited to approved browser or application programming interfaces (APIs) and that prohibits file-system writes, network calls outside predefined domains, orAtty Docket No. 0120-920W01 direct operating-system commands. The task executor 172 may thus safely execute automation tasks — such as generating a document, opening a new tab 126, or populating a form field — without exposing the underlying operating system 105 or applications 106 to arbitrary code execution. In some examples, the task executor 172 validates the model response 132 against a defined schema before execution and rejects instructions that fall outside expected data types or parameter ranges.

[0125] In some examples, the model response 132 may conform to a command schema defining a set of permissible operations. For example, the schema may specify fields such as “action,” “target,” and “parameters,” where the “action” field identifies the operation to perform (e.g., create_tab, generate_document, summarize_text), the “target” field identifies the resource 280 or interface element to which the operation applies, and the “parameters” field provides structured data values or user-visible text. When the task executor 172 receives a model response 132 encoded in this format (for example, in JSON), it parses the fields, verifies the action against an approved command list, and dispatches the corresponding API call to the browser application 108 or collaborative session manager 110. These safeguards ensure that execution of model-generated instructions is deterministic, auditable, and confined to explicitly authorized functions of the shared computing session 120.

[0126] In some examples, the model response 132 may specify a communication task for the collaboration group. The task executor 172 may generate a notification or message element identifying an update, recommendation, or assigned collaborator action and transmit the message to one or more user devices 102 participating in the shared computing session 120. In some examples, the task executor 172 may post the notification directly to the interactive interface 128 or transmit the notification through an external messaging or collaboration service. The notification may include contextual references to the shared tab group 125, such as links to relevant resources 280 or citations to specific collaborator contributions, thereby improving group awareness and coordination.

[0127] In some examples, the model response 132 may specify a data analysis or insight-generation task. The task executor 172 may collect engagement data 140, contextual data 118, or resource metadata to compute summary statistics or trend analyses for the collaboration group. The language model 154 may assist by generating natural-language summaries or visual descriptions of the analytic results. In certain implementations, the results may be rendered in the interactive interface 128 as charts, tables, or textual summariesAtty Docket No. 0120-920W01 that capture activity patterns across the shared computing session 120, such as collaborator participation rates, topic frequency, or the number of updates per resource.

[0128] In some examples, the model response 132 may specify an integration task that involves interaction with an external application or service. For instance, the task executor 172 may call a remote application programming interface (API) to create a calendar event, populate a form, or store a generated document in a connected file repository. In some examples, the task executor 172 may authenticate with the external service using credentials associated with the collaboration group and may insert metadata (e.g., resource locators 135, timestamps, or collaborator identifiers) to maintain continuity between the shared computing session 120 and the external application. These integration tasks enable the collaboration group to extend language-model-driven operations across multiple application domains and services while maintaining centralized control and context synchronization.

[0129] A browser application 108 is a web browser configured to access information on the Internet. The browser application 108 may launch one or more tabs 126 in the context of one or more browser windows on a display 122 of the user device 102-1. A tab 126 may display content (e.g., web content) associated with a web document (e.g., webpage, PDF, images, videos, etc.) and / or an application 106 such as a web application, progressive web application (PWA), and / or extension. A web application may be an application program that is stored on a remote server (e.g., a web server) and delivered over the network 150 through the browser application 108 (e.g., a tab 126). In some examples, a progressive web application is similar to a web application but can also be stored (at least in part) on the user device 102-1 and used offline. An extension adds a feature or function to the browser application 108. In some examples, an extension may be HTML, CSS, and / or JavaScript based (for browser-based extensions).

[0130] The language model 154 may include any type of pre-trained large language model (LLM) configured to generate a model response 132 in response to a prompt 114. The language model 154 includes weights. The weights are numerical parameters that the language model 154 leams during the training process. The weights are used to compute the output (e.g., the model response 132) of the language model 154. In some examples, the language model 154 includes a pre-trained language model configured to generate a model response 132 in response to a prompt 114 but has been fine-tuned with additional training data to generate a model response about a shared computing session 120. In some examples, the language model 154 is stored on one or more server computers 160. In some examples, the language model 154 is stored locally on a user device.Atty Docket No. 0120-920W01

[0131] The language model 154 may receive text input. In some examples, the language model 154 may be a multi-modal model configured to receive image or text. The text input includes the information from the prompt 1 14. The language model 154 includes a pre-processing engine configured to pre-process the text input. Pre-processing may include converting the text input to individual tokens (e.g., words, phrases, or characters). Preprocessing may include other operations such as removing stop words (e.g., “the'’, “and", “of’) or other terms or syntax that do not impart any meaning to the language model 154. The language model 154 includes an embedding engine configured to generate word embeddings from the pre-processed text input. The word embeddings may be vector representations that assist the language model 154 to capture the semantic meaning of the input tokens and may assist the language model 154 to better understand the relationships between the input tokens.

[0132] The language model 154 includes one or more neural networks configured to receive the word embeddings and generate an output. A neural network includes multiple layers of interconnected neurons (e.g., nodes). The neural network may include an input layer, one or more hidden layers, and an output layer. The output may include a sequence of output word probability distributions, where each output distribution represents the probability7of the next word in the sequence given the input sequence so far. In some examples, the output may be represented as a probability distribution over the vocabulary or a subset of the vocabulary. The neural network(s) is configured to receive the word embeddings and generate an output, and, in some examples, the query activity (e.g., previous natural language queries and textual responses from the language model 154). The output may represent a version of the textual response. The output may include a sequence of output word probability distributions, where each output distribution represents the probability of the next word in the sequence given the input sequence so far. In some examples, the output may be represented as a probability7distribution over the vocabulary^ or a subset of the vocabulary7. The decoder is configured to receive the output and generate the model response 132. In some examples, the decoder may select the most likely instruction, sampling from a probability distribution, or using other techniques to generate coherent and valid model response 132. The language model 154 includes a decoder configured to receive the output and generate a model response 132.

[0133] The user device 102-1 may be any type of computing device that includes one or more processors 101, one or more memory devices 103, a display 122, and an operating system 105 configured to execute (or assist with executing) one or more applications 106,Atty Docket No. 0120-920W01 including a browser application 108. In some examples, the user device 102-1 is a laptop computer. In some examples, the user device 102-1 is a desktop computer. In some examples, the user device 102-1 is a tablet computer. In some examples, the user device 102- 1 is a smartphone. In some examples, the user device 102-1 is a wearable device. In some examples, the display 122 is the display of the user device 102-1. In some examples, the display 122 may also include one or more external monitors that are connected to the user device 102-1.

[0134] The operating system 105 is a system software that manages computer hardware, software resources, and provides common services for the applications 106. In some examples, the operating system 105 is an operating system designed for a larger display 122 such as a laptop or desktop (e.g., sometimes referred to as a desktop operating system). In some examples, the operating system 105 is an operating system for a smaller display 122 such as a tablet or a smartphone (e.g., sometimes referred to as a mobile operating system). User device 102-2 may include any of the features as explained with reference to user device 102-1.

[0135] The processor(s) 101 may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof. The processor(s) 101 can be semiconductor-based - that is, the processors can include semiconductor material that can perform digital logic. The memory device(s) 103 may include a main memory that stores information in a format that can be read and / or executed by the processor(s) 1 1 . The memory device(s) 103 may store the operating system 105, including the collaborative session manager 110. In some examples, the memory device(s) 103 includes a non-transitory computer-readable medium that includes executable instructions that cause at least one processor (e.g.. the processors 101) to execute the operations discussed herein.

[0136] The server computer(s) 160 may be computing devices that take the form of a number of different devices, for example a standard server, a group of such servers, or a rack server system. In some examples, the server computer(s) 160 may be a single system sharing components such as processors and memories. In some examples, the server computer(s) 160 may be multiple systems that do not share processors and memories. The network 150 may include the Internet and / or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, satellite network, or other types of data networks. The network 150 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) that are configured to receive and / orAty Docket No. 0120-920W01 transmit data within network 150. Network 150 may further include any number of hardwired and / or wireless connections.

[0137] In some examples, the language model 154 is stored on the server computer(s) 160. In some examples, the language model 154 is stored on the user device 102-1. The sen' er computer(s) 160 may include one or more processors 161 formed in a substrate, an operating system (not shown) and one or more memory devices 163. The memory device(s) 163 may represent any kind of (or multiple kinds of) memory (e.g., RAM, flash, cache, disk, tape, etc.). In some examples (not shown), the memory devices may include external storage, e.g., memory physically remote from but accessible by the sener computer(s) 160. The processor(s) 161 may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof. The processor(s) 161 can be semiconductor-based - that is, the processors can include semiconductor material that can perform digital logic. The memory device(s) 163 may store information in a format that can be read and / or executed by the processor(s) 161. In some examples, the memory device(s) 163 includes anon-transitory computer-readable medium that includes executable instructions that cause at least one processor (e.g., the processor(s) 161) to execute operations.

[0138] In some examples, the language model 154 may be deployed on a server computer 160 that is remote from the user devices participating in the shared computing session 120. In some examples, prompts 114 generated by the collaborative session manager 1 10 are transmitted over a network 150 to the server computer 160, and the model response 132 is returned to the user devices for display in the interactive interface 128. In some examples, the language model 154 may be deployed locally on a user device such that the prompt is processed entirely on-device without transmitting contextual data beyond the device.

[0139] In some examples, the system 100 may employ a hybrid architecture in which certain operations are performed locally while other operations are performed remotely. For example, the collaborative session manager 110 may generate the contextual data and pre- process the prompt 1 14 on the user device, while the server computer 160 performs inference to generate the model response 132. In some examples, lightweight models may be executed on the user devices to provide rapid feedback, while more complex or resource-intensive tasks are delegated to a remote server. In some examples, the system 100 may implement a privacy-preserving execution environment for handling contextual data. For example, the collaborative session manager 110 or portions thereof may operate within a trusted executionAtty Docket No. 0120-920W01 environment (TEE) or secure enclave such that contextual data is encrypted and isolated from other system components while being processed.

[0140] In some examples, the interactive interface 128 may be presented in an immersive environment such as an augmented reality (AR) or virtual reality (VR) system. In such cases, the shared computing session 120 may include spatially arranged resources (e.g., browser windows, 3D objects, or collaborative whiteboards), and the contextual data 118 may further include spatial coordinates, orientation data, or user gaze tracking information. In some examples, the user query 116 may be expressed in modalities other than text, such as voice commands, pen input, gestures, head movements, eye movement, or hand movements, or captured images. The model response 132 may likewise be provided in multimodal forms including synthesized speech, rendered graphics, or dynamically generated 3D objects. In some examples, the collaborative session manager 110 may proactively generate or update the contextual data 118 in anticipation of potential queries. For example, the system 100 may detect patterns of collaborator activity and pre-compute contextual embeddings, thereby reducing latency when a user query 116 is received. In some examples, the task executor 172 may be configured to communicate with external services or connected devices. For example, a model response 132 may trigger operations such as scheduling calendar events, controlling Internet of Things (loT) devices, or initiating secure financial transactions through external APIs.

[0141] FIG. 2 illustrates a user interface depicting a shared computing session with a shared tab group 225 and an interactive interface 228 with a plurality of messages (e.g., message 230-1, message 230-2, message 230-3, message 230-4, message 230-5) posted by users (e.g., user A, user B, user C, user D) of a collaboration group. The user interface of FIG. 2 may be an interface provided by the system 100 of FIGS. 1A and IB and may include any of the details discussed herein. In some examples, the language model is invoked when a model identifier 275 (e.g., @LLM) is included in a message 230. A model identifier 275 may refer to a token, string, keyword, or other signal included in a user input that indicates the input should be processed by a language model 154. For example, a model identifier may be an explicit tag such as the name of the model included within a message posted to the interactive interface 128. In some examples, a model identifier 275 may be a voice command, a gesture, or another input modality that invokes a language model 154 to process the input content.

[0142] The shared tab group 225 includes a browser tab with a webpage, and the interactive interface 228 is located adjacent to the webpage. In response to user A posting aAtty Docket No. 0120-920W01 message 230-4 that identifies a language model using a model identifier 275 to the interactive interface 228 via an input field 268, a collaborative session manager (e.g., the collaborative session manager 110 of FIG. IB) may detect the submission of a user query, and the collaborative session manager may generate a prompt (e.g., prompt 114 of FIG. IB) with the textual data of the message 230-4 along with the contextual data (e.g., the contextual data 118 of FIG. IB). In response to the prompt, a language model (e.g., the language model 154 of FIG. IB) may generate a model response 232 that responds to the message 230-4. The model response 232 is displayed in the interactive interface 228. In some examples, the contextual data includes engagement data 240 from the interactive interface 228. In some examples, the model response 232 may identify a resource 280 that assists with answering the message 230- 4, where the resource 280 may be any computer resource identifiable by a resource locator (e.g., a URL).

[0143] In some examples, the model response 232 may include or reference a selectable element corresponding to the resource 280. The selectable element may be a hyperlink, button, or other graphical user interface control that, when selected, causes the browser application to display the resource 280 in a browser tab. If the resource 280 corresponds to a webpage or other network-addressable object that is already associated with one of the tabs in the shared tab group 225, selection of the element may cause the browser application to switch focus to that existing tab, thereby bringing the existing tab to the foreground on the display. If the resource 280 is not already associated with any tab in the shared tab group 225, selection of the element may instead cause the browser application to create and render a new tab containing the resource 280, where the new tab becomes part of the shared tab group 225 so that it is visible to the collaborators participating in the shared computing session.

[0144] In some examples, upon switching to or rendering the tab with the resource 280, the browser application may highlight or emphasize a relevant portion of the content of the resource 280 that pertains to the subject matter of the message 230-4 or the user query7. The highlighting may include visually distinguishing a section of the webpage, scrolling the display to a particular position, or temporarily applying a UI effect (e.g., a color overlay or animation) to the relevant text or image. For example, in the example of FIG. 2, the browser application may programmatically scroll to and highlight the portion of the webpage that discusses parking availability7for the rental property so that collaborators can immediately view the information referenced by the language model without performing manual navigation.Atty Docket No. 0120-920W01

[0145] FIG. 3 is a flowchart 300 depicting example operations of initiating model responses for a shared computing session that uses contextual data across a plurality of user devices according to an aspect. The flowchart 300 may depict operations of a computer- implemented method. Although the flowchart 300 of FIG. 3 illustrates the operations in sequential order, it will be appreciated that this is merely an example, and that additional or alternative operations may be included. Further, operations of FIG. 3 and related operations may be executed in a different order than that shown, or in a parallel or overlapping fashion.

[0146] The example operations that implement a technical solution involving an application (e.g., a browser application, an operating system, or non-browser application) that integrates a language model in a shared computing session betw een multiple user devices (e.g., collaborators) in a manner that overcomes one or more technical problems relating to security or privacy. For example, the application generates contextual data about the shared computing session and includes the contextual data in a prompt to a language model, which causes the language model to generate a model response that responds to a user query posted to an interactive interface by a collaborator. In other words, in response to a message posted to the interactive interface by a collaborator, the language model can be invoked to generate a model response that responds to a user query included in the message. Because the operations discussed herein generates the contextual data for information that is shared to the shared computing session by the collaborators and the language model does not store the contextual data in memory, the technical solution provides one or more technical benefits of maintaining security and / or privacy. The operations of FIG. 3 may enable collaborators to interact together in real time with the language model, which is aware of the shared context (e.g., the common contextual data) to execute a task in an application in an integrated manner.

[0147] Operation 302 includes rendering a shared tab group on a user device, the shared tab group including a plurality of tabs. A plurality of tabs may refer to two or more browser tabs, application tabs, or other visual containers of computing resources that are part of a shared tab group. Each tab may correspond to a distinct computer resource, such as a webpage, a document, a web application, a native application window, or another interface. The plurality of tabs may be rendered within a single browser window, across multiple windows, or within another application environment that supports tabbed interfaces.

[0148] Operation 304 includes detecting a user query from input to an interactive interface. An interactive interface may be a user-accessible interface that enables collaborators to provide inputs or view outputs associated with a shared computing session.Atty Docket No. 0120-920W01Examples include but are not limited to interactive interfaces, document editing interfaces, or other application interfaces. Operation 306 includes generating contextual data about the plurality of tabs in the shared tab group. Operation 308 includes transmitting a prompt to a language model, the prompt including the user query and the contextual data. Operation 310 includes receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data. Operation 312 includes initiating display of the model response in the interactive interface.

[0149] Clause 1. A method comprising: rendering a shared tab group on a user device, the shared tab group including a plurality of tabs; detecting a user query from an input to an interactive interface associated with the shared tab group; generating contextual data about the plurality of tabs in the shared tab group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query7and the contextual data; and initiating display of the model response in the interactive interface.

[0150] Clause 2. The method of clause 1, wherein detecting the user query from the input includes: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query.

[0151] Clause 3. The method of clause 1 or 2. wherein the model response includes a selectable element corresponding to a resource identified by the language model, the method further comprising: in response to a selection of the selectable element, rendering a new tab with the resource in the shared tab group.

[0152] Clause 4. The method of any one of clauses 1 to 3. wherein the user device is a first user device, the plurality of tabs including a first tab added by the first user device, and a second tab added by a second user device, the contextual data including a first resource locator of a first webpage displayed in the first tab, and a second resource locator of a second webpage displayed in the second tab.

[0153] Clause 5. The method of clause 4, wherein the contextual data includes information, obtained from an index, about the first resource locator and the second resource locator and at least a portion of a history' associated with the interactive interface.

[0154] Clause 6. The method of any one of clauses 1 to 5. wherein the model response includes structured instructions configured to cause the user device to perform a task including generating, updating, or displaying a resource.Atty Docket No. 0120-920W01

[0155] Clause 7. The method of clause 6, further comprising: generating the resource; adding content, generated by the language model, to the resource; and adding the resource to the shared tab group.

[0156] Clause 8. A computing device comprising: at least one processor; and a non- transitory computer-readable medium storing executable instructions that cause the at least one processor to: render a shared tab group on the computing device, the shared tab group including a plurality of tabs; detect a user query from an input to an interactive interface associated with the shared tab group; generate contextual data about the plurality of tabs in the shared tab group; transmit a prompt to a language model, the prompt including the user query and the contextual data; receive a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiate display of the model response in the interactive interface.

[0157] Clause 9. The computing device of clause 8, wherein the executable instructions include instructions that cause the at least one processor to: determine that the input includes a language model identifier; and in response to the input being determined as including the language model identifier, identity' at least some content of the input as the user query.

[0158] Clause 10. The computing device of clause 8 or 9, wherein the computing device is a first user device, the plurality of tabs including a first tab added by the first user device, and a second tab added by a second user device, the contextual data including a first resource locator of a first webpage displayed in the first tab, and a second resource locator of a second webpage displayed in the second tab, wherein the contextual data includes information, obtained from an index, about the first resource locator and the second resource locator.

[0159] Clause 11. The computing device of any one of clauses 8 to 10, wherein the executable instructions include instructions that cause the at least one processor to: generate a proposed user query for an action of the language model; and display the proposed user query as a selectable element in interactive interface.

[0160] Clause 12. The computing device of any one of clauses 8 to 11, wherein the model response includes a selectable element corresponding to a resource identified by the language model, wherein the executable instructions include instructions that cause the at least one processor to: in response to a selection of the selectable element, switch to a tab of the plurality of tabs that displays the resource.Atty Docket No. 0120-920W01

[0161] Clause 13. The computing device of any one of clauses 8 to 12, wherein the model response includes structured instructions that cause the computing device to: generate a resource; add content, generated by the language model, to the resource: and add the resource to the shared tab group.

[0162] Clause 14. The computing device of any one of clauses 8 to 13, wherein the executable instructions include instructions that cause the at least one processor to: detect that the input includes an invocation of the language model without inclusion of a language model identifier in the input; and identify the user query from at least a portion of the input.

[0163] Clause 15. A non-transitory computer-readable medium storing executable instructions that cause at least one processor to execute operations, the operations comprising: rendering a shared tab group on a user device, the shared tab group including a plurality of tabs; detecting a user query from an input to an interactive interface associated with the shared tab group; generating contextual data about the plurality of tabs in the shared tab group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of the model response in the interactive interface.

[0164] Clause 16. The non-transitory computer-readable medium of clause 15, wherein the operations further comprise: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query.

[0165] Clause 17. The non-transitory computer-readable medium of clause 15 or 16, wherein the model response includes a selectable element corresponding to a resource identified by the language model, wherein the operations further comprise: in response to a selection of the selectable element, rendering a new tab with the resource in the shared tab group.

[0166] Clause 18. The non-transitory' computer-readable medium of clause 17, wherein the model response includes structured instructions that cause the user device to: generate a resource; add content, generated by the language model, to the resource; and add the resource to the shared tab group.

[0167] Clause 19. The non-transitory' computer-readable medium of any one of clauses 15 to 18, wherein the operations further comprise: generate a proposed user query for an action of the language model; and display the proposed user query as a selectable element in interactive interface.Atty Docket No. 0120-920W01

[0168] Clause 20. The non-transitory computer-readable medium of any one of clauses 15 to 19, wherein the model response includes a proposed user query for an action of the language model, wherein the operations further comprise: displaying the proposed user query as a selectable element; and in response to a selection of the selectable element, prompting the language model to perform the action.

[0169] Clause 21. A method comprising: rendering a shared group on a user device, the shared group including a plurality of web resources; detecting a user query from an input to an interactive interface associated with the shared group; generating contextual data about the plurality of web resources in the shared group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of at least a portion of the model response in the interactive interface.

[0170] Clause 22. The method of clause 21, wherein detecting the user query from the input includes: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query.

[0171] Clause 23. The method of clause 21 or 22, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, the method further comprising: in response to a selection of the selectable element, rendering the web resource in the shared group.

[0172] Clause 24. The method of any one of clauses 21 to 23, wherein the user device is a first user device, the plurality of web resources including a first web resource added by the first user device, and a second web resource added by a second user device, the contextual data including a first resource locator of the first web resource, and a second resource locator of the second web resource.

[0173] Clause 25. The method of any one of clauses 21 to 24, w herein the plurality of web resources are rendered in a plurality of tabs that form the shared group.

[0174] Clause 26. The method of any one of clauses 21 to 25, wherein the model response includes structured instructions, generated by the language model, configured to cause the user device to perform a task.

[0175] Clause 27. The method of any one of clauses 1 to 7 or any one of clauses 21 to 26. wherein the model response includes structured instructions, generated by the language model, configured to cause the user device to: launch a web application; add content,Atty Docket No. 0120-920W01 generated by the language model, to the web application; and add the web application to the shared group.

[0176] Clause 28. A computing device comprising: at least one processor; and a non- transitory computer-readable medium storing executable instructions that cause the at least one processor to: render a shared group on the computing device, the shared group including a plurality of web resources; detect a user query from an input to an interactive interface associated with the shared group; generate contextual data about the plurality of web resources in the shared group; transmit a prompt to a language model, the prompt including the user query and the contextual data; receive a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiate display of at least a portion of the model response in the interactive interface.

[0177] Clause 29. The computing device of clause 28, wherein the executable instructions include instructions that cause the at least one processor to: determine that the input includes a language model identifier; and in response to the input being determined as including the language model identifier, identify at least some content of the input as the user query.

[0178] Clause 30. The computing device of clause 28 or 29, wherein the computing device is a first user device, the plurality of web resources including a first web resource added by the first user device, and a second web resource added by a second user device, the contextual data including a first resource locator of the first web resource, and a second resource locator of the second web resource, wherein the contextual data includes information, obtained from an index, about the first resource locator and the second resource locator.

[0179] Clause 31. The computing device of any one of clauses 28 to 30, wherein the executable instructions include instructions that cause the at least one processor to: generate a proposed user query for an action of the language model; and display the proposed user query7as a selectable element in the interactive interface.

[0180] Clause 32. The computing device of any one of clauses 28 to 31, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, wherein the executable instructions include instructions that cause the at least one processor to: in response to a selection of the selectable element, switch to a tab that displays the web resource.Atty Docket No. 0120-920W01

[0181] Clause 33. The computing device of any one of clauses 8 to 12 or any one of clauses 28 to 32, wherein the model response includes structured instructions that cause the computing device to: launch a web application; add content, generated by the language model, to the web application; and add the web application to the shared group.

[0182] Clause 34. The computing device of any one of claims 8 to 13 or any one of clauses 28 to 33, wherein the executable instructions include instructions that cause the at least one processor to: detect that the input invokes the language model without inclusion of a language model identifier in the input; and identify the user query from at least a portion of the input.

[0183] Clause 35. A non-transitory computer-readable medium storing executable instructions that cause at least one processor to execute operations, the operations comprising: rendering a shared group on a user device, the shared group including a plurality of web resources; detecting a user query from an input to an interactive interface associated with the shared group; generating contextual data about the plurality of web resources in the shared group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of at least a portion of the model response in the interactive interface.

[0184] Clause 36. The non-transitory computer-readable medium of clause 35. wherein the operations further comprise: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query.

[0185] Clause 37. The non-transitory computer-readable medium of clause 35 or 36, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, wherein the operations further comprise: in response to a selection of the selectable element, rendering the web resource in the shared group.

[0186] Clause 38. The non-transitory computer-readable medium of any one of clauses 15 to 18 or any one of clauses 35 to 37, wherein the model response includes structured instructions that cause the user device to: launch generate a resource; add content, generated by the language model, to the resource; and add the resource to the shared group.

[0187] Clause 39. The non-transitory' computer-readable medium of any one of clauses 35 to 38, wherein the operations further comprise: generate a proposed user query forAtty Docket No. 0120-920W01 an action of the language model; and display the proposed user query as a selectable element in the interactive interface.

[0188] Clause 40. The non-transitory computer-readable medium of any one of clauses 35 to 39, wherein the model response includes a proposed user query for an action of the language model, wherein the operations further comprise: displaying the proposed user query as a selectable element; and in response to a selection of the selectable element, prompting the language model to perform the action.

[0189] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0190] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms “machine-readable medium’" “computer-readable medium” refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0191] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.Atty Docket No. 0120-920W01

[0192] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN’'), a wide area network (“WAN”), and the Internet.

[0193] The computing system can include clients and servers. A client and server are remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship with each other.

[0194] In this specification and the appended claims, the singular forms "a," "an" and "the" do not exclude the plural reference unless the context clearly dictates otherwise. Further, conjunctions such as “and,” “or,” and “and / or” are inclusive unless the context clearly dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B. Further, connecting lines or connectors shown in the various figures presented are intended to represent example functional relationships and / or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the implementations disclosed herein unless the element is specifically described as “essential” or “critical”.

[0195] Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary7skill in the art.

[0196] Moreover, use of terms such as up. dow n, top, bottom, side, end, front, back, etc. herein are used with reference to a currently considered or illustrated onentation. If they are considered w ith respect to another orientation, it should be understood that such terms must be correspondingly modified.

[0197] Although certain example methods, apparatuses and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that terminology employed herein is for the purpose of describing particularAtty Docket No. 0120-920W01 aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.

Claims

Atty Docket No. 0120-920W01WHAT IS CLAIMED IS:

1. A method comprising: rendering a shared group on a user device, the shared group including a plurality of web resources; detecting a user query from an input to an interactive interface associated with the shared group; generating contextual data about the plurality of web resources in the shared group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of at least a portion of the model response in the interactive interface.

2. The method of claim 1, wherein detecting the user query from the input includes: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query-.

3. The method of claim 1 or 2, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, the method further comprising: in response to a selection of the selectable element, rendering the web resource in the shared group.

4. The method of any one of claims 1 to 3, wherein the user device is a first user device, the plurality of web resources including a first web resource added by the first user device, and a second web resource added by a second user device, the contextual data including a first resource locator of the first web resource, and a second resource locator of the second web resource.

5. The method of any one of claims 1 to 4, wherein the plurality of web resources are rendered in a plurality of tabs that form the shared group.Atty Docket No. 0120-920W016. The method of any one of claims 1 to 5, wherein the model response includes structured instructions, generated by the language model, configured to cause the user device to perform a task.

7. The method of any one of claims 1 to 6, wherein the model response includes structured instructions, generated by the language model, configured to cause the user device to: launch a web application; add content, generated by the language model, to the web application; and add the web application to the shared group.

8. A computing device comprising: at least one processor; and a non-transitory computer-readable medium storing executable instructions that cause the at least one processor to: render a shared group on the computing device, the shared group including a plurality of web resources; detect a user query from an input to an interactive interface associated with the shared group; generate contextual data about the plurality of web resources in the shared group; transmit a prompt to a language model, the prompt including the user query and the contextual data; receive a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiate display of at least a portion of the model response in the interactive interface.

9. The computing device of claim 8, wherein the executable instructions include instructions that cause the at least one processor to: determine that the input includes a language model identifier; andAtty Docket No. 0120-920W01 in response to the input being determined as including the language model identifier, identify at least some content of the input as the user query.

10. The computing device of claim 8 or 9, wherein the computing device is a first user device, the plurality of web resources including a first web resource added by the first user device, and a second web resource added by a second user device, the contextual data including a first resource locator of the first web resource, and a second resource locator of the second web resource, wherein the contextual data includes information, obtained from an index, about the first resource locator and the second resource locator.

11. The computing device of any one of claims 8 to 10, wherein the executable instructions include instructions that cause the at least one processor to: generate a proposed user query for an action of the language model; and display the proposed user query7as a selectable element in the interactive interface.

12. The computing device of any one of claims 8 to 11, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, wherein the executable instructions include instructions that cause the at least one processor to: in response to a selection of the selectable element, switch to a tab that displays the web resource.

13. The computing device of any one of claims 8 to 12, wherein the model response includes structured instructions that cause the computing device to: launch a web application; add content, generated by the language model, to the web application; and add the web application to the shared group.

14. The computing device of any one of claims 8 to 13, wherein the executable instructions include instructions that cause the at least one processor to: detect that the input invokes the language model without inclusion of a language model identifier in the input; and identify the user query from at least a portion of the input.Atty Docket No. 0120-920W0115. A non-transitory computer-readable medium storing executable instructions that cause at least one processor to execute operations, the operations comprising: rendering a shared group on a user device, the shared group including a plurality of web resources; detecting a user query from an input to an interactive interface associated with the shared group; generating contextual data about the plurality of web resources in the shared group; transmitting a prompt to a language model, the prompt including the user query and the contextual data; receiving a model response from the language model, the model response being generated by the language model based on the user query and the contextual data; and initiating display of at least a portion of the model response in the interactive interface.

16. The non-transitory computer-readable medium of claim 15, wherein the operations further compnse: determining that the input includes a language model identifier; and in response to determining that the input includes the language model identifier, identifying at least some content of the input as the user query.

17. The non-transitory computer-readable medium of claim 15 or 16, wherein the model response includes a selectable element corresponding to a web resource identified by the language model, wherein the operations further comprise: in response to a selection of the selectable element, rendering the web resource in the shared group.

18. The non-transitory computer-readable medium of any one of claims 15 to 17, wherein the model response includes structured instructions that cause the user device to: launch generate a resource; add content, generated by the language model, to the resource; and add the resource to the shared group.

19. The non-transitory computer-readable medium of any one of claims 15 to 18. wherein the operations further comprise:Atty Docket No. 0120-920W01 generate a proposed user query for an action of the language model; and display the proposed user query as a selectable element in the interactive interface.

20. The non-transitory computer-readable medium of any one of claims 15 to 19, wherein the model response includes a proposed user uery for an action of the language model, wherein the operations further comprise: displaying the proposed user query as a selectable element; and in response to a selection of the selectable element, prompting the language model to perform the action.

Images