Computer-implemented multi-user messaging applications

Abstract

The computing system includes a processor and memory for storing instructions, which, when executed by the processor, cause the processor to perform several actions. These actions include receiving multiple messages from multiple users in a messaging application that supports group conversations, and the multiple messages are included in the group conversation. These actions also include providing a prompt to a generative model, and the prompt includes multiple messages. In addition, these actions include receiving output generated by the generative model based on the prompt and including the output as a turn in the group conversation.

Description

Technical Field

[0001] Cross - Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 451,547, filed on March 10, 2023, entitled "COMPUTER-IMPLEMENTED MULTI-USER MESSAGING APPLICATION WITH AI BOT". The entire content of this application is incorporated herein by reference.

Background Art

[0002] Background

[0002] There are various types of computer - implemented messaging applications that enable multiple users of multiple client computing devices to participate in a group conversation with each other (synchronously and / or asynchronously). Examples of computer - implemented messaging applications include, but are not limited to, text - messaging applications, instant - messaging applications, unified communication (UC) applications, and the like among numerous others.

[0003]

[0003] Relatively recently, generative models have been developed, which include generative language models (GLMs) (also called large-scale language models (LMS)), models that generate images based on input (input can be text, audio, images, etc.), and models that generate videos based on input. One example of a GLM is the Generative Pre-trained Transformer 4 (GPT-4) model. Another example of a GLM is the BigScience Large Open-science Open-access Multilingual language (BLOOM) model, which is also a transformer-based model. Briefly, a generative model is configured to generate output (such as text in a human-readable language, source code, music, video, and the like) based on prompts provided as input to the generative model, and the generative model generates the output in near real time (e.g., within seconds of receiving a prompt).

[0004]

[0004] Computer implementation applications are developed to incorporate generative models. For example, a generative model is incorporated into a chat application, allowing one user to interact with the generative model through the chat application. Accordingly, when a user enters input into the chat application, the generative model generates output based on that input and presents that output to the user. However, conventional generative models are designed to interact with one user at a time, and such a design has prevented their incorporation into computer implementation messaging applications that support group conversations. For example, conventional generative models maintain the conversational context between the user and the generative model through the conversation between the user and the generative model. Because conventional generative models cannot distinguish between different users participating in a conversation (due to the design of the generative model), they cannot maintain the conversational context of a group conversation. [Overview of the project] [Problems that the invention aims to solve]

[0005] overview

[0005] The following is a brief overview of the subject matter of the patent, which is described in more detail herein. This overview is not intended to limit the scope of the claims.

[0006]

[0006] This specification describes various techniques for incorporating generative models into computer-implemented messaging applications that support group messaging (messaging between at least two participants). Computer-implemented messaging applications include text messaging applications, instant messaging applications, unified communications applications, and other applications that support synchronous messaging between participants in a group conversation (computer-implemented messaging applications may also support asynchronous messaging).

[0007]

[0007] In one example, a server computing system receives multiple messages from multiple participants in a group conversation. Messages within multiple messages may include text, images, audio (e.g., music, voice input, etc.), video, or other computer-readable input that can be processed by a generative model. The generative model generates output based on the multiple messages from the multiple participants. The output may include text, images, video, audio, or the like.

[0008]

[0008] There are various techniques that can be employed to enable a generative model to generate output based on multiple messages. In one example, a computer-implemented messaging application (in which a group conversation is taking place) constructs a prompt and provides it to the generative model. The prompt identifies, for example, that the conversation is a group conversation involving multiple users, includes the identities of the users in the group conversation, and further includes multiple messages from the group conversation (each message identifying the user who generated the message). The generative model then generates output based on the prompt. In another example, the generative model is trained to inherently recognize multi-user conversations.

[0009]

[0009] By incorporating a generative model into a computer-implemented messaging application that supports group messaging, many use cases that were previously unavailable become possible. For example, a generative model can summarize group conversations, identify specific user input on a topic, suggest follow-up actions based on group conversations, help participants schedule events within a group conversation, help users adjust their schedules within a group conversation, generate graphics specific to the group conversation (images, avatars, memes, etc.), help rewrite and check the content of group conversations (e.g., modifying messages to be more professional or humorous, correcting typos and grammatical inconsistencies, translating conversations into different languages, etc.), answer questions about the content of uniform resource locators (URLs) included in group conversations (e.g., summarizing news articles, identifying facts mentioned in articles, reading structured data such as corporate financial statements, answering specific questions, etc.), contribute to brainstorming sessions conducted within group conversations, function as a translation engine to translate parts of a group conversation into a language understandable to other members within the group conversation, and start and run games for the purpose of entertaining members within the group conversation.

[0010]

[0010] The above summary is a simplified overview to provide a basic understanding of some aspects of the systems and / or methods discussed herein. This summary is not a comprehensive overview of the systems and / or methods discussed herein, nor is it intended to identify major / important elements or to clarify the scope of such systems and / or methods. Its sole purpose is to present some concepts in a simplified form as a prelude to further detailed explanations to be given later. [Brief explanation of the drawing]

[0011] Brief explanation of the drawing [Figure 1]

[0011] This is a functional block diagram of a computing system in which a generative model is incorporated into a messaging application that supports group conversations. [Figure 2]

[0012] This is a functional block diagram of a client computing device displaying a group conversation that includes entries generated by a bot that uses a generative model to generate output. [Figure 3]

[0013] This is a schematic diagram including a functional block diagram of a computer-implemented messaging application that supports group conversations. [Figure 4]

[0014] This shows the prompts that can be provided to a bot, including a generative model, by a computer-implemented messaging application. [Figure 5]

[0015] This shows the prompts that can be provided to a bot, including a generative model, by a computer-implemented messaging application. [Figure 6]

[0016] This flowchart illustrates how a bot can participate in a group conversation using a computer-implemented messaging application. [Figure 7]

[0017] This is a flowchart illustrating how to construct prompts provided to a bot that includes a generative model. [Figure 8]

[0018] This is a schematic diagram of a computing device. [Modes for carrying out the invention]

[0012] Detailed explanation

[0019] This section describes various techniques for incorporating bot functionality into computer-implemented messaging applications that support group conversations, with reference to the diagrams. Throughout the diagrams, similar elements are indicated using similar numbering. The following descriptions include many specific details to facilitate understanding of one or more embodiments. However, it will be clear that such embodiments can be implemented without these specific details. In other examples, well-known structures and devices are shown in block diagram form to facilitate the description of one or more embodiments. Furthermore, it should be understood that functionality described as being performed by a particular system component can be performed by multiple components. Similarly, functionality described as being performed by multiple components can be configured to be performed by a single component.

[0013]

[0020] Furthermore, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or it is clear from the context, the expression “X adopts A or B” is intended to mean either of the natural and inclusive substitutions. That is, the expression “X adopts A or B” is satisfied by any of the following examples: “X adopts A,” “X adopts B,” or “X adopts both A and B.” In addition, the articles “a” and “an” used in this application and the attached claims should generally be interpreted as meaning “one or plural” unless otherwise specified or it is clear from the context that they refer to a singular form.

[0014]

[0021] Furthermore, as used herein, the terms “component,” “module,” and “system” are intended to encompass computer-readable data storage configured to store computer-executable instructions that, when executed by a processor, perform specific functionalities. Computer-executable instructions may include routines, functions, or similar. It should also be understood that a component or system may be localized to a single device or distributed across multiple devices. Furthermore, as used herein, the term “exemplary” is intended to mean that something serves as an example or illustration of something, and not to indicate a preference.

[0015]

[0022] This specification describes various techniques for incorporating bots containing generative models into computer-implemented messaging applications that support group conversations. As mentioned above, traditional generative models, when incorporated into chat applications, are limited to conversations with a single user. The techniques described herein aim to enable the incorporation of generative models into computer-implemented messaging applications that support group conversations, thereby enabling several use cases that were previously impossible. As will be described in more detail herein, generative models can be incorporated into messaging applications, and such incorporation is made possible on the basis that the messaging application provides prompts to the generative model, which identify that the conversation is a group conversation and further include information about the group conversation (among many pieces of information, the identities of the participants, messages previously sent within the group conversation, etc.). In another example, incorporating a generative model into a messaging application is made possible by training the generative model to recognize that it is involved in a group conversation that includes multiple users (instead of a single user).

[0016]

[0023] Referring now to FIG. 1, there is shown a functional block diagram of a computing architecture 100 that facilitates incorporating a bot that includes a generation model into a computer-implemented messaging application that supports group messaging. Architecture 100 includes a server computing system 102 and a number of client computing devices 104-106. Server computing system 102 communicates with client computing devices 104-106 via a network 108 such as the Internet. Although not shown, each client computing device 104-106 has installed thereon a client messaging application. The computer-implemented messaging application supports synchronous messaging and thus may be an instant messaging application, a unified communication application, or the like. In another example, the messaging application supports asynchronous communication and thus may be a text messaging application. Other computer-implemented messaging applications that support group conversations are also contemplated. The client messaging application installed on client computing devices 104-106 may be a stand-alone application, incorporated into a browser, or the like.

[0017]

[0024] The server computing system 102 includes a processor 110 and memory 112, the memory 112 containing instructions executed by the processor 110. As shown in Figure 1, the memory 112 contains a computer implementation messaging application 114 for enabling users of client computing devices 104-106 to participate in a group conversation. Accordingly, in the embodiment shown in Figure 1, the messaging application 114 has a client-server architecture. However, it should be understood that the functionality described herein can be used in peer-to-peer messaging applications, in which case the bot functions as a peer.

[0018]

[0025] Memory 112 further includes a bot 116 that contains or can access a generative model 118. In one example, the generative model 118 is a transformer-based model. The generative model 118 could be a GLM, and it could be configured to receive text input and produce text output. In such a case, the bot 116 could be a chatbot. In other examples, the generative model 118 could produce audio, images, video (with audio), or other appropriate output based on a variety of inputs such as voice, text, video, images, and audio. Although the bot 116 is shown as being outside the messaging application 114, it is also intended that the bot 116 may be included in the messaging application 114. As will be described in more detail herein, the messaging application 114 receives messages from client computing devices 104-106, which are part of a group conversation between users of client computing devices 104-106. Bot 116 uses the generative model 118 to generate output based on received messages, and this output is provided to client computing devices 104-106 as part of a group conversation. Thus, Bot 116 can participate in the group conversation.

[0019]

[0026] Further, the server computing system 102 optionally also includes a search system 120. More specifically, the server computing system 102 may include several data sources 122-124, and the search system 120 can search the data sources 122-124 based on messages included in the conversation and / or queries generated by the bot 116. In that case, the bot 116 generates a query based on at least one message in the group conversation. In one example, the data sources 122-124 can be, among other numerous information, data sources that store past information such as search engine indexes, instant answer indexes, image indexes, knowledge graphs, previous group conversations conducted among members of the group, or data sources that include the calendar information of the participants in the group conversation, or may include them. Further, although not shown, the search system 120 can access external data sources such as a web server hosting a website, a document repository, etc.

[0020]

[0027] Here, an exemplary operation of the server computing system 102 will be described. Users of the client computing devices 104-106 participate in a group conversation through the messaging application 114. For example, the messaging application 114 receives a first message from the first client computing device 114 and sends the first message to the computing device operated by the user participating in the group conversation. Thereafter, the messaging application 114 receives an nth message from the nth client computing device 106 and sends the nth message to the computing device operated by the user participating in the group conversation.

[0021]

[0028] Following the example, messaging application 114 receives a notification that it should invoke bot 116. Messaging application 114 provides bot 116 with a first message and an nth message, as well as information related to the requested invoke of bot 116. Bot 116 generates output based on the information provided to bot 116 by messaging application 114, and bot 116 provides the generated output to messaging application 114. Messaging application 114 sends the output to client computing devices 104-106. For example, the output may be sent to client computing devices 104-106 so that it appears that bot 116 is a member of a group and is participating in a group conversation with users of client computing devices 104-106. In addition, bot 116 can generate output based on data obtained by search system 120, which obtains data by searching for data in one or more data sources 122-124 and / or data from external data sources.

[0022]

[0029] Referring to Figure 2, a functional block diagram of the first client computing device 104 is presented. The first client computing device 104 includes a processor 202 and memory 204, the memory 204 containing a client messaging application 206. When executed by the processor 202, the client messaging application 206 establishes communication with the messaging application 114 of the server computing system 102, enabling data to be transmitted between the client messaging application 206 and the messaging application 114. For example, the client messaging application 206 can receive a message from a user of the first client computing device 104 to be included in a group conversation and send such a message to the messaging application 114. Upon receiving the message, the messaging application 114 can send the message to computing devices operated by other users participating in the group conversation. Similarly, when the messaging application 114 receives a message in a group conversation from the nth client computing device 106, the messaging application 114 can send the message to computing devices operated by other users participating in the group conversation (including the first client computing device 104).

[0023]

[0030] The first client computing device 104 also includes a display 208, which can show a group conversation 210 being conducted by client messaging applications 206 and 114. The group conversation 210 contains many messages written by several users participating in the group conversation 210. In some examples, each message is called a "turn". In the example of the group conversation 210 shown in Figure 2, first, the first user writes the first message in the group conversation 210, followed by the second user writing the second message, and then the third user writing the third message. Next, the first user posts the fourth message in the group conversation 210, and then the third user writes the fifth message in the group conversation 210. The group conversation 210 also includes a sixth message written by the second user, which invokes bot 116 and includes a request for output from bot 116. For example, by using the identifier of bot 116 following the "@" symbol, a second user can indicate to the messaging application 114 that they are calling bot 116, and a request for output can follow the identifier of bot 116.

[0024]

[0031] The generative model 118 of bot 116 generates output based on the request contained in the sixth message, and further based on previous messages in the group conversation 210. As shown in the group conversation 210, bot 116 generates output and provides it to the messaging application 114, which in turn sends the output to the first client computing device 104. It should also be noted that other participants in the group conversation can invoke bot 116. For example, later in the conversation, the first user indicates to the messaging application 114 that they want to invoke bot 116, and the generative model 118 of bot 116 generates output based on several messages in the group conversation 210.

[0025]

[0032] Referring to Figure 3, a functional block diagram of the messaging application 114 is shown. The messaging application 114 includes a call detection module 302, a prompt generation module 304, and conversation information 306. As previously shown, users in a group conversation can invoke bot 116 (for example, request bot 116 to join the group conversation). The call detection module 302 can monitor the group conversation and detect requests from users participating in the group conversation to invoke bot 116. For example, as confirmed above, a user can invoke bot 116 using the "@" symbol. In another example, a user can issue a voice command to request bot 116 to be invoked.

[0026]

[0033] When the call detection module 302 detects a request to call bot 116, it can notify the prompt generation module 304 that it should generate a prompt for bot 116. The prompt is an input provided to the generation model 118, which in turn instructs the generation model 118 regarding the output it should generate. In some examples, the prompt includes text used to instruct the generation model 118 regarding the output it should generate. The prompt generation module 304 generates the prompt 308 based on conversation information 306. The conversation information 306 may include, among other things, the time the group conversation started, the identity of the users in the group conversation, an indication that the conversation is a group conversation, the time the message was received by the messaging application 114, and the content of the message in the group conversation.

[0027]

[0034] Referring to Figure 4, an exemplary prompt 400 output by the prompt generation module 304 is shown. The prompts previously provided to the generative model assume that bot 116 is communicating directly with a single user and that there are no other participants in the conversation. Furthermore, such prompts include examples that generative model 118 would adopt when responding to a query from a single user. In contrast, prompt generation module 304 adds new instructions to the typical prompt (below where the context of generative model 118 conversing one-on-one with a user is traditionally provided) to help generative model 118 understand the context of a group conversation. The prompt 400 generated by prompt generation module 304 includes a statement that bot 116 is in a group conversation. In addition, the prompt 400 generated by prompt generation module 304 includes the identity of the users participating in the group conversation. This information is important because without knowing the identity of the users in the group conversation, generative model 118 might become confused about who the participants are and which participant is providing which message when reading the messages in the group conversation. To obtain a list of participant identities, the prompt generation module 304 can retrieve identities from client applications running on client computing devices 104-106, or by making backend calls to the roster application programming interface (API) maintained by the messaging application 114.

[0028]

[0035] Next, the prompt generation module 304 includes a transcript of the messages in the group conversation in the prompt, the transcript including the message content, a timestamp indicating when the message was retrieved by the messaging application 114, and the identity of the user who submitted the message. The prompt generation module 304 also identifies the user who requested the call to bot 116 in the prompt 400 and optionally includes the user's location in the prompt 400. The prompt generation module 304 may also include instructions to be given to bot 116 in the prompt 400 to limit the output of bot 116. For example, the instructions may indicate that the user requesting the call to bot 116 is interested in an answer from the provided context of the group conversation (e.g., from the group conversation transcript). Furthermore, the prompt 400 may include requests to bot 116, such as not to perform extrapolation, not to provide personal information about the people participating in the group conversation, and not to make inferences about the users participating in the group conversation. Since bot 116 is limited to responding using information within the context, these instructions can facilitate the reduction of hallucinatory output by bot 116.

[0029]

[0036] For example, if bot 116 is not restricted by the context within prompt 400, the user might write a call request to bot 116 that includes a phrase such as, "Do you remember when John traveled to Australia?" Without the restricting instructions mentioned above, since the group conversation does not contain any information that John traveled to Australia, bot 116 might produce output that completely fabricates the fact that John traveled to Australia.

[0030]

[0037] Optionally, the prompt generation model 304 may include in the prompt 400 other information about the user in the group conversation, such as topics of interest to the user, keywords and facts mentioned by the user in other conversations, and information retrieved from the user's profile, including the user's demographic information (such as age), so that the bot 116 can tailor its response to the user.

[0031]

[0038] Returning to Figure 3, the prompt 308 generated by the prompt generation module 304 is provided to the bot 116, which then provides the prompt 308 to the generation model 118. The generation model 118 generates an output 310 based on the prompt, and the bot 116 provides the output 310 to the messaging application 114. The messaging application 114 includes the output 310 in the group conversation so that it appears as though the bot 116 is participating in a group conversation with other users.

[0032]

[0039] Bot 116 can interact with the search system 120, with the optional creation of an updated prompt used by bot 116 to generate output. In a non-limiting example, a user in a group conversation might request a call to bot 116, with the input: "We'd like to go see a movie this afternoon. What time works for everyone? Are there any movies playing at that time?" The prompt generation module 304 can generate a prompt for bot 116 based on this call request. In this case, the prompt 308 generated by the prompt generation model 304 does not include any restrictions limiting the response to the context contained in the prompt 308. The generation model 118 receives the prompt 308, generates a query based on such a prompt 308, and provides the query to the search system 120. The search system 120 searches one or more data sources 122-124 (Figure 1) based on the query and provides at least a portion of the search results as part of the prompt that the generation model 118 should adopt to respond to the user request. For example, the first data source 122 may include the time and location of a movie, and the m-th data source 124 may include calendar information of participants in the group conversation. Based on such prompts, the generative model 118 generates an output 310 and provides it to the messaging application 114, which then includes the output 310 in the group conversation taking place among the users of client computing devices 104-106.

[0033]

[0040] Prompt 400 is described as an exemplary prompt associated with a conventional generative model, and the prompt has been modified so that the generative model 118 can be incorporated into the messaging application 114. In another example, the prompt generation module 304 can generate a prompt that appropriately explains to bot 116 from the outset that bot 116 is in a multi-user group conversation.

[0034]

[0041] Referring briefly to Figure 5, another exemplary prompt 500 that may be generated by the prompt generation module 304 is shown. Prompt 500 is similar to prompt 400, except that the conversation information 306 contained in prompt 500 includes output from bot 116. It has been observed that when bot 116 generates output based on its own previous output, the subsequently generated output can be unpredictable, unstable, and generally undesirable. To address this problem, the prompt generation module 304 may, at its discretion, redact one or more outputs of bot 116 contained in the conversation information 306. Various techniques can be used to determine how many outputs of bot 116 to include in prompt 500 and / or how many outputs of bot 116 to redact. For example, a sliding window can be employed, in which case only the most recent threshold number of outputs of bot 116 are included in prompt 500 (other outputs are redacted). In another example, a rolling window of bot 116's outputs can be maintained, with other outputs being redacted. An example of this rolling window is shown below.

[0035]

[0042] In one example, the threshold number for bot 116's output is 5. This means that when one of the users in the group calls bot 116 for the 6th time, bot 116 will be provided with the prompt with the previous 5 replies. When the user calls the bot for the 7th time, the prompt generation module 304 reduces all of the bot's output in the prompt. This can be expressed algorithmically as follows: Therefore, resetConversationInTurn=5, Next, botTurnsToKeep = previousTurnCount % (resetConversationInTurn + 1), ○If this is the 6th call to the bot, then previousTurnCount=6-1=5, botTurnsToKeep=5%(5+1)=5 → The prompt will contain the 5 most recent bot replies, and any older ones will be replaced with [redacted]. ○If this is the 7th call to the bot, then previousTurnCount=7-1=6, botTurnsToKeep=6%(5+1)=0 → the bot's output is not included in the prompt, and all bot outputs are replaced with [redacted]. ○If this is the 8th call to the bot, then previousTurnCount=8-1=7 and botTurnsToKeep=1, so the transcript will be as follows: ■ Statement by Person A: XXXXXXX ■ Statement by Person B: YYYYY ■ Bot's statement: [Redact] ■ Human A's statement: @bot Question ■ Bot's message: Reply ■ Human B's comment: @bot New question This can be represented in a table as follows:

[0036] [Table 1]

[0037]

[0043] Again, Figure 5 shows that prompt 500 includes a reduction of the output of bot 116.

[0038]

[0044] Bot 116 is restricted to receiving prompts of a specific size (e.g., a threshold number of tokens). Accordingly, the prompt generation module 304 may perform one or more actions to ensure that the number of tokens in the prompt 308 provided to Bot 116 is below a threshold so that Bot 116 can consume the prompt 308. As previously mentioned, the prompt generation module 304 may include various types of information in the prompt, and / or the search system 120 may include information in the prompt 308, such information may include general instructions to Bot 116 on how Bot 116 should produce output, including hypothetical examples of questions and answers between Bot 116 and the user in a group conversation. The prompt 308 may also include JSON documents and web snippets of search results for searches initiated by Bot 116 and retrieved by the search system 120. The prompt 308 may also include at least some of the conversation information 306, including a transcript of the group conversation prior to receiving the call request. Finally, the prompt 308 may include requests to the Bot associated with the call.

[0039]

[0045] All of this information consumes space in prompt 308. To keep prompt 308 within an acceptable size (for example, so that the number of tokens is below a threshold), prompt generation module 304 can truncate the transcript of a group conversation if the conversation becomes too long. Prompt generation module 304 can employ a variety of different strategies when truncating the transcript of a conversation. For example, prompt generation module 304 can obtain the entire transcript of a conversation sorted chronologically from newest to oldest. Prompt generation module 304 can then proceed line by line from newest to oldest, filling a buffer with a certain number of tokens (note that the term "token" is not necessarily equivalent to a single word or character, but rather refers to an entity that bot 116 semantically understands, where a token in one language might represent 4 characters, and a token in another (semantically dense) language might represent 1 character).

[0040]

[0046] For example, the prompt generation module 304 does not truncate the content of any individual messages written by the user. When the prompt generation module 304 determines that adding the next message to the transcript would cause the token count in the buffer to exceed a certain number, that message is not added to the context. The prompt generation module 304 then reverses the order of the messages in the buffer so that they are arranged from oldest to newest, and adds the buffer to the context.

[0041]

[0047] In another embodiment, if the message from bot 116 is relatively long, the prompt generation module 304 can request bot 116 to truncate (summarize) its own output, thereby reducing the number of tokens needed to represent bot 116's output. In another example, when the number of tokens representing the transcript of a conversation exceeds a certain number, such a transcript of a conversation may be provided to bot 116 as input, along with a request to truncate (summarize) the transcript of the conversation (for example, thereby reducing the transcript from X characters to Y characters, where Y is less than X). In yet another example, the prompt generation model 304 can use multiple different models to truncate portions of the transcript of a conversation. For example, if the transcript of a conversation is relatively long, the prompt generation module 304 can select the next line, call another model to summarize the argument, and then repeat the same process in the next batch until all lines are covered.

[0042]

[0048] Furthermore, to reduce the use of computing resources, the messaging application 114 can be configured to identify certain bot calls as requiring less computation and other calls as requiring more computation. For example, the messaging of application 114 communicates with several different bots, each using different amounts of computing resources to generate output. For instance, one bot might be trained to perform only summarization and therefore require fewer computing resources to generate output. Conversely, a second bot might require significantly more computing resources than the first bot because it can retrieve information generated by the search system 120 and generate images based on that information. The messaging application 114 can conserve computing resources by identifying which bot should answer a call request and providing prompts to the appropriate bot.

[0043]

[0049] Furthermore, as mentioned above, the generative model 118 may utilize a considerable amount of computing resources when generating certain types of output, such as images, music, or videos. In some cases, users participating in a group conversation may have accounts associated with the messaging application 114, and these accounts may contain units of value (such as reward points). When a user who invoked bot 116 makes a request that requires bot 116 to utilize a considerable amount of computing resources to generate output, the user's account may be charged for that output. For example, a certain number of reward points may be deducted from the account of the user who invoked bot 116. In another example, participants in a group conversation are randomly selected, and units of value are deducted from the accounts of the randomly selected users. In yet another example, a round-robin method is employed to deduct units of value from a user's account when bot 116 generates output based on a user's request.

[0044]

[0050] As described above, the technology described herein relates to messaging applications that enable group conversations and enables use cases that were previously impossible. For example, bot 116 can answer not only general questions but also questions relevant to the context of a group conversation. In some implementations, bot 116 generates output only when invoked by a user participating in the group conversation. In another example, bot 116 is provided with input for each turn of the group conversation, and bot 116 determines the appropriate timing for itself to intervene in the group conversation by generating output. In yet another example, messaging application 114 is associated with a UX canvas displayed on the displays of client computing devices 104-106, the UX canvas being continuously updated by the output of bot 116 (the output being suggestions to include in the group conversation). When a user taps a suggestion, that suggestion is entered into the group conversation as a turn (and identified as having been generated by bot 116).

[0045]

[0051] Bot116 can assist users in group conversations through a variety of tasks. Examples of tasks Bot116 can assist with include, but are not limited to, 1) summarizing the current conversation, 2) answering questions about what someone has said about a specific topic in the conversation, 3) suggesting follow-up action items based on information in the group conversation, 4) helping with group event planning such as vacations, 5) generating images, avatars, memes, etc. related to messages in the group conversation, 6) helping rewrite and check texts the group is working on (making text more professional, making text more humorous, correcting typos, translating text into different languages, etc.), and 7) news reporting. This includes answering questions about the content of web pages pointed to by URLs pasted into the group conversation, such as summarizing events, identifying facts mentioned in news articles, reading structured data like corporate financial statements, and answering specific questions about the content of web pages; 8) brainstorming ideas, such as creating roadmaps for product development; 9) performing text translation from one language to another so that language differences among participants in the group do not become an issue, as Bot 116 can translate and communicate information between participants in the language requested by the participants; and 10) starting and running text and image-based games to entertain users in the group conversation.

[0046]

[0052] Furthermore, Bot 116 can generate text in its output, such as answers to questions that follow the flow of a conversation in plain text, and it can understand pasted structured text, such as tables. In another example, Bot 116 can receive audio as input, allowing users to provide audio notes in group conversations, and Bot 116 can understand the user's speech using speech-to-text technology. Bot 116 can respond to input text in text or using text-to-speech conversion. By receiving audio, Bot 116 can listen to and understand audio / video calls, and can take URLs pasted into group conversations as input so the bot can follow them, download HTML, or answer questions about the page content. In another example, Bot 116 can access links to documents in a shared storage space. Regarding output, Bot 116 can generate output in any of the modes mentioned above (text, audio, images, etc.). Bot 116 can generate output with generated images, fill in meme templates, or generate graphs. Bot 116 can generate output in a tone selected by the user within a group chat, and Bot 116's output can be shared with other applications.

[0047]

[0053] Figures 6 and 7 illustrate methodologies for incorporating a generative model into a computer-implemented messaging application according to one or more embodiments described herein. While these methodologies are presented and described as a series of actions performed in sequence, it should be understood and acknowledged that these methodologies are not limited by the order of that sequence. For example, some actions may occur in a different order than those described herein. Furthermore, some actions may occur simultaneously with others. Moreover, in some examples, not all actions are necessary to implement the methodologies described herein.

[0048]

[0054] Furthermore, the actions described herein may be computer-executable instructions that can be performed by one or more processors and / or stored in one or more computer-readable media. Computer-executable instructions may include routines, subroutines, programs, execution threads and / or similar. Furthermore, the results of the actions of the methodology may be stored in computer-readable media, displayed on a display device, and / or similar.

[0049]

[0055] Referring only to Figure 6, a flowchart is shown illustrating methodology 600, which facilitates the integration of a bot into a computer-implemented messaging application that supports group conversations. Methodology 600 begins at 602, and at 604, a first message is received from a first client computing device operated by a first user participating in a group conversation that includes several other users in the messaging application that supports group conversations. At 606, the first message is sent to a client computing device operated by a user participating in the group conversation.

[0050]

[0056] In step 608, the messaging application receives a second message from a second client computing device operated by a second user participating in the group conversation. In step 610, the second message is sent to the client computing device operated by the user participating in the group conversation. Therefore, it can be confirmed that the group conversation contains messages generated by multiple users.

[0051]

[0057] In 612, a third message is generated by a bot that includes a generative model, and the third message is generated based on the first and second messages in the group conversation (i.e., previous messages in the group conversation). In one example, as described above, the bot generates the third message in response to receiving a call request from a user participating in the group conversation. In 614, the third message is sent to a computing device operated by the user participating in the group conversation. Methodology 600 is completed in 616.

[0052]

[0058] Moving on to Figure 7, a flowchart illustrating methodology 700 for including bot output in a group conversation is shown. Methodology 700 begins at 702, and at 704, a computer-implemented messaging application receives an indication that the bot has been invoked by a user participating in the group conversation. At 706, in response to receiving the indication, a prompt is constructed for the bot, which includes two messages previously received in the group conversation from two different users. These two previously received messages can be represented by tokens in the prompt. At 708, the bot receives the prompt and generates output based on the prompt. At 710, the output generated by the bot is sent to the client computing devices of the users participating in the group conversation. Methodology 700 is completed at 710.

[0053]

[0059] Referring here to Figure 8, a high-level diagram of an exemplary computing device 800 that may be used in accordance with the systems and methodologies disclosed herein is shown. For example, computing device 800 may be a client computing device that runs a client messaging application. As another example, computing device 800 may be a server computing system that runs a server-side messaging application and / or a generative model. Computing device 800 includes at least one processor 802 that executes instructions stored in memory 804. Instructions may be, for example, instructions for performing the functionality described above as being performed by one or more components discussed above, or instructions for performing one or more of the methods described above. Processor 802 can access memory 804 by system bus 806. In addition to storing executable instructions, memory 804 may also store content, graphical icons, profile information, and the like.

[0054]

[0060] The computing device 800 also includes a data store 808 accessible to the processor 802 via the system bus 806. The data store 808 may contain executable instructions, graphical icons, profile information, content, etc. The computing device 800 also includes an input interface 810 for enabling external devices to communicate with the computing device 800. For example, the input interface 88 can be used to receive instructions from external computer devices, users, etc. The computing device 800 also includes an output interface 812 for interfacing the computing device 800 with one or more external devices. For example, the computing device 800 can display text, images, etc., via the output interface 812.

[0055]

[0061] External devices communicating with the computing device 800 via the input interface 810 and output interface 812 are intended to be included in an environment that provides substantially any type of user interface with which the user can interact. Examples of user interface types include graphical user interfaces and natural user interfaces. For example, a graphical user interface can receive input from a user employing an input device such as a keyboard, mouse, remote control, or similar, and provide output to an output device such as a display. Furthermore, a natural user interface allows the user to interact with the computing device 800 in a way that is free from the constraints imposed by input devices such as keyboards, mice, remote controls, and similar. Rather, a natural user interface may rely on speech recognition, touch and stylus recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, machine intelligence, etc.

[0056]

[0062] In addition, although computing device 800 is presented as a single system, it should be understood that it may be a distributed system. For example, several devices may communicate via network connectivity and collectively perform tasks described as being performed by computing device 800.

[0057]

[0063] The various functions described herein can be implemented in hardware, software, or any combination thereof. When implemented in software, the functions can be stored in or transmitted through a computer-readable medium as one or more instructions or codes. Computer-readable medium includes computer-readable storage media. Computer-readable storage media can be any available storage medium accessible to a computer. Such computer-readable storage media, not as an limitation but as an example, may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other computer-accessible medium that can be used to carry or store desired program code in the form of instructions or data structures. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs (BDs), where disk typically reproduces data magnetically, and disc typically reproduces data optically using a laser. Furthermore, propagated signals are not included in the scope of computer-readable storage media. Furthermore, computer-readable media also include communication media, which include any medium that facilitates the transfer of computer programs from one location to another. For example, a connection can also be a communication medium. For instance, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, and microwaves, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio waves, and microwaves are included in the definition of communication media. Combinations of the above may also be included within the scope of computer-readable media.

[0058]

[0064] Alternatively, or in addition to the above, anything functionally described herein can be performed at least partially by one or more hardware logic components. For example, but not limited to, exemplary types of usable hardware logic components include field-programmable gate arrays (FPGAs), integrated circuits for specific programs (ASICs), standard products for specific programs (ASSPs), system-on-chip systems (SOCs), and complex-programmable logic devices (CPLDs).

[0059]

[0065] This specification discloses techniques for using generative models in a multi-user messaging context, at least according to the following examples.

[0060]

[0066] (A1) In one embodiment, the method disclosed herein includes receiving a plurality of messages from a plurality of client computing devices operated by a plurality of users in a messaging application that supports a group conversation, wherein the plurality of messages are included in the group conversation. The method also includes providing a prompt to a generative model, wherein the prompt includes a plurality of messages. In addition, the method includes receiving output from the generative model, wherein the output is generated by the generative model based on the prompt. The method further includes including the output as a turn in the group conversation.

[0061]

[0067] (A2) In some embodiments of the method of (A1), the multiple users include a first user. The method also includes receiving a message from the first user, wherein the message includes a request to invoke a chatbot in a group conversation and a request to obtain information from the chatbot. In addition, the method includes detecting the request to invoke a chatbot in the message. The method further includes constructing a prompt in response to detecting the request to invoke a chatbot, wherein the request to obtain information is included in the prompt.

[0062]

[0068] (A3) In some embodiments of at least one of the methods of (A1) to (A2), the messaging application is an instant messaging application.

[0063]

[0069] (A4) In some embodiments of at least one of the methods of (A1) to (A3), the method also includes constructing a transcript of messages in a group conversation. The transcript includes: 1) a first message received from a first client computing device of a plurality of client computing devices; 2) a first identifier of a first user of the first client computing device, which is the first identifier assigned to the first message; 3) a second message received from a second client computing device of a plurality of client computing devices; and 4) a second identifier of a second user of the second client computing device, which is the second identifier assigned to the second message. In addition, the method also includes constructing prompts based on the transcript of messages in the group conversation.

[0064]

[0070] (A5) In some embodiments of the method of (A4), the transcript includes 5) a third message previously generated by the generative model, and 6) a third identifier of the generative model, including the third identifier assigned to the third message, and the prompts provided to the generative model include a first message and a first identifier assigned to the first message, a second message and a second identifier assigned to the second message, a third message and a third identifier assigned to the third message.

[0065]

[0071] (A6) In some embodiments of the method of (A4), the transcript of messages in a group conversation includes 1) a third message previously generated by the generative model, and 2) a third identifier of the generative model, which is assigned to the third message. The method also includes reducing the third message generated by the generative model from the transcript in order to create an updated transcript, wherein the third message is reduced based on the fact that a third identifier is assigned to the third message, and further, the prompt includes the updated transcript.

[0066]

[0072] (A7) In some embodiments of at least one of the methods of (A1) to (A6), the method includes receiving a command from a first client computing device to invoke a generative model, wherein the command to invoke the generative model includes a request for output from the generative model, and the command is received before a prompt is provided to the generative model. In addition, the method includes constructing a transcript of a group conversation, wherein the transcript includes several messages previously generated by the generative model in the group conversation. The method further includes comparing the number of messages in several messages with a predefined threshold. The method also includes determining that the number of messages in several messages is equal to a predefined threshold. In addition, the method includes reducing several messages previously generated by the generative model in the transcript to create an updated transcript, wherein the prompt includes the updated transcript, and several messages are determined to be equal to a predefined threshold.

[0067]

[0073] (A8) In some embodiments of at least one of the methods of (A1) to (A7), the prompt includes an indication that the generative model is participating in a group conversation.

[0068]

[0074] (A9) In some embodiments of at least one of the methods of (A1) to (A8), the method includes obtaining identifiers of multiple users from the user profiles of multiple users, wherein the identifiers are obtained before a prompt is provided to the generative model.

[0069]

[0075] (A10) In at least one of its embodiments of (A1) to (A9), the method includes receiving a command from a first client computing device to invoke a generative model, wherein the command to invoke the generative model includes a request for output from the generative model, and further includes receiving the command before a prompt is provided to the generative model. The method further includes constructing a prompt based on the request for output, the generative model generating a query based on the prompt, providing the query to a search engine, the generative model receiving at least a portion of the search results identified by the search engine based on the query, and further, the generative model generating output based on at least a portion of the search results identified by the search engine.

[0070]

[0076] (B1) In another embodiment, a method performed by a computing system running a messaging application includes receiving a first message in a group conversation from a first client computing device communicating with the computing system by the messaging application, the first client computing device being operated by a first user. The method also includes receiving a second message in a group conversation from a second client computing device communicating with the computing system by the messaging application, the second client computing device being operated by a second user. In addition, the method includes constructing a prompt based on the first and second messages. The method further includes providing the prompt to a generative model, the third message being generated and provided by the generative model based on the prompt. The method also includes sending a third message to the first and second client computing devices for display as part of a group conversation, the third message being identified in the group conversation as being generated by a generative model.

[0071]

[0077] (B2) In some embodiments of the method of (B1), the method includes receiving a command from a first client computing device to invoke a generative model, wherein the command to invoke the generative model includes a request for a fourth message from the generative model, and the command is received after the third message has been sent to the first and second client computing devices. In addition, the method includes constructing a second prompt based on the command, wherein the second prompt includes the third message. The method further includes providing the second prompt to the generative model, which provides the generative model to generate a fourth message based on the second prompt. In addition, the method includes sending the fourth message to the first and second client computing devices for presentation as part of a group conversation, wherein the fourth message is identified in the group conversation as having been generated by the generative model.

[0072]

[0078] (B3) In some embodiments of at least one method of (B1) to (B2), the prompt includes an identifier for a first user assigned to a first message and an identifier for a second user assigned to a second message.

[0073]

[0079] (B4) In some embodiments of at least one method of (B1) to (B3), constructing a prompt includes 1) constructing a transcript of a group conversation, and 2) truncating the transcript of the group conversation so that the number of tokens in the truncated transcript is less than a predefined threshold, and the prompt includes the truncated transcript.

[0074]

[0080] (B5) In some embodiments of at least one of the methods of (B1) to (B4), the method includes receiving a command from a first client computing device to invoke a generative model, wherein the command to invoke the generative model includes a request for a fourth message from the generative model, and further includes receiving the command after the third message has been sent to the first and second client computing devices. In addition, the method includes constructing a second prompt based on the command to invoke the generative model, wherein the second prompt includes the first and second messages but does not include the third message. The method further includes providing the second prompt to the generative model, which then provides the generative model to generate a fourth message based on the second prompt. The method also includes sending the fourth message to the first and second client computing devices for presentation as part of a group conversation, wherein the fourth message is identified in the group conversation as having been generated by the generative model.

[0075]

[0081] (B6) In some embodiments of at least one of the methods of (B1) to (B5), the method also includes obtaining a first identifier for the first user and a second identifier for the second user from a first user profile and a second user profile, respectively, constructing a prompt includes including the first user identifier and the second user identifier in the prompt, and the generative model generates a third message based on at least one of the first user identifier and the second user identifier.

[0076]

[0082] (B7) In some embodiments of at least one method of (B1) to (B6), the generation model generates a summary that summarizes the first message and the second message based on the first message and the second message, and further constructs a prompt which includes including the summary in the prompt.

[0077]

[0083] (B8) In some embodiments of at least one of the methods of (A1) to (A7), the method also includes obtaining a topic identified in the user profile of a first user, and constructing a prompt includes including the topic in the prompt.

[0078]

[0084] (C1) In another embodiment, the computing system includes a processor and memory, the memory storing instructions, which, when executed by the processor, cause the processor to execute at least one of the methods disclosed herein (e.g., either method (A1) to (10) or method (B1) to (B8)).

[0079]

[0085] (D1) In yet another embodiment, a computer-readable storage medium includes instructions, which, when executed by a processor, cause the processor to execute at least one of the methods disclosed herein (g, any of the methods (A1) to (10) or (B1) to (B8)).

[0080]

[0086] The foregoing includes examples of one or more embodiments. Naturally, it is impossible to describe all possible changes and modifications of the above-described devices or methodologies in order to illustrate the aforementioned embodiments, but those skilled in the art will recognize that many further changes and substitutions of various embodiments are possible. Accordingly, the embodiments described are intended to incorporate all such modifications, changes, and variations that fall within the spirit and scope of the appended claims. Furthermore, where the term “includes” is used in the detailed description or claims, such term is intended to have a comprehensive meaning, as is the term “comprising,” and the term “comprising,” where adopted, is interpreted as a transitional term in the claims.

Claims

1. Processor and Memory for storing instructions A computing system including, where the instruction is executed by the processor, In a messaging application that supports group conversations, the receipt of multiple messages from multiple client computing devices operated by multiple users, wherein the multiple messages are included in the group conversation. Providing a prompt to the generation model, wherein the prompt includes the plurality of messages, Receiving the output generated by the generation model, wherein the output is generated by the generation model based on the prompt, To include the output as a turn in the group conversation. A computing system that causes the processor to perform an action including the following.

2. The aforementioned multiple users include the first user, and the aforementioned action is Receiving a message from the first user, wherein the message includes a request to invoke the chatbot in the group conversation and a request to obtain information from the chatbot. To detect the request within the message that calls the chatbot, In response to detecting the request to invoke the chatbot, construct the prompt such that the request for information is included in the prompt. The computing system according to claim 1, further comprising:

3. The computing system according to claim 1, wherein the messaging application is an instant messaging application.

4. The aforementioned act, To construct a transcript of the messages in the group conversation, wherein the transcript is A first message received from the first client computing device of the plurality of client computing devices, A first identifier of the first user of the first client computing device, which is the first identifier assigned to the first message, A second message received from a second client computing device of the plurality of client computing devices, and The second identifier of the second user of the second client computing device, which is the second identifier assigned to the second message. This includes building, Constructing the prompt based on the transcript of the message in the group conversation. The computing system according to claim 1, further comprising:

5. The transcript of the message in the group conversation is The third message previously generated by the aforementioned generation model, and The computing system according to claim 4, wherein a third identifier of the generative model further includes a third identifier assigned to the third message, and the prompt provided to the generative model includes a first message, a first identifier assigned to the first message, a second message, a second identifier assigned to the second message, a third message, and a third identifier assigned to the third message.

6. The transcript of the message in the group conversation is The third message previously generated by the aforementioned generation model, and A third identifier of the generation model, further comprising a third identifier assigned to the third message, wherein the action is The computing system according to claim 4, further comprising reducing the third message generated by the generative model from the transcript in order to create an updated transcript, wherein the third message is reduced on the basis that the third identifier is assigned to the third message, and further the prompt includes the updated transcript.

7. The aforementioned act, Receiving a command from the first client computing device to invoke the generative model before the prompt is provided to the generative model, wherein the command to invoke the generative model includes a request for the output from the generative model. Constructing a transcript of the group conversation, wherein the transcript includes several messages previously generated by the generative model within the group conversation. The number of messages in some of the aforementioned messages is compared with a predefined threshold, The determination that the number of messages within some of the aforementioned messages is equal to the predefined threshold, The computing system according to claim 1, further comprising, when it is determined that the number of messages in the several messages is equal to the predefined threshold, redacting the several messages previously generated by the generative model in the transcript in order to create an updated transcript, wherein the prompt includes the updated transcript.

8. The computing system according to claim 1, wherein the prompt includes an indication that the generative model is participating in the group conversation.

9. The aforementioned act, Before the prompt is provided to the generation model, the identifiers of the multiple users are obtained from the user profiles of the multiple users, Constructing the prompt to include the identifiers of the multiple users The computing system according to claim 1, further comprising:

10. The aforementioned act, Receiving a command from the first client computing device to invoke the generative model before the prompt is provided to the generative model, wherein the command to invoke the generative model includes a request for the output from the generative model. The computing system according to claim 1, further comprising constructing the prompt based on the request for the output, wherein the generative model generates a query based on the prompt, provides the query to a search engine, the generative model receives at least a portion of the search results identified by the search engine based on the query, and the generative model further generates the output based on at least a portion of the search results identified by the search engine.

11. A method performed by a computing system running a messaging application, Receiving a first message in a group conversation from a first client computing device communicating with the computing system via the messaging application, wherein the first client computing device is operated by a first user to receive, Receiving a second message in the group conversation from a second client computing device communicating with the computing system via the messaging application, wherein the second client computing device is operated by a second user to receive, Based on the first message and the second message, construct a prompt, The means of providing the prompt to the generative model, wherein the third message is generated by the generative model based on the prompt. Sending the third message to the first client computing device and the second client computing device for display as part of the group conversation, wherein the third message is identified in the group conversation as having been generated by the generative model. Methods that include...

12. Receiving a command from the first client computing device to invoke the generative model after the third message has been transmitted to the first client computing device and the second client computing device, wherein the command to invoke the generative model includes a request for a fourth message from the generative model. Constructing a second prompt based on the aforementioned command, wherein the second prompt includes the third message, Providing the second prompt to the generation model, wherein the generation model generates and provides the fourth message based on the second prompt, Sending the fourth message to the first client computing device and the second client computing device for presentation as part of the group conversation, wherein the fourth message is identified in the group conversation as having been generated by the generative model. The method according to claim 11, further comprising:

13. The method according to claim 11, wherein the prompt includes the identifier of the first user assigned to the first message and the identifier of the second user assigned to the second message.

14. Constructing the aforementioned prompt To construct a transcript of the aforementioned group conversation, The method according to claim 11, comprising truncating the transcript of the group conversation so that the number of tokens in the truncated transcript is less than a predefined threshold, wherein the prompt includes the truncated transcript.

15. Receiving a command from the first client computing device to invoke the generative model after the third message has been transmitted to the first client computing device and the second client computing device, wherein the command to invoke the generative model includes a request for a fourth message from the generative model. Constructing a second prompt based on the command for calling the generation model, wherein the second prompt includes the first message and the second message, but does not include the third message. Providing the second prompt to the generation model, wherein the generation model generates and provides the fourth message based on the second prompt, Sending the fourth message to the first client computing device and the second client computing device for presentation as part of the group conversation, wherein the fourth message is identified in the group conversation as having been generated by the generative model. The method according to claim 11, further comprising:

Images