Collaborative search session with automated assistant

Abstract

This disclosure relates to collaborative search sessions with automated assistants. Techniques for collaborative search sessions with automated assistants are described herein. One method includes receiving a first query in a query session from a first user of a first client device, providing a first set of search results to the first user, determining that the first query is relevant to a second user of the first client device based on at least one term in the first query, providing a selectable option to the second user to join the query session, adding the second user to the query session in response to receiving an indication from the second user that the selectable option is accepted, receiving additional input from the second user, generating a modified set of search results based on the additional input received from the second user, and providing the modified set of search results to the first user and the second user.

Description

Technical Field

[0001] This disclosure relates to collaborative search sessions via automated assistants. Background Technology

[0002] Humans can engage in human-computer dialogue through interactive software applications referred to herein as “automatic assistants” (also known as “digital assistants,” “digital agents,” “interactive personal assistants,” “intelligent personal assistants,” “assistant applications,” “conversational agents,” etc.). For example, a human (who may be referred to as a “user” when interacting with an automatic assistant) can provide commands and / or requests to the automatic assistant using spoken natural language input (i.e., speech). In some cases, spoken natural language input can be converted into text by providing text (e.g., typed) and / or by touch and / or nonverbal physical movements (e.g., gestures, eye gaze, facial movements, etc.) for processing. Automatic assistants respond to requests by providing responsive user interface outputs (e.g., auditory and / or visual user interface outputs), controlling one or more intelligent devices, and / or controlling one or more functions of the device implementing the automatic assistant (e.g., controlling other applications of the device).

[0003] An automated assistant can be a software application running on a client device. The client device can be a standalone interactive speaker, a standalone interactive display device (which may also include speakers and / or a camera), a smart appliance such as a smart TV (or a standard TV with automated assistant capabilities equipped with a network radar detector), a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in the user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), and / or a wearable device for the user that includes a computing device (e.g., a user's watch with a computing device, glasses for the user with a computing device, a virtual or augmented reality computing device).

[0004] In some cases, an automated assistant can be used by multiple users (e.g., multiple members of a particular household or family), especially when the automated assistant operates on a client device that can be shared among users, such as a standalone interactive speaker, a standalone interactive display, or a smart appliance. In other cases, an automated assistant can be used by a single user, particularly when the automated assistant operates on a client device that is not frequently shared, such as a mobile phone computing device.

[0005] Multiple users can perform similar or redundant searches using an automated assistant running on a shared client device. For example, two members of a particular household buying a new television can each use an automated assistant running on a separate interactive display device to perform individual searches for the television. Each of these users can perform multiple searches as they separately refine their searches using newly added constraints. Multiple activations of the automated assistant can waste network and / or computing resources when multiple users separately perform and refine searches on the same topic, such as television. Furthermore, the user experience can be negatively impacted when a second user repeats a search already performed by the first user. Summary of the Invention

[0006] Some embodiments disclosed herein involve collaborative search sessions via an automated assistant. As described in more detail herein, collaborative search sessions (e.g., product search sessions) can be detected and maintained across multiple users. These search sessions can be longer (e.g., lasting hours or days) and can be available on an automated assistant device for two or more participants. The search session can be used as an input method to continuously refine the product search within the search session until a product is selected and the purchase is completed. In some embodiments, these collaborative product search sessions can be used to search for and purchase a single item (e.g., a television) or multiple items (e.g., household goods).

[0007] In various embodiments, a method executed by one or more processors may include: receiving a first query from a first user of the first client device in a query session by a first automated assistant executing on a first client device; providing the first user with a first set of search results for the first query by the first automated assistant; determining that the first query is relevant to a second user of the first client device based on at least one term in the first query; in response to determining that the first query is relevant to the second user, providing the second user of the first client device with an optional option to join the query session by the first automated assistant; adding the second user to the query session in response to receiving an instruction from the second user to accept the optional option to join the query session; receiving additional input from the second user to refine the first query; generating a modified set of search results based on the additional input received from the second user; and providing the modified set of search results to the first user and the second user by the first automated assistant.

[0008] In some implementations, the query session is a shopping session; the first set of search results 0 includes a first set of products; and the modified set of search results includes the modified set of products. In some implementations, the first client device is an automated assistant device; providing the first set of search results includes the first automated assistant displaying the first set of search results on the display of a first user's mobile device; and providing the modified set of search results includes the first automated assistant displaying the modified set of search results on the display of a first user's mobile phone and the display of a second user's mobile device.

[0009] In some implementations, the method may further include providing a selectable option from a first automated assistant to a first user to allow a second user of the first client device to join the query session. Providing the selectable option to the second user to join the query session may further respond to receiving an instruction from the first user to accept the selectable option to allow the second user of the first client device to join the query session.

[0010] In some implementations, the method may further include determining the predicted level of interest of a second user in a query session. Determining that the first query is relevant to the second user may further be based on the second user's predicted level of interest in the query session meeting a threshold. The predicted level of interest of the second user in the query session may be based on the second user's query history.

[0011] In some embodiments, the method may further include determining an environmental context. Determining the relevance of the first query to the second user may further be based on the environmental context. In some embodiments, the method may further include determining the frequency of interaction between the second user and the first client device. Determining the relevance of the first query to the second user may further be based on the frequency of interaction between the second user and the first client device satisfying a threshold. In some embodiments, determining the relevance of the first query to the second user of the first client device may be based on the score of at least one term in the first query satisfying a threshold.

[0012] In some implementations, the method may further include automatically determining filter terms by a first automated assistant based on the inferred preferences of a second user. The generation of a modified set of search results may further be based on the filter terms. In some implementations, a second client device receives additional input.

[0013] In some implementations, generating the modified set of search results may include: a first automated assistant determining a second query based on a first query and additional input; and generating the modified set of search results based on a second set of search results from the second query. In some implementations, generating the modified set of search results may include filtering a first set of search results based on additional input.

[0014] In some additional or alternative embodiments, the computer program product may include one or more computer-readable storage media having program instructions commonly stored on the one or more computer-readable storage media. The program instructions are executable to: receive a first query in a query session from a first user of the first client device by a first automated assistant executing on the first client device; provide the first user with a first set of search results for the first query by the first automated assistant; receive additional input from a second user of the first client device by the first automated assistant to refine the first query; add a second user to the query session in response to receiving additional input from the second user to refine the first query; generate a modified set of search results based on the additional input received from the second user; and provide the modified set of search results to both the first user and the second user by the first automated assistant. The modified set of search results may be provided to the second user based on adding the second user to the query session.

[0015] In some implementations, the query session can be a shopping session; the first set of search results can include a first set of products; and the modified set of search results can include the modified set of products.

[0016] In some implementations, the first client device may be an automated assistant device; providing a first set of search results may include the first automated assistant providing the first set of search results on the display of a first user's mobile device; and providing a modified set of search results may include the first automated assistant providing the modified set of search results on the display of a first user's mobile phone and the display of a second user's mobile device.

[0017] In some implementations, the program instructions may be further executable for automatically determining filter terms by a first automated assistant based on the inferred preferences of a second user. The generated set of modified search results may be further based on the filter terms.

[0018] In some implementations, the program instructions may be further executable for: identifying the second user based on the second user's voice detected by the microphone of the first client device, or based on the second user's face detected by the camera of the first client device; and identifying the second user's mobile device based on the second user's identity. Adding the second user to the query session in response to receiving additional input may include adding the second user's mobile phone to the query session. Providing the second user with a modified set of search results may include the first automated assistant causing the modified set of search results to be displayed on the second user's mobile phone based on adding the second user's mobile phone to the query session.

[0019] In some implementations, the program instructions may be further executable for: determining the identity of a second user based on the second user's voice detected by the microphone of the first client device, or based on the second user's face detected by the camera of the first client device; and identifying the second user's user account based on the second user's identity. Adding the second user to the query session in response to receiving additional input may include adding the second user's user account to the query session. Providing the second user with a modified set of search results may be based on subsequently detecting the second user based on the second user's voice or face.

[0020] In some additional or alternative embodiments, the system may include a processor, a computer-readable storage device, one or more computer-readable storage media, and program instructions commonly stored on one or more computer-readable storage media. The program instructions are executable to: receive a first query from a first user of the first client device in a query session by a first automated assistant executing on the first client device; provide the first user with a first set of search results for the first query by the first automated assistant; determine that the first query is relevant to a second user of the first client device based on at least one term in the first query; in response to determining that the first query is relevant to the second user, provide the second user of the first client device with an optional option to join the query session; add the second user to the query session in response to receiving an instruction from the second user to accept the optional option to join the query session; receive additional input from the second user to refine the first query; generate a modified set of search results based on the additional input received from the second user; and provide the modified set of search results to both the first user and the second user by the first automated assistant.

[0021] By utilizing one or more of the techniques described herein, the occurrence of multiple activations of automated assistants, which can waste network and / or computing resources when multiple users perform similar or redundant searches, can be reduced. This leads to improved performance by allowing automated assistants to reduce the number of activations.

[0022] The above description serves as an overview of some embodiments of this disclosure. These embodiments and other embodiments are further described in more detail below.

[0023] Various embodiments may include a non-transitory computer-readable storage medium storing instructions executable by one or more processors (e.g., a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), and / or tensor processing unit (TPU)) to perform methods such as those described herein. Other embodiments may include an automated assistant client device (e.g., a client device including at least one automated assistant for interfacing with a cloud-based automated assistant component), the automated assistant client device including a processor operable to execute the stored instructions to perform methods such as those described herein. Other embodiments may include a system comprising one or more servers, the servers including one or more processors operable to execute the stored instructions to perform methods such as those described herein. Attached Figure Description

[0024] Figure 1 An example environment in which selected aspects of this disclosure can be implemented according to various implementation schemes is schematically depicted.

[0025] Figure 2 A flowchart illustrating an example method for practicing a selected aspect of this disclosure is provided.

[0026] Figure 3 Another flowchart is provided to illustrate example methods for practicing selected aspects of this disclosure.

[0027] Figure 4 Describe the instance architecture of the computing device. Detailed Implementation

[0028] Figure 1 An example environment 100 is schematically depicted, according to various embodiments, in which selected aspects of this disclosure may be implemented. Figure 1 Any computing device depicted in or elsewhere in the figure may include logic such as one or more microprocessors (e.g., a central processing unit or “CPU”, a graphics processing unit or “GPU”) that execute computer-readable instructions stored in memory, or other types of logic such as application-specific integrated circuits (“ASICs”), field-programmable gate arrays (“FPGAs”), etc. Figure 1 Some of the systems described, such as the cloud-based automated assistant component 130, can be implemented using one or more server computing devices that form what is sometimes called “cloud infrastructure,” although this is not required.

[0029] In some embodiments, environment 100 may include client device 110 (or multiple client devices 110) implementing automated assistant client 120. Client device 110 is illustrated as having one or more microphones 111, one or more speakers 112, one or more cameras and / or other visual components 113, and a display 114 (e.g., a touch-sensitive display). Client device 110 may further include pressure sensors, proximity sensors, accelerometers, magnetometers, and / or other sensors for generating sensor data other than audio data captured by the one or more microphones 111. Client device 110 executes automated assistant client 120 at least selectively. Automated assistant client 120 may include an on-device voice capture engine 121, an on-device visual capture engine 122, an on-device hotword detection engine 123, an on-device speech recognizer 124, an on-device natural language understanding (NLU) engine 125, and / or an on-device execution engine 126. The automatic assistant client 120 can include additional and / or alternative engines, such as a voice activity detector (VAD) engine, an endpoint detector engine, and / or other engines.

[0030] One or more cloud-based automated assistant components 130 may optionally be implemented on one or more computing systems (collectively referred to as "cloud" computing systems) communicatively coupled to client device 110 via one or more local area networks and / or wide area networks (e.g., the Internet), typically indicated at 150. The cloud-based automated assistant components 130 may be implemented, for example, via a cluster of high-performance servers. In various embodiments, an instance of the automated assistant client 120 may form a logical instance of an automated assistant from the user's perspective through its interaction with one or more cloud-based automated assistant components 130, which the user can utilize for human-computer interaction (e.g., verbal interaction, gesture-based interaction, and / or touch-based interaction).

[0031] Client device 110 can be used by two or more users and can be, for example: a standalone interactive speaker, a standalone interactive display device (which may also include a speaker and / or a camera), a smart appliance such as a smart TV (or a standard TV with automatic assistant capabilities equipped with a network radar detector), a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in the user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), and / or a wearable device for the user that includes a computing device (e.g., a watch for the user with a computing device, glasses for the user with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client devices may be provided.

[0032] One or more vision components 113 can take various forms, such as an integrated camera, a stereo camera, a LiDAR component (or other laser-based component), a radar component, etc. One or more vision components 113 can be used, for example, by a vision capture engine 122 to capture visual frames (e.g., image frames, laser-based visual frames) of the environment in which the client device 110 is deployed. In some embodiments, such visual frames can be used to determine whether a user is near the client device 110 and / or whether the user (e.g., the user's face) is at a distance relative to the client device 110. For example, this determination can be used to determine whether activation is required. Figure 1 Machine learning engines and / or other engines on various devices described herein.

[0033] The voice capture engine 121 can be configured to capture the user's voice and / or other audio data captured via microphone 111. Additionally, the client device 110 may include pressure sensors, proximity sensors, accelerometers, magnetometers, and / or other sensors used to generate sensor data in addition to the audio data captured by microphone 111. As described herein, the hot word detection engine 123 and / or other engines can utilize this audio data and other sensor data to determine whether to activate one or more currently dormant automatic assistant functions, avoid activating one or more currently dormant automatic assistant functions, and / or deactivate one or more currently active automatic assistant functions. Automatic assistant functions can include an on-device speech recognizer 124, an on-device NLU engine 125, an on-device execution engine 126, and additional and / or alternative engines. For example, the on-device speech recognizer 124 can utilize an on-device speech recognition model to process the audio data of captured spoken utterances to generate recognized text corresponding to the spoken utterances. The on-device NLU engine 125 may optionally utilize an on-device NLU model to perform on-device natural language understanding on the recognized text to generate NLU data. For example, NLU data can include the intent corresponding to the spoken utterance and, optionally, parameters of the intent (e.g., slot values). Furthermore, the on-device execution engine 126 optionally generates execution data based on the NLU data using an on-device execution model. This execution data can define local and / or remote responses to the spoken utterance (e.g., answers), interactions with locally installed applications based on the spoken utterance, commands transmitted to Internet of Things (IoT) devices based on the spoken utterance (directly or via a corresponding remote system), and / or other parsed actions performed based on the spoken utterance. The execution data is then provided for local and / or remote execution / enforcement of the determined actions to parse the spoken utterance. Execution can include, for example, presenting local and / or remote responses (e.g., visual and / or auditory presentation (optionally utilizing a local text-to-speech model)), interacting with locally installed applications, transmitting commands to IoT devices, and / or other actions.

[0034] Display 114 can be used to display recognized text from the speech recognizer 124 on the device, and / or from one or more results of an execution (e.g., search results in response to a search query). Display 114 can also be one of the user interface output components through which a responsive visual portion is presented from the automated assistant client 120.

[0035] In some implementations, the cloud-based automated assistant component 130 may include a remote ASR engine 131 performing speech recognition, a remote NLU engine 132 performing natural language understanding, and / or a remote performance engine 133 generating performance. A remote execution module may also be optionally included, which performs remote execution based on locally or remotely determined performance data. Additional and / or alternative remote engines may be included. In various implementations, on-device speech processing, on-device NLU, on-device performance, and / or on-device execution may be prioritized at least because they provide reduced latency and / or network usage when parsing spoken utterance (due to the elimination of client-server round trips for parsing spoken utterance). However, one or more cloud-based automated assistant components 130 may be selectively utilized at least. For example, such components may be used in parallel with on-device components, and their output may be used when a local component fails. For example, on-device performance engine 126 may fail in certain situations (e.g., due to the relatively limited resources of client device 110), and remote performance engine 133 may utilize the more powerful resources of the cloud in such cases to generate performance data. The remote execution engine 133 can operate in parallel with the on-device execution engine 126 and utilize its results when the on-device execution fails, or it can invoke the remote execution engine in response to determining that the on-device execution engine 126 has failed.

[0036] In various implementations, the NLU engine (on-device and / or remote) is capable of generating NLU data that includes one or more annotations of the recognized text and one or more (e.g., all) terms of the natural language input. In some implementations, the NLU engine is configured to identify and annotate various types of grammatical information in the natural language input. For example, the NLU engine may include a morphological module that can, for example, separate individual words into morphemes and / or, for example, annotate morphemes using their categories. The NLU engine may also include a portion of a speech annotator configured to annotate terms using their grammatical rules. Furthermore, for example, in some implementations, the NLU engine may additionally and / or alternatively include a dependency analyzer configured to determine grammatical relationships between terms in the natural language input.

[0037] In some implementations, the NLU engine may additionally and / or alternatively include an entity annotator configured to annotate entity references in one or more fragments, such as references to people (including, for example, literary characters, celebrities, public figures, etc.), organizations, locations (real and virtual), etc. In some implementations, the NLU engine may additionally and / or alternatively include a coreference resolver (not depicted) configured to group or "cluster" references to the same entity based on one or more contextual cues. In some implementations, one or more components of the NLU engine may depend on annotations from one or more other components of the NLU engine.

[0038] The NLU engine may also include an intent matcher configured to determine the intent of a user engaging with the automated assistant client 120. The intent matcher is capable of using various techniques to determine the user's intent. In some implementations, the intent matcher may access one or more local and / or remote data structures that include, for example, multiple mappings between grammars and response intents. For example, the grammars included in the mappings can be selected and / or learned over time and may represent common user intents. For instance, a grammar “play <artist>” can be mapped to an intent to invoke a response action that causes music by <artist> to play on client device 110. Another grammar, “[weather|forecast] today”, can be matched with user queries such as “What’s the weather like today?” and “What’s the weather forecast for today?”. In addition to or instead of grammars, in some implementations, the intent matcher is capable of employing one or more trained machine learning models, individually or in combination, one or more grammars. These trained machine learning models can be trained, for example, by embedding recognized text from spoken utterances into a dimensionality-reduced space, and then, for example, using techniques such as Euclidean distance, cosine similarity, etc., to determine which other embeddings (and therefore intents) are closest to identify the intent. As seen in the “Play <Artist>” example syntax above, some syntaxes have slots (e.g., <Artist>) that can be filled with slot values ​​(or “parameters”). Slot values ​​can be determined in various ways. Users often actively provide slot values. For example, for the syntax “Give me a <toppings> pizza,” a user might say the phrase “Give me a sausage pizza,” in which case the slot <toppings> is automatically filled. Other slot values ​​can be inferred based on factors such as user location, currently presented content, user preferences, and / or other cues.

[0039] The fulfillment engine (local and / or remote) can be configured to receive predicted / estimated intents output by the NLU engine, along with any associated slot values, and fulfill (or “parse”) the intents. In various implementations, the fulfillment (or “parse”) of a user intent can result in, for example, the generation / acquisition of various fulfillment information (also referred to as fulfillment data) by the fulfillment engine. This can include determining local and / or remote responses to spoken utterances (e.g., answers), interactions with locally installed applications based on spoken utterances, commands transmitted to Internet of Things (IoT) devices based on spoken utterances (directly or via corresponding remote systems), and / or other parsed actions performed based on spoken utterances. On-device fulfillment can then initiate the local and / or remote execution / implementation of the determined actions used to parse the spoken utterances.

[0040] In implementations, environment 100 may further include user devices 140-1, ..., 140-n, which in various implementations may include a user input engine 141 and a rendering engine 142. User devices 140-1, ..., 140-n may communicate with client device 110 and / or cloud-based automated assistant component 130 via computer network 150. Each of user devices 140-1, ..., 140-n may be a user's personal device (e.g., a device typically used by a single user and not frequently used by multiple users) and may be, for example, a mobile phone computing device, a desktop computing device, a laptop computing device, a tablet computing device, and / or a wearable device for the user including a computing device (e.g., a watch for a user with a computing device, glasses for a user with a computing device, a virtual or augmented reality computing device). Additional and / or alternative user devices may be provided.

[0041] User input engine 141 is capable of detecting various types of user input at user devices 140-1, ..., 140-n. User input detected at user devices 140-1, ..., 140-n can include verbal input detected via the microphone of user device 140-1, ..., 140-n, touch input detected via the user interface input device (e.g., touchscreen) of user device 140-1, ..., 140-n, and / or typed input detected via the user interface input device (e.g., via a virtual keyboard on the touchscreen) of user device 140-1, ..., 140-n. In some embodiments, user devices 140-1, ..., 140-n can be used to provide input to the automated assistant client 120 on client device 110. For example, user input detected by user input engine 141 may include touch input relating to a search being performed using the automatic assistant client 120, 5-key typing input including one or more terms of a search query, verbal input including one or more terms of a search query, and / or any other input relating to client devices 140-1, ..., 140-n on the automatic assistant client 120.

[0042] In some implementations, the Auto Assistant client 1200 on client device 110 can use user devices 140-1, ..., 140-n to present the visual portion of the response from Auto Assistant client 120 (e.g., using rendering engine 142).

[0043] Figure 2 This is a flowchart illustrating an example method 200 for a collaborative search session via an automated assistant, according to an embodiment disclosed herein. For convenience, the operations of the flowchart are described with reference to the system performing operation 5. This system may include various components of various computer systems, such as one or more components of the client device 110. Furthermore, although the operations of method 200 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0044] At block 205, the system receives a first query in a query session from a first user on the first client device via a first automated assistant executing on the first client device. In some embodiments, at block 205, an automated assistant client 120 executing on client device 110 may receive the first query in a query session from the first user on client device 110. In some embodiments, the query session may be a shopping session. For example, the first user may be buying a television, and the automated assistant client 120 may receive a query that might be spoken by the first user, such as "Computer, show me the best-selling 65-inch television."

[0045] At box 210, the system provides a first set of search results for a first query to a first user via a first automated assistant. In some embodiments, at box 210, an automated assistant client 120 executing on client device 1100 may provide the first user with a first set of search results for the first query received at box 205. In some embodiments, the automated assistant client 120 may provide the first set of search results by rendering the first set of search results, for example, on the display 114 of client device 110 by rendering engine 142 and / or on the first user's user device (e.g., one of user devices 140-1, ..., 140-n). In some embodiments, the first set of search results includes a first set of products. In the above example, the first set of search results may be a first set of televisions (e.g., a set of best-selling 65-inch televisions).

[0046] Referring again to block 210, in some embodiments, the first client device (e.g., client device 110) may be an automated assistant device, and the first set of search results provided may include a first automated assistant (e.g., automated assistant client 120) that causes the first set of search results to be provided on the display of a first user's mobile device (e.g., one of user devices 140-1, ..., 140-n).

[0047] At box 215, the system determines whether the first query is relevant to a second user of the first client device based on at least one term in the first query. In some embodiments, at box 215, an automated assistant client 120 executing on client device 110 may determine whether the first query is relevant to a second user of client device 110 based on at least one term in the first query received at box 205. In some embodiments, the automated assistant client 120 may determine that the first query is relevant to a second user of client device 110 based on a score for one of the at least one term in the first query meeting a threshold. In some embodiments, a score may be assigned to a term and / or a score may be determined based on the expected relevance of the term to other users of the client device. For example, terms such as “television” and “dishwasher” that are associated with commonly shared products may be associated with a relatively high score (e.g., a score that meets the threshold), while terms such as “toothbrush” and “socks” that are associated with less commonly shared products may be associated with a relatively low score (e.g., a score that does not meet the threshold).

[0048] In some implementations, the account associated with the first user and the account associated with the second user may belong to the same family of accounts. In other implementations, both the account associated with the first user and the account associated with the second user can log in on the client device 110.

[0049] In some implementations, the automated assistant client 120 may determine that the first query is relevant to the second user of the first client device based on the fact that the second user has previously performed searches similar to the first query, and / or based on the fact that the second user has previously performed searches that indicate a high probability of interest in the first query. For example, if the second user has previously performed multiple searches related to new technology products, the automated assistant client 120 may infer that the second user is interested in new technology products and may determine that the first query related to television is relevant to the second user. In some implementations, BERT (Bidirectional Encoder Representation from Transformer) or other machine learning methods may be used to determine query similarity.

[0050] Referring back to box 215, in response to the automatic assistant client 120 determining that the first query is unrelated to the second user of the first client device, the process proceeds to box 220, and the method ends. On the other hand, in response to the automatic assistant client 120 determining that the first query is related to the second user of the first client device, the process proceeds to box 225.

[0051] Referring again to block 215, in some embodiments, the automated assistant client 120 executing on client device 110 can determine the predicted interest level of the second user in a query session. The automated assistant client 120 can further determine whether a first query is relevant to the second user based on whether the predicted interest level of the second user in the query session meets a threshold. In some embodiments, the automated assistant client 120 can determine the predicted interest level of the second user in a query session based on the second user's query history.

[0052] Referring again to block 215, in some embodiments, the automated assistant client 120 executing on client device 110 can determine the environmental context. The automated assistant client 120 can further determine, based on the environmental context, whether the first query is relevant to a second user.

[0053] For example, the environmental context when the first query is received at frame 205 may include one or more signals that can be used to determine whether the first query is relevant to the second user. If the first user provides the first query in a "private" setting, where no other person is detected in the camera frame and / or the microphone does not detect another person's voice during a threshold time period before and / or after the first query is received at frame 205, the automated assistant client 120 executing on client device 110 may determine that the first query is irrelevant to the second user based on one or more identified environmental signals. Similarly, if the automated assistant client 120 executing on client device 110 determines that the first query was provided by the first user when no one else is home, the automated assistant client 120 executing on client device 110 may determine that the first query is irrelevant to the second user based on one or more identified environmental signals. Furthermore, if the first user provides the first query after the user enters "private" or "incognito" mode (e.g., by saying "let's keep it a secret"), the automated assistant client 120 executing on client device 110 may determine that the first query is irrelevant to the second user.

[0054] On the other hand, if other users are present in the room where client device 110 is located, and / or when the automated assistant client 120 executing on client device 110 determines that another user (e.g., another person, such as the second user) is nearby in the camera frame and / or the voice of another person is detected by the microphone during a threshold time period before and / or after the first query is received at frame 205, the automated assistant client 120 executing on client device 110 can determine that the first query is related to the second user. Similarly, if other users frequently use the automated assistant client 120 executing on client device 110 (e.g., at least a threshold percentage of queries are made by other people such as the second user, and / or at least a threshold number of queries are made by other people such as the second user), the automated assistant client 120 executing on client device 110 can determine that the first query is related to the second user.

[0055] Specifically, when a first query is received at box 205, the automatic assistant client 120 may identify one or more environmental signals associated with the environment in which the first user of the client device 110 is located. The one or more environmental signals may include, for example, location information corresponding to the location of the first user when the first query is received at box 205, audio data capturing ambient noise when the first query is received at box 205, and / or visual data capturing the environment when the first query is received at box 205.

[0056] For example, continuing the above example, suppose the first user of client device 110 submits a search query, "Computer, show me the best-selling 65-inch TVs." Furthermore, suppose that when the user submits the search query, one or more environmental signals indicate that the first user is in their living room at home, and another person is present in the living room next to the first user. In this example, the automated assistant client 120 can determine that the first query is relevant to a second user based on one or more identified environmental signals, such as the proximity of the first location to the first user and location information (living room).

[0057] Alternatively, suppose that when a user submits a search query, one or more environmental signals indicate that the first user is at home in their office and that no one else is in the office. In this example, the automated assistant client 120 can determine that the first query is irrelevant to the second user based on one or more identified environmental signals, such as the absence of other people near the first user and location information (office).

[0058] Referring again to block 215, in some embodiments, an automated assistant client 120 executing on client device 110 can determine the frequency of interaction between the second user and client device 110. The automated assistant client 120 can further determine whether the first query is relevant to the second user based on whether the frequency of interaction between the second user and client device 110 meets a threshold.

[0059] At block 225, the system provides selectable options to the first user via a first automated assistant to allow a second user of the first client device to join the query session. In one embodiment, at block 225, in response to determining at block 215 that the first query is relevant to the second user of client device 110, the automated assistant client 120 executing on client device 110 may provide selectable options to the first user to allow the second user of client device 110 to join the query session. In some embodiments, the automated assistant client 120 may provide selectable options, for example, by visually presenting selectable options (e.g., “Would you like to allow user 2 to participate in the search?”) on the user interface of client device 110, and / or by audibly presenting selectable options on client device 110.

[0060] At box 230, the system determines whether an instruction to accept the optional option to allow a second user of the first client device to join the query session has been received. In some embodiments, the first user can provide user input (e.g., a tap or click) via the user interface of client device 110, which is an instruction to accept or reject the optional option to allow a second user of the first client device to join the query session. Alternatively, the first user can provide a verbal response (e.g., "yes" or "no"), which is an instruction to accept or reject the optional option to allow a second user of the first client device to join the query session.

[0061] Referring again to block 230, in one embodiment, in response to the automated assistant client 120 determining that it has not yet received an instruction to accept the optional option provided at block 225 to allow the second user of the first client device to join the query session (e.g., receiving a rejection of the optional option), the process proceeds to block 220, and the method ends. On the other hand, in response to the automated assistant client 120 determining that it has received an instruction to accept the optional option provided at block 225 to allow the second user of the first client device to join the query session, the process proceeds to block 235.

[0062] At block 235, in response to determining that the first query is relevant to the second user, the system provides the second user of the first client device with an optional option to join the query session via the first automated assistant. In some embodiments, at block 235, in response to determining at block 215 that the first query is relevant to the second user of client device 110, the automated assistant client 120 executing on client device 110 may provide the second user of the first client device 110 with an optional option to join the query session. In some embodiments, the automated assistant client 120 providing the optional option to join the query session may further respond to receiving an instruction from the first user at block 230 to accept the optional option to allow the second user of the first client device 110 to join the query session. In some embodiments, the automated assistant client 120 may provide the optional option, for example, by visually presenting the optional option (e.g., “User 2, would you like to participate in the search?”) on the user interface of client device 110, and / or by audibly presenting the optional option on client device 110.

[0063] At box 240, the system determines whether an indication has been received from the second user to accept the optional option of joining the query session. In some embodiments, the second user can provide user input (e.g., a tap or click) via the user interface of client device 110, which is an indication to accept or reject the optional option of joining the query session. Alternatively, the second user can provide a verbal response (e.g., "yes" or "no"), which is an indication to accept or reject the optional option of joining the query session.

[0064] Referring again to box 240, in one implementation, in response to the automated assistant client 120 determining that it has not yet received an indication to accept the optional option to join the query session provided at box 235 (e.g., a rejection of the optional option), the process proceeds to box 220 and the method ends. Alternatively, in response to the automated assistant client 120 determining that it has received an indication to accept the optional option to join the query session provided at box 235, the process proceeds to box 245.

[0065] At block 245, in response to receiving an instruction from the second user to accept the optional option to join the query session, the system adds the second user to the query session. In one embodiment, at block 245, in response to receiving an instruction from the second user to accept the optional option to join the query session at block 240, the automatic assistant client 120 executing on the client device 110 can add the second user to the query session.

[0066] Referring again to box 245, in some embodiments, a member of the query session (e.g., a second user) may be able to send messages or comments (e.g., about products or other search results, such as interests, feedback, desired options / constraints, etc.) to other members of the query session (e.g., a first user) via an automated assistant client 120 executing on a client device or via an automated assistant client executing on one of the user devices 140-1, ..., 140-n (e.g., the user's mobile device), or tag search results of other members of the query session. In some embodiments, members of the query session may be able to add / remove products in a shared shopping cart. In some embodiments, the automated assistant client 120 may obtain approval from one member of the query session (e.g., a designated administrator), multiple members (e.g., two parents), or all members before allowing products to be added / removed from the cart. In other embodiments, members may be able to add / remove products from the cart without obtaining approval from other members of the query session.

[0067] At box 250, the system receives additional input from the second user to refine the first query. In one embodiment, at box 250, the automated assistant client 120 executing on client device 110 can receive additional input from the second user to refine the first query received at box 205. The additional input can be provided by the second user via the user interface of client device 110, such as via a touchscreen. For example, the second user can tap the touchscreen to select filters for the first set of search results (e.g., “price under $1000”, “no red,” “HDMI 2.1 only,” etc.), applying, deleting, or modifying filters based on user input. Alternatively, the second user can speak the additional input. For example, the second user can say “I want a 4K model.” In other embodiments, the received additional input can be the second user's ranking of the first search results.

[0068] Referring again to frame 250, in some embodiments, additional input provided by a second user may be displayed on the user interface of client device 110 or user device 140-1, ...

[0069] In one of the user interfaces 140-n. For example, additional input from a second user (e.g., preferences) can be displayed visually separately, for example, through separate columns or other visual cues on the display 114 of the client device 110. For example, each user can be represented by a circle on the user interface of the display 0 114, and tapping one of the circles can display additional input (e.g., preferences) from the selected user. For example, tapping the circle representing the second user on the user interface may result in the second user's additional input "I want a 4K model" being displayed on the user interface, and the user (e.g., the second user) can annotate with the added additional input.

[0070] Referring again to block 250, in some embodiments, the second client device receives additional input. The second client device may be another client device 110, or one of user devices 140-1, ..., 140-n. Specifically, in some embodiments, additional input may be provided by a second user via a user interface of another client device 110 or one of user devices 140-1, ..., 140-n, which user device may then provide the additional input to the automated assistant client 120 on client device 110.

[0071] At box 255, the system generates a modified set of search results based on additional input received from the second user. In one implementation, at box 255, the automated assistant client 120 executing on client device 110 can generate the modified set of search results based on additional input received from the second user at box 250.

[0072] Referring again to box 255, in some embodiments, generating the modified set of search results includes the system determining a second query based on a first auto-assistant and additional input; and generating the modified set of search results based on the second set of search results for the second query. Specifically, the auto-assistant client 120 executing on client device 110 can determine the second query based on the first query received at box 205 and additional input received at box 250, and generate the modified set of search results based on the second set of search results for the second query. For example, in the case of the first query “Computer, show me the best-selling 65-inch TVs” and the additional input “I want a 4K model”, the auto-assistant client 120 can determine the second query “Best-selling 65-inch 4K TVs”.

[0073] Referring again to box 255, in other embodiments, the set of modified search results generated includes a first set of search results filtered by the system based on additional input. Specifically, the automated assistant client 120 executing on client device 110 can filter the first set of search results from box 210 based on additional input received at box 250. For example, in the case of the first query “Computer, show me the best-selling 65-inch TVs” and the additional input “I want 4K models”, the automated assistant client 120 can determine that “4K” is the attribute it filters, and the automated assistant client 120 can filter the first set of search results from box 210 to identify and display a subset of the first set of search results associated with the “4K” attribute, and hide other search results in the subset of the first set of search results not associated with the “4K” attribute.

[0074] Referring again to box 255, in other embodiments, generating the modified set of search results includes the system reordering a first set of search results based on the rankings provided at box 250, and optionally hiding or removing products not included in the rankings. In some embodiments, when the rankings provided at box 250 include similar or nearly identical items, users can vote on the item they deem most suitable given the available options. In some embodiments, the ranking of displayed advertisements can be determined based on the rankings provided at box 250.

[0075] Referring again to box 255, in other embodiments, instead of generating a set of revised search results (e.g., by incorporating additional input received from the second user into the search session), the second user's additional input (e.g., preferences) can be displayed visually separately, for example, through separate columns or other visual cues on the display 114 of the client device 110. For example, each user can be represented by a circle on the user interface on the display 114, and tapping one of the circles can display the preferences of the selected user. In some embodiments, the preferences of one or more other users may be temporarily enabled / disabled, or completely removed, for example, by providing the option of providing explanatory messages to other users in the session (e.g., "I really don't want to buy a red TV, is that okay?").

[0076] Referring again to box 255, in some embodiments, the system automatically determines filter terms based on the inferred preferences of a second user by a first automated assistant. The system can further generate a modified set of search results based on the filter terms. In some embodiments, the inferred preferences can be based on the second user's past searches.

[0077] At box 260, the system provides a set of modified search results to a first user and a second user via a first automated assistant. In some embodiments, at box 260, an automated assistant client 120 executing on client device 110 may provide the first user and the second user with the set of modified search results generated at box 255. In some embodiments, the automated assistant client 120 may provide the set of modified search results by having the set of modified search results, for example, rendered by rendering engine 142 on the display 114 of client device 110 and / or on the user device of the first user (e.g., one of user devices 140-1, ..., 140-n), and / or on the user device of the second user (e.g., one of user devices 140-1, ..., 140-n), for example, rendered by rendering engine 142. In some embodiments, the set of modified search results includes a set of modified products. In the above example, the set of modified search results could be a set of modified televisions (e.g., a set of best-selling 65-inch 4K televisions).

[0078] Referring again to box 260, in some embodiments, providing the modified set of search results may include a first automated assistant (e.g., automated assistant client 120) that causes the modified set of search results to be provided on the display of a first user's mobile phone (e.g., one of user devices 140-1, ..., 140-n) and on the display of a second user's mobile device (e.g., another of user devices 140-1, ..., 140-n).

[0079] In some implementations, some or all of the operations in boxes 215 to 260 may optionally be repeated relative to another user (e.g., a third user). Additionally, during subsequent iterations, further input received at box 250 may be used to further refine the first query and may be received from the first user, the second user, and / or another user.

[0080] Figure 3 This is a flowchart illustrating an example method 300 for a collaborative search session via an automated assistant, according to an embodiment disclosed herein. For convenience, the operations of the flowchart are described with reference to the system performing the operations. This system may include various components of various computer systems, such as one or more components of the client device 110. Furthermore, although the operations of method 300 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0081] At block 310, the system receives a first query in a query session from a first user on the first client device via a first automated assistant executing on the first client device. In some embodiments, at block 310, an automated assistant client 120 executing on client device 110 may receive the first query in a query session from the first user on client device 110. In some embodiments, the query session may be a shopping session. For example, the first user may be buying a television, and the automated assistant client 120 may receive a query that the first user might utter, "Computer, show me the best-selling 65-inch television."

[0082] At box 320, the system provides a first set of search results for a first query to a first user via a first automated assistant. In some embodiments, at box 320, an automated assistant client 120 executing on client device 110 may provide the first user with a first set of search results for the first query received at box 310. In some embodiments, the automated assistant client 120 may provide the first set of search results by rendering the first set of search results, for example, on the display 114 of client device 110 by rendering engine 142 and / or on the first user's user device (e.g., one of user devices 140-1, ..., 140-n). In some embodiments, the first set of search results includes a first set of products. In the above example, the first set of search results may be a first set of televisions (e.g., a first set of best-selling 65-inch televisions).

[0083] Referring again to block 320, in some embodiments, the first client device (e.g., client device 110) may be an automated assistant device, and providing the first set of search results may include a first automated assistant (e.g., automated assistant client 120) that causes the first set of search results to be displayed on the display of a first user's mobile device (e.g., one of user devices 140-1, ..., 140-n).

[0084] At block 330, the system receives additional input from a second user on the first client device via a first automated assistant to refine the first query. In one embodiment, at block 330, an automated assistant client 120 executing on client device 110 may receive additional input from the second user to refine the first query received at block 310. This additional input may be provided by the second user via the user interface of client device 110, such as via a touchscreen. For example, the second user may tap the touchscreen to select filters for a first set of search results (e.g., “price under $1000”, “no red”, “HDMI 2.1 only”, etc.). Alternatively, the second user may speak additional input. For example, the second user may say, “I want a 4K model.”

[0085] At block 340, in response to receiving additional input from the second user to refine the first query, the system adds the second user to the query session. In an embodiment, at block 340, in response to receiving additional input from the second user to refine the first query at block 330, the automated assistant client 120 executing on client device 110 can add the second user to the query session.

[0086] Referring again to box 340, in some embodiments, a member of the query session (e.g., a second user) may be able to send messages or comments (e.g., about products or other search results, such as interests, feedback, desired options / constraints, etc.) to other members of the query session (e.g., a first user) via an automated assistant client 120 executing on a client device or via an automated assistant client executing on one of the user devices 140-1, ..., 140-n (e.g., the user's mobile device), or tag search results of other members of the query session. In some embodiments, members of the query session may be able to add / remove products from a shared shopping cart. In some embodiments, the automated assistant client 120 may obtain approval from one member of the query session (e.g., a designated administrator), multiple members (e.g., two parents), or all members before allowing products to be added / removed from the cart. In other embodiments, members may be able to add / remove products from the cart without obtaining approval from other members of the query session.

[0087] Referring again to block 340, in some embodiments, the system (e.g., the auto-assistant client 120 of client device 110) may determine the identity of the second user based, for example, on the second user's voice detected by the microphone of the first client device (e.g., one or more microphones 111 of client device 110), on the second user's face detected by the camera of the first client device (e.g., one or more cameras and / or other vision components 113 of client device 110), and / or using other wireless signals such as Bluetooth signals that can be used to detect the presence of the second user's smartwatch. The system (e.g., the auto-assistant client 120 of client device 110) may identify the second user's mobile device based on the second user's identity. In some embodiments, adding the second user to a query session in response to receiving additional input may include adding the second user's mobile phone to the query session. Additionally, in some implementations, providing the modified set of search results to the second user may include a first automated assistant (e.g., automated assistant client 120 of client device 110) providing the modified set of search results on the display of the second user's mobile phone (e.g., one of user devices 140-1, ..., 140-n) by adding the second user's mobile phone to the query session.

[0088] Referring again to block 340, in some embodiments, the system (e.g., the automated assistant client 120 of client device 110) may determine the identity of the second user, for example, based on the second user's voice detected by the microphone of the first client device (e.g., one or more microphones 111 of client device 110), or based on the second user's face detected by the camera of the first client device (e.g., one or more cameras and / or other visual components 113 of client device 110). The system (e.g., the automated assistant client 120 of client device 110) may identify the second user's user account based on the second user's identity. In some embodiments, adding the second user to a query session in response to receiving additional input may include adding the second user's user account to the query session. Additionally, in some embodiments, providing the second user with a modified set of search results may be based on subsequent detection of the second user (e.g., by the automated assistant client 120 of client device 110) based on the second user's voice or face.

[0089] At box 350, the system generates a modified set of search results based on additional input received from the second user. In one implementation, at box 350, the automated assistant client 120 executing on client device 110 can generate the modified set of search results based on additional input received from the second user at box 340.

[0090] Referring again to block 350, in some embodiments, generating modified search results includes the system determining a second query by a first automated assistant based on a first query and additional input; and generating a set of modified search results based on a second set of search results from the second query. Specifically, the automated assistant client 120 executing on client device 110 may determine the second query based on the first query received at block 310 and additional input received at block 330, and generate a set of modified search results based on a second set of search results from the second query. For example, in the case of the first query “Computer, show me the best-selling 65-inch TVs” and the additional input “I want a 4K model”, the automated assistant client 120 may determine the second query “Best-selling 65-inch 4K TVs”.

[0091] Referring again to box 350, in other embodiments, the set of modified search results generated includes a first set of search results filtered by the system based on additional input. Specifically, the automated assistant client 120 executing on client device 110 can filter the first set of search results from box 320 based on additional input received at box 330. For example, in the case of the first query “Computer, show me the best-selling 65-inch TVs” and the additional input “I want 4K models”, the automated assistant client 120 can determine that “4K” is the attribute it filters, and the automated assistant client 120 can filter the first set of search results from box 320 to identify a subset of the first set of search results associated with the “4K” attribute.

[0092] Referring again to box 350, in some implementations, the system automatically determines filter terms by a first automated assistant based on the preferences of a second user inferred from one or more searches previously performed by the user. The system can further generate a modified set of search results based on the filter terms.

[0093] At box 360, the system provides a set of modified search results to a first user and a second user via a first automated assistant. In one embodiment, the set of modified search results may be provided to a second user based on adding the second user to the query session at box 340. In another embodiment, at box 360, an automated assistant client 120 executing on client device 110 may provide the first user and the second user with the set of modified search results generated at box 350. In some embodiments, the automated assistant client 120 may provide the set of modified search results by having the set of modified search results rendered, for example, by rendering engine 142 on the display 114 of client device 110 and / or on the user device of the first user (e.g., one of user devices 140-1, ..., 140-n), and / or on the user device of the second user (e.g., one of user devices 140-1, ..., 140-n) by rendering engine 142. In some embodiments, the set of modified search results includes a set of modified products. In the above example, the set of modified search results could be a set of modified TVs (e.g., a set of best-selling 65-inch 4K TVs).

[0094] Referring again to box 360, in some embodiments, providing the set of modified search results may include a first automated assistant (e.g., automated assistant client 120) that causes the set of modified search results to be provided on the display of a first user's mobile phone (e.g., one of user devices 140-1, ..., 140-n) and on the display of a second user's mobile device (e.g., another of user devices 140-1, ..., 140-n).

[0095] Referring again to box 360, in some embodiments, providing the modified set of search results may include, in response to determining that a first user and / or a second user are near the client device 110 (e.g., in the same room as the client device 110), the automated assistant client 120 causes the modified set of search results to be provided on the display 114 of the client device 110. For example, the automated assistant client 120 may determine that the first user and / or the second user are near the client device 110 based on the detection of the first user and / or the second user's voice by one or more microphones 111 of the client device 110, or based on the detection of the first user and / or the second user's face by one or more cameras and / or other visual components 113 of the client device 110.

[0096] Still referring to box 360, in some embodiments, the automated assistant client 120 may avoid providing a modified set of search results in response to determining (e.g., using one or more microphones 111 and / or one or more cameras and / or other visual components 113 of the client device 110) that someone other than the first user and / or the second user is approaching the client device 110, for example, to avoid spoiling the surprise element about the gift.

[0097] In some implementations, some or all of the operations in boxes 330 to 360 may optionally be repeated relative to another user (e.g., a third user). Additionally, during subsequent iterations, further input received at box 330 may be used to further refine the first query and may be received from the first user, the second user, and / or another user.

[0098] Figure 4 This is a block diagram of an example computing device 410 that can be optionally used to perform one or more aspects of the techniques described herein. In some embodiments, one or more of a client device, a cloud-based automated assistant component, and / or other components may include one or more components of the example computing device 410.

[0099] Computing device 410 typically includes at least one processor 412 that communicates with a plurality of peripheral devices via a bus subsystem 414. These peripheral devices may include: a storage subsystem 424, which includes, for example, a memory subsystem 425 and a file storage subsystem 426; a user interface output device 420; a user interface input device 422; and a network interface subsystem 416. The input and output devices allow users to interact with computing device 410. The network interface subsystem 416 provides an interface to an external network and couples to corresponding interface devices in other computing devices.

[0100] User interface input device 422 may include a keyboard, pointing devices such as a mouse, trackball, touchpad or graphics tablet, scanner, touchscreen integrated into a display, audio input devices such as a voice recognition system, microphone, and / or other types of input devices. Generally, the term "input device" is used to include all possible types of means and methods for inputting information into computing device 410 or a communication network.

[0101] User interface output device 420 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for producing a visible image. The display subsystem may also provide a non-visual display, for example, via an audio output device. Generally, the term "output device" is used to encompass all possible types of means and methods for outputting information from computing device 410 to a user or another machine or computing device.

[0102] Storage subsystem 424 stores the programming and data structures that provide some or all of the functionality of the modules described herein. For example, storage subsystem 424 may include selected aspects that perform the methods disclosed herein and implementations in... Figure 1 The logic of each component described in the text.

[0103] These software modules are typically executed by processor 414 alone or in combination with other processors. Memory subsystem 425, included in storage subsystem 424, may include multiple memories, including main random access memory (RAM) 430 for storing instructions and data during program execution and read-only memory (ROM) 432 for storing fixed instructions. File storage subsystem 426 provides persistent storage for program and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical disk drives, or removable media cartridges. Modules implementing the functionality of certain embodiments may be stored by file storage subsystem 426 within storage subsystem 424 or in other machines accessible to processor 414.

[0104] Bus subsystem 412 provides a mechanism for enabling the various components and subsystems of computing device 410 to communicate with each other as intended. Although bus subsystem 412 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0105] The computing device 410 can be of various types, including workstations, servers, computing clusters, blade servers, server groups, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, therefore... Figure 4 The description of the computing device 410 depicted herein is intended only as a specific example for illustrating some implementation schemes. Many other configurations of the computing device 410 may have more... Figure 4 The computing device depicted in the text has more or fewer components.

[0106] While several embodiments have been described and illustrated herein, various other ways and / or structures may be used to perform functions and / or obtain results and / or one or more advantages described herein, and each of these variations and / or modifications is considered to be within the scope of the embodiments described herein. More generally, all parameters, dimensions, materials, and configurations described herein are exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications to which the teachings are applied. Those skilled in the art will recognize or be able to identify many equivalents of the specific embodiments described herein using only conventional experimentation. Therefore, it should be understood that the foregoing embodiments are presented by way of example only, and that embodiments may be practiced in ways other than those specifically described and claimed within the scope of the appended claims and their equivalents. Embodiments of this disclosure relate to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of this disclosure if such features, systems, articles, materials, kits, and / or methods are not contradictory.

Claims

1. A method implemented by one or more processors, the method comprising: A first automated assistant, executed on a first client device, receives a first query from a first user of the first client device in a query session; The first set of search results for the first query is provided to the first user by the first automated assistant. Determine the frequency of interaction between the second user of the first client device and the first client device; Determine a score for at least one term in the first query, wherein the score is determined based on the expected relevance of the at least one term to users of the first client device other than the first user, and the expected relevance is determined based on the degree to which the use of items associated with the at least one term is shared among multiple users; Based on (i) the frequency of the interaction between the second user and the first client device meeting a threshold and (ii) the score of at least one term in the first query, it is determined that the first query is related to the second user; In response to determining that the first query is related to the second user, the first automated assistant provides the second user of the first client device with an option to join the query session; In response to receiving an indication from the second user to accept the optional option to join the query session, the second user is added to the query session; Receive additional input from the second user to refine the first query; A set of modified search results is generated based on the additional input received from the second user; and The first automated assistant provides the modified set of search results to the first user and the second user.

2. The method according to claim 1, wherein: The query session is a shopping session; The first set of search results includes a first set of products; and The set of modified search results includes the set of modified products.

3. The method according to claim 1, wherein: The first client device is an automatic assistant device; The first set of search results is provided by the first automated assistant, which causes the first set of search results to be displayed on the display of the first user's mobile device. as well as The set of modified search results is provided by the first automated assistant, which causes the set of modified search results to be displayed on the display of the first user's mobile device and the display of the second user's mobile device.

4. The method of claim 1, further comprising providing the first user with a selectable option by the first automated assistant to allow a second user of the first client device to join the query session, and The option to join the query session is provided to the second user in response to receiving an instruction from the first user to accept the option to allow the second user of the first client device to join the query session.

5. The method of claim 1, further comprising determining the predicted level of interest of the second user in the query session, The determination that the first query is related to the second user is further based on the second user's predicted level of interest in the query session satisfying the level of interest threshold.

6. The method of claim 5, wherein determining the predicted level of interest of the second user in the query session is based on the query history of the second user.

7. The method of claim 1, further comprising determining the environmental setting, The determination that the first query is related to the second user is further based on the environmental context.

8. The method of claim 1, wherein determining that the first query is related to the second user of the first client device is based on the score of the at least one term in the first query satisfying a score threshold.

9. The method of claim 1, further comprising the first automatic assistant automatically determining filter terms based on the inferred preferences of the second user. The set of modified search results is generated further based on the filter terms.

10. The method of claim 1, wherein the second client device receives the additional input.

11. The method of claim 1, wherein generating the modified set of search results comprises: The first automated assistant determines the second query based on the first query and the additional input; as well as The modified set of search results is generated based on the second set of search results from the second query.

12. The method of claim 1, wherein generating the set of modified search results includes filtering the first set of search results based on the additional input.

13. The method according to claim 1, wherein, Terms associated with more commonly shared items were associated with relatively higher scores compared to terms associated with less commonly shared items.

14. A computer program product comprising one or more computer-readable storage media, the computer program product having program instructions commonly stored on the one or more computer-readable storage media, the program instructions being executable for: A first automated assistant, executed on a first client device, receives a first query from a first user of the first client device in a query session; The first set of search results for the first query is provided to the first user by the first automated assistant. Determine the frequency of interaction between the second user of the first client device and the first client device; Determine a score for at least one term in the first query, wherein the score is determined based on the expected relevance of the at least one term to users of the first client device other than the first user, and the expected relevance is determined based on the degree to which the use of items associated with the at least one term is shared among multiple users; Based on (i) the frequency of the interaction between the second user and the first client device meeting a threshold and (ii) the score of at least one term in the first query, it is determined that the first query is related to the second user; In response to determining that the first query is related to the second user, the first automated assistant provides the second user of the first client device with an option to join the query session; In response to receiving an indication from the second user to accept the optional option to join the query session, the second user is added to the query session; The first automated assistant receives additional input from the second user to refine the first query; A set of modified search results is generated based on the additional input received from the second user; and The first automated assistant provides the modified set of search results to the first user and the second user.

15. The computer program product according to claim 14, wherein: The query session is a shopping session; The first set of search results includes a first set of products; and The set of modified search results includes the set of modified products.

16. The computer program product according to claim 14, wherein: The first client device is an automatic assistant device; The first set of search results is provided by the first automated assistant, which causes the first set of search results to be displayed on the display of the first user's mobile device. as well as The set of modified search results is provided by the first automatic assistant, which causes the set of modified search results to be provided on the display of the first user's mobile device and on the display of the second user's mobile device.

17. The computer program product according to claim 14, wherein: The program instructions are further executable to provide selectable options from the first automated assistant to the first user, allowing a second user of the first client device to join the query session, and Providing the second user with the optional option to join the query session is further in response to receiving the following instruction from the first user: accepting the optional option to allow the second user of the first client device to join the query session.

18. The computer program product according to claim 14, wherein: The program instructions are further executable to determine the predicted level of interest of the second user in the query session; and Determining that the first query is relevant to the second user is further based on the second user's predicted level of interest in the query session meeting an interest level threshold.

19. The computer program product of claim 18, wherein determining the predicted level of interest of the second user in the query session is based on the query history of the second user.

20. A system comprising: A processor, a computer-readable storage medium, one or more computer-readable storage media, and program instructions stored together on the one or more computer-readable storage media, the program instructions being executable to: A first automated assistant, executed on a first client device, receives a first query from a first user of the first client device in a query session; The first set of search results for the first query is provided to the first user by the first automated assistant. Determine the frequency of interaction between the second user of the first client device and the first client device; Determine a score for at least one term in the first query, wherein the score is determined based on the expected relevance of the at least one term to users of the first client device other than the first user, and the expected relevance is determined based on the degree to which the use of items associated with the at least one term is shared among multiple users; Based on (i) the frequency of the interaction between the second user and the first client device meeting a threshold and (ii) the score of at least one term in the first query, it is determined that the first query is related to the second user; In response to determining that the first query is related to the second user, the first automated assistant provides the second user of the first client device with an option to join the query session; In response to receiving an indication from the second user to accept the optional option to join the query session, the second user is added to the query session; Receive additional input from the second user to refine the first query; A set of modified search results is generated based on the additional input received from the second user; and The first automated assistant provides the modified set of search results to the first user and the second user.

Images