Method and system for presenting privacy-friendly query activity based on environmental signals

Abstract

Implementations are directed to receiving a search query from a user, obtaining an environmental signal(s) associated with an environment in which the user is located when the search query is received, processing the environmental signal(s) to generate a privacy measure associated with the submission of the search query, obtaining additional environmental signal(s) associated with the environment in which the user is located when a user input is received for a search interface, processing the additional environmental signal(s) to generate an additional privacy measure associated with the user input, selecting a subset of historical search queries from a superset of historical search queries of the user based at least on the privacy measure and the additional privacy measure, and causing presentation of the subset of historical search queries to the user in response to receiving the user input for the search interface.

Description

Background Technology

[0001] Users can submit search queries via various client devices, such as smartphones, tablets, wearable devices, automotive systems, and standalone personal assistant devices. Furthermore, search queries can be submitted using various input modalities (e.g., verbal, touch, and / or typing). Search results in response to the corresponding search query can be provided as output at the corresponding client device (e.g., visually and / or audibly).

[0002] Users can interact with the search interface to submit search queries. Previously submitted search queries are typically presented as suggestions on the search interface before the actual search query is received, or as autocomplete suggestions based on the user's input. However, for privacy reasons, users may wish for some historical search queries not to be presented on the search interface, depending on their environment when interacting with it. To ensure privacy, users can selectively enter private browsing mode or anonymous mode to submit search queries. However, in these modes, historical search queries may never be presented as suggestions, and users may need to provide more input to resubmit one of their historical search queries. Summary of the Invention

[0003] The embodiments disclosed herein relate to generating a privacy metric associated with user input received at a user's client device, and adjusting the presentation of historical search queries based on this privacy metric. The privacy metric represents the level of privacy in the environment in which the user on the client device is located when user input is received for the search interface. The privacy metric can be determined based on processing one or more environmental signals associated with the environment in which the user on the client device is located when the user input is received. The one or more environmental signals can include, for example, location information corresponding to the user's location when the search query is received, audio data capturing ambient noise in the environment when the search query is received, and / or visual data capturing the environment when the search query is received. The privacy metric can be additionally or alternatively determined based on one or more terms of the received search query. Furthermore, the privacy metric associated with user input for the search interface can be compared with multiple corresponding privacy metrics associated with the user's historical search queries to determine whether historical search queries should be presented to the user.

[0004] For example, suppose a user on a client device submits a search query for "symptoms of infectious disease #19" at the client device's search interface. Further suppose that when the user submits the search query, one or more environmental signals indicate that the user is at home and there is no one else at home. In this example, the privacy measure indicating the user's environment at the time the search query was submitted could be highly private. Additionally or alternatively, the privacy measure could indicate that the search query could be highly private based on one or more terms in the search query including terms related to medical conditions (e.g., a request for disease symptoms). Furthermore, suppose the user subsequently commutes to work by train, and user input to the client device's search interface is detected. Further suppose that when the user submits the search query, one or more environmental signals indicate that the user is commuting to work and there are other people on the train. In this example, the additional privacy measure indicating the user's environment at the time the search query was submitted is public. As a result, a subset of historical search queries presented to the user in response to received user input to the search interface can exclude the search query for "symptoms of infectious disease #19," and can exclude any other historical search queries associated with the corresponding privacy measure indicating that the user's environment at the time the historical search query was submitted might be private.

[0005] In various implementations, it is possible to use corresponding machine learning (ML) models to process one or more environmental signals, and to generate privacy metrics based on the outputs generated across the ML models. For example, in an implementation where one or more environmental signals include location information corresponding to a user's location when a search query is received, it is possible to use multiple classification models to process the location information to generate outputs associated with one or more types of environments (e.g., public environments, semi-public environments, privacy environments, and / or other types of environments). Additionally or alternatively, in an implementation where one or more environmental signals include audio data capturing ambient noise in the environment when a search query is received, it is possible to use multiple acoustic models to process the audio data to generate outputs associated with one or more acoustic properties of the environment (e.g., the noise level of the environment, the classification of ambient noise detected in the environment, and / or other acoustic properties of the environment). Additionally or alternatively, in embodiments where one or more environmental signals include capturing visual data of the environment when a search query is received, it is possible to use multiple visual models to process the visual data to generate outputs associated with one or more visual attributes of the environment (e.g., indications of the presence of additional users in the environment, indications of objects present in the environment, and / or other visual attributes of the environment).

[0006] In these implementations, it is possible to process the output generated across multiple ML models to generate a privacy metric associated with the submission of a search query. The privacy metric can include, for example, the type of environment in which the user of the client device is located when the search query is received (e.g., public, semi-public, private, and / or other types of environment); a score or measure of likelihood (e.g., probabilities such as probabilities, where a probability closer to 0.0 corresponds to a more public environment and a probability closer to 1.0 corresponds to a more private environment) representing the level of privacy in which the user of the client device is located when the search query is received; a binary value representing the level of privacy in which the user of the client device is located when the search query is received (e.g., where a value of 0 corresponds to a public environment and a value of 1 corresponds to a private environment); and / or other representations indicating the level of privacy in which the user of the client device is located when the search query is received. Furthermore, the privacy metric can be based on the output generated across multiple ML models.

[0007] For example, in an implementation where the output generated across multiple ML models includes the type of environment in which the user of the client device is located when a search query is received, suppose the location information indicates the user is at home and the audio data excludes any noise, but the visual data indicates the presence of three other people in that environment. In this example, the privacy metric could correspond to a semi-public environment based on the presence of other people, rather than a private environment. As another example, in an implementation where the output generated across multiple ML models includes a probability representing the level of privacy in the environment in which the user of the client device is located when a search query is received, suppose the location information indicates the user is at home and the audio data excludes any noise, but the visual data indicates the presence of three other people in that environment. In this example, the privacy metric could correspond to a probability of 0.6 to indicate that the environment is very likely private (e.g., based on location information and audio data), but may not be highly private (e.g., based on the presence of other people in that environment). As yet another example, in an implementation where the output generated across multiple ML models includes binary values ​​representing the privacy level of the environment in which a user on a client device is located when a search query is received, it is assumed that location information indicates the user is at home and audio data does not contain any noise (e.g., both are associated with binary values ​​representing a private environment), but visual data indicates the presence of three other people in the environment (e.g., associated with binary values ​​representing a public environment). In this example, the privacy metric could correspond to a binary value representing a public environment based on the presence of other people in the environment.

[0008] In various implementations, when user input is subsequently received for the search interface of a user's client device, corresponding environmental signals associated with the reception of the user input can be obtained and processed to generate an additional privacy metric associated with the reception of the user input. The additional privacy metric associated with the reception of the user input can be compared with a privacy metric associated with the submission of the search query to determine whether the additional privacy metric matches the privacy metric. Furthermore, a subset of historical search queries presented to the user can be selected from a superset of historical search queries based on whether the additional privacy metric matches the privacy metric (e.g., based on type matching of the environment and / or based on the additional privacy metric meeting a privacy level threshold determined based on a privacy level). For example, if a search query was submitted in a privacy environment (e.g., as indicated by the privacy metric) and the user input was received in a public environment (e.g., as indicated by the additional privacy metric), the subset of historical search queries presented to the user can be restricted to those also received in a public environment, such that the search query is excluded from the subset of historical search queries presented to the user. However, in various implementations, if an additional instance of a search query previously received in a private environment is received in a public environment, that search query may subsequently be included in a subset of the historical search queries presented to the user.

[0009] In some implementations, historical search queries presented to the user can be displayed as query suggestions on the search interface. For example, assuming an additional privacy metric matches the privacy metric, a subset of historical search queries can be presented as query suggestions when the search interface is accessed, and when such a query suggestion is selected, the search engine executes a search query associated with the selected query suggestion. In some additional or alternative implementations, historical search queries presented to the user can be displayed on the search interface as autocomplete suggestions for partial queries entered. For example, assuming an additional privacy metric matches the privacy metric, when the partial query "Sym" is entered, a subset of historical search queries can be used to generate the autocomplete query suggestion "ptoms of Infectious Disease #19", and when such a query suggestion is selected, the search engine executes the search query "Symptoms of Infectious Disease #19 (symptoms of infectious disease #19)".

[0010] In various implementations, users can interact with a superset of historical search queries. In some implementations, users can edit the privacy metrics associated with historical search queries. For example, a user can change the corresponding privacy metric from being associated with a private environment to being associated with a public environment, such that when it is determined that the user is in a public environment, the corresponding search query can be included in a subset of the historical search queries presented to the user. As another example, a user can change the corresponding privacy metric from being associated with a public environment to being associated with a private environment, such that when it is determined that the user is in a public environment, the corresponding search query can be excluded from the subset of the historical search queries presented to the user. In some additional or alternative implementations, users can remove historical search queries from the superset of historical search queries based on the privacy metrics associated with them. For example, a user can specify that historical search queries associated with the corresponding privacy metric associated with a private environment should be removed from the superset of historical search queries.

[0011] By adapting the presentation of historical search queries based on privacy metrics using the techniques described herein, user privacy can be protected while reducing the amount of user input required to submit additional instances of previously submitted search queries. As a result, computational and network resources can be saved. For example, by adapting the presentation of historical search queries based on privacy metrics using the techniques described herein, those historical search queries that might be resubmitted in the user's current context can be presented. The user can then select a given one of those historical search queries to resubmit without having to retype or provide verbal input including that given historical search query, thereby reducing the amount of user input received and / or processed at the client device.

[0012] The above description is merely an overview of some embodiments disclosed herein. These embodiments, as well as others, will be described in detail herein. Attached Figure Description

[0013] Figure 1 A block diagram depicts an example environment that illustrates various aspects of this disclosure and enables the implementation of the embodiments disclosed herein.

[0014] Figure 2 A flowchart is depicted illustrating example methods for generating privacy metrics associated with the submission of search queries according to various implementations, and providing a subset of historical search queries based on the generated privacy metrics.

[0015] Figure 3 A flowchart is depicted illustrating an example method that shows a privacy metric associated with the submission of a search query according to various implementations, and provides a subset of historical search queries based on the modified privacy metric.

[0016] Figure 4A , Figure 4B and Figure 4C Various non-limiting examples are described, according to various implementations, of user interfaces associated with generating privacy metrics related to the submission of search queries in a privacy environment and providing a subset of historical search queries based on the generated privacy metrics.

[0017] Figure 5A and Figure 5B Various non-limiting examples are described, according to various implementations, of a user interface associated with generating a privacy metric related to the submission of a search query in a public environment and providing a subset of historical search queries based on the generated privacy metric.

[0018] Figure 6 Example architectures of computing devices according to various implementation methods are depicted. Detailed Implementation

[0019] Now go to Figure 1 A block diagram depicts an example environment illustrating various aspects of this disclosure and in which embodiments disclosed herein may be implemented. Client device 110 in... Figure 1 As shown in the figure, and in various embodiments including a user input engine 111, a rendering engine 112, one or more sensors 120 and a search engine 130A1.

[0020] User input engine 111 is capable of detecting various types of user input at client device 110. User input detected at client device 110 can include verbal input detected via microphone(s) of client device 110, touch input detected via user interface input device(s) of client device 110 (e.g., touchscreen), and / or typed input detected via user interface input device(s) of client device 110 (e.g., via a virtual keyboard on the touchscreen). For example, user input detected by user input engine 111 can include touch input for a search interface, typed input including one or more terms of a search query, verbal input including one or more terms of a search query, and / or any other input for client device 110 described herein.

[0021] In various implementations, when user input is detected at client device 110 via user input engine 111, one or more environmental signals generated by one or more sensors 120 of client device 110 can be obtained (and these environmental signals can optionally be stored in an environmental signal database 120A associated with the user input). The one or more sensors 120 of client device 110 can include, for example, multiple GPS sensors, multiple microphones, multiple vision components, and / or other sensors. The one or more environmental signals generated by the one or more sensors 120 of client device 110 can include, for example, location information generated by multiple GPS sensors of client device 110, audio data generated by multiple microphones of client device 110, visual data generated by multiple vision components of client device 110, and / or other environmental signals generated by other sensors of client device 110. The one or more environmental signals can be processed to generate multiple privacy metrics associated with the user input detected at client device 110 via user input engine 111 (e.g., as described with respect to privacy metric engine 140).

[0022] In various implementations, user input detected at client device 110 via user input engine 111 can be directed to a search interface. User input directed to the search interface can include, for example, accessing the search interface, searching for one or more terms of a query via touch input, typing input, or verbal input, and / or interacting with search results presented at the search interface. In some implementations, and before any terms of a search query are received, previously submitted historical search queries by the user of client device 110 can be rendered as query suggestions at the search interface of client device 110 (e.g., via rendering engine 112 and as described with respect to query suggestion engine 160). In some additional or alternative implementations, and in response to receiving one or more terms of a search query at the search interface, previously submitted historical search queries by the user of client device 110 can be used to generate autocomplete suggestions for portions of the query entered at the search interface, and can be rendered at the search interface of client device 110 (e.g., via rendering engine 112 and as described with respect to query suggestion engine 160).

[0023] In some implementations, search queries received at client device 110 can be processed locally on client device 110. For example, client device 110 can use search engine 130A1 to locally process search queries against multiple databases on client device 110, such as email databases, calendar databases, document databases, note databases, contact databases, and / or other databases. In some additional or alternative implementations, search queries received at client device 110 can be transmitted to a remote system (e.g., multiple servers) communicating with client device 110 via one or more networks 199 (e.g., Wi-Fi, Bluetooth, Near Field Communication (NFC), Local Area Network (LAN), Wide Area Network (WAN), Ethernet, the Internet, and / or any combination of other networks) for remote processing. For example, client device 110 can transmit search queries to a remote system that uses search engine 130A2 to process the search queries. Any search queries received at client device 110 can be stored in query activity database 130A. The query activity database 130A can include a superset of historical search queries received at client device 110 and / or other client devices associated with the user of the client device.

[0024] In various implementations, when user input detected at client device 110 is directed to a search interface, client device 110 can utilize privacy search system 180 to determine a subset of historical search queries to be presented to the user via client device 110 (e.g., as query suggestions before any query terms are received and / or as autocomplete suggestions for parts of the query). Privacy search system 180 in Figure 1 As shown, and in various embodiments, it includes a search engine 130A2, a privacy measurement engine 140, a privacy measurement comparison engine 150, a query suggestion engine 160, and a privacy measurement modification engine 170. In various embodiments, the privacy measurement engine 140 includes a location engine 141, an audio engine 142, and a visual engine 143. Although the privacy search system 180... Figure 1 The system is described as a remote system communicating with client device 110 via one or more networks; however, it should be understood that this is for illustrative purposes and is not intended to be restrictive. For example, an instance of privacy search system 180 could be implemented locally on one or more client devices of the user (e.g., client device 110 and / or additional client devices of the user).

[0025] Privacy metric engine 140 is capable of processing one or more environmental signals to generate multiple privacy metrics associated with user input detected at client device 110 via user input engine 111. The multiple privacy metrics can represent the privacy level of the environment in which the user of client device 110 is located when user input is received. The multiple privacy metrics can include, for example, the type of environment in which the user of client device 110 is located when user input is received (e.g., public environment, semi-public environment, private environment, and / or other types of environment), a probability representing the privacy level of the environment in which the user of client device 110 is located when user input is received (e.g., where a probability closer to 0.0 corresponds to a more public environment, and a probability closer to 1.0 corresponds to a more private environment), a binary value representing the privacy level of the environment in which the user of client device 110 is located when user input is received (e.g., where a value of 0 corresponds to a public environment, and a value of 1 corresponds to a private environment), and / or other representations indicating the privacy level of the environment in which the user of client device 110 is located when user input is received.

[0026] In some implementations, the privacy search system 180 enables the privacy metric engine 140 to process one or more environmental signals using one or more machine learning (ML) models stored in a database of ML models 140A. The ML models stored in the database of ML models 140A can be trained using, for example, supervised training techniques. For instance, multiple training instances can be used to train the ML models, each including a training instance input and a corresponding training instance output. The training instance input can include one or more training environmental signals associated with the training search query, and the corresponding training instance output can include one or more ground truth labels associated with the privacy level of the training search query. Notably, multiple ML models can be trained to process different environmental signals.

[0027] In embodiments where one or more environmental signals include location information generated by multiple GPS sensors of client device 110 upon receiving user input, location engine 141 is capable of using multiple classification models to process the location information to generate outputs associated with one or more types of environments (e.g., public environments, semi-public environments, privacy environments, and / or other types of environments). To train the multiple classification models, training instance inputs can each include location information, and corresponding training instance outputs can include ground truth labels corresponding to the type of environment at the location represented by the location information.

[0028] In some implementations, the output generated across multiple classification models can be a label corresponding to an environment type predicted to correspond to the type of environment in which the user is located when the user input is received. For example, if the location information corresponds to the residential address of a user of client device 110, the output generated across multiple classification models based on the location information could be a "privacy" label. As another example, if the location information corresponds to a coffee shop, train station, or other public location, the output generated across multiple classification models based on the location information could be a "public" label. As yet another example, if the location information corresponds to the residential address of a user who is not the residential address of a user of client device 110, the output generated across multiple classification models based on the location information could be a "semi-public" label (or a "semi-privacy" label).

[0029] In additional or alternative implementations, the output generated across the classification model(s) may be one or more labels corresponding to an environment type predicted to correspond to the type of environment in which the user is located when the user input is received, and a corresponding metric (e.g., binary value, probability, log-likelihood, and / or other metric) associated with each of the one or more labels. For example, if the location information corresponds to the residential address of a user of client device 110, the output generated across the classification model(s) based on processing the location information may be a "privacy" label with an association probability of 0.9 and a "semi-public" label with an association probability of 0.1, or a "privacy" label with an association value of 1.0 and a "semi-public" label with an association value of 0.0 and a "public" label. As another example, if the location information corresponds to a coffee shop, train station, or other public location, the output generated across the classification model(s) based on processing the location information may be a "public" label with an association probability of 0.7 and a "semi-public" label with an association probability of 0.3, or a "public" label with an association value of 1.0 and a "semi-public" label with an association value of 0.0 and a "privacy" label. As another example, if the location information corresponds to the residential address of a user who is not the residential address of the user of client device 110, then the output generated based on the location information across (multiple) classification models can be a “semi-public” label (or “semi-private” label) with an association probability of 0.8 and a “private” label with an association probability of 0.2, or a “semi-public” label with an association value of 1.0 and a “public” label and a “private” label with an association value of 0.0.

[0030] In embodiments where one or more environmental signals additionally or alternatively include audio data generated by the microphone(s) of client device 110 upon receiving user input, audio engine 142 may use (multiple) acoustic models(s) to process the audio data to generate outputs associated with one or more acoustic properties of the environment (e.g., the noise level of the environment, the classification of ambient noise detected in the environment, and / or other acoustic properties of the environment). To train (multiple) acoustic models(s), training instance inputs may each include audio data, and the corresponding training instance outputs may include ground truth labels corresponding to the noise level captured in the audio data and / or the type of noise included in the audio data (e.g., people talking, vehicle movement, a television program or movie playing in the background, and / or other types of noise).

[0031] In some implementations, the output generated across multiple acoustic models may be a value corresponding to the noise level (e.g., decibel level) of the user's environment when the user input is received and / or one or more labels corresponding to the type of noise detected in the user's environment when the user input is received. For example, if the audio data corresponds to audio data captured when a user of client device 110 is commuting to work by bus, the output generated across multiple acoustic models based on processing the audio data may be a "commuting" and / or "bus" label based on noise generated by the bus, and / or a decibel level detected based on bus movement (e.g., 90 dB, etc.). As another example, if the audio data corresponds to audio data captured when a user of client device 110 is in a coffee shop, the output generated across multiple acoustic models based on processing the audio data may be a "conversation" label based on other people talking in the coffee shop and / or a "music" label if music is playing in the coffee shop, and a decibel level detected based on the people talking or the music (e.g., 60 dB, 70 dB, etc.). As yet another example, if the audio data corresponds to audio data captured when the user of client device 110 is at home, the output generated based on processing the audio data across (multiple) acoustic models could be a “TV show” or “movie” label based on whether a TV show or movie is playing in the background, and based on the decibel level detected by the TV show or movie (e.g., 40 dB, etc.).

[0032] In additional or alternative implementations, the output generated across the acoustic models(s) may be one or more labels corresponding to the type of noise detected in the user's environment when user input is received, and a corresponding metric (e.g., binary value, probability, log-likelihood, and / or other metric) associated with each of the one or more labels. For example, if the audio data corresponds to audio data captured when the user of client device 110 commutes to work by bus, the output generated across the acoustic models(s) based on processing the audio data may be a "bus" label with an association probability of 0.6 and a "car" label with an association probability of 0.4, or a "bus" label with an association value of 1.0 and a "car" label with an association value of 0.0. As another example, if the audio data corresponds to audio data captured when the user of client device 110 is in a coffee shop, the output generated across the acoustic models(s) based on processing the audio data may be a "talking" label with an association probability or a value of 1.0 assuming others are talking in the coffee shop. As yet another example, if the audio data corresponds to audio data captured when the user of client device 110 is at home, the output generated based on processing the audio data across (multiple) acoustic models could be a “TV show” or “movie” label with an associated probability or a value of 1.0 based on whether a TV show or movie is playing in the background.

[0033] In embodiments where one or more environmental signals additionally or alternatively include visual data generated by the visual components of the client device 110 upon receiving user input, the visual engine 143 is capable of using multiple visual models to process the visual data to generate outputs associated with one or more visual attributes of the environment (e.g., indications of the presence of additional users in the environment, indications of objects present in the environment, and / or other visual attributes of the environment). To train the multiple visual models, training instance inputs can each include visual data, and the corresponding training instance outputs can include ground truth labels corresponding to the presence of other people in the environment (and optionally, the identities of those people are available) and / or classifications of objects included in the environment (e.g., coffee tables, train seats, bookshelves, and / or any other objects that may exist in the environment).

[0034] In some implementations, the output generated across multiple visual models may be one or more labels corresponding to whether other people are in the environment when user input is received (and optionally, the identities of those other people) and / or the type of objects in the environment when user input is received. For example, if the visual data corresponds to visual data captured when a user of client device 110 is commuting to work by bus, the output generated across multiple visual models based on the processed visual data may be a “bus seat” and / or “other people” label (if other people are present in the environment). As another example, if the visual data corresponds to visual data captured when a user of client device 110 is in a coffee shop, the output generated across multiple visual models based on the processed visual data may be a “other people” label (if other people are present in the coffee shop and are captured in the visual data), and / or a “coffee cup” label (if a coffee cup is captured in the visual data), or a “table” or “booth” label (if a table or booth is captured in the visual data). As yet another example, if the visual data corresponds to visual data captured when the user of client device 110 is at home, the output generated based on processing the visual data across visual models(s) could be a “sofa” label (if a sofa is captured in the visual data), and / or a “spouse” label (if the user’s spouse is captured in the visual data and the spouse’s visual embedding is available).

[0035] In additional or alternative implementations, the output generated across the visual models(s) may be one or more labels corresponding to whether other people are in the environment when user input is received (and optionally, the identities of those other people) and / or the type of object in the environment when user input is received, and a corresponding metric (e.g., binary value, probability, log-likelihood, and / or other metric) associated with each of the one or more labels. For example, if the visual data corresponds to visual data captured when a user of client device 110 commutes to work by bus, the output generated across the visual models(s) based on the processed audio data may be a “bus seat” label with an association probability of 0.6 and a “chair” label with an association probability of 0.4, or a “other people” label with an association value of 1.0 if the visual data predicts the presence of other people in the environment. As another example, if the visual data corresponds to visual data captured when a user of client device 110 is in a coffee shop, the output generated across the visual models(s) based on the processed visual data may be a “coffee cup” label with an association probability of 0.6, a “thermos bottle” label with an association probability of 0.4, or a “coffee cup” label with a value of 1.0. As another example, if the visual data corresponds to visual data captured when the user of client device 110 is at home, the output generated based on processing audio data across visual models(s) could be a label “sofa” with an association probability of 0.8, a label “coffee table” with an association probability of 0 / 75, a label “other people” with an association probability of 0.9 (or a label “spouse” with an association probability of 0.9).

[0036] Although the location engine 141, audio engine 142, and vision engine 143 are described herein with respect to specific examples having specific labels and associated metrics for those labels, it should be understood that this is for illustrative purposes and not intended to be limiting. For example, other labels exist, and any metrics associated with those labels can be utilized. For instance, the labels generated across (multiple) ML models can vary as (multiple) ML models are trained to predict sounds and / or objects and / or sounds and / or objects present in the user's environment when user input is received.

[0037] Privacy metric engine 140 is capable of processing the output generated by one or more of the location engine 141, audio engine 142, or vision engine 143 to generate privacy metrics associated with the reception of user input. Privacy metric engine 140 is capable of processing the output generated by one or more of the location engine 141, audio engine 142, or vision engine 143 using multiple ML models or multiple rules (e.g., multiple ML rules or multiple heuristically defined rules) stored in multiple ML model databases 140A. To train the multiple ML models for processing the output, training instance inputs can each include one or more outputs generated by one or more of the location engine 141, audio engine 142, or vision engine 143, and the corresponding training instance outputs can include ground truth privacy metrics associated with the environment in which the user input was received, generating outputs for that user input. Privacy metrics generated by privacy metric engine 140 can be stored in multiple privacy metric databases 140B. Furthermore, privacy metrics generated by privacy metric engine 140 can be optionally stored in privacy metric database(s) 140B in association with any subsequent queries received at client device 110, or mapped to subsequent queries stored in query activity database 130A.

[0038] For example, suppose the output generated by location engine 141 indicates that the user of client device 110 is at a restaurant indicated by the “Public” label with an association probability of 0.9. Further suppose that additional or alternative output generated by audio engine 142 indicates that other users are present in the environment of the user of client device 110, such as indicated by a relatively high noise level (e.g., 80 dB) and an association probability of 0.8 with the “Talking” label. Further suppose that additional or alternative output generated by vision engine 143 indicates that other users are present in the environment of the user of client device 110, such as indicated by the “Others” label with an association probability also of 0.8. In this example, privacy metric engine 140 is capable of processing these outputs using multiple ML models or multiple rules to generate a privacy metric of 0.75 associated with the “Public” label and / or a privacy metric of 0.25 associated with the “Privacy” label based on processing these outputs from one or more of the location engine 141, audio engine 142, or vision engine 143. In other words, the privacy metrics generated based on these outputs in this example, associated with the receipt of user input, indicate that the user is in a public environment, and any search query submitted by the user of client device 110 in that environment is unlikely to be considered highly private by the user. Therefore, as described below regarding the privacy metric comparison engine and query suggestion engine 160, historical search queries associated with privacy metrics indicating that the user was in a more private environment than when the user input was received may not be presented to the user as query suggestions and / or autocomplete suggestions for parts of the query.

[0039] As another example, suppose the output generated using location engine 141 indicates that the user of client device 110 is in their home, as indicated by the “privacy” label with an association probability of 1.0. Further suppose that the additional or alternative output generated using audio engine 142 indicates that no other people are present in the environment, as indicated by a relatively low sound level (e.g., 20 dB), and no other noise is detected. Further suppose that the additional or alternative output generated using visual engine 143 indicates that no other users are present in the user's environment (e.g., only the user of client device 110). In this example, privacy metric engine 140 is able to process these outputs using (multiple) ML models or (multiple) rules to generate a privacy metric of 0.95 associated with the “privacy” label and / or 0.05 associated with the “public” label. In other words, in this example, the privacy metrics generated based on these outputs and associated with the reception of user input indicate that the user is in a private environment, and any search query submitted by the user of client device 110 in that environment can be considered highly private by the user. Therefore, as described below regarding the privacy metric comparison engine and query suggestion engine 160, any historical search query can be presented to the user as a query suggestion and / or an autocomplete suggestion for parts of the query, regardless of the environment in which it is received (e.g., public, semi-public, or private environment).

[0040] The privacy metric comparison engine 150 is capable of comparing a privacy metric associated with user input with multiple additional privacy metrics stored in the privacy metric database(s) 140B that are associated with historical search queries. Based on the comparison of the privacy metric with the multiple additional privacy metrics, the privacy metric comparison engine 150 is capable of determining whether the privacy metric matches any of the multiple additional privacy metrics. In implementations where the privacy metric corresponds to a tag (e.g., public, semi-public, private, and / or any other tag optionally defined at different levels of granularity, such as highly public, moderately public, highly private, moderately private, etc.), the privacy metric comparison engine 150 is capable of identifying those additional privacy metrics that match the privacy metric, as well as the historical search queries associated with those additional privacy metrics. For example, suppose the privacy metric corresponds to a privacy tag. In this example, the privacy metric comparison engine 150 is capable of identifying those additional privacy metrics with any tag and the historical search queries associated with them, because the privacy metric indicates a privacy environment and the user of client device 110 can submit any search query regardless of the environment in which it was initially submitted. As another example, suppose the privacy metric corresponds to a public tag. In this example, the privacy metric comparison engine 150 is able to identify additional privacy metrics that are limited to those with public labels, as well as the historical search queries associated with them, because the privacy metrics indicate a public environment and the user of the client device 110 can only submit search queries that were previously submitted in a public environment.

[0041] In implementations where privacy metrics correspond to associated values ​​(e.g., probability, log-likelihood, binary values, etc.), the privacy metric comparison engine 150 is capable of identifying additional privacy metrics that have associated values ​​that satisfy a threshold privacy metric level determined based on the privacy metric associated with the user input. For example, suppose the privacy metric corresponds to a probability of 0.35 associated with a privacy tag (and an implicit probability of 0.65 associated with a public tag). In this example, the threshold privacy metric level could be a privacy threshold level of 0.35 or a public threshold level of 0.65. The privacy metric comparison engine 150 is capable of identifying those additional privacy metrics associated with a privacy threshold level with a probability below 0.35 or a public threshold level with a probability above 0.65 as satisfying the threshold privacy metric level. Furthermore, the privacy metric comparison engine 150 is capable of identifying historical search queries associated with those privacy metrics that satisfy the threshold privacy metric level. As a result, in this example, the identified historical search queries include those submitted by the user of client device 110 in similar or more public environments that the user might not consider private. As another example, suppose the privacy metric corresponds to a probability of 0.95 associated with a privacy label (and an implicit probability of 0.05 associated with a public label). In this example, the threshold privacy metric level can be either a privacy threshold level of 0.95 or a public threshold level of 0.05. The privacy metric comparison engine 150 is able to identify those additional privacy metrics associated with a privacy threshold level with a probability below 0.95 or a public threshold level with a probability above 0.05 as meeting the threshold privacy metric level. Furthermore, the privacy metric comparison engine 150 is able to identify historical search queries associated with those privacy metrics that meet the threshold privacy metric level. As a result, in this example, the identified historical search queries include those submitted by the user of client device 110 in similar or more public environments that the user might consider private.

[0042] The query suggestion engine 160 is capable of generating query suggestions and / or autocomplete query suggestions for portions of the query entered at the search interface of the client device 110, utilizing historical search queries associated with additional privacy metrics identified by the privacy metric comparison engine 150. The query suggestions and / or autocomplete query suggestions can be visually presented to the user via the display of the client device 110, and / or audibly presented to the user via the speaker(s)(s) of the client device 110 (e.g., using the rendering engine 112). In some embodiments, the query suggestion engine 160 is capable of selecting a subset of historical search queries from a superset of identified historical search queries to present as query suggestions to the user of the client device 110. Query suggestions can be presented at the search interface of the client device 110 before and / or while the user is entering any terms of the search query. For example, a subset of historical search queries can be selected to present a predefined number of historical search queries to the user (e.g., three, four, six, and / or any other number), and can optionally present selectable elements that, when selected, present additional historical search queries to the user via the search interface, or present an additional subset of historical search queries to the user via the search interface. In some versions of those implementations, a subset of historical search queries can optionally be presented along with other query suggestions, such as popular or active search queries that are popular or active among multiple users (and optionally limited to users in the same country, region, or city as the user of client device 110). In some additional or alternative implementations, historical search queries can be used to generate autocomplete suggestions for portions of a query entered at the search interface of client device 110. For example, suppose the user previously submitted the search query “Symptomsof Infectious Disease#19”. Further suppose the user has already entered the partial query “Sym” at the search interface of client device 110. In this example, query suggestion engine 160 enables the presentation of autocomplete suggestions such as "ptoms of Infectious Disease #19" and "phony tickets" to the user at the search interface of client device 110. The following describes providing query suggestions based on privacy metrics and (multiple) additional privacy metrics (e.g., see reference). Figures 4A to 4C and Figures 5A to 5B It is worth noting that the historical search queries presented to the user may be limited to those search queries associated with additional privacy metrics that match the privacy metrics associated with the user's input.

[0043] In some implementations, the privacy metric modification engine 170 can modify previously generated privacy metrics for the historical search queries based on one or more environmental signals obtained when additional instances of the historical search queries are received at client device 110. For example, suppose the search query “Score of the Louisville game” has been received five times previously. Further suppose that each time the search query was previously submitted, the user of client device 110 was alone at home, causing the search query to be associated with a privacy metric indicating that the search query is private. As a result, the search query “Score of the Louisville game” can be excluded from a subset of the historical search queries presented to the user. However, further suppose that an additional instance of the search query “Score of the Louisville game” is received when the user of client device 110 is in a noisy bar. The resulting privacy metric generated based on one or more environmental signals obtained when the additional instance of the search query is received indicates that the search query is not private, and the privacy metric associated with the search query can be updated in the privacy metric database 140B. As a result, the search query “Score of the Louisville game” can subsequently be included in a subset of the historical search queries presented to the user.

[0044] Although client device 110 and privacy search system 180 are described in this document as including specific engines, such as Figure 1The engines shown are not included in this description; however, it should be understood that client device 110 and / or privacy search system 180 may include additional or alternative engines. For example, client device 110 and / or privacy search system 180 may include multiple speech recognition engines, multiple natural language understanding (NLU) engines, multiple speech synthesis engines, and / or multiple other engines. For example, multiple speech recognition engines may use multiple speech recognition models to process audio data capturing spoken input from a user of client device 110 to generate recognized text corresponding to the spoken input; multiple NLU engines may use multiple NLU models to process the recognized text generated by multiple speech recognition engines to determine multiple intentions included in the spoken input; and multiple speech synthesis engines may use multiple speech synthesis models to generate synthetic speech audio data, which includes synthesized speech presented via multiple speakers of client device 110 (e.g., using rendering engine 112) in response to spoken input and / or any other input received at client device 110. In addition, the client device 110 and / or the privacy search system 180 may additionally or alternatively include instances of automated assistants capable of engaging in human-computer dialogue with the user of the client device 110.

[0045] In embodiments where client device 110 and / or privacy search system 180 include multiple NLU engines, multiple classification models can be used to process multiple intents determined by the multiple NLU engines (e.g., based on multiple terms of a search query typed by a user and / or multiple identified terms of a search query included in spoken input) to generate output associated with one or more types of search queries. The one or more types of search queries can be defined at different levels of granularity and can include, for example, medical search queries, sports search queries, travel search queries, restaurant search queries, and / or search queries of any other category. Furthermore, each of the one or more types of queries can be mapped to one or more types of environments. For example, any medical search query can be mapped to a privacy environment because the user of client device 110 may consider medical search queries to be privacy, while any sports search query, travel search query, and restaurant search query can be mapped to a privacy environment because the user of client device 110 may not consider these search queries to be privacy. In addition to or in place of one or more environmental signals, the privacy metric engine 140 is able to take into account one or more search query types to which a search query is classified when generating a privacy metric.

[0046] For example, suppose privacy metric engine 140 generates a privacy metric for a given search query based on one or more environmental signals, indicating that the user might not consider the search query to be a private query (e.g., when the user of client device 110 receives the search query while commuting to work by train). However, if the search query is a query type considered private (e.g., a medical search query), privacy metric engine 140 can generate a privacy metric associated with the submission of the search query, indicating that the user of client device 110 considers the search query to be private because it is a query type generally considered private. Conversely, suppose privacy metric engine 140 generates a privacy metric for a given search query, indicating that the user can consider the search query to be a private query based on one or more environmental signals (e.g., when the user receives the search query while alone at home). However, if the search query is a query type considered public (e.g., a sports query), privacy metric engine 140 can generate a privacy metric associated with the submission of the search query, indicating that the user of client device 110 does not consider the search query to be private because it is a query type generally considered public.

[0047] Furthermore, although described for a single client device Figure 1 However, it should be understood that this is for illustrative purposes only and not intended to be limiting. For example, one or more additional client devices of the user may also communicate with the privacy search system 180 (or additional instances implementing the privacy search system 180) and / or with the client device 110 via one or more networks 199. For example, an initial search query may be received and processed at the client device 110, but user input for the search interface may be received at one or more additional client devices of the user, and the privacy search system 180 may be used to limit the subset of historical search queries presented at one or more additional client devices of the user.

[0048] By adapting the presentation of historical search queries based on privacy metrics using the techniques described herein, user privacy can be protected while reducing the amount of user input required to submit additional instances of previously submitted search queries. As a result, computational and network resources can be saved. For example, by adapting the presentation of historical search queries based on privacy metrics using the techniques described herein, those historical search queries that might be resubmitted in the user's current context can be presented. The user can then select a given one of those historical search queries to resubmit without having to retype or provide verbal input including a given one of those historical search queries, thereby reducing the amount of user input received and / or processed at the client device.

[0049] Turn now Figure 2A flowchart illustrating an example method 200 is provided, showing the generation of a privacy metric associated with a search query submission and the provision of a subset of historical search queries based on the generated privacy metric. For convenience, the operation of method 200 is described with reference to a system performing the operation. This system of method 200 includes (e.g., multiple) computing devices (e.g., Figure 1 , Figures 4A to 4C , Figures 5A to 5B Client device 110 and / or Figure 6 The computing device 610, one or more servers and / or other computing devices, and one or more processors and / or other (multiple) components. Furthermore, although the operations of method 200 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, and / or added.

[0050] At box 252, the system receives a search query via a user's client device. Search queries can be received at the search interface on the client device using one or more of the following input methods: typing, touch, or verbal input. The search interface can be displayed via a graphical user interface on the client device. In some implementations, the search interface may be associated with a software application accessible on the client device, such as a browser-based application, an automation assistant application, a contact application, a navigation application, a calendar application, an email application, a task or reminder application, and / or any other application accessible on the client device that includes search functionality. In some additional or alternative applications, the search interface can be a web browser, the home screen of a mobile device, and / or any other interface capable of receiving search queries.

[0051] At box 254, the system acquires one or more environmental signals associated with the user's environment at the time the search query is received. The one or more environmental signals associated with the environment can be generated by multiple sensors of the client device and can include, for example, location information generated by multiple GPS sensors of the client device, audio data generated by multiple microphones of the client device, visual data generated by multiple vision components of the client device, and / or one or more other environmental signals that can be captured by multiple sensors of the client device. One or more environmental signals can be captured within a threshold time range relative to the time the search query is received (e.g., several seconds, milliseconds, and / or other durations before and after the search query is received).

[0052] At box 256, the system processes one or more environmental signals to generate a privacy metric associated with the submission of a search query. The system is capable of processing one or more environmental signals using multiple ML models to generate multiple outputs, and is capable of generating a privacy metric based on the multiple outputs generated across the multiple ML models. In implementations where one or more environmental signals include location information about the user's environment at the time the search query is received, the system is capable of processing the location information using multiple classification models to generate multiple outputs (e.g., regarding...). Figure 1 (As described in location engine 141). In one or more of the environmental signals, including audio data capturing noise in the user's environment at the time the search query is received, in an additional or alternative implementation, the system is able to process the audio data using (multiple) acoustic models to generate (multiple) outputs (e.g., as per [reference to...]). Figure 1 (As described in acoustic engine 142). In one or more of the environmental signals, including visual data captured of the user's environment at the time the search query is received, in an additional or alternative implementation, the system is capable of processing the visual data using (multiple) visual models to generate (multiple) outputs (e.g., as per [reference to...]). Figure 1 (As described in the visual engine 143). Furthermore, the system is able to use additional (multiple) ML models to process one or more of these outputs to generate privacy metrics.

[0053] At box 258, the system determines whether user input has been received for the search interface of the user's client device or an attached client device. User input can be one or more of, for example, typing, touch, or verbal input for the search interface. For example, user input for the search interface can include accessing the search interface, entering part of the query into the search field of the search interface, submitting a query through the search field of the search interface, invoking an automation assistant (e.g., by squeezing the client device or an attached client device, by speaking a specific word or phrase, and / or by invoking an automation assistant by other means) without receiving any additional user input (and optionally after a period of time without receiving any additional user input), and / or other user interactions with the search interface. If, in the iteration of box 258, the system determines that no user input has been received for the search interface, the system continues to monitor user input at box 258. If, in the iteration of box 258, the system determines that user input has been received for the search interface, the system proceeds to box 260.

[0054] At box 260, the system acquires one or more additional environmental signals associated with the user's environment at a subsequent time after the user input is received. The subsequent time after receiving the user input at box 258 is after the time the search query is received at box 252. It is worth noting that the user's environment at the subsequent time after receiving the user input at box 258 can be the same environment as the environment in which the search query is received at box 252, or a different environment. The one or more additional environmental signals associated with the environment can be generated by the client device or multiple sensors of the additional client device, and can include the environmental signals described above with respect to box 254.

[0055] At box 262, the system processes one or more additional environmental signals to generate an additional privacy metric associated with the user input. The system is capable of processing one or more additional environmental signals to generate the additional privacy metric associated with the user input in the same or similar manner as described in reference box 256 above.

[0056] At box 264, the system determines whether the additional privacy metric associated with the user input generated at box 262 matches the privacy metric associated with the submission of the search query generated at box 256 (e.g., regarding...). Figure 1 (As described in the privacy metric comparison engine 150). In some implementations, the privacy metric and the additional privacy metric can be labels corresponding to one or more different types of environments (e.g., public environments, semi-public environments, private environments, and / or other types of environments defined at different levels of granularity). In some versions of those implementations, determining whether the additional privacy metric matches the privacy metric can be based on the label. In some additional or alternative implementations, the privacy metric and the additional privacy metric can be probabilities, log-likelihoods, binary values, and / or other values ​​representing the degree of privacy or publicness of the environment in which the search query and user input are received. In some versions of these implementations, determining whether the additional privacy metric matches the privacy metric can be based on these values.

[0057] In the iteration at box 264, if the system determines that the additional privacy metric does not match the privacy metric, the system proceeds to box 266. For example, in an implementation where the privacy metric and the additional privacy metric are labels corresponding to one or more different types of environments, it is assumed that the privacy level associated with the submission of the search query received at box 252 corresponds to a privacy label, and that the additional privacy level associated with the user input received at box 258 corresponds to a public label. In this example, the system could determine that the additional privacy metric does not match the privacy metric because the user input was received in an environment less private than the environment in which the search query was received. As another example, in an implementation where the privacy metric and the additional privacy metric are values ​​representing the degree of privacy or publicness of the environment in which the search query and user input were received, it is assumed that the privacy level associated with the submission of the search query received at box 252 corresponds to a probability of 0.8 (e.g., indicating a more private environment), and that the additional privacy level associated with the user input received at box 258 corresponds to a probability of 0.4 (e.g., indicating a less private environment). In this example, the system can determine a mismatch probability of 0.4 or a probability greater than 0.8, indicating that the environment in which the user input was received is less private than the environment in which the search query was received.

[0058] At box 266, the system selects a subset of historical search queries from a superset of historical search queries to present to the user, based on the privacy metric associated with the submission of the search query generated at box 256 and the additional privacy metric associated with the user input generated at box 262. The superset of historical search queries can be stored in one or more databases or storage(s) accessible to the client device and / or additional client devices. Furthermore, the subset of historical search queries selected from the superset can be limited to those historical search queries associated with the privacy metric(s) that match the privacy metric associated with the submission of the search query generated at box 256. It is noteworthy that the subset of historical search queries excludes the search query received at box 252 because the additional privacy metric does not match the privacy metric. The system then proceeds to box 268. Box 268 is described below.

[0059] In the iteration of box 264, if the system determines that the additional privacy metric matches the privacy metric, the system proceeds to box 268. For example, in an implementation where the privacy metric and the additional privacy metric are labels corresponding to one or more different types of environments, it is assumed that the privacy level associated with the submission of the search query received at box 252 corresponds to a public label, and that the additional privacy level associated with the user input received at box 258 corresponds to a privacy label. In this example, the system may determine that the additional privacy metric matches the privacy metric because the user input was received in a more private (or at least equally private) environment than the environment in which the search query was received. As another example, in an implementation where the privacy metric and the additional privacy metric are values ​​representing the degree of privacy or publicness of the environment in which the search query and user input were received, it is assumed that the privacy level associated with the submission of the search query received at box 252 corresponds to a probability of 0.4 (e.g., indicating a more private environment), and that the additional privacy level associated with the user input received at box 258 corresponds to a probability of 0.6 (e.g., indicating a more private environment). In this example, the system can determine a match probability of 0.6 or greater than 0.4, indicating that the environment in which the user input was received is more private (or at least equally private) than the environment in which the search query was received. The system then proceeds to box 268.

[0060] At box 268, the system causes the system to present the user with historical search queries (or a subset thereof selected at box 266). Based on the location where user input is received at box 264, historical search queries (or a subset thereof selected at box 266) can be presented to the user via a client device or an additional client. Historical search queries (or a subset thereof) can be presented to the user before any terms of the additional search query are received at the search interface, while terms of the additional search query are being entered at the search interface, and / or in response to the search results of the additional search query being displayed at the search interface. In some embodiments, historical search queries (or a subset thereof) can be presented to the user as query suggestions at the search interface. In some additional or alternative embodiments, historical search queries (or a subset thereof) can be used to generate autocomplete suggestions for portions of the query entered at the search interface. The following describes (for example, refer to...) Figures 4A to 4C and Figures 5A to 5B It displays historical search queries (or a subset thereof).

[0061] Turn now Figure 3 A flowchart illustrating an example method 300 is provided, showing the modification of a privacy metric associated with a search query submission and the provision of a subset of historical search queries based on the modified privacy metric. For convenience, the operation of method 300 is described with reference to a system performing the operation. This system of method 300 includes (e.g., multiple) computing devices (e.g., Figure 1, Figures 4A to 4C , Figures 5A to 5B Client devices 110 and / or Figure 6 The computing device 610, one or more servers and / or other computing devices, and one or more processors and / or other components. Furthermore, although the operations of method 300 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, and / or added.

[0062] At box 352, the system receives the search query via the user's client device. The search query can be displayed on the client device's search interface, as referenced above. Figure 2 The same or similar manner described in box 252 is received.

[0063] At box 354, the system processes one or more environmental signals associated with the user's environment at the time the search query is received to generate a privacy metric. The one or more environmental signals can be related to... Figure 2 Obtained in the same or similar manner as described in box 254, and in relation to Figure 2 The same or similar methods are used to generate privacy metrics as described in box 256.

[0064] At box 356, the system determines whether user input has been received for the search interface of the user's client device or an attached client device. The system can then refer to... Figure 2 The same or similar method described in box 258 determines whether user input for the search interface has been received at the client device or an attached client device. In the iteration of box 356, if the system determines that no user input for the search interface has been received, the system continues to monitor for user input in box 356. In the iteration of box 356, if the system determines that user input for the search interface has been received, the system proceeds to box 358.

[0065] At box 358, the system processes one or more additional environmental signals associated with the user's environment at a subsequent time when user input is received to generate an additional privacy metric. The additional privacy metric generated at box 358 is associated with the user input received at box 356. The one or more additional environmental signals can be used to... Figure 2 The same or similar manner described in box 260 is obtained, and in relation to Figure 2 The same or similar methods described in box 262 are used to generate additional privacy metrics.

[0066] At box 360, the system, based on the privacy metric associated with the search query submission generated at box 354 and the additional privacy metric associated with user input generated at box 358, allows it to present the user with restricted historical search queries. It is worth noting that restricted historical search queries exclude the search query itself. For example, suppose the additional privacy metric does not match the privacy metric (e.g., compared to the above regarding...). Figure 2 (Determined in the same or similar manner as described in box 264), and assuming that a subset of historical search queries is selected from the superset of historical searches to be presented to the user, wherein the subset of historical search queries excludes the search received at box 352 (e.g., in accordance with the above regarding...). Figure 2 (The same or similar selection method described in box 266). In this example, excluding a subset of historical search results for a search query can be considered as a restricted historical search query. Furthermore, it can be referenced... Figure 2 Box 268 describes a similar or identical way of presenting limited historical search queries to the user.

[0067] At box 362, the system determines whether an additional instance of the search query received at box 352 has been received at the client device or an additional client device. The additional search query can be displayed on the client device's search interface in conjunction with the above reference. Figure 2 The same or similar method described in box 252 is received. In the iteration of box 362, if the system determines that no additional instance of the search query has been received, the system continues to monitor for additional instances of the search query at box 362. In the iteration of box 362, if the system determines that an additional instance of the search query has been received, the system proceeds to box 364.

[0068] At box 364, the system processes one or more additional environmental signals associated with the user's environment at a subsequent time when an additional instance of a search query is received to modify the privacy metric. These one or more additional environmental signals can be configured to... Figure 2 The same or similar manner described in box 260 is obtained, and in relation to Figure 2 The process is handled in the same or similar manner described in box 262 to generate additional privacy metrics associated with the submission of the search query. The privacy metric generated at box 354 can then be modified based on these additional privacy metrics. For example, suppose the additional privacy metric indicates that an additional instance of the search query received at box 362 was received in a public environment, but the original privacy metric indicates that the original instance of the search query received at box 352 was received in a private environment. In this example, the privacy metric can be modified from being associated with a private environment to being associated with a public environment because it can be inferred that the user might not consider the search query private because it was submitted in a public environment.

[0069] At box 366, the system, based on a modified privacy metric, subsequently presents the user with unrestricted historical search queries. It is worth noting that unrestricted historical search queries can include the search query itself. For example, suppose additional user input is received for the search interface, and the user is in a public environment, as indicated by yet another additional privacy metric. In this example, based on the modified privacy metric associated with the submission of the search query, the search query can be included in a subset of the historical search queries presented to the user.

[0070] Turn now Figures 4A to 4C and Figure 5A Figure B depicts various non-limiting examples of user interfaces associated with generating privacy metrics in various environments and providing different subsets of historical search queries based on the generated privacy metrics. Figures 4A to 4C Each depicts a client device 110 with a graphical user interface 190, and may include... Figure 1 One or more components of the client device. Privacy search system (e.g., Figure 1 One or more aspects of the privacy search system 180 can communicate with the client device 110 and / or with the client device 110 via a network (e.g., via...). Figure 1 The network 199 is implemented locally at (multiple) remote computing devices (e.g., (multiple) servers). However, for simplicity, Figures 4A to 4C The operation is described in this document as being performed by client device 110. Although Figures 4A to 4C The client device 110 is described as a mobile phone, but it should be understood that this is not intended to be limiting. The client device 110 can be, for example, a standalone assistant device (e.g., having multiple microphones, multiple speakers, and / or a display), a laptop computer, a desktop computer, an in-vehicle computing device, and / or any other client device capable of receiving search queries, processing search queries, and / or displaying historical search queries.

[0071] Figures 4A to 4C The graphical user interface 190 also includes a text reply interface element 194 and a voice reply interface element 195. The user can select the text reply interface element 194 via a virtual keyboard 196 (e.g., ...). Figure 4CUser input can be generated via touch and / or typing input (as shown) or other touch and / or typing input, and the user can select voice response interface element 195 to generate user input via microphone(s) of client device 110. In some embodiments, the user can generate user input via microphone(s) without selecting voice response interface element 195. For example, audible user input via microphone(s) can be actively monitored to avoid the need for the user to select voice response interface element 195. In some and / or other embodiments, voice response interface element 195 can be omitted. Furthermore, in some embodiments, text response interface element 194 can be additionally and / or alternatively omitted (e.g., the user can provide only audible user input). Figures 4A to 4C The graphical user interface 190 also includes system interface elements 191, 192, and 193, with which users can interact to enable the computing device 110 to perform one or more actions.

[0072] In various implementations, user input for a search interface displayed on a graphical user interface 190 can be received from user 401 of client device 110. Client device 110 can utilize privacy search system 180 to process one or more environmental signals to generate a privacy metric associated with the environment 400 in which user 401 is located when user input is received. Figure 4A As shown, it is assumed that user 401 is the only user present in environment 400, as indicated by other user 402 located outside environment 400 (e.g., outside the dashed box). Further assumptions, and as... Figure 4B As shown, user input for the search interface (e.g., a browser-based software application accessible by client device 110, indicated by URL 411 of “www.exampleurl0.com / ”) includes user 401 submitting search query 415B “Symptoms of Infectious Disease #19”. Search results in response to the search query can be presented to user 401 and can include, for example, a first search result 420 “Government Disease Website” and a second search result 430 “Virtual Doctor Website”.

[0073] exist Figure 4A and Figure 4BIn the example shown, it is assumed that the sensors(s) of client device 110 capture one or more environmental signals of the environment 400 in which user 401 is located when search query 415B is received. The one or more environmental signals can include, for example, location information generated using the GPS(s) of client device 110 corresponding to the location of user 401 when search query 415B is received, audio data generated using the microphone(s) of client device 110 capturing ambient noise of environment 400 when search query 415B is received, and / or visual data generated using the microphone(s) of client device 110 capturing environment 400 when search query 415B is received. Furthermore, it is possible to use (e.g., stored in multiple ML models) Figure 1 The database of (multiple) ML models 140A processes one or more environmental signals to generate (multiple) outputs and is able to generate a privacy metric associated with the submission of search query 415B based on the (multiple) outputs generated across (multiple) ML models.

[0074] For example, suppose the location information indicates that user 401 is located at a residential address (e.g., user 401's home). In this example, client device 110 enables privacy search system 180 to process the location information using multiple classification models to generate multiple outputs (e.g., regarding...). Figure 1 (As described by location engine 141). The outputs can include, for example, one or more types of environments corresponding to the location information of user 401 when search query 415B is received, and / or corresponding values ​​associated with each of the one or more types of environments. The one or more types of environments can be defined at different levels of granularity. For example, the one or more types of environments can correspond to public environments or private environments, or more specifically, coffee shops, libraries, train stations, and / or other more specific types of public environments or residences (e.g., the residence of the user of client device 110), private offices, and / or other more specific types of private environments. Furthermore, the values ​​associated with each of the one or more environment types can be probabilities, log-likelihoods, binary values, and / or any other values ​​indicating the level of confidence associated with the classification of the environment type in which user 401 is located when search query 415B is received. By processing location information using the privacy search system 180, client device 110 can determine that user 401 is at home, which can be considered a private environment.

[0075] Additionally or alternatively, it is assumed that the audio data does not include the speech of user 401 or any other user (e.g., other user 402), but captures audio data corresponding to a television program playing in the background of environment 400. In this example, client device 110 enables privacy search system 180 to process the audio data using multiple acoustic models to generate multiple outputs (e.g., regarding...). Figure 1 (As described in the acoustic engine 142). The outputs (multiple) can include, for example, one or more acoustic properties captured in the audio data and / or corresponding values ​​associated with each of the one or more acoustic properties. The one or more acoustic properties can include, for example, the noise level of the environment 400 in which user 401 is located when search query 415B is received, one or more types of noise captured in the environment 400 in which user 401 is located when search query 415B is received, and / or corresponding values ​​associated with each of the one or more types of noise. Furthermore, the values ​​associated with each of the one or more types of noise can be probabilities, log-likelihoods, binary values, and / or any other values ​​indicating the confidence level associated with the classification of the type of noise captured in the environment 400 in which user 401 is located when search query 415B is received. By using the privacy search system 180 to additionally or alternatively process the audio data, the client device 110 can determine that the audio data does not include the voice of user 401 or any other user, but the audio data is captured from a television program playing in the background of environment 400.

[0076] Additionally or alternatively, assume that the visual data does not capture any other users in the environment (e.g., other user 402), but a sofa is present in environment 400. In this example, client device 110 enables privacy search system 180 to process visual data using multiple visual models to generate multiple outputs (e.g., regarding...). Figure 1(As described in visual engine 143). The outputs can include, for example, one or more acoustic attributes captured in audio data and / or corresponding values ​​associated with each of the one or more acoustic attributes. The one or more acoustic attributes can include, for example, an indication of whether other users (e.g., other users 402) exist in the environment 400 where user 401 is located when search query 415B is received (and optionally, the identity of other users if visual embeddings of (multiple) other users are available), one or more types of objects captured in the environment 400 where user 401 is located when search query 415B is received, and / or corresponding values ​​associated with each of the one or more types of objects. Furthermore, the values ​​associated with each of the one or more types of objects in the environment can be probabilities, log-likelihoods, binary values, and / or any other values ​​indicating the confidence level associated with the classification of the object types captured in the environment 400 where user 401 is located when search query 415B is received. By using the privacy search system 180 to additionally or alternatively process visual data, the client device 110 is able to determine that the visual data does not indicate the presence of any other user in the environment, but the visual data is captured on the sofa in the environment 400.

[0077] Furthermore, client device 110 enables privacy search system 180 to process one or more of these outputs generated based on one or more of processing location information, audio data, or visual data to generate a privacy metric associated with the submission of search query 415B. One or more of these outputs can be processed using multiple ML models or multiple rules (e.g., multiple ML rules and / or multiple heuristically defined rules). For example, based on the fact that user 401 exists in an environment 400 corresponding to user 401's home, as indicated by location information, the noise level of environment 400 is relatively low (e.g., a television program in the background), and that no other users are present in environment 400, as indicated by audio data, and / or as indicated by visual data, the resulting privacy metric associated with the submission of search query 415B can instruct user 401 to consider search query 415B as private. Therefore, the resulting privacy metric can be a label of "privacy" and may optionally include relevant values ​​indicating a private environment (e.g., a probability of 0.95 associated with the label of "privacy"). When additional user input for the search interface is received at client device 110 or an additional client device of user 401, the resulting privacy metric can then be used to determine whether search query 415B should be included in a subset of the historical search queries presented to user 401.

[0078] In various implementations, additional user input for a search interface displayed on the graphical user interface 190 can be received from user 401 of client device 110. For example, such as Figure 4C As shown, assume that user 401 subsequently provides additional user input to access a search interface (e.g., a browser-based software application accessible by client device 110, indicated by URL 411 of “www.exampleurl0.com / ”). When the search interface is subsequently accessed at client device 110, at least a subset of historical search queries can be presented to user 401 at the search interface of client device 110. The subset of historical search queries presented to user 401 can be selected from a superset of historical search queries based on a privacy metric associated with the submission of search query 415B and an additional privacy metric associated with the reception of additional user input for the search interface (e.g., user access to the search interface as described above). The additional privacy metrics can be generated in the same or similar manner as described above regarding privacy metrics, but based on one or more additional environmental signals obtained at the time the additional user input for the search interface is received in environment 400.

[0079] For example, suppose user 401 exists in reference Figure 4A and Figure 4BThe described environment 400 (e.g., user 401 is alone in environment 400), but at subsequent times, such as the next day, the following week, and / or any other time after the time search query 415B is received. In this example, additional privacy metrics associated with additional user input for the search interface may match the privacy metrics associated with the submission of search query 415B. As a result, a subset of historical search queries presented to user 401 (e.g., 425C1, 425C2, 425C3, and 425C4) may include search query 415B. In some implementations, the subset of historical search queries presented to user 401 can be presented as a list or search query 425, which includes search query 415B (e.g., as shown in 425C1) and / or other search queries associated with privacy metrics that match at least the privacy metrics associated with the submission of search query 415B. In some implementations, the search query list 435 may optionally include additional search queries that are not part of a superset of historical search queries from which the subset is selected. For example, in a subsequent time when additional user input is received for the search interface, the search query list 425 may optionally include one or more popular search queries that are popular among the user group (e.g., as indicated by 425C5). In some additional or alternative implementations, the search query list 425 may optionally include an optional element 425C6, which, when selected, causes one or more additional historical search queries to be presented to the user 401. For example, the search query list 425 may be expanded to include one or more additional historical search queries, or an additional subset of historical search queries may replace those included in the search query list 425. In some additional or alternative implementations, a subset of historical search queries may be used to generate one or more autocomplete suggestions for a portion of the query entered by the user 401 for the search interface. For example, suppose the user 401 provides a partial query 415C for “Sym” via the virtual keyboard 196. In this example, search query 415B may be used to generate an autocomplete suggestion 415C1 for “ptoms of Infectious Disease #19”. Let's further assume that user 401 selected autocomplete suggestion 415C1. In this example, it is possible to resubmit an additional instance of search query 415B and to present the search results to user 401 in response to the additional instance of search query 415B.

[0080] Conversely, suppose that when additional user input is received for the search interface, user 401 exists in an environment different from environment 400, such as... Figure 5A The environment depicted in the text is 500. For example... Figure 5AAs shown, it is assumed that user 401 coexists in environment 500 with another user 402, as indicated by another user 402 located inside environment 500 (e.g., inside the dashed box). Further assumptions are made, and as... Figure 5B As shown, additional user input for the search interface (e.g., as from URL 411) www.exampleurl0.com / The browser-based software application (accessible by client device 110) includes user 401 accessing a search interface. When the search interface is subsequently accessed at client device 110, at least one subset of historical search queries can be presented to user 401 at the search interface of client device 110. The subset of historical search queries presented to user 401 can be selected from the superset of historical search queries based on a privacy metric associated with the submission of search query 415B and an additional privacy metric associated with the reception of additional user input for the search interface (e.g., as described above when the user accesses the search interface). The additional privacy metrics can be generated in the same or similar manner as described above regarding privacy metrics, but based on one or more additional environmental signals obtained at the time when additional user input for the search interface is received in environment 500.

[0081] For example, suppose user 401 is present in environment 500 along with another user 401, but at a later time, such as the next day, the following week, and / or any other time after the time search query 415B is received. Further suppose that location information captured at a later time after receiving additional user input indicates the user is in a restaurant (e.g., a public environment), audio data captures a relatively high noise level caused by people talking in the restaurant, and / or visual data indicates that other people (e.g., at least other user 402) are present in environment 500. In this example, the resulting additional privacy metric associated with the additional user input for the search interface cannot match the privacy metric associated with the submission of search query 415B. As a result, a subset of historical search queries presented to user 401 (e.g., 525B1, 525B2, and 525B3) can exclude search query 415B. In some implementations, a subset of historical search queries presented to user 401 can be presented as a list or search query 525 that excludes search query 415B (e.g., as indicated by the absence of search query 415B in the search query list 525) and / or includes other search queries associated with privacy metrics previously submitted in a public environment. In some implementations, the search query list 525 can optionally include additional search queries that are not part of a superset of historical search queries from which the subset is selected. For example, at a subsequent time after receiving additional user input for the search interface, the search query list 525 can optionally include one or more popular search queries that are popular among the user group (e.g., as indicated by 525B4).

[0082] In some additional or alternative implementations, the search query list 525 may optionally include an optional element 525B5, which, when selected, causes one or more additional historical search queries to be presented to the user 401. The optional element 525B5 may optionally include an indication that the historical search queries presented to the user 401 in the search query list 525 have been filtered or restricted based on the environment 500. For example, the search query list 425 may be expanded to include one or more additional historical search queries, or an additional subset of historical search queries may replace those included in the search query list 425. Notably, one or more additional search queries presented to the user 401 in response to the selection of the optional element 525B5 may optionally include one or more historical search queries submitted in a more private environment (e.g., search query 415B). As another example, and in response to the selection of the optional element 525B5, the user 401 may be prompted to select whether the user 401 wants the privacy search system 180 to filter or restrict the presentation of search queries included in the search query list 525 in response to subsequent user input to the search interface. In some additional or alternative implementations, a subset of historical search queries can be used to generate one or more autocomplete suggestions for a portion of the query entered by user 401 at the search interface, as referenced above. Figure 4C As described. However, because the environment 500 in which user 401 is currently located does not match the environment 400 in which the search query 415B was initially received, the search query 415B may not be used to generate autocomplete suggestions. For example, suppose user 401 provides a partial query 515C for "Sym" via virtual keyboard 196. In this example, an autocomplete suggestion 515B1 for "phony Tickets" can be generated using a different search query.

[0083] Furthermore, when an additional instance of search query 415B is received, the privacy metrics associated with the submission of search query 415B can be modified based on the environment in which user 401 exists. For example, suppose user 401 exists in Figure 5A In the environment 500 shown, an additional instance of query 415B is submitted. In this example, the privacy metric associated with the submission of search query 415B can be modified to indicate that the user may not consider search query 415B as a privacy search query. As a result, if user 401 provides additional user input for the search interface in environment 500, search query 415B can be considered as described above. Figure 4C The same or similar descriptions are included in a subset of the historical search queries presented to the user in a 401 redirect.

[0084] although Figures 4A to 4C and Figures 5A to 5BThis document describes specific environmental signals acquired in a specific environment; however, it should be understood that these are provided for illustrative purposes and are not intended to be limiting. Furthermore, it should be understood that the techniques described herein can be implemented using any environmental signal alone or in any combination of the environmental signals described herein, and the environmental signals acquired by client device 110 can be based on the sensors of client device 110. In other words, some client devices may not include multiple GPS sensors, multiple microphones, and / or multiple vision components. However, as long as client device 110 is able to acquire at least one environmental signal, the techniques described herein can still be used to adapt the presentation of historical search queries to the user.

[0085] Figure 6 This is a block diagram of an example computing device 610, which may optionally be used to perform one or more aspects of the techniques described herein. In some implementations, one or more of a client device, a plurality of cloud-based automation assistant components, and / or other components may include one or more components of the example computing device 610.

[0086] Computing device 610 typically includes at least one processor 614 that communicates with a plurality of peripheral devices via a bus subsystem 612. These peripheral devices may include a storage subsystem 624, including, for example, a memory subsystem 625 and a file storage subsystem 626, a user interface output device 620, a user interface input device 622, and a network interface subsystem 616. The input and output devices allow users to interact with computing device 610. The network interface subsystem 616 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0087] User interface input device 622 may include a keyboard, a pointing device such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen integrated into a display, an audio input device such as a voice recognition system, a microphone, and / or other types of input devices. In general, the term "input device" is used to encompass all possible types of devices and methods for inputting information into computing device 610 or a communication network.

[0088] User interface output device 620 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating visual images. The display subsystem may also provide non-visual displays, such as via an audio output device. In general, the term "output device" is used to encompass all possible types of devices and methods for outputting information from computing device 610 to a user or another machine or computing device.

[0089] Storage subsystem 624 stores programs and data structures that provide functionality for some or all of the modules described herein. For example, storage subsystem 624 may include selected aspects for implementing the methods disclosed herein. Figure 1 The logic of the various components described in the text.

[0090] These software modules are typically executed by processor 614 alone or in conjunction with other processors. The memory 625 used in storage subsystem 624 can include multiple memories, including main random access memory (RAM) 630 for storing instructions and data during program execution and read-only memory (ROM) 632 for storing fixed instructions. File storage subsystem 626 provides persistent storage for program and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical drives, or removable media cartridges. Functional modules implementing certain embodiments may be stored by file storage subsystem 626 in storage subsystem 624 or in other machines accessible to processor(s) 614.

[0091] The bus subsystem 612 provides a mechanism for allowing various components and subsystems of the computing device 610 to communicate with each other as intended. Although the bus subsystem 612 is schematically shown as a single bus, alternative implementations of the bus subsystem 612 may use multiple buses.

[0092] The computing device 610 can be of various types, including workstations, servers, computing clusters, blade servers, server groups, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, Figure 6 The description of the computing device 610 depicted herein is intended only as a specific example for illustrating some implementation methods. Many other configurations of the computing device 610 may have... Figure 6 The computing device depicted in the text has more or fewer components.

[0093] In cases where the system described herein collects or otherwise monitors personal information about a user, or may utilize personal and / or monitored information, the user may be given the opportunity to control whether the program or feature collects user information (e.g., information about the user's social networks, social behavior or activities, occupation, user preferences, or the user's current geographic location), or to control whether and / or how content that may be more relevant to the user is received from the content server. Furthermore, certain data may be processed in one or more ways before being stored or used to remove personally identifiable information. For example, a user's identity may be processed so that the user's personally identifiable information cannot be determined, or the user's geographic location may be generalized (e.g., generalized to a city, zip code, or state level) where geographic location information is available, making it impossible to determine the user's specific geographic location. Therefore, the user can control how information about them is collected and / or used.

[0094] In some implementations, a method implemented by one or more processors is provided, the method comprising: receiving a search query via a user's client device; obtaining one or more environmental signals associated with the user's environment at the time the search query is received; processing the one or more environmental signals to generate a privacy metric associated with the submission of the search query; and, after generating the privacy metric associated with the submission of the search query: receiving user input to a search interface for the user's client device or an additional client device; obtaining one or more additional environmental signals associated with the user's environment at a subsequent time after the time the user input is received; processing the one or more additional environmental signals to generate an additional privacy metric associated with the user input; selecting a subset of historical search queries from a superset of the user's historical search queries, the selection being based at least on the privacy metric and the additional privacy metric; and, in response to receiving user input to the search interface, causing the subset of historical search queries to be presented to the user via the client device or an additional client device.

[0095] These and other embodiments of the technology disclosed herein may optionally include one or more of the following features.

[0096] In some implementations, one or more environmental signals associated with the user's environment may include one or more of the following: location information corresponding to the user's location when a search query is received, audio data capturing environmental noise of the environment when a search query is received, or visual data capturing the environment when a search query is received.

[0097] In some versions of these implementations, one or more environmental signals associated with a search query may include at least location information corresponding to the user's location when the search query is received. In other versions of these implementations, processing one or more environmental signals to generate a privacy metric associated with the submission of a search query may include using a machine learning model to process the location information corresponding to the user's location when the search query is received to generate output associated with one or more types of environments. Generating the privacy metric associated with the submission of a search query may be based on the output associated with one or more types of environments. In yet another version of these implementations, the type of environment may include one or more of the following: a public environment, a semi-public environment, or a private environment.

[0098] In additional or alternative versions of these embodiments, one or more environmental signals associated with the search query may include at least audio data capturing ambient noise in the environment at the time the search query is received. In some further versions of these embodiments, processing one or more environmental signals to generate a privacy metric associated with the submission of the search query may include using a machine learning model to process the audio data capturing ambient noise in the environment at the time the search query is received to generate an output associated with one or more acoustic properties of the environment. Generating the privacy metric associated with the submission of the search query may additionally or alternatively be based on the output associated with one or more acoustic properties of the environment. In still other versions of these embodiments, one or more acoustic properties of the environment may include one or more of the following: the noise level of the environment, or the classification of ambient noise detected in the environment.

[0099] In additional or alternative versions of these implementations, one or more environmental signals associated with a search query may include at least visual data capturing the environment at the time the search query is received. In some further versions of these implementations, processing one or more environmental signals to generate a privacy metric associated with the submission of a search query may include using a machine learning model to process the visual data capturing the environment at the time the search query is received to generate output associated with one or more visual attributes of the environment. Generating the privacy metric associated with the submission of the search query may additionally or alternatively be based on one or more visual attributes of the environment. In still other versions of these implementations, one or more visual attributes of the environment may include one or more of the following: indications of the presence of one or more other users in the environment, or indications of the presence of one or more objects in the environment.

[0100] In some implementations, the method may further include: comparing a privacy metric associated with a submission of a search query with an additional privacy metric associated with user input to a search interface; and based on the comparison, determining whether the additional privacy metric associated with user input to the search interface matches the privacy metric associated with the submission of the search query.

[0101] In some versions of these implementations, selecting a subset of historical search queries may be in response to determining that an additional privacy metric associated with user input to the search interface does not match the privacy metric associated with the submission of the search query. In some other versions of those implementations, determining that an additional privacy metric associated with user input to the search interface does not match the metric associated with the submission of the search query may include determining that the additional privacy metric associated with the user input fails to meet a threshold privacy metric level associated with the privacy metric associated with the submission of the search query.

[0102] In additional or alternative versions of these implementations, the method may further include presenting selectable elements along with a subset of historical search queries. When a selectable element is selected, it may present one or more additional historical search queries that were not selected and included in the subset to the user via a client device or an additional client device.

[0103] In some implementations, the method may further include generating a prompt based on a privacy metric associated with the submission of a search query to solicit a user's choice regarding whether the search query should be included in a subset; and causing the prompt to be presented to the user via a client device or an additional client device. In some versions of these implementations, presenting the prompt to the user may be in response to determining a current privacy metric associated with the user's environment at an intermediate time, wherein the intermediate time is after that time and before a subsequent time.

[0104] In some implementations, the method may further include receiving additional user input to remove a subset of historical search queries from the superset of historical search queries; and, in response to receiving additional user input, removing historical search queries included in the subset from the superset.

[0105] In some implementations, a subset of historical search queries may be presented to the user as autocomplete query suggestions in response to additional search queries entered in response to user input for the search interface.

[0106] In some implementations, a subset of historical search queries can be presented to the user as query suggestions for additional search queries to be entered at the search interface.

[0107] In some implementations, processing one or more environmental signals to generate a privacy metric associated with the submission of a search query may include processing the one or more environmental signals using a machine learning model. Processing one or more additional environmental signals to generate an additional privacy metric associated with user input may include processing the one or more additional environmental signals using a machine learning model. In some versions of those implementations, the method may further include, prior to receiving the search query: obtaining a plurality of training instances, each of the plurality of training instances including a training instance input and a training instance output, the training instance input including one or more training environmental signals associated with training the search query, and the training instance output including one or more ground truth labels associated with the privacy level of training the search query. The method may also include training the machine learning model based on the plurality of training instances.

[0108] In some implementations, the method may further include processing the search query to identify one or more terms of the search query. Generating a privacy metric associated with the submission of the search query may further be based on processing one or more terms of the search query. In some versions of these implementations, processing one or more terms of the search query may include using a machine learning model to process one or more terms of the search query to generate output associated with one or more types of search queries. Generating a privacy metric associated with the submission of the search query may be based on the output associated with one or more types of search queries.

[0109] In some implementations, a method implemented by one or more processors is provided, the method comprising: receiving a search query via a user's client device; processing a set of environmental signals associated with the user's environment at the time the search query is received to generate a privacy metric associated with the submission of the search query; and, after generating the privacy metric associated with the submission of the search query: receiving user input to a search interface for the user's client device or an additional client device; processing an additional set of additional environmental signals associated with the user's environment at a subsequent time after the time the user input is received to generate an additional privacy metric associated with the user input, the subsequent time being after the time; in response to receiving user input to the search interface, presenting a restricted history search query via the client device or an additional client device, based on the privacy metric and the additional privacy metric, the restricted history search query being restricted based on the privacy metric and the additional privacy metric; receiving an additional search query via the client device or an additional client device, the additional search query being an additional instance of the search query; and processing an additional set of additional environmental signals associated with the user's environment at a further subsequent time after the time the additional search query is received to modify the privacy metric associated with the submission of the search query, the further subsequent time being after the time and the subsequent time.

[0110] These and other embodiments of the technology disclosed herein may optionally include one or more of the following features.

[0111] In some implementations, the method may further include, after modifying a privacy metric associated with the submission of a search query: receiving additional user input for a search interface on a client device or an additional client device; processing a further set of additional environmental signals associated with the user's environment at a further subsequent time when the additional user input is received to generate a further additional privacy metric associated with the user input, the further subsequent time being after the time, the subsequent time, and the further subsequent time; and, in response to receiving additional user input for the search interface, causing an unrestricted historical search query to be presented via the client device or an additional client device, the unrestricted historical search query being unrestricted based on the further additional privacy metric.

[0112] In some versions of those implementations, restricted historical search queries may exclude the search query, while unrestricted historical search queries may at least include the search query.

[0113] These and other embodiments of the technology disclosed herein may optionally include one or more of the following features.

[0114] Furthermore, some embodiments include one or more processors (e.g., multiple central processing units (CPUs), multiple graphics processing units (GPUs), and / or multiple tensor processing units (TPUs)) of one or more computing devices, wherein the one or more processors are operable to execute instructions stored in associated memory, and wherein the instructions are configured to cause the execution of any of the foregoing methods. Some embodiments also include one or more non-transitory computer-readable storage media storing computer instructions executable by one or more processors to perform any of the foregoing methods. Some embodiments also include a computer program product comprising instructions executable by one or more processors to perform any of the foregoing methods.

[0115] It should be understood that all combinations of the foregoing concepts and additional concepts described in more detail herein are considered part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the subject matter disclosed herein.

Claims

1. A method implemented by one or more processors, the method comprising: Search queries are received via the user's client device; Obtain one or more environmental signals associated with the environment in which the user is located at the time the search query is received, wherein the one or more environmental signals associated with the user's environment include one or more of the following: capturing audio data of environmental noise of the environment when the search query is received and capturing visual data of the environment when the search query is received; Process the one or more environmental signals to generate a privacy metric associated with the submission of the search query, wherein the privacy metric represents a measure of the presence of persons other than the user in the environment in which the user of the client device is located at the time the search query is received; and After generating the privacy metric associated with the submission of the search query: Receive user input for the search interface of the user's client device or attached client device; Obtain one or more additional environmental signals associated with the environment in which the user is located at a subsequent time after the user input is received, wherein the subsequent time is after the time when the search query is received, wherein the one or more additional environmental signals associated with the user's environment include one or more of the following: capturing audio data of ambient noise of the environment when the user input is received and capturing visual data of the environment when the user input is received; The one or more additional environmental signals are processed to generate an additional privacy metric associated with the user input, wherein the additional privacy metric represents a measure of the presence of persons who are not the user in the environment in which the user is located when the user input is received on the client device; A subset of the user's historical search queries is selected from a superset of the user's historical search queries, wherein the selection is based at least on the privacy metric and the additional privacy metric; and In response to receiving user input for the search interface, a subset of the historical search queries is presented to the user via the client device or the additional client device.

2. The method according to claim 1, wherein, The one or more environmental signals associated with the user's environment include location information corresponding to the user's location when the search query is received.

3. The method according to claim 2, wherein, The one or more environmental signals associated with the search query include at least the location information corresponding to the user's location when the search query is received.

4. The method according to claim 3, wherein, Processing the one or more environmental signals to generate the privacy metric associated with the submission of the search query includes: Machine learning models are used to process the location information corresponding to the user's location when the search query is received to generate output associated with one or more types of the environment. The generation of the privacy metric associated with the submission of the search query is based on the output associated with one or more types of the environment.

5. The method according to claim 4, wherein, The types of environments include one or more of the following: public environment, semi-public environment, and private environment.

6. The method according to claim 2, wherein, The one or more environmental signals associated with the search query include at least the audio data that captures the environmental noise of the environment at the time the search query is received.

7. The method according to claim 6, wherein, Processing the one or more environmental signals to generate the privacy metric associated with the submission of the search query includes: The audio data, which captures ambient noise of the environment when the search query is received, is processed using a machine learning model to generate an output associated with one or more acoustic properties of the environment. The privacy metric associated with the submission of the search query is generated based on the output associated with one or more acoustic properties of the environment.

8. The method according to claim 7, wherein, The one or more acoustic properties of the environment include one or more of the following: the noise level of the environment, and the classification of environmental noise detected in the environment.

9. The method according to claim 2, wherein, The one or more environmental signals associated with the search query include at least the visual data of the environment captured when the search query is received.

10. The method according to claim 9, wherein, Processing the one or more environmental signals to generate the privacy metric associated with the submission of the search query includes: A machine learning model is used to process the visual data of the environment captured when the search query is received to generate output associated with one or more visual attributes of the environment. The privacy metric generated in connection with the submission of the search query is based on one or more visual attributes of the environment.

11. The method according to claim 10, wherein, The one or more visual attributes of the environment include one or more of the following: the presence of one or more additional user indications in the environment, and indications of one or more objects present in the environment.

12. The method according to claim 1, further comprising: The privacy metric associated with the submission of the search query and the additional privacy metric associated with the user input for the search interface are compared. as well as Based on the comparison, it is determined whether the additional privacy metric associated with the user input for the search interface matches the privacy metric associated with the submission of the search query.

13. The method according to claim 12, wherein, Selecting the subset of the historical search queries is in response to determining that the additional privacy metric associated with the user input for the search interface does not match the privacy metric associated with the submission of the search query.

14. The method according to claim 13, wherein, Determining that the additional privacy metric associated with the user input for the search interface does not match the metric associated with the submission of the search query includes: determining that the additional privacy metric associated with the user input fails to meet a threshold privacy metric level for the privacy metric associated with the submission of the search query.

15. The method of claim 12, further comprising: The selectable element is presented together with the subset of the historical search queries, wherein, when the selectable element is selected, one or more additional historical search queries that were not selected and included in the subset are presented to the user via the client device or the additional client device.

16. The method according to any one of claims 1-15, further comprising: Based on the privacy metric associated with the submission of the search query, a prompt is generated to solicit user selection related to whether the search query should be included in the subset; as well as The prompt is then presented to the user via the client device or the additional client device.

17. A method implemented by one or more processors, the method comprising: Search queries are received via the user's client device; A set of environmental signals associated with the user's environment at the time the search query is received is processed to generate a privacy metric associated with the submission of the search query, wherein the set of environmental signals associated with the environment includes one or more of the following: capturing audio data of environmental noise in the environment at the time the search query is received and capturing visual data of the environment at the time the search query is received, and wherein the privacy metric represents a measure of the presence of persons who are not the user in the environment in which the user is located on the client device at the time the search query is received; and After generating the privacy metric associated with the submission of the search query: Receive user input for the search interface of the user's client device or attached client device; An additional set of additional environmental signals associated with the environment in which the user is located at a subsequent time when the user input is received is processed to generate an additional privacy metric associated with the user input, wherein the subsequent time is after the time, wherein the additional set of additional environmental signals associated with the environment includes one or more of the following: capturing audio data of ambient noise of the environment when the user input is received and capturing visual data of the environment when the user input is received, and wherein the additional privacy metric represents a measure of the presence of people who are not the user in the environment in which the user is located on the client device at the time the user input is received; In response to receiving user input for the search interface, based on the privacy metric and the additional privacy metric, a restricted historical search query is presented via the client device or the additional client device, wherein the restricted historical search query is restricted based on the privacy metric and the additional privacy metric; Receive additional search queries via the client device or the additional client device, wherein the additional search query is an additional instance of the search query; and An additional set of environmental signals associated with the user's environment at a further subsequent time when the additional search query is received is processed to modify the privacy metric associated with the submission of the search query, wherein the additional subsequent time is after the time and the subsequent time.

18. The method of claim 17, further comprising: After modifying the privacy metric associated with the submission of the search query: Receive additional user input for the search interface of the client device or the attached client device; Processing another set of additional environmental signals associated with the user's environment at yet another subsequent time when the additional user input is received, to generate another additional privacy metric associated with the user input, wherein the yet another subsequent time is after the time, the subsequent time, and the additional subsequent time; and In response to receiving the additional user input for the search interface, an unrestricted historical search query is presented via the client device or the additional client device, wherein the unrestricted historical search query is unrestricted based on the additional privacy metric.

19. The method according to claim 18, wherein, The restricted historical search query does not include the search query, and the unrestricted historical search query includes at least the search query.

20. A system comprising: At least one processor; as well as A memory for storing instructions, which, when executed, cause the at least one processor to perform the method according to any one of claims 1 to 19.

21. A non-transitory computer-readable storage medium storing instructions, which, when executed, cause at least one processor to perform the method of any one of claims 1 to 19.

Images