A user interface for making web browser history data visible.

Abstract

A system and method are described that includes generating a repository of metadata based on a plurality of web pages accessed and saved in a browser history of a web browser executing on a computing device, and generating a history cluster including a portion of the plurality of web pages related to a topic based on the metadata, the history cluster generation being based on source events and access timestamps of the web pages in the portion, the system and method further including assigning respective scores to the web pages in the portion. In response to a request to view browser activity associated with the topic, the system and method can generate and display a history cluster listing for the topic, the history cluster listing including a list of visits associated with web pages in the history cluster determined to have a score that meets a threshold score.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Application No. 63 / 260,164, filed on August 11, 2021, entitled "USER INTERFACES FOR SURFACING WEB BROWSER HISTORY DATA", the disclosure of which is incorporated herein by reference in its entirety.

Background Art

[0002] Background A web browser enables a user to navigate a number of websites during various browser sessions that are executed over time. When the user navigates a website, the web browser can save a browser history indicating the web page visits. The web page visits are sequentially saved based on the date and time of access. It can be difficult for the user to look back at previous web page visits using the saved history. For example, the user may have to manually search a list of saved navigation information to find data on previous web page visits. Searching the saved navigation information extensively can consume a large amount of resources, which can be particularly problematic for mobile computing devices with limited battery capacity.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Summary The systems and methods described herein function as browser history organization tools for generating a list of history clusters representing search activities associated with a web browser. In particular, the history cluster list can, with user consent, visualize browser history (e.g., search history data, download history data, cookies, cache data, and any relevant search activities associated with a web browser) as a consistent search journey. With user consent, the systems and methods described herein can generate history clusters using browser history data and / or other metadata generated with respect to (or stored within) the browser history of a web browser. A history cluster may represent a search journey over time, containing several search activities related to a particular topic. Any number of history clusters can be generated using models (e.g., clustering algorithms, machine learning models) running on a local computing device.

[0004] A history cluster represents the basis upon which one or more history cluster lists are generated by the systems and methods described herein. A history cluster list may include search activities that are analyzed, selected, organized, and formatted and displayed in a user interface (e.g., UI) without user intervention. In some implementations, history cluster lists may be generated using ML algorithms. For example, a web browser may, with user authorization, perform ML analysis on locally stored browser history data using an ML algorithm without accessing a server device or other connected device. In this case, local data and on-device analysis are employed to analyze, organize, and format search activities for display in the UI. Information can then be provided to the user securely and efficiently. In addition to the above, the user is provided with the ability to make choices regarding whether and when local data is saved to browser history data, whether and when and to what extent the browser history data is analyzed to generate history clusters, and whether or not content or communications are sent from the server. Furthermore, certain data may be handled in one or more ways so that user information is removed before it is saved or used. For example, a user's identity may be handled in a way that does not identify the user, or a user's geographical location may be generalized to the level of where the location information is obtained (such as city, zip code, or state level) so that the user's exact location cannot be determined from browser history data. In this way, users can control what information is collected about them, how that information is used, and what information is provided to them.

[0005] In some implementations, the history cluster list may include browser history data, search history data, search activity information, and related search result information described according to determined topics, scores, and / or rankings. Scores and / or rankings may be based at least partially on metadata corresponding to browser activities performed in the web browser (e.g., engagement activities, click events, search events, tab / bookmark events, etc.) and / or specific browser history data (e.g., access timestamps, topics, subtopics, cookies, etc.).

[0006] In some implementations, the history cluster list may be organized according to topics and / or subtopics based on any number of history clusters generated. In some implementations, the history cluster list may include controls and / or indicators to allow the user to resume a search or perform additional investigations associated with a particular history cluster list, web page, and / or search result. For example, the systems and methods described herein may provide access points, which may be represented as UI elements configured to allow re-entry into a search. Access points may be provided in browser pages, search history pages, browser history pages, browser address toolbars (e.g., omniboxes or other search controls), actions associated with browser address toolbars (or other search controls), and / or menus or controls associated with such access points.

[0007] A system consisting of one or more computers can be configured to perform a specific operation or action by installing software, firmware, hardware, or a combination thereof on the system that causes the system to perform an action when it is running. One or more computer programs can be configured to perform a specific operation or action by including instructions that cause a data processing device to perform an action when it is executed by that device. The first general embodiment describes a method performed by a computer. The method may include generating a repository of metadata based on a number of web pages that have been accessed and saved in the browser history of a web browser running on a computing device, the metadata for each web page including a source event, an access timestamp, and at least one topic. The method may also include generating a history cluster based on the metadata, which includes a portion of a number of web pages related to a given topic, the history cluster generation being based on the source event and the access timestamps of the web pages in that portion. The method may also include assigning a score to each of the web pages in that portion, the score being at least partially based on the access timestamp and the engagement score generated for each web page. In response to a request to view browser activity associated with a topic, the method may include generating a history cluster list for the topic, the history cluster list including a list of visits associated with web pages in the history cluster that are determined to have a score that meets a threshold score and are organized according to their respective scores. The method may also include displaying the history cluster list.

[0008] The implementation may include one or all of the following features: In some implementations, a visit list within the visit list contains a snippet for at least one webpage from among several webpages in the aforementioned part, and the history cluster contains associated action controls. In some implementations, the history cluster list further contains multiple visit lists corresponding to at least one webpage in the aforementioned part, and the multiple visit lists are presented within the web browser's browser history. In some implementations, the history cluster list also includes action controls configured to resume previous searches associated with the multiple visit lists.

[0009] In some implementations, the metadata further includes, for a particular webpage among several webpages, a determined webpage entity, several suggested related searches, and the time spent on the webpage. In some implementations, the history cluster is generated as the output of a machine learning model that runs on a computing device and uses the metadata as input. In some implementations, the metadata further includes a webpage identifier defined for the webpages in the above-mentioned section, which indicates whether each webpage in the above-mentioned section is part of a tab group, bookmark, or search results page accessed by the computing device. In some implementations, the generated engagement score includes an engagement metric for each webpage in the above-mentioned section, which is used to select the prominence level when displaying snippets for at least one webpage among several webpages in the above-mentioned section in the history cluster list.

[0010] In some implementations, generating a metadata repository further involves deduplicating multiple web pages that have been accessed and saved in the browser history, and deduplicating involves determining duplicate web page accesses among the accessed web pages, selecting the web page with the most recent access timestamp with respect to the access timestamp associated with the determined duplicate web page access, and generating metadata to be stored in the repository for the web page with the most recent access timestamp.

[0011] In some implementations, the history cluster has a title, and the method further includes displaying the title as a suggested link, and a request to view the history cluster is issued by selecting the link. In some implementations, the suggested link is displayed as a search suggestion in a search engine, or as an option in the search history of a web browser. In some implementations, we describe a computing device configured to perform one of the methods of any combination of the above features.

[0012] In a second general embodiment, a system is described. The system may include a computing device running a web browser, a renderer, and a web browser user interface generator. The user interface generator is configured to generate a search history user interface based on a history cluster associated with a topic in response to a request to view search activity associated with that topic, the history clusters including multiple web pages that have been accessed and saved in the web browser's search history, and the search history user interface includes a history cluster list and action controls. The history cluster list may include at least a portion of multiple web pages that have been depicted in the web browser's history and determined to be related to the topic, the portion of the web pages including a number of visit lists corresponding to the cluster and search history data from the search history indicating that the portion belongs to a web browser tab group or bookmark. The action controls may be configured to resume previous searches associated with the cluster. The renderer may be configured to display the generated search history user interface together with the portion of the web pages and the search history data.

[0013] The implementation may include one or all of the following features: In some implementations, the history cluster is generated based on metadata associated with the web pages in the above section, and the metadata includes a source event, an access timestamp, an engagement score, and subtopics associated with at least one topic. In some implementations, the engagement score includes the respective engagement metric for the web pages in the above section, and the respective engagement metric is used to select the prominence level when displaying a snippet for at least one web page in the above section in the history cluster list. In some implementations, the user interface generator receives a request to delete a web page rendered in the history cluster list and is further configured to clear the record of access to the stored web page from the search history based on this request.

[0014] In a third general embodiment, the description includes a non-temporary computer-readable medium, which, when executed by at least one processor, includes generating a repository of metadata on the at least one processor based on a plurality of web pages saved in the browser history of a web browser accessed and running on a computing device, wherein the metadata for each web page includes a source event, an access timestamp, and at least one topic, and which includes generating a history cluster based on the metadata, where the history cluster generation is based on the source event and the access timestamp of the web pages in the plurality, and which includes assigning a score to each web page in the plurality, where the score is at least in part based on the access timestamp and an engagement score generated for each web page, and which includes generating a history cluster list for a topic in response to a request to view browser activity associated with the history cluster, where the history cluster list is organized according to its score, and which includes a list of visits associated with the web pages in the plurality that are determined to have a score that satisfies a threshold score, and which includes memory that stores instructions causing an operation to display the history cluster list.

[0015] The implementation may include one or all of the following features: In some implementations, a visit list in the visit list includes a snippet for at least one webpage among several webpages in the above part, and the history cluster includes associated action controls. In some implementations, the history cluster list further includes several visit lists corresponding to at least one webpage among several webpages, the several visit lists are presented in the web browser's browser history, and the history cluster list further includes action controls configured to resume previous searches associated with the several visit lists. In some implementations, the metadata further includes, for a webpage among several webpages, a determined webpage entity, several associated searches, and the time spent on the webpage.

[0016] In some implementations, the history cluster is generated as output of a model (such as a machine learning model or clustering algorithm) running on a computing device that uses metadata as input. In some implementations, the metadata further includes webpage identifiers defined for the webpages in the above collection, which indicate whether each webpage in the above collection is part of a tab group, bookmark, or search results page accessed by the computing device. In some implementations, the generated engagement score includes an engagement metric for each webpage in the above collection, which is used to select the prominence level when displaying snippets for at least one of the multiple webpages in the above collection in the history cluster list.

[0017] In some implementations, generating a metadata repository further includes deduplicating multiple web pages that have been accessed and saved in the browser history. Deduplicating includes determining duplicate web page accesses among the accessed web pages, selecting the web page with the most recent access timestamp with respect to the access timestamp associated with the determined duplicate web page access, and generating metadata to be stored in the repository for the web page with the most recent access timestamp. In some implementations, the history cluster has a title, and the operation further includes displaying the title as a suggested link, and a request to view the history cluster is issued by selecting the link.

[0018] The above systems and embodiments may be configured to perform any combination of the above embodiments, each of which may be performed in any preferred combination of the features and embodiments listed above.

[0019] Implementations of the described technology may include hardware, methods or processes, or computer software on a computer-accessible medium. Details of one or more implementations are described in the accompanying drawings and the following description. Other features will become apparent from this description and drawings, as well as from the claims. [Brief explanation of the drawing]

[0020] [Figure 1] This is a block diagram showing an example history cluster list user interface (UI) relating to the implementation form described herein. [Figure 2] This block diagram shows an exemplary computing system configured to use search activity data and browser history data to generate and add to a user interface (UI) according to the implementation described herein. [Figure 3]A flowchart showing an example of obtaining and analyzing web browser history data for use in generating history clusters according to the implementation forms described in this specification. [Figure 4A] An exemplary user interface showing an example of accessing the history cluster list according to the implementation forms described in this specification. [Figure 4B] An exemplary user interface showing an example of accessing the history cluster list according to the implementation forms described in this specification. [Figure 4C] An exemplary user interface showing an example of accessing the history cluster list according to the implementation forms described in this specification. [Figure 5A] An exemplary user interface showing an example of accessing the history cluster list and interacting with it according to the implementation forms described in this specification. [Figure 5B] An exemplary user interface showing an example of accessing the history cluster list and interacting with it according to the implementation forms described in this specification. [Figure 5C] An exemplary user interface showing an example of accessing the history cluster list and interacting with it according to the implementation forms described in this specification. [Figure 5D] An exemplary user interface showing an example of accessing the history cluster list and interacting with it according to the implementation forms described in this specification. [Figure 5E] An exemplary user interface showing an example of accessing the history cluster list and interacting with it according to the implementation forms described in this specification. [Figure 6A] An exemplary user interface showing an example of modifying the browser history by modifying the history cluster list according to the implementation forms described in this specification. [Figure 6B] An exemplary user interface showing an example of modifying the browser history by modifying the history cluster list according to the implementation forms described in this specification. [Figure 6C]This is an illustrative user interface illustrating an example of modifying browser history by modifying the history cluster list, according to the implementation configuration described herein. [Figure 7] This is a flowchart illustrating an example process for generating user interface content related to browser search activities, according to the implementation described herein. [Figure 8] This figure shows examples of computer devices and mobile computer devices that can be used to implement the techniques described herein. [Modes for carrying out the invention]

[0021] The use of similar or identical reference numbers in various drawings is intended to indicate the presence of similar or identical elements or features.

[0022] Detailed explanation This document describes a browser history organizing tool that can generate a history cluster list that visualizes browser history data as a consistent search process. The process may include searches performed, pages visited, bookmarks created, tab groups created, and / or pages added to tab groups, all related to a specific topic. Such a search process visualization may include machine-organized and machine-classified webpage content (e.g., images, text, links, metadata, etc.). The machine classification of webpage content associated with the search process may be based on user (or browser) search activity, as well as an analysis of search activity related to a specific topic or subtopic that occurs during a particular period and is stored in the search history and / or browser history.

[0023] A history cluster list may include one or more user interfaces generated by analyzing, clustering, and organizing (e.g., classifying, ranking, scoring, etc.) browser history data (e.g., activity) to allow users to easily return to previous search sessions and complete their research without having to start the search again. Machine learning (ML) algorithms are executed on the client / browser side to manipulate browser history data and generate one or more data clusters. In some implementations, the ML algorithms may be executed on the server side (or on another remote computing device) with user consent.

[0024] Traditional search history pages may store search history data and / or browser history data that users manually review and search. However, a technical problem with such traditional search history pages is that the search history content is not presented or accessible in a way that allows users to evaluate the data in terms of search topics and / or search processes. For example, traditional search history pages generally contain links that are sequentially organized by access date and time, are repetitive, and are optimized for electronic processing rather than human comprehension. Furthermore, links related to different search intentions are often interspersed, so that links related to a particular user intention (e.g., a process) are not clearly distinguishable or continuous on the history page. Thus, traditional search history views lack a practical way to relate previous searches and page views together around a specific intention, such as a topic, subtopic, and / or entity, so that users can see a consistent search process. Traditional search history pages and / or browser history pages do not offer any next steps in a search. Moreover, there is no practical way to clear search history and / or browser history on a specific topic, for example, to maintain user privacy and security.

[0025] At least one technical solution to this technical problem is to analyze and organize data on search history pages and / or browser history pages using ML models and algorithms. For example, search history data and / or browser history data may be analyzed using one or more ML models based on user-provided authorizations. Furthermore, one or more ML models may also be employed to generate metadata and data to add to the UI based on metadata input and user-provided authorizations. One or more ML models may run in a local web browser or on the operating system of the computing device running that browser. ML models can be trained for search and knowledge base analysis. In some implementations, ML models may be trained to analyze searches and knowledge bases on the server side, for example, when a web browser user gives authorization to share data with such a server. For example, based on user authorization to do so, the web browser described herein may access user account data on a server (or remote) computing device as part of the process of generating history clusters.

[0026] The ML models described herein can accept browser history data as input to produce output (e.g., data for a history cluster list) that can be added to a new UI in a web browser or to fields in an existing UI that has been accessed. For example, an ML model can be used to generate data to add to a UI, which is configured to present browser history data (e.g., a history cluster list) about the user, analyzed and organized according to a specific topic or subtopic, based on an analysis of the user's previous search history data and / or browser history data.

[0027] The technical solutions described herein can use search history data and / or browser history data to build a database of user web page accesses (e.g., web page visits) and corresponding metadata, all of which can be clustered into topical search processes (e.g., representing clusters of web page visits) using an on-device ML model. In some implementations, search processes (e.g., clusters) can be ranked and indexed for use in presenting the search history data and / or browser history data in the web browser UI. Users can then search and explore the search history data from within the web browser, from search history pages and / or browser history pages, and / or from new windows, fields, frames, controls, and / or web components configured to present the search history data and / or browser history data. Ranking may include, but is not limited to, page ranking, cluster ranking, on-device ML clustering, access timestamp ranking, and / or natural language processing query evaluation.

[0028] Clusters of topical search processes (e.g., search history data and / or search activity and / or browser history data) may be accessed in response to user authorization to use such data and activity information and / or associated data for the purpose of generating a history cluster list. For example, if the user does not give such authorization, the conventional search history page and / or browser history page may be displayed, rendering web pages sequentially organized according to the date and time of access, in response to the user requesting to view the search history page and / or browser history page.

[0029] While in operation, the browser history maintenance tools described herein may generate a repository of web page visits (e.g., accesses) representing the web browser's search history and / or browser history. The repository may include metadata generated and / or retrieved using context-based and content-based data associated with web page content, user search activity, and access timestamps of web page visits. Generally, content-based data is associated with the content being displayed, and context-based data is associated with how the user interacts with the content being displayed.

[0030] Metadata and / or related data may be used to generate history clusters representing a user's search journey. These history clusters may be used to generate and add UI elements to facilitate user interaction with previously performed search and / or browsing activities. For example, history clusters may be used as a basis for generating a history cluster list, which a user can interact with in their browser to view and evaluate previously performed searches in a format organized according to topic similarity and access timestamps. The history cluster list described herein may be interactive and may be accessed to review search data and to continue searches from where the user last viewed content associated with the search.

[0031] In some implementations, the repository includes metadata that is generated, scored, and / or ranked based on ML-based clustering and analysis. For example, the systems and methods described herein can build a database of webpage accesses (e.g., webpage visits). The database may include metadata that is ingested locally with user approval when a user uses a web browser. The metadata may include, but is not limited to, one or more webpage access timestamps, one or more source events, one or more webpage engagement scores, one or more webpage topics, one or more webpage subtopics, search query inputs, one or more dwell times on the webpage, determined webpage entities, and / or search data related to any or all of the accessed web content and / or webpages. In some implementations, the webpage visit repository may be deduped to remove repeated webpage visits so that webpages with the most recent timestamp are retained for the purpose of generating historical clusters and / or historical cluster lists, and so that repeated visits are not considered important and / or ignored.

[0032] The systems and methods described herein can be broadly adapted to a variety of devices, including smaller devices such as mobile devices, tablets, smartwatches, and convertible laptops / tablets, as well as larger devices such as laptops and desktops. Furthermore, the systems and methods described herein can provide organized access to search history data by providing multiple history cluster lists and search fields, which may enable smaller devices to display access to more content, providing greater available space for search content and history cluster lists, since any number of history cluster lists can be collapsed while a single history cluster list of interest is expanded.

[0033] The technical solutions described herein may result in technical effects such as improved search activity management and improved search content management. Such improved management can reduce the computing resources used to access information from previous activity on a computing system (e.g., a device), and facilitate the efficient and secure delivery of such information. Furthermore, the technical solutions described herein may result in technical effects such as improved access to search and / or browser history content, novel UIs enabling new functionality, and improved UI interaction with search and / or browser history content. For example, the browser history organizing tool used to generate the history cluster lists described herein can be broadly adapted to a variety of devices, including small devices such as wearables and mobile devices, as well as larger devices such as laptops and desktops, enabling users to quickly find previously searched content and identify related content in a logical and often automated manner through history cluster lists containing related searches, related content, and suggested additional information, all of which can be organized according to topic. In short, the browser history organizing tool described herein can provide easy access to search and / or browser history content that accurately reflects the user's needs.

[0034] Figure 1 is a block diagram showing an exemplary history cluster list 100 relating to an implementation described herein. The history cluster list 100 may be generated based on user activity in the web browser 102. For example, user activity may include data stored in history 104, which may include timestamped page or content access data, web page data, web activity data, etc. Generally, history 104 may represent web browser history and / or search history, including any combination of browser history data, search history data, download history data, cookies, cached data, and any related search activity associated with the web browser. The data stored in history 104 may be used to generate one or more repositories of metadata 106 and history clusters 108. The data in history clusters 108 and / or repositories may be used to generate the history cluster list 100.

[0035] As used herein, metadata (e.g., metadata 106) may include, but is not limited to, one or more webpage access timestamps, one or more source events, one or more webpage engagement scores, one or more webpage topics, one or more webpage subtopics, search query inputs, one or more dwell times on a webpage, determined webpage entities, and / or related search data. Source events may include events that indicate how a user came to access / navigate to a particular webpage (e.g., a source event is an event that indicates the source of a webpage opened in a web browser). Illustrative source events may include executed, generated, or accessed queries, bookmarks, tab groups, advertisements, search results pages, search history pages and / or browser history pages or other source webpages, null events or attributes, etc. In some implementations, metadata 106 may also include or represent one or more next steps that a user can take based on a user search. In some implementations, metadata 106 may also include or represent user actions on a particular webpage, such as sharing, voice queries, in-page searches, and sharing links by highlighting. In some implementations, metadata 106 may include or represent how the user interacted with one or more webpages. In some implementations, metadata 106 may further include images and associated image data, commercial data, and / or shopping data. In some implementations, metadata 106 may include or represent the reason for page exit (e.g., tab closed, browser closed, etc.). In some implementations, metadata 106 may include or represent page bookmark status, page bookmark timing (e.g., page bookmarked during visit), and / or page addition status per network time protocol.

[0036] In some implementations, the exemplary history cluster list 100 may be presented (or displayed) as part of the search history page and / or browser history page of the web browser 102. For example, the history cluster list 100 may be provided as an interactive UI nested within the search history list (search history page). In another example, the history cluster list 100 may instead be provided as an interactive UI near a conventional search history page or browser history page that enumerates previously accessed web pages, and may be presented without modifying the search history page / browser history page. In some implementations, the history cluster list 100 may be presented in a separate location associated with the web browser.

[0037] Referring to Figure 1, a user can access the computing system 110 and perform a search using the web browser 102. The web browser 102 may generate metadata 106 and history clusters 108 based on the history 104 and / or other search activities associated with the history 104 of the web browser 102. For example, a metadata repository 106 may be generated based on multiple web pages that have been accessed and saved in the history 104 of the web browser 102. Based on the metadata in the generated repository, a history cluster 108 can be generated that contains portions of multiple web pages related to a particular topic. The web browser 102 can use the metadata 106 and history clusters 108 to score and rank the content displayed in the history cluster list 100.

[0038] For example, the web browser 102 may generate a history cluster list 100 based on web activity related to a search (e.g., web activity including the topic of kitchen ranges). In some examples, in response to a request to view search activity associated with a topic, the web browser 102 may generate a history cluster list for that topic. The web browser 102 may analyze the history 104 to determine the search behavior of the user that generated the data stored in the history 104. For example, the web browser 102 may perform an analysis of the history 104 to generate metadata and history clusters representing the data stored in the history 104. In some implementations, the analysis may be performed and stored in real time, for example, when the user performs a search. In some implementations, the analysis is performed by the operating system of the computing system 110.

[0039] Using metadata 106 generated from the history 104 of accessed web pages (e.g., web page visits), a web browser 102 can generate history clusters 108 using any number of ML models. For example, the web browser 102 can use at least the generated metadata 106 as input to generate one or more history clusters 108. For example, an ML model may be configured to generate clusters 108 by identifying a first web page (of a determined portion of accessed web pages) as the top node according to a topic determined to be common to the determined portion. This portion may be determined, for example, based on an analysis of context-based data and / or content-based data associated with the web pages in the history 104. In some implementations, this portion is selected based on its similarity to a specific topic defining the web page (or some similarity to another web page). Similarity may be used as the basis for scoring web pages. Web pages may also be scored according to other criteria associated with the metadata, such as access timestamps. Scores may also be assigned to web pages based on an engagement score (e.g., web page engagement score) for each web page. The engagement score may form part of metadata 106, or it may be generated separately. In that case, this part may include specific web pages that meet or comply with a certain threshold score.

[0040] The ML model may then be configured to assign the remaining web pages from the above portion to specific clusters, which may relate to, for example, a topic or a subtopic of a topic, based on the metadata 106. In some implementations, the ML model may also take into account the access timestamps of the web pages to prevent web pages (e.g., web page visits) that exceed a certain access timestamp from being selected for a cluster or from being displayed in the history cluster list 100.

[0041] As shown in Figure 1, the history cluster list 100 includes several clusters 108A, 108B, 108C, and 108D. Each cluster contains any number of web page visits, controls, associated searches, links, images, and / or text. For example, cluster 108A contains web page visit 112. Cluster 108B contains web page visit 114 and additional web page visits 128, 130, and 132. Cluster 108C contains web page visit 116. Cluster 108D contains web page visit 118. Each web page visit 112, 114, 128, 130, 132, 116, and 118 corresponds to a user's visit (e.g., access) to a specific web page during their respective period. When generating a specific display order for accessed web pages, additional data and / or metadata, such as time spent and page entities, may also be considered.

[0042] A webpage visit (e.g., visits 112, 114, 128, 130, and 132, or other associated webpages in the cluster) may relate to actual webpage visits stored in the browser's search history, but may be presented as a visit list in the UI. The visit list may include webpages, but may also refer to searches, related webpages or content, or other data that may be associated with or otherwise deemed relevant to the original webpage visit.

[0043] Each webpage visit may relate to a topic or subtopic associated with topic 120 in the history cluster list 100. For example, each webpage visit 112, 114, 116, and 118, as well as 128-132 (and / or additional visits not depicted), may correspond to a specific matching topic (e.g., topic 120 shown as "kitchen range" in this example). For example, webpage visit 114 may correspond to topic 120 that, in some implementations, may match a specific part of the keywords in a search query. That is, the web browser 102 may determine that visit 114, representing a particular webpage access, is the top visit 124 in history cluster 108B, and that visits 128, 130, and 132 also have some similarity to topic 120. In some implementations, the top visit 114 represents a search query, which may be the source event that triggered the results and subsequent webpage visits by the user, such as webpage visits 128, 130, and 132. In this example, if the top visit 124 is the top visit related to cluster 108B, then visits 128, 130, and 132 may be nested webpage visits related to the search query source event. Therefore, visits 128, 130, and 132 may be generated as a visit list by the computing system 110 so that they appear in the history cluster list 100 based on the determined score of each associated webpage visit. Other webpage visits may have occurred with respect to the source event (e.g., the top visit 114 associated with cluster 108B), but none of these visits received a score high enough to be enumerated as a visit list representing visits 128, 130, and 132 in the history cluster list 100. In some examples, the history cluster list may be organized according to their respective scores and may include visit lists associated with webpages in history clusters that have been determined to have scores that meet a threshold score. Meeting a threshold score means that the score meets a threshold.

[0044] The UI content, web pages (e.g., visitor list), and controls shown in the history cluster list 100 generally relate to the same topic 120, but any number of topics can be used to generate clusters for the history cluster list. Therefore, in some implementations, multiple clusters can be generated for multiple topics. Such clusters may be ranked based on metadata and / or a user-selected filter or ranking scheme and presented in the history cluster list.

[0045] Web page visits in cluster 108B may have occurred according to the time and / or date of access timestamp 134, or may have occurred over a general period of time. Generally, each visit (e.g., web page access) includes an access timestamp, but in a particular historical cluster list, a general range of access timestamps may be depicted to account for any number of visits that occurred with respect to the web pages associated with the historical cluster list. In some implementations, clusters may include associated access controls. For example, access control 126 may be provided to view additional clusters and / or visit data nested under visit 128. The user can select access control 126 to expand a list of additional content items related to the topic of cluster 108B.

[0046] Generally, the timestamps of web page visits selected for a cluster can be correlated with search activity associated with cluster 108B to ensure that relevant (e.g., recent) data is selected for the visit list in the history cluster list 100. This ensures that outdated search content does not appear in the history cluster list generated based on the generated clusters.

[0047] In some implementations, a cluster may include web pages relating to additional subtopics or related data, such as related search 138 and related search 140. Related searches 138 and 140 are presented as relating to the content, topic, or subtopic of cluster 108B, and each includes access controls (e.g., buttons, links) selected to access additional content. The history cluster list 100 may show additional queries and / or searches relating to web activity and / or accessible clusters 108A-D, such as related search 138 and related search 140. Furthermore, controls may be provided for viewing more content, web page visits, suggested queries, etc., as indicated by control 142.

[0048] Although not shown in Figure 1, any number of history clusters 108 (e.g., 108A-108D) may be generated and depicted as part of a history cluster list 100. Furthermore, any number of web page visits (e.g., depicted as a visit list) may also be generated and / or displayed together with such history clusters in a hierarchy determined based on metadata 106 taken from the browser search history data described herein.

[0049] Figure 2 is a block diagram of an exemplary system 200, which includes a computing system 202 configured to generate a user interface (UI) and use search activity data and search history data to add to it, according to the implementation described herein. System 200 may be configured with computing devices (e.g., computing system 202 and / or server computing system 204) and / or other devices (not shown in Figure 2) to generate a history cluster list 100 based on history clusters generated by a computing device that renders a history cluster list within a web browser, for example.

[0050] The computing system 202 can annotate specific web pages browsed by consumers (e.g., accessed web pages) (e.g., using metadata) to verify content-based and context-based metadata. The system 202 can then define topics, subtopics, etc., as metadata annotations. The annotations and associated web pages may then be clustered into history clusters, which can be deduppled and used as the basis for generating a history cluster list according to topics or subtopics, access timestamps for the pages, and / or source events. The history cluster list may be displayed within a UI and / or field associated with the web browser.

[0051] As shown in Figure 2, the computing system 202 includes an operating system (O / S) 206 capable of executing and / or otherwise managing an application 208 based on application information 210 and / or session data 212. The O / S 206 may function to execute and / or control the application 208, UI interactions, accessed services, and / or device communications not depicted in Figure 2. For example, the O / S 206 may enable a web browser 214 to use the resources of the system 202 to perform web browser functions. The computing system 202 may include one or more modules or engines representing specially programmed software, such as those described in the algorithms and methods described herein.

[0052] The web browser 214 represents a web browser configured to access information on the Internet. The web browser 214 can launch one or more browser processes (not shown) to generate, modify, and / or otherwise configure the browser repository 216. Furthermore, the web browser 214 may be configured to generate browser content, generate search history data, and / or perform other browser-based actions. The web browser 214 may also store browser tab / tab group data 218, browser history 220, history clusters 222, and metadata 224.

[0053] Browser history 220 may contain data representing user activity within a web browser. For example, browser history 220 may include previously accessed web pages 221 accessed by browser 214. Browser history 220 and accessed web pages 221 may be presented on the browser's search history page and / or browser history page. Search history pages and browser history pages are generally organized sequentially by web page access date and timestamp, are iterative, and contain links optimized for electronic processing rather than human comprehension.

[0054] Using the browser history 220, the search history analyzer 234 can use the browser history 220 to ingest and / or generate specific data. For example, the analyzer 234 can retrieve (and / or generate) metadata 224 from the browser history 220. In some implementations, the metadata 224 may be generated based on the browser history 220 and / or another database available to the web browser 214. As shown, the metadata 224 includes an access timestamp 238, an engagement score 240, a topic 242 (and subtopics), a score 244, and a source event 245.

[0055] Generally, the search history analyzer 234 may run in a web browser 214 using one or more browser processes (not shown). The analyzer 234 can use metadata 224 to generate content-based annotations (e.g., content-based metadata) and context-based annotations (e.g., context-based metadata). For example, content-based annotations may include categories for a particular webpage (e.g., non-local public or private webpages). Content-based annotations may also include indicators that a webpage contains specific content (e.g., user data, public data, specific types of data). Content-based annotations may also include indicators of which entities are associated with (or present on) the webpage. Content-based metadata represents information describing the actual content on a webpage (e.g., images, text, symbols, etc.) or a specific topic or subtopic describing the webpage. Context-based metadata represents information describing how a user engaged with and / or otherwise interacted with the webpage.

[0056] Generally, content-based annotations can be obtained by using a document object model API to capture page content associated with the web pages visited by the user via the browser process. In some implementations, ML inference tasks may be performed to annotate the output from the API and to update the history cluster 222 when the annotation task is complete. As used herein, the history cluster 222 may include search history clusters, history clusters, or other clusterings of browser data related to user interactions in the web browser 214.

[0057] In some implementations, context-based annotations may include URL-keyed anonymized data retrieved from the web browser 214 based on user authorization to access such data. The data may include the URL of the web page the user is visiting. Additional context-based annotations may also include indicators of tab group status (e.g., whether the web page is in a tab group and the timestamp when the web page was placed in the tab group). Context-based annotations may also include indicators of whether a particular web page is a bookmark, a favorite bookmark, a frequently accessed bookmark, or a new bookmark. Context-based annotations may further include indicators of whether a particular web page is a shortcut, a frequently accessed shortcut, or a new shortcut. Context-based annotations may also include indicators of the duration since the most recent visit to the URL associated with the particular web page or bookmark. Context-based annotations can be obtained using a browser history 220 and an ML model that clusters the browser history 220 according to topics, subtopics, scores, source events, web page access timestamps, or other retrieved metadata 224, and / or any combination of the above.

[0058] Context-based annotations or metadata may also include context signals indicating how the user navigated the web page. In some implementations, context-based annotations (e.g., metadata) may be generated based on clipboard actions associated with the web browser 214. For example, the web browser 214 may be configured to monitor for explicit copy operations (e.g., Ctrl-C or other inputs to trigger a copy operation). For example, the web browser 214 may detect when a copy action has been invoked, such as in the URL toolbar, in a document within the browser, or in a search field. In some implementations, the web browser may also monitor for sharing operations that may extend beyond copying to the clipboard. For example, the web browser 214 may detect sharing of web content via upload, link sharing, printing, text messaging, etc. Context signals may also indicate, for example, scrolling or resizing the web page being viewed.

[0059] Content-based and context-based annotations may be used by the analyzer 234 to generate descriptive search routes for specific search sessions, users, topics, and / or for several search sessions (for example, to generate one or more history clusters 222). Parts of the search route and / or metadata associated with the search route may be used as the basis for generating content for the history cluster list 228. In other words, the history cluster list 228 is generated based on the clusters 222, which are generated based on the metadata 224 (for example, content-based and context-based annotations).

[0060] A search process can include any or all of a specific topic-centric user activity in a web browser, such as searches performed, pages visited, bookmarks created, or pages added to tab groups. For example, a search process may include user activities performed that are determined to be associated with a particular topic (or subtopic), and these activities are performed within a predetermined time interval that allows the user to return to and resume searching for that topic or subtopic.

[0061] The search history analyzer 234 may determine that the browser history 220 relates to several specific topics. Each of these topics (and / or related subtopics) may be used to organize previously visited web pages. That is, various aspects of the search process (i.e., topic-centric user activity in the web browser) related to a given topic can be used to generate visually appealing clusters of related web page visits (e.g., accesses), UI data, and controls for accessing information about the topic. The search history analyzer 234 may use the ML model 236 to generate such clusters 222. The score / rank generator 232 may be used to organize the web pages within the clusters 222. Any number of topics can be represented in the browser history 220, and therefore the analyzer 234 can generate any number of history clusters 222 based on such topics. In some implementations, multiple clusters 222 can be generated for multiple topics. Such clusters can be ranked and presented in the history cluster list.

[0062] In some implementations, the search history analyzer 234 can use one or more ML models 236 to automatically analyze conventional search history data (e.g., browser history 220) into history clusters 222, which can then be used to associate previously performed searches and page views / visits together, centered around specific topics, subtopics, and / or page entities. As a result, a consistent search path and any next steps for the search can be presented to the user. The presented consistent search path and next steps may take the form of a search history UI, such as a history cluster list 100. The browser repository 216 can generally be used in any combination, with user approval, to generate history clusters 222 and any resulting history cluster list 228 based on the history clusters 222.

[0063] For example, with user approval, system 200 may generate a history cluster 222 that may represent a search process performed on the web browser 214. The history cluster 222 may be generated based on browser history 220 collected by the web browser 214, and / or other data obtained from (or generated for) the browser repository 216. The history cluster 222 may represent the basis for generating one or more history cluster lists and displaying them to the user of computing system 202.

[0064] In some implementations, the history cluster 222 is generated as the output of one or more ML models 236 running on a local computing device. The ML model 236 may take metadata 224 as input. The ML model 236 can be configured to identify a first web page in a part as the top node according to its topic, and based on the metadata, it can assign a web page from the remaining web pages in the part to a subnode (for example, a web page in a particular cluster that is identified as being associated with the top node or top visit).

[0065] The generation of history clusters 222 may include identifying and grouping accessed web pages 221 (e.g., from browser history 220). For example, accessed web pages 221 may be grouped based on similarity to a determined topic, the top cluster node, or metadata 224 associated with one or more of the accessed web pages 221. For example, a computing system 202 may iteratively process the metadata 224 associated with each individual web page access / visit.

[0066] In some implementations, the history clusters 222 may be generated by one or more ML models 236 using entity tagging to attach a vector of keywords (e.g., topics, subtopics, entities on the page, etc.) to each cluster. These keywords may be used as metadata and may not be provided to the user during browsing, but they may be used to generate the clusters 222. For example, the ML model 236 may be configured to compute entities that are determined to be (or probabilistically determined to be) included in a particular web page content.

[0067] Page entities can be an example of metrics used to generate UI content relevant to a user's search session (e.g., content in a history cluster list) when a user wants to return to a session and continue searching for a topic and / or subtopic. Since users typically accomplish tasks over multiple sessions (e.g., days, weeks, etc.) in a web browser, using knowledge about web page entities related to specific web pages can result in accurate clustering of web page visits. For example, instead of having to recall the title of a particular web page previously visited, the user might be able to enter a query about the entities of that page, which can improve the user's visual and content-based experience and allow system 202 to link search journeys across multiple browser sessions. Using these linked search journeys, a history cluster list can be generated to help users review previously searched content and evaluate additional relevant topics and the next steps in their search.

[0068] During operation, system 202 may retrieve metadata about identified entities of one or more accessed web pages (e.g., web page visits) according to the browser history 220. For example, system 202 may retrieve the entity name and the category to which the entity belongs for each accessed web page. Similar to annotating entities in documents for a knowledge graph, system 202 may locally annotate the web page entities of visited web pages within the web browser 214, as described in detail above. One or more ML models 236 can use the entity metadata and any other metadata to generate clusters of web page accesses.

[0069] In some implementations, ML model 236 may employ any number of neural networks (not shown) to perform machine learning tasks on system 202. Neural networks may be used with ML model 236 and / or in conjunction with operations to predict and / or analyze search history data to generate and / or update search clusters. In some implementations, processing tasks associated with web page visits and metadata are called machine learning (ML) inference operations. An inference operation may refer to a processing operation, step, or substep involving an ML model that makes (or leads to) one or more predictions. Certain processing performed by system 202 may use ML models to make such predictions. For example, machine learning may use statistical algorithms that learn data from existing data to make decisions about new data; this is a process called inference. In other words, inference refers to the process of using a pre-trained model and making predictions using that trained model. Some examples of inference include topic or subtopic recognition, web page activity recognition, and / or cognition of such web page activity or user activity related to that web page.

[0070] In some implementations, the ML model includes one or more neural networks. Such networks can transform inputs received by an input layer through a series of hidden layers and produce outputs through output layers. Each layer consists of a subset of a set of nodes. Nodes in a hidden layer are fully connected to all nodes in the previous layer and supply their respective outputs to all nodes in the next layer. Nodes in a single layer function independently of each other (i.e., they do not share connections). The nodes in the output layer supply the transformed inputs to the requesting process. In some implementations, the network used by system 202 includes a convolutional neural network, which is a neural network that is not fully connected.

[0071] In general, system 202 may employ networks and classifiers (not shown) to generate training data and to perform ML tasks. For example, processor 256 may be configured to run a classifier that determines whether a particular web page relates to a certain topic or subtopic, and whether such web pages can be combined and / or selected to be presented with a particular list of historical clusters. Furthermore, processor 256 may be configured to run a classifier that further determines whether a particular web page belongs to a certain cluster based on a topic / subtopic or other metadata associated with a particular web page. The ML model 236 may include at least candidate generation, ML confidence ranking, heuristic scoring, and metadata analysis. For candidate generation, system 202 may use a predefined set of entity names hashed together in machine-readable code. System 202 can then perform substring matching on a particular web page visit to determine possible annotations about the web page visit.

[0072] After candidate generation is complete, system 202 can send the candidates and hashed entity names to the ML model to calculate confidence scores for those candidates. The ML model can be trained, for example, using a public web corpus over a certain time frame (e.g., daily, weekly, monthly, etc.). During training, if evaluation determines new information, the content from the corpus can be re-annotated with such information and / or labels. After determining the confidence scores, any number of heuristic scores can be applied, for example, to resolve ambiguity between similar annotations.

[0073] The confidence score may enable the ML model (or browser 214 and / or O / S 206) to rank web page visits with respect to specific metadata and / or heuristic scores. In some implementations, additional classification may be performed. For example, the browser 214 may classify the entities of page content on accessed web pages from the browser history 220. In some implementations, the web browser 214 may add new annotation fields for each web page visit (e.g., access) indicating entities, keywords, topics, subtopics, etc. Such annotations may be stored locally on the computing system 202 associated with the browser 214.

[0074] The annotations (e.g., metadata 224) can then be used to generate a history cluster 222. For example, the ML model 236 may capture localized category names and entity names for each web page identifier (e.g., access timestamp of a page visit) that has been determined to be clustered. The categories, entity names, and web page identifiers can be used as input to the ML model 236 to determine the similarity of entities between different web page visits within the cluster 222, as well as the similarity between clusters in order to group specific clusters (and / or related content) together. The determined categories and entities can then be stored locally on the computing system 202 associated with the browser 214. The clusters can then be queried, and the data of the cluster 222 can be surfaced on various UIs associated with the browser. In some implementations, the ML model 236 may use any or all of the signals of the metadata 106, as described throughout this disclosure.

[0075] One ML task that ML model 236 can perform is to generate history clusters 222 using metadata obtained and / or generated based on browser history 220. Each cluster 222 represents several web page visits (e.g., accessed web pages 221). ML model 236 can obtain (or generate) metadata 224, which can be used as input to one or more ML models 236 to make decisions about web page content and web page context. For example, system 200 can determine topics related to a web page (e.g., keywords, categories, etc.). Furthermore, system 200 can determine contextual features that indicate the recency of activity or access to a web page. For example, metadata 224 may include access timestamps for one or more web pages and engagement scores for interactions with one or more web pages. Engagement scores may be calculated per page. In some implementations, a total engagement score can be calculated using all engagement scores related to a particular topic or subtopic. Furthermore, the number of page loads during a certain period before and after a visit to a specific webpage can also be calculated. The number of unique URLs visited during a specific period can also be determined.

[0076] In some implementations, ML model 236 may be trained using URL-keyed metrics and data collected based on user authorization. The actual processing of the data (e.g., web page accesses, browser search inputs, search history data) may be performed on a local computing system 202 to ensure the data is secure and private. ML model 236 is generated to run based on specific topic relevance and browser behavior depending on a specific platform or device type.

[0077] During operation, the search history analyzer 234 may generate each history cluster 222 by analyzing search process data, such as web page accesses (e.g., visits), search queries associated with the searches performed, tab group status for a particular web page visit, and / or bookmark status for a particular web page visit. Generally, each history cluster 222 may represent a portion of any number of accessed web pages 221 in the browser history 220. The portions of web pages may be defined and / or organized according to topic, subtopic, and / or access timestamp.

[0078] In some implementations, the ML model 236 may perform clustering on demand (e.g., in real time) to generate the most recent clusters based on the browser history 220 and / or related data. Thus, when a user accesses the history cluster list, the system 202 can retrieve the cluster content to add to the content (e.g., cluster 108B in the history cluster list 100). Furthermore, as the user scrolls and views more content, additional clusters are retrieved and used to add to further clusters. Since a list of clustered content is added to the history cluster list, clusters may include web page visits accessed at a later time than, for example, the first cluster list in the history cluster list 100 (e.g., cluster 108B). This real-time approach can reduce the processing resources that might be used to perform clustering at regular intervals in the background. Thus, the use of computing resources can be improved.

[0079] In some implementations, additional web page visits are captured in response to requests to view additional content (e.g., scroll, show more controls, etc.). ML Model 236 can then generate clusters in real time based on the captured web page visits. In some implementations, generated clusters are not maintained once the user navigates away from the history cluster list. This can reduce the demand on data storage capacity, which may be limited on mobile computing devices. In some implementations, generated clusters are maintained based on a user request (or authorization) to do so.

[0080] In some implementations, the ML model 236 is executed by the browser process of the web browser 214. For example, each ML model 236 can take in metadata and / or related data associated with the browser history 220. The taken-in metadata and / or related data may include text and / or images, which can be parsed, for example, split into words based on whitespace, and then tokenized. Each word is compared to a fixed-size word dictionary and mapped to a predetermined unknown token. A fixed-size integer vector is then determined and can be used as input to the ML model 236. The output of the model 236 may include a fixed-size vector based on the number of categories used to train the model 236.

[0081] Each webpage visit may be analyzed in this manner, according to the webpage's content and context, in order to generate new metadata on the device (e.g., on the computing system 202) to be stored along with metadata 224. An anonymized version of the new metadata can be used as training data for future model iterations. In some implementations, such data collection is not performed for anonymous browsing sessions. Furthermore, users have the option to clear such data on their devices by clearing their browser history.

[0082] In some implementations, the history cluster list 228 may depict clusters 222 in reverse chronological order based on access timestamps and / or ranges of access timestamps. The history cluster list 228 may be presented within the browser 214 along with other UI controls, including but not limited to search toolbars, page links, and menus. If a user wants to search for a specific cluster 222, they can use the search toolbar within or near the history cluster list 228 to enter keywords that may match the cluster 222 in order to retrieve (or generate) the specific history cluster list 228.

[0083] In particular, history clusters 222 may be generated by one or more models 236 using entity tagging to attach a vector of keywords (e.g., topics, subtopics, etc.) to each cluster. When a user searches for a query in the UI of the history cluster list 228, or within the search history page, browser history page, and / or search toolbar, the system 202 can find clusters containing such keywords using string tokenization and full and / or prefix string matching against a predefined list of keywords. In some implementations, the tokenized query may also be matched against the URL of a particular cluster 222 and / or the respective title associated with the cluster 222. Substring matching, fuzzy logical string matching, and / or other matching techniques may be used. Furthermore, additional metadata other than keywords may be used to evaluate matching to a particular cluster.

[0084] In some implementations, it may be ensured that users can manage their browser history by running ML model 236 locally in the browser. Unlike traditional systems, the UI generated by system 200 (e.g., system 202 or system 204) (e.g., a history cluster list depicting clusters) is configured to group and organize webpage visits (e.g., accesses) to allow the user access to a curated UI that can be used to quickly clear groups of navigation. This feature may provide the ability to precisely clear parts of the search history without having to manually find and clear webpage visits one by one, as is the case with traditional browser and / or search history webpages. For example, by selecting an action control associated with the history cluster list, it may be possible to perform an action to clear the search history and / or browser history of the associated cluster. Facilitating such precise clearing of parts of history can improve user privacy and security, especially when other users may have access to the same computing system.

[0085] In some implementations, the ML model 236 is executed by the server computing system 204 when the user has given authorization to use the browser history 220 on a remote device, such as the server computing system 204. For example, if the user has given authorization to share data between user accounts by saving the data on a cloud server or other server device, the system 200 can retrieve all browser search history for the user account because the system 200 can store the browser history data remotely. The remote browser search history is retrieved on demand and integrated with the local browser search history in the local history. In some implementations, the remote browser search history is updated periodically along with the local browser search history, in which case integration is not performed.

[0086] At some point, a user may request to view their search history (or a portion of their search history). System 200 can generate any number of history clusters 222 based on (local and remote) browser history 220, and can use the clusters 222 to generate a history cluster list 228. For example, System 202 can generate clusters 222 locally on System 202 using an ML model 236 and use this to add to the history cluster list 228. The history cluster list can be displayed to the user in response to a request. In some implementations, a server computing system 204 can instead use an ML model on the server side to generate the clusters and / or history cluster list and send the history cluster list to the user in response to a request.

[0087] Each history cluster 222 may include a determined top webpage visit (e.g., top node) that represents a very prominent visit within the cluster 222. The top node can be determined, for example, by the search history analyzer 24 and the score / rank generator 232 and may function to represent a cluster. For example, the top node may be a node to which other webpage visits (e.g., accesses) are mapped. If the top node represents a topic, the mapped webpage visits may represent the same topic or subtopic. In some implementations, the top node is a list that is prominently presented in the history cluster list 228, and the rest of the webpage visits, some of the webpages, are nested and / or indented from the top node. The nested and / or indented webpage visits may represent related webpage visits.

[0088] The top node may be determined based on a score 244 generated for each webpage visit. The score may be based at least in part on the access timestamp and the generated engagement score for each webpage. The score 244 may be generated using, for example, an ML model 236. The score 244 for a particular webpage visit may be calculated based on metadata 224, which includes, for example, the determined engagement with the webpage being visited (e.g., an engagement score 240 representing the length of time the webpage tab was in the foreground). Additional heuristics may be used to identify the top node. For example, a navigation graph may be generated and evaluated in combination with the engagement score 240 for each spoke of the graph. In some implementations, the score 244 is based on the access timestamp 238 and metadata 224 of the webpage visit. For example, by scoring a particular webpage visit using the access timestamp 238, it may be ensured that a recent visit to a webpage may be scored higher than an earlier visit to that webpage. This can improve any resulting list of historical clusters 228 based on a particular cluster 222 by avoiding the use of old or less relevant website visits (based on timestamps).

[0089] By generating and assigning a score 244 for each webpage visit, the score / rank generator 232 can rank webpage visits related to a particular topic and / or subtopic according to the score 244. For example, webpage visits within a history cluster 222 are in descending order based on the score 244 of each visit. For example, the webpage visit with the highest score is selected as the top webpage visit (e.g., top node), while the remaining webpage visits are identified as related webpage visits. In some implementations, the UI generator 226 may be triggered to hide certain related webpage visits based on whether their score is below a predefined threshold. In some implementations, the UI generator 226 may be triggered to hide certain related webpage visits if such visits are below a certain rank, for example, to reduce clutter in the resulting history cluster list 228. User controls may be provided by the UI generator 226 within or near the content in the history cluster list to allow the user to switch such hidden webpage visits to a visible state.

[0090] Source Event 245 can include events that indicate how a user came to access / navigate to a particular webpage. Examples of Source Event 245 include executed, generated, or accessed queries, bookmarks, tab groups, advertisements, search results pages, search history pages, other source webpages, null events, or attributes. Such source events can be used to generate history clutter. For example, the same or similar source events might indicate that webpages related to a topic should be included in a history cluster.

[0091] Ranked webpage visits can be evaluated by the search history analyzer 234 to generate a history cluster list 228 of webpage visits viewed in the browser. For example, the search history analyzer 234 can generate a history cluster list 228 for a particular history cluster according to access timestamps 238 associated with a specific webpage visit (e.g., access) within one or more determined subtopics and / or clusters. The history cluster list 228 may include webpage visits that are associated with the cluster and have been determined to have a score that meets a threshold score. Furthermore, the history cluster list 228 may also include several search history events corresponding to the webpages (e.g., webpage visits / accesses) in the history cluster list. Examples of search history events include search results, parts of search results, images responding to the entered search query, relevant search recommendations, relevant search queries, summarized search results, and summarized webpage visits.

[0092] In some implementations, the search history analyzer 234 can dedup web page visits. For example, web page visits associated with a particular cluster may still be represented in metadata 224 even if the visit is deduped (e.g., duplicate content is removed from the UI view). For example, metadata about duplicate content is retained in metadata 224, but the system 200 may dedup web page visits (e.g., accessed web pages 221) saved in the browser history 220. Deduplication may involve determining duplicate accessed web pages 221 among multiple accessed web pages (i.e., within the browser history 220). In some implementations, accessed web pages 221 represent the search history of the web browser 214, which includes a list of all web pages accessed over a certain period of time.

[0093] Next, for the web page with the most recent access timestamp, system 200 may select the web page with the most recent access timestamp relative to the determined duplicate web page access in order to generate metadata 224 to be stored in the repository. Deduped content may remain stored, but may not be used for the purpose of generating the history cluster list 228. In some implementations, deduped content may be deleted from the data of the history cluster 222. In some implementations, deduped content may be indicated as deprecated content. Any number of clusters can be deduped in a similar manner.

[0094] In some implementations, the search history analyzer 234 may generate relevant search queries for each cluster. These relevant search queries may be provided to be displayed in the history cluster list. For example, the history cluster list 100 includes a search query related to "dual-fuel range" in visit 124, and additional relevant searches 138 and 140 are depicted and shown as relevant. In some implementations, relevant search queries may be taken from search results pages associated with a particular search query. Such relevant search queries can be analyzed and stored in the metadata repository as metadata 224, indicating that they are associated with a specific webpage visit. When displaying dates associated with a history cluster 222 in the history cluster list, for example, relevant search queries may be aggregated from multiple webpage visits or selected from the highest-ranked webpage visits. Such aggregated and / or selected content may be prominently displayed, for example, in the history cluster list UI, and may also be displayed under the determined top node of a particular history cluster. In some implementations, to minimize the obsolescence of relevant search query suggestions, relevant searches from recent web page visits replace relevant searches from any preceding, overlapping web page visits.

[0095] In some implementations, the search history analyzer 234 may also annotate webpage visits according to whether the webpage is part of a tab group or bookmark. Such annotations may be shown later in the history cluster list as badges or other indicators to help the user identify highly ranked / or very relevant webpage visits during a particular search journey. In some implementations, annotations can be used to ensure that a particular webpage visit is associated with a particular cluster 222.

[0096] As shown in Figure 2, the browser 214 includes a UI generator 226 that functions to generate a user interface, such as a history cluster list 228, either within a browser field or as part of a browser frame or web page. The history cluster list 228 may represent organized search activities displayed within the UI, as shown in the illustrative history cluster list 100 (Figure 1). In some implementations, the history cluster list 228 may include analyzed and organized search history information, search activity information, and related search result information, depicted according to scores and rankings determined (e.g., performed by a score / rank generator 232) using metadata (e.g., metadata 224) corresponding to activities performed in the web browser 214, a specific browser history 220, etc. In some implementations, the history cluster list 228 may be organized within the UI layout according to topics and / or subtopics 242, engagement scores 240, timestamps 238, and / or other scores 244.

[0097] In some implementations, the history cluster list 228 may include controls and / or indicators to allow the user to return to performing additional investigations and / or searches associated with a particular history cluster list. For example, system 200 may provide an access point represented as a UI element configured to allow re-entry into a search on a search history page or a web browser's browser history page. In some implementations, system 200 may provide an access point represented as a UI element configured to allow re-entry into a search on a browser address toolbar (or, for example, another search control). In some implementations, system 200 may provide an access point represented as a UI element configured to allow re-entry into a search at an entry point associated with a browser address toolbar (or another search control), and / or as a menu or control associated with such an entry point. For example, system 200 may enable a search re-entry option in a browser menu, bookmark menu, tab group menu, etc.

[0098] During operation, the UI generator 226 can display data based on performed searches, content item analysis, search history analysis, and / or other processing activities related to the web browser 214. The UI generator 226 can utilize a renderer 246 that can render UI content such as browser windows, browser tabs, search history data, history cluster lists 228, browser tab groups, browser tabs, and content associated with such elements. The renderer 246 may work with a display (not shown) associated with the computing system 202 to, for example, depict user interface objects or other content to a user of the computing system 202. The renderer 246 may be triggered to render content from the UI generator 226, the score / rank generator 232, the search history analyzer 234, the browser 214 or associated browser process, the application 208, and / or other inputs or outputs received by the computing system 202.

[0099] In some implementations, the computing system 202 may be running a browser session and can receive requests to display search activity (e.g., associated with web browser history content). If the system 202 determines that a user accessing a web browser on the computing system 202 has given authorization to access browser history data, the system 202 can trigger the UI generator 226 to retrieve specific browser history data and generate and render a list of history clusters based on the search data associated with the generated history clusters.

[0100] For example, the UI generator 226 can request, retrieve, and / or receive history clusters representing web pages that have been accessed and saved in the browser history 220 of the web browser 214. Such web pages may be ranked, for example, based at least in part on the metadata associated with each web page in the history cluster 222. The UI generator 226 can use the cluster 222 to generate a search history UI, for example, a history cluster list 100 or other UIs that depict search history data based on the browser history 220.

[0101] A request to display search activity may be associated with a specific topic, in which case the request may trigger the display of a search history UI with content, web pages, and search activity related to that specific topic added. In some implementations, the search history UI may also include subtopics related to the topic and / or web pages related to those subtopics.

[0102] In some implementations, a history cluster list (e.g., history cluster list 100) is generated and displayed within the web browser's search history page. The history cluster list may include at least some of several web pages determined to be related to at least one topic. For example, visits 128, 130, and 132 may represent previously accessed web pages related to a range of topics in a binary fuel formula, as indicated by top visit 124. Visits 128, 130, and 132 may also include web pages nested under the visit that represent additional subtopics or web pages related to visits 128, 130, or 132. A cluster may include other content, including but not limited to visit lists containing any or all of additional web pages, search results, related searches, shopping content, advertisements, etc.

[0103] In some implementations, visits 128, 130, and 132 may represent (or include) a list of visits corresponding to a topic, and the order in which visits 128, 130, and 132 are displayed may be based on search history data and / or metadata obtained from the first webpage visits 128, 130, and 132. In some implementations, the search history data and / or metadata may relate to source events associated with one or more webpage visits, access timestamps associated with one or more webpage visits, engagement scores associated with one or more webpage visits, and / or at least one topic and subtopics associated with one or more webpage visits. In some implementations, source events may include click events, bookmark events, tab group events, copy events, access events, etc.

[0104] An engagement score may include engagement metrics for one or more web pages represented during a web page visit (e.g., access). Engagement metrics may be used, for example, to determine and / or select a prominence level when displaying a snippet (e.g., "Top 10 Binary Fuel Ranges" for visit 130) in the Historical Cluster List 100 for at least one web page (e.g., "tribune.com") among several web pages in a subset of multiple clusters depicted in cluster 108B or in the Historical Cluster List 100. Engagement metrics may be based, for example, on the foreground and / or background duration of a web page. The duration a web page is in the foreground may be used as a surrogate for user engagement. For example, an engagement score may be based on foreground duration. Additionally, or alternatively, the duration a web page is open in the background (rather than being closed by the user) during an active browser session may also be an indicator of user engagement with that web page.

[0105] In some implementations, search history data from browser history 220 may indicate that a particular webpage or part of a webpage belongs to a tab group or bookmark in the web browser 214. This metadata can be used to increase or decrease visits to specific webpages and / or cluster content within a history cluster list.

[0106] In some implementations, the history cluster list also includes one or more action controls configured to resume previous searches associated with multiple search results. For example, control 143 indicates that the user can select this control to resume the search process related to web page visit 114 and the “dual-fuel range” search. Other action controls are, of course, possible. An action control may be a control button or a selectable UI element configured to perform an action associated with the history cluster list when selected.

[0107] The UI generator 226 may include and / or have access to the renderer 246, which may be configured to display any generated search history UI (e.g., a history cluster list 100, a filter section 532, a dropdown 538, and / or other UI presented together with the search history data). In some implementations, the search history UI may be rendered using at least some of multiple web pages and search history data ordered based on metadata.

[0108] In some implementations, the UI generator 226 may be configured to receive a request to delete a webpage rendered in the history cluster list, as described in detail with respect to Figures 6A to 6C, and to clear the record of access to the webpage stored in the search history or browser history based on this request.

[0109] In some implementations, system 202 is a computing device that runs a web browser. System 202 may include a user interface generator for the web browser. The user interface generator may consist of instructions for generating a search history user UI (e.g., a history cluster list 100, etc.) based on a history cluster (e.g., 108B) associated with a certain topic (e.g., topic 120). The generation of such clusters may be performed in response to a request received by the browser to view search activities associated with a topic. For example, a user may access web browser 214 and choose to view previously performed search steps (by the user) related to a topic. The history cluster 108B may include multiple accessed web pages 221 saved in the browser history 220 of web browser 214.

[0110] In some implementations, the history cluster is generated by system 202 based on metadata associated with the web pages mentioned above. The metadata may include the source event, the access timestamp, the engagement score, and subtopics associated with at least one topic. The metadata may be retrieved or generated based on information within the web pages mentioned above.

[0111] In some implementations, the engagement score includes an engagement metric for each of the web pages in the above section, and each engagement metric is used to select the prominence level when displaying snippets for at least one of the web pages in the above section in the history cluster list.

[0112] In some implementations, the search history UI may include a list of history clusters depicted in the web browser's search history, as shown in Figure 4B and / or Figure 5B. In some implementations, the search history UI (e.g., history cluster list 100) may include at least a portion of several web pages that have been determined to be related to a topic. For example, visits 128, 130, and 132 relate to a list of visits representing web pages related to the topic 120, "kitchen range". In some embodiments, the above portion of several web pages includes a list of visits in cluster 108B, and the above portion corresponds to a list of visits corresponding to search history data from the search history indicating that the portion belongs to a web browser tab group or bookmark.

[0113] In some implementations, the search history UI also includes an action control configured to resume a previous search associated with a cluster, as shown by control 143 in Figure 1. System 202 may include a renderer configured to display the generated search history UI along with parts of multiple web pages and search history data. In some implementations, the search history UI represents a browser history UI that encompasses the search history.

[0114] Computing system 202 may also be associated with session data 212, which represents open browser tabs and browser windows during a single login to web browser 214. The session can be saved and reopened after computing system 202 running web browser 214 is closed or restarted. Specific history cluster lists 228 and history clusters 222 may be generated based on the session data 212.

[0115] Application 208 can be any type of computer program that can be executed / delivered by computing system 202 (or by server computing system 204, or via external services or website 248). Application 208 may provide a user interface (e.g., a browser application window, tabs, etc.) to allow a user to interact with the functions of each application 208. The application window of a particular application 208 may display any type of control, such as menus, icons, widgets, etc., in addition to application data.

[0116] Application 208 may include or access application information 210 and session data 212, both of which may be used to generate content and / or data and provide such content and / or data to the user and / or O / S 206 via the device interface. Application information 210 may correspond to information executed or otherwise accessed by a particular application 208. For example, application information 210 may include text, images, inputs, outputs, or control signals associated with interaction with application 208. In some implementations, application information 210 may include data from the browser repository 216.

[0117] In some implementations, application information 210 may include, but is not limited to, metadata, tags, timestamp data, URL data, tab group assignment data, bookmark assignment data, and other data associated with a specific application 208. In some implementations, application 208 includes a web browser 214.

[0118] In some implementations, the OS 206 and / or application 208 may include or access services and / or websites 248. Services 248 may include online storage, website / content access, account session or profile access, authorization data access, etc. In some implementations, services 248 may function to replace the server computing system 204, in which case user account data is accessed via the services.

[0119] In some implementations, the OS 206 and / or application 208 may include, or have access to, a communication module 250, a camera 252, memory 254, and a CPU / GPU 256. The computing system 202 may also include, or have access to, policies and approvals 262 and preferences 264. Furthermore, the computing system 202 may also include, or have access to, input devices 258 and / or output devices 260.

[0120] The computing system 202 may generate and / or distribute policies and approvals 262 and preferences 264. Policies and approvals 262 and preferences 264 may be configured by the device manufacturer of the computing system 202 and / or by users accessing the system 202. Policies and preferences 262 may include routines (i.e., sets of actions) that are executed based on commands triggered by the user. Needless to say, other policies 262 and preferences 264 may be configured to modify and / or control content associated with the system 202 configured with those policies and approvals 262 and / or preferences 264. In some implementations, approvals and policies 262 may be configured for specific applications, browser tabs, browser tab groups, and inputs associated with such entities.

[0121] The input device 258 may provide the system 202 with data received via, for example, a touch input device capable of receiving haptic user input, a keyboard, a mouse, a hand controller, a mobile device (or other portable electronic device), or a microphone capable of receiving audible user input. The output device 260 may include, for example, a device that generates content for a display for visual output, a speaker for audio output, and so on.

[0122] The server computing system 204 may include any number of computing devices, taking the form of several different devices, such as standard servers, groups of such servers, or rack server systems. In some examples, the server computing system 204 may be a single system sharing components such as a processor 266 and memory devices 268. A user account 270 may be associated with the configuration of the system 204 and session data 272 and / or the configuration of profile 274, according to user authorization data 276 based on policies and authorizations 262, which may be provided to the system 202 at the request of a user of a web browser 214, for example.

[0123] Network 280 may include the Internet and / or other types of data networks, such as local area networks (LANs), wide area networks (WANs), cellular networks, satellite networks, or other types of data networks. Network 280 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within Network 280. Network 280 may further include any number of hardwired and / or wireless connections.

[0124] The server computing system 204 may include one or more processors 266 formed on a circuit board, an operating system (not shown), and one or more memory devices 268. The memory devices 268 may represent any (or more) types of memory (e.g., RAM, flash, cache, disk, tape, etc.). In some examples (not shown), the memory devices 268 may include external storage, such as memory that is physically separate from but accessible from the server computing system 204. The server computing system 204 may include one or more modules or engines representing specially programmed software.

[0125] In general, computing systems 202 and 204 can communicate with each other, for example, via the communication module 250, and / or transfer data wirelessly via the network 280, using the systems and techniques described herein. In some implementations, each system 202, 204 may be configured to communicate within system 200 with other devices associated with system 200.

[0126] Figure 3 is a flowchart 300 illustrating an example of acquiring and analyzing web browser history data for use in generating history clusters, according to the implementation described herein. In particular, flowchart 300 depicts an example of acquiring browser history 220 to generate metadata 224. For example, flowchart 300 can collect context-based annotations and content-based annotations.

[0127] Context-based annotations may be obtained using browser history 220 and an ML model that clusters the browser history 220 according to topics, subtopics, and / or metadata 224. Content annotations may use document object model content to ingest and / or annotate web page content using a browser process to perform further computations. For example, a page content annotation service may perform a low-priority ML inference task to annotate content ingested from the document object model. Annotated content may be used to update web page visits in a history service repository (stored, for example, in history clusters 222 and / or metadata 224 repositories).

[0128] Context-based annotations can be obtained based on browser history 220 and / or from application programming interfaces (APIs) such as the URL-keyed anonymized data (UKM) API. In operation, the flow diagram 300 may begin with the history tab helper component 302 notifying the history cluster tab helper component 304 about detected new web page visits (e.g., accesses) associated with the web browser 214. Furthermore, the history cluster tab helper component 304 can access and / or receive UKM page load metrics 306.

[0129] The History Cluster Tab Helper Component 304 may then record the page start metric associated with the accessed web page, and may log incomplete visits until the page end metric is detected and saved. When the lifespan of a web page ends (for example, when the user navigates away from the accessed web page), the History Cluster Tab Helper Component 304 is requested to record the page end metric. Once the page end metric is recorded, the History Cluster Tab Helper Component 304 may log the page end metric to the History Service Component 308. Furthermore, the UKM page load metric 306 may be uploaded to the UKM Upload Component 310, for example, as part of a page load event. The History Cluster Tab Helper Component 304 may also trigger the Cluster Service 312 to save metadata such as page load events and data metrics (and, for example, maintain that metadata locally on the device).

[0130] The history cluster tab helper component 304 may function to execute logic and analysis associated with the ML model 236 on the device in order to generate history clusters 222. The logic can utilize one or more ML models 236 to identify groups of similar web page accesses using one or more algorithms and iteratively process the metadata 224 associated with each individual page load to build clusters. Generally, clusters can cover similar topics but focus on different parts of a topic (e.g., tomatoes or a vegetable garden). Performing analysis on the device helps ensure that user data and / or associated content remain private and reduces network resource usage compared to processing and cluster generation off-device.

[0131] In some implementations, the search history analyzer 234 iteratively processes the metadata 224 associated with each individual page load to build an indexable set of clusters of web page accesses. Analysis and clustering may be performed periodically (e.g., every x page loads or every y hours) in the background process of the web browser 214.

[0132] In some implementations, a visit identifier (e.g., an access timestamp) may be used to associate a context signal with a specific web page. The history cluster tab helper component 304 may obtain the access timestamp for the accessed web page from the history service component 308. Such page metadata and access timestamp may be stored on the device as metadata 224 and may be keyed according to the access timestamp (e.g., a visit identifier).

[0133] Figure 4A is an illustrative user interface showing an example of access to the history cluster list according to the implementation described herein. The user can access information (e.g., history cluster list 100) which may be presented as one or more UIs depicting a search process organized based on one or more generated history clusters 222. At some point, the user may want to continue the search process or review previous search data. To this end, the user can access the browser 214 and enter one or more keywords that may be related to previous searches. For example, Figure 4A depicts an illustrative search box 402 in which the user has entered the search query "kitchen range". System 202 can receive the search query and determine whether any of the query terms match keywords associated with previously searched query terms. In some implementations, system 202 can also determine whether any of the query terms match keywords and / or content in the browser history 220.

[0134] In the example in Figure 4A, system 202 determined that the previous search 404 was performed for the search query "kitchen range". The user may be provided with a link to select that search to return to the previous search. System 202 may also provide any search process 406 that may match such search terms from the browser history 220. As shown, a single search process control 408 is provided as an option for the user to select to resume the search process for "kitchen range". In some implementations, the process "kitchen range" represents a topic generated based on the entered keyword "kitchen range" from, for example, search 404. In some implementations, the topic is identified as an entity related to the content of the visit, such as a query, web page content, or contextual information, using conventional or future-developed techniques for identifying entities in text or images. The topic could instead be "kitchen appliances", or "range uses", or any other such topic generated by the system based on the specific search entered, search activity, and visited web pages. Additional processes may be displayed in the list below process 406. If the user selects control 408 as the entry point to the journey, window 410 may be provided.

[0135] In response to receiving a selection in control 408, for example, system 202 can trigger the generation of a history cluster list based on a previous search step, "kitchen range". The depicted history cluster list is history cluster list 100, but content for any history cluster list may be generated and added depending on previous searches, topics, and past interactions the user has had with the web browser 214.

[0136] Window 410 displays the history page of browser 214. In some implementations, the same options and history cluster list content may instead be provided in a different web page, window, control, or toolbar. Furthermore, as shown by menu 412, journeys and previous searches may be displayed as menu items. The user can select a previous search, for example search 404a, to be presented during the previous search results from search 404. For example, if system 202 has generated other records related to the history cluster list and / or other previous searches, such searches may also be displayed in menu 412.

[0137] In some implementations, menu 412 may be a tabbed navigation menu, in which case elements 404a, 414, 408a, and 416 (and / or other items not shown here) are presented in separate tabs. In some implementations, such elements are presented with additional nested items that may be accessed and presented as part of the history cluster list 100.

[0138] The user can also view other previously performed steps, as indicated by control 408a. The user can also view their past browser history 220 by selecting control 414. Furthermore, the user can select control 416 to view content open in browser tabs on other devices, which may be associated with the user account performing the search shown in window 410, for example.

[0139] Figure 4B depicts another example of a history cluster list 100A. The history cluster list 100A can provide the user with a visual way to start a search task from an incomplete location. For example, the user's last steps are depicted in the form of suggested searches, recent tabs and bookmarks, as well as traditional search history page data or browser history page data, so that the user can get a sense of where the search task ended and where the user can logically continue the search.

[0140] In this example, as shown in the history cluster list 100, web activity related to the topic "kitchen range" is depicted, but additional information is also depicted. For example, history cluster list 100A also includes relevant content from tabs and / or bookmarks in section 418 related to the journey associated with the topic. In particular, section 418 depicts several web page visits, represented as part of the recent tab group 420. Recent related bookmarks 422, 424, and 426 are also shown. Any number of tab groups and bookmarks may be displayed in section 418 in response to the system 202 determining that the content is related to a particular cluster displayed during the search journey. In some implementations, the content provided in section 418 may be related to a relevance condition so as to ensure that outdated search content is not depicted.

[0141] History cluster list 100A may also include recent searches, which are depicted within section 428. Recent searches are depicted as a conventional search history page list, but may include additional features and nesting of search content, based on the fact that they are generated to be displayed within history cluster list 100A, rather than being generated to be displayed within a search history page or browser history page. For example, duplicate content can be removed from the list within section 428.

[0142] In some implementations, the content listed within Section 428 may include website content in which the user has had a high level of engagement (e.g., content interaction, clicks, etc.) and / or a high time spent. For example, System 202 may determine metadata indicating engagement scores and / or time spent scores associated with any number of previously accessed websites. Sites with the highest engagement scores may be selected to be displayed within Section 428. In some implementations, other sections of the history cluster list may also include or exclude content based on the engagement scores of the associated accessed web pages. In some implementations, secondary items with lower engagement scores and / or time spent scores may be nested under content with higher engagement scores and / or time spent scores.

[0143] The history cluster list 100A is depicted as a content-based structure, in which case the content is used as the basis for displaying items within a particular section. In some implementations, system 202 may instead use a time-based structure in which previously accessed content is visually enumerated based on the time it was accessed. The same visual content may be depicted, but the display order may differ if the access timestamp is used as the basis for ordering the display.

[0144] Figure 4C depicts another example of a history cluster list 100B. The history cluster list 100B can provide users with a visual way to start a search task from an incomplete location. For example, the user's last steps are depicted in the form of suggested searches, recent tabs and bookmarks, as well as traditional search history page data or browser history page data, so that the user can see where their previous search task ended and where they can logically continue their search.

[0145] In this example, web activity related to the topic "Kitchen Range" is depicted, as shown in the history cluster list 100B, but additional information is also depicted. In particular, section 430 includes webpage visit 114, which in this example represents the top visit (e.g., top visit 124 for history cluster 108B). Section 430 also includes visits 128, 130, and 132, which represent visits that also have some degree of similarity to a particular topic (e.g., topic 120 in Figure 1). Visits 128, 130, and 132 may represent related visits nested within visit 114. For example, if visit 114 is the top visit for cluster 108B, then visits 128, 130, and 132 may be nested webpage visits related to the search query source event. Thus, visits 128, 130, and 132 may be generated as a visit list, which is generated by the computing system 110 to appear in the history cluster list 100B based on a determined score for each associated webpage visit.

[0146] Furthermore, the history cluster list 100B includes indicators representing analyzed metadata. The analyzed metadata may include metadata 224, which is collected by the browser 214 and analyzed for the purpose of generating UI content for the history cluster list. In this example, the metadata was used to generate one or more metadata badges, such as badges 432, 434, 436, and 438. In some implementations, users can choose to view or hide badges in the history cluster list. In some implementations, users can use badges to change the status of a particular visit. For example, if a visit is indicated as a bookmarked webpage, the user can select a badge to set the visit as a favorite instead of a bookmark, revoke the bookmark metric, and / or perform another classification change for that visit.

[0147] Badge 432 indicates that visit 128 is a bookmarked webpage. Badge 432 may be generated by the UI generator 226 in response to determining that visit 128 is bookmarked in the browser 214. In some implementations, badge 432 may indicate the time of the bookmark. In some implementations, badge 432 may indicate the bookmark's rank relative to other bookmarks.

[0148] Badge 434 indicates that visit 130 is saved in a tab group. Badge 432 may be generated by the UI generator 226 in response to determining that visit 130, associated with browser 214, is part of a tab group. In some implementations, the visit list may contain more than one badge. As shown, visit 130 includes a second badge 436 indicating that visit 130 was opened again three days ago. In some implementations, badge 436 is applied to visit 130. In some implementations, badge 436 may be applied to another badge, such as badge 434. In such an example, badge 436, indicating that it was opened three days ago, may be applied to the tab group indicated by badge 434.

[0149] Badge 438 indicates that visit 132 has been visited 5 times by the user. Badge 438 may be generated by the UI generator 226 in response to determining that visit 132 has previously been visited 5 times in the browser 214. The badge may be updated when the user visits the web page. The history cluster list 100B may also include related queries, such as related searches 138 and 140, as described in detail above. The history cluster list 100B may further include a show more control 440 for displaying additional related visits.

[0150] In some implementations, metadata may be captured but not displayed in a particular history cluster list. In such cases, users can configure the history cluster list to display (or hide) metadata-related information (e.g., badges, timestamps, etc., generated based on metadata associated with web activity). The option to display (or hide) metadata-related information allows users to view the history cluster list in a less data-intensive form, potentially making it easier to review the history cluster list. In some implementations, the option to display (or hide) metadata-related information allows users to make informed choices based on the presented metadata, potentially avoiding reviewing content or links that are less important to the user's next steps.

[0151] Figures 5A to 5C are illustrative user interfaces illustrating examples of access to and interaction with the history cluster list according to the implementation described herein. The history cluster list 100 may be created based on the history cluster 222, which is generated according to the metadata 224 and browser history 220. Generally, the user can access the history cluster list (e.g., history cluster list 100) from several locations within the web browser 214. For example, the user can access the history cluster list using direct URL navigation, suggested search controls, queries entered in query controls, menu items and / or controls presented on the history page, etc.

[0152] Figure 5A shows the user interface of history page 500A, where the user has chosen to search for a search itinerary using control 502. In this example, any number of previous search itineraries that the user has previously performed may be presented to the user. The presented search itineraries may be depicted based on a determined time period, which can be selected by the system or by the user. For example, computing system 202 may decide to generate and display the last month of a search itinerary using access timestamps associated with the accessed and considered web pages in one or more of the search itineraries.

[0153] For example, ML model 236 may generate historical clusters based on search runs performed in the previous month. These clusters may be ranked and / or indexed. One or more of these ranked and / or indexed clusters may be retrieved in response to the decision to display which search runs. The retrieved clusters can then be used to generate a list of historical clusters relevant to a particular search run. In some implementations, the list of historical clusters is generated in response to a user selection of a specific search run.

[0154] As shown in Figure 5A, the user entered a search process page associated with the browser and / or search history page using direct navigation to the browser URL toolbar 504 via a URL. The process page depicts the search processes the user performed in the previous month. The processes include the bar sink process 506, the kitchen process 508, the living room process 510, and the patio process 512. In some implementations, the number of processes that may be depicted on the example page 500A may be more or less. Furthermore, the duration of the search processes that may be used is arbitrary.

[0155] Figure 5B is a user interface illustrating an example of filtering browser history 220 on a browser and / or search history page 500B. Page 500B includes controls 514 for searching events and content enumerated in the browser history 220. Furthermore, the browser history 220 may include other controls, such as controls 514 and / or chronologically enumerated web page accesses 516 and 518.

[0156] In some implementations, system 202 can also use the generated history clusters 222 to present content on the browser and / or search history page 500B. Such clusters can be used to filter content in the chronologically enumerated webpage accesses 516 and 518. For example, system 202 may generate clusters 222 to include the topics (or subtopics) of bathrooms 520, kitchens 522, living rooms 524, patios 526, houses 528, and brands 530. In this case, the search history and / or browser history is presented based on the relevance of the webpages to specific topics, rather than simply on the chronological order shown in 516, 518.

[0157] Topics (or subtopics) may be provided as selectable controls within page 500B (e.g., controls within section 532) for filtering browser history 220 according to a topic or subtopic. For example, a user can select the topic ybrand 530 and filter browser history 220 to display specific web pages related to ybrand. Such web pages may contain data, topics, subtopics, metadata, and / or related content about the keyword ybrand. As shown, section 532 contains visual images and text content representing web pages accessed with respect to the enumerated topics. Other variations of the visual images and text content may be depicted to enable the user to interpret the topics of a particular filter.

[0158] Figure 5C shows the user interface 500C, illustrating the user's access to the browser history 220 via the browser toolbar 534. In this example, the user entered a search query for "kitchen range" into the toolbar 534 of the browser 214. The browser toolbar 534 can also be called the search box, omnibox, or query input area. The user has previously viewed the searched content 536 shown behind the dropdown 538 and has decided to search for different content. Thus, the user enters a search query (including an incomplete / partial query) into the toolbar 534, and in response, the browser 214 receives the search query (or partial query) and provides several search terms that may match part of the search query, for example, through an autocomplete mechanism (suggested completion). In some of the suggested search queries, the browser 214 may also provide a search process control 540 to allow the user to return to and resume a previous search process in the browser. The user can select control 540, and the user may be provided with a list of history clusters related to search processes that include the query terms “search” and / or “range,” as shown in the example history cluster list 100 in Figures 1 and 4A. In some implementations, the browser 214 may also provide an option to search all search processes, as shown by control 542. In some implementations, the browser 214 may also provide, in addition, autocomplete entries based on the context (e.g., metadata) associated with previous search processes that match one or more keywords entered in the browser toolbar 534.

[0159] Figure 5D shows a user interface 500D, an example of a list view of the browser's browser history 220 and / or search history page. Here, the user has chosen to view the search history (or browser history) as a reverse chronological list of web page visits by selecting a list control 544. User interface 500D includes a control 514 for searching events and content enumerated in the browser history 220. Furthermore, the browser history 220 may include other controls, such as control 514 and / or the reverse chronologically enumerated web page access lists 516 and 518. In some implementations, the user may choose the option to view web visits from other tabs from other devices saved in the browser 214, for example. Web visits from other tabs may then be included in the list view according to the time and / or date of the visit. In some implementations, the user may instead prefer to view the search history or browser history in a process view. To do so, the user can select a route control 546 to switch the viewing of historical data (e.g., search history data, browser history data) from a list view to a route view provided by the user interface 500E.

[0160] As shown in Figure 5D, the list control 544 and the itinerary control 546 are depicted as selectable controls that can be used to navigate to a specific list view or itinerary view. In some implementations, the list view and itinerary view may be provided in a tabbed view so that the user can switch between the list view tab and the itinerary view tab within the browser and / or search history.

[0161] Figure 5E is a user interface 500E showing an example of a journey view for browser history 220 and / or search history page 500E. A journey view can represent a history cluster list having, for example, any number of sections. The sections shown in Figure 5E include a first section 548, a second section 550, and a third section 552. Section 548 is similar to section 430 in Figure 4C. In particular, web activity related to the topic "kitchen range" is depicted in this example to include at least one web page visit 114, which represents the top visit (e.g., the top visit 124 for history cluster 108B). Section 548 also includes visits 128, 130, and 132, which also represent visits that have some degree of similarity to a particular topic (e.g., topic 120 in Figure 1). Visits 128, 130, and 132 may represent related visits nested in visit 114. Badges 432, 434, and 436 related to specific web page visits are also shown, as detailed in Figure 4C. Section 548 also includes related searches 138 and 140 (queries). Any number of related queries may be displayed. In some implementations, related search queries may change as the itinerary is updated with new information triggered by the user's new web activity. Additional content related to visit 114 may also be displayed in full within interface 500E, or it may be provided in response to a user requesting to view additional data (e.g., via a Show More control, itinerary update control, etc.).

[0162] Section 550, like Section 548, includes the top visits 553 related to web activity for “kitchen range.” In this example, the subtopic could be related to “gas range.” The subtopic could be generated as a separate history cluster from the initial history cluster associated with Section 548, for example. The top visits 553 may also include ranked related visits 554 and 556, one of which may include a badge, such as badge 558, indicating that visit 554 is bookmarked. Although not depicted, related searches, additional controls, and / or additional visits may be enumerated within or accessible therein in Section 550.

[0163] Section 552 contains the top 560 visits related to the web activity for “curtains.” Section 552 may be generated based on search paths associated with user-entered searches or web activities about curtain-related topics. In this example, the topic or subtopic about curtains might be related to “tips for hanging curtains.” The topic or subtopic may be generated as a separate history cluster from the initial history cluster associated with sections 548 and 550, for example. The top 560 visits may also include ranked related visits 562 and 564, one of which may include badges, images, links, indicators, etc. Related searches, additional controls, and / or additional visits, though not depicted, may be enumerated within or accessible therein in Section 550.

[0164] In some implementations, the browser 214 may provide the user with an option to turn off itinerary generation, as indicated by control 566. Turning off itinerary generation results in the web browser 214 ceasing to cluster visits according to topics / subtopics. When itinerary generation is set to off mode, the browser 214 may not generate additional search itineraries. Thus, to review browser and / or search history, the user can browse a list view to access recent and historical browser and / or search history data / activity.

[0165] Figures 6A to 6C are illustrative user interfaces illustrating an example of browser history modification by modifying the history cluster list according to the implementation described herein. For example, Figure 6A shows a history cluster list 600 having several clusters (e.g., clusters 602 and 604). Cluster 602 includes visit lists 606, 608, and 610 representing any number of web page visits performed by the user in relation to a specific search process and the topic "kitchen range". Related search queries 612 may be provided, indicating additional ways to continue the search process related to that cluster. In the example in Figure 6A, history cluster 602 is ranked higher and displayed earlier than history cluster 604 based on the recency of the access timestamps associated with the pages and / or on similarity to a particular topic or subtopic.

[0166] Other clusters may be presented in the example history cluster list 600. Clusters generally represent a search process or part of a search process previously performed by the user.

[0167] Users can access web pages of any cluster and subcluster and view additional content associated with such web pages. For example, cluster 602 may represent several web pages, and the user can choose to navigate to those web pages or additional web pages associated with the selected web page. In some implementations, the user may select a Show More control 622 to view additional content associated with cluster 602.

[0168] Users can also make selections about the content and web pages associated with a subcluster. For example, a user might select subcluster 608 to view additional content associated with it. In some implementations, users may access additional menus associated with a cluster or subcluster. For example, a user might access menu 624 associated with cluster 602. In particular, menu 624 can be opened for visit list 608. Menu 624 depicts an option to open visit list 608 (and / or related web pages or information) in a new browser tab.

[0169] Furthermore, menu 624 depicts an option to clear visit list 608 from cluster 602, which can also trigger the removal of the underlying webpage visit from the search history. If the user chooses to clear visit list 608 from cluster 602, the browser 214 may remove list 608 and its associated webpage from the cluster (e.g., cluster 602) used to generate the history cluster list 600. This targeted removal method allows sensitive content to be removed from the device in a less impactful way without clearing the entire search and / or browser history.

[0170] Figure 6B shows a history cluster list 600 with another example menu 626. Menu 626 includes an option to open all web pages related to the visit list 608 as a tab group. Such an option can open a new browser tab group and generate tabs associated with the web pages in the visit list 608. Browser tab groups may include an indicator (e.g., text, color, etc.) that a particular web page belongs to the newly generated browser tab group.

[0171] Menu 626 also includes an option to clear everything from the history. This option is selected with respect to the visit list 608, and therefore, when selected, the browser 214 clears the visit list 608 and its associated web pages from both the cluster that generated the history cluster list 600 and the web browser's browser history 220.

[0172] Figure 6C shows UI628 opened in association with the history cluster list 600, particularly the visit list of cluster 602. UI628 provides the user with the option to delete web pages in bulk based on which visit list (and / or web page) is selected in UI628. In response to selecting the bulk delete control, the browser 214 can retrieve data including at least the list visit list and associated web pages of cluster 602. The retrieved data may be provided within UI628, in which case the user can, for example, select any number of items within UI628 and delete those items from the browser history 220. This may have the advantage of allowing the user to delete entire search concepts / processes from the search history without having to sort through all chronological records that are not shown according to topic. Thus, the history cluster list provides a novel and efficient way to manage search history.

[0173] In some implementations, the UI generator 226 can receive a request to delete a webpage associated with a rendered cluster in the history cluster list 600. The request may be received in UI 628. The request may trigger the clearing of the record of access to the saved webpage from the browser history 220. In some implementations, the clearing may remove the content from the search process so that the content does not appear on the history cluster list in future web sessions. For example, if a user has opted out of purchasing a range of a particular brand, the user may not want to see that brand's range in future history cluster lists and therefore may choose to delete the browsing and / or interaction with webpages associated with that brand from their search history. This ensures that the user is never presented with such content, as clusters are generated based on search history events (e.g., webpage access / visits). If a visit related to that brand's range is deleted, such a visit will not be part of a future cluster or subcluster used in the history cluster list.

[0174] Figure 7 is a flowchart of an exemplary process 700 that generates user interface content related to browser search activity according to the implementation forms described herein. In particular, process 700 may be used to generate and display several search history UIs representing browser activities and / or search activities performed by a user. The search history UIs are supplemented with search history data, browser history data, and / or data based on associated metadata. Search activities and / or browser activities may include, for example, one or more searches and / or interactions performed by a user accessing a web browser 214. With user approval, process 700 may generate a history cluster (representing a search process including several searches and / or browser activities) using, for example, the browser history 220 of the web browser 214 and / or activities stored in the accessed web pages 221. Specific input and / or requests from the web browser user may trigger the display of data associated with the history cluster 222.

[0175] Generally, process 700 generates and renders a history cluster list showing history search data performed in a web browser, utilizing the systems and algorithms described herein. Process 700 may utilize one or more computing systems having at least one processing device and a memory storing instructions that, when executed, cause the processing device to perform a plurality of operations and computer steps described in the claims.

[0176] In block 702, process 700 includes generating a repository of metadata based on multiple web pages that have been accessed and saved in the browser history of a web browser running on a computing device. For example, metadata 224 may be generated using browser history 220, accessed web pages 221, and / or metadata generated and / or taken from such browser history 220 and accessed web pages 221. Browser history 220 may be saved by system 202 when a user accesses a web browser and / or interacts with web content in the process of making a search and / or reviewing web content.

[0177] The metadata 224 may include, for each webpage (i.e., for each webpage visit / access), a source event 245 (e.g., an event indicating the source of the webpage opened in the web browser), an access timestamp 238, and at least one topic 242. In some implementations, the metadata 224 may further include a webpage identifier defined for the webpages in the history cluster 222. The webpage identifier may indicate whether each webpage is part of a tab group, bookmark, or search results page accessed by a computing device. In some implementations, the metadata may further include, for one or more webpages, a determined webpage entity, multiple associated searches, and / or time spent on the webpage.

[0178] In some implementations, generating a metadata repository further involves deduplication of multiple web pages that have been accessed and saved in the browser history. Deduplication may include determining duplicate web page accesses among the accessed web pages, selecting the web page with the most recent access timestamp relative to the access timestamp associated with the determined duplicate web page access, and generating metadata to be stored in the repository for the web page with the most recent access timestamp. This can, for example, ensure that recent web pages are retained while older web pages are discarded or not used for display in the history cluster list.

[0179] In block 704, process 700 includes generating a history cluster that represents a subset of several web pages related to a given topic, based on metadata. For example, history cluster 108B may include a subset of web pages accessed by the user that were related to the topic "Kitchen Range" associated with history cluster list 100 (e.g., a visit list representing visits 128, 130, and 132). History cluster 108B may also include web page visits (e.g., a visit list) based on the source events and access timestamps of specific web pages within the above subset. For example, history cluster 108B may include additional content that may be related searches performed by the user and / or web pages that the user has visited, all of which may relate to source events representing previous search results and / or investigations performed with respect to cluster 108B and / or topic 120. In some implementations, access timestamp 134 may relate to cluster 108B, while each visit 128-130 may be associated with an individual access timestamp, which can be used for ranking or otherwise ordering visits within a particular cluster or within the entire history cluster list 100.

[0180] In some implementations, for example, a history cluster associated with visit 116 has a title, and the UI generator 226 generates a link to display the title as a suggested link, such as the title "30-inch gas range". Selecting the link issues a request to view the data associated with history cluster 108C. In some implementations, the suggested link is displayed as a search suggestion in a search engine (e.g., controls 152, 154, and 156 leading to links related to cluster 116 and / or topic 120). In some implementations, the suggested link may be displayed as an option in the search history, as shown in Figure 4B (e.g., links related to related searches 138 and 140 and / or links associated with web page visit 114), and / or Figure 5B (e.g., links 520-530).

[0181] In some implementations, a portion of a history cluster 222 may contain or reference a snippet for at least one webpage among several webpages in that portion. A snippet is a short description or identifier (e.g., one or two lines of text) of a webpage and / or its content. A snippet for a webpage may contain a shortened URL for that webpage, e.g., a domain, or a domain and a shortened path. A snippet for a webpage may contain concise text extracted from the webpage. A snippet may contain a title that identifies the content of the webpage. A snippet may contain two or more of the following: shortened URL, concise text, or title. A cluster may contain one or more associated action controls. Associated actions are actions that can be performed on a history cluster or a portion of a history cluster. Associated actions may include restarting the search process by reissuing the query associated with the history cluster. Associated actions may include clearing the saved search history and / or browser history of the history cluster. Associated actions may include a proposed associated search. The suggested related searches may not be user-submitted but could be queries that extend the journey represented by the history cluster in a relevant direction. Related actions may include bookmarking a location (web page) in the cluster. Related actions may include opening a web page in a new tab. Related action controls may be provided by or represented using selectable UI elements. In other words, control buttons may be provided for performing actions related to the cluster. For example, cluster 108B, which includes visit 130, may depict a resume search journey control 143 to allow the user to return to the web page and continue the search, in addition to a snippet 147 and an image snippet 149 about the best dual-fuel range.In general, history clusters and subclusters can be generated as the output of an ML model (e.g., ML model 236) that can be run on computing system 202 using metadata 224 as input. ML model 236 is described in detail with reference to Figure 2.

[0182] In block 706, process 700 includes assigning a score 244 to each webpage in the above-mentioned subset. For example, score / rank generator 232 may assign scores to a subset of webpages in cluster 108B. Each score may be based at least in part on an access timestamp 134 and a determined engagement score associated with each webpage in the above-mentioned subset of webpages for history cluster 108B. In some implementations, the score 244 for a particular webpage visit may be calculated based on metadata 224, which includes, for example, a determined engagement (e.g., interaction) with the webpage being visited (e.g., an engagement score 240 based on an engagement metric). In some implementations, the engagement score includes an engagement metric for each webpage in the above-mentioned subset. Each engagement metric may be used to select a prominence level when displaying a snippet for at least one webpage from among the multiple webpages in the above-mentioned subset in history cluster list 100.

[0183] Engagement metrics may include, but are not limited to, the length of time a webpage tab was in the foreground, the number of times a webpage was visited (e.g., return page visits), whether a webpage is a bookmark, whether a webpage belongs to a tab group, the number of accesses, visits, and interactions with bookmarks or tab groups, selected ads, search results that were interacted with, page views, selected entities, scroll depth, exit rate, bounce rate, abandonment rate, comments provided, reviews provided, and click-throughs.

[0184] In block 708, process 700 includes generating a history cluster list 100 for a topic (and / or subtopic) that includes one or more history clusters associated with the topic, in response to a request to view search activity associated with the topic. The history cluster list may be organized according to the history clusters, and within each history cluster, according to the respective scores of the web pages in that history cluster. In some implementations, the history cluster list may include a visit list representing the web pages in the above part if each web page is determined to have a score that meets a threshold score.

[0185] The example threshold scores may indicate the level of similarity between a topic or subtopic and a specific webpage in the browser history 220. For example, if a webpage is determined to have at least 50% similarity to a particular topic and / or subtopic, that webpage may have a score that meets the threshold score. In some implementations, the threshold score may be related to the topic, and a score of approximately 80% or more similarity between the topic and the webpage may be indicated as the threshold score that triggers the selection of webpages to be included in a cluster and / or the selection of the resulting history cluster list. Other threshold scores, as well as combinations of similarity determinations between topics, subtopics, and webpages, are also possible. For example, additional scores, thresholds, and / or rankings may be used, at least partially, based on metadata corresponding to activities performed in the web browser (e.g., engagement activity / metrics, click events, search events, tab / bookmark events, etc.) and / or specific search history data (e.g., access timestamps, topics, subtopics), etc.

[0186] In block 710, process 700 includes displaying a history cluster list. For example, renderer 246 may be triggered by system 202 to display a history cluster list 100 in response to receiving an indicator to view a particular search run, web page, or topic, or an associated history cluster list 100. In some implementations, the history cluster list may be rendered once the generation of the history cluster list is complete.

[0187] In some implementations, a history cluster list may include search history information organized by topic, source event, and / or access timestamp, search activity information, and relevant search result information depicted according to determined scores and / or rankings. For example, a history cluster list may include a visit list corresponding to at least one of the web pages in the above part. For example, a history cluster list 100 may include several search results (e.g., web pages in clusters such as the visit list and / or the visit list related to visit 128) for a particular web page that can be accessed using control 126. In some implementations, multiple visit lists may be presented within the browser history 220 of the web browser 214. In some implementations, a history cluster list may also include action controls configured to resume previous searches associated with multiple visit lists. For example, cluster 108B may include a resume search process control 143, in addition to the image snippet 149, to allow the user to return to the web page and continue the search.

[0188] Figure 8 shows examples of computer devices 800 and mobile computer devices 850 that may be used with the technologies described herein. Computing device 800 is intended to represent various forms of digital computers, including laptops, desktops, tablets, workstations, personal digital assistants, smart devices, appliances, electronic sensor-based devices, televisions, servers, blade servers, mainframes, and other suitable computing devices. Computing device 850 is intended to represent various forms of mobile devices, including personal digital assistants, mobile phones, smartphones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be illustrative only and are not intended to limit the implementation of any implementation described and / or claimed herein.

[0189] The computing device 800 includes a processor 802, memory 804, a storage device 806, a high-speed interface 808 connected to memory 804 and a high-speed expansion port 810, and a low-speed interface 812 connected to an external device port 814 and the storage device 806. The processor 802 may be a semiconductor-based processor. The memory 804 may be semiconductor-based memory. Each of the components 802, 804, 806, 808, 810, and 812 may be interconnected using various buses and mounted on a common motherboard or in other configurations as appropriate. The processor 802 can process instructions to be executed within the computing device 800, including instructions for displaying graphical information related to a GUI on an external input / output device such as a display 816 coupled to the high-speed interface 808, which is stored in memory 804 or in the storage device 806. In other implementations, multiple processors and / or multiple buses may be used as appropriate, along with multiple memories and types of memory. Furthermore, multiple computing devices 800 may be connected to each other, and each device may provide some of the necessary functions (for example, as a server bank, a group of blade servers, or a multiprocessor system).

[0190] Memory 804 stores information within the computing device 800. In one implementation, memory 804 is one or more volatile memory units. In another implementation, memory 804 is one or more non-volatile memory units. Memory 804 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk. Generally, the computer-readable medium may be a non-transient computer-readable medium.

[0191] The storage device 806 may provide mass storage to the computing device 800. In one implementation, the storage device 806 may be or include computer-readable media such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. Computer program products may be tangibly embodied in the information carrier. Computer program products may also include instructions that, when executed, perform one or more methods and / or methods performed by the computer, such as those described above. The information carrier is computer or machine-readable media such as memory 804, the storage device 806, or memory on the processor 802.

[0192] The high-speed interface 808 manages bandwidth-intensive operations related to the computing device 800, while the low-speed interface 812 manages bandwidth-intensive operations. This assignment of functions is merely an example. In one implementation, the high-speed interface 808 is coupled to memory 804, a display 816 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 810 that can accept various expansion cards (not shown). In this implementation, the low-speed interface 812 is coupled to storage device 806 and an external device port 814. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices such as a keyboard, pointing device, or scanner, or to network devices such as a switch or router via a network adapter, for example.

[0193] The computing device 800 can be implemented in several different forms, as shown in the figure. For example, it may be implemented as a standard server 820, or it may be implemented multiple times within a group of such servers. It may also be implemented as part of a rack server system 824. Furthermore, it may be implemented in a computer such as a laptop computer 822. Alternatively, the components of the computing device 800 may be combined with other components in a mobile device (not shown), such as device 850. Each of such devices may contain one or more computing devices 800, 850, and the entire system may consist of multiple computing devices 800, 850 communicating with each other.

[0194] The computing device 850 includes, among other components, a processor 852, memory 864, input / output devices such as a display 854, a communication interface 866, and a transceiver 868. Device 850 may also include storage devices to provide additional storage, such as a microdrive or other device. Each of the components 850, 852, 864, 854, 866, and 868 are interconnected using various buses, and some of these components may be mounted on a common motherboard or in other configurations as appropriate.

[0195] The processor 852 can execute instructions within the computing device 850, including instructions stored in memory 864. The processor may be implemented as a chipset of chips including multiple individual analog and digital processors. The processor can coordinate other components of the device 850, such as user interface control, applications run by the device 850, and wireless communication by the device 850.

[0196] The processor 852 can communicate with the user via a control interface 858 and a display interface 856 coupled to the display 854. The display 854 may be, for example, a TFT LCD (thin-film transistor liquid crystal display), an OLED (organic light-emitting diode) display, or other suitable display technology. The display interface 856 may include appropriate circuitry for driving the display 854 to present graphic information and other information to the user. The control interface 858 can receive commands from the user, translate them, and submit them to the processor 852. Furthermore, an external interface 862 can be provided to communicate with the processor 852, enabling short-range communication of the device 850 with other devices. The external interface 862 may provide, for example, wired communication in some implementations and wireless communication in other implementations, and multiple interfaces may be used.

[0197] Memory 864 stores information within the computing device 850. Memory 864 may be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Extended memory 874 is also provided and may be connected to device 850 via an extension interface 872, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such extended memory 874 may provide additional storage space to device 850, or it may store applications or other information related to device 850. In detail, extended memory 874 may contain instructions for executing or supplementing the processes described above, and may also contain secure information. For example, extended memory 874 may be provided as a security module for device 850 and may be programmed with instructions that enable secure use of device 850. Furthermore, secure applications may be provided via a SIMM card, along with additional information, such as by placing identification information on the SIMM card in a hack-proof manner.

[0198] Memory may include, for example, flash memory and / or NVRAM memory, as will be discussed later. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more actions, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 864, extended memory 874, or memory on processor 852, which can be received, for example, via transceiver 868 or external interface 862.

[0199] Device 850 may communicate wirelessly via a communication interface 866, which may include digital signal processing circuitry as needed. The communication interface 866 may provide communication in various modes or protocols, including, among others, GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA®, CDMA2000, or GPRS. Such communication may be performed, for example, via a transceiver 868. Furthermore, short-range communication may be performed using Bluetooth, Wi-Fi, or other such transceivers (not shown). Additionally, a GPS (Global Positioning System) receiver module 870 may provide device 850 with additional navigation and location-related wireless data, which may be used as appropriate by applications running on device 850.

[0200] Device 850 can also perform voice communication using the audio codec 860, which can receive speech information from the user and convert it into usable digital information. Similarly, the audio codec 860 can generate audible sound for the user, for example, through the speaker in the handset of device 850. Such sound may include sounds from telephone voice calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 850.

[0201] The computing device 850 can be implemented in several different forms, as shown in the figure. For example, it can be implemented as a mobile phone 880. It can also be implemented as part of a smartphone 882, a personal digital assistant, or other similar mobile device.

[0202] The various implementations of the systems and technologies described herein can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include one or more computer programs executable and / or interpretable on a programmable system, comprising at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0203] These computer programs (also known as modules, programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, as well as in assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” mean any computer program product, apparatus, and / or device (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, and include machine-readable mediums that receive machine instructions as machine-readable signals. The term “machine-readable signal” means any signal used to provide machine instructions and / or data to a programmable processor.

[0204] To provide user interaction, the systems and technologies described herein can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, or an LED (light-emitting diode)) and a keyboard and pointing device (e.g., a mouse or trackball) on which the user can provide input to the computer. Other types of devices for providing user interaction may also be used. For example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic input, voice input, or tactile input.

[0205] The systems and technologies described herein may be implemented in computing systems, or in any combination of such backend, middleware, or frontend components, including backend components (e.g., as data servers), middleware components (e.g., application server devices), or frontend components (e.g., client computers having a graphical user interface or web browser that allows users to interact with implementations of the systems and technologies described herein). The components of the system may be interconnected by digital data communications (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the internet.

[0206] A computing system may include a client and a server. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises when computer programs run on each other's computers and have a client-server relationship with one another.

[0207] In some implementations, the computing device depicted in Figure 8 may include sensors that interface with virtual reality or a headset (VR headset / AR headset / HMD device 890). For example, one or more sensors included on computing device 850 or other computing devices depicted in Figure 8 may provide input to the AR / VR headset 890, or generally provide input to the AR / VR space. Examples of sensors include, but are not limited to, touchscreens, accelerometers, gyroscopes, pressure sensors, biometric sensors, temperature sensors, humidity sensors, and ambient light sensors. Computing device 850 can use sensors to determine the absolute position and / or detected rotation of the computing device in the AR / VR space, which can then be used as input to the AR / VR space. For example, computing device 850 can be incorporated into the AR / VR space as a virtual object such as a controller, laser pointer, keyboard, or weapon. The user's placement of the computing device / virtual object when incorporated into the AR / VR space allows the user to position the computing device so that the virtual object appears in a particular manner within the AR / VR space.

[0208] In some implementations, one or more input devices included on or connected to the computing device 850 can be used as inputs to the AR / VR space. Examples of input devices include, but are not limited to, touchscreens, keyboards, one or more buttons, trackpads, pointing devices, mice, trackballs, joysticks, cameras, microphones, input-enabled earphones or pads, game controllers, or other connectable input devices. When the computing device is integrated into the AR / VR space, users interacting with the input devices included on the computing device 850 can perform specific actions within the AR / VR space.

[0209] In some implementations, one or more output devices included on the computing device 850 may provide output and / or feedback to the user of the AR / VR headset 890 in the AR / VR space. The output and / or feedback may be visual, tactical, or audible. Examples of output and / or feedback include, but are not limited to, rendering of the AR / VR space or virtual environment, vibration, turning one or more lights or strobes on and off or blinking and / or flashing, sounding alarms, playing chimes, playing music, and playing audio files. Examples of output devices include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light-emitting diodes (LEDs), strobes, and speakers.

[0210] In some implementations, a computing device 850 can be placed inside an AR / VR headset 890 to create an AR / VR system. The AR / VR headset 890 may include one or more positioning elements that allow the computing device 850, such as a smartphone 882, to be positioned appropriately within the AR / VR headset 890. In such implementations, the display of the smartphone 882 can render a stereoscopic image representing the AR / VR space or virtual environment.

[0211] In some implementations, the computing device 850 may appear as another object in a computer-generated 3D environment. User interaction with the computing device 850 (e.g., rotating, shaking, touching the touchscreen, swiping a finger on the touchscreen) can be interpreted as interaction with an object in the AR / VR space. As just one example, the computing device could be a laser pointer. In such an example, the computing device 850 appears as a virtual laser pointer in the computer-generated 3D environment. When the user manipulates the computing device 850, the user in the AR / VR space sees the laser pointer moving. The user receives feedback from their interaction with the computing device 850, either on the computing device 850 itself or in the AR / VR environment on the AR / VR headset 890.

[0212] In some implementations, the computing device 850 may include a touchscreen. For example, the user can interact with the touchscreen in a specific manner, where what happens on the touchscreen can be mimicked to what happens in the AR / VR space. For instance, the user can use a pinch-type gesture to zoom in on the content displayed on the touchscreen. This pinch-type gesture on the touchscreen allows the user to zoom in on the information provided in the AR / VR space. In another example, the computing device may be rendered as a virtual book within a computer-generated 3D environment. In the AR / VR space, the pages of this book can be displayed within the AR / VR space, and the user's finger swiping on the touchscreen can be interpreted as turning / flipping the pages of the virtual book. As each page is turned / flipped, the user can not only see the page content changing, but may also be provided with audio feedback, such as the sound of turning the pages of a book.

[0213] In some implementations, in addition to computing devices, one or more input devices (e.g., mouse, keyboard) can be rendered within the computer-generated 3D environment. The rendered input devices (e.g., rendered mouse, rendered keyboard) can then be used in their rendered state within the AR / VR space to control objects within the AR / VR space.

[0214] Further examples of the systems and methods relating to this disclosure are described below. The first example relates to a method performed by a computer, which includes generating a repository of metadata based on multiple web pages that have been accessed and saved in the browser history of a web browser running on a computing device. The metadata includes, for each web page, a source event, an access timestamp, and at least one topic. The method performed by the computer further includes generating a history cluster based on the metadata, which includes a portion of multiple web pages related to a given topic. The history cluster generation is based on the source events and access timestamps of the web pages in the portion. The method performed by the computer further includes assigning a score to each web page in the portion. Each score is at least in part based on the access timestamp and the generated engagement score for each web page. Generate a history cluster list for a topic in response to a request to view browser activity associated with the topic. The history cluster list is organized according to its respective score and includes a list of visits associated with web pages in the history cluster that are determined to have a score that meets a threshold score. Display the history cluster list.

[0215] The first example relates to a system, which includes a computing device running a web browser and a web browser user interface generator. The user interface generator is configured to generate a search history user interface based on history clusters associated with a topic in response to a request to view search activity associated with a topic, where a history cluster includes multiple web pages that have been accessed and saved in the web browser's search history, and the search history user interface includes a history cluster list, which is rendered within the search history user interface and supplemented with data from history clusters associated with the topic, where the history cluster list includes multiple visit lists corresponding to at least some of the multiple web pages included in the history cluster, and action controls configured to resume previous searches associated with the history cluster. The system further includes a renderer configured to display the search history user interface.

[0216] In the second example, based on the first example, the user interface generator is further configured to receive at least a partial query in a query input area, determine that the partial query is related to a topic, and, in response to determining that the partial query is related to a topic, provide a process control configured to provide a request to view search activities associated with the topic.

[0217] In the third example, based on the second example, the process control is provided with partial query completion suggestions.

[0218] In the fourth example, which is based on any one of the first to third examples, the multiple visit list includes an indicator that some of the above-mentioned web pages belong to a web browser tab group or bookmark.

[0219] In Example 5, which is based on any one of Examples 1 through 4, the search history user interface further includes suggested related searches.

[0220] In Example 6, which is based on any one of Examples 1 through 5, the history cluster is generated based on metadata associated with the web pages in the above parts, and the metadata includes source events, access timestamps, engagement scores, or subtopics associated with topics.

[0221] In Example 7, based on Example 6, the engagement score includes the respective engagement metrics for each webpage in the above section, and each engagement metric is used to select the prominence level when displaying snippets for at least one webpage in the above section in the history cluster list.

[0222] In Example 8, which is based on any one of Examples 1 through 7, the user interface generator receives a request to delete a webpage that was rendered in one of several visit lists, and is further configured to, based on the request, clear the record of access to the stored webpage from the search history.

[0223] In the ninth example, the technique described herein relates to a method, the method comprising receiving a request to view search activity associated with a certain topic, and in response to the request, generating a search history user interface based on history clusters associated with the topic. A history cluster includes a plurality of web pages that have been accessed and saved in the search history of a web browser, the search history user interface includes a history cluster list, which is depicted within the search history user interface and includes data from history clusters associated with the topic, the history cluster list includes a plurality of visit lists corresponding to at least some of the plurality of web pages included in the history cluster, and action controls configured to resume previous searches associated with the history cluster. The method further comprises displaying the search history user interface.

[0224] A tenth example, based on the ninth example, further includes receiving at least a partial query in a query input area, determining whether the partial query is related to a topic, and providing a process control configured to provide a request to view search activities associated with the topic in response to determining that the partial query is related to a topic.

[0225] In the 11th example, based on the 10th example, the process control is provided with partial query completion suggestions.

[0226] In Example 12, which is based on any one of Examples 9 through 11, the multiple visit list includes an indicator that some of the above-mentioned web pages belong to a web browser tab group or bookmark.

[0227] In Example 13, which is based on any one of Examples 9 through 12, the search history user interface further includes suggested related searches.

[0228] In Example 14, which is based on any one of Examples 9 through 13, the history cluster is generated based on metadata associated with the web pages in the above parts, and the metadata includes source events, access timestamps, engagement scores, or topics and their associated subtopics.

[0229] In Example 15, based on Example 14, the engagement score includes the respective engagement metrics for each webpage in the above section, and each engagement metric is used to select the prominence level when displaying snippets for at least one webpage in the above section in the history cluster list.

[0230] In Example 16, which is based on any one of Examples 9 through 15, the method may also include receiving a request to delete a webpage that has been rendered in one of several visit lists, and, based on the request, clearing the record of access to the stored webpage from the search history.

[0231] In the 17th example, the technique described herein relates to a method performed by a computer, the method comprising generating a repository of metadata based on a plurality of web pages that have been accessed and saved in the browser history of a web browser running on a computing device. The metadata for each web page comprises a source event, an access timestamp, and at least one topic. The method further comprises generating a history cluster based on the metadata, which comprises some of the web pages of a plurality of web pages related to a certain topic, and assigning a score to each of the web pages in the plurality. Each score is at least in part based on an access timestamp and an engagement score for the respective web page. The method further comprises generating a history cluster list for a topic in response to a request to view browser activity associated with the topic. The history cluster list comprises a list of visits associated with web pages that have been determined to have scores that meet a threshold score in the history cluster, and the list of visits is organized according to the respective score. The method further comprises displaying the history cluster list.

[0232] In Example 18, based on Example 17, historical cluster generation is based on the source events and access timestamps of the web pages mentioned above.

[0233] In Example 19, based on Example 17 or Example 18, a visit list in the visit list contains a snippet for at least one web page from among the multiple web pages in the above part, and a history cluster contains associated action controls for initiating actions related to the history cluster.

[0234] In Example 20, based on any one of Examples 17 through 19, the history cluster list further includes multiple visit lists corresponding to at least one web page in the above parts, and action controls configured to resume previous searches associated with the multiple visit lists. The multiple visit lists are presented within the web browser's browser history.

[0235] In Example 21, which is based on any one of Examples 17 through 20, the metadata further includes, for a particular webpage among several webpages, a determined webpage entity, multiple related searches, or time spent on the webpage.

[0236] In Example 22, which is based on any one of Examples 17 through 20, the metadata further includes a webpage identifier defined for the webpages in the above section, the webpage identifier indicating whether each webpage in the above section is part of a tab group, bookmark, or search results page accessed by a computing device.

[0237] In Example 23, which is based on any one of Examples 17 through 22, the engagement score includes the respective engagement metrics for the web pages in the above section, and each engagement metric is used to select the prominence level when displaying snippets for at least one of the multiple web pages in the above section in the history cluster list.

[0238] In Example 24, based on any one of Examples 17 through 23, generating a metadata repository further includes deduplicating multiple web pages that have been accessed and saved in the browser history, the deduplicating includes determining duplicate web page accesses among the accessed web pages, selecting the web page with the most recent access timestamp with respect to the access timestamp associated with the duplicate web page access, and generating metadata to be stored in the repository for the web page with the most recent access timestamp.

[0239] In Example 25, which is based on any one of Examples 17 through 24, the history cluster has a title, and the method further includes displaying the title as a suggested link, and a request is issued to view the history cluster by selecting the suggested link.

[0240] In example 26, based on example 25, the suggested link appears either as a search suggestion in a search engine or as an option in the search history of a web browser.

[0241] Several implementations have been described. Nevertheless, it should be understood that various modifications may be made without deviating from the spirit and scope of this disclosure.

[0242] Furthermore, the logical flow depicted in the diagram does not require a specific order or sequence shown to obtain the desired result. Additionally, other steps may be provided to the described flow, or steps may be deleted, and other components may be added to or removed from the described system. Therefore, other implementations exist within the scope of the following claims.

[0243] A computer system (e.g., a computing device) can be configured to communicate wirelessly with a network server via a communication link, which is established with the network server using any known wireless communication technology and protocol adapted for network communication, including radio frequency (RF), microwave frequency (MWF), and / or infrared frequency (IRF) wireless communication technologies and protocols.

[0244] In accordance with aspects of this disclosure, the various implementations of the technologies described herein may be implemented in digital electronic circuits, or in computer hardware, firmware, software, or a combination thereof. The implementations may be implemented as computer program products (e.g., computer programs tangibly embodied in information carriers, machine-readable storage devices, computer-readable media, or tangible computer-readable media) for processing by or controlling the operation of data processing devices (e.g., programmable processors, computers, or multiple computers). In some implementations, tangible computer-readable storage media may be configured to store instructions that, when executed, cause a processor to execute a process. Computer programs, such as the computer programs described above, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, such as as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs may be deployed to be processed on a single computer or on multiple computers located in one place or distributed across multiple locations interconnected by a communication network.

[0245] The specific structural and functional details disclosed herein are representative only for illustrative purposes to illustrate exemplary implementations. However, these exemplary implementations can be embodied in many alternative forms and should not be construed as being limited to those explicitly stated herein.

[0246] The terminology used herein is intended solely to describe specific implementations and is not intended to limit them to those implementations. Where used herein, the singular forms “a,” “an,” and “the” are intended to include the plural unless otherwise explicitly indicated in the context. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or “including” do not exclude the presence or addition of one or more other features, steps, actions, elements, components, and / or groups thereof.

[0247] To facilitate descriptions of the relationship between one element or feature and another, as shown in the diagram, terms describing spatial relationships, such as “beneath,” “below,” “lower,” “above,” and “upper,” may be used herein. It will be understood that these terms describing spatial relationships are intended to encompass different orientations of the device in use or operation, in addition to the orientation depicted in the diagram. For example, if the device in the diagram is inverted, elements described as “below” or “beneath” other elements or features will consequently be oriented “above” those other elements or features. Thus, the term “below” may encompass both above and below orientations. The device may be otherwise oriented (rotated 70 degrees or in other orientations), and the terms describing spatial relationships used herein may be interpreted accordingly.

[0248] While terms such as “first,” “second,” and so on may be used in this specification to describe various elements, it should be understood that these elements are not limited by these terms. These terms are used solely to distinguish one element from another. For example, an element “first” may be referred to as an element “second” without departing from the teachings of this disclosure.

[0249] Unless otherwise specified, terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art to which the concepts belong. Terms such as those defined in commonly used dictionaries should be interpreted in a way that is consistent with their meaning in the context of the relevant art and / or herein, and it will be further understood that they should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0250] While the specific features of the described implementations have been explained as described herein, those skilled in the art will likely conceive of numerous modifications, substitutions, changes, and equivalents. Therefore, it should be understood that the accompanying claims are intended to encompass modifications and changes that fall within the scope of those implementations. The accompanying claims are presented as examples only, not as limitations, and various modifications in form and detail are possible. Any part of the apparatus and / or method described herein can be combined in any combination, except for mutually exclusive combinations. The implementations described herein may include various combinations and / or partial combinations of the functions, components, and / or features of the different implementations described.

Claims

1. It is a system, A computing device comprising, the computing device comprising, at least one processor, and memory storing instructions that, when executed by the at least one processor, cause the computing device to perform an operation, the operation is, Through machine learning analysis of metadata associated with multiple web pages based on browsing history saved in the web browser's search history, historical clusters representing activity are generated from the multiple web pages, which are grouped by topic. This includes generating a search history user interface, and the search history user interface is The search history user interface includes a list of history clusters to which data from the aforementioned history clusters has been added, the list of history clusters includes a list of visits corresponding to at least some of the multiple web pages included in the history clusters, and the search history user interface includes a list of history clusters to which data from the aforementioned history clusters has been added, the list of history clusters includes a list of visits corresponding to at least some of the multiple web pages included in the aforementioned, and the search history user interface includes a list of history clusters to which data from the aforementioned history clusters has been added, the list of history clusters includes a list of visits, and the search history user interface includes a list of history clusters to which data from the aforementioned history clusters has been added, the list of history clusters includes a list of visits, and the search history user interface includes a list of history clusters to which data from the aforementioned history clusters has been added, the list of history clusters includes a list of visits, and the search history user interface includes a list of history clusters to which data from the aforementioned history cluster The system further includes a graphical action control programmatically linked to the history cluster, the graphical action control being configured to initiate an action relating to the history cluster in response to a selection. The aforementioned system, A system further comprising a renderer configured to display the aforementioned search history user interface.

2. The aforementioned operation is, The query input area accepts at least a partial query, Determining that the aforementioned partial query matches the topic of the historical cluster, The system according to claim 1, further comprising providing a process control configured such that a search process based on the history cluster is resumed in response to determining that the partial query matches a topic in the history cluster.

3. The system according to claim 2, wherein the process control is provided with suggestions to complete the partial queries.

4. The system according to claim 1, wherein the initiation of an action relating to the history cluster includes opening at least a portion of the plurality of web pages as tabs in a new tab group.

5. The system according to claim 1, wherein the initiation of an action relating to the history cluster further includes initiating a search for proposed topics related to the topics of the history cluster.

6. The system according to claim 1, wherein the machine learning analysis further involves an analysis using the engagement scores or time spent on the multiple web pages.

7. The system according to claim 6, wherein the engagement score includes an engagement metric for each of the web pages in the portion, and each of the engagement metrics is used to select a prominence level when displaying a snippet for at least one web page in the portion in the history cluster list.

8. The system according to any one of claims 1 to 7, wherein the initiation of an action relating to the history cluster includes, in response to a selection, clearing the stored records of access to the plurality of web pages included in the history cluster list.

9. It is a method, Through machine learning analysis of metadata associated with multiple web pages based on browsing history saved in the web browser's search history, historical clusters representing activity are generated from the multiple web pages, which are grouped by topic. This includes generating a search history user interface, and the search history user interface is: The history cluster list includes a list of history clusters to which data from the aforementioned history cluster has been added, and the history cluster list includes a list of visits corresponding to at least some of the multiple web pages included in the history cluster, The aforementioned search history user interface is: The system further includes a graphical action control programmatically linked to the history cluster, the graphical action control being configured to initiate an action relating to the history cluster in response to a selection. The aforementioned method, A method further comprising displaying the aforementioned search history user interface.

10. The query input area accepts at least a partial query, Determining that the aforementioned partial query matches the topic of the historical cluster, The method of claim 9, further comprising providing a process control configured to expand the history cluster in response to determining that the partial query matches a topic in the history cluster.

11. The method according to claim 10, wherein the process control is provided with suggestions to complete the partial query.

12. The method according to claim 9, wherein the plurality of visit lists include information indicating that the portion of the plurality of web pages belongs to a tab group or bookmark of the web browser.

13. The method according to claim 12, wherein the initiation of an action relating to the history cluster includes opening the portion of the plurality of web pages as tabs in the tab group.

14. The method according to claim 9, wherein the machine learning analysis further comprises a machine learning analysis using the engagement scores or time spent on the multiple web pages.

15. The method according to claim 14, wherein the engagement score includes an engagement metric for each of the web pages in the portion, and each of the engagement metrics is used to select a prominence level when displaying a snippet for at least one web page in the portion in the history cluster list.

16. Receiving a request to delete a webpage that has been rendered in one of the aforementioned multiple visitor lists, The method according to any one of claims 9 to 15, further comprising deleting the records of access to the stored web page from the search history based on the aforementioned request.

17. A method by which a computer performs an action. This includes generating a metadata repository based on multiple web pages that have been accessed and saved in the browser history of a web browser running on a computing device, wherein the metadata for each web page includes a source event, an access timestamp, and at least one topic. The method that the aforementioned computer will perform is: Through machine learning analysis of metadata associated with multiple web pages based on the browsing history saved in the browser history of the web browser, historical clusters representing activity are generated from the multiple web pages, which are grouped by topic. The method further includes assigning a score to each of the web pages corresponding to at least a portion of the plurality of web pages, wherein each score is at least partially based on the access timestamp and the engagement score for each of the web pages. The method that the aforementioned computer will perform is: In response to a request to view browser activity associated with the topic, the method further includes generating a history cluster list for the topic, wherein the history cluster list is: The history cluster includes a list of visits associated with web pages that are determined to have a score that satisfies a threshold score, and the list of visits is organized according to the respective scores. The system further includes a graphical action control programmatically linked to the history cluster, the graphical action control being configured to initiate an action relating to the history cluster in response to a selection. The method that the aforementioned computer will perform is: A method performed by a computer, further comprising displaying the aforementioned list of history clusters.

18. A method for generating a history cluster, performed by a computer according to claim 17, based on the source event and access timestamp of the web page in the part thereof.

19. The computer method according to claim 17, wherein one of the visitor lists includes a snippet relating to at least one webpage among the plurality of webpages in the part.

20. The method performed by a computer according to claim 17, wherein the initiation of an action relating to the history cluster further includes initiating a search for proposed topics related to the topics of the history cluster.

21. The computer-operated method according to claim 17, wherein the metadata further includes, for one of the plurality of web pages, a determined web page entity, a plurality of related searches, or the time spent on the web page.

22. The method performed by a computer according to claim 17, wherein the metadata further includes a webpage identifier defined for the webpage in the portion, the webpage identifier indicating whether each webpage in the portion is part of a tab group, a bookmark, or a search results page accessed by the computing device.

23. The method performed by a computer according to claim 17, wherein the engagement score includes an engagement metric for each of the web pages in the portion, and each of the engagement metrics is used to select a prominence level when displaying a snippet for at least one of the plurality of web pages in the portion in the history cluster list.

24. Generating the metadata repository further includes deduplicating the multiple web pages that have been accessed and saved in the browser history, and deduplicating means Determining duplicate web page accesses among the multiple web pages accessed, With respect to the duplicate web page accesses, the web page with the most recent access timestamp associated with the duplicate web page accesses is selected. A method performed by a computer according to claim 17, comprising generating metadata to be stored in the repository for the web page having the most recent access timestamp.

25. The method performed by a computer according to claim 17, wherein the history cluster has a title, the method further comprises displaying the title as a suggested link, and the request to view the history cluster is issued by selecting the suggested link.

26. The proposed link will be displayed as a search suggestion in the search engine. The proposed link will be displayed as an option in the search history of the web browser. The method performed by the computer according to claim 25.

27. A computing device configured to perform the method described in any one of claims 17 to 26.

Images