Method and system for generating action elements of suggested content for an ongoing task

Abstract

This disclosure relates to methods and systems for generating action elements for suggested content in an ongoing task. Computational systems and methods can be used to expose optional action elements for at least one ongoing task. Specifically, this disclosure provides a general pipeline for identifying potential tasks that a user is continuously interested in or has not yet completed, thereby enabling the generation of content item suggestions to further advance the identified user's task. This pipeline can incorporate probability transition graphs, machine learning models, and / or historical data to determine the relevance and completion status of tasks that the user may wish to continue performing.

Description

Technical Field

[0001] This disclosure generally relates to identifying ongoing tasks associated with a user. More specifically, this disclosure relates to action elements that generate suggested content for ongoing tasks identified based on historical user web browsing data. Background Technology

[0002] A web browser application (“browser”) can retrieve content from a web server and display it on a user’s device. The user can enter a uniform resource locator (URL) that allows the browser to retrieve data (e.g., content) associated with that URL (e.g., by using the Hypertext Transfer Protocol to communicate with the web server).

[0003] In some cases, communication between browsers and web servers can be encrypted for privacy and security purposes. Once a browser retrieves a webpage, its rendering engine can display it on the user's device, including image and video formats supported by the browser. Most browsers can use internal caching of webpage resources to improve load times for subsequent visits to the same page. Caching can store many items, such as large images, so they don't need to be downloaded from the server again.

[0004] Generating personalized search queries is the process of providing users with personalized content using web browsing technology. Some standard query personalization techniques rely on aggregated user information, such as common search queries. However, existing methods may not attempt to identify a specific user's specific task.

[0005] While progress has been made in the field of personalized search queries, existing methods are often limited to users making single, isolated queries using pre-existing search bars, without taking into account any alternative or overall purpose of a given user's search. Summary of the Invention

[0006] Aspects and advantages of embodiments of this disclosure will be set forth in part in the description which follows, or may be learned from the description or by practice of the embodiments.

[0007] One example aspect of this disclosure relates to a computer-implemented method for generating task-specific action elements. The method includes a computing system obtaining historical user data describing historical user actions taken in one or more past online sessions of the user. The historical user data has been annotated with annotations describing content attributes associated with the user actions. The method includes the computing system identifying one or more tasks based on the historical user data. The method includes the computing system determining suggested content items for each of the one or more ongoing tasks. The method includes the computing system exposing optional action elements for displaying at least one ongoing task to a user. The optional action elements are configured to provide access to at least one suggested content item for at least one ongoing task.

[0008] Other aspects of this disclosure relate to various systems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices.

[0009] These and other features, aspects, and advantages of the various embodiments of this disclosure will be better understood by referring to the following description and the appended claims. The accompanying drawings, which are incorporated in and form a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the relevant principles. Attached Figure Description

[0010] A detailed discussion of embodiments applicable to those skilled in the art is set forth in the description with reference to the accompanying drawings, in which:

[0011] Figure 1 A block diagram of an example computing system according to an example embodiment of the present disclosure is depicted.

[0012] Figure 2 An exemplary client device is shown, including one or more processors and a user interface.

[0013] Figure 3 This illustrates exemplary web content displayed in a browser window within the user interface of a client device.

[0014] Figure 4 A block diagram depicts an example content recommendation model according to an example embodiment of the present disclosure.

[0015] Figure 5 An example user interface according to an exemplary embodiment of this disclosure is shown, illustrating how action elements are presented to a user.

[0016] Figure 6 An example user interface according to an exemplary embodiment of the present disclosure is shown, illustrating how a dedicated dashboard can be used to present a user with multiple ongoing tasks having associated action elements.

[0017] Figure 7 An example user interface according to an example embodiment of the present disclosure is shown, illustrating how a user can interact with a specific, continuously ongoing task.

[0018] Figure 8 An example user interface according to an example embodiment of the present disclosure is shown, illustrating how a user can interact with a specific discrete ongoing task.

[0019] Figure 9 A flowchart depicts an example method for generating action elements of suggested content for performing an ongoing task, according to an example embodiment of this disclosure.

[0020] The repeated reference numerals in multiple figures are intended to identify the same features in various implementations.

[0021] Specific implementation

[0022] Overview

[0023] Generally, this disclosure relates to a computational system and method that can be used to reveal optional action elements of at least one ongoing task, such as action elements configured to provide access to at least one suggested content item associated with the ongoing task. Specifically, this disclosure provides a general pipeline for identifying potential tasks that a user is continuously interested in or has not yet completed, thereby enabling the provision of suggestions for content items to further advance the identified user's task. This pipeline can incorporate probability transition graphs, machine learning models, and / or historical data to determine the relevance and completion status of tasks that the user may wish to continue performing.

[0024] More specifically, the example computational system can identify ongoing tasks. For instance, the computational system can identify ongoing tasks by analyzing historical user data to infer what tasks a user is likely to engage in in the future. By leveraging what the user has done in the past, the computational system can determine the overall categories of activities, and based on these categories, it can evaluate a pipeline of predicted steps the user is most likely to take for the identified task. For example, if the computational system identifies that a user has already engaged in a shopping activity, it can generate (e.g., using probability transition graphs, machine learning models, etc.) the most likely shopping pipeline (e.g., inspiration, discovery, consideration, confirmation, purchase, post-purchase) and / or evaluate the user's state or position along such a pipeline. Thus, the computational system can use historical user data to determine which stage of the pipeline the user is most likely to engage in next.

[0025] In some implementations, the computing system can optimize the pipeline of activities a user is likely to engage in to complete a specific task. The computing system can optimize the pipeline by predicting the most likely subsequent steps a user might take and identifying relevant content. This relevant content can then be presented to the user, eliminating the need for manual searching. Relevant content can be presented to the user by exposing optional action elements. These optional action elements can be displayed in specific dashboards that the user can navigate to view multiple tasks identified by the computing system. For example, a user can interact with a dedicated dashboard to indicate which of several available tasks they wish to interact with. Furthermore, optional action elements can be overlaid on the user's current browsing session to provide suggestions with which the user can interact, thereby driving the user further into the ongoing task.

[0026] Example methods for revealing optional action elements of at least one ongoing task may include obtaining historical user data. Specifically, historical user data can describe historical user actions taken in one or more past online sessions. For example, historical user data can be annotated with comments. In particular, comments can describe attributes of content associated with user actions. For example, a computing system could obtain historical user data and annotate a portion of the historical data as related to searching for or purchasing headphones. Even more specifically, comments can be based on metadata associated with the content.

[0027] A computing system can identify one or more ongoing tasks. One or more ongoing tasks can be identified based on historical user data (e.g., using historical user web browsing data). Ongoing tasks can be discrete tasks (e.g., buying items, planning vacations, etc.). Ongoing tasks can also be continuous tasks (e.g., cooking, watching movies, etc.). For example, a computing system can identify a discrete task such as buying a set of headphones.

[0028] The computing system can determine one or more suggested content items. One or more suggested content items can be determined for each of one or more ongoing tasks. For example, if a user is undertaking the task of buying a pair of headphones, the computing system can suggest content items related to the purchase. For example, the computing system can suggest content items such as headphone buying guides, headphone reviews viewed by the user based on historical browser data, various headphones themselves, suggested search queries, comparison guides, or related products (e.g., microphone attachments, headphone covers, etc.). The suggested content items can change based on the identified user's ongoing task. In particular, suggested content items can include adding content to a user database (e.g., a list). For example, if the computing system determines that the user is interested in cooking, the suggested content items could be recipes that the user can add to their cookbook by combining other suggested content (e.g., cooking utensils, grocery lists, grocery store locations, etc.). As another example, if the computing system determines that the user is interested in movies, the suggested content items could include movies to add to a watchlist by combining other suggested content (e.g., movie reviews, related movies, etc.).

[0029] Optional action elements can be exposed to the user. Optional action elements can be associated with at least one ongoing task. Specifically, optional action elements can be configured to provide access to at least one suggested content item for at least one ongoing task. Optional action elements can include an image associated with the ongoing task to which the action element relates. For example, an optional action element associated with a user attempting to purchase headphones could include an image of headphones on the action element itself.

[0030] In some implementations, historical user actions can be retrieved across multiple web pages. For example, historical user actions can be retrieved from multiple web pages that a user has opened, regardless of whether those pages were closed after being opened. Furthermore, historical user actions can be retrieved across multiple web browser sessions. Specifically, historical user actions can be retrieved from multiple instances where a user uses a web browser to initiate a search, closes the web browser for a period of time, then returns to the web browser and initiates another search (e.g., a different search). For example, historical user actions can be retrieved for a predetermined amount of time for a web browser session. Specifically, historical user actions can be retrieved within 30 days prior to the timeframe in which they were retrieved. A web browser session can correspond to: the loading and unloading of discrete instances of a web browser application (e.g., a session corresponds to loading and unloading instances of a web browser application into device memory); the use of a web browser application over a period of time (e.g., each different day of using the web browser application corresponds to a session); a given user's use of a web browser during login; different tabs within the web browser; and / or other segments of browser application usage over time.

[0031] In some implementations, the computational system can identify one or more ongoing tasks. Furthermore, one or more ongoing tasks can be annotated based on historical user data. This historical user data can be annotated to describe attributes of content associated with user actions. As an example, historical user data instructing a user to search for the best headphones of 2021 can be annotated to describe it as related to purchasing headphones. Specifically, the annotated historical user data can be arranged into multiple clusters. Furthermore, one or more ongoing tasks can be identified based on historical user data, at least in part, based on multiple clusters. In one example, identifying one or more ongoing tasks may include performing an edge thresholding algorithm to identify one or more master clusters among multiple clusters. Examples of “edges” or “dimensions” that product image searches might match include recognition-derived attributes such as categories (e.g., “headphones”), attributes (e.g., “noise cancellation”, “gaming”, etc.), or other semantic dimensions and / or visual attributes (e.g., “dark with light,” “light is a thin line that makes up 40% of the entire color space”, etc.) including machine-generated visual attributes (such as machine-extracted visual features or machine-generated visual embeddings).

[0032] In some implementations, one or more ongoing tasks can be identified based on historical user data. Specifically, historical user data can be fed into a machine learning model. Even more specifically, the machine learning model can generate one or more ongoing tasks based on the input historical user data.

[0033] In some implementations, identifying one or more ongoing tasks based on historical user data can be at least partially based on relevance scores. Specifically, relevance scores can be at least partially based on timestamps associated with historical user data. Even more specifically, content with higher relevance scores can be weighted more heavily than content with lower relevance scores. For example, a web browsing search performed within the previous 24 hours may have a higher relevance score than a web browsing search performed within the previous 30 days. Furthermore, timestamps can be associated with the duration spent browsing specific content. Specifically, relevance scores can be at least partially based on the duration spent browsing specific content. For example, content viewed for a longer period may have a higher relevance score. Additionally, relevance scores can be at least partially based on the repetition of web browsing searches. For example, content that is repeatedly viewed may have a higher relevance score. Even more specifically, relevance scores may be required to meet a relevance threshold in order for associated content to qualify for the identification of potential user tasks.

[0034] In some implementations, identifying one or more ongoing tasks based on historical user data may include generating a probability distribution. Specifically, the probability distribution may include a set of subsequent step intentions (i.e., intentions to perform actions) based at least in part on historical user data. Even more specifically, historical user data may be extracted from multiple users. Furthermore, suggested subsequent steps may be identified based at least in part on completion metrics. Specifically, identifying one or more ongoing tasks based on historical user data may include identifying suggested subsequent steps. Even more specifically, suggesting subsequent steps may be based in part on completion metrics, where completion metrics indicate the user's state regarding completion of predetermined checkpoints associated with one or more ongoing tasks. Even more specifically, predetermined checkpoints may indicate what information the user has consumed. For example, the identified suggested subsequent steps may differ depending on the degree of completion of the task the user is experiencing or what checkpoints the user has already completed. For example, if a user is trying to buy headphones, if the user is in the initial stages (e.g., the user has not hit any checkpoints), the user may get the most help by viewing a shopping guide. On the other hand, if the user is in a later stage of the process (e.g., the user has hit multiple checkpoints, such as viewing a shopping guide), the user may be more interested in reviews of a particular pair of headphones. Furthermore, while historical data can indicate a user's strong interest in completing a task, the user may have already recently completed it, leaving little further information about the completed task to reveal. Specifically, completion metrics can be based, at least in part, on URLs associated with at least one predetermined checkpoint. Even more specifically, URLs associated with at least one predetermined checkpoint can be associated with one or more ongoing tasks. For example, a URL from a purchase confirmation page could indicate that a user has purchased a set of headphones and has completed that task. Therefore, in response, the computing system can remove the headphones from the list of potential ongoing tasks already identified by the system.

[0035] In some implementations, identifying one or more ongoing tasks based on historical user data may include accessing a continuously updated probability transition graph that describes a predicted pipeline of steps a user can take to complete the task. Specifically, the probability transition graph can identify subsequent step intentions for the identified tasks. More specifically, the probability transition graph may be based on multiple historical user tasks, which may span multiple different users. Even more specifically, the probability transition graph may reflect different subsequent step intentions within the ongoing task requirements of different categories of tasks. For example, even in a shopping task, differences may exist in the probability transition graph depending on the product (e.g., TV vs. mobile phone vs. sofa). Specifically, a unique distribution may be generated for each task category and subcategory. Based on the probability transition graph, a computational system can identify subsequent step intentions for one or more ongoing tasks. Specifically, the probability transition graph may include a hierarchical representation of the subsequent step intentions for one or more ongoing tasks, at least partially based on historical user data. Even more specifically, the computational system may select one or more content intentions based at least partially on the hierarchical representation of subsequent step intentions. As an example, determined or predicted user preferences can determine the hierarchical representation of subsequent step intentions (e.g., common product attributes of all explored products, brand similarity, etc.).

[0036] In some implementations, suggested content items for each of one or more subsequent step intentions within an ongoing task can be ranked. Specifically, suggested content items for each of one or more subsequent step intentions within an ongoing task can be ranked based on at least one quality attribute. Even more specifically, quality can be one or more other user engagement levels (e.g., the percentage of other users who interact with the suggested content item when presented as an option), comments (e.g., comments on articles that can be offered as suggested content), freshness (e.g., how new or old an article is), or content relevance (e.g., a headphone buying guide compared to a gaming computer review).

[0037] In some implementations, exposing optional action elements for displaying at least one ongoing task to the user includes displaying optional elements on a dedicated dashboard. For example, a dedicated dashboard can be configured to display multiple ongoing tasks. For instance, multiple ongoing tasks (e.g., “Headphone Shopping”) can be depicted under a title representing an explicit task unit of the user, allowing the user to interact with one of the multiple ongoing tasks to move to another interface that allows the user to interact with a specific ongoing task at a more granular level.

[0038] Interacting with a specific ongoing task at a greater granularity allows users to have a more detailed impact on the content revealed by the computing system, and expands the breadth of suggestions users can view. For example, users can see current suggestions and preview implicit task units, such as suggested follow-up steps. Products can be tracked and viewed, and users can interact with tracked products by turning alerts on or off. Users can interact with specific revealed content to indicate whether the content is accurate in predicting useful information or whether the user is not interested (e.g., if a user indicates disinterest in a specific revealed content, that content can be reordered, and alternative content can be revealed). Another way users can interact more deeply with a specific ongoing task is by interacting with a timeline of content they have already interacted with while pursuing that particular ongoing task.

[0039] Dedicated dashboards can present optional action elements that indicate suggested content items in various ways. For example, optional action elements can be presented on a carousel (e.g., suggested movies to watch), as a preview of a suggested webpage (e.g., a headphone buying guide), or as a notification of a price drop for a previously viewed item (e.g., a sale on a certain brand of headphones). Furthermore, the ranking of suggested content items can determine how action elements for specific suggested content items are presented. For example, suggested content items ranked above a certain threshold can be presented with larger icons (e.g., images describing the content), while suggested content items ranked below a certain threshold can be presented with smaller icons (e.g., only words below higher-ranked content). Even more specifically, badges can be overlaid on optional action elements to remind users of relevant information (e.g., a streaming service offering suggested movies, the time the user last viewed the content, etc.).

[0040] In some implementations, revealing optional action elements for displaying at least one ongoing task to the user includes obtaining current user web browser data. Current user web browser data may include one or more of text content or image content. One or more ongoing tasks can be identified based on historical user data, at least in part, of the text content or image content. Specifically, identifying one or more ongoing tasks based on historical user data may be at least in part based on semantic analysis of text content. Even more specifically, the content type can be determined at least in part based on semantic analysis of text content from the current user web browser data. Furthermore, identifying one or more ongoing tasks based on historical user data may be at least in part based on one or more compositional features of an identified image. Even more specifically, the content type can be determined at least in part based on one or more compositional features of an image from the current user web browser data. Continuing the example above, if the user is currently browsing an article about earplugs, the computing system can reveal optional action elements suggesting an article related to purchasing a set of headphones, at least in part, based on the work "earplugs" appearing in the article or an image of earplugs appearing in the article.

[0041] Therefore, this disclosure provides a computational system and method that can be used to expose optional action elements for at least one ongoing task, such as action elements configured to provide access to at least one suggested content item. Specifically, this disclosure provides a general pipeline to identify potential tasks that a user is continuously interested in or has not yet completed, thereby enabling the suggestion of content items to further advance the identified user's task. This pipeline combines hierarchical graphs, machine learning models, and historical data to determine the relevance and completion status of tasks that the user may wish to continue performing, and presents optional action elements in various interfaces.

[0042] The systems and methods disclosed herein offer numerous technical effects and benefits. As an example of a technical effect, the proposed techniques provide users with an immersive and beneficial experience by predicting subsequent steps in a task a user is attempting to complete or continue engaging with. Specifically, providing helpful suggestions about the task can significantly overcome user frustration, as many users engage in long tasks involving multiple searches and lack the ability to efficiently and productively utilize previously completed work by filtering out irrelevant information. Rather than predicting what a user might be searching for based solely on initial text query input, this disclosure demonstrates a way to directly provide the results of historical searches to more effectively offer the most helpful content to the user. Furthermore, this disclosure allows users to more fully and effectively utilize historical content and combine it with their future search needs to complete initiated tasks in ways that cannot be predicted based solely on initial input. Specifically, the proposed techniques reduce redundant or fruitless searches, thereby saving computational resources (e.g., processor usage, memory usage, network bandwidth, etc.) and user time and frustration. Specifically, by revealing relevant content for an ongoing task, the total number of searches required to perform that task can be reduced, which can save computational resources such as those described above. Furthermore, the proposed technology reveals content that is more relevant to the user, which may not be displayed based on a general search that the user may have already entered, thus reducing the consumption of meaningless content.

[0043] Exemplary embodiments of this disclosure will now be discussed in more detail with reference to the accompanying drawings.

[0044] Example devices and systems

[0045] Figure 1 A block diagram of an example computing system 100 according to an exemplary embodiment of the present disclosure is depicted, which performs personalized and / or intelligent search in at least a portion of response to a visual query. The computing system 100 includes a user computing device 102 and a web server 104 communicatively coupled via a network 180.

[0046] User computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0047] User computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be one or more processors operatively connected. Memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 114 can store data 116 and instructions 118 executed by processor 112 to cause user computing device 102 to perform operations.

[0048] In some implementations, the web browser application 124 of the user computing device 102 retrieves content related to objects mentioned in the user input component 122 of the user computing device 102. For example, the web browser may retrieve content related to a webpage requested by the user and then display that page to the user interface 158 of the device 102.

[0049] Although web browser application 124 Figure 1 While shown as being included in device 102, in other implementations, some or all of the functionality of web browser application 124 may be implemented on web server 104.

[0050] Web server 104 includes one or more front-end servers 136 and one or more back-end servers 140. Front-end server 136 can receive user input component 122 from a user computing device, such as user computing device 102 (e.g., from web browser 124). Front-end server 136 can provide image data to back-end server 140. Back-end server 140 can identify content related to objects identified in the user input data and provide that content to front-end server 136. Furthermore, front-end server 136 can provide content to mobile devices that receive image data from it.

[0051] Backend server 140 includes one or more processors 142 and memory 146. The one or more processors 142 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be one or more processors operatively connected. Memory 146 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 146 can store data 148 and instructions 150 executed by processor 142 to cause network server 104 to perform operations. Backend server 140 may also include a query processing system 154.

[0052] Alternatively, the backend server 140 may not be able to access a pre-existing file to provide to the frontend server 136. Therefore, the file can be generated upon request by another program that provides content 160 for communication with the web server 104. Furthermore, the web server 104 and the frontend server 136 can provide content to mobile devices receiving image data from them.

[0053] In some implementations, network server 104 includes one or more server computing devices or is implemented by one or more server computing devices. When network server 104 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0054] In some implementations, query processing system 154 includes multiple processing systems. An example system may allow the system to identify multiple candidate search results. For example, the system may identify multiple candidate search results when it first receives a user input component. Alternatively, the system may identify multiple search results after it has completed further processing. Specifically, the system may identify multiple search results based on a more targeted query that the system has already generated. Even more specifically, when the system first receives a user input component, it may generate multiple candidate search results, and then regenerate multiple candidate search results after further processing based on a more targeted query that the system has already generated.

[0055] After content is selected, it can be provided to user computing device 102, from which user input components are received, stored in content cache 138 of web server 104, and / or stored at the top of the memory stack of front-end server 136. In this way, content can be quickly presented to the user in response to a user request. If content is provided to user computing device 102, web browser 124 can store the content in content cache 134 or other fast access memory. For example, web browser 124 can store the content of an object and a reference to that object, allowing web browser 124 to identify the appropriate content for that object in response to determining how to present it.

[0056] User computing device 102 may also include one or more user input components 122 for receiving user input. For example, user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). Touch-sensitive components can be used to implement a virtual keyboard. Other example user input components include a microphone, a conventional keyboard, or other devices that a user can use to provide user input.

[0057] Network 180 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication over network 180 can be carried via any type of wired and / or wireless connection using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0058] Figure 1 An example computing system that can be used to implement this disclosure is shown. Other different distributions of the components may also be used. For example, some or all of the aspects of the visual search system may alternatively be located at and / or implemented at user computing device 102.

[0059] Figure 2 An exemplary client device 102 is shown, including one or more processors 210 and a user interface 212. Client 102 may be, for example, a personal computer, laptop computer, personal handheld device, mobile phone, game console, set-top box, embedded system, etc., but not limited to these devices. Client 102 may include any computing device capable of requesting network content from a network content server such as server 104.

[0060] Client 102 includes a user interface 212, which may include various types of inputs and outputs that allow the user to interact with client 102. Example inputs may include, but are not limited to, a mouse, keyboard, keypad, touchscreen, microphone, etc. Example outputs may include, but are not limited to, a display for visual output, a speaker for auditory output, etc. Many (if not all) of the above interface examples are driven, supported, or enhanced by hardware, firmware, and / or software located within or running within client 102. For example, to view web content, user interface 212 may include a web browser 214. Browser 214 includes software running on client 102 that allows the user to request and view web content (i.e., content provided from a server (e.g., server 104) connected to client 102 via one or more networks (e.g., the Internet or the World Wide Web). Examples of browser 214 include, but are not limited to, Google. TM The company's Chrome, Internet The company The company OperaSoftware TM ASA wait.

[0061] Figure 3 An exemplary browser window 314 displayed to a user of client device 102 is shown. A typical browser window 314 may include a title bar 316, a menu bar 318, a toolbar 320, a URL (Uniform Resource Locator) field 324, and a body 325. The title bar 316 may contain a title given to the webpage by its designer. The menu bar 318 may contain various drop-down menus, including tools and options related to the webpage being viewed. The toolbar 320 may contain various buttons 322, representing tools and options related to navigation to the currently viewed webpage and / or other webpages. The URL field 324 may contain the address of the webpage being viewed and may allow the user to type in the addresses of other webpages to view. The body 325 may contain the content of the webpage, including, for example, text 326 and images 330. The text in the body 325 may also represent hyperlinks 328, which are typically highlighted and / or underlined to distinguish them from regular text. If a user selects or clicks hyperlink 328, a new webpage may appear, for example, replacing or appending to the currently viewed webpage related to the text displayed in hyperlink 328.

[0062] When a user chooses to view a webpage, for example by entering an address in a URL field in a browser window such as browser window 314, clicking a hyperlink (in the currently viewed webpage, email, electronic document, etc.), using menu bar 318 or toolbar 320, or selecting to view specific web content on the webpage (e.g., a new view of an image), a request for the corresponding web content is sent to the appropriate web content server 104 via one or more networks 106. Server 104, for example, retrieves or obtains the requested web content from database 108 and provides (sends) it to the requesting client device 102. Client device 102 receives the requested web content and presents it for display, for example, in browser window 314.

[0063] Example model arrangement

[0064] Figure 4A block diagram of an example content recommendation model 200 according to an example embodiment of the present disclosure is depicted. In some implementations, the content recommendation model 200 is trained to receive a set 204 of input data describing historical user data and, as a result of receiving the input data 204, provides output data 206 identifying content recommended to a user by a machine learning model to complete subsequent steps of prediction in the identified task. Therefore, in some implementations, the task generation model 200 may include a task generation model 202 operable to generate one or more tasks that the user is currently engaging in. Specifically, the task generation model 202 may utilize the input data 204 to determine the current task based on historical user actions determined according to historical user data. For example, the task generation model may generate annotations for the historical user data (e.g., based on metadata associated with the historical user data). Based on the annotations associated with the historical user data, the task generation model 202 may generate one or more predicted tasks that the user may be engaging in. More specifically, the content recommendation model can utilize input data 204 by combining one or more predicted tasks that the user may be participating in, generated by task generation model 202, to generate output data 206, specifically, one or more contents predicted to complete subsequent steps in the predicted tasks that the user may be participating in.

[0065] Example Method

[0066] Figure 9 A flowchart depicts an example method performed according to an example embodiment of the present disclosure. Although for illustrative and discussion purposes, Figure 9 The steps are described in a specific order, but the method of this disclosure is not limited to the specific order or arrangement described. The steps of method 900 may be omitted, rearranged, combined, and / or modified in various ways without departing from the scope of this disclosure.

[0067] At point 902, the computing system can obtain historical user data describing historical user actions taken in one or more past online user sessions. Specifically, the historical user data describing historical user actions can be annotated with annotations describing attributes associated with the user actions.

[0068] At position 904, the computing system can identify one or more ongoing tasks based on historical user data.

[0069] At position 906, the computing system can determine suggested content items for each of one or more ongoing tasks.

[0070] At point 908, the computing system can expose optional action elements for displaying at least one ongoing task to the user. Specifically, the optional action elements can be configured to provide access to at least one suggested content item for at least one outing task.

[0071] Example Application

[0072] Figure 5 It shows Figure 9 A first exemplary application of the disclosed method is shown. 500 illustrates an exemplary user interface in which a user can participate in continuing the prediction task. For example, a user can participate in continuing the prediction task (e.g., a task-specific dashboard prompting the content of the prediction) by interacting with optional action elements 502 (e.g., touch, select, toggle, etc.). Specifically, optional action elements 502 may include icons 508 symbolizing the predicted task. For example, if the predicted task is to buy a set of headphones, optional action elements 502 may include an icon 508 of a set of headphones. The user interface 500 may additionally include at least one piece of content 512 directly recommended to the user. Specifically, a user can interact with at least one piece of directly recommended content 512 without participating in optional action elements 502. For example, a user can directly participate in directly recommended content 512 without interacting with a task-specific dashboard prompting the content of the prediction by interacting with optional action elements 502. The user interface 500 may include an overlay 504 illustrating how the directly recommended content 512 may affect the predicted task. Additionally, the user interface 500 may include labels 506 describing keywords associated with the predicted task. Additionally, the user interface 500 may include a preference switch 510, where a user can interact with the preference switch 510 to indicate whether they prefer directly recommended content 512. If the user activates the preference switch 510, the computing system can save the directly recommended content 512 to another recoverable location. The computing system may also have a button 514 that the user can interact with to launch an immersive dashboard related to the predicted task.

[0073] Figure 6 It shows Figure 9Another exemplary application of the method disclosed herein. 600 illustrates an exemplary immersive dashboard user interface in which a user can engage in continuing more than one predicted task. For example, the dashboard may indicate more than one predicted task with a text title 602, allowing the user to browse which task the user wishes to continue. The predicted task may include a panel 610 on a conveyor belt feature 604 indicating the next steps in the user's prediction for that task. For example, the panel 610 on the conveyor belt feature 604 may include features for the user to continue browsing options. The panel 610 on the conveyor belt feature 604 may include prediction items of interest. The panel 610 on the conveyor belt feature 604 may include content saved by the user. In particular, for example, the predicted task may be cooking. Thus, as a further example, the panel 610 on the conveyor belt feature 604 indicating the prediction items of interest may include recipes for predictions of interest. As another example, if the predicted task is to buy headphones, the panel 610 on the conveyor belt feature 604 indicating the prediction items of interest may include headphones or earplugs for sale. Panel 610 may include an information overlay diagram 612. Information overlay 612 can indicate information related to the content associated with panel 610. For example, information overlay 612 can indicate the price or source of the content associated with panel 610. Users can interact with detail icon 606 to indicate that they want to see more detailed suggestions related to the task. When interacting with detail icon 606, users can view additional content related to the task. When interacting with detail icon 606, users can participate in displaying a further dashboard of additional content related to a specific task. Additional content can have information overlay 608 that indicates why the computing system displays that specific content. For example, content can have information overlay 608 that indicates that the content was viewed by the user in the past (e.g., "viewed four days ago"), or that the content has similar characteristics to content saved by the user or content previously viewed by the user (e.g., "inspired by saved content").

[0074] Figure 7 It shows Figure 9Another exemplary application of the method disclosed herein is shown in Figure 700. An exemplary immersive dashboard user interface is illustrated, in which a user can engage in continuing a task based on a specific prediction. Specifically, Figure 700 illustrates how an immersive dashboard associated with a specific predicted task can allow a user to perform a specific task with increased specificity. For example, card 712 may indicate the predicted content associated with a specific task. Specifically, a designated action element 702 may be exposed, allowing the user to interact with the designated action element 702 to trigger a predicted action tailored to the content of the specific prediction in conjunction with the specific task. For example, the designated action element 702 may connect the predicted content to other features of a smart home (e.g., Nest, Echo, etc.). As another example, for card 712 suggesting a recipe, the designated action element 702 may include actions that the user can engage in, such as purchasing ingredients. Card 712 may also have a bookmark feature 704, allowing the user to indicate that they wish to revisit the content associated with card 712. In some cases, an overlay map may appear on the interface, prompting the user to indicate their feedback in response to the predicted content. For example, users can interact with affirmative icons 706 or negative icons 708 on the overlay map to indicate their feedback on the indicated content. Additionally, users can insert notes 710 associated with specific content snippets, where the notes can contain user-written memos about the related content. The immersive dashboard user interface 700 may also include a share icon 714, allowing users to share tasks with other users. Second users can be passive observers during the task or can be added as active contributors. In cases where more than one user contributes to a particular task, certain predictions can be attributed to specific users, and representations of those specific users (e.g., images, symbols, names, etc.) can be used to indicate which user a particular prediction is attributed to.

[0075] Figure 8 It shows Figure 9Another exemplary application of the method disclosed herein is shown in 800. Another exemplary immersive dashboard user interface is illustrated, in which a user can engage in continuing a specific predicted task. Specifically, 800 further illustrates how an immersive dashboard associated with a specific predicted task can allow a user to perform a specific task with increased specificity. For example, the immersive dashboard 800 may include a related activity search 802, wherein the related activity search 802 may reveal which keywords are included in the search. In some cases, the keywords included in the search may be hyperlinked, such that interaction with them initiates a complete search by the user. Alternatively, results from the related activity search 802 may be displayed in a results panel 808 adjacent to the related activity search 802. The results panel 808 may be displayed in a conveyor belt manner. The immersive dashboard user interface 800 may include a step counter 810, which indicates the number of subsequent steps that the calculation system has predicted. The number of subsequent steps indicated on the step counter 810 may be the number of the most highly recommended steps (e.g., steps above a certain confidence threshold). Furthermore, the predicted steps may be indicated in a step summary panel 804, wherein the step summary panel 804 may be hyperlinked, allowing the user to interact with the step summary panel 804 to initiate the associated steps. For example, a predicted step could be reviewing price drops for items the user has already indicated as of interest (e.g., by viewing them in the past, indicating them as favorites, adding them to a wishlist, bookmarking them, etc.). Another predicted step could be browsing similar products (e.g., browsing similar products below a certain price). The immersive dashboard 800 may also include a user interface prompt 806. The user interface prompt 806 can allow the user to interact with it to present even more detailed predictions output by the computing system. As an example, the user interface prompt 806 could be associated with a step counter 810 and a step summary panel 804, allowing the user to select the user interface prompt to view all the subsequent steps of the predictions, rather than just viewing the most highly recommended subsequent steps selected by the computing system.

[0076] Additional Publication

[0077] This paper discusses technologies related to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and received from these systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and partitions of tasks and functions among components. For example, the processes discussed in this paper can be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0078] While this subject matter has been described in detail with respect to various specific example embodiments, each example is provided by way of explanation and not as a limitation of this disclosure. Those skilled in the art will readily recognize changes, variations, and equivalents to these embodiments upon understanding the foregoing. Therefore, this disclosure does not preclude such modifications, variations, and / or additions to the subject matter, which will be apparent to those skilled in the art. For example, features shown or described as part of one embodiment may be used with another embodiment to produce further embodiments. Therefore, this disclosure is intended to cover such changes, variations, and equivalents.

Claims

1. A computer-implemented method for generating action elements for a specific task, the method comprising: The computing system obtains historical user data describing historical user actions taken in one or more past online user sessions, wherein the historical user data has been annotated with annotations describing the content associated with the user actions; The computing system identifies one or more ongoing tasks based on historical user data; For the corresponding ongoing task in the one or more ongoing tasks: The computing system accesses a continuously updated probability transition graph for the corresponding ongoing task; and The computing system identifies the intended next steps of the corresponding ongoing task based at least in part on the continuously updated probability transition graph; The computing system determines the suggested content items for at least the corresponding ongoing task based on the intent of the subsequent steps; and The computing system exposes optional action elements for displaying at least the corresponding ongoing tasks to the user, the optional action elements being configured to provide access to at least one suggested content item for at least the corresponding ongoing task.

2. The computer-implemented method according to claim 1, wherein, Historical user data describing historical user actions obtained by the computing system includes: The computing system extracts historical user actions from multiple web pages across multiple web browser sessions.

3. The computer-implemented method according to claim 1, wherein, The computing system identifies one or more ongoing tasks based on historical user data, including: The computing system annotates historical user data with attributes that describe content associated with user actions; The computational system clusters historical user data into multiple clusters; and One or more ongoing tasks are identified, at least in part, based on the multiple clusters and historical user data.

4. The computer-implemented method according to claim 1, wherein, The computing system identifies one or more ongoing tasks based on historical user data, including: Historical user data is processed by a computing system using machine learning models; and One or more ongoing tasks are generated by a machine learning model based on historical user data.

5. The computer-implemented method according to claim 1, wherein, The computing system identifies one or more ongoing tasks based at least in part on relevance scores from historical user data, wherein identifying one or more ongoing tasks based on historical user data includes: Content with higher relevance scores is given a greater weight compared to content with lower relevance scores.

6. The computer-implemented method according to claim 5, wherein, The relevance score is based at least in part on timestamps associated with historical user data.

7. The computer-implemented method according to claim 1, wherein, The computing system identifies one or more ongoing tasks based on historical user data, which also includes: A probability distribution is generated by a computing system, wherein the probability distribution includes a set of intentions based at least in part on aggregated historical user data from multiple users; and The computing system identifies the suggested next steps based at least in part on a completion metric, wherein the completion metric indicates the user’s status regarding the completion of predetermined checkpoints associated with the one or more ongoing tasks, wherein the checkpoints indicate information that the user has consumed.

8. The computer-implemented method according to claim 7, wherein, The completion metric is based at least in part on a URL associated with at least one predetermined checkpoint, which is associated with one or more ongoing tasks.

9. The computer-implemented method according to claim 1, wherein: The probability transition graph describing content intent includes a hierarchical representation of the subsequent steps of the one or more ongoing tasks, based at least in part on historical user data. and The identification of subsequent steps of the one or more ongoing tasks by the computing system includes the selection of one or more content intentions by the computing system based at least in part on a hierarchical representation of the subsequent step intentions.

10. The computer-implemented method according to claim 1, wherein, The content items that the computing system determines for each of the one or more ongoing tasks include the content items for which the computing system ranks each of the one or more ongoing tasks based on at least one quality attribute.

11. The computer-implemented method according to claim 10, wherein, Quality attributes include one or more of the following: Other user engagement levels; Comment; Freshness; or Content relevance.

12. The computer-implemented method according to claim 1, wherein, The optional action elements displayed by the computing system for showing the user at least one ongoing task include optional action elements displayed by the computing system on a dedicated dashboard.

13. The computer-implemented method according to claim 1, wherein, Optional action elements exposed by the computing system for displaying at least one ongoing task to the user include: The computing system obtains the current user's web browser data, which includes one or more of the text content or image content currently presented to the user; The computing system selects at least one ongoing task based, at least in part, on the one or more text or image contents according to historical user data; The computing system exposes optional action elements for displaying at least one ongoing task to the user, the optional action elements being overlaid on or adjacent to the current user's web browser data.

14. The computer-implemented method according to claim 13, wherein, The selection of at least one ongoing task by the computing system based at least in part on one or more of the text content or image content according to historical user data includes: The computing system performs semantic analysis on the text content of the current user's web browser data; and The content type is determined by the semantic analysis of the text content by the computing system, based at least in part on the current user's web browser data.

15. The computer-implemented method according to claim 13, wherein, The selection of the at least one or more ongoing tasks by the computing system based at least in part on one or more of the text content or the image content, according to historical user data, includes: One or more constituent features of the image content identified by the computing system in the current user's web browser data; and The content type is determined by the computing system based at least in part on one or more constituent features of the image content from the current user's web browser data.

16. A computing system, comprising: One or more processors; and A non-transitory computer-readable storage medium storing instructions executable by the one or more processors, and causing a computing system to perform operations when the instructions are executed, said operations including: The computing system obtains historical user data describing historical user actions taken in one or more past online user sessions, wherein the historical user data has been annotated with annotations describing the content associated with the user actions; The computing system identifies one or more ongoing tasks based on historical user data; For the corresponding ongoing task in the one or more ongoing tasks: The computing system accesses a continuously updated probability transition graph for the corresponding ongoing task; and The computing system identifies the intended next steps of the corresponding ongoing task based at least in part on the continuously updated probability transition graph; The computing system determines the suggested content items for at least the corresponding ongoing task based on the intent of the subsequent steps; and The computing system exposes optional action elements for displaying at least the corresponding ongoing tasks to the user, the optional action elements being configured to provide access to at least one suggested content item for at least the corresponding ongoing task.

17. The computing system according to claim 16, wherein, The computing system identifies one or more ongoing tasks based on historical user data, including: Annotations describe historical user data related to attributes associated with user actions; Arrange the annotated historical user data into multiple clusters; One or more ongoing tasks are identified, at least in part, based on the multiple clusters and historical user data.

18. The computing system according to claim 16, wherein, The computing system identifies one or more ongoing tasks based on historical user data, including: Inputting historical user data into a machine learning model; and One or more ongoing tasks are generated by a machine learning model based on historical user data.

19. A non-transitory computer-readable storage medium storing instructions executable by a computing system, and causing the computing system to perform operations when the instructions are executed, comprising: The computing system obtains historical user data describing historical user actions taken in one or more past online user sessions, wherein the historical user data has been annotated with annotations describing the content associated with the user actions; The computing system identifies one or more ongoing tasks based on historical user data; For the corresponding ongoing task in the one or more ongoing tasks: The computing system accesses a continuously updated probability transition graph for the corresponding ongoing task; and The computing system identifies the intended next steps of the corresponding ongoing task based at least in part on the continuously updated probability transition graph; The computing system determines the suggested content items for at least the corresponding ongoing task based on the intent of the subsequent steps; and The computing system exposes optional action elements for displaying at least the corresponding ongoing tasks to the user, the optional action elements being configured to provide access to at least one suggested content item for at least the corresponding ongoing task.

Images