Explainable content recommendation system

Abstract

Devices and techniques are generally described for explainable content recommendation systems. A respective first category label for each content item of a first set of content items may be determined. A weighted category graph comprising a plurality of nodes may be generated where each node is associated with a respective first category label for the first set of content items, and where a first edge connecting two nodes represents the respective category labels associated with the two nodes having been selected by a first user during a past time period. At least one cluster of the weighted category graph may be selected. A first content item for recommendation to a user may be determined from among category labels of the selected cluster based on the user's past content selection history.

Description

EXPLAINABLE CONTENT RECOMMENDATION SYSTEMBACKGROUND

[0001] Online content catalogs are often searched using keyword search queries to find content of interest to a user. Various approaches may be used to recommend content to users based on previously selected and / or previously viewed content. In some examples, users may be able to browse content by selecting an initial content item and then selecting content items that are recommended based on the initial content item, and so on. until a suitable content item is located or until the user abandons the search. In some cases, users may be unaware of the appropriate search terms to use to surface a particular product or product feature. Additionally, in some examples, recommendations systems recommend content only based on the currently- displayed content. In some examples, deep learning algorithms, such as neural networks, may be used to recommend content of interest to a user.BRIEF DESCRIPTION OF DRAWINGS

[0002] FIG. 1 is a diagram of an example explainable content recommendation system configured to generate per-user content recommendations, according to various embodiments of the present disclosure.

[0003] FIG. 2 depicts block diagrams that may be used to filter historical content selections and transform content items into category labels, in accordance with various aspects of the present disclosure.

[0004] FIG. 3A depicts an example of a weighted category graph, in accordance with various aspects of the present disclosure.

[0005] FIG. 3B depicts a thresholding operation that may be applied to a weighted category graph, in accordance with various aspects of the present disclosure.

[0006] FIG. 3C depicts another example of a weighted category graph and an example of a clustering operation that may be performed in accordance with various aspects of the present disclosure.

[0007] FIG. 3D depicts an example of cluster selection based on historical content item selection history for a user, in accordance with various aspects of the present disclosure.

[0008] FIG. 3E depicts an example of category label selection from a selected cluster based on the historical content item selection history for the user, in accordance with various aspects of the present disclosure.

[0009] FIG. 4 is a flow chart depicting an example process for explainable content recommendation, in accordance with various aspects of the present disclosure.

[0010] FIG. 5 is a block diagrams showing an example architecture of a computing device that may be used in accordance with various embodiments described herein.

[0011] FIG. 6 is a diagram illustrating an example system for sending and providing data that may be used in accordance with the present disclosure.DETAILED DESCRIPTION

[0012] In the following description, reference is made to the accompanying drawings that illustrate several example embodiments of the present disclosure. It is understood that other examples may be utilized and various operational changes may be made without departing from the scope of the disclosure. The follow ing detailed description is not to be taken in a limiting sense, and the scope of the embodiments of the present invention is defined only by the claims of the issued patent.

[0013] The problem of optimally recommending content items (movies, music, items, offers / promotions, advertisements, etc.) to users in a personalized manner is a common problem in various industries using online interfaces. Many existing computer-implemented recommendation systems recommend content items by identifying individual content items that are similar in some respect to content items that the user has previously selected. Selection in this context can depend on the type of content item. For example, selection may indicate that the user has navigated to a web page for the content item, has played back the content item, has clicked on the content item, has added the content item to a virtual shopping cart, etc. Many current computer-implemented recommendation systems use machine-learning based approaches such as deep learning-based approaches (e g., neural network -based systems) to recommend content items to users based on some learned representation of the type of content item in which the user may be interested.

[0014] However, a key technical problem facing such machine learning based recommendation systems is that such systems lack explainability to the end user. This is because the high dimensional representations learned by such models during training may lack human interpretability. As such, machine learning recommendation systems (such as those employing neural networks) are typically considered to be “black boxes’" where only the inputs and outputs are human interpretable. This can cause issues and questions, particularly when the model recommends something that a user does not like or when a user is wondering why such a recommendation has been made. Even in the case of unsupervised machine learning systems the relationship between different content items may depend on a high-dimensional encoding, which may lack interp retability / explainabilify.

[0015] Currently, computer-implemented recommendation systems are unable to both 1) generate high qualify content item recommendations to a user, and 2) offer explainabilify, at any desired scale. The computer-implemented techniques described herein are able to determine related groups of content items that are related to user goals, even when there is no explicitlabeling or category for such items and / or when the relationship between such items may not be apparent to a human (at least prior to the explainability provided by the systems and techniques discussed herein). For example, the explainable recommendation systems described herein may use historical data to determine a relationship between t-shirts, nutrition bars, and a jump rope. Although these three items may have different explicit content item categories (e.g., apparel, food, exercise equipment), the explainable recommendation systems and techniques described herein may use available signals to determine that such content items are related and pertain to, for example, martial arts equipment and may explain why a content item of such a category- has been recommended to a user. As previously described, currently-available content item recommendation systems may make recommendations at the individual content item level and / or for explicit categories, and are unable to provide human explainability and determine such nonexplicit and seemingly unrelated category labels. Accordingly, the systems and techniques described herein provide technical improvements over existing machine learning based recommendation systems by discovering potentially hidden content associations (e.g., categories) while providing explainability of the output recommendations, at any scale.

[0016] Storage and / or use of data related to a particular user, account, and / or device (e.g., device identifier data, device names, names of device groups, contextual data, and / or any personal data) may be controlled by a user using privacy controls associated with the account, service(s), and / or device being used. Users may opt out of storage of historical information, such as historical content selections and / or contextual data and / or may select particular types historical content selections that may be stored while preventing aggregation and storage of other types of historical content selections. Additionally, aggregation, storage, and use of historical content selections, as described herein, may be compliant with privacy controls, even if not legally subject to them. For example, historical content selections, and other data described herein may be treated as if it was subject to acts and regulations, such as the Health Insurance Portability and Accountability Act (HIPAA) and the General Data Protection Regulation (GDPR), even if it is not actually subject to these acts and regulations. In various examples, the account data, user data, device and / or device identifiers and / or any historical content selections captured by such devices may be used only in accordance with user permission, in compliance with any relevant laws and / or policies. Additionally, users may opt out of data collection, and / or may opt to delete some or all of the data used by the various techniques described herein, even where deletion or non-collection of various data may result in reduced functionality and / or performance of various aspects of the systems described herein.

[0017] FIG. 1 is a diagram of an example explainable content recommendation system 100 configured to generate per-user content recommendations, according to various embodiments ofthe present disclosure. As depicted in FIG. 1, computing device(s) 120 may include a non- transitory computer-readable memory 103 and / or may be configured in communication with non-transitory computer-readable memory 103, such as over network 141. In various examples, network 141 may represent a local area network (LAN) and / or a wide area network (WAN) such as the internet. Non-transitory computer-readable memory 103 may store computer-executable instructions that may be effective to program at least one processor of a computing device to perform one or more of the various techniques described herein. Computing device(s) 120 may be used to execute one or more components of the explainable content recommendation system 100. For example, the instructions stored by the non-transitory computer-readable memory 103 may perform various operations described herein in order to generate the predictions, generated the weighted category graph 122, perform cluster selection 130, etc.

[0018] In order to generate personalized, goal-based content item recommendations, as described herein, associated category-based (not content item based) content selections may be determined. The goal may be to determine content selections that often go together (e.g., selection of a how-to video for a video game and selection of the video game itself, selection of toothpaste and a toothbrush, etc.). Accordingly, for a group of users 106 over a predetermined time period (which may vary according to the desired implementation), historical content selections 124 may be determined. Historical content selections 124 may be a list of past content selections for the users 106 over the selected past time period. For example, past content selections may include one or more of views, searches, clicks (and / or other selections), purchases, add-to-cart selections, playback, etc. In various examples, in order to determine habitual behavior (as opposed to one-off behavior), only those content items that have been selected at least twice by the same user may be included in the historical content selections 124.

[0019] Category labels 118 may be determined for the historical content selections 124. The category' labels 118 may be generated in a variety of ways. For example, a function and / or mapping technique may be used to transform a particular content item into a category’ for the content item (where the category label 118 represents data identifying the particular category). Categories may be general ledger categories for a particular content item, short titles for the content item, browse nodes for the content item, concept nodes for the content item (e.g., a more granular categorization relative to a browse node that may be used to find the content item in an ontology), content type metadata, etc. In still other examples, a large language model (LLM) may be given information about a content item (e.g., a title of the content item, a description of the content item, structured data (e.g., metadata) representing the content item, and / or even images of the content item (in the case of a multi-modal LLM)) and the LLM may be thefunction that generates the category label for the item. In general, any desired techniques may be used to transform the particular content item into a category label 118.

[0020] Category7labels 118 may be used to generate a weighted category graph 122. Each node in the weighted category graph 122 may represent an individual category label 118. Edges connecting nodes may represent the condition that the two connected nodes appeared together in historical content selections of at least one single user of the users 106. For example, a first node may represent the category' label 118 “shampoo” and a second node may represent a category' label 118 “soap.” If at least one user selected both a content item having the category label “shampoo” and another content item having the category label “soap” (as determined from the historical content selections 124), then the “shampoo” node and the “soap” node are connected by an edge in the weighted category graph 122. As described in further detail below in reference to FIG. 3 A, the weighted category graph 122 may be generated such that nodes and edges may have respective weight values. The weight for a node may represent the total number of users that selected any content item having the category label 118 for that node or the total number of selections by all users of any content item having the category label 118. The weight for an edge represents the total number of times that content items of the two connected nodes appeared together in the historical content selections 124 for the same user 106 (aggregated over all users). For example, if user 1 and user 2 were the only users among users 106 that each selected a single content item having category label A and a single content item having category label B during the past time period (e.g., the past time period associated with the historical content selections 124), then the edge connecting the nodes for category label A and category label B may have an edge weight of 2. In various examples, the edge weights and / or node weights may be normalized so that the weight values are of a similar scale. Normalization of the weights in the weighted category graph may be important for thresholding the graph to determine clusters, as described in further detail below. Normalization may correct for large variance in popularity among different nodes of the weighted category graph 122. Example techniques for weight normalization are described in further detail below in reference to FIG. 3A.

[0021] After generating the weighted category graph 122, the explainable content recommendation system 100 may execute thresholding and clustering 126 to generate one or more unique clusters of nodes from the weighted category graph 122. In various examples, the thresholding may be performed on the basis of the node weights and / or edge weights from the weighted category graph 122. Thresholding techniques are described in further detail below in reference to FIG. 3B. Similarly, various techniques may be used to determine the individual clusters (e.g., after thresholding). Various clustering techniques are discussed in further detail below in reference to FIG. 3C.

[0022] The resulting clusters (e.g., cluster 1, cluster 2, , etc.) may represent category labels 118 that are strongly associated with one another as evidenced by the historical content selections 124 from the plurality of users 106 over the relevant past time period. Note that the clusters do not represent associations at the content item level, but instead at the category label level, and thus may be conceptually thought of as associated user desires (representing past selections that are associated with the different category labels) rather than more granular associations between particular items.

[0023] After generating the clusters for the particular past time period, individual, personalized recommendations may be provided by the explainable content recommendation system 100. For example, user 108 may be a subject for whom one or more personalized recommendations are to be generated. The per-user historical content selections 128 may be determined (e.g., over the same past time period as historical content selections 124 or over a different past time period, as desired according to the particular implementation). In the example depicted in FIG. 1, the per-user historical content selections 128 indicate that the user 108 has selected content item 1, content item 3, content item 7, etc., over the past time period.

[0024] Category7labels 129 may be determined for each content item in the per-user historical content selections 128. The function and / or mapping technique used to generate the category labels 129 from the content items in the per-user historical content selections 128 may be the same as the function used to transform the historical content selections 124 into the category labels 118.

[0025] After transforming the per-user historical content selections 128 into the category labels 129, one or more clusters may be selected for personalized recommendation(s) (cluster selection 130). Clusters may be selected based on the per-user historical content selections 128. Cluster selection is described in further detail below in reference to FIG. 3D.

[0026] After cluster selection, category' labels from the selected cluster(s) that have not previously been selected by the user 108 may be determined using the category labels 129 (at block 132). The notion may be to determine the category labels 118 which have not been selected by the user 108 during the past time period, but which are associated with category labels 129 which were selected by the user 108 during the past time period. After determining the non-selected category labels (from the selected cluster(s)), processing may continue to block 134, at which one or more specific content items for these category labels may be determined for recommendation to the user 108. There are a variety7of ways to select the content items at block 134. Examples are discussed below in reference to FIG. 3E. The content item(s) determined at block 134 may be output to the user 108 as recommendation(s) (e.g., a graphical representationof the content items and / or text representing the content items may be output on a graphical user interface).

[0027] The recommendation(s) may be output using a variety of user interfaces and / or user experiences. For example, an LLM may be used to provide a descriptor for the selected cluster from which the content item is determined. Then, the user experience may inform the user that based on their interest in [natural language cluster descriptor], they may also be interested in [determined content item]. For example, the LLM may generate an explainable title for a widget that represents the selected cluster (and / or previously-unselected content items from the selected cluster (e.g., non-overlapping category labels between the cluster and the historical content item selections)). Some examples may include: “Because of your interest in aquarium care”, “Other action movies having a female lead”, “Bathroom essentials”, etc. Visual user experiences may depict multiple categories labels (e.g., images representative of content items from previously- unselected category labels) for a matched cluster, multiple images of content items for a topscored category label in a matched cluster, etc.

[0028] FIG. 2 depicts block diagrams that may be used to filter historical content selections and transform content items into category7labels, in accordance with various aspects of the present disclosure. As previously described, historical content selections 124 (e.g., content selections made over a past time period by the plurality of users 106) may be filtered such that only the content item selections that were selected at least twice by the same user 202 are considered for generation of the weighted category graph 122. Thereafter, the set of content items that were selected at least twice by the same user 202 may be transformed into respective category labels 118 that describe each of the content items of the set of content items that were selected at least twice by the same user 202.

[0029] As previously described, the category labels 118 may be generated in a variety of ways. For example, a function may transform a particular content item into a category7for the content item (where the category label 118 represents data identifying the particular category). Categories may be general ledger categories for a particular content item, short titles for the content item, browse nodes for the content item, concept nodes for the content item (e.g., a more granular categorization relative to a browse node that may be used to find the content item in an ontology), content type metadata, etc. In still other examples, a large language model (LLM) may be given information about a content item (e.g., a title of the content item, a description of the content item, structured data (e.g., metadata) representing the content item, and / or even images of the content item (in the case of a multi-modal LLM)) and the LLM may be the function that generates the category7label for the item. In general, any desired techniques may beused to transform the set of content items that were selected at least twice by the same user 202 into representative category labels 118.

[0030] FIG. 3A depicts an example of a weighted category graph 122, in accordance with various aspects of the present disclosure. Category labels 118 may be used to generate a weighted category graph 122. Each node in the weighted category graph 122 may represent an individual category label 118. Edges connecting nodes may represent that the two connected nodes appeared together in historical content selections of at least one single user of the users 106.

[0031] In the simple example depicted in FIG. 3A, there are five distinct category labels (i.e., [Label 1 , Label 2, Label 3, Label 4, Label 5], Additionally, there is a list of four Users 1 -4 and a representation of category labels of content selections made by each user during the relevant past time period. For example, User 1 selected content items associated with Label 1, Label 2, Label 3. and Label 4, but did not select any content items associated with Label 5 during the past time period. User 2 selected content items associated with Label 3 and Label 4, but did not select any content items associated with Label 1, Label 2, or Label 5. User 3 and User 4 each only selected content items associated with Label 5.

[0032] The node for Label 1 in the weighted category graph 122 has a node weight of 1 since only User 1 selected a content item associated with the category label: Label 1. Label 4 has a node weight of 2 since both User 2 and User 1 selected a content item with the category label: Label 4, and so on.

[0033] If two category labels w ere selected at least once by the same user during the past time period, these nodes are connected by an edge in the weighted category graph 122. For example, the node for Label 1 and the node for Label 4 are connected by an edge since User 1 selected a content item with category' label: Label 1 and a content item with the category label: Label 4 during the past time period. The edge weight for this edge is 1 since User 1 is the only- user that selected content items from both category labels 1 and 4 during the past time period. Similarly, the node for Label 3 and the node for Label 4 are connected by an edge since User 1 and User 2 both selected content items from category label: Label 1 and category label: Label 4 during the past time period. The edge weight for this edge is 2 both Users 1 and 2 each selected content items Labeled with 1 and 4 during the past time period. The node for Label 5 is not connected to any other node since content items labeled with Label 5 were not selected together with content items labeled with any other label by any user during the past time period. As previously described, although not depicted in FIG. 3A (due to the relatively simplistic example), the edge weights and / or node w eights may be normalized so that the weight values are of asimilar scale. Normalization of weights may be used to account for large degrees of variance in popularity of different nodes so that the weights are normalized to the same scale.

[0034] For example, edge weights may be normalized to the interval [0,1] in the following way:

[0035] Edge weight between nodes A and B = # of users that selected content from node A and content from node B / min(# of users that selected content from node A, # of users that selected content from node B).

[0036] The minimum between the two nodes A, B may be selected to correct for varying popularity. For example, if node A is much more popular than node B, but all of the users that selected content from node B also selected content from node A, a strong connection would be indicated.

[0037] As an example, there may be 192,282 users selecting content from node A (e.g., Node A = action movies) and 14,132 users electing content from node B (e.g., 80s musicals). The number of users selecting from both node A and node B may be 13,326. In this case, the edge weight between node A and node B = 13,326 / min(192,282, 14,132) = 13,326 / 14,132 = 0.94 (indicating a strong connection between nodes A and B).

[0038] FIG. 3B depicts a thresholding operation that may be applied to the weighted category graph 122 of FIG. 3 A, in accordance with various aspects of the present disclosure. In various examples, edge weight thresholds and / or node weight thresholds may be used to filter the weighted category graph 122 such that the filtered weighted category graph may represent only strong connections between category labels (based on habitual user behavior). The resulting weighted category graph 122 may represent categories of content items that were often selected together by users in the past. In the example thresholding operation of FIG. 3B, a minimal edge weight threshold may be 2 and a minimal node weight threshold may be 1. Accordingly, as depicted by black Xs in FIG. 3B. edges with weights less than the minimal edge weight threshold may be eliminated. Similarly, nodes with node weights less than the minimal node weight threshold may be eliminated. However, since the simplistic example in FIG. 3B uses a minimal node weight threshold of 1 , no nodes are eliminated based on the minimal node weight threshold. After thresholding, any disconnected nodes may be removed. Accordingly, the nodes for Label 5. Label 1, and Label 2 are removed to generate the resulting weighted category- graph 322.

[0039] In various examples, the thresholds (e.g.. the minimal edge weight threshold and / or the minimal node weight threshold) may be empirically determined and / or may be determined using a machine learning model based on the data. In general, higher threshold values may lead to clusters having fewer category labels and ultimately fewer content items to recommend to users, but with stronger connections to one another (in terms of historical selections of items withsuch category labels). Conversely, lower thresholds may lead to larger clusters and ultimately more content items to recommend to users, but the nodes may be less strongly connected. The threshold values may be plotted on a curve where higher threshold values are generally- associated with higher precision (e.g., stronger node connections) and lower coverage (e.g., fewer nodes per cluster) and lower threshold values are associated with higher coverage (e.g., more clustered nodes) and lower precision (e.g., weaker node connections).

[0040] FIG. 3C depicts another example of a weighted category- graph 324 and an example of a clustering operation that may be performed in accordance with various aspects of the present disclosure. For example, as described in FIG. 1, after generating the weighted category graph (e.g., weighted category graph 122 of FIG. 1 or weighted category graph 324 of FIG. 3C) and after thresholding (as described above in reference to FIG. 3B), the explainable content recommendation system 100 may execute clustering to generate one or more unique clusters of nodes from the weighted category graph. For example, as described in FIG. 3C, clustering 304 may generate the result clusters from the weighted category graph 324.

[0041] For example, one clustering algorithm may be the clique-finding algorithm, which may find all nodes that are connected to each other and label such nodes as a single cluster. This example has been used in FIG. 3C to generate two unique clusters. The first cluster includes: [Label 5, Label 6, Label 7] since each of these nodes are connected to each other node in the cluster. The second cluster includes [Label 1, Label 2, Label 3, Label 4] since each of these nodes are connected to each other node in the cluster. Note that the node for Label 4 is not included in the first cluster since the node for Label 4 is not connected via an edge to the node for Label 6 or the node for Label 7. Similarly, the node for Label 5 is not included in the second cluster since the node for Label 5 is not connected via an edge to the nodes for Labels 1, 2, or 3.

[0042] Other sub-graph-based clustering approaches may be used instead of clique-finding. For example, Connected Components may generate subgraphs such that every- two nodes are connected via a path (e.g., an edge). Another example approach is to cluster similar nodes together in a Community Structure approach based on common and / or similar information for nodes. Other example algorithms that may7be used to find the clusters from the w eighted category- graph include Breadth-first Search, Depth-first Search, the Bron-Kerbosch algorithm, the Louvain algorithm, the Min-cut algorithm, etc.

[0043] FIG. 3D depicts an example of cluster selection based on historical content item selection history for a user, in accordance with various aspects of the present disclosure. After generating the clusters for the particular past time period, individual, personalized recommendations may be provided by the explainable content recommendation system 100. For example, user 108 may be a subject for whom one or more personalized recommendations are tobe generated. The set of content selections for user 108 over the past time period 306 may be determined. In the example depicted in FIG. 3D, the set of content selections for user 108 over the past time period 306 indicate that the user 108 has selected content item 1, content item 2, and content item 3, etc., over the past time period.

[0044] Category labels may be determined for each content item in the per-user histoneal content selections. The function used to generate the category labels from the user-specific content items may be the same as the function used to transform the historical content selections 124 into the category labels 118. In the example of FIG. 3D, content item 1 has been determined to pertain to category label 3, content item 2 has been determined to pertain to category label 4, and content item 3 has been determined to pertain to category label 6. Accordingly, user 108 has selected content items associated with Label 3, Label 4, and Label 6 (the “user categories”) over the past time period.

[0045] The user categories may be matched with the result clusters determined as described above in the various examples of FIGS. 3A-3C. In one example, the cluster that has maximum overlap with the user categories may be determined. For example, cluster 1 has one matching category with respect to the user categories (i. e. , Label 6, bolded in FIG. 3D). Cluster 2 has two matching categories with respect to the user categories (i.e., Label 3 and Label 4, bolded in FIG. 3D). In this example, cluster 2 may be selected since there is a 50% overlap with respect to the user categories selected over the past time period (relative to a 33% overlap for cluster 1). However, cluster selection 130 may be selected in other manners. For example, the total number of times the user 108 selected each content item may be used to select the cluster. For example, if user 108 selected content item 3 (Category Label 6) ten times, but only selected content item 1 twice and content item 2 once, cluster 1 may be selected over cluster two. Additionally, the overlap percentage may be multiplied by the number of individual content item selections (e.g., in a weighted or unweighted combination). In general, each cluster may be scored with respect to the user categories over the past time period. The respective cluster scores (e.g., overlap, per- category selections, combinatorial scores, etc.) may be used to score each cluster. In various examples, node weight and / or edge weight may also be used to score the clusters. In various further examples, the clusters may be seasonally weighted and / or the clustering may be determined on a seasonal basis in order to take seasonal effects into account. In some other examples, clusters may be scored and / or ranked with respect to the user categories using a machine learning model (e.g., a ranker model). For example, the machine learning model may take node weights (per-cluster) and overlaps between the cluster node categories and the user categories as input and may generate a per-cluster score that may then be used to rank the different clusters for the user.

[0046] In some further examples, runtime context data may be used to select a particular cluster. For example, if a user is currently playing a particular video (or has recently done so), has selected a particular content item, etc., such runtime information may be used during cluster selection and / or content item selection. For example, such runtime context data (e.g., indiciating currently selected content items and / or content items selected during a user session) may be stored in association with a current user session in memory. For example, the video currently or recently played may be associated with a particular category' label. Clusters that have a matching category label may be ranked / scored more highly relative to other clusters. In another example, runtime context (current and / or recently-selected items) may be given higher weights (e.g.. when determining cluster overlap with user category’ labels) relative to historical user selections. In some examples, user category’ labels may be weighted based on a recency of such selections and such weights may be accounted for during cluster selection and / or content item selection.

[0047] In still other examples, the runtime context (current and / or recently-selected items) may be used as the per-user historical content selections 128. Accordingly, in at least some examples, only’ current and / or recently-selected items (and not longer-ago historical content selections past some recency cut-off) may be used during cluster selection 130 and / or content item selection. In this way, recommendations may be generated based solely on recent user content selections.

[0048] FIG. 3E depicts an example of category label selection from a selected cluster based on the historical content item selection history’ for the user, in accordance with various aspects of the present disclosure. After one or more clusters are selected for the user 108, categories labels that were not selected by the user 108 during the past time period may be determined. In the cunent example, the user 108 has selected content items from categories Label 3 and Label 4. Accordingly, for the selected cluster (cluster 2 in the current example), the user 108 has not selected content items from the categories Label 1 and Label 2. These category labels may be selected from within the selected cluster. In some examples, these selected category’ labels maybe ranked (e.g.. according to node weights of the nodes). For example, Label 1 may have a node weight of 37 while Label 2 may have a node weight of 14. Accordingly, Label 1 may be ranked higher than node 2.

[0049] Content items for the labels may be determined and recommended to the user. For example, if Label 1 is the highest rated previously -unselected label, the highest ranked content items from category Label 1 may be determined and recommended to the user. In another example, the category’ Label 1 may be used as an input query’ into a search engine and the top results may be recommended or otherwise output to the user. For example, if Label 1 is Electronic dance music, the query “electronic dance music” may be input into a search engineand a ranked list of search results may be recommended to the user. In various examples, the search engine may be personalized for the user account. Accordingly, inputting the category label as part of the search query may lead to personalized search results for the particular user. For example, category Label 1 may be “toothbrushes / ’ When the term “toothbrush” is input into a personalized search engine, the results may be skewed toward the individual preferences of the particular user account (e.g., with preferred brands of toothbrushes being higher ranked relative to other brands). The content items may be selected in other manners as well, depending on the desired implementation. For example, trending content items, deals, most re-ordered and / or replayed content items, content items of a user’s preferred brand, studio, artist, producer, director, actor, etc. may be determined, etc.

[0050] The recommendation outputs of the explainable content recommendation system 100 may be explainable to the user 108 such that the user can understand why the recommendation was made (unlike with many machine learning-based recommendation systems). The clusters and connected category nodes may be used (along with past categories of content selections by the user) to explain why a particular content item and / or category of content item was recommended to the user. Optionally, the actual content that has been selected by the user in the past (together with the category of the content) can be shown as part of the explanation. For example, a user may ask via a chat form, email, customer service system, etc., why a particular content item was recommended. In various examples, an LLM may be used to explain the recommendation.

[0051] For example, an LLM may be prompted using a prompt such as: “For a given group of content items, describe the content items in short, broader terms that describes the group. For example, for [toothbrush, floss, toothpaste] the answer could be “dental hygiene.” Thereafter, the individual category' labels for nodes of a cluster could be input to provide a representation of the cluster. Examples may include:

[0052] Category label input: Vitamins, Herbal Supplements, Glucosamine - LLM output: “Health Support.”

[0053] Category label input: Eyelash Adhesives, Mascara brushes, false lashes, lip brushes, gels, kits - LLM output: “Cosmetic Enhancements.”

[0054] Category label input: Dryer sheets, liquid fabric softener, laundry detergent - LLM output: “Laundry Essentials.”

[0055] These LLM-generated explanations can be used to show the user the clusters matched to their histories. In some examples, the connection between individual categories in a cluster may not be apparent at first glance. For example, it may initially be unclear why content items such as t-shirts, protein bars, and jump ropes would be related content items that have often beenselected together, but these items may all be associated with athletic training. The LLM can recognize such connections and output the explanation to the user.

[0056] In another example, instead of prompting the LLM to provide a general description for a cluster, the LLM may be prompted by telling the LLM the category label that was selected for recommendation (and / or the particular content that was selected for recommendation) and instructing the LLM to provide the connection between the selected category label (and / or content item) and one or more other non-selected category' labels of the cluster. In some cases this may provide more specific and / or clearer explanations.

[0057] FIG. 4 is a flow chart depicting an example process 400 for explainable content recommendation, in accordance with various aspects of the present disclosure. Those portions of FIG. 4 that have been previously discussed in reference to FIGS. 1-3E may not be described again for purposes of clarity and brevity. The actions of the process 400 may represent a series of instructions comprising computer-readable machine code executable by one or more processing units of one or more computing devices. In various examples, the computer-readable machine codes may be comprised of instructions selected from a native instruction set of and / or an operating system (or systems) of the one or more computing devices. Although the figures and discussion illustrate certain operational steps of the system in a particular order, the steps described may be performed in a different order (as well as certain steps removed or added) without departing from the intent of the disclosure.

[0058] Process 400 may begin at action 410, at which a first set of content items may be determined. In various examples, the first set of content items may be content items that were selected by a plurality of users over a past time period. In various examples, the first set of content items may be limited to those items that were selected at least twice by the same user so that content items that are repeatedly selected may be considered (for prediction and / or recommendation).

[0059] Processing may continue at action 415. at which a respective first category label may be determined for each content item of the first set of content items. The category labels (e.g., category labels 118) may be generated in a variety of ways. For example, a function may transform a particular content item into a category' for the content item (where the category' label represents data identifying the particular category). Categories may be general ledger categories for a particular content item, short titles for the content item, browse nodes for the content item, concept nodes for the content item (e.g., a more granular categorization relative to a browse node that may be used to find the content item in an ontology ), content ty pe metadata, etc. In still other examples, a large language model (LLM) may be given information about a content item (e.g., a title of the content item, a description of the content item, structured data (e.g., metadata)representing the content item, and / or even images of the content item (in the case of a multimodal LLM)) and the LLM may be the function that generates the category label for the item. In general, any desired techniques may be used to transform the particular content item into a category label.

[0060] Processing may continue at action 420. at which a weighted category graph including a plurality of nodes may be generated. Each node may be associated with a unique category label. The weight for a node may represent the total number of users that selected any content item having the category label for that node or the total number of selections by all users of any content item having the category label. The weight for an edge represents the total number of times that content items of the two connected nodes appeared together in the historical content selections for the same user.

[0061] Processing may continue at action 425, at which clusters from the weighted category graph may be determined. For example, as described in FIG. 1, after generating the weighted category graph (e.g., weighted category graph 122 of FIG. 1 or weighted category graph 324 of FIG. 3C) and after thresholding (as described above in reference to FIG. 3B), the explainable content recommendation system 100 may execute clustering to generate one or more unique clusters of nodes from the weighted category graph. For example, as described in FIG. 3C clustering 304 may generate the result clusters from the weighted category graph 324.

[0062] For example, one clustering algorithm may be the clique-finding algorithm, which may find all nodes that are connected to each other and label such nodes as a single cluster. Other sub-graph-based clustering approaches may be used instead of clique-finding. For example, Connected Components may generate subgraphs such that every two nodes are connected via a path (e.g., an edge). Another example approach is to cluster similar nodes together in a Community Structure approach based on common and / or similar information for nodes. Other example algorithms that may be used to find the clusters from the weighted category graph include Breadth-first Search, Depth-first Search, the Bron-Kerbosch algorithm, the Louvain algorithm, the Min-cut algorithm, etc.

[0063] Processing may continue at action 430, at which a plurality of category labels associated with previous content selections during a past time period for a first user may be determined. For example, past content selections may be determined for the first user (over the past time period) and category labels may be determined for each of the past content selections.

[0064] Processing may continue at action 435, at which a first cluster may be selected based at least in part on the plurality of category labels for the first user. The first cluster may be matched to the category labels for the past content selections for the first user. In various examples, each cluster may be scored with respect to the plurality of category labels for the firstuser to determine a per-cluster score. As described above, overlap percentage, node weight, edge weight, total number of content item selections, etc., may be used to score each cluster.

[0065] Processing may continue at action 440, at which a first content item may be determined for recommendation to the first user based at least in part on an unselected category’ label of the first cluster. For example, after selecting a cluster for the first user (e.g., based on cluster scoring), an unselected category label may be determined for that cluster. The unselected category label may be a node from the selected cluster which is not associated w ith selections by the first user during the past time period. After determining the unselected category label, a ranked list of content items for that category label may be determined and one or more of the top-ranked content items may be selected for recommendation. Different techniques may be used to select the particular content item(s) to be recommended for a given category label. Various examples are described above in reference to FIG. 3E.

[0066] FIG. 5 is a block diagram showing an example architecture 500 of a computing device that may be used to implement, at least in part, the explainable content recommendation system 100, in accordance with various aspects of the present disclosure. It will be appreciated that not all devices will include all of the components of the architecture 500 and some user devices may include additional components not shown in the architecture 500. The architecture 500 may include one or more processing elements 504 for executing instructions and retrieving data stored in a storage element 502. The processing element 504 may comprise at least one processor. Any suitable processor or processors may be used. For example, the processing element 504 may comprise one or more digital signal processors (DSPs). The storage element 502 can include one or more different types of memory, data storage, or computer-readable storage media devoted to different purposes within the architecture 500. For example, the storage element 502 may comprise flash memory, random-access memory, disk-based storage, etc. Different portions of the storage element 502, for example, may be used for program instructions for execution by the processing element 504, storage of images or other digital works, and / or a removable storage for transferring data to other devices, etc.

[0067] The storage element 502 may also store software for execution by the processing element 504. An operating system 522 may provide the user with an interface for operating the computing device and may facilitate communications and commands between applications executing on the architecture 500 and various hardware thereof. A transfer application 524 may be configured to receive images, audio, and / or video from another device (e.g., a mobile device, image capture device, and / or display device) or from an image sensor 532 and / or microphone 570 included in the architecture 500. The storage element 502 may also store the various components of the explainable content recommendation system 100.

[0068] When implemented in some user devices, the architecture 500 may also comprise a display component 506. The display component 506 may comprise one or more light-emitting diodes (LEDs) or other suitable display lamps. Also, in some examples, the display component 506 may comprise, for example, one or more devices such as cathode ray tubes (CRTs), liquidcrystal display (LCD) screens, gas plasma-based flat panel displays, LCD projectors, raster projectors, infrared projectors or other types of display devices, etc. As described herein, display component 506 may be effective to display input images in accordance with the various techniques described herein.

[0069] The architecture 500 may also include one or more input devices 508 operable to receive inputs from a user. The input devices 508 can include, for example, a push button, touch pad, touch screen, wheeljoystick, keyboard, mouse, trackball, keypad, light gun, game controller, or any other such device or element whereby a user can provide inputs to the architecture 500. These input devices 508 may be incorporated into the architecture 500 or operably coupled to the architecture 500 via wired or wireless interface. In some examples, architecture 500 may include a microphone 570 or an array of microphones for capturing sounds, such as voice requests. In various examples, audio captured by microphone 570 may be streamed to external computing devices via communication interface 512.

[0070] When the display component 506 includes a touch-sensitive display, the input devices 508 can include a touch sensor that operates in conjunction with the display component 506 to permit users to interact with the image displayed by the display component 506 using touch inputs (e.g., with a finger or stylus). The architecture 500 may also include a power supply 514, such as a wired alternating current (AC) converter, a rechargeable battery operable to be recharged through conventional plug-in approaches, or through other approaches such as capacitive or inductive charging.

[0071] The communication interface 512 may comprise one or more wired or wireless components operable to communicate with one or more other computing devices. For example, the communication interface 512 may comprise a wireless communication module 536 configured to communicate on a network, such as the network 141, according to any suitable wireless protocol, such as IEEE 802.11 or another suitable wireless local area network (WLAN) protocol. A short range interface 534 may be configured to communicate using one or more short range wireless protocols such as. for example, near field communications (NFC). Bluetooth, Bluetooth LE, etc. A mobile interface 540 may be configured to communicate utilizing a cellular or other mobile protocol. A Global Positioning System (GPS) interface 538 may be in communication with one or more earth-orbiting satellites or other suitable positiondetermining systems to identity a position of the architecture 500. A wired communicationmodule 542 may be configured to communicate according to the USB protocol or any other suitable protocol.

[0072] The architecture 500 may also include one or more sensors 530 such as, for example, one or more position sensors, image sensors, and / or motion sensors. An image sensor 532 is shown in FIG. 5. Some examples of the architecture 500 may include multiple image sensors 532. For example, a panoramic camera system may comprise multiple image sensors 532 resulting in multiple images and / or video frames that may be stitched and may be blended to form a seamless panoramic output. An example of an image sensor 532 may be a camera configured to capture color information, image geometry information, and / or ambient light information.

[0073] As noted above, multiple devices may be employed in a single system. In such a multi-device system, each of the devices may include different components for performing different aspects of the system’s processing. The multiple devices may include overlapping components. The components of the computing device(s) 120, as described herein, are exemplary, and may be located as a stand-alone device or may be included, in whole or in part, as a component of a larger device or system.

[0074] An example system for sending and providing data will now be described in detail. In particular, FIG. 6 illustrates an example computing environment in which the embodiments described herein may be implemented. For example, the computing environment of FIG. 6 may be used to implement the explainable content recommendation system 100 as a sendee over a network wherein one or more of the techniques described herein may be requested by a first computing device and may be performed by a different computing device configured in communication with the first computing device over a network. FIG. 6 is a diagram schematically illustrating an example of a data center 65 that can provide computing resources to users 60a and 60b (which may be referred herein singularly as user 60 or in the plural as users 60) via user computers 62a and 62b (which may be referred herein singularly as user computer 62 or in the plural as user computers 62) via network 141. Data center 65 may be configured to provide computing resources for executing applications on a permanent or an as-needed basis. The computing resources provided by data center 65 may include various types of resources, such as gateway resources, load balancing resources, routing resources, networking resources, computing resources, volatile and non-volatile memory resources, content delivery resources, data processing resources, data storage resources, data communication resources and the like. Each type of computing resource may be available in a number of specific configurations. For example, data processing resources may be available as virtual machine instances that may be configured to provide various web services. In addition, combinations of resources may be madeavailable via a network and may be configured as one or more web services. The instances may be configured to execute applications, including web services, such as application services, media services, database services, processing services, gateway sendees, storage services, routing sen ices, security services, encryption services, load balancing services, application sen ices and the like. In various examples, the instances may be configured to execute one or more of the various machine learning techniques described herein.

[0075] These services may be configurable with set or custom applications and may be configurable in size, execution, cost, latency, type, duration, accessibility and in any other dimension. These web services may be configured as available infrastructure for one or more clients and can include one or more applications configured as a system or as software for one or more clients. These web services may be made available via one or more communications protocols. These communications protocols may include, for example, hypertext transfer protocol (HTTP) or non-HTTP protocols. These communications protocols may also include, for example, more reliable transport layer protocols, such as transmission control protocol (TCP), and less reliable transport layer protocols, such as user datagram protocol (UDP). Data storage resources may include file storage devices, block storage devices and the like.

[0076] Each type or configuration of computing resource may be available in different sizes, such as large resources — consisting of many processors, large amounts of memory and / or large storage capacity — and small resources — consisting of fewer processors, smaller amounts of memory and / or smaller storage capacity. Customers may choose to allocate a number of small processing resources as web servers and / or one large processing resource as a database server, for example.

[0077] Data center 65 may include servers 66a and 66b (which may be referred herein singularly as server 66 or in the plural as servers 66) that provide computing resources. These resources may be available as bare metal resources or as virtual machine instances 68a-d (which may be referred herein singularly as virtual machine instance 68 or in the plural as virtual machine instances 68). In at least some examples, server manager 67 may control operation of and / or maintain servers 66. Virtual machine instances 68c and 68d are rendition switching virtual machine (“RSVM”) instances. The RSVM virtual machine instances 68c and 68d may be configured to perform all, or any portion, of the techniques for improved rendition switching and / or any other of the disclosed techniques in accordance with the present disclosure and described in detail above. As should be appreciated, while the particular example illustrated in FIG. 6 includes one RSVM virtual machine in each server, this is merely an example. A server may include more than one RSVM virtual machine or may not include any RSVM virtual machines.

[0078] The availability of virtualization technologies for computing hardware has afforded benefits for providing large scale computing resources for customers and allowing computing resources to be efficiently and securely shared between multiple customers. For example, virtualization technologies may allow a physical computing device to be shared among multiple users by providing each user with one or more virtual machine instances hosted by the physical computing device. A virtual machine instance may be a software emulation of a particular physical computing system that acts as a distinct logical computing system. Such a virtual machine instance provides isolation among multiple operating systems sharing a given physical computing resource. Furthermore, some virtualization technologies may provide virtual resources that span one or more physical resources, such as a single virtual machine instance with multiple virtual processors that span multiple distinct physical computing systems.

[0079] Referring to FIG. 6, network 141 may, for example, be a publicly accessible network of linked networks and possibly operated by various distinct parties, such as the Internet. In other embodiments, network 141 may be a private network, such as a corporate or university network that is wholly or partially inaccessible to non-privileged users. In still other embodiments, network 141 may include one or more private networks with access to and / or from the Internet.

[0080] Network 141 may provide access to user computers 62. User computers 62 may be computers utilized by users 60 or other customers of data center 65. For instance, user computer 62a or 62b may be a server, a desktop or laptop personal computer, a tablet computer, a wireless telephone, a personal digital assistant (PDA), an e-book reader, a game console, a set-top box or any other computing device capable of accessing data center 65. User computer 62a or 62b may connect directly to the Internet (e.g., via a cable modem or a Digital Subscriber Line (DSL)). Although only tw o user computers 62a and 62b are depicted, it should be appreciated that there may be multiple user computers.

[0081] User computers 62 may also be utilized to configure aspects of the computing resources provided by data center 65. In this regard, data center 65 might provide a gateway or w eb interface through which aspects of its operation may be configured through the use of a w eb browser application program executing on user computer 62. Alternately, a stand-alone application program executing on user computer 62 might access an application programming interface (API) exposed by data center 65 for performing the configuration operations. Other mechanisms for configuring the operation of various web sendees available at data center 65 might also be utilized.

[0082] Servers 66 shown in FIG. 6 may be servers configured appropriately for providing the computing resources described above and may provide computing resources for executing one ormore web services and / or applications. In one embodiment, the computing resources may be virtual machine instances 68. In the example of virtual machine instances, each of the servers 66 may be configured to execute an instance manager 63a or 63b (which may be referred herein singularly as instance manager 63 or in the plural as instance managers 63) capable of executing the virtual machine instances 68. The instance managers 63 may be a virtual machine monitor (VMM) or another ty pe of program configured to enable the execution of virtual machine instances 68 on server 66, for example. As discussed above, each of the virtual machine instances 68 may be configured to execute all or a portion of an application.

[0083] It should be appreciated that although the embodiments disclosed above discuss the context of virtual machine instances, other types of implementations can be utilized with the concepts and technologies disclosed herein. For example, the embodiments disclosed herein might also be utilized with computing systems that do not utilize virtual machine instances.

[0084] In the example data center 65 shown in FIG. 6, a router 61 may be utilized to interconnect the servers 66a and 66b. Router 61 may also be connected to gateway 64, which is connected to network 141. Router 61 may be connected to one or more load balancers, and alone or in combination may manage communications within networks in data center 65, for example, by forwarding packets or other data communications as appropriate based on characteristics of such communications (e.g., header information including source and / or destination addresses, protocol identifiers, size, processing requirements, etc.) and / or the characteristics of the private network (e.g., routes based on network topology, etc.). It will be appreciated that, for the sake of simplicity, various aspects of the computing systems and other devices of this example are illustrated without showing certain conventional details. Additional computing systems and other devices may be interconnected in other embodiments and may be interconnected in different ways.

[0085] In the example data center 65 shown in FIG. 6, a data center 65 is also employed to at least in part direct various communications to. from and / or between servers 66a and 66b. While FIG. 6 depicts router 61 positioned between gateway 64 and data center 65. this is merely an exemplary configuration. In some cases, for example, data center 65 may be positioned between gateway 64 and router 61. Data center 65 may, in some cases, examine portions of incoming communications from user computers 62 to determine one or more appropriate servers 66 to receive and / or process the incoming communications. Data center 65 may determine appropriate servers to receive and / or process the incoming communications based on factors such as an identity7, location or other attributes associated with user computers 62, a nature of a task with which the communications are associated, a priority of a task with which the communications are associated, a duration of a task with which the communications are associated, a size and / orestimated resource usage of a task with which the communications are associated and many other factors. Data center 65 may, for example, collect or otherwise have access to state information and other information associated with various tasks in order to, for example, assist in managing communications and other operations associated with such tasks.

[0086] It should be appreciated that the netw ork topology illustrated in FIG. 6 has been greatly simplified and that many more networks and networking devices may be utilized to interconnect the various computing systems disclosed herein. These network topologies and devices should be apparent to those skilled in the art.

[0087] It should also be appreciated that data center 65 described in FIG. 6 is merely illustrative and that other implementations might be utilized. It should also be appreciated that a server, gateway or other computing device may comprise any combination of hardware or software that can interact and perform the described types of functionality, including without limitation: desktop or other computers, database servers, network storage devices and other network devices, PDAs, tablets, cellphones, wireless phones, pagers, electronic organizers, Internet appliances, television-based systems (e.g., using set top boxes and / or personal / digital video recorders) and various other consumer products that include appropriate communication capabilities.

[0088] A network set up by an entity, such as a company or a public sector organization, to provide one or more web services (such as various types of cloud-based computing or storage) accessible via the Internet and / or other networks to a distributed set of clients may be termed a provider network. Such a provider network may include numerous data centers hosting various resource pools, such as collections of physical and / or virtualized computer servers, storage devices, networking equipment and the like, used to implement and distribute the infrastructure and web services offered by the provider network. The resources may in some embodiments be offered to clients in various units related to the web service, such as an amount of storage capacity’ for storage, processing capability for processing, as instances, as sets of related services and the like. A virtual computing instance may, for example, comprise one or more servers with a specified computational capacity (which may be specified by indicating the type and number of CPUs, the main memory size and so on) and a specified software stack (e.g., a particular version of an operating system, which may in turn run on top of a hypervisor).

[0089] A number of different types of computing devices may be used singly or in combination to implement the resources of the provider network in different embodiments, for example computer servers, storage devices, network devices and the like. In some embodiments a client or user may be provided direct access to a resource instance, e.g., by giving a user an administrator login and password. In other embodiments the provider network operator mayallow clients to specify execution requirements for specified client applications and schedule execution of the applications on behalf of the client on execution systems (such as application server instances, Java™ virtual machines (JVMs), general-purpose or special-purpose operating systems that support various interpreted or compiled programming languages such as Ruby, Perl, Python. C, C++ and the like or high- performance computing systems) suitable for the applications, without, for example, requiring the client to access an instance or an execution system directly. A given execution system may utilize one or more resource instances in some implementations; in other implementations, multiple execution systems may be mapped to a single resource instance.

[0090] In many environments, operators of provider networks that implement different types of virtualized computing, storage and / or other network-accessible functionality may allow customers to reserve or purchase access to resources in various resource acquisition modes. The computing resource provider may provide facilities for customers to select and launch the desired computing resources, deploy application components to the computing resources and maintain an application executing in the environment. In addition, the computing resource provider may provide further facilities for the customer to quickly and easily scale up or scale down the numbers and types of resources allocated to the application, either manually or through automatic scaling, as demand for or capacity requirements of the application change. The computing resources provided by the computing resource provider may be made available in discrete units, which may be referred to as instances. An instance may represent a physical server hardware system, a virtual machine instance executing on a server or some combination of the two. Various t pes and configurations of instances may be made available, including different sizes of resources executing different operating systems (OS) and / or hypervisors, and with various installed software applications, runtimes and the like. Instances may further be available in specific availability zones, representing a logical region, a fault tolerant region, a data center or other geographic location of the underlying computing hardware, for example. Instances may be copied within an availability zone or across availability zones to improve the redundancy of the instance, and instances may be migrated within a particular availability zone or across availability zones. As one example, the latency for client communications with a particular server in an availability zone may be less than the latency for client communications with a different server. As such, an instance may be migrated from the higher latency server to the lower latency server to improve the overall client experience.

[0091] In some embodiments the provider network may be organized into a plurality of geographical regions, and each region may include one or more availability zones. An availability zone (which may also be referred to as an availability container) in turn maycomprise one or more distinct locations or data centers, configured in such a way that the resources in a given availability zone may be isolated or insulated from failures in other availability zones. That is, a failure in one availability zone may not be expected to result in a failure in any other availability zone. Thus, the availability profile of a resource instance is intended to be independent of the availability profile of a resource instance in a different availability zone. Clients may be able to protect their applications from failures at a single location by launching multiple application instances in respective availability zones. At the same time, in some implementations inexpensive and low latency network connectivity may be provided between resource instances that reside within the same geographical region (and network transmissions between resources of the same availability zone may be even faster).

[0092] Clauses

[0093] Embodiments of the present disclosure can be described in view of the following clauses:Clause 1. A computer-implemented method, comprising: determining a first set of content items, wherein each content item of the first set of content items was previously selected at least twice by a given user of a plurality of users during a past time period; determining, using a first computer-implemented function, a respective category label for each content item of the first set of content items; generating a weighted category' graph comprising a plurality' of nodes, wherein each node is associated with a unique category label, wherein a first edge of the weighted category' graph that connects a first node and a second node represents a condition of a first category label of the first node being selected by a first user during the past time period and a second category label of the second node being selected by the first user during the past time period; determining, using a first computer-implemented clustering algorithm, a plurality of clusters of the weighted category graph based at least in part on the first edge, wherein the plurality of clusters comprises a first cluster and a second cluster; determining, for a second user, a second set of content items previously selected by7the second user; determining, using the first computer-implemented function, a respective category label for each content item of the second set of content items; determining a first overlap between the respective category labels for the second set of content items and the first cluster; determining a second overlap between the respective category labels for the second set of content items and the second cluster;selecting the first cluster based on a comparison of the first overlap and the second overlap; determining a third category7label of the first cluster that is not included in categorylabels associated with the second set of content items; determining a first content item associated with the third category label; and outputting a first graphical representation of the first content item on a display.Clause 2. The computer-implemented method of clause 1, wherein the first edge is associated with a first edge weight, the first edge weight representing a total number of times that the first category label and the second category label were both selected during the past time period by respective users of the plurality' of users.Clause 3. The computer-implemented method of clause 2, wherein: a weight of the first node represents a first total number of users that selected a content item having the first category7label during the past time period; and a weight of the second node represents a second total number of users that selected a content item having the second category' label during the past time period.Clause 4. The computer-implemented method of clause 3, further comprising: determining an edge weight threshold and a node weight threshold stored in memory; prior to determining the plurality of clusters of the weighted category graph, removing those nodes with node weights less than the node weight threshold and those edges with edge weights less than the edge weight threshold from the weighted category7graph; and removing disconnected nodes from the weighted category graph.Clause 5. A method comprising: determining a first set of content items; determining a respective first category label for each content item of the first set of content items; generating a weighted category graph comprising a plurality of nodes, wherein each node is associated with a respective first category label for the first set of content items, and wherein a first edge connecting two nodes represents the respective category7labels associated with the two nodes having been selected by a first user during a past time period; determining, for a second user, a plurality of second category labels associated with previous content selections; selecting, from among at least one cluster of the weighted category graph, at least a first cluster, wherein the first cluster is selected based at least in part on the plurality7of second category7labels;determining, based at least in part on the first cluster, a first content item for recommendation to the second user; and generating output data representing a recommendation of the first content item.Clause 6. The method of clause 5, further comprising: filtering the first set of content items from among a second set of content items, wherein the first set of content items comprises content items of the second set of content items that were previously selected at least twice by a given user of a plurality of users during a past time period. Clause 7. The method of clause 5, wherein the weighted category graph comprises a first edge between a first node and a second node, wherein the first edge is associated with a first edge weight, the first edge weight representing a total number of times that a first category label of the first node and a second category label of the second node were both selected during a past time period by respective users of a plurality of users.Clause 8. The method of clause 7, wherein: a weight of the first node represents a first total number of users that selected a content item having the first category label during the past time period; and a weight of the second node represents a second total number of users that selected a content item having the second category label during the past time period.Clause 9. The method of clause 8, further comprising: determining an edge weight threshold and a node weight threshold stored in memory; removing nodes from the weighted category graph that have node weights less than the node weight threshold and removing edges from the weighted category graph that have edge weights less than the edge weight threshold; and determining at least the first cluster and a second cluster from the weighted category graph after removing the nodes.Clause 10. The method of clause 5, further comprising determining the first cluster as a unique cluster different from any other cluster of the weighted category graph based at least in part by determining that each node of the first cluster is connected to each other node of the first cluster. Clause 11. The method of clause 5, further comprising: determining, for the first cluster, a first cluster score by determining an overlap between the plurality of second category labels and category labels of nodes of the first cluster, wherein the first cluster is selected based at least in part on a comparison of the first cluster score and a second cluster score of the second cluster.Clause 12. The method of clause 5, further comprising: determining, from among the plurality of second category labels, a category label with a highest count of past selections among the previous content selections by the second user; andselecting the first cluster from among the at least one cluster based at least in part on the category label with the highest count of past selections being associated with a node of the first cluster.Clause 13. The method of clause 5, further comprising: determining, from among category labels of the first cluster, at least a first nonoverlapping category label that does not overlap with any category label of the plurality of second category7labels; and determining the first content item based at least in part on the first content item having the first non-overlapping category label.Clause 14. The method of clause 5, further comprising: determining at least one content item selection associated with a current user session; determining a third category label associated with the at least one content item selection; and selecting the first cluster based at least in part on the third category label and the current user session.Clause 15. A system, comprising: at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to: determine a first set of content items; determine a respective first category label for each content item of the first set of content items; generate a weighted category graph comprising a plurality of nodes, wherein each node is associated with a respective first category label for the first set of content items, and wherein a first edge connecting two nodes represents the respective category7labels associated with the two nodes having been selected by a first user during a past time period; determine, for a second user, a plurality7of second category labels associated with previous content selections; select, from among at least one cluster of the weighted category graph, at least a first cluster, wherein the first cluster is selected based at least in part on the plurality of second category labels; determine, based at least in part on the first cluster, a first content item for recommendation to the second user; and generate output data representing a recommendation of the first content item.Clause 16. The system of clause 15, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: filter the first set of content items from among a second set of content items, wherein the first set of content items comprises content items of the second set of content that were previously selected at least twice by a given user of a plurality of users during a past time period. Clause 17. The system of clause 15, wherein the weighted category graph comprises a first edge between a first node and a second node, wherein the first edge is associated with a first edge weight, the first edge weight representing a total number of times that a first category label of the first node and a second category label of the second node were both selected during a past time period by respective users of a plurality of users.Clause 18. The system of clause 17, wherein: a weight of the first node represents a first total number of users that selected a content item having the first category label during the past time period; and a weight of the second node represents a second total number of users that selected a content item having the second category label during the past time period.Clause 19. The system of clause 18, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine an edge weight threshold and a node weight threshold stored in memory; remove nodes from the w eighted category graph that have node w eights less than the node w eight threshold and removing edges from the weighted category7graph that have edge weights less than the edge weight threshold; and determine at least the first cluster and a second cluster from the weighted category graph after removing the nodes.Clause 20. The system of clause 15, the non-transitory7computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine the first cluster as a unique cluster different from any other cluster of the weighted category graph based at least in part by determining that each node of the first cluster is connected to each other node of the first cluster.

[0094] Although various systems described herein may be embodied in software or code executed by general purpose hardware as discussed above, as an alternate the same may also be embodied in dedicated hardware or a combination of software / general purpose hardw are and dedicated hardware. If embodied in dedicated hardw are, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals,application specific integrated circuits having appropriate logic gates, or other components, etc. Such technologies are generally well known by those of ordinary skill in the art and consequently, are not described in detail herein.

[0095] The flowcharts and methods described herein show the functionality and operation of various implementations. If embodied in software, each block or step may represent a module, segment, or portion of code that comprises program instructions to implement the specified logical function(s). The program instructions may be embodied in the form of source code that comprises human-readable statements written in a programming language or machine code that comprises numerical instructions recognizable by a suitable execution system such as a processing component in a computer system. If embodied in hardware, each block may represent a circuit or a number of interconnected circuits to implement the specified logical function(s).

[0096] Although the flowcharts and methods described herein may describe a specific order of execution, it is understood that the order of execution may differ from that which is described. For example, the order of execution of two or more blocks or steps may be scrambled relative to the order described. Also, two or more blocks or steps may be executed concurrently or with partial concurrence. Further, in some embodiments, one or more of the blocks or steps may be skipped or omitted. It is understood that all such variations are within the scope of the present disclosure.

[0097] Also, any logic or application described herein that comprises software or code can be embodied in any non-transitory computer-readable medium or memory for use by or in connection with an instruction execution system such as a processing component in a computer system. In this sense, the logic may comprise, for example, statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a "computer-readable medium" can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. The computer-readable medium can comprise any one of many physical media such as magnetic, optical, or semiconductor media. More specific examples of a suitable computer-readable media include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium may be a random access memory (RAM) including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-onlymemory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory' device.

[0098] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the drsclosure. Many variations and modifications may be made to the abovedescribed example(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A method comprising: determining a first set of content items; determining a respective first category label for each content item of the first set of content items; generating a weighted category' graph comprising a plurality' of nodes, wherein each node is associated with a respective first category label for the first set of content items, and wherein a first edge connecting two nodes represents the respective category labels associated with the two nodes having been selected by a first user during a past time period; determining, for a second user, a plurality7of second category7labels associated w ith previous content selections; selecting, from among at least one cluster of the weighted category graph, at least a first cluster, wherein the first cluster is selected based at least in part on the plurality of second category labels; determining, based at least in part on the first cluster, a first content item for recommendation to the second user; and generating output data representing a recommendation of the first content item.

2. The method of claim 1, further comprising: filtering the first set of content items from among a second set of content items, wherein the first set of content items comprises content items of the second set of content items that were previously selected at least twice by a given user of a plurality7of users during a past time period.

3. The method of any of claims 1 to 2, wherein the weighted category' graph comprises a first edge between a first node and a second node, wherein the first edge is associated with a first edge weight, the first edge weight representing a total number of times that a first category label of the first node and a second category7label of the second node w ere both selected during a past time period by respective users of a plurality' of users.

4. The method of claim 3, wherein: a weight of the first node represents a first total number of users that selected a content item having the first category label during the past time period; and a weight of the second node represents a second total number of users that selected a content item having the second category label during the past time period.

5. The method of claim 4, further comprising: determining an edge weight threshold and a node weight threshold stored in memory; removing nodes from the weighted category graph that have node weights less than the node weight threshold and removing edges from the weighted category graph that have edge weights less than the edge weight threshold; and determining at least the first cluster and a second cluster from the weighted category' graph after removing the nodes.

6. The method of any of claims 1 to 2, further comprising determining the first cluster as a unique cluster different from any other cluster of the weighted category graph based at least in part by determining that each node of the first cluster is connected to each other node of the first cluster.

7. The method of any of claims 1 to 2, further comprising: determining, for the first cluster, a first cluster score by determining an overlap between the plurality of second category labels and category labels of nodes of the first cluster, wherein the first cluster is selected based at least in part on a comparison of the first cluster score and a second cluster score of the second cluster.

8. The method of any of claims 1 to 2, further comprising: determining, from among the plurality of second category labels, a category label with a highest count of past selections among the previous content selections by the second user; and selecting the first cluster from among the at least one cluster based at least in part on the category label with the highest count of past selections being associated with a node of the first cluster.

9. The method of any of claims 1 to 2, further comprising: determining, from among category labels of the first cluster, at least a first nonoverlapping category label that does not overlap with any category label of the plurality of second category labels: and determining the first content item based at least in part on the first content item having the first non-overlapping category label.

10. The method of any of claims 1 to 2, further comprising: determining at least one content item selection associated with a current user session; determining a third category label associated with the at least one content item selection; and selecting the first cluster based at least in part on the third category' label and the current user session.

11. A system, comprising: at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to: determine a first set of content items; determine a respective first category label for each content item of the first set of content items; generate a weighted category graph comprising a plurality of nodes, wherein each node is associated with a respective first category label for the first set of content items, and wherein a first edge connecting two nodes represents the respective category labels associated with the two nodes having been selected by a first user during a past time period; determine, for a second user, a plurality of second category labels associated with previous content selections; select, from among at least one cluster of the weighted category graph, at least a first cluster, wherein the first cluster is selected based at least in part on the plurality of second category’ labels; determine, based at least in part on the first cluster, a first content item for recommendation to the second user; and generate output data representing a recommendation of the first content item.

12. The system of claim 11, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: filter the first set of content items from among a second set of content items, wherein the first set of content items comprises content items of the second set of content that were previously selected at least twice by a given user of a plurality of users during a past time period.

13. The system of any of claims 11 to 12, wherein the weighted category graph comprises a first edge between a first node and a second node, wherein the first edge is associated with a first edge weight, the first edge weight representing a total number of times that a first category label of the first node and a second category label of the second node were both selected during a past time period by respective users of a plurality of users.

14. The system of claim 13, wherein: a weight of the first node represents a first total number of users that selected a content item having the first category label during the past time period; and a weight of the second node represents a second total number of users that selected a content item having the second category label during the past time period.

15. The system of claim 14, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine an edge weight threshold and a node weight threshold stored in memory; remove nodes from the weighted category graph that have node weights less than the node weight threshold and removing edges from the weighted category graph that have edge weights less than the edge weight threshold; and determine at least the first cluster and a second cluster from the weighted category' graph after removing the nodes.

Images