Understanding-based identification of educational content among multiple content types

Abstract

Implementations disclose understanding-based identification of educational content of multiple content types. A method includes determining a respective understanding ranking signal for a content item corresponding to a user request, the understanding ranking signal based on a learning attribute score generated for the content item from at least one machine learning model; determining a learning level of a user corresponding to the user request; ranking the content item based on a mapping between the learning level and the respective understanding ranking signal for the content item; and providing a recommendation for the content item in accordance with the ranking of the content item.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of content sharing platforms, and in particular, to understanding-based identification of educational content among multiple content types. BACKGROUND

[0002] On the Internet, content sharing platforms allow users to connect with each other and share information with each other. The content sharing aspect of such platforms allows users to upload, view, and share content such as video content, image content, audio content, text content, websites, applications, channels, playlists, and the like (which can be collectively referred to as "media items" or "content items"). Such viewable and shareable content items can include audio clips, movie clips, TV clips, and music videos, as well as amateur content such as video blogs, short original videos, pictures, photos, other multimedia content, and the like. Users can use computing devices such as smart phones, cellular phones, laptop computers, desktop computers, netbooks, tablet computers, network-connected televisions to use, play, and / or consume media items (e.g., watch digital videos and / or listen to digital music). SUMMARY

[0003] The following is a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is neither intended to identify key or critical elements of the disclosure nor to delineate any scope of the particular implementations of the disclosure or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the disclosure, a method includes determining, by a processing device, a respective understanding rank signal for a content item corresponding to a user request, the understanding rank signal being based on a learning attribute score generated for the content item from at least one machine learning model, determining a learning level of a user corresponding to the user request, ranking the content item based on a mapping between the learning level and the respective understanding rank signal for the content item, and providing, by the processing device, a recommendation for the content item in accordance with the ranking of the content item.

[0005] In some implementations, the method further includes identifying a change in the learning level of the user to a new learning level, updating the recommendation for the content item based on a mapping between the new learning level and the respective understanding rank signal for the content item, and automatically updating the recommendation for the content item in accordance with the updated ranking of the content item.

[0006] In one implementation, the user request of the method includes at least one of a search query or navigation to a portion of an application. Further, the application can be an application launcher on a home screen user interface of the client device. Additionally, the user request can correspond to a topic associated with a learning experience.

[0007] In one implementation, the content items include at least one of an application, a video, a book, or a webpage. Further, providing the recommendations of the method can include recommending higher ranked content items in preference to lower ranked content items. Additionally, the method can further include modifying the rankings of the content items based on a level of interest of the user in the content items.

[0008] In some implementations, the learning attribute scores correspond to learning attributes including two or more of appeal, depth, spark, learning impact, or development appropriateness. The at least one machine learning model includes a machine learning model for each learning attribute for each of a plurality of learning levels. Further, the at least one machine learning model can be trained using manual ratings for sample content items and predetermined learning levels. In one implementation, the learning attribute scores generated by the at least one machine learning model are combined to generate an understanding ranking signal.

[0009] In one implementation, combining the learning attribute scores includes applying weights to the learning attribute scores. Further, the understanding ranking signal can be used to place at least one of the content items in a learning tree. Additionally, the learning level of the user is based on at least one of a user artifact of the learning level, a user metric, or a manual user input. In some implementations, the method can further include at least one of automatically installing, automatically uninstalling, or automatically updating the recommended content items provided via a user interface, UI, of the client device based on updated recommendations for the content items.

[0010] Computing devices for performing the operations of the above-described methods and various implementations described herein are disclosed. Computer-readable media storing instructions for performing operations associated with the above-described methods and various implementations described herein are also disclosed. BRIEF DESCRIPTION OF DRAWINGS

[0011] The disclosure is illustrated by way of example, and not by way of limitation, in the accompanying drawings.

[0012] Figure 1 is a block diagram illustrating an example network architecture in which implementations of the present disclosure can be implemented.

[0013] Figure 2 is a flow diagram illustrating a method for understanding-based identification of educational content in a plurality of content types, according to an implementation.

[0014] Figure 3is a flowchart illustrating a method for identifying a learning level of a user based on comprehension-based identification of educational content in a plurality of content types, according to an embodiment.

[0015] Figure 4A and Figure 4B is a flowchart illustrating a method for generating a comprehension ranking signal and using the comprehension ranking signal for comprehension-based identification of educational content in a plurality of content types, according to an embodiment.

[0016] Figure 5 is a flowchart illustrating a method for training a machine learning model for comprehension-based identification of educational content in a plurality of content types, according to an embodiment.

[0017] Figure 6 is a flowchart illustrating a method for progressive content recommendation updates for comprehension-based identification of educational content in a plurality of content types, according to an embodiment.

[0018] Figure 7 illustrates an example screenshot of a learning application home screen user interface providing comprehension-based educational content suggestions, according to an embodiment of the disclosure.

[0019] Figure 8 is a block diagram illustrating one embodiment of a computer system, according to an embodiment. DETAILED DESCRIPTION

[0020] Aspects and embodiments of the present disclosure relate to comprehension-based identification of educational content in a plurality of content types. Embodiments of a learning content system are described that are capable of identifying, curating, and presenting educational content that is appropriate and relevant to a requesting user’s level of understanding. Educational content can refer to one or more content items that are instructive, informative, academic, and / or safe (e.g., not mature, violent, or explicit).

[0021] Embodiments of the present disclosure can identify content that is educational and at a corresponding level of understanding of a user interacting with the content. For example, a learning content system can identify educational topics (e.g., subjects, people, places, things, etc.) for a user, identify content items (e.g., applications, videos, books, websites, etc.) for different user understanding level groups for the topics, rank the identified content items according to comprehension and other metrics (e.g., interest level, etc.), and publish the content to a learning content interface of a content sharing platform. In one embodiment, the learning content interface can include a home screen user interface (UI) of a learning dedicated application associated with the content sharing platform.

[0022] Existing content curation solutions for learning and / or educational environments generally do not automate the process of identifying, curating, and presenting educational content that is appropriate and relevant to a user's different levels of learning (understanding). Existing content curation solutions for learning and / or educational environments generally provide a whitelist approach, where a small set of content is manually selected from an entire corpus of content. However, in current platforms for content curation, there can be more uploaded content to parse than is actually used for human review (e.g., 300 hours of video uploaded to many platforms every minute; a large number of apps and books and websites, etc. available and introduced every day). Unlike aspects of the present disclosure, these existing solutions are generally not algorithmically performed, do not use a large amount of content, and / or do not have a high rate of incoming, new content and a large base of content.

[0023] The technical problem addressed by embodiments of the present disclosure is that unnecessary processing resources and network bandwidth resources can be consumed by excessive communication between a client device and a server when attempting to determine an ideal set of content items curated for a user in terms of educational value and understanding of the user. For example, typing a search query for relevant educational content corresponding to an understanding level of a particular user can generally be attempted multiple times in order to drill down to the most relevant content for the user. Multiple requests can undesirably consume processing resources on the client device and server device. This is particularly problematic for client devices that have limited battery life and deplete the battery and consume network bandwidth of the device by processing multiple requests over a network.

[0024] Further, each time an irrelevant and undesirable content item is presented to a user as a result of a search query or user navigation within an application, the user can continue to request updated content via a new search request or additional navigation within the application. These additional requests are sent from the client device to the server to find content items based on the updated request. The server can send new results including different content items as suggestions to the client device that is displaying the search or navigation user interface. The data payloads for transmitting the updated suggestions can include thumbnails, full video files of media content items, and metadata (e.g., titles and descriptions of media content items). Sending multiple data payloads when the user enters a search query or continues to navigate can consume processing resources at the server to perform the finds, processing resources at the client device to display the list of suggested content items, network bandwidth resources to send multiple data payloads when an updated request is entered, etc. This results in slow processing for the user to reach a desired content item suggestion.

[0025] Ranked search results have also been utilized in existing search engines. However, such known solutions suffer from inaccuracies in determining educational content that matches and / or is consistent with a requesting user's level of learning (understanding). Moreover, these known solutions do not progressively update suggested content for a user as the user's level of understanding changes.

[0026] A technical solution to the above technical problems can include identifying educational content that is at an appropriate level of learning (understanding) for a user interacting with the content. For example, a learning content system can utilize machine learning to identify educational content (e.g., applications, videos, books, websites, etc.), algorithmically select such content for different user learning level (understanding level) groups, rank the selected content according to the user's level of learning and other metrics (e.g., level of interest, etc.), and publish the content to a learning content interface of a content sharing platform. In one implementation, the learning content interface can include a home screen UI of a learning dedicated application associated with the content sharing platform. Alternatively, the learning content platform can provide a service that identifies educational content at an appropriate level of learning for various third party platforms (e.g., various social networking platforms, search engine platforms, media service platforms, online news platforms, content sharing platforms, etc.).

[0027] Moreover, the learning content system can continuously and / or periodically reevaluate the user's level of learning in order to respond to changes in the user's level of learning by dynamically and progressively updating the content curation provided to the user. If the user's level of learning improves to the next level (or degrades to a lower level), the learning content system can reevaluate the mapping of the understandability ranking signals of the content items being accessed by the user to the updated level of learning of the user. As a result, the learning content system can dynamically present new and / or different content for the user and provide such content to the user without the user having to request such updated content. This can be performed without any user intervention, thus providing a seamless, progressive, and relevant experience in terms of content curation (with a learning application or with other services). Moreover, processing resources and network bandwidth are reduced in order to present the most relevant, educational, and understandable content to the user as a result of less communication occurring between the client device and the server.

[0028] In one implementation, content items can be automatically installed, uninstalled, and / or updated on a computing device based on content recommendations. For example, as a user's learning level improves to a next level, new content appropriate for the user's new level can be automatically installed on the user's computing device without user intervention. Similarly, old content that is no longer appropriate for the user's new level can be automatically uninstalled from the user's computing device without user intervention. As another example, content that is no longer appropriate for the user's new level can be automatically updated (e.g., by replacing the content with a different version, or by installing a plug-in) so that the updated content is appropriate for the user's new level. This can overcome or mitigate problems associated with manual installation, uninstallation, and updating of content on a computing device, such as the user having the appropriate security permissions on the device and / or sufficient technical knowledge of how to install or uninstall content. Moreover, by automatically uninstalling content that is no longer appropriate for the user's learning level, the limited data storage capacity of the computing device can be better used.

[0029] Accordingly, a technical effect can include reducing the number of communications required to present educational content corresponding to a user's learning (understanding) level. Because the improved content item identification and user understanding (learning level) matching occurs on a backend server based on a machine learning model, the reduced number of communications can also reduce processing resources (across all of the remote control, client device, and backend server) and network bandwidth resources expended during searches.

[0030] In contrast to known solutions that rank content items based on affinity scores and user history, in embodiments of the present disclosure, machine learning is used to identify educational content that matches and / or is consistent with a requesting user's learning level (understanding level), and moreover, the educational content is progressively updated without user intervention to match the user's changing learning (understanding) level.

[0031] For simplicity and brevity, the present disclosure often refers to videos. However, the teachings of the present disclosure are generally applicable to media items, and can be applied to various types of content or media items, including, for example, videos, audio, text, images, program instructions, etc.

[0032] Figure 1An example system architecture 100 according to one embodiment of the present disclosure is illustrated. The system architecture 100 includes client devices 110A-110Z, a network 105, a data store 106, a content sharing platform 120, a server 130, and other platform servers 150. In one embodiment, the network 105 can include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), a wired network (e.g., Ethernet network), a wireless network (e.g., an 802.11 network or a Wi-Fi network), a cellular network (e.g., a Long Term Evolution (LTE) network), routers, hubs, switches, server computers, and / or a combination thereof. In one embodiment, the data store 106 can be a memory (e.g., a random access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, or another type of component or device capable of storing data. The data store 106 can also include multiple storage components (e.g., multiple drives or multiple databases) that can also span multiple computing devices (e.g., multiple server computers).

[0033] The client devices 110A-110Z can each include a computing device, such as a personal computer (PC), a laptop computer, a mobile phone, a smart phone, a tablet computer, a netbook computer, a network-connected television, and the like. In some embodiments, the client devices 110A-110Z can also be referred to as "user devices." Each client device includes a media viewer 111. In one embodiment, the media viewer 111 can be an application that allows a user to view content, such as images, videos, webpages, documents, and the like. For example, the media viewer 111 can be a web browser that is capable of accessing, retrieving, rendering, and / or navigating content provided by a web server (e.g., a webpage such as a HyperText Markup Language (HTML) page, a digital media item, and the like). The media viewer 111 can render, display, and / or present content (e.g., a webpage, a media viewer) to a user. The media viewer 111 can also display an embedded media player (e.g., a Flash® player or an HTML5 player) embedded in a webpage (e.g., a webpage that can provide information about a product sold by an online merchant).

[0034] ​In another example, the media viewer 111 can be a standalone application that allows a user to view digital media items (e.g., digital videos, digital images, e-books, etc.). The media viewer 111 can be an application launcher (also referred to as a launcher application, launcher app, or launcher). An application launcher can refer to a computer program that helps users to locate and start other computer programs. An application launcher can provide the backbone of the application experience on a mobile device, perform basic functions such as opening by default when booting, default assignment to the home button on a mobile device, provide a location to store applications ("apps") once installed, etc. According to aspects of the present disclosure, the media viewer 111 can be an application that is educationally or learning- specific that allows a user to view and search for content that is appropriate for a user's level of learning (understanding).

[0035] The media viewer 111 can be provided to the client devices 110A-Z by the server 130 and / or the content sharing platform 120. For example, the media viewer 111 can be an embedded media player that is embedded in a web page provided by the content sharing platform 120. In another example, the media viewer 111 can be an application that is downloaded from the server 130.

[0036] Generally, functions described in one implementation as being performed by the content sharing platform 120 or by the servers 130, 150 can also be performed in other implementations, on the client devices 110A-Z, as appropriate. Additionally, functions attributed to particular components can be performed by different or multiple components operating together. The content sharing platform 120 can also be accessed as a service provided to other systems or devices through appropriate application programming interfaces, and thus is not limited to use in websites.

[0037] In one implementation, the content sharing platform 120 and / or the servers 130, 150 can be one or more computing devices (such as rack-mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc.), data stores (e.g., hard disks, memories, databases), networks, software components, and / or hardware components that can be used to provide users with access to media items and / or provide users with media items. For example, the content sharing platform 120 can allow users to consume, upload, search, like, dislike, and / or comment on media items. The content sharing platform 120 can also include a website (e.g., web pages) or application backend software that can be used to provide users with access to media items.

[0038] In implementations of the present disclosure, a "user" can be represented as a single individual. However, other implementations of the present disclosure encompass a "user" as an entity controlled by a set of users and / or automated sources. For example, a set of individual users that are federated as a community in a social network can be considered a "user." In another example, an automated consumer can be an automated ingestion pipeline of the content sharing platform 120, such as a topic channel.

[0039] The content sharing platform 120 can include a plurality of channels (e.g., channel A 122A through channel Z 122Z). The channels 122A-122Z can be data content available from a common source or data content having a common topic, subject, or substance. The data content can be digital content selected by a user, digital content available to a user, digital content uploaded by a user, digital content selected by a content provider, digital content selected by a broadcaster, etc. For example, channel X can include videos Y and Z. The channels 122A-122Z can be associated with an owner, which is a user that can perform actions on the channel. Based on actions of the owner, such as the owner making digital content available on the channel, the owner selecting (e.g., liking) digital content associated with another channel, the owner commenting on digital content associated with another channel, etc., different activities can be associated with the channels 122A-122Z. The activities associated with the channels 122A-122Z can be collected into an activity feed for the channel. In addition to the owner of the channels 122A-122Z, a user can subscribe to one or more channels in which they are interested. The concept of "subscribing" can also be referred to as "liking," "following," "friending," etc.

[0040] Once a user subscribes to a channel 122A-122Z, the user can be presented with information from the activity feed of the channel. If the user subscribes to multiple channels, the activity feeds of each channel to which the user subscribes can be combined into a federated activity feed. The user can be presented with information from the federated activity feed. The channels 122A-122Z can have their own feeds. For example, when navigating to a homepage of a channel on the content sharing platform, feed items generated by the channel can be shown on the channel homepage. A user can have a federated feed, which is a feed composed of at least a subset of content items from all channels to which the user subscribes. The federated feed can also include content items from channels to which the user does not subscribe. For example, the content sharing platform 120 or other social network can insert recommended content items into the user's federated feed, or can insert content items associated with relevant connections of the user into the federated feed.

[0041] Each channel 122A-122Z can include one or more content items 121. Examples of content items 121 can include, but are not limited to, digital videos, digital movies, digital photos, digital music, website content, social media updates, ebooks, electronic magazines, digital newspapers, digital audio books, electronic magazines, web blogs, Really Simple Syndication (RSS) feeds, electronic comic books, software applications, etc. In some implementations, content items 121 are also referred to as media items.

[0042] Content items 121 can be consumed via the Internet and / or via mobile device applications. For brevity and simplicity, throughout the document, online videos (hereinafter also referred to as videos), applications ("apps"), books, and / or websites are used as examples of content items 121. As used herein, "content," "content item," "media," "media item," "online media item," "digital media," and "digital media item" can include an electronic file that can be executed or loaded using software, firmware, or hardware configured to present the digital media item to an entity. In one implementation, content sharing platform 120 can use data store 106 to store content items 121. In some implementations, servers 130, 150, and other external data sources can also store and provide content items 121.

[0043] In one implementation, servers 130 can be one or more computing devices (e.g., rack-mounted servers, server computers, etc.). In one implementation, servers 130 can be included as part of content sharing platform 120. In other implementations, servers 130 are a separate service from content sharing platform 120. Servers 130 can include a learning content system 140 that is capable of identifying, curating, and presenting educational content that is appropriate and relevant to the learning (understanding) level of an accessing user. Educational content can refer to one or more content items that are instructional, informative, academic, and / or in many cases, safe (e.g., not mature, violent, or explicit).

[0044] Embodiments of the present disclosure can identify educational and at an appropriate learning (understanding) level of a user interacting with and / or accessing content. For example, the learning content system 140 can identify educational topics (e.g., subjects, people, places, things, etc.) for a user, algorithmically select and recommend content items (e.g., apps, videos, books, websites, etc.) for different user learning level (understanding level) groups for the topics, rank the recommended content according to understanding and other attributes (e.g., interest level, depth, etc.), and publish the content to a learning content interface of the learning content system 140 or the content sharing platform 120. In one embodiment, the learning content interface can include a home screen UI of a learning dedicated application associated with the content sharing platform 120 and / or with the server 130. Alternatively, the learning content system 140 can provide a service to various third party platforms (e.g., various social networking platforms, media service platforms, search engine platforms, online news platforms, content sharing platforms, etc.) to identify educational content at an appropriate learning level of a user.

[0045] In some embodiments, the learning content system 140 of the server 130 can interact with the content sharing platform 120 and / or with other platform servers 150 to provide embodiments of the present disclosure. Although embodiments of the present disclosure are discussed in the context of a content sharing platform, embodiments can also be applied generally to any type of social networking that provides connections between users, any type of media service platform that provides content to users, any type of search engine platform, any type of online news platform, etc. Embodiments of the present disclosure are not limited to a content sharing platform that provides channel subscriptions to users.

[0046] In situations in which the systems discussed here collect personal information about users, or can make use of personal information, the users can be provided with an opportunity to control whether learning content system 140 and / or content sharing platform 120 collects user information (e.g., information about a user's social network, social actions or activities, profession, a user's preferences, or a user's current location), or to control whether and / or how to receive content from the content server that can be more relevant to the user. In addition, certain data can be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity can be treated so that no personally identifiable information can be determined for the user, or a user's geographic location can be generalized where location information is obtained (such as to a city, postal code, or state level), so that a particular location of a user cannot be determined. Thus, the user can have control over how information is collected about the user and used by learning content system 140 and / or content sharing platform 120.

[0047] Further to the descriptions herein, a user can be provided with controls to allow the user to make an election as to whether and when, or on what terms, the user is willing to allow the collection of user information (e.g., information about a user's social network, social actions or activities, profession, a user's preferences, a user's current location, or a user's learning level) by or with the systems described herein, and can be provided with controls to allow the user to make an election not to allow the sending of content or communications to the user from a server. In addition, certain data can be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. As one example, a user's identity can be treated so that no personally identifiable information can be determined for the user, or a user's geographic location can be generalized where location information is obtained (such as to a city, postal code, or state level), so that a particular location of a user cannot be determined. Thus, the user can have control over what information is collected about the user, how that information is used, and what information is provided to the user.

[0048] The learning content system 140 can interact with a single content platform (e.g., a social network, a media service provider, a search engine, an online news provider, etc.) or can be utilized across multiple content platforms (e.g., provided as a service of a content sharing platform utilized by other third party content platforms). In one implementation, the learning content system 140 includes a machine learning (ML) model 141, a training engine 142, a learning tree 143, a signal generation engine 144, and a content recommendation engine 145 and a UI generation engine 146. More or fewer components can be included in the learning content system 140 without loss of generality. For example, two of the modules can be combined into a single module, or one of the modules can be split into two or more modules. In one implementation, one or more of the modules can reside on different computing devices (e.g., different server computers, on a single client device, or distributed among multiple client devices, etc.). Further, one or more of the modules can reside on different content sharing platforms, third party social networks, and / or external servers.

[0049] The learning content system 140 can be communicably coupled to the data store 106. For example, the learning content system 140 can be coupled to the data store 106 via a network (e.g., via a network 104 as described above, via a direct connection, etc.). In one implementation, the learning content system 140 can be coupled to the data store 106 via a network 104 and via a direct connection. Figure 1The network 105 shown in FIG. 1 is coupled to a data store 106. In another example, the learning content system 140 can be directly coupled to a server in which the learning content system 140 resides (e.g., can be directly coupled to the server 130). The data store 106 can be a memory (e.g., a random access memory), a cache, a drive (e.g., a hard disk drive), a flash drive, a database system, or another type of component or device capable of storing data. The data store 106 can also include multiple storage components (e.g., multiple drives or multiple databases) that can also span multiple computing devices (e.g., multiple server computers). The data store 106 includes content item data 190, training data 191, comprehensibility data 192, and learning level data 193.

[0050] As described above, in embodiments of the present disclosure, the learning content system 140 is able to identify, curate, and present educational content that is appropriate and relevant to the learning (comprehension) level of an accessing user. Educational content can refer to one or more content items that are instructive, informative, academic, and / or in many cases, safe (e.g., not mature, violent, or explicit).

[0051] Embodiments of the present disclosure generate ML models that are able to determine a score and associate that score with various learning attributes associated with a content item. Such “learning attributes” can include, but are not limited to, appeal, depth, sparkability, learning impact, and development appropriateness. Embodiments of the present disclosure can utilize more or fewer learning attributes than described herein. The discussion of learning attributes provided herein is not meant to be exhaustive, and other learning attributes can also be utilized in embodiments of the present disclosure and / or other terminology can be used to describe the learning attributes described above.

[0052] The learning attribute of appeal refers to how likely a user is to engage with a content item. The learning attribute of depth refers to how deep or how in-depth a content item is in teaching a user about a topic. The learning attribute of sparkability refers to a measure of how much a content item stimulates a user to learn more about the content of the content item and / or engage with other content items similar to the content item. The learning attribute of learning impact refers to a measure of whether a content item conveys new learning or value to a user. The learning attribute of development appropriateness refers to a measure of how difficult a content item can be for a user (e.g., too easy, just right, too difficult, etc.).

[0053] The combination of the above learning attributes can be indicative of an understanding level associated with the content item. An understanding level (also referred to herein as a “learning level”) refers to the ability to understand and interpret something that is presented to a user. For example, with respect to reading, understanding refers to more than just the act of correctly identifying the words being read. Rather, for the example of reading, understanding refers to correctly understanding and interpreting the words being read. Further, with respect to reading, understanding involves (1) decoding the content being read, (2) making connections between the content being read and known content, and (3) thinking deeply about the content being read.

[0054] As such, embodiments of the present disclosure utilize one or more machine learning (ML) models to generate a learning attribute score for each learning attribute of a content item. The generated learning attribute scores can be used to indicate an understanding level associated with the content item. The understanding level indicated by the learning attribute scores can be referred to herein as an understanding ranking signal. The understanding ranking signal for a content item can be used to rank, recommend, and / or curate content items to users having an understanding level corresponding to the understanding ranking signal of the content item. In some embodiments, the learning attribute scores can be used individually to rank, recommend, and / or curate content items. In such cases, the learning attribute scores can each be considered an understanding ranking signal for the content item.

[0055] In some embodiments, the learning attributes of content items are scored for different learning levels. In one embodiment, the learning levels can be analogous to degree levels in an educational background (e.g., pre-K, K, 1st grade, …, 12th grade, undergraduate level, graduate level, post-graduate level, etc.). Thus, in some embodiments, the ML models can be trained for each learning attribute at each learning level. For example, a first ML model can be trained for the engagement learning attribute at a first learning level (e.g., a pre-K level), a second ML model can be trained for the engagement learning attribute at a second learning level (e.g., a K level), and so on. As a result, a content item can have a learning attribute score (for each learning attribute) for each learning level generated by multiple ML models 141.

[0056] Accordingly, the training engine 142 can be used to train one or more ML models 141 that are trained to generate learning attribute scores (and, thus, comprehensibility ranking signals) for content items. The training engine 142 can include processing device(s) such as computers, microprocessors, logic devices, or other devices or processors configured with hardware, firmware, and software to perform some embodiments described herein. The training engine 142 can include or have access to a set of training data files and a corresponding summary of each of the training data files that are used by the training engine 142 as training data (e.g., stored as training data 191 in the data store 106) to train the ML models 141 to perform learning attribute score generation.

[0057] The ML models 141 can refer to model artifacts created by the training engine 142 using training inputs and corresponding target outputs (e.g., as found in the training data 191 of the data store 106). The training inputs can include the set of training data files, and the corresponding target outputs can be learning attribute scores for the respective training inputs. In some embodiments, the training data files and corresponding target outputs can include a particular format (e.g., bullet point lists). In one implementation, the training data can include sample content items identified with a topic and / or a learning level. In one implementation, the metadata of the content items includes the topic and / or the learning level. (Such metadata can be stored in the data store 106, e.g., as content item data 190.) The sample content is manually rated with a learning attribute score for each of the learning attributes at each learning level. These manual ratings are the target outputs. In one implementation, a teacher or other educational or subject matter expert can provide the manual ratings for each learning attribute at each learning level for the sample content.

[0058] The ML models 141 can use the training inputs and target outputs to learn features of words, phrases, or sentences in the text that can predict and / or lead to the target outputs. The features can include occurrences of particular words in the text, frequencies of words or phrases, co-occurrences of words or phrases, number of words in a sentence, placement of words in the text, etc. As noted above, multiple ML models 141 can be trained. For example, a first ML model 141 can be trained for the appeal learning attribute at a first level of learning, a second ML model 141 can be trained for the appeal learning attribute at a second level of learning, and so on for the appeal learning attribute at each level of learning. Similarly, a third ML model 141 can be trained for the depth learning attribute at a first level of learning, a fourth ML model 141 can be trained for the depth learning attribute at a second level of learning, and so on for the depth learning attribute at each level of learning. In this case, an ML model 141 can be trained for each learning attribute at each level of learning. In other implementations, a single ML model 141 can be trained to generate learning attribute scores for each level of learning.

[0059] In some implementations, the ML models 151 can be trained by a combination of content type and learning attribute. For example, an ML model can be trained for videos for each learning attribute at each level of learning. A further ML model can be trained for apps for each learning attribute at each level of learning. Similarly, an ML model can be trained for books, websites, and other content types for each learning attribute at each level of learning.

[0060] Once trained, the ML models 141 can be applied to new content items to obtain learning attribute scores for each learning attribute at each level of learning for the new content items. In this way, a new content item can have various different learning attribute scores (e.g., an appeal score at a first level of learning can be different than an appeal score at a second level of learning, etc.) for the same learning attribute (e.g., appeal, depth, thrift, learning impact, development appropriateness) based on the ML model 141 for the level of learning being applied. As discussed further below, the learning attribute scores generated from the ML models 141 can be used to indicate for which level of learning a content item is ideal.

[0061] In one implementation, the learning attribute scores can be numerical values. For example, the learning attribute scores can be numerical values on a scale (e.g., from zero to one, one to five, zero or one to one hundred, etc.) that indicate how central / relevant a content item is for a particular learning attribute, where higher values can indicate that the content item is more central / relevant to the learning attribute. In some implementations, content items with learning attribute scores below a threshold value can be filtered out from the result set.

[0062] In one implementation, the learning attribute scores generated for content items can be stored for later use. For example, the learning attribute scores for a content item can be stored with the content item in content item data 190 or as interpretative data 192 of data store 106. In some implementations, the generated learning attribute scores are used in real-time for on-the-fly ranking, recommendation, and / or curation of content items.

[0063] In one implementation, different types of content items can be processed differently for ingestion by ML model 141. For example, books and texts can be parsed to analyze frequency and co-occurrence of words, placement of words, etc. For videos, optical character recognition (OCR) and natural language processing (NLP) techniques can be applied to the video to extract audio and text and identify what is being discussed. In addition, metadata of the video can also be used as input to ML model 141. For applications (“apps”), relevant terms can be extracted from the app by simulating interactions with the app (e.g., navigating through the app) and taking screenshots of the generated interactions. The captured screenshot images can then be analyzed via OCR on the text (and NPL on any audio) to extract the content of the app for ingestion by ML model 141. For webpages, a combination of the above techniques can be used to extract content for ingestion by ML model 141.

[0064] In some implementations, the trained ML model 141 can be periodically updated based on received feedback regarding the generated learning attribute scores. This allows for continuous improvement and development of ML model 141 based on real-time feedback received regarding the learning attribute scores.

[0065] For purposes of ranking, recommending, and / or curating content items for a user, signal generation engine 144 can cause ML model 141 to be applied to new content items. In one implementation, content items can be analyzed by signal generation engine 144 in response to a user requested search query. In some implementations, content items can be analyzed by signal generation engine 144 in response to a user navigating to a particular section of an application. In some implementations, content items can be analyzed by signal generation engine 144 in response to a user initializing an application (e.g., content curated on a login page of the application).

[0066] The signal generation engine 144 can utilize the learning attribute scores generated by the ML model 141 in various ways. In one implementation, the signal generation engine 144 utilizes the learning attribute scores generated by the ML model 141 to provide curated content items to a learning content interface of a learning- dedicated application associated with the content sharing platform 120 and / or with the server 130. The learning-dedicated application can be provided to users via the media viewers 111 of the client devices 110A-110Z. More specifically, the signal generation engine 144 can utilize the learning attribute scores of content items to curate content items that are educational and at an appropriate level of understanding for users interacting with content via the learning-dedicated application. Example user interfaces (UIs) of the learning-dedicated application are discussed in further detail below with respect to FIGS. 6-8. Figure 7 Example user interfaces (UIs) of the learning-dedicated application are discussed in further detail below with respect to FIGS. 6-8.

[0067] In some implementations, the signal generation engine 144 utilizes the learning attribute scores generated by the ML model 141 to generate ranking signal(s) of content items for other uses of the learning content system 140 (e.g., the learning tree 143 discussed further below) and for uses of other platform servers 150. For example, the other platform servers 150 can provide services for recommending content items, such as but not limited to search engines, social networks, media service providers, online news providers, and the like. The other platform servers 150 can utilize the understanding ranking signals generated by the signal generation engine 144 from the learning attribute scores in order to rank and / or recommend content provided by the other platform servers 150.

[0068] The signal generation engine 144 can utilize the learning attribute scores provided by the ML model 141 in different ways to provide different content curation results. For example, the signal generation engine 144 can combine the learning attribute scores into a single combined understanding ranking signal that is used for ranking, recommending, and / or curating content items. The learning attribute scores can be combined based on various functions, such as an average, a weighted average, a mean, a median, or any known mathematical function for combining numerical values to generate one or more values.

[0069] When combining the scores based on the purpose of curation, the signal generation engine 144 can weight one or more of the attributes higher than others. For example, to curate content items that can encourage a user to gain additional interest in a certain subject area, the signal generation engine 144 can weight the attractiveness and / or spark learning attribute scores of content items higher than the other learning attribute scores of depth, learning impact, and development appropriateness. Similarly, when attempting to challenge a user with more difficult content, the development appropriateness learning attribute score can be weighted higher than the other learning attribute scores, and so on. In some implementations, the signal generation engine 144 can consider a subset of the learning attribute scores while ignoring others of the learning attribute scores.

[0070] In some implementations, the signal generation engine 144 can construct a combination of learning attribute scores in order to generate an understandability ranking signal that identifies content items that a user at a particular learning level will understand but still be challenged by. In this way, the generation of the understandability ranking signal can more strongly weight those learning attributes (e.g., developmental appropriateness, depth) and then select content items with higher scores that indicate that the content items are slightly more difficult than average.

[0071] Once the signal generation engine 144 generates the understandability ranking signal, the signal can be passed to the content recommendation engine 145 in order to identify content items for recommendation and / or curation. For the purpose of recommending and / or curating content to a user of an application, such as but not limited to the learning-specific applications described herein, the content recommendation engine 145 determines a learning level of a user that accesses curated content. As noted below, the determined learning level can also be for an associated user of the user (e.g., a family member). The learning level of the user can be stored and maintained in the learning level data 193 of the data store 106. In some implementations, the learning level of the user can be related to an educational grade level that the user is at or that the user has recently completed. In one implementation, the learning level can be a numerical value that indicates the learning level of the user. However, other representations of the learning level of the user are also contemplated.

[0072] In some implementations, the learning level of the user can be based on an age provided by the user. Within a particular age, the learning level can be adapted based on how long the user has been at that age and / or at that particular learning level. In some implementations, the learning level of the user can be based on a user artifact, including a history of the user’s interactions with the learning-specific application and / or with the content sharing platform 120 and / or third-party content platforms. In some implementations, the learning level of the user can be based on a learning level metric, such as a vocabulary level provided by the user or determined via interactions with the user. In further implementations, the learning level of the user can be based on a user-indicated input that indicates the learning level of the user. The learning level of the user can be based on any of the above factors and / or a combination of the above factors, as well as other learning level determinations. For example, the learning level of the user can be determined based on an ML application to artifacts generated by the user (e.g., a user history within the learning-specific application).

[0073] In some implementations, a user can have more than one associated learning level. For example, a user can have a first learning level associated with a first subject area (e.g., math), while having a second (different) learning level associated with a second subject area (e.g., English Language Arts (ELA)), and so on. Further, a user's learning level can be adjusted as the user progresses (or regresses). As such, the content recommendation engine 145 can periodically evaluate or determine a user's learning level in order to identify any changes in the user's learning level. In some implementations, a user's learning level associated with a primary user can be utilized. For example, a user's learning level of a family member of the primary user can be determined and utilized in order to curate content specific to that family member.

[0074] The content recommendation generation engine 145 can map or otherwise correspond the determined user's learning level to the comprehensibility ranking signal generated by the signal generation engine 144 for the content items curated for the user. The content recommendation engine 145 can map the determined user's learning level to the content items curated by the engine 145 by identifying which learning attribute scores of the content items most closely align with the determined user's learning level. The content recommendation engine 145 can utilize various different methods to map the learning attribute scores to the user's learning level. In one implementation, those content items having the highest learning attribute scores matching the user's learning level are ranked above other content items. For example, if a user is determined to be at a third learning level, the content recommendation engine 145 can rank the content items having the highest learning attribute scores at the third learning level above other content items having lower learning attribute scores at the third learning level. As such, the content recommendation engine 145 is able to surface those content items that are deemed to be the most educational and most comprehensible to a user at the third learning level.

[0075] In one implementation, the learning content system 140 can continuously and / or periodically reevaluate the user's learning level in order to update the content curation provided to the user by the content recommendation engine 145. If the user's learning level improves to the next level (or regresses to a lower level), the content recommendation engine 145 can reevaluate the mapping of the understandability ranking signals of the content items being accessed by the user to the updated learning level of the user. As a result, the content recommendation engine 145 can present new and / or different content for the user and provide such content to the UI generation engine 146 for presentation to the user. This can be performed automatically without any user intervention, providing a seamless, progressive, and relevant experience in terms of content curation (with a learning application or with other services). As discussed herein, automatically can refer to performing a process without any manual input and / or user intervention from a client device or other device. Additionally, processing resources and network bandwidth are reduced as less communication occurs between the client devices 110A-110Z and the server 130 in order to present the most relevant, educational, and understandable content to the user.

[0076] In one implementation, content items can be automatically installed, uninstalled, and / or updated on a computing device based on content recommendations, as discussed above. For example, as a user's learning level improves to the next level, new content appropriate for the user's new level can be automatically installed on the user's computing device without user intervention. Similarly, old content that is no longer appropriate for the user's new learning level can be automatically uninstalled from the user's computing device without user intervention. As another example, content that is no longer appropriate for the user's new learning level can be automatically updated (e.g., by replacing the content with a different version, or by installing a plug-in) such that the updated content is appropriate for the user's new level. This can overcome or mitigate problems associated with manual installation, uninstallation, and updating of content on a computing device, such as the user having the appropriate security permissions on the client device and / or sufficient technical knowledge of how to install or uninstall content. Furthermore, by automatically uninstalling content that is no longer appropriate for the user's learning level, the limited data storage capacity of the computing device can be better utilized.

[0077] In one implementation, the specific use case of the understandability ranking signals generated by the signal generation engine 144 can utilize a learning tree 143. The learning tree 143 is a data structure that includes information related to educational topics. The learning tree 143 is a structure in which learning standards can be uniformly mapped to domains, subdomains, topics, and skills. The learning tree 143 can be structured as a tree graph with a root node and corresponding child nodes connected to each other via edges. As such, the learning tree 143 can be able to map from everything a person can learn in a standardized way, including educational topics as well as social and emotional skills.

[0078] Each node in the learning tree 143 can be drilled down into more detailed information about educational topics, where base nodes (i.e., bottom nodes) include an identification of content related to the base node. In the learning tree 143, base nodes can be organized according to a learning level, and can list conditions for mastering a particular topic, question, or skill at that learning level. For example, a base node corresponding to photosynthesis questions can be specific to a first learning level, while another base node corresponding to photosynthesis questions can be specific to a second learning level, and so on. The learning tree 143 can be utilized similar to a knowledge graph in order to map various educational topic areas and developmental milestones to a structured setting, and provide corresponding support content for the information given by the learning tree 143.

[0079] In one implementation, the content recommendation engine 145 can use the understanding ranking signal to place content in the learning tree 143. Content items can be placed in base nodes of the learning tree 143 that correspond to a learning level. Accordingly, the understanding ranking can be used to place particular content items into corresponding base nodes. For example, content items having their highest learning attribute score in a learning level "X" can be placed accordingly in a base node of the learning tree 143 associated with the learning level "X".

[0080] Once the content recommendation engine 145 identifies content items to recommend and / or curate, the identified content can then be provided to the UI generation module 146. The UI generation module 146 can associate formatting and other UI elements to generate a page or screen that displays the selected content as educationally and understandably relevant to the accessing user. As described above, the content can be displayed via the media viewer 111 on the client device 110A-110Z. In some implementations, there is optional human (e.g., human) review of the selected content before the selected content is displayed in the UI.

[0081] In further implementations, a learning interface can be generated and / or selected for display in the UI. The learning interface is discussed further below Figure 7 An example learning application home screen UI 700 is provided that provides content curated as educationally and understandably relevant to the accessing user. As described above, the learning application can be an application launcher on the client device that allows for streamlined application experiences for the user by automating the management of the client device and programs of the client device. For example, the application launcher can provide for the automatic downloading of applications without having to navigate to an application store, the automatic launching of applications, the automatic removal of applications, and / or ease of settings management (e.g., nested battery indicators, etc.) and account management (e.g., parental zones) for family members.

[0082] The description herein discusses content curation in identifying educational and understandable content items that are relevant to a user's learning level of a content sharing platform. Different types of content can be identified, including but not limited to individual content items (e.g., videos, apps, websites, books, etc.), playlists, and channels of the content sharing platform. In one implementation, the learning content system 140 230 curates content for display on a home screen of a learning dedicated application or on a webpage of a learning dedicated application. The learning dedicated application can be provided by a content sharing platform such as the content sharing platform 120 or by another type of platform. In other implementations, the learning content system module 140 can curate educational and understandable content for other purposes such as search results, recommendations, watch next / related content, etc.

[0083] Figure 2 is a flow diagram illustrating a method 200 for understanding-based identification of educational content for multiple content types, in accordance with implementations of the present disclosure. The method 200 can be performed by processing logic that comprises hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof. In one implementation, the method 200 can be performed by the learning content system 140 as shown in Figure 1

[0084] For the sake of explanation, the methods of the present application are depicted and described in terms of a series of actions. However, the acts of the present disclosure can occur in various orders and / or concurrently, and with other acts not presented and described herein. Additionally, not all illustrated acts can be required to implement a method in accordance with the present disclosure. Furthermore, those skilled in the art will recognize and appreciate that a methodology could alternatively be represented as a series of interrelated states utilizing a state diagram or event diagram. Additionally, it will be understood by those skilled in the art that the methods disclosed in the specification can be stored on an article of manufacture to facilitate transporting and transferring such methods to computing devices. The term article of manufacture as used herein is intended to encompass a computer program accessible from any computer-readable device or storage media.

[0085] ​Method 200 begins at block 210, where a processing device can determine respective understandability ranking signals for content items corresponding to a user request. The user request can be a search query, a request to navigate content in an application, or any other user interaction with a content provider platform (e.g., various social networking platforms, search engine platforms, media service platforms, online news platforms, content sharing platforms, etc.). In one implementation, the understandability ranking signals are based on learning attribute scores generated for the content items from at least one ML model. The learning attribute scores can correspond to the learning attributes of engagement, depth, spark, learning impact, and development appropriateness. In one implementation, the ML model can be used to generate a learning attribute score for each learning attribute at each learning level.

[0086] At block 220, the processing device can determine a learning level of the user corresponding to the user request. In one implementation, the learning level of the user can be based on one or more of a user-provided age, a user artifact, a learning level metric, a user profile indicating the learning level of the user, and / or a combination of the above factors, among other learning level determinations. For example, the learning level of the user can be determined based on the ML to the application of the artifact generated by the user (e.g., the user history within the learning-specific application) and / or other characteristics of the user (e.g., user age, grade level, etc.).

[0087] At block 230, the processing device can rank the content items based on a mapping between the learning level and the respective understandability ranking signals for the content items. In one implementation, the mapping can identify the learning attribute scores for the content items that most closely align with the determined learning level of the user. In some implementations, various other different methods can be used to map the learning attribute scores to the learning level of the user. For example, those content items with the highest learning attribute scores that are most relevant to the learning level of the user are ranked above other content items. As a result, those content items that are deemed to be most educational and most understandable to a user at a particular learning level can be surfaced to the user.

[0088] Finally, at block 240, the processing device can provide a recommendation for the content items according to the ranking of the content items. In one implementation, the content items for which the recommendation is provided can be generated as a page or screen of content that is relevant to the user educationally and understandability. For example, the page or screen can be an interface of a learning-specific application that the user is accessing.

[0089] Figure 3is a flowchart illustrating a method 300 for identifying a learning level of a user for understanding-based identification of educational content in a plurality of content types, in accordance with implementations of the present disclosure. The method 300 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof. In one implementation, the method 300 can be performed by the learning content system 140 as shown in Figure 1

[0090] The method 300 begins at block 310, where the processing device can identify a user corresponding to a user request. Then, at block 320, the processing device can identify user artifacts. In one implementation, the user artifacts include a user history of consumption of content items. At block 330, the processing device can identify user metrics that measure a learning level of the user. At block 340, the processing device can identify any manual user input of the learning level of the user in a configuration setting. Finally, at block 350, the processing device can determine the learning level of the user based on the identified user artifacts, user metrics, and manual user input.

[0091] Figure 4A and 4B are flowcharts illustrating methods 400, 450 for generating an understanding rank signal and using the understanding rank signal for understanding-based identification of educational content in a plurality of content types, in accordance with some implementations of the present disclosure. The methods 400, 450 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof. In one implementation, the methods 400 and 450 can be performed by the learning content system 140 as shown in Figure 1

[0092] Referring to Figure 4A ​​According to one embodiment of the present disclosure, the method 400 determines an understandability ranking signal. The method 400 begins at block 410, where the processing device can receive a request for an understandability ranking signal for a content item. At block 420, the method 400 performs blocks 422 and 424 for each learning level configured in the underlying system. At each learning level, the processing device can apply one or more ML models for each of a plurality of learning attributes to the content item at block 422. The one or more ML models generate a learning attribute score for each learning attribute for the corresponding learning level. Then, at block 424, the processing device combines the learning attribute scores generated from the ML models into a final understandability ranking signal for the content item for the corresponding learning level. As described above, in some embodiments, the learning attribute scores can be weighted according to desired outcomes for the type of content to be recommended (e.g., to recommend more “engaging” content or to recommend content that is more “difficult” to understand, etc.). In some embodiments, the learning attribute scores can not be combined, but can be utilized individually for purposes of ranking, recommending, and / or curation.

[0093] Once the understandability ranking signals for the content item for each learning level have been generated via blocks 420-424, the method 400 proceeds to block 430, where the processing device can return the generated individual learning attribute scores and the final understandability ranking signal for the content item for each learning level. Finally, at block 440, the processing device can store the generated individual learning attribute scores and the final understandability ranking signal for the content item for each learning level.

[0094] Referring to Figure 4B According to embodiments of the present disclosure, the method 450 utilizes the determined understandability ranking signals to place the content item in a learning tree. The method 450 begins at block 460, where the processing device can receive a request for an understandability ranking signal for a content item being placed in a learning tree. In one embodiment, the learning tree refers to a data structure that includes information related to educational topics. At block 470, the processing device can apply one or more ML models to the content item to generate an understandability ranking signal for the content item for each learning level. One embodiment, Figure 4A The method 400 can be used to generate an understandability ranking signal for a content item.

[0095] At block 480, the processing device can determine the generated highest understandability ranking signal for the content item among the learning levels. Finally, at block 490, the processing device can place the content item in the learning tree at a learning level corresponding to the determined learning level of block 480.

[0096] Figure 5is a flowchart illustrating a method 500 for training ML models for understanding-based identification of educational content in multiple content types in accordance with embodiments of the present disclosure. The method 500 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof. In one embodiment, the method 500 can be performed by the learning content system 140 as shown in Figure 1

[0097] The method 500 begins at block 510, where the processing device can obtain sample content items having metadata indicating an identified learning level of the content item and an educational topic fit (e.g., a subject area of the content item) for the content item. Subsequently, at block 520, the processing device can obtain manual ratings of the sample content items with respect to different learning attributes for each learning level configured for the underlying system. In one embodiment, the different learning attributes include, but are not limited to, appeal, depth, spark, learning impact, and development appropriateness.

[0098] At block 530, the processing device can train one or more ML models using the training data. In one embodiment, the training data includes the features of the metadata determined at block 510 and the labels of the manual ratings obtained at block 520. The one or more ML models are trained for each of the learning attributes including appeal, depth, spark, learning impact, and development appropriateness. Finally, at block 540, the processing device can update the trained ML models based on feedback received regarding the understanding-based ranking signals generated by the ML models.

[0099] Figure 6 is a flowchart illustrating a method 600 for progressive content recommendation updates for understanding-based identification of educational content in multiple content types in accordance with embodiments of the present disclosure. The method 600 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof. In one embodiment, the method 600 can be performed by the learning content system 140 as shown in Figure 1

[0100] Reference is made to Figure 6 ​​The method 600 begins at block 610, where the processing device can determine a respective understandability ranking signal for a content item corresponding to the user request. In one implementation, the understandability ranking signal is based on a learning attribute score generated from at least one ML model for the content item. The learning attribute score can correspond to the learning attributes of appeal, depth, spark, learning impact, and development appropriateness. In one implementation, the ML model can be utilized to generate the learning attribute score for each learning attribute at each learning level.

[0101] At block 620, the processing device can determine a learning level of the user corresponding to the user request. Subsequently, at block 630, the processing device can rank the content item based on a mapping between the learning level and the respective understandability ranking signal of the content item. In one implementation, the mapping can identify the learning attribute score of the content item that most closely matches the determined learning level of the user. At block 640, the processing device can provide a recommendation for the content item according to the ranking of the content item. In one implementation, the content item for which the recommendation is provided can be generated as a page or screen of content that is educationally and understandability relevant to the user. For example, the page or screen can be an interface of a learning dedicated application that the user is accessing.

[0102] Subsequently, at block 650, the processing device can identify a change in the learning level of the user to a new learning level. In one implementation, the learning level of the user can be periodically and / or continuously evaluated to determine whether the learning level has changed. At block 660, the processing device can update the recommendation for the content item based on a mapping between the new learning level and the respective understandability ranking signal of the content item. Finally, at block 670, the processing device can automatically update the recommendation for the content item according to the updated ranking of the content item from block 660. In one implementation, the automatic update of the recommendation includes updating the recommendation without any user intervention or manual input from the user via the client device.

[0103] Figure 7 FIGURE 8 illustrates an example screenshot of a learning application home screen UI 800 that provides understandability-based educational content suggestions, in accordance with implementations of the present disclosure. Figure 7 FIGURE 8 illustrates an example screenshot of a learning application home screen UI 800 that provides understandability-based educational content suggestions, in accordance with implementations of the present disclosure.

[0104] The learning application home screen UI 700 can include a navigation bar 730 running at the bottom of the UI 700, including a home icon 730, a book icon 732, a play icon 733, a video icon 740, and an activity icon 735. The navigation bar 730 can also include a status icon 740, which includes a favorites (e.g., star icon) and device status indicators (network connectivity, battery status, time, etc.). A search icon 705 is also displayed in the UI 700.

[0105] The learning application home screen UI 700 can also provide content recommendations in different sections of the UI 700. For example, a "Top pick" section 710 can display content icons 711-714. A "Recents" section 720 of the UI 700 can display content icons 721-726. The content icons 711-714, 721-716 can correspond to content items, such as but not limited to videos, apps, books, websites, channels, or playlists, etc. Content icon 711 is Top App 1, content icon 712 is Top Video 1, content icon 713 is Top Video 2, and content icon 714 is Top Book 1, content icon 721 is App 1. Content icon 722 is App 2, content icon 73 is Website 1, content item 724 is Website 2, content icon 725 is Book 1, and content icon 726 is Video 1.

[0106] If a user selects one or more of the content icons 711-714, 721-726, the content corresponding to the content icon can be caused to be downloaded to the underlying client device displaying the UI 700. In some implementations, selection of one or more of the content icons 711-714, 721-726 can cause the particular content corresponding to the content icon 711-714, 721-726 to be opened and displayed on the UI 700. In some implementations, if the content icon 711-714, 721-726 is interacted with in a predetermined manner (e.g., left click, two-finger selection, etc.), a quick options window 715 can be displayed, including options to save 716 and / or remove 717 the content icon 711-714, 721-726.

[0107] The content icons 711-714, 721-726 displayed for a particular section 710, 720 can be curated by the learning content system in accordance with the processes described above with respect to Figures 1 to 6 The content described above is educational and understandable content curated for the user by the learning content system. For example, as described above with respect to FIG. 6, the content icons 711-714, 721-726 displayed in the Top pick section 710 and the Recents section 720 can be curated by the learning content system in accordance with the processes described above with respect to FIG. 6. Figure 7As shown, content icons 711-714, 721-726 correspond to videos, applications, websites, books, channels, playlists, etc. that are curated for the user as educationally and intellectually relevant to the user that is accessing UI 700. Content icons 711-714 can be content items that are selected and ranked as the most matching to the learning level of the user that is accessing UI 700. Content icons 721-726 can be those content items that the user has recently accessed in the learning application.

[0108] As shown in UI 700, content icons 711-714, 721-726 can vary based on the type of content that the icons correspond to. For example, video content icons 712, 713, 726 are displayed with rounded edges, while other content icons of different content types (e.g., applications, books, websites, etc.) are displayed with square, angular edges. Different display options for content types different than those illustrated herein are possible and contemplated.

[0109] Figure 8 FIGURE 11 illustrates a schematic representation of a machine in the example form of a computer system 800 within which instructions, for causing the machine to perform any one or more of the methodologies discussed herein, can be executed. In alternative implementations, the machine can be connected (e.g., networked) to other machines in a Local Area Network (LAN), an intranet, an extranet, or the Internet. The machine can operate in the capacity of a server or a client machine in client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or other) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. In one implementation, the computer system 800 can represent a server, such as server 102, as described with respect to Figure 1 FIGURE 1, that executes a learning content system 140.

[0110] The example computer system 800 includes a processing device 802, a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 818, which communicate with each other via a bus 808. Any of the signals provided over various buses described herein can be time-multiplexed onto a single physical bus line, and the information can be provided via multiple physical bus lines, as can be appreciated by one of ordinary skill in the art. Additionally, the interconnection between circuit components or blocks can be replaced with a bus and the functions of the circuit components or blocks can be combined or divided into further circuit components or blocks. Each of the buses can be a single physical bus or a plurality of physical buses. In addition, each of the functions of the various entities described herein can be performed by multiple entities and so on.

[0111] The processing device 802 represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computer (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. The processing device 802 can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. The processing device 802 is configured to execute the processing logic 826 for performing the operations and steps discussed herein.

[0112] The computer system 800 can further include a network interface device 822. The computer system 800 also can include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generation device 820 (e.g., a speaker).

[0113] The data storage device 818 can include a computer-readable storage medium 824 (also referred to as machine-readable storage medium) on which is stored one or more sets of instructions 826 embodying any one or more of the methodologies of functions described herein (e.g., software). The instructions 826 can also reside, completely or at least partially, within the main memory 804 and / or within the processing device 802 during execution thereof by the computer system 800; the main memory 804 and the processing device 802 also constitute machine-readable storage media. The instructions 826 can further be transmitted or received over a network 874 via the network interface device 822.

[0114] The computer-readable storage medium 824 can also be used to store instructions to perform understanding-based recognition methods for educational content in a plurality of content types as described herein. While the computer-readable storage medium 824 is shown in an example embodiment to be a single medium, the term“computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). The machine-readable medium can include, but is not limited to, magnetic storage media (e.g., floppy disks); optical storage media (e.g., CD-ROMs); magneto-optical storage media; read-only memory (ROM); random-access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or another type of medium suitable for storing electronic instructions.

[0115] The foregoing description has provided a number of specific details for the purpose of providing a thorough understanding of several embodiments of the present disclosure as described in the specification and illustrated in the figures. However, it will be apparent to those skilled in the art that the specific details need not be implemented to practice at least some embodiments of the present disclosure. The specific details can, therefore, be subsumed by other specific details in other cases. No portion of the disclosure is intended to be dedicated to or reserved for any particular embodiment.

[0116] Reference throughout this specification to“an embodiment” or“embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase“in one embodiment” or“in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the term“or” is intended to mean an inclusive“or” rather than an exclusive“or” when used in this specification.

[0117] Although the operations of the methods herein are shown and described in a particular order, the order of the operations can be altered so that certain operations can be performed in an inverse order or so that certain operations can be performed, at least partially, concurrently with other operations. In another embodiment, instructions or sub-operations of distinct operations can be in an intermittent and / or alternating manner.

Claims

1. A method comprising: Using at least one machine learning model, a learning attribute score is generated for each of a plurality of first educational content items corresponding to a user's request for educational content, wherein the at least one machine learning model is trained using training inputs including a plurality of second educational content items and a target output of a learning attribute rating including the learning attributes of each of the plurality of second educational content items. The processing device combines the learning attribute scores of each first educational content item to determine a comprehension ranking signal for the corresponding first educational content item, wherein the comprehension ranking signal indicates the level of comprehension associated with the corresponding first educational content item; Determine the user's learning level corresponding to the user's request, wherein the user's learning level is determined in response to receiving the user's request for educational content; The plurality of first educational content items are ranked based on the mapping between the user's learning level and the corresponding comprehension ranking signals of the plurality of first educational content items; The processing device provides recommendations to be presented on the user interface (UI) of the user's device, the recommendations comprising a subset of the plurality of first educational content items identified according to the ranking of the plurality of first educational content items, wherein providing the recommendations includes automatically installing the subset of the plurality of first educational content items provided via the UI of the user device; Identify a change in the user's learning level to a new learning level, wherein the new learning level is determined in response to at least one of the following: receiving user input indicating the new learning level, or detecting a change in the user's level of interest in the plurality of first educational content items; Based on the mapping between the user's new learning level and the corresponding comprehension ranking signals of the plurality of first educational content items, a revised subset of the plurality of first educational content items is determined; and Automatically updating the UI of the user's device includes recommending revisions to the subset of revisions of the plurality of first educational content items, wherein automatically updating the UI of the user's device includes automatically unloading or automatically updating the subset of first educational content items provided via the UI of the user device using the subset of revisions of the plurality of first educational content items.

2. The method according to claim 1, wherein, The user request includes at least one of the following: a search query related to the educational content or navigation to a part of an application associated with the educational content.

3. The method according to claim 2, wherein, The application includes an application launcher on the home screen user interface of the client device.

4. The method according to claim 1, wherein, The user request corresponds to a topic that is relevant to the educational content and associated with the learning experience.

5. The method according to claim 1, wherein, The plurality of primary educational content items include at least one of the following: an application, a video, a book, or a webpage.

6. The method of claim 1, further comprising modifying the ranking of the plurality of first educational content items in response to recognizing the change in the user's learning level to the new learning level.

7. The method according to claim 1, wherein, The learning attribute scores correspond to two or more learning attributes, including attractiveness, depth, spark, learning impact, or developmental appropriateness.

8. The method according to claim 7, wherein, The at least one machine learning model includes a machine learning model for each learning attribute of each of the multiple learning levels.

9. The method according to claim 1, wherein, Combining the learning attribute scores includes applying weights to the learning attribute scores.

10. The method according to claim 1, wherein, The comprehension ranking signal is used to place at least one of the plurality of first educational content items in the learning tree.

11. The method according to claim 1, wherein, The user's learning level is based on at least one of the following: user artifacts, user metrics, or manual user input.

12. A system comprising: Memory; as well as A processing device coupled to the memory, wherein the processing device is used for: Using at least one machine learning model, a learning attribute score is generated for each of a plurality of first educational content items corresponding to a user's request for educational content, wherein the at least one machine learning model is trained using training inputs including a plurality of second educational content items and a target output of a learning attribute rating including the learning attributes of each of the plurality of second educational content items. The processing device combines the learning attribute scores of each first educational content item to determine a comprehension ranking signal for the corresponding first educational content item, wherein the comprehension ranking signal indicates the level of comprehension associated with the corresponding first educational content item; Determine the user's learning level corresponding to the user's request, wherein the user's learning level is determined in response to receiving the user's request for educational content; The plurality of first educational content items are ranked based on the mapping between the user's learning level and the corresponding comprehension ranking signals of the plurality of first educational content items; The processing device provides recommendations to be presented on the user interface (UI) of the user's device, the recommendations comprising a subset of the plurality of first educational content items identified according to the ranking of the plurality of first educational content items, wherein providing the recommendations includes automatically installing the subset of the plurality of first educational content items provided via the UI of the user device; Identify a change in the user's learning level to a new learning level, wherein the new learning level is determined in response to at least one of the following: receiving user input indicating the new learning level, or detecting a change in the user's level of interest in the plurality of first educational content items; Based on the mapping between the user's new learning level and the corresponding comprehension ranking signals of the plurality of first educational content items, a revised subset of the plurality of first educational content items is determined; and Automatically updating the UI of the user's device includes recommending revisions to the subset of revisions of the plurality of first educational content items, wherein automatically updating the UI of the user's device includes automatically unloading or automatically updating the subset of first educational content items provided via the UI of the user device using the subset of revisions of the plurality of first educational content items.

13. The system according to claim 12, wherein, The user request includes at least one of the following: a search query related to the educational content or navigation to a part of an application associated with the educational content.

14. The system according to claim 12, wherein, The plurality of primary educational content items include at least one of the following: an application, a video, a book, or a webpage.

15. The system according to claim 12, wherein, The at least one machine learning model includes a machine learning model for each of the multiple learning levels for each learning attribute, and wherein the learning attribute score corresponds to two or more learning attributes including attractiveness, depth, spark, learning impact, or developmental appropriateness.

16. The system according to claim 12, wherein, The user's learning level is based on at least one of the following: user artifacts, user metrics, or manual user input.

17. A non-transitory machine-readable storage medium storing instructions, said instructions, when executed, causing a processing device to perform operations, said operations including: Using at least one machine learning model, a learning attribute score is generated for each of a plurality of first educational content items corresponding to a user's request for educational content, wherein the at least one machine learning model is trained using training inputs including a plurality of second educational content items and a target output of a learning attribute rating including the learning attributes of each of the plurality of second educational content items. The processing device combines the learning attribute scores of each first educational content item to determine a comprehension ranking signal for the corresponding first educational content item, wherein the comprehension ranking signal indicates the level of comprehension associated with the corresponding first educational content item; Determine the user's learning level corresponding to the user's request, wherein the user's learning level is determined in response to receiving the user's request for educational content; The plurality of first educational content items are ranked based on the mapping between the user's learning level and the corresponding comprehension ranking signals of the plurality of first educational content items; The processing device provides recommendations to be presented on the user interface (UI) of the user's device, the recommendations comprising a subset of the plurality of first educational content items identified according to the ranking of the plurality of first educational content items, wherein providing the recommendations includes automatically installing the subset of the plurality of first educational content items provided via the UI of the user device; Identify a change in the user's learning level to a new learning level, wherein the new learning level is determined in response to at least one of the following: receiving user input indicating the new learning level, or detecting a change in the user's level of interest in the plurality of first educational content items; Based on the mapping between the user's new learning level and the corresponding comprehension ranking signals of the plurality of first educational content items, a revised subset of the plurality of first educational content items is determined; and Automatically updating the UI of the user's device includes recommending revisions to the subset of revisions of the plurality of first educational content items, wherein automatically updating the UI of the user's device includes automatically unloading or automatically updating the subset of first educational content items provided via the UI of the user device using the subset of revisions of the plurality of first educational content items.

18. The non-transitory machine-readable storage medium according to claim 17, wherein, The user request includes at least one of the following: a search query related to the educational content or navigation to a part of an application associated with the educational content.

19. The non-transitory machine-readable storage medium according to claim 17, wherein, The plurality of primary educational content items include at least one of the following: an application, a video, a book, or a webpage.

20. The non-transitory machine-readable storage medium of claim 17, the operation further comprising modifying the ranking of the content item in response to recognizing the change of the user's learning level to the new learning level.

21. The non-transitory machine-readable storage medium according to claim 17, wherein, The at least one machine learning model includes a machine learning model for each learning attribute for each of a plurality of learning levels, and wherein the learning attribute score corresponds to two or more learning attributes including attractiveness, depth, spark, learning impact, or developmental appropriateness.

Images