Casting media content on network-connected television devices

Abstract

To provide a method for presenting a unified user interface on a network-connected television device; a non-transitory computer-readable medium; and a network-connected television device.SOLUTION: A unified user interface includes a focus area for displaying a series of media content items (e.g., an advertisement) sequentially; and displays media content recommendations selected and organized based on knowledge of a user (e.g., search queries, search results, watch history, purchase history, physical activities). When playback of a media content item presented on the unified user interface involves a restricted mode, the playback relies on a collaborative implementation of a corresponding media player application and a media content casting application, . When the media player application does not operate in the restricted mode, the media content casting application is enabled to play the media content item at the restrict mode.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] This application generally relates to display technologies, including, but not limited to, methods and systems for providing content-based media recommendations, displaying an integrated user interface with content-based media recommendations, mixing a plurality of media content items displayed in a focus area, and / or casting controlled media content associated with a network-connected television device.

Background Art

[0002] Background Media display devices are used to stream various media content (such as TV programs, movies, live broadcasts, personal videos, etc.) via one or more communication networks. Media display devices include various connected display devices such as mobile phones, set-top boxes, or connected television devices. A connected television device is often controlled via a remote control interface that can be displayed directly on the media display device or separately on a second-screen mobile device (such as a tablet computer, laptop, or mobile phone). A user can browse a list of content items and search for media content on the user interface displayed on the television device while controlling the playback of the media content via a dedicated remote device or a second-screen mobile device (such as a tablet computer, laptop, or mobile phone). However, the list of content items is often compiled based on a media playback application configured to play the content items within the list.

Summary of the Invention

Means for Solving the Problems

[0003] Summary This application is directed toward presenting an integrated user interface with content-based media recommendations on a network-connected television (TV) device (sometimes called a smart TV). The integrated user interface is configured to display recommendations for multiple media content selected for a particular user, at least in part, based on knowledge of the user's interests and activities (e.g., the user's search queries, search results, previous viewing history, purchase history, and the user's physical activity). In one embodiment, the media content recommendations are organized based on one or more categories such as content type (e.g., "action movies"), viewing history (e.g., "I watched Movie ABC"), and release time (e.g., "trending"). In one embodiment, media content provided by different content sources (e.g., action movies from two different streaming services and broadcast TV channels) may be recommended within the same cluster.

[0004] In one embodiment, the integrated user interface includes a focus area (e.g., a prominently displayed area of ​​the user interface) configured to sequentially display a series of media content items (e.g., advertisements, movie trailers, product images) in chronological order for the purpose of promoting media content items, products, or events.

[0005] In one embodiment, the user starts playback of a content item from the integrated user interface by selecting an icon or thumbnail corresponding to the content item. In one embodiment, playback of media content items selected from an integrated user interface is handled by a dedicated application associated with a specific media streaming service or broadcast channel, a cable or wireless (OTA) tuner, a web browser, or an internal casting application associated with the connected TV that displays the user interface. In another embodiment, playback of content items is handled collaboratively by the media playback application and a local content casting application, particularly when playback occurs in a restricted mode not supported by the media playback application (for example, in the context of an age-restricted system such as a media playback interface customized for children). In one embodiment, where the integrated user interface provides both restricted and unrestricted modes, and only the media playback application supports unrestricted mode, the media playback application is used to play media content when the user interface is in unrestricted mode, and the local casting application is used to play media content items when the system is in restricted mode media playback.

[0006] In various embodiments, the integrated user interface is coupled to a server system that hosts a virtual user domain containing user accounts associated with users of the user interface. Hardware devices and software applications are integrated with each other through user accounts within the virtual user domain. In one embodiment, a user account is associated with a network-connected television device, and the integrated media application is configured to present the integrated user interface on the television device. The user account may also be associated with one or more home devices (e.g., a camera, doorbell, and / or a hazard detector), one or more media playback devices (e.g., a set-top box and / or an associated television device), one or more assistant devices (e.g., a speaker assistant and / or a display assistant device), and their corresponding home security, media playback, and assistant applications. The user account may also be associated with common user applications, such as email applications, social media applications, mapping applications, and online retail store applications.

[0007] In one embodiment, to the extent authorized by the user, the server system learns about the user's interests and activities from information reported by various applications associated with the user's account, and from this information generates recommendations for media content relevant to that user. These recommendations are displayed on an integrated user interface of a network-connected television device, which is communicably coupled to the server system, third-party content providers, electronic program guide (EPG) providers, and advertising providers via one or more communication networks.

[0008] Specifically, in one embodiment, a method for identifying media content items to be presented on a network-connected television device is implemented on a server system hosting a virtual user domain containing user accounts. This method includes associating user accounts with an integrated TV application, multiple media playback applications, and multiple user applications. The integrated TV application is associated with a network-connected television device, and the multiple user applications include at least an internet search engine. This method includes storing activity data for each of the integrated TV application, media playback applications, and user applications, and such storage includes storing search queries and search results for multiple search operations performed by the internet search engine from the user account. The method includes storing viewing information associated with a user account for a subset of media playback applications, and storing purchase information associated with a user account for a subset of user applications. The method further includes calculating multidimensional integrated user activity characteristics from the stored activity data, and obtaining program information associated with multiple media playback applications, thereby obtaining deduplicated program information for multiple unique media programs. The method further includes identifying ordered subsets of the unique programs according to the user activity characteristics and the program information for the multiple unique programs, and enabling the display of an ordered sequence of information items corresponding to the ordered subsets of the unique programs via an integrated TV application. Each information item represents a unique program provided by the corresponding media playback application.

[0009] In another embodiment, the program presentation method is implemented on a network-connected television device linked to a user account in a virtual user domain hosted by a server system. The program presentation method includes running an integrated TV application to display an integrated user interface and retrieving a media recommendation stream from the server containing an ordered sequence of selectable information items corresponding to a unique media content item or ordered set of programs. For each selectable information item, the media recommendation stream contains the respective commands to be executed in response to a user selection of that selectable information item on the network-connected television device. The program presentation method further includes displaying multiple clusters on the integrated user interface. Each cluster contains a subset of the ordered sequence of selectable information items, the first cluster containing a first selectable information item and a second selectable information item that are simultaneously displayed on the integrated user interface. The program presentation method further includes displaying information about a first media content item provided by a first content provider in response to a first user selection of the first selectable information item. This program presentation method further includes displaying information about a second media content item provided by a second content provider in response to a second user selection of a second selectable information item. The second content provider is different from the first content provider.

[0010] In yet another embodiment, a method for presenting selected content provided by a media playback application is implemented on a network-connected television device linked to a user account in a virtual user domain hosted by a server system. This method includes running an integrated TV application to enable the display of an integrated user interface on the network-connected television device. The integrated TV application has a normal mode and a restricted mode and is associated with one or more standard media playback applications and a local content casting application. This method further includes displaying multiple clusters on the integrated user interface, each having multiple selectable information items, and in restricted mode, includes identifying content selection criteria and selecting multiple selectable information items from program information associated with one or more standard media playback applications according to the content selection criteria. This method further includes receiving user action on a first selectable information item, the first selectable information item corresponding to a first media content item provided by a first content provider in a first media playback application. This method, in response to a user action on a first selectable information item, displays a first media content item corresponding to the first selectable information item on a network-connected television device via a local content casting application, according to the determination that the integrated TV application operates in restricted mode and the determination that the first media playback application does not operate in restricted mode, and according to the determination that the integrated TV application operates in normal mode. The further includes displaying a first media content item corresponding to a first selectable information item on a network-connected television device via a first media playback application.

[0011] In yet another embodiment, the method for displaying focus content is implemented in a server system hosting a virtual user domain containing user accounts. This method includes associating the user accounts with an integrated TV application and multiple media playback applications. The integrated TV application is associated with a network-connected television device and configured to enable the display of media content on the television device, and each of the multiple media playback applications is configured to display media content provided by its respective content provider. This method further includes running the integrated TV application to enable the display of an integrated user interface on the network-connected television device. The integrated user interface has a focus area configured to display multiple focus items. This method further includes sequentially displaying the multiple focus items within the focus area in chronological order. Each focus item is associated with its respective focus item provider selected from the sales module, advertisers, and media recommendation engine. The multiple focus items include a first focus item. More specifically, this method includes sending a request to a first advertiser server for a first focus item, the request including information on several predetermined user interface elements of the focus area of ​​the integrated user interface; receiving several media content items associated with the first focus item from the first advertiser server in response to the request; and integrating the several media content items into the first focus item for display on the focus area during the time slot corresponding to the first focus item.

[0012] A non-temporary computer-readable medium has stored instructions, and when these instructions are executed by one or more processors, they cause those processors to perform one of the methods described above. A computer system includes one or more processors and memory having stored instructions, and when these instructions are executed by one or more processors, they cause those processors to perform one of the methods described above.

[0013] To better understand the various embodiments described below, refer to the following descriptions of embodiments in conjunction with the following drawings, and throughout the drawings, similar reference numbers refer to corresponding parts. [Brief explanation of the drawing]

[0014] [Figure 1] An exemplary media environment, according to one embodiment, in which a network-connected television device, a server system, and a third-party content source interact with each other through one or more communications. [Figure 2] This is an exemplary operating environment in which a server system 104 provides knowledge-based media content recommendations to a network-connected television device, according to one embodiment. [Figure 3] This is an exemplary integrated user interface for display on a network-connected television device, according to one embodiment. [Figure 4] This is an illustrative flowchart of a method for providing knowledge-based media content recommendations to a network-connected television device, according to one embodiment. [Figure 5] This is an illustrative flowchart of a method for presenting program information on a network-connected television device according to one embodiment. [Figure 6A] These are two exemplary media content pages displayed on a network-connected television device, according to one embodiment. [Figure 6B]These are two exemplary media content pages displayed on a network-connected television device, according to one embodiment. [Figure 7] This is an exemplary operating environment in which an integrated TV application and a media playback application collaborate to play media content items on a network-connected television device 12, according to one embodiment. [Figure 8] This is an exemplary sequence 800 of a user interface that is sequentially displayed on a network-connected television device in response to user actions on a remote control device coupled to the network-connected television device, according to one embodiment. [Figure 9] An exemplary operating environment in which an integrated TV application, a media playback application, and a content casting application for a network-connected television device cooperate to enable playback of media content items, according to one embodiment. [Figure 10A] This is an exemplary integrated user interface displayed on a network-connected television device in restricted mode, according to one embodiment. [Figure 10B] This is an exemplary sequence of user interfaces displayed sequentially on a network-connected television device in response to user actions on a remote control device coupled to the network-connected television device, according to one embodiment. [Figure 10C] This is an exemplary sequence of user interfaces displayed sequentially on a network-connected television device in response to user actions on a remote control device coupled to the network-connected television device, according to one embodiment. [Figure 11] This is an illustrative flowchart of a method for presenting selected media content, provided by a media playback application on a network-connected television device, according to one embodiment. [Figure 12]An exemplary operating environment in which focus items (also referred to as tiles) provided by a recommendation engine, one or more advertising providers, and a sales module according to an embodiment are mixed for display on a network-connected television device. [Figure 13] An exemplary flowchart of a method for mixing focus content (including VAST-based advertisements) on a network-connected television device according to an embodiment. [Figure 14A] A block diagram showing a server system of a media environment according to an embodiment. [Figure 14B] An exemplary device and application registry of a server system that stores information of one or more user accounts managed by the server system and information of user devices and applications associated with each of the one or more user accounts according to an embodiment. [Figure 15] A block diagram showing an exemplary network-connected television device configured to display an integrated user interface and media content provided by a media playback application according to an embodiment.

Best Mode for Carrying Out the Invention

[0015] Throughout several views of the drawings, like reference numerals refer to corresponding parts.

[0016] Description of Embodiments In various embodiments, the media environment includes a network-connected television device (also called a smart television device) having one or more processors and memory for storing instructions to be executed by the one or more processors. The network-connected television device is linked to a user account in a virtual user domain hosted by a server system. The network-connected television device runs an integrated TV application to display an integrated user interface in which media content provided by various media content providers is selected and displayed for the user of the network-connected television device. In some embodiments, the network-connected The television system is controlled by a dedicated remote control unit or a client device having a remote control application for playing media content items for the user. In one embodiment, the dedicated remote control unit is located close to the television system. In another embodiment, the client device having the dedicated remote control unit or remote control application is not physically close to the television system but is associated with a user account maintained in a server system of the media environment.

[0017] When a network-connected television system runs the integrated TV application, it retrieves a media recommendation stream from a server system. The media recommendation stream contains an ordered sequence of selectable information items corresponding to an ordered set of unique media content items or programs. The selectable information items within the media recommendation stream are identified for a particular user, at least in part, based on the user's knowledge of the network-connected television system (e.g., the user's interests and activities). Specifically, a user account associated with the integrated TV application is also associated with multiple media playback applications and multiple user applications (e.g., search engine, map application). Activity data from these applications is stored in association with the user account and used to calculate the user's multidimensional integrated user activity characteristics (also known as a knowledge graph). According to these user activity characteristics, the ordered sequence of selectable information items is determined from program information associated with multiple media playback applications and recommended to the user via the integrated TV application.

[0018] An ordered sequence of selectable information items is displayed on the integrated user interface of the network-connected television device. For each selectable information item, the media recommendation stream includes the respective commands executed in response to the user selection of each selectable information item on the network-connected television device. The selectable information items are organized into clusters and arranged without referencing their content providers. Thus, two information items associated with media content items provided by separate content providers can be placed adjacent to each other, thereby allowing the integrated user interface to focus on the content and become independent of the content provider in that regard.

[0019] The integrated user interface displays multiple focus items sequentially in a focus area in a chronological order, in addition to selectable information items, that is, each of the multiple focus items sequentially occupies the entire focus area. The focus area may be located in the upper half of the integrated user interface. Each focus item is associated with a respective focus item provider selected from the sales module, advertisers, and media recommendation engine. Focus items provided by the sales module of the server system optionally correspond to media content items provided by TVOD content providers that require payment for review. In one embodiment, the focus item is an advertising item provided by an advertiser, and the advertising item conforms to a customized video ad serving template (VAST). Specifically, the television device sends a request for a focus item to the advertiser, which includes information on predetermined user interface elements in the focus area. The advertising item is aggregated from multiple media content items provided by the advertiser according to the information on predetermined user interface elements.

[0020] In one embodiment, the integrated TV application has a restriction mode in which content selection criteria are applied to restrict media content provided via a standard media content application for playback on the integrated user interface. For example, the content selection criteria may be related to age-based content ratings, and the restriction mode The system is defined as kid-friendly. In restricted mode, the integrated TV application arbitrarily determines whether the standard media content application operates in restricted mode. If the standard media content application is determined to operate in restricted mode, the media content is displayed on the integrated user interface via the standard media content application. Alternatively, if the third-party media content application is determined not to operate in restricted mode, the media content is displayed on the integrated user interface via a local content casting application different from the standard media content application.

[0021] Examples of embodiments are shown in the accompanying drawings and will be described in detail with reference to those embodiments. The following detailed description will provide numerous specific details to give a complete understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments can be carried out without these specific details. In other examples, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0022] Figure 1 shows an exemplary media environment 100 according to one embodiment, in which a network-connected television device 102, a server system 104, and third-party content sources 106-110 interact with each other via one or more communication networks 112. The media environment 100 corresponds to a virtual user domain created and hosted by the server system 104, which includes multiple user accounts. The third-party content sources 106-110 include an advertising source 106, an EPG source 108, and a media content source 110. For each user account, the server system 104 is coupled to the third-party content sources 106-110 and one or more media devices 102, 116-126, and is configured to recommend and stream media content for user review via the respective user account.

[0023] Specifically, one or more media devices associated with a user and user account are located in a media environment 100 and provide the user with media content stored in and streamed from a remote content source. The remote content source is optionally a third-party media content source 110 or an internal media source hosted by a server system 104. In one embodiment, one or more media devices include a networked television device 102 that integrates an embedded casting unit configured to stream media content directly from the remote content source or to stream media content for display to its audience. The networked television device 102 is communicably coupled to a dedicated remote control device 114 and / or a client device having a remote control application. The dedicated remote control device 114 may be located in close proximity to the television device 102 and may be configured to communicate with the television device 102 using digitally encoded pulses of infrared signals. Alternatively, in certain situations, a dedicated remote control device 114 may be configured to communicate with the television device 102 via a communication network 112 (i.e., via a local wide area network and / or a wide area network), and it may not be physically close to the television device 102.

[0024] The network-connected television device 102 includes one or more processors and memory for storing instructions to be executed by the one or more processors. The instructions stored in the network-connected television device 102 include an integrated TV application, a local content casting application, and third-party code. This includes one or more media playback applications associated with content sources 106-110, and / or internal media sources associated with server system 104. These applications are linked to user accounts within the virtual user domain of media environment 100.

[0025] Alternatively, in one embodiment, the media device located in the media environment 100 includes a display device 116 that outputs media content directly to the viewer and a casting device 118 that is coupled and configured to stream the media content to the display device 116. Examples of the display device 116 include, but are not limited to, television (TV) displays and music players. Examples of the casting device 118 include, but are not limited to, set-top boxes (STBs), DVD players, and TV boxes. In the example shown in Figure 1, the display device 116 includes a TV display device wired to a DVD player or set-top box 118. In contrast, in one embodiment, the media device located in the media environment 100 includes a computer screen 120A that outputs media content to the viewer and a desktop computer 120B that streams the media content to the computer screen 120A. In one embodiment, the media device located in the media environment 100 includes mobile devices, such as a mobile phone 122, a tablet computer 124, and a laptop computer 126. Each of the media devices 118 to 126 includes one or more media playback applications configured to receive and play media content items provided by third-party content sources 106 to 110 or internal media sources associated with the server system 104.

[0026] The server system 104 includes an integrated media platform (UMP) 128 configured to manage the recommendation and streaming distribution of media content to one or more media devices within the media environment 100. The UMP 128 is configured to receive multidimensional integrated user activity characteristics associated with a user account and program information of programs or content provided by media content sources 110 from a knowledge engine 130, and to generate personalized media content recommendations for the user account. The integrated user activity characteristics associated with a user account are obtained from multiple information sources 132, e.g., a search engine, a mapping application, and an online retail store associated with the user account, each of which provides activity data associated with the user account's activities through its respective software program or application. Optionally, the media content recommendations generated by the UMP 128 are presented on a network-connected television device 102 via a server-side TV application 134, which enables the display of media content on the integrated TV application on the television device 102 in response to a user selection from the media content recommendations. Furthermore, UMP128 may also function as a centralized media content management module configured to provide media content recommendations to other media devices 118-126, in addition to the television device 102.

[0027] In one embodiment, activity data associated with a user account is collected from the TV application 134 and the casting service module 136 (which are therefore part of the information source 132) and returned to the knowledge graph engine 130 to update the integrated user activity characteristics and media content recommendations associated with the user account.

[0028] In one embodiment, the UMP128 receives a search query from the server-side TV application 134 or the casting service module 136 and retrieves one or more program information from the program or content item provided by the media content source 110. This also includes a media search engine configured to identify media content items (for example, engine 222 in Figure 2).

[0029] It should be noted that user accounts in the virtual user domain hosted by the server system 104 are also associated with one or more other types of devices, such as intelligent, multisensing, network-connected auxiliary devices 138 and home devices 140 installed in the media environment 100 associated with the user of the network-connected television device 102. Examples of auxiliary devices 138 include a speaker auxiliary device 142 and a display auxiliary device 144. The speaker auxiliary device 142 can collect audio input, recognize user commands from the audio input, and perform actions in response to user commands (e.g., playing music, answering questions). The display auxiliary device 144 can collect audio and / or video input, recognize user commands from the audio and / or video input, and perform actions in response to user commands (e.g., playing music, displaying images or video clips, answering questions). Examples of home devices 140 include one or more intelligent, multi-sensing, and networked hazard detectors 146, camera systems 148, thermostats, accessway interface devices (e.g., smart doorbells and smart door locks), alarm systems, microphone devices, wall switches, wall plug interfaces, and appliances (e.g., refrigerators, stoves, ovens, televisions, washing machines, dryers, lighting, stereos, intercom systems, garage door openers, floor fans, ceiling fans, wall air conditioning units, pool heaters, irrigation systems, security systems, space heaters, window AC units, and electric duct vents).

[0030] Each of the auxiliary devices 138 and home devices 140 is optionally managed by an integrated home device application or a dedicated user application and is linked to a user account in a virtual domain in conjunction with the integrated TV application of a network-connected television device. The activity data collected by the knowledge graph engine 130 may also include data collected by a subset of the auxiliary devices 138 and home devices 140, thereby enabling recommendations for media content provided to the user to reflect this activity data collected directly from the physical surroundings associated with the media environment 100.

[0031] In one embodiment, the server system 104 includes a device and application registry 150 configured to store information about one or more user accounts managed by the server system 104 and information about user devices and applications associated with each of the one or more user accounts. For example, the device and application registry 150 stores information about a network-connected television device 102, a remote control device 114, media devices 116-126, an auxiliary device 138, and a home device 140, as well as information about the corresponding integrated TV application, remote control application, media playback application, integrated home device application, and dedicated user applications associated with the auxiliary device 138 and the home device 140. Optionally, these media devices, auxiliary devices, and home devices associated with the same user account are distributed across different geographical areas. Optionally, these devices are located in the same physical location. Each media, auxiliary, or home appliance communicates with another device or the server system 104 using one or more communication networks 112. The communication network 112 used may include, but is not limited to, the Internet, intranet, local area network (LAN), cellular network, Ethernet®, storage area network (SAN), telephone network, Bluetooth® personal area network (PAN), etc., and may be one or more networks having one or more types of topology. In one embodiment, two or more devices within a subnetwork are connected via a wired connection, while at least some of the devices within the same subnetwork are connected via a local wireless communication network (e.g., ZigBee, Z-Wave, Insteon, Bluetooth, Wi-Fi, and other wireless communication networks).

[0032] Figure 2 shows an exemplary operating environment 200 according to one embodiment, in which a server system 104 provides knowledge-based media content recommendations to a network-connected television device 102. In the exemplary operating environment 200, the server system 104 includes multiple information sources 132, a knowledge graph engine 130, a UMP 128, and a server-side TV application 134. The server system 104 hosts a virtual user domain that includes user accounts associated with the server-side TV application 134 and an integrated TV application 202 that runs on the network-connected television device 102. The user accounts are also associated with multiple media playback applications 204 and multiple user applications, such as a search engine 206. The server system 104 stores activity data for each of the integrated TV application 202, the media playback applications 204, and the user applications. The knowledge graph engine 130 of the server system 104 calculates multidimensional integrated user activity characteristics 208 (for example, regarding user preferences or similarities) from stored activity data, retrieves program information associated with multiple media playback applications 204, and as a result obtains deduplicated program information 210 for multiple unique programs (also called media content items). The UMP 128 includes a recommendation engine 212 configured to compare the user activity characteristics 208 and the program information 210 for multiple unique programs in order to identify an ordered subset of unique programs 214 for recommendation to the user via the integrated TV application 202.

[0033] In one example, the knowledge graph engine 130 receives several search queries related to "Star Wars" from the search engine 206 and determines that the user associated with the user account is interested in "Star Wars." Multiple unique programs associated with program information are ranked based on the user's interests, preferences, or similarities (i.e., represented by the integrated user activity characteristics 208). A certain number of top-ranked unique programs are selected and categorized into clusters. Given the user's interest in "Star Wars" as indicated by the search queries, the top-ranked unique programs include one or more media content items related to "Star Wars," and the clusters include clusters associated with "Star Wars," such as "Science Fiction Movies Like Star Wars."

[0034] An ordered subset of unique programs 214 identified by the recommendation engine 212 is represented by an ordered sequence of information items, which is provided to the server-side TV application 134 and made available for display via the integrated TV application 202 on the network-connected television device 102. In some situations, the activity data used by the knowledge graph engine 130 and the resulting integrated user activity characteristics 208 are constantly changing. The ordered subset of unique programs 214 can be identified dynamically. For example, the ordered subset of unique programs 214 may be updated periodically or in response to a request from a user of the network-connected television device 102. In some situations, each unique program within the ordered subset 214 has a lifetime determined based on the respective content type of each unique program, and must be removed from the recommended ordered program after its lifetime expires. In one embodiment, the integrated user activity characteristics 208 vary depending on the time of day and day of the week, as well as an ordered subset of unique programs 214 recommended via the integrated TV application 202.

[0035] Multiple user applications act as multiple information sources 132 for providing user activity data of the television device 102, and include at least an internet search engine 206. The server system 104 stores search queries and search results for multiple search operations performed by the internet search engine 206 from the user account and provides them to the knowledge graph engine 130. Furthermore, in some embodiments, the server system 104 stores viewing information associated with the user account for a subset or all of the media playback applications 204, and purchase information associated with the user account for a subset of user applications (e.g., an online retailer application). Such viewing and purchase information, along with search queries and search results, is provided to the knowledge graph engine 130. In some embodiments, the multiple user applications include audio or video assistance applications 218 associated with speaker assistance devices 142 or display assistance devices 144. In some embodiments, the multiple user applications include integrated home device applications and / or dedicated home device applications 220 associated with one or more home devices 140. Activity data collected by these auxiliary applications 218 and home device applications 220 is also stored and provided to the knowledge graph engine 130.

[0036] Optionally, one of the media playback applications 204 is provided by the server system 104 itself. Optionally, the media playback application 204 is provided by a separate and / or remote third-party media content source 110. In one example, the virtual user domain is associated with a Google® user account (e.g., abc@gmail.com). Translation 204 is provided by Google Inc. and YouTube®, a registered trademark of Google Inc. This service is not provided by Google, but is hosted independently by other media playback applications (for example, Netflix, Amazon Prime, Hulu). (Hulu) and others are included.

[0037] In one embodiment, program information associated with multiple media playback applications 204 includes one or more of the following: schedule information, a short plot summary, casting information, rating information, and comments on current or scheduled programs that are available or will become available in each of the media playback applications. Optionally, program information is provided to the server system 104 in a program catalog by each of the third-party media content sources 110 associated with the multiple media playback applications 204. Optionally, program information is provided to the server system 104 by an EPG source 108. Alternatively, in one embodiment, program information associated with multiple media playback applications 204 includes public media information downloaded from a public database independent of user accounts. Note that program information associated with media playback applications may be provided to the server system 104 regardless of whether user accounts associated with the television device 102 and the integrated TV application 202 subscribe to the media playback applications and have access rights to the media content provided through those media playback applications.

[0038] In one embodiment, program information provided by different media content sources 110 for the same media content item may not be identical and may vary to different levels, thereby requiring duplication of program information associated with multiple media playback applications. Specifically, in one embodiment, a first media content item is provided by a first media playback application and associated with first program information. A second media content item is provided by a second media playback application. The first program information is associated with the second program information within the second media playback application. The second program information is different from the first program information. The knowledge graph engine 130 identifies the similarity level of the first and second program information and determines that the similarity level exceeds a similarity threshold. In accordance with this determination, the knowledge graph engine 130 aggregates the first program information of the first media content item and the second program information of the second media content item by deduplication and determines that one of several unique programs is associated with both the first and second media content items. In other words, the first and second media content items provided by two separate media playback applications correspond to the same unique program or media content item. Furthermore, in one embodiment, the knowledge graph engine 130 identifies the similarity level of the first and second program information by identifying a first keyword from the first program information and a second keyword from the second program information. The first and second keywords are semantically compared to identify the similarity between the first and second program information items. For example, two program information items, "police" and "inspector," are generally consistent but semantically similar, citing different keywords. These two program information items correspond to a relatively high similarity, above the similarity threshold.

[0039] In one embodiment, in addition to deduplication, program information associated with multiple media playback applications is processed to calculate program characteristics for one of the unique programs. Multidimensional integrated user activity characteristics are then compared to the calculated program characteristics to arrive at a consistency score. The recommendation engine 212 determines whether the consistency score exceeds a consistency threshold, and if so, is configured to identify one of the unique programs that should be recommended to the user via the integrated TV application. Alternatively, in one embodiment, to identify each of an ordered subset of unique programs, the recommendation engine 212 identifies one or more keywords based on the multidimensional integrated user activity characteristics and searches for one or more keywords in the deduplication program information of the multiple unique programs to identify each unique program within the ordered subset of unique programs.

[0040] In one embodiment, the recommendation engine 212 receives credentials for a subset of multiple media playback applications. For each ordered subset of unique programs, the recommendation engine 212 determines that each unique program corresponds to a media playback application to which a user of a network-connected television device has credential-based access. According to the credentials, a user may have access to each unique program because they are subscribed to the respective media playback application, or because each unique program can be freely reviewed regardless of whether they are signed up. Each unique program that can be freely reviewed includes, for example, media content that is freely available in the air via broadcast services or IPTV distribution feeds, and media content that is made available by local regulations or credentials.

[0041] In other words, personal preferences, as indicated by multidimensional integrated user activity characteristics, are used to select a set of unique programs from the unique programs provided by the media content source 110 and to rank the selected set of unique programs. The selected set of unique programs is further filtered according to rights information indicating whether the user account has rights (e.g., access rights) to the media content provided by the media content source 110 via the corresponding media playback application 204. This credentials associated with the media playback application are optionally entered by the user of the integrated TV application during the initial application personalization process or the execution phase of the integrated TV application 202. Optionally, this credentials associated with the media playback application are automatically extracted from the media playback application.

[0042] In one embodiment, the UMP128 also includes a media search engine 222 configured to receive search queries from a server-side TV application 134 and to identify one or more media content items from program information 210 of a program or content item provided via a media playback application 204.

[0043] The server system 104 includes one or more processors and memory for storing instructions to be executed by the one or more processors. In one embodiment, the server system 104 includes a single server computer configured to implement the functions of a knowledge graph engine 130, a recommendation engine 212, a media search engine 222, and a server-side TV application 134. Alternatively, in one embodiment, the server system 104 includes two or more separate server computers, each of which is configured to implement one or more functions of the knowledge graph engine 130, the recommendation engine 212, the media search engine 222, and the server-side TV application 134.

[0044] Figure 3 shows an exemplary integrated user interface 300 for display on a network-connected television device 102, according to one embodiment. The integrated TV application 202 runs locally on the network-connected television device 102 and displays the integrated user interface 300. As described above with reference to Figure 2, the server system 104 identifies an ordered subset of unique programs 214 based on user activity characteristics 208 and program information 210 for a plurality of unique programs. The integrated TV application 202 retrieves a media recommendation stream from the server system 104, which includes an ordered sequence of selectable information items 302-332 corresponding to the identified ordered subset of unique media content items or programs 214. For each selectable information item, the media recommendation stream includes a corresponding command that is executed in response to the user selection of each selectable information item displayed on the network-connected television device 102.

[0045] In one embodiment, the corresponding ordered sequence of selectable information items is classified into an ordered list of clusters 344-340. That is, multiple clusters 334-340 are created for display on the integrated user interface 300 and physically arranged according to a predetermined cluster order. Optionally, these clusters are dynamically created and arranged based on multidimensional integrated user activity characteristics 208. In one embodiment, the clusters recommended via the integrated TV application 202 have a certain level of heterogeneity, i.e., have several different clusters greater than a predetermined number. For example, three or more different clusters must be recommended via the integrated TV application 202. Alternatively, in one embodiment, the clusters displayed on the integrated user interface 300 have a similarity score controlled to be below an inter-cluster similarity threshold for several adjacent clusters. This is to avoid similar clusters or information items being placed adjacent to each other in highly relevant clusters. In one embodiment, a benchmark is established to define an inter-cluster similarity threshold across the number of adjacent clusters.

[0046] Each cluster contains one or more information items in an ordered sequence, organized into an ordered sublist of information items within each cluster. For each cluster (e.g., the first cluster 338), each subset of the ordered sequence of selectable information items (e.g., information items 318-324) is physically arranged according to a predetermined order of information items determined based on the multidimensional integrated user activity characteristics 208 of the user account associated with the ordered sequence of selectable information items. It can be done.

[0047] Examples of clusters shown in the integrated user interface 300 include "New Releases" 334, "Next Playback" 336, "Currently Trending" 338, "I just watched Movie ABC" 340, and "Action Movies." In another example, a cluster is dedicated to a single media playback application (e.g., YouTube), and the information items associated with that cluster are... A subset of these clusters corresponds to a mix of shows, movies, and TV shoulder content provided by a single media playback application. Thereafter, each cluster corresponds to a corresponding ordered sublist of information items and is defined according to one or more categories: namely, content type (e.g., "action movies"), viewing history (e.g., because I watched Movie ABC), release date (e.g., trending), or content source (e.g., different streaming services and broadcast TV channels offering action movies). In particular, information items in the "New Arrivals" cluster are determined based on one or more of the following associated with the media content item or program corresponding to the "New Arrivals" information item: content type / format, content quality, newness, popularity, seasonality, and application lifecycle.

[0048] In one embodiment, media content items corresponding to information items within a "New Arrivals" or "Trending" cluster are provided by a transactional video-on-demand (TVOD) content provider, and the selection of media content items in "New Arrivals" or "Trending" is configured such that a user interface prompted by the user of the television device 102 allows for the collection of payment for reviewing the media content items. Optionally, this "New Arrivals" or "Trending" cluster is provided as part of an ordered sequence of information items corresponding to an ordered subset of unique programs 214 determined based on integrated user activity characteristics 208. Optionally, this "New Arrivals" or "Trending" cluster is provided to all user accounts as a supplement to the ordered sequence of information items corresponding to an ordered subset of unique programs 214, independently of the integrated user activity characteristics 208 of individual user accounts.

[0049] In one embodiment, media content items corresponding to information items in the "Action Movies" cluster are provided by a subscription video-on-demand (SVOD) content provider. The selection of a media content item in this cluster is configured to activate the page of the media content item (also known as the summary page, e.g., page 600 in Figure 6A) or to start displaying the media content item without prompting immediate payment. Furthermore, in one embodiment, media content items corresponding to information items in the "Play Next" cluster are optionally provided by an SVOD or TVOD content provider. In response to the selection of a media content item in this "Play Next" cluster, the television device 102 resumes displaying the media content item, enabling the user of the television device 102 to have one-click access to continue watching their favorite show or movie.

[0050] For each cluster on the integrated user interface 300, some or all of the ordered sublists of information items are displayed according to the total number of information items in each sublist, the size of each information item, and the available display space associated with each cluster. In one embodiment, each ordered subset of a unique program 214 is different from any other unique program in the ordered subset of the unique program 214. Any two of the information items 302-332 displayed on the integrated user interface 300 are different from each other. In some situations, the same cluster on the integrated user interface 300 Any two of the information items 302–332 displayed are different from each other. Alternatively, in some situations, two of the information items 302–332 may be identical but recommended in different clusters; for example, information item 310 associated with "Downton Abbey" is shown in both the "Next Play" 336 cluster and the "Drama" cluster.

[0051] In one embodiment, the integrated user interface 300 includes an affordance item 342 distinct from an ordered list of clusters, and a corresponding selectable information item displayed therein. In response to a user action on the affordance item 342 (e.g., selection of item 342 via the remote control device 114), the integrated TV application 202 updates the integrated user interface 300 on a page of information items, each representing a respective media playback application 204. In an embodiment not shown in Figure 3, the integrated user interface 300 further includes alternative clusters of information items, each representing one of a plurality of media playback applications 204.

[0052] In one embodiment, the integrated user interface 300 includes an EPG affordance item 348. In response to a user action on the affordance item 348 (e.g., selection of item 348 via the remote control device 114), the integrated TV application 202 is provided by broadcast services (e.g., TV, cable, satellite, YouTube). The system displays an integrated EPG page containing a timetable of multiple content items and programs organized in chronological order. The content items and programs are selected at least partially based on their location on the network-connected television device 102. In one embodiment, the multiple content items and programs are also determined by comparing a multidimensional integrated user activity characteristic 208 with deduplicated program information corresponding to media content items and programs provided by the broadcast service.

[0053] Information items 302-332 are provided by two or more media content sources 110, but are organized and displayed on the integrated user interface 300 based on the media content items or program content (not media content sources) recommended by the recommendation engine 212 of the server system 104. Specifically, each cluster displayed on the integrated user interface 300 contains a subset of an ordered sequence of selectable information items. The first cluster 338 contains a first selectable information item 318 and a second selectable information item 320. The first and second selectable information items 318 and 320 of the first cluster 338 are displayed simultaneously on the integrated user interface 300. In response to a first user selection of the first selectable information item 318, the television device 102 displays information about a first media content item provided by a first content provider. In response to a second user selection of the second selectable information item 320, the television device 102 displays information about a second media content item provided by a second content provider. The second content provider is different from the first content provider, but the information of the first and second content providers is optionally hidden on the integrated user interface 300 which shows clusters 334-340 and selectable information items 302-332.

[0054] In one embodiment, each of the first and second content providers is selected from broadcast television services, broadcast satellite services, cable services, subscription video-on-demand (SVOD) content providers, transactional video-on-demand (TVOD) content providers, ad-supported video-on-demand (AVOD) services, and playback and time-shifted content sources. Optionally, the first or second content provider relies on tuner-based distribution of wireless media content. Optionally, the first or second content provider relies on Internet Protocol Television (IPTV) or other internet-based distribution. The system delivers live TV streams based on trusted, linear, live, or synthesized live content sources, where the first or second content provider is an internet-based live streaming service. If one of the information items 302–332 corresponds to a broadcast television service (e.g., NBC, ABC, ESPN), then a unique broadcast program corresponding to one of the information items 302–332 is permitted to be displayed at the location of television equipment 102 in accordance with the geographical restrictions associated with the broadcast television service.

[0055] It should be noted that the integrated TV application 202 is associated with a user account in a virtual user domain hosted by the server system 104. In one embodiment, a first content provider is associated with a first media playback application, which is optionally associated with a user account in a virtual user domain hosted by the server system 104. In another embodiment, a second content provider is associated with a second media playback application, which is optionally associated with a user account in a virtual user domain hosted by the server system 104.

[0056] In some situations, the first cluster 338 has more selectable information items than the number of corresponding slots in the integrated user interface 300 assigned to the first cluster 338. A subset of the selectable information items corresponding to the first cluster 338 is physically positioned and configured to move along the first axis 344 of the integrated user interface 300, so that different information items within the subset of selectable information items corresponding to the first cluster 338 can be displayed in the integrated user interface 300 and recommended to the user of the television device 102. Specifically, a first user action on information item 318 (for example, the user pressing the "forward" button on the remote control device 114) can allow information items 318-322 to shift one slot to the right in order to remove information item 324 from the integrated user interface 300 and allow new information items associated with the first cluster 338 to enter the integrated user interface 300. Similarly, a second user action on information item 324 (for example, the user pressing the "Next" button on the remote control unit 114) can shift information items 320-324 one slot to the left, causing information item 318 to exit the integrated user interface 300 and allowing another new information item associated with the first cluster 338 to enter the integrated user interface 300. When one of the first and second user actions is applied consecutively, the start or end information item in the first cluster 338 is displayed. When this happens, one of the first and second user actions may optionally either not change the information item displayed on the integrated user interface 300, or continue to display the end or start from a subset of selectable information items in the first cluster 338 as a new information item following the start / end information item, respectively.

[0057] Furthermore, in one embodiment, multiple clusters are distributed along a second axis 346 of the integrated user interface 300, and include a first subset of clusters and a second subset of clusters. The first subset of clusters may be moved outside the integrated user interface, and the second subset of clusters may be moved within the integrated user interface along the second axis. For example, the first subset of clusters includes cluster 334. A third user action for one of the information items 302-308 (e.g., the user presses the "Up" button on the remote control 114) can cause clusters 334-338 to shift down by one slot so that cluster 340 exits the integrated user interface 300, and a new cluster preceding cluster 334 enters the integrated user interface 300 from the top. Similarly, a fourth user action for one of the information items 326-332 (e.g., the user presses the "Down" button on the remote control 114) can cause clusters 336-340 to shift down by one slot. This shift allows cluster 334 to exit the integrated user interface 300 and another new cluster following cluster 340 to enter the integrated user interface 300. If one of the third and fourth user actions is continuously applied, the top or bottom cluster among the multiple clusters will be displayed. When this happens, one of the third and fourth user actions may optionally either not change the clusters displayed on the integrated user interface 300, or continue to display the bottom or top cluster among the multiple clusters as new clusters following the top or bottom cluster, respectively.

[0058] Figure 4 is an exemplary flowchart of Method 400, which provides knowledge-based media content recommendations to a networked television device 102 according to one embodiment. Method 400 is executed by a server system 104 and is optionally controlled by instructions stored in a non-temporary computer-readable storage medium and executed by one or more processors of the server system 104. Each of the operations shown in Figure 4 may correspond to an instruction stored in computer memory or a non-temporary computer-readable storage medium (e.g., memory 1406 of the server system 104 in Figure 14A). The computer-readable storage medium may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or other non-volatile memory devices. The instructions stored in the computer-readable storage medium may include one or more of source code, assembly language code, object code, or other instruction formats interpreted by one or more processors. Some operations in Method 400 may be combined, and / or the order of some operations may be changed.

[0059] The server system 104 hosts a virtual user domain containing user accounts. The server system 104 associates user accounts with an integrated TV application 202, multiple media playback applications 204, and multiple user applications (402). The integrated TV application 202 is associated with a network-connected television device 102, and the multiple user applications include at least an internet search engine 206. For each of the integrated TV application, media playback applications, and user applications, the server system stores activity data (404). Specifically, the server system 104 stores search queries and search results for multiple search operations performed by the internet search engine 206 from the user account (406), stores viewing information associated with the user account for a subset of the media playback applications 204 (408), and stores purchase information associated with the user account for a subset of the user applications (410). Thereafter, the multidimensional integrated user activity characteristics 208 are calculated from the stored activity data (412).

[0060] The server system 104 also obtains program information associated with multiple media playback applications 204 (414), thereby obtaining deduplicated program information 210 for multiple unique programs. In one embodiment, a first media content item is provided by a first media playback application and associated with first program information, and a second media content item is provided by a second media playback application and associated with second program information within the second media playback application. The server system 104 identifies the similarity levels of the first and second program information (416). In accordance with the determination that the similarity level exceeds the similarity threshold (418), the server system 104 deduplicates the first program information of the first media content item and the second program information of the second media content item (420), and determines that a unique program is associated with both the first and second media content items (422).

[0061] Based on user activity characteristics 208 and program information 210 for multiple unique programs, the server system 104 identifies an ordered subset of unique programs 214 to recommend to the user via the integrated TV application 202 (424). The server system 104 enables the display of an ordered sequence of information items (e.g., items 302-332) corresponding to the ordered subset of unique programs 214 via the integrated TV application 202 (426). Each information item represents a unique program or media content item provided by the corresponding media playback application 204.

[0062] Figure 5 is an illustrative flowchart of a method 500 for presenting program information on a network-connected television device 102, according to one embodiment. Method 500 is performed by the network-connected television device 102 and is optionally controlled by instructions stored in a non-temporary computer-readable storage medium and executed by one or more processors of the network-connected television device 102. Each of the operations shown in Figure 5 may correspond to an instruction stored in computer memory or a non-temporary computer-readable storage medium (e.g., memory 1506 of the network-connected television device 102 in Figure 15). The computer-readable storage medium may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or other non-volatile memory devices. Instructions stored in the computer-readable storage medium may include source code, assembly language code, object code, or one or more other instruction formats interpreted by one or more processors. Some operations in Method 500 may be combined, and / or the order of some operations may be changed.

[0063] The network-connected television device 102 is linked to a user account in a virtual user domain hosted by the server system 104. The network-connected television device 102 runs the integrated TV application 202 (502) and displays the integrated user interface 300. The media recommendation stream is obtained from the server system 104 (504) and includes an ordered sequence of selectable information items 302-332 corresponding to an ordered set of unique media content items or programs 214. For each selectable information item, the media recommendation stream includes the respective command executed in the network-connected television device 102 in response to the user selection of that selectable information item (506).

[0064] The network-connected television device 102 displays multiple clusters on the integrated user interface 300 (508). Each cluster contains a subset of an ordered sequence of selectable information items 302 to 332 (510). The first cluster 338 includes a first selectable information item 318 and a second selectable information item 320. The first and second selectable information items 318 and 320 of the first cluster 338 are displayed simultaneously on the integrated user interface 300 (510). In one embodiment, information of a content provider associated with the first and second selectable information items is hidden on the integrated user interface 300 that displays the first and second selectable information items within the first cluster 338.

[0065] In response to a first user selection of a first selectable information item, the network-connected television device 102 displays information about a first media content item provided by the first content provider (for example, via a first media playback application associated with the first content provider) (512). In response to a second user selection of a second selectable information item, the network-connected television device 102 displays information about a second media content item provided by the second content provider (for example, via a second media playback application associated with the second content provider). Displaying information (514). The second content provider is different from the first content provider. In some situations, displaying information for the first media content item includes directly starting playback of the first media content item provided by the first content provider from the first media playback application, and displaying information for the second media content item includes directly starting playback of the second media content item provided by the second content provider from the second media playback application. Alternatively, in some situations, information for the first and second media content items is displayed via the integrated TV application 202 on the respective pages of those media content items (for example, page 600 in Figure 6A).

[0066] Figures 6A and 6B are two exemplary media content item pages 600 and 650, respectively, displayed on a network-connected television device 102 according to one embodiment. In one embodiment, media content item page 600 is displayed to present information about a first media content item provided by a first media playback application associated with a first content provider, in response to a user selection of a selectable information item (e.g., item 318) within the integrated user interface 300. The information about the first media content item includes one or more of the following: schedule information, a short summary of plot 612, casting information 614, evaluation information 616, and comments about the first media content item. Optionally, the information about the first media content item includes one or more recommendations 618 related to the first media content item.

[0067] A list of content providers 602 that provide the first media content item is displayed on page 600 of the media content item. The first content provider 604 is listed at the top of the list of content providers 602 and has the highest priority thereon. A user account associated with the integrated TV application 202 is subscribed to a subset or all of the content providers in list 602. In one embodiment, the user account is subscribed to at least the first content provider 604, thereby enabling the recommendation engine 212 to rely on the credentials of the first content provider 604 and add the first content item to an ordered sequence of selectable information items provided to the network-connected television device 102.

[0068] Furthermore, in certain circumstances, the list of content providers 602 includes a second content provider 606 that also provides the first media content item, and the first content provider 604 has a higher priority than the second content provider 606 on the list of content providers 602 that provide the first media content item. On the other hand, it should be noted that a separate media content item page may be displayed to present information about the second media content item in response to a user selection of a separate selectable information item (e.g., item 320) in the integrated user interface 300. The separate list of content providers is displayed in relation to the second media content item, along with the second content provider 606, which is listed at the top of the separate list of content providers.

[0069] Referring to Figure 6B, in one embodiment, the media content item page 650 is displayed to present information about the first media content item provided by a first media playback application associated with a first content provider, in response to a user selection of a selectable information item (e.g., item 318) within the integrated user interface 300. The user account does not have a subscription to the first media playback application, and the media content item page 650 is displayed in response to a user action on affordance item 608, indicating that the user account has a subscription to the first content provider. To enable subscription to the provider's services, affordance item 608 is included, linked to a first media playback application. For example, if the first media content item is relatively new, for instance, recently shown in theaters and now available from a TVOD content provider, affordance item 608 is linked to a uniform resource locator (URL) associated with the first media playback application, and reviews of the first media content item are supported on a pay-per-view basis.

[0070] Figure 7 shows an exemplary operating environment 700 in which an integrated TV application 202 and a media playback application 204 cooperate on a network-connected television device 102 to play media content items, according to one embodiment. Both the integrated TV application 202 and the media playback application 204 are run on the network-connected television device 102. Playback of media content items is initiated in the integrated TV application 202. In some situations, playback of a first media content item provided by a first content provider is initiated directly from the first media playback application in response to a user action on a corresponding selectable information item 318 displayed on the integrated user interface 300. Alternatively, in some situations, playback of a first media content item is initiated from the page 600 of the corresponding media content item in response to a user selection of a media content provider from a list of media content providers 602, for example.

[0071] When playback of the first media content item begins, the integrated TV application 202 relies on the first or other selected media playback application 204 to play the first media content item. In one embodiment, the first media content item is played by the first media playback application, but the user interface elements of the first media playback application are bypassed and stopped so as not to be displayed to the user of the network-connected television device 102. These bypassed user interface elements include those associated with the home user interface of the first media playback application. In contrast, alternative user interface features do not have to be skipped, and an example of these non-skipped user interface features is a splash image frame indicating that the first media item is provided by the first media content provider. By bypassing the user interface elements of the first media playback application, the integrated TV application 202 and the media playback application 204 are seamlessly integrated together to enable playback of the first media content item without requiring user intervention to switch between these two applications.

[0072] Figure 8 is an exemplary sequence 800 of a user interface that is sequentially displayed on a network-connected television device 102 in response to a user action on a remote control device 114 coupled to the network-connected television device 102, according to one embodiment. When a page 600 (also called a summary page) of a media content item is displayed on the television device 102 for a first media content item, a first media content provider 604 is selected from a list of media content providers 602 by a first user action (for example, by pressing the play button 802 on the remote control device 114) with instructions for the first media content provider. In response to this user action, the first media content item is provided by the first media content provider 604 and played on the media playback interface 804 via a first media playback application associated with the first media content provider 604. The media playback application runs and is hidden in the background to enable playback of the first media content item.

[0073] In one embodiment, a user request is received to stop playback of a first media content item on a media playback interface 804 enabled by a first media playback application. In response to the user request to stop playback of the first media content item, the home user interface 806 of the first media playback application is displayed, and the network-connected television device 102 operates in a media environment managed by the first media playback application. The user of the network-connected television device 102 can exit the media environment by selecting an exit affordance item on the home user interface 806 of the first media playback application, or by pressing one or more predetermined shortcut buttons on the remote control device 114.

[0074] Alternatively, in one embodiment, in response to a user request to stop playback of a first media content item on the media playback interface 804, display control is returned directly to the integrated TV application 202 associated with the network-connected television device 102. That is, in response to the user request, the network-connected television device 102 displays pages 600 or multiple clusters of the media content item of the first media content item on the integrated user interface 300 via the integrated TV application 202. These means make the involvement of the first media playback application completely invisible to the user of the network-connected television device 102, thereby enabling the media playback application 204 to be seamlessly integrated into the integrated TV application 202.

[0075] Figure 9 shows an exemplary operating environment 900 in which an integrated TV application 202, a media playback application 204, and a content casting application 902 of a network-connected television device 102 cooperate to enable playback of media content items, according to one embodiment. The media playback application 204 is a dedicated application associated with a specific media streaming service or broadcast channel, a cable or over-the-air (OTA) tuner, or a web browser. The content casting application 902 resides within the connected television device 102 running the integrated TV application 202. The network-connected television device 102 has a normal mode and a restricted mode, the restricted mode being associated with content selection criteria. In normal mode, media content permitted to be played via the integrated TV application 202 is selected from media content provided by each media content source 110 associated with the media playback application 204, without any restrictions other than user account entitlements (i.e., the right to access media content permitted by the subscription). In contrast, in restricted mode, the media content permitted to be played via the integrated TV application 202 is selected from the media content provided by each media content source 110 associated with the media playback application 204, according to content selection criteria in addition to user account eligibility. Thus, implementing restricted mode involves operations on both the server side and the TV device side.

[0076] On the server side, the recommendation engine 212 compares user activity characteristics 208 with program information 210 of multiple unique content items or programs to identify an ordered subset of unique programs 214 and recommends them to the user in normal mode. In restricted mode, in the process of obtaining an ordered subset of unique programs 214, the recommendation engine 212 applies content filtering criteria along with (if any) user activity characteristics 208 and other filtering conditions (e.g., credentials of the media playback application 204). In other words, in restricted mode, the ordered subset of unique programs 214 is filtered by the content filter The selection is determined at least partially based on the ring criteria. The selectable information items displayed on the television device 102 correspond to an ordered subset of unique programs 214 and are therefore selected from program information 210 associated with one or more media playback applications 204 according to the content selection criteria.

[0077] On the television device side, the integrated TV application 202 is executed to enable the display of the integrated user interface 300 on the network-connected television device 102. In normal mode, the integrated TV application 202 does not impose any restrictions on content selection criteria on media content provided via the media playback application 204, and the media playback application 204 is called by the integrated TV application 202 to play the media content. Conversely, in restricted mode, content selection criteria are applied via the integrated TV application 202. The integrated TV application 202 receives user actions on a first selectable information item on the integrated user interface 300. The first selectable information item corresponds to a first media content item provided by a first content provider in the first media playback application. In response to a user action on the first selectable information item, the integrated TV application 202 determines that the integrated TV application is operating in restricted mode and the first media playback application is not operating in restricted mode, and the local content casting application is called by the integrated TV application 202 to display the first media content item corresponding to the first selectable information item on the network-connected television device 102.

[0078] Alternatively, in one embodiment, the integrated TV application 202 determines that it operates in restricted mode and that the first media playback application can operate in restricted mode, and the first media playback application is invoked by the integrated TV application 202 to display a first media content item corresponding to a first selectable information item on the network-connected television device 102. In restricted mode, the media playback application 204 has priority over the content casting application 902, which is invoked only if the media playback application 204 cannot operate in restricted mode. The content casting application 902 is available locally to function on behalf of the media playback application 204 that cannot meet the content selection criteria.

[0079] In one embodiment, the media playback application 204 is associated with each third-party media content source 110 independently of an integrated TV application configured to introduce content selection criteria to restrict the media content provided through each media playback application. Alternatively, in one embodiment, the media playback application 204 is associated with a server system 104 that also implements the integrated TV application 202. Furthermore, in one embodiment, the content casting application 902 includes multiple media receiver modules 904 corresponding to multiple media playback applications 204. In restriction mode, the integrated TV application 202 applies content selection criteria to the media content provided by the media content sources 110 associated with the media playback applications 204 via each media receiver module 904 of the content casting application 902.

[0080] Figure 10A is an exemplary integrated user interface 1000 displayed on a network-connected television device 102 in restricted mode, according to one embodiment. Figures 10B and 10C show, respectively, a network connection in response to user actions on a remote control device 114 coupled to the network-connected television device 102, according to one embodiment. These are two exemplary sequences of user interfaces 1040 and 1080 that are sequentially displayed on the television device 102. In restricted mode, content selection criteria define age-based content ratings, and first media content items corresponding to a plurality of selectable information items displayed in the integrated user interface 1000 conform to the age-based content ratings. For example, a kids' zone may be created within the corresponding integrated TV application 202. In one embodiment, the first content provider is configured to provide one or more second media content items, each of which is different from the first media content items corresponding to selectable information items displayed in the integrated user interface 1000. In normal mode, one or more second media content items can be played by a first media playback application associated with the first content provider, but in restricted mode, they are prohibited from being played by the local content casting application 902.

[0081] In one embodiment, the integrated TV application 202 loads a baseline user interface and receives input of the child's age associated with a user account associated with the integrated TV application 202 on the baseline user interface. The integrated TV application 202 is configured to automatically identify age-based content ratings based on the input of the child's age. While operating in restricted mode, the server system 104 identifies age-based content ratings defined based on the child's age and selects a number of selectable information items according to the age-based content ratings to be displayed on the integrated user interface 1000, thereby creating a kid-friendly media environment on the television device 102.

[0082] As described above, in restricted mode, an ordered subset of unique programs 214 is determined at least in part on content filtering criteria. The selectable information items displayed on the television device 102 correspond to an ordered subset of unique programs 214 and are selected from program information 210 associated with one or more media playback applications 204 according to content selection criteria. In one embodiment, the program information 210 associated with one or more media playback applications 204 includes several characteristic fields for each media content item or program, such as an age rating field. The recommendation engine 212 compares the value of the age rating field with an age-based content rating to determine whether the corresponding media content item or program conforms to the content selection criteria that define the age-based content rating.

[0083] Furthermore, in one embodiment, the first selectable information item is selected from first program information associated with a first media playback application according to content selection criteria, and the first program information is provided at least partially by an information source different from the first media playback application. For example, the first media playback application (e.g., HBO) does not have a kids zone, and the corresponding program information 210 for the media content item provided by it does not include an age rating field and may not be used to determine whether the media content item conforms to an age-based content rating. As one solution, the recommendation engine 212 can extract the program information 210 from an information source different from the first media playback application and determine whether the media content item provided through the first media playback application conforms to an age-based content rating.

[0084] Referring to Figure 10B, in one embodiment, when the integrated user interface 1000 is displayed on the network-connected television device 102 and presents a number of selectable information items, the television device 102, for example via the remote control device 114, can select a first selectable item. The television device 102 receives a user selection of an information item. In response to the user selection, the television device 102 begins playback of a first media content item corresponding to the first selectable information item. In one example, the television device 102 sequentially displays a splash image frame 1002 indicating that the first media content item is provided by a first media content provider, followed by a frame of the first media content item 1004. Referring to Figure 10C, in a certain situation, the television device 102 responds to a user selection of a first selectable information item by beginning playback of a first media content item 1006 corresponding to the first selectable information item without displaying the splash image frame 1002.

[0085] When the first media content item is played on the television device 102, the television device 102 receives a user request to stop playing the first media content item, for example, via the remote control device 114. In one embodiment (Figure 10B), in response to a decision that the integrated TV application 202 is operating in restricted mode, the television device 102 stops displaying the first media content item and redisplays a plurality of clusters having selectable information items (i.e., the integrated user interface 1000). Alternatively, in one embodiment (Figure 10C), in response to a decision that the integrated TV application 202 is operating in restricted mode, the television device 102 stops displaying the first media content item and displays the home user interface 1008 of the first media playback application in response to a user request. Furthermore, in response to an additional user request, the television device 102 exits the home user interface and redisplays a plurality of clusters having selectable information items (i.e., the integrated user interface 1000). Conversely, in response to a determination that the integrated TV application is operating in normal mode, the television device 102 either stops displaying the first media content item and redisplays multiple clusters on the integrated user interface 1000, or displays the home user interface 1008 of the first media playback application.

[0086] Figure 11 is an illustrative flowchart of a method 1100 for presenting selected media content provided by a media playback application on a network-connected television device 102, according to one embodiment. Method 1100 is performed by the network-connected television device 102 and is controlled by instructions, optionally stored in a non-temporary computer-readable storage medium and executed by one or more processors of the network-connected television device 102. Each of the operations shown in Figure 5 may correspond to an instruction stored in computer memory or a non-temporary computer-readable storage medium (e.g., memory 1506 of the network-connected television device 102 in Figure 15). The computer-readable storage medium may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or other non-volatile memory devices. Instructions stored in the computer-readable storage medium may include one or more of source code, assembly language code, object code, or other instruction formats interpreted by one or more processors. Some operations in Method 1100 may be combined, and / or the order of some operations may be changed.

[0087] The television device 102 is linked to a user account in a virtual user domain hosted by the server system 104. The television device 102 runs an integrated TV application 202 to enable the display of the integrated user interface 1000 on the network-connected television device (1102). The integrated TV application 202 has a normal mode and a restricted mode (1104) and is associated with one or more standard media playback applications 204 and a local content casting application 902. Multiple clusters are displayed in the integrated user interface 1000 (1106). Each cluster has multiple selectable information items. The television device 102 identifies content selection criteria (1108) and, according to those content selection criteria, one or (1110) selects multiple selectable information items from program information 210 associated with multiple media playback applications 204.

[0088] The television device 102 receives a user action on a first selectable information item corresponding to a first media content item provided by a first content provider in a first media playback application (1112). In response to the user action on the first selectable information item (1114), depending on whether the integrated TV application is operating in restricted mode or whether the first media playback application is not operating in restricted mode, the television device 102 displays the first media content item corresponding to the first selectable information item on the network-connected television device via a local content casting application (1116). Alternatively, in response to the user action on the first selectable information item (1114), depending on whether the integrated TV application is operating in normal mode, the television device 102 displays the first media content item corresponding to the first selectable information item on the network-connected television device 102 via the first media playback application (1118).

[0089] In one embodiment, the first content provider is configured to provide one or more second media content items, each distinct from a plurality of media content items, each corresponding to a plurality of selectable information items. The first media playback application is configured to play one or more second media content items in normal mode. In restricted mode, one or more second media content items are prohibited from being played by the local content casting application.

[0090] Figure 12 shows an exemplary operating environment 1200 in which focus items (also called tiles) provided by a recommendation engine 212, one or more advertising providers, and a sales module 1202 are mixed together for display on a network-connected television device 102, according to one embodiment. The one or more advertising providers include an internal advertising provider 1204 hosted and managed by a server system 104, and / or an external third-party advertising source 106 not hosted by the server system 104. An advertising management module 1208 is coupled to one or more advertising providers and is configured to manage the advertising content provided by one or more advertising providers. The recommendation engine 212 is configured to compare program information 210 and multidimensional integrated user activity characteristics 208 for multiple unique programs provided by a media content source 204 to identify an ordered subset of unique programs 214 for recommendation to the user via an integrated TV application 202. In particular, the recommendation engine 212 can recommend media content provided by broadcast services (television, satellite, or cable) and video-on-demand (VOD) content providers to users of the television device 102, based on user activity characteristics 208. The sales module 1202 is hosted and managed by the server system 104 and is configured to provide focus items selected by the server system 104 for sales promotion. The focus items selected by the sales module 1202 are optionally independent of user activity characteristics 208 or determined based on the user's user activity characteristics 208. In one embodiment, the focus items selected by the sales module 1202 are advertisements suggesting to the user to subscribe to a service provided by an SVOD content provider, purchase or rent certain media content, or play an online game. For example, the focus items selected by the sales module 1202 include trailer clips associated with newly released movies.In one embodiment, the focus item selected by the sales module 1202 is TVOD content, independently of the user activity characteristics 208 of any particular user. The focus items are provided by the provider (i.e., the focus items are associated with the TVOD content provider and pushed to the television device 102 by the sales module 1202). Alternatively, in some situations, the focus items selected by the sales module 1202 are determined by the server system 104 based on the user activity characteristics 208 of numerous users across different use cases in a virtual user domain hosted by the server system 104. For example, each of the focus items selected by the sales module 1202 is determined based on one or more of the following for each focus item: content type / format, content quality, newness, popularity, seasonality, and application lifecycle.

[0091] Referring to Figure 3, the integrated TV application 202 runs to enable the display of the integrated user interface 300 on a network-connected television device 102, the integrated user interface 300 having a focus area 362 (e.g., a prominently displayed area of ​​the user interface) configured to display multiple focus items. The focus area 362 is optionally located in the upper half of the integrated user interface 300. Multiple focus items are displayed sequentially in chronological order in the focus area 362. Optionally, multiple focus items are organized chronologically as a rotary conveyor belt, smoothly transitioning from the last focus item to the starting focus item after the last focus item is displayed. Each focus item is associated with its respective focus item provider selected from the sales module 1202, the advertising management module 1208, and the media recommendation engine 212. The server system 104 includes a mixer 1206 configured to automatically mix the multiple focus items sequentially according to chronological order. In one embodiment, the focus area 362 extends to partially overlap with a subset of selectable information items 302-332. In one example, the focus area 362 covers the entire integrated user interface 300 and overlaps with all of the selectable information items 302-332. That is, a subset or all of the selectable information items 302-332 are displayed in the background of the focus area 362, and these multiple focus items are sequentially displayed in the background for a subset or all of the selectable information items 302-332.

[0092] Furthermore, the ad management module 1208 of the server system 104 further includes an ad transformation module (not shown in Figure 12) configured to customize media content items received from an external ad source 106 for display in the focus area 362 of the integrated user interface 300. For example, the multiple focus items include a first focus item. The television device 102 sends a request for the first focus item to the first advertiser server 106, the request including information on a plurality of predetermined user interface elements of the focus area of ​​the integrated user interface. In response to the request, the television device 102 receives a plurality of media content items associated with the first focus item from the first advertiser server 106 and integrates the plurality of media content items into the first focus item for display on the focus area during the time slot corresponding to the first focus item. In one embodiment, the plurality of media content items include a subset of media content items that conform to the Video Ad Placement Template (VAST) standard for communication requirements between the first advertiser server and the network-connected television device. Therefore, the ad conversion module of the ad management module 1208 is configured to integrate a subset of content items into a first focus item and to customize the first focus item for display on the integrated user interface 3000 of the network-connected television device 102.

[0093] Examples of predefined user interface elements in focus area 362 include background images, stylized title / logo images, descriptions including catchphrases, calls to actions, title text, application names, source labels, package names, and clicks. It includes a web link and one or more video clips. The background image may include a text image without text. A request for the first focus item sent to the first advertiser server defines which section of the text image should be left blank to make room for a title. In one embodiment, a logo image is required, but a stylized font is optional and facilitates a one-line description of the media content associated with the first focus item. Calls to actions include selectable actions for the first focus item, such as "Sign Up Now," "Available for Rent," "Watch Now," "Subscription Required," "Coming to Theaters," "Add App to Favorites," and "Add Channel to Home Screen," and the server system 104 associates the first focus item with a subset of the selectable actions. Title text is used when a stylized title is unavailable. The first focus item carries a source label that highlights to the user of the television device 102 whether the content is an advertisement, a commercialized unit, or a recommendation. Click-through links include links to an application, content within an application, or a commercial webpage associated with the primary focus item, where a click on the primary focus item automatically triggers the link.

[0094] In one example, the first focus item includes a static advertisement image of a movie displayed in a theater, and the first focus item includes a link 364 for purchasing tickets and / or a link 366 for pre-ordering a digital multipurpose disc (DVD). In one embodiment, the static advertisement image indicates "Available on DVD" and has a link for renting a DVD of the corresponding movie.

[0095] In one embodiment, multiple focus items include video clips or still images configured to highlight media content items provided by broadcast television services, broadcast satellite services, cable services, SVOD content providers, or AVOD services. The recommendation engine 212 of the server system 104 identifies the multidimensional user activity characteristics 208 of the user of the network-connected television device 102, retrieves program information 210 associated with multiple media playback applications 204, and identifies highlighted media content items for recommendation to the user via the integrated TV application. Specifically, the highlighted media content items are presented to the user of the television device 102 in the focus area 362 of the integrated user interface 300. Information on the highlighted media content is further provided to the recommendation engine 212 so that the multidimensional user activity characteristics 208 and subsequent recommendations for media content items can be updated.

[0096] In one embodiment, a plurality of focus items sequentially displayed within the focus area 362 further include a second focus item, the second focus item including a video clip or still image configured to highlight a media content item provided by a TVOD content provider. The video clip or still image is displayed during a time slot corresponding to the second focus item. In response to user actions on the displayed video clip or still image, a TVOD media application is executed, enabling the user to order and play the media content item corresponding to the second focus item. In some situations, this video clip or still image is sponsored by a TVOD content provider or selected by a server system 104 to advertise the corresponding media content item, independently of the user activity characteristics 208 of the user of the network-connected television device 102.

[0097] Referring to Figure 3, in one embodiment, multiple focus items are displayed simultaneously with multiple selectable information items organized into clusters. The selectable information items in each cluster are: Focus areas and multiple clusters are distributed parallel to the first axis 344, but along a second axis 346 of the integrated user interface 300, which is different from the first axis 344. Focus areas are fixed on the integrated user interface 300, independent of the display of multiple selectable information items on non-focus areas that are separate from and do not overlap with the focus area. That is, regardless of whether the selectable information moves within, enters, or leaves the non-focus area, the focus item is displayed in the fixed focus area. In one embodiment, multiple focus item indicators 368 are displayed along with the focus item. Each focus item indicator 368 represents one of the multiple focus items. For each of the multiple focus items, the television device 102 displays the respective focus item indicator according to the active specification 370a, depending on the determination that the respective focus item is displayed in the focus area. The television device 102 displays the respective focus item indicator according to the inactive specification 370b, which is different from the active specification 370a, depending on the determination that the respective focus item is not displayed in the focus area.

[0098] In a certain situation, while one of several focus items is displayed in the focus area 362, the television device 102 receives a predetermined user action associated with each focus item (for example, clicking the previous or next button on the remote control device 114). In response to the predetermined user action, the television device 102 stops displaying each focus item and displays either (1) a subsequent focus item that follows each focus item in chronological order, or (2) a preceding focus item that precedes each focus item in chronological order, within the focus area of ​​the integrated user interface.

[0099] Focus items are often updated within the focus area 362 to maintain their freshness. For example, the server system 104 determines the number of times multiple focus items have been displayed sequentially in chronological order in the focus area 362. If the server system 104 determines that this number is greater than or equal to a predetermined threshold (e.g., 2 times), it replaces at least one of the multiple focus items with an alternative focus item.

[0100] Furthermore, in one embodiment, the server system 104 is configured to deduplicate focus items that are sequentially displayed within the focus area 362. Multiple focus items include a second focus item in addition to the first focus area. The mixer 1206 receives two or more original items each from separate content providers selected from the sales module 1202, advertiser 106, and media recommendation engine 212. The mixer 1206 deduplicates two or more original items according to a preferred order of the sales module 1202, advertiser 106, and media recommendation engine 212 to produce a second focus item. In one example, the preferred order defines a decreasing priority for advertiser 106, sales module 1202, and media recommendation engine 212, which gives advertiser 106 priority over any other content source.

[0101] In one embodiment, information on user actions associated with a focus area is provided to one of the following: the recommendation engine 212, the advertising management module 1208, and the sales module 1202, which provides focus items displayed in the focus area. The user action information is information indicating the user preferences of the user of the network-connected television device 102 regarding a plurality of focus items displayed in the focus area. Each of the recommendation engine 212, the advertising management module 1208, and the sales module 1202 provides one or more recommendations for the user based on the user preferences, or based on the user preferences of a plurality of users associated with the user of the network-connected television device 102. You can determine additional focus items.

[0102] Figure 13 is an exemplary flowchart of a method 1300 for mixing focus content (including VAST-based advertisements) on a network-connected television device 102, according to one embodiment. Method 1300 is performed by a server system 104 and is optionally controlled by instructions stored in a non-temporary computer-readable storage medium and executed by one or more processors of the server system 104. Each of the operations shown in Figure 13 may correspond to an instruction stored in computer memory or a non-temporary computer-readable storage medium (e.g., memory 1406 of the server system 104 in Figure 14A). The computer-readable storage medium may include magnetic or optical disk storage devices, solid-state storage devices such as flash memory, or other non-volatile memory devices. Instructions stored in the computer-readable storage medium may include source code, assembly language code, object code, or one or more other instruction formats interpreted by one or more processors. Some operations in Method 1300 may be combined, and / or the order of some operations may be changed.

[0103] The server system 104 hosts a virtual user domain containing user accounts. User accounts are associated with the integrated TV application 202 and multiple media playback applications 204 (1302). The integrated TV application 202 is associated with the network-connected television device 102 (1304) and is configured to enable the display of media content on the television device 102, and each of the multiple media playback applications 204 is configured to display media content provided by its respective content provider 110. The integrated TV application 202 runs in such a way that it can display the integrated user interface on the network-connected television device 102 (1306). The integrated user interface 300 has a focus area 362 configured to display multiple focus items. Multiple focus items (also called tiles) are displayed sequentially within the focus area 362 in chronological order, for example, like a rotary conveyor belt (1308). Each focus item is associated with its respective focus item provider selected from the sales module, advertisers, and media recommendation engine.

[0104] Multiple focus items include a first focus item. Server system 104 sends a request for the first focus item to the first advertiser server (1310). The request includes information on multiple predetermined user interface elements of the focus area of ​​the integrated user interface. In response to the request, server system 104 receives multiple media content items associated with the first focus item from the first advertiser server (1312) and integrates the multiple media content items into the first focus item for display on the focus area 362 during the time slot corresponding to the first focus item.

[0105] It should be understood that the specific order in which the operations are described in Figures 4, 5, 11, and 13 is merely illustrative and does not indicate that the described order is the only order in which the operations can be performed. Those skilled in the art will recognize the various methods for displaying information items and focusing content in the integrated user interface as described herein. Furthermore, it should be noted that the details described with respect to one of processes 400, 500, 1100, and 1300 are also applicable in a similar manner to any other of methods 400, 500, 1100, and 1300. For the sake of brevity, similar details will not be repeated.

[0106] Figure 14A shows a block illustrating a server system 104 in a media environment according to one embodiment. This is a diagram. The server system 104 typically includes one or more processing units (CPUs) 1402, one or more network interfaces 1404, memory 1406, and one or more communication buses 1408 (sometimes called a chipset) for interconnecting these components. The server system 104 may include one or more input devices 1410 that facilitate user input, such as a keyboard, mouse, voice command input unit or microphone, touchscreen display, touch sensor input pad, gesture capture camera, or other input buttons or controls. Furthermore, the server system 104 may use a microphone and voice recognition, or a camera and gesture recognition, to supplement or replace the keyboard. In one embodiment, the server system 104 includes one or more cameras, scanners, or photosensor units for acquiring images of graphic series codes printed on, for example, an electronic device. The server system 104 may also include one or more output devices 1412 that enable the presentation of a user interface and the display of content, including one or more speakers and / or one or more visual displays.

[0107] Memory 1406 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid-state storage devices. Memory 1406 optionally includes one or more storage devices located remotely from one or more processing units 1402. Memory 1406, or alternatively the non-volatile memory within Memory 1406, includes a non-temporary computer-readable storage medium. In one embodiment, Memory 1406, or the non-temporary computer-readable storage medium of Memory 1406, stores the following programs, modules, and data structures, or subsets or supersets thereof: • Operating system 1416, which handles various basic system services and includes procedures for performing hardware-dependent tasks. • A network communication module 1418 for connecting the server system 104 to other devices (e.g., various servers within the server system 104, advertising sources 106, EPG sources 108, media content sources 110, network-connected television equipment 102, remote control devices 114, media devices 114-126, auxiliary devices 138, home devices 140, and mobile devices) via one or more network interfaces 1404 (wired or wireless) and one or more networks 112 such as the Internet, other wide area networks, local area networks, and metropolitan area networks. • A casting device application 1420 is performed to provide server-side functions for device provisioning, device control, and user account management associated with the casting device 118. • One or more internal media playback applications 1422 (e.g., YouTube and Google Play) that are executed to provide server-side functionality for media display and user account management. - One or more user applications 1424, including one or more of the following: an internet search engine, a mapping application, a social media application, a retail application, and a device application, which is performed to provide server-side functions for device provisioning, device control, data processing, and data review of the corresponding auxiliary device 138 or home device 140. A server-side TV application 134 is executed to provide server-side functions for device provisioning, device control, data processing, and data review of a network-connected television device 102. A knowledge graph engine 130 is executed to collect activity data from media playback applications and user applications associated with a user account, calculate multidimensional integrated user activity characteristics 208 based on the collected activity data, and collect (deduplicated or non-duplicate) program information associated with the media playback application 204. A recommendation engine 212 is executed to compare user activity characteristics 208 with program information 210 for multiple unique programs in order to identify an ordered subset of unique programs 214 for recommendation to the user via the integrated TV application 202. • Sales module 1202 is executed to provide focus items selected to promote media content, regardless of the user activity characteristics 208 of individual user accounts. • An ad management module 1208, which is executed to manage ad content provided by an internal ad provider 1204 or an external ad source 106, and which includes an ad transformation module that is executed to customize media content items received from the external ad source 106 for display within the focus area 362 of the integrated user interface 3000, for example, according to a customized VAST format. A mixer 1206 (also called a focus item aggregation module) is executed to sequentially blend multiple focus items provided by two or more content sources (e.g., recommendation engine 212, sales module 1202, and ad provider 1204 or 106) in chronological order. A media search engine 222 receives a search query from the server-side TV application 134 or the casting service module 136 and is executed to identify one or more media content items from the program information of the programs or content items provided by the media content source 110. Server system data 1440 that stores at least the following data associated with controlling media display on network-connected television device 102: A user account setting 1442 for storing account settings for one or more user accounts and settings for devices and applications associated with each user account and managed by the server system 104, further including: • TV device settings 1444 for storing information associated with the TV device 102, including common device settings (e.g., service layer, device model, memory capacity, processing power, communication capabilities, etc.) and information associated with the corresponding integrated TV application 202 (including one or more of the following: account access information, user preferences for media content types, and viewing history data). • Information associated with the casting device 118, including one or more of the following: account access information, device settings (e.g., service layer, device model, memory capacity, processing power, communication capabilities, etc.), and casting device settings 1446 for storing user accounts for the casting device application 1420. • Media playback application settings 1448 for storing information associated with a user account of one or more media playback applications 204, including one or more of the following: account access information, user preferences for media content types, and viewing history data. • Remote control settings 1450 for storing information associated with one or more remote control devices 114, such as common device settings (e.g., service layer, device model, memory capacity, processing power, communication capabilities, etc.), media control functions linked to individual buttons, and commands and data for performing each of the media content functions. A device and application registry 150 for storing information about one or more user accounts managed by the server system 104, and information about user devices and applications associated with each of the one or more user accounts. • Data generated by various applications of multiple devices associated with each user account managed by the server system 104, for example, user account data 1454 for storing display information associated with media content currently displayed on the television device 102. • Knowledge graph data 1456 associated with the knowledge graph engine 130 performed by the server system 104, which includes activity data 1458 of the integrated TV application 202, media playback application 204, and user application 1424, multidimensional integrated user activity characteristics 208 derived from said activity data, and one or more program information 210 associated with multiple media playback applications 204. One or more content selection criteria 1460 applied by the integrated TV application 202 to control the media content displayed on the television device 102.

[0108] Each of the elements identified above may be stored in one or more of the aforementioned memory devices and corresponds to a set of instructions for performing the functions described above. The modules or programs (i.e., sets of instructions) identified above do not need to be implemented as separate software programs, procedures, modules, or data structures, and therefore various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 1406 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 1406 optionally stores additional modules and data structures not described above.

[0109] Figure 14B shows an exemplary device and application registry 150 of the server system 104, according to one embodiment, which stores information about one or more user accounts managed by the server system 104, and information about user devices and applications associated with each of the one or more user accounts. In this example, user account 1482 is defined according to an email address (for example, abc@gmail.com). This is associated with multiple user devices, including a casting device 118 (e.g., Google Chromecast), a surveillance camera 148 (e.g., a Nest camera), and a remote control device 114. The device and application registry 150 stores one or more of the following for each user device associated with a user account: a device identifier (ID), device type, device manufacturer ID, and Internet Protocol (IP) address. The device ID is assigned to a user device when it is linked to user account 1482. The manufacturer ID includes the product serial number provided by the manufacturer to uniquely identify the user device. When each user device is linked to user account 1482, its respective manufacturer ID is required to authenticate its respective user device. The IP address of the user device is used to track the location of the user device on the Internet. These are obtained during the process of linking the user device to user account 1482 and are updated when the location of the user device changes on the Internet.

[0110] In one embodiment, a first user device associated with user account 1482 is linked to a second user device associated with user account 1482. For example, a remote control device 114 (device ID=10) is linked to a network-connected television device 102 (device ID=11) but not to a casting device 118 (device ID=1). In one embodiment, user account 1482 includes two or more remote control devices 114, each of which is uniquely linked to a specific network-connected television device 102 or casting device 118. In one embodiment, user account 1482 is linked to two or more casting devices or television devices according to predetermined linking rules (for example, according to the distance of a remote control device 114 to one of two or more casting devices or television devices). Includes location 114.

[0111] In one embodiment, it is defined according to an email address (for example, abc@gmail.com) The user account 1482 is also associated with an integrated TV application 202, multiple media playback applications 204, and multiple user applications 1424. Each of the media playback applications 204 is configured to display media content items or programs provided by broadcast services (television, satellite, or cable) and VOD content providers. The VOD content providers are optionally the server system 104 or third-party content providers separate from the server system 104. Examples of media playback applications 204 include, but are not limited to, YouTube, Google Play, Netflix, Hulu, Amazon Prime, HBO, and several TV channel applications (e.g., ESPN, NBC, ABC, and PBS). Conversely, examples of multiple user applications 1424 include email applications (e.g., Google Gmail), map applications (e.g., Google Maps), assistance applications (e.g., Google Assistance), home device applications (e.g., Nest Camera, Nest Hello), and online retail applications (e.g., Google Shopping). Optionally, each media player or user application is stored with one or more of the following: an application identifier (ID), an application type, a flag indicating whether it is a first-party or third-party application, and enrollment / subscription information (e.g., membership type, credentials).

[0112] Figure 15 is a block diagram showing an exemplary network-connected television device 102 configured to display an integrated user interface and media content provided by a media playback application 204, according to one embodiment. The network-connected television device 102 typically includes one or more processing units (CPUs) 1502, one or more network interfaces 1504, memory 1506, and one or more communication buses 1508 (sometimes called a chipset) for interconnecting these components. Memory 1506 includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and also optionally includes non-volatile memory such as one or more magnetic disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid-state storage devices. Memory 1506 optionally includes one or more storage devices located remotely from one or more processing units 1502. Memory 1506, or the non-volatile memory within Memory 1506, includes non-temporary computer-readable storage media. The television device 102 may include one or more input devices 1510 to facilitate user input, such as input buttons or controls. Furthermore, the television device 102 may use a microphone and voice recognition, or a camera and gesture recognition, to supplement or replace buttons. The television device 102 may also include one or more output devices 1512, including speakers and / or visual displays, to enable the presentation of a user interface and the display of content.

[0113] In one embodiment, memory 1506, or the non-temporary computer-readable storage medium of memory 1506, stores the following programs, modules, and data structures, or subsets or supersets thereof: • Operating system 1514, which handles various basic system services and includes procedures for performing hardware-dependent tasks. • One or more network interfaces 1504 (wired or wireless), and the Internet, other wide area networks, local area networks, metropolitan networks A network communication module 1516 for connecting a network-connected television device 102 to other computers or systems (e.g., a server system 104, a media content source 110, a remote control device 114) via one or more networks 112 such as a local area network, a cable television system, a satellite television system, or an IPTV system. • An integrated TV application 202 presents recommended media content to the integrated user interface 300 and plays the media content provided by the media content source 204 in response to the user's selection of recommended media content. A local content casting application 902 is configured to be called by an integrated TV application 202 in a restricted mode for playing media content items selected according to content selection criteria, and includes multiple media receiver modules 904, each corresponding to a respective media content source 110, which are called to play media content items provided by each media content source, bypassing the corresponding media playback application 204. • A media playback application 204 that is associated with a corresponding media content source 110 and is executed to provide device-side functions for media display and user account management. Device data 1526 that stores data including, at least, the following, associated with the control of media display on the television device 102: Casting settings 1528 for storing information associated with a user account of a casting device application, including one or more of the following: account access information, device settings (e.g., service layer, device model, memory capacity, processing power, communication capabilities, etc.), and information for automatic media display control. • Media playback application settings 1530 that store information associated with the user account of the media playback application 204, including one or more of the user's preferences for media content types and viewing history data. Integrated TV application settings 1532 for storing information associated with a user account of the integrated TV application 202, including one or more account access details, user preferences for media content types, and viewing history data associated with the integrated TV application 202.

[0114] Each of the elements identified above may be stored in one or more of the aforementioned memory devices and corresponds to a set of instructions for performing the functions described above. The modules or programs (i.e., sets of instructions) identified above do not need to be implemented as separate software programs, procedures, modules, or data structures, and therefore various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 1506 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 1506 optionally stores additional modules and data structures not described above.

[0115] Furthermore, while terms such as "first," "second," etc., are used herein in some examples to describe various elements, it will be understood that these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the embodiments described in various ways, a first type of phonetic feature can be called a second type of phonetic feature, and similarly, a second type of phonetic feature can be called a first type of phonetic feature. Both the first type of phonetic feature and the second type of phonetic feature are types of phonetic features, but they are not the same type of phonetic feature.

[0116] The technical terms used in the description of the various embodiments described herein are specific to the present invention. This description is intended solely to illustrate, and not to limit, the embodiments. As used in the various embodiments and the accompanying claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context explicitly indicates otherwise. As used herein, the terms “and / or” refer to, and also encompass, one or more of any and all possible combinations of the enumerated items relating to them. As used herein, the terms “include,” “encompass,” “equip,” and / or “equip” identify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0117] As used herein, the term "if" is to be interpreted, optionally, depending on the context, as meaning "when..." or "if..." or "in response to a decision" or "in response to detection" or "in accordance with the decision that...". Similarly, the expression "if it is decided" or "if [the stated condition or event] is detected" is to be interpreted, optionally, depending on the context, as meaning "if it is decided" or "in response to making a decision" or "in response to detecting a predetermined condition or event" or "in response to detecting a predetermined condition or event" or "in accordance with the determination that a predetermined condition or event has been detected".

[0118] While various drawings illustrate several logical stages in a specific order, the order-independent stages may be rearranged, and other stages may be combined or separated. Some rearrangements or other groupings are specifically mentioned, while others are obvious to those skilled in the art; therefore, the orderings and groupings presented herein are not an exhaustive list of alternatives. Furthermore, it should be recognized that such stages may be implemented in hardware, firmware, software, or any combination thereof.

[0119] The above description has been provided with reference to specific embodiments for illustrative purposes. However, the above exemplary description is not intended to be exhaustive or to limit the claims to the exact form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were selected to best illustrate the principles underlying the claims and their practical applications, thereby enabling other persons skilled in the art to best utilize the various modified embodiments for their intended specific applications.

Claims

1. It is a method, A television application running on a network-connected television device presents content-based media recommendations in the user interface on the network-connected television device. The television application includes receiving a selection of a media content item from the content-based media recommendations presented in the user interface, and the selection initiates playback of the media content item on the network-connected television device, and the method is as follows: The television application further includes processing the playback of the media content item in cooperation with a media playback application associated with a media content source for the media content item and a local content casting application configured to play the media content item from the media content source in restricted mode, and the processing is Determining that the aforementioned playback is performed in the aforementioned restricted mode, Determining that the aforementioned restriction mode is not supported by the media playback application, A method comprising: determining that the restriction mode is not supported by the media playback application, the local content casting application requesting to play the media content item on the network-connected television device.

2. The method further includes determining that the user interface provides the restriction mode, The method according to claim 1, wherein the local content casting application requests playback of the media content item on the network-connected television device, the local content casting application requests playback of the media content item in the user interface.

3. In response to a user request to stop playback of the aforementioned media content item, To stop playback of the media content item on the network-connected television device, The method according to claim 2, further comprising presenting the content-based media recommendations in the user interface on the network-connected television device.

4. The collaborative processing of playback of the aforementioned media content items by a media playback application and a local content casting application is: Determining that the user interface does not provide the restriction mode, Determining that the playback of the media content item is performed in unrestricted mode, The method according to any one of claims 1 to 3, further comprising the media playback application requesting to play the media content item on the network-connected television device based on the determination that the playback is performed in the unrestricted mode.

5. The method according to any one of claims 1 to 4, wherein the local content casting application includes a receiver module corresponding to the media content source associated with the media playback application.

6. The method according to claim 5, wherein the local content casting application obtains the media content item from the receiver module.

7. The method according to any one of claims 1 to 6, wherein the television application is interfaced to a server-side television application included in a server system that hosts a virtual user domain including a user account associated with a user of the television application.

8. A program including executable instructions, wherein when the executable instructions are executed by at least one processor of a network-connected television device, the at least one processor causes the operation to be performed, and the operation is: The TV application presents content-based media recommendations in the user interface on the network-connected TV device, The television application includes receiving a selection of a media content item from the content-based media recommendations presented in the user interface, and the selection initiates playback of the media content item on the network-connected television device, and the operation is as follows: The television application further includes processing the playback of the media content item in cooperation with a media playback application associated with a media content source for the media content item and a local content casting application configured to play the media content item from the media content source in restricted mode, and the processing is Determining that the aforementioned playback is performed in the aforementioned restricted mode, Determining that the aforementioned restriction mode is not supported by the media playback application, A program that includes, based on the determination that the restriction mode is not supported by the media playback application, the local content casting application requesting to play the media content item on the network-connected television device.

9. The operation further includes determining that the user interface provides the restricted mode, The program according to claim 8, wherein the local content casting application requests the playback of the media content item on the network-connected television device, and the local content casting application requests the playback of the media content item in the user interface.

10. The aforementioned operation is, In response to a user request to stop playback of the aforementioned media content item, To stop playback of the media content item on the network-connected television device, The program according to claim 9, further comprising presenting the content-based media recommendations in the user interface on the network-connected television device.

11. The collaborative processing of playback of the aforementioned media content items by a media playback application and a local content casting application is: Determining that the user interface does not provide the restriction mode, Determining that the playback of the media content item is performed in unrestricted mode, The program according to any one of claims 8 to 10, further comprising the media playback application requesting to play the media content item on the network-connected television device based on the determination that the playback is performed in the unrestricted mode.

12. The program according to any one of claims 8 to 11, wherein the local content casting application includes a receiver module corresponding to the media content source associated with the media playback application.

13. The program according to claim 12, wherein the local content casting application obtains the media content item from the receiver module.

14. The program according to any one of claims 8 to 13, wherein the television application is interfaced to communicate with a server-side television application included in a server system that hosts a virtual user domain including a user account associated with a user of the television application.

15. It is a system, At least one processor, The system includes a non-temporary computer-readable medium storing instructions, and when the instructions are executed by the at least one processor, they are used in a television application. To present content-based media recommendations in the user interface on a network-connected television device, The user interface is configured to receive a selection of a media content item from the content-based media recommendations presented, and upon selection, playback of the media content item is started on the network-connected television device, and the television application is configured to perform the following actions: Further, the playback of the media content item is processed collaboratively by a media playback application associated with the media content source for the media content item and a local content casting application configured to play the media content item from the media content source in restricted mode, and the processing is performed by Determining that the aforementioned playback is performed in the aforementioned restricted mode, Determining that the aforementioned restriction mode is not supported by the media playback application, A system including the local content casting application requesting to play the media content item on the network-connected television device based on the determination that the restricted mode is not supported by the media playback application.

16. When the instruction is executed by the at least one processor, it causes the television application to determine that the user interface provides the restricted mode. The system according to claim 15, wherein the local content casting application requests playback of the media content item on the network-connected television device, and the local content casting application requests playback of the media content item in the user interface.

17. When the instruction is executed by the at least one processor, it is directed to the television application. In response to a user request to stop playback of the aforementioned media content item, To stop playback of the media content item on the network-connected television device, The system according to claim 16, further comprising the function of presenting the content-based media recommendations in the user interface on the network-connected television device.

18. The collaborative processing of playback of the aforementioned media content items by a media playback application and a local content casting application is: Determining that the user interface does not provide the restriction mode, Determining that the playback of the media content item is performed in unrestricted mode, The system according to any one of claims 15 to 17, further comprising the media playback application requesting to play the media content item on the network-connected television device based on the determination that the playback is performed in the unrestricted mode.

19. The local content casting application includes a receiver module corresponding to the media content source associated with the media playback application, The system according to any one of claims 15 to 18, wherein the local content casting application obtains the media content item from the receiver module.

20. The system according to any one of claims 15 to 19, wherein the television application is interfaced to a server-side television application included in a server system that hosts a virtual user domain including a user account associated with a user of the television application.

Images