Predictive Media Routing

Abstract

The present disclosure relates to predictive media routing. In some specific implementations, a user device may predictively route media content to a remote playback device based on playback scenario information obtained by the user device. The playback scenario may include local playback scenario information related to the state and / or scenario of the user device. The playback scenario may include remote playback scenario information related to the state and / or scenario of an available remote playback device. Based on the playback scenario information obtained by the user device, the user device may generate a prediction score for each available playback device, which indicates or predicts the possibility that the user wishes to send media content to a corresponding playback device. The user device may generate and present a graphical user interface, which may identify playback devices with a prediction score higher than a threshold score. In some instances, the user device may automatically route selected media content to a predicted playback device.

Description

[0001] This application is a divisional application of the invention patent application with application number 202010004938.6, application date January 3, 2020, and invention name “Predictive Media Routing”. Technical Field

[0002] The present disclosure generally relates to routing media content from a user device to a playback device. Background Art

[0003] Modern user devices (e.g., smart phones, tablet computers, laptop computers, etc.) generally include functions that allow users to present the media content selected by the user on the user device (e.g., local device) on a playback device (e.g., TV, set-top box, smart speaker, etc.). On some user devices, the user will need to provide a series of user inputs so that the user device sends the selected media content to the playback device. For example, the user will generally open a software application, select a media content item, select a play button to start playing the selected content item locally on the user device, select another button to indicate that the user wishes to stream or send the media content to the playback device, navigate in the list of available playback devices, and select the playback device so that the user device starts streaming or sending the selected content item to the selected remote playback device. This multi-input process for routing media to a playback device may be cumbersome, time-consuming and frustrating. Summary of the invention

[0004] In some specific implementations, user equipment can predictively route media content to a remote playback device based on playback context information obtained by the user equipment. The playback context may include local playback context information related to the state and / or context of the user equipment. The playback context may include remote playback context information related to the state and / or context of an available remote playback device. Based on the playback context information obtained by the user equipment, the user equipment can generate a prediction score for each available playback device, which indicates or predicts the possibility that the user wishes to send media content to a corresponding playback device. The user equipment can generate and present a graphical user interface, which can identify playback devices with a prediction score higher than a threshold score. In some instances, the user equipment can automatically route selected media content to a predicted playback device.

[0005] Specific implementations provide at least the following advantages. Media content can be routed to a remote playback device without requiring the user to select a remote playback device. When the user is prompted to select a playback device, the process of selecting a remote playback device can be made more efficient by presenting the playback device that the user is most likely to select in an easily accessible location. User privacy can be protected by collecting context information and generating a predicted score for each playback device entirely on the user device. Since the context information is not distributed outside the user device, unauthorized entities (e.g., people, businesses, etc.) cannot access the context information. Since the processes associated with predictive media routing (e.g., GUI creation, obtaining context information, playback device identification, playback device selection, etc.) are performed by the operating system of the user device, third-party applications do not have access to information associated with predictive media routing, which further enhances the user privacy protection provided by the disclosed system.

[0006] The details of one or more specific implementations are set forth in the following drawings and detailed description. Other features, aspects, and potential advantages will be apparent in the detailed description and drawings, and in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is an exemplary graphical user interface for automatically routing media content to predicted remote playback devices.

[0008] Figure 2 is an exemplary graphical user interface for intelligently presenting playback devices for user selection.

[0009] Figure 3 is a block diagram of an exemplary system for predictive media routing.

[0010] Figure 4 is a block diagram of an exemplary system for managing media sessions.

[0011] Figure 5 is a block diagram of an exemplary system for generating target playback device prompts.

[0012] Figure 6 is a flow chart of an exemplary process for automatically routing media content to a predicted remote playback device.

[0013] Figure 7 is a flow chart of an exemplary process for efficiently presenting available playback devices based on prediction scores.

[0014] Figure 8 is a flow chart of an exemplary process for routing different content items from a single computing device to different playback devices.

[0015] Fig. 9is a flow chart of an exemplary process for determining when to interrupt a remote playback session.

[0016] Fig.10 is a flow chart of an exemplary process for resuming a routing session based on location.

[0017] Fig.11 It is achievable Figure 1-Figure 10 A block diagram of an exemplary computing device illustrating features and processes of the invention.

[0018] Like reference symbols in the various drawings indicate like elements. DETAILED DESCRIPTION

[0019] Exemplary User Interface

[0020] Figure 1 1 is an exemplary graphical user interface (GUI) 100 for automatically routing media content to a predicted remote playback device. For example, the GUI 100 may be presented on a display of a user device (eg, user device 302 described below).

[0021] In some implementations, the GUI 100 may be generated by a software application (e.g., a media application) executed on a user device. The software application may be a first-party application created by a manufacturer of the user device and / or an operating system of the user device. The software application may be a third-party application created by a software developer different from the manufacturer of the user device and / or the operating system of the user device. In either case, the media routing process and user interface elements may be implemented by the operating system of the user device to provide consistent behavior between all applications executed on the user device. As an additional benefit, because the media routing process and user interface elements are provided by the operating system, user information used to perform the media routing process described herein is protected from disclosure to third-party applications, thereby enhancing user privacy protection.

[0022] In some implementations, the GUI 100 may be configured to enable a user to initiate playback of media content. For example, media content may include audio content, such as music, podcasts, audiobooks, radio content, talk shows, etc. Media content may include video content, which may or may not include audio content. Video content may include long-form video content, such as movies, television programs, commercially produced video content, or other content that exceeds a threshold time period. Video content may include short-form video content, such as amateur video content, social media video content, video content sent or received via an electronic messaging system (e.g., instant messaging, short messages, emails, etc.), etc.

[0023] In some implementations, the GUI 100 can include a content presentation area 102. For example, when presenting media content on the GUI 100, an application (e.g., a software application) can present video content in the presentation area 102. If the application is presenting video content, the video content can be presented in the area 102. If the application is presenting audio-only content, the application can present a representation of the content (e.g., artwork, album cover, artist drawing, etc.) in the area 102.

[0024] In some implementations, the GUI 100 may include playback control graphical elements. For example, the GUI 100 may include a fast forward control, a rewind control, and a play control 104 (e.g., a button). In some implementations, when the user interacts with the play control 104, the application will start playback and present the content selected by the user using a display of the user device and / or a speaker connected to the user device, depending on the type of media content.

[0025] However, in some specific implementations, user equipment may be configured to automatically select a playback device for presenting selected media content from a plurality of available playback devices. As used herein, a playback device may refer to a remote playback device, a local user device, or a playback device grouping (e.g., two or more playback devices). For example, a playback device grouping may be considered as a single playback device by system 300, or may be processed as a single playback device. If the playback devices included in the first playback device (e.g., group) and the second playback device (e.g., group) are not identical, then the first playback device is different from the second playback device.

[0026] In some specific implementations, the user device may generate a prediction score for each available playback device based on various criteria (e.g., scenario attributes, machine learning model features, historical playback data, etc.), which indicates the possibility that the user will want to present the media content selected by the user on the corresponding playback device, as described in detail below. The user device may then automatically select the playback device with the highest prediction score for presenting the content selected by the user. When the user device (e.g., operating system) automatically selects a playback device, an identifier (e.g., name) of the selected playback device may be presented on a GUI element 106 (e.g., a button or other graphical object). For example, the selected playback device may be a remote playback device, such as a set-top box, a television, a streaming media receiver, a smart speaker, or other device capable of receiving content from a user device. In some cases, the selected playback device may be a (e.g., local) user device.

[0027] When the playback device receives a user input (e.g., touch input, cursor input, etc.) selecting a play control 104 (e.g., a button) or a voice input command for initiating playback of the selected media content, the user device can automatically route playback of the media content to the automatically selected playback device identified on the graphical element 106 without the user providing any input to select a playback device. Thus, the content selected by the user can be presented to (e.g., streamed to, handed over to, etc.) a remote playback device that (e.g., historically) is a device preferred by the user without the user having to perform multiple inputs to navigate among the available remote playback devices and select a playback device for presenting the selected media content.

[0028] In some implementations, when sending media content to a remote playback device for presentation, the user device may stream the media content to the remote playback device. For example, during streaming, the user device may send incremental portions of selected media content to the remote playback device, and the remote playback device may reassemble the incremental portions of the selected media content and present the media content to the user via a display and / or speaker of the remote playback device.

[0029] In some specific implementations, when sending media content to a remote playback device for presentation, the user device can hand over the playback of the media content to the remote playback device by sending playback state information associated with the media content to the remote playback device. For example, the playback state information may include an identifier of the media content, an identifier of an application program that presents the media content to the remote playback device, and a current playback position in the media content. When the remote playback device receives the playback state information, the remote playback device may call an application program identified by an application program identifier, and provide the media content identifier and the current playback position to the application program, so that the application program can present the identified media content on the remote playback device from the current position identified in the media content. Therefore, when the playback of the media content is handed over to the remote playback device, the resources (e.g., battery power, processing cycles, memory resources, network resources, etc.) of the user device will not be consumed by streaming the media content to the remote playback device.

[0030] In some implementations, the GUI 100 may include a status bar 108. For example, the status bar 108 may be generated and / or presented by an operating system of the user device. The operating system may present conventional device status information in the status bar 108, such as the date, time, current battery level, etc.

[0031] In some implementations, the status bar 108 may include a graphical element 110 for indicating when the selected content item will be automatically routed to the remote playback device. For example, when the selected content item will be routed to the remote playback device, the graphical element 110 may be highlighted (e.g., colored with a specific eye-catching color, presented with a higher contrast, etc.). When the selected content item will be presented on the local user device, the graphical element 110 may be faded (e.g., colored with a specific color that will be integrated into the UI 100, presented with a lower contrast, etc.) or hidden. In general, the graphical element 110 may provide the same or similar features, interactions, and behaviors as described herein for the graphical element 106. However, the graphical element 110 may be presented in an area of ​​the GUI 100 controlled and / or generated by the operating system, while the graphical element 106 may be presented in an area of ​​the GUI 100 controlled and / or generated by an application on the user device.

[0032] In some specific implementations, status bar 108 can be modified to indicate that the selected content item will be automatically routed to a remote playback device. For example, user equipment 302 can change the color of status bar 108 (for example, changing status bar from gray to blue), so that when the user subsequently provides an input indicating that the user wants to play the selected content item, the selected content item will be automatically routed to a remote playback device. Alternatively, in response to the user providing the input of playing the selected content item, a colored banner can be displayed above status bar 108 or just below status bar 108 to indicate that the selected content item will be presented on a remote playback device. In response to the user calling a media application that meets the automatic routing qualification of the media item, status bar modification or banner display can be presented. Status bar modification or banner display can be presented for a period of time (for example, 5 seconds, 30 seconds, 1 minute, etc.). For example, after presenting for a period of time, status bar modification or banner display can be hidden, but leaving graphic element 110 to indicate to the user that any media item provided by the media application will be routed to a remote playback device.

[0033] In some implementations, graphical element 106 and graphical element 110 may be mutually exclusive. For example, when graphical element 106 is presented, graphical element 110 may not be presented. When graphical element 110 is presented, graphical element 106 may not be presented. Thus, when graphical element 110 is presented, the user may be notified that the selected content item will be routed to the remote playback device in a minimally intrusive manner; while still providing the automatic routing, user prompting, and user playback device selection behaviors described above and below in conjunction with graphical element 106.

[0034] Figure 2It is an exemplary graphical user interface 200 for intelligently presenting playback devices for user selection. For example, GUI 200 may correspond to the above-mentioned GUI 100. In some cases, users may view graphic elements 106 and determine that they do not want to present the selected media content on the playback device identified by graphic elements 106. Therefore, in some specific implementations, users may select graphic elements 106 so that user equipment presents graphic elements 202. In some cases, user equipment may not automatically select the playback device for presenting the content selected by the user with high confidence. Therefore, in some specific implementations, in response to receiving the user's selection of playback button 104, user equipment can automatically present graphic elements 202. For example, graphic elements 202 can be a playback device selector graphical user interface presenting the available playback devices detected by the user equipment. The user can provide input to graphic elements 202 to select the desired playback device for presenting the selected media content. In response to receiving user input, the user equipment can send the selected media content to the selected playback device for presentation.

[0035] In some cases, selecting a playback device may be a cumbersome or tedious task. For example, in a large office setting or in a home with many different playback devices, the user device may detect many different available playback devices (e.g., living room device, bedroom device, office device, etc.) and present a long list of playback devices on graphical element 202. In order to find the desired playback device, the user may need to browse through the long list of playback devices and may need to provide input to scroll through the long list of playback devices.

[0036] In order to make the user's interaction with the graphical element 202 and the GUI 200 more efficient, the user device may generate a prediction score for each available playback device (as described above and in more detail below), and order the available playback devices presented on the graphical element 202. For example, the playback devices may be ordered so that playback devices with higher prediction scores are listed in the most accessible locations on the graphical element 202. For example, the playback devices may be arranged from highest prediction score to lowest prediction score in a top-down, bottom-up, left to right, right to left, etc. scheme, or according to some other organizational scheme that makes it easy for the user to quickly view and find the playback device that the user is most likely to select.

[0037] Automatically select playback device

[0038] Figure 3is a block diagram of an exemplary system 300 for predictive media routing. For example, system 300 can be configured to predictively route media to playback devices based on prediction scores generated for playback devices available in system 300. The prediction score for each playback device can be generated based on historical user behavior (e.g., which playback device the user previously selected) and the current playback context (e.g., what media type was selected, the location of the user's device, what playback devices are available, the current state of the playback device, etc.). The prediction score can then be used to automatically select a playback device for presenting the content selected by the user (as described above with reference to FIG. 1 ). Figure 1 ), and / or when in a playback device selector user interface (e.g., Figure 2 When the playback devices are presented on the graphical element 202, the available playback devices are sorted according to the corresponding prediction scores.

[0039] In some specific implementations, system 300 may include user equipment 302. For example, user equipment 302 may be a computing device, such as a smart phone, tablet computer, laptop computer or other computing devices. User equipment 302 and / or the operating system of user equipment 302 may be configured to send media content (e.g., streaming or handing over) to various playback devices (e.g., playback device 330, playback device 340, playback device 350, etc.) that can be accessed by network 320 (e.g., etc.) to present. For example, network 320 may be a local area network, a Wi-Fi network, a Bluetooth connection, a peer-to-peer Wi-Fi, or other technologies for communicating between computing devices. Playback device 330, playback device 340 and playback device 350 may be configured to receive streaming and / handed over media content from user equipment 302 or other similar user equipment by network 320.

[0040] In some specific implementations, user device 302 may include application 304 and / or application 306. For example, application 304 may be configured to present long-format (e.g., movies, television, etc.) media content. For example, application 304 may be a software application specifically configured to facilitate streaming of movies and television programs to user device 302. Application 306 may be configured to present short-format (e.g., amateur videos, social media videos, etc.) media content. For example, application 306 may be a software application configured to provide access to a social media website, where a user of user device 302 is able to view and / or listen to media content created by other users of the social media website. For another example, application 306 may be a software application configured to facilitate electronic messaging (e.g., instant messaging, short messaging, email, etc.), which may include transmitting media content between different user devices. A user of user device 302 may interact with application 304 and / or application 306 to select a media content item and cause application 304 / application 306 to present GUI 100 and / or GUI 200 described above on a display of user device 302 .

[0041] Media playback session management

[0042] In some implementations, the user device 302 can include a media session manager 316. For example, the media session manager 316 can manage multiple simultaneous media sessions on the user device 302. By managing multiple simultaneous sessions, the media session manager 316 enables the user device 302 to simultaneously present multiple different media items on multiple different playback devices (e.g., local playback devices, remote playback devices, user devices, etc.).

[0043] Figure 4 4 is a block diagram of an exemplary system 400 for managing media sessions. For example, system 400 may correspond to system 300 described above. System 400 may include a media session manager 316. In some implementations, media session manager 316 may associate media applications (e.g., application 304, application 306) and / or media content with different types of media playback sessions. For example, media session manager 316 may manage long-form video sessions, long-form audio sessions, rich media sessions, and / or non-media sessions.

[0044] In some specific implementations, each media playback session can store the data defining the playback session. For example, the media playback session data can include the data of the media application associated with the playback session that recently presented the media content. The media playback session data can include the data of the playback target (e.g., local playback device, remote playback device, etc.) identifying the media playback session. In some specific implementations, if the user changes the playback target of the playback session, the media session manager 316 will terminate the current playback session and create a new playback session that routes the selected media item to the new playback target. The media playback session data can include the data of the type (e.g., long form video, long form audio, rich media, non-media, etc.) identifying the media playback session.

[0045] The media playback session data may include data identifying the time of occurrence of the last playback event associated with the media session. For example, the last playback event may be a playback termination event, a playback start event, a playback communication or transmission, a presentation of content, or other events associated with playing content on a playback device. In some implementations, the media session manager 316 may use the time of the last playback event to determine when to terminate the media playback session. For example, the media session manager 316 may automatically terminate the media playback session after a period of time (e.g., 3 hours) has passed since the last playback event. If a network connection has been established, the media playback session data may include data identifying the established network connection (e.g., Wi-Fi, Bluetooth, etc.) between the user device 302 and the identified playback device.

[0046] In some implementations, the media session manager 316 can manage media playback sessions associated with long-form video content and / or applications. For example, movies, television programs, etc. can be classified as long-form video content. Media applications that provide access to or are used to play movies, television programs, etc. can be classified as long-form video applications. When a user invokes (e.g., playback event 402) a long-form video media application (e.g., a movie application, a television program application, etc.) or selects (e.g., playback event 402) a long-form video media item to be played back, the media session manager 316 can create a new long-form video session (e.g., long-form video session 410) to manage the playback and / or presentation of the long-form video content.

[0047] In some implementations, the media session manager 316 can manage media playback sessions (e.g., long-form audio sessions 412) associated with music content and / or music applications. For example, music media items presented by a music-focused software application can be classified as long-form audio media content. Media applications that provide access to or are used to play commercial music can be classified as long-form audio applications. When a user invokes (e.g., playback event 402) a long-form audio media application or selects (e.g., playback event 402) a long-form audio media item to be played back, the media session manager 316 can create a new long-form audio session (e.g., long-form audio session 412) to manage the playback and / or presentation of the long-form audio content.

[0048] In some specific implementations, the media session manager 316 can manage media playback sessions (e.g., rich media sessions 414) associated with media content and / or applications that are not specifically focused on long-form video and / or long-form audio. For example, audio and / or video media items presented by a software application that focuses on text content enhanced by audio and / or video content will be classified in the rich media session category. For example, a news application that presents text news-related content and may include audio and / or video content to enhance news presentation can be classified as a rich media application that provides rich media content. General media applications such as web browsers can be classified as rich media applications because browsers are not specifically focused on providing or presenting long-form video and / or long-form audio content. For another example, social media applications are generally not specifically focused on presenting long-form video and / or long-form audio content, and therefore will be classified as rich media applications that provide rich media content. When a user invokes (e.g., playback event 402) a rich media application or selects (e.g., playback event 402) a rich media item to be played back, media session manager 316 can create a new rich media session (e.g., rich media session 414) to manage playback and / or presentation of the rich media content.

[0049] In some implementations, the media session manager 316 can manage media playback sessions associated with non-media audio and / or video (e.g., non-media sessions 416). For example, audio and / or video presented by a communication-focused software application (e.g., chat, instant messaging, video conferencing, phone, etc.) that focuses on real-time audio and / or video content can be managed using a non-media playback session. When a user invokes (e.g., playback event 402) a non-media application or initiates (e.g., playback event 402) a real-time audio and / or video communication, the media session manager 316 can create a new non-media session (e.g., non-media session 414) to manage the presentation of the non-media content.

[0050] Multiple simultaneous playback sessions

[0051] In some specific implementations, playback sessions can be configured to present media content items simultaneously. For example, long-form video sessions 410, long-form audio sessions 412, rich media sessions 414, and non-media sessions 416 are examples of different types of media sessions. A user can provide input (e.g., select a movie to be played) to initiate a long-form video session 412 on a user device 302. A user can provide input (e.g., select a song to be played) to initiate a long-form audio session 410 on a user device 302. As described herein, a user can provide input to a user device 302 to specify a target playback device (e.g., user device 302, remote playback device, etc.) for media content associated with a playback session. If the target playback device (e.g., playback device 340) for a long-form video session 410 is different from the target playback device (e.g., playback device 330) for a long-form audio session 412, the user device 302 can cause the corresponding long-form video playback session 410 and the long-form audio playback session to play their corresponding media items simultaneously on the corresponding target devices. For example, user device 302 may cause a selected movie to be presented on playback device 340 while simultaneously causing a selected song to be presented on playback device 330 .

[0052] Interrupt a playback session

[0053] In some implementations, playing the selected content item may interrupt the current playback session. For example, once a playback device is selected for a particular type of playback session, media applications and / or content associated with the particular type of playback session may be routed to the selected playback device for the duration of the playback session, or until the user selects a different playback device. Thus, if a user of user device 302 selects playback device 360 ​​for presenting media content associated with rich media session 414, all media content provided by rich media applications will be routed to playback device 360 ​​for presentation to the user until rich media session 414 terminates or until the user selects a different playback device for rich media session 414.

[0054] In some implementations, the media session manager 316 manages only one playback session of a particular session type at a time. For example, if a long form video session 410 is in progress and the user device 302 receives user input selecting a new long form video application or long form video content item, the media session manager 316 will not create a new long form video playback session. Instead, the media session manager 316 may interrupt playback of the media items corresponding to the current long form video session 410, reconfigure the long form video session 410 based on the newly selected long form video application and / or long form video content item, and then initiate playback of the new long form video content item on the same playback device that previously presented the previous long form video content item.

[0055] In some implementations, the media session manager 316 can manage multiple playback sessions of a particular type at once. For example, if a long form video session 410 is in progress and the user device 302 receives user input selecting a new long form video application or long form video content item, the media session manager 316 can create a new long form video playback session and manage two active long form video playback sessions simultaneously. For example, the media session manager 316 can route the long form video content of a first long form video playback session to one playback device for presentation of the content while routing the long form video content of a second long form video playback session to a different playback device for presentation of the content. Thus, the user device 302 can cause two different playback devices to simultaneously present different content associated with the same playback session type.

[0056] In some instances, a first playback session may not be an active session (e.g., not playing or routing content, but the session has not been terminated), while a second playback session may be an active session in which content is routed to and rendered by a playback device. Thus, there may be multiple playback sessions of the same type, but only one session may be an active playback session.

[0057] In some implementations, playing the selected content item associated with the first playback session type may interrupt the current playback session corresponding to the second playback session type. For example, if the long-form audio content item associated with the long-form audio session 412 is currently being routed to the playback device 340, and the user initiates a long-form video session 410 configured to route video playback to the playback device 340, the user device 302 may interrupt the long-form audio playback session 412 so that the video playback session 410 may present the long-form video on the playback device 340. However, in some instances, it may be preferred to avoid interrupting the playback of the long-form audio session 412 on the playback device 412. For example, if the user has chosen to initiate a rich media session by selecting a rich media item (rather than a long-form video) provided by a social media application, the user may not wish to interrupt the playback of the long-form audio item with the presentation of the rich media item, but may prefer to use the rich media item locally on the user device 302. Therefore, user equipment 302 can be configured to have the following standard: can be evaluated to determine when to play media content items should or should not interrupt the current playback of another content item.In some specific implementations, media session manager 316 can be configured to have various rules or standards, and these rules or standards are used to determine when to interrupt the presentation of the media content items associated with the media playback session.For example, the media content items to be played back selected by the user of user equipment 302 will be played on a remote playback device or on a local playback device (for example, user equipment 302, and / or a personal listening device connected to user equipment 302, such as headphones, earphones, earplugs, etc.).If after assessing the standard described below, media session manager 316 determines that the playback of the content items on the remote device should not be interrupted by the playback of the content items newly selected, then media session manager 316 can make the playback of the content items newly selected be presented on the local playback device.If the local playback device has presented the media items, the currently presented media items can be interrupted so that the media items newly selected can be presented to the user according to the wishes of the user.

[0058] In some implementations, the remote playback interruption criteria may include whether the selected media item is provided by a media application included in a media application blacklist. For example, the media session manager 316 may be configured with an application identifier blacklist that identifies media applications that should never interrupt a remote playback media session or media content item. When determining whether to interrupt a remote media playback session, the media session manager 316 may compare the application identifier of the media application that provides the newly selected media item with the application identifier in the blacklist. If the application identifier is included in the blacklist, the remote playback media session will not be interrupted, and the newly selected media item may be played locally on the user device 302. If the application identifier is not included in the blacklist, the remote playback media session may be interrupted (e.g., pausing evaluation of other criteria described herein), and the newly selected media item may be presented on the remote playback device.

[0059] In some specific implementations, the remote playback interruption criteria may include whether the selected media item is provided by a media application included in a media application whitelist. For example, the media session manager 316 may be configured to have an application identifier whitelist that can identify media applications that should always interrupt remote playback media sessions or media content items. When determining whether to interrupt a remote media playback session, the media session manager 316 may compare the application identifier of the media application that provides the newly selected media item with the application identifier in the whitelist. If the application identifier is included in the whitelist, the remote playback media session will be interrupted, and the newly selected media item will be played on the remote playback device. If the application identifier is not included in the whitelist, the remote playback media session may or may not be interrupted (e.g., pausing to evaluate other criteria described herein), and the newly selected media item may or may not be presented on the remote playback device.

[0060] In some specific implementations, the remote playback interruption criteria may include whether the selected media item is provided by a media application that is determined to be an important media application. For example, the media session manager 316 may determine whether the media application is important based on various important tags. An example of an important indicator is whether the media application that provides the selected media item participates in operating system-level user interface features, such as playback status and control user interface provided by the operating system of the user device 302 and / or remote control functions provided by the user device 302. A second example of an important indicator is whether the media application has the right to present audio content when running as a background application. If any one of these tags is observed (for example, the media application participates in operating system functions and / or has the right to present audio content when running as a background application), the remote playback media session can be interrupted and the newly selected media item can be played on the remote playback device.

[0061] In some implementations, the remote playback interruption criteria may include whether the media application providing the selected media item has indicated that the media application is a long-form video application and / or a long-form audio application. For example, the media application may be configured to interact with an application programming interface of the operating system of the user device 302 to provide information indicating that the media application is a long-form video application and / or a long-form audio application. If the media application is a long-form video application and / or a long-form audio application, the remote playback media session may be interrupted and the newly selected media item provided by the media application may be played on the remote playback device.

[0062] In some implementations, remote playback interruption criteria may be evaluated based on a priority or hierarchy of criteria. For example, the results of a whitelist evaluation may take precedence over the results of a blacklist evaluation. A blacklist evaluation may take precedence over an application importance evaluation.

[0063] In some implementations, the whitelist can be configured to include long-form video applications and / or long-form audio applications, including applications using the above-mentioned application programming interface. In some implementations, the whitelist can be dynamically generated based on historical user behavior patterns that indicate which media applications should interrupt remote playback of media items provided by other media applications.

[0064] In some implementations, user input overrides all rules and criteria for interrupting remote playback. For example, if a user provides user input that explicitly indicates that the user wants a media content item to be presented on a specific remote playback device, the media session manager 316 will configure the corresponding playback session to route the selected content item to the specified remote playback device, regardless of which media content item is currently playing on the specified remote playback device and will be interrupted.

[0065] In some implementations, application 306 can be associated with a higher level of privacy protection than application 304. For example, when application 306 corresponds to a social media application, an electronic instant messaging application, etc., the media content processed by application 306 may be of a more personalized nature than the media content processed by application 304, which processes professionally produced long-form media content that is accessible to a wider public. The privacy protection level associated with the application that presents the user-selected media content can be considered when automatically routing the media content to a playback device, as further described below.

[0066] Rating playback devices

[0067] In some implementations, the user device 302 may include a scoring module 308. For example, the scoring module 308 may be configured to generate a prediction score for playback devices that the user device 302 can access. In some implementations, the scoring module 308 may be included in an operating system (OS) of the user device 302. The scoring module 308 may be configured as a rule-based system that compares playback context data (described below) with a set of rules for determining prediction scores for various playback devices. The scoring module 308 may be configured as a machine learning model that processes the playback context data to generate prediction scores for various playback devices.

[0068] In some implementations, in response to a user selecting a media content item to be played back, the scoring module 308 can generate a predicted score. For example, because the scoring process can take into account the type of media content that the user has selected to be played back, the scoring process can be performed after the user selects the media content item to be played back, but preferably before the user selects to initiate playback of the media content item.

[0069] Prior to generating a predicted score, the scoring module 308 may obtain information identifying playback devices currently available on the network 320. For example, playback device 330, playback device 340, playback device 360 ​​may periodically broadcast messages indicating (e.g., announcing) their availability. These messages may be broadcast using peer-to-peer communication technology (e.g., Bluetooth, peer-to-peer Wi-Fi, etc.). The message may include information that a receiving device (e.g., user device 302) may use to establish a connection with the broadcasting playback device via the network 320. The user device 302 may store information indicating which playback devices may be used or accessed via the network 320. When generating a predicted score for a playback device, the scoring module 308 may obtain information indicating which playback devices may be used or accessed via the network 320.

[0070] In some implementations, the scoring module 308 can generate a predicted score for the available playback devices. In some implementations, the available playback devices can include local playback devices (e.g., user device 302). Therefore, the scoring module 308 can generate a predicted score for the remote playback device 330, the remote playback device 340, the remote playback device 360, and / or the user device 302.

[0071] Scoring module 308 can generate prediction scores based on scoring attributes corresponding to playback scenario data and / or historical user behavior patterns. When implemented as a rule-based scoring system, scoring module 308 can assign a single score to each scoring attribute, and then combine the scores into an overall prediction score. When implemented as a machine learning model, the scoring attribute can correspond to the features used to create the machine learning model. When the user selects the content item to be played back, the value of the scoring attribute can be fed back into the machine learning model based on the overall analysis of the current playback scenario according to historical user behavior to generate a prediction score for each playback device. In some specific implementations, the machine learning module can generate a playback device list arranged in order from the user's most likely selection to the user's least likely selection, without associating each playback device with a prediction score. For example, when media content is automatically routed to a playback device, user device 302 can select the most likely playback device from a sorted list. When a playback device is presented for user selection, a sorted list of playback devices can be presented so that the user can first view the most likely playback device.

[0072] Scoring playback devices based on context

[0073] In some implementations, the scoring attributes may include context attributes corresponding to the current playback context. The current playback context may include a local playback context associated with the user device 302. In some implementations, the local playback context of the user device 302 may include local context attributes of the user device 302. For example, the local context attributes may be context attributes, context data, etc. that the user device 302 does not have to obtain from other playback devices.

[0074] Local playback scenario

[0075] In some implementations, the local context attributes may include features of the media content item selected by the user. In some implementations, the features may include whether the media content item is a long-form content item or a short-form content item. For example, the user may be more likely to send the long-form content item to the remote playback device, so when the long-form content item is selected, a higher prediction score may be generated for the remote playback device than for the local playback device. Conversely, the user may be more likely to play the short-form content item on the local playback device, so when the short-form content item is selected, a higher prediction score may be generated for the user device 302 than for the remote playback device.

[0076] In some specific implementations, content item features may include the type of media content selected by the user. For example, the type of media content may include audio / video content (e.g., movies, TV shows, short videos, etc.). When selecting audio / video content items, scoring module 308 may generate a lower prediction score than a playback device (e.g., smart TV, set-top box, etc.) that can present audio and video for playback devices that can only present audio (e.g., smart speakers). For another example, the type of media content may include pure audio content (e.g., music, podcasts, audiobooks, radio broadcasts, etc.). When selecting pure audio content items, scoring module 308 may generate a higher prediction score than a playback device (e.g., smart TV, set-top box, etc.) that can present audio and video for playback devices that can only present audio (e.g., smart speakers).

[0077] The content item features may include ratings associated with the selected media content items. For example, the ratings may include parental guidance ratings indicating content maturity, whether the content includes descriptions of violence, sex or other scenes, language, etc. for mature audiences. For example, content item rating information combined with the time of day may affect the prediction score generated by the scoring module 308. For example, content items with adult content selected in the evening (e.g., after 9 p.m.) may result in a remote playback device obtaining a higher prediction score than adult content items selected in the morning or at noon, because children may be awake and close to a remote playback device in the morning or at noon. Content items rated as suitable for children (e.g., G-rated, PG-rated, etc.) selected during the day (e.g., before 6 p.m.) may result in a remote playback device obtaining a higher prediction score than content items rated as suitable for adults selected during the period of the day when children are able to watch and / or hear adult content presented by the remote playback device.

[0078] Content item features may include the type of content item. Similar to the discussion above on ratings, some types (e.g., cartoons, children's educational television programs, etc.) may be more suitable for children, while some types (e.g., action movies, TV series, etc.) may be more suitable for adults. Type combined with other scoring attributes (e.g., time of day) and / or historical playback behavior may result in a higher or lower predicted score for a playback device.

[0079] The content item features may include content item identifiers. For example, the user device 302 may store historical data (e.g., historical data 310) that is used to identify various scoring attributes and historical user behaviors associated with sending media content from the user device 302 to the remote playback device. If the historical data indicates a pattern of sending content items with specific identifiers (e.g., TV show titles, movie titles, etc.) to the remote playback device, when the user selects a content item with the specific identifier, the scoring module 308 may generate a higher predicted score for the remote playback device than for the local user device 302. The use of historical data (e.g., historical user behavior data) to generate predicted scores is discussed in more detail below.

[0080] Content item features may include the resolution of the selected content item. For example, a user may be more likely to send high-resolution content (e.g., 4k video, high-definition audio, etc.) to a remote playback device that can accurately render the high-resolution content, rather than playing the high-resolution content on a user device that may have a small display and small speakers. Therefore, for a playback device that can handle high-resolution content, selecting high-resolution content may result in a higher prediction score.

[0081] The content item feature may include the source of the selected media item. For example, the source may correspond to the application program through which the user selects the media item. The source may correspond to the network resource (e.g., network server, website, etc.) from which the selected media item is obtained. For example, when the source is a short-form media source (e.g., a video sharing website or application program) or a media source (e.g., a social media website or application program) with more user personal properties or private properties, the scoring module 308 may generate a prediction score that is less than the local playback device for the remote playback device, because the user may be unlikely to wish to present content from these types of sources on the remote playback device where other people can see or hear the content. For example, the application program 304 and / or the application program 306 may be associated with a privacy level indicating the personal properties of the content provided by the application program. When the privacy level is high (e.g., the application program provides content with a high degree of personal properties), the scoring module 308 may generate a prediction score that is conducive to presenting content on the local user device 302. When the privacy level is low (e.g., the application provides content that is not of a user-personal nature), then the scoring module 308 may generate a predicted score that favors presenting the content on the remote playback device 330, remote playback device 340, or remote playback device 360. When the source is a long-form media source (e.g., a television show or movie streaming website or application) or a professionally produced media source (e.g., a music streaming website or application), the scoring module 308 may generate a higher predicted score for the remote playback device than for the local playback device. Of course, the predicted scores generated for the playback devices based on these content item features may be adjusted up or down based on user history data indicating whether the user typically streams content from these sources to various playback devices.

[0082] In some implementations, the local context attribute may include a location of the user device 302 determined by the user device 302. For example, the location may include a geographic location, such as the user's home, office, school, etc. The geographic location may include grid coordinates, latitude / longitude location, etc.

[0083] In some specific implementations, the location of the user device 302 may include a fixed-point location identified or defined by a combination of signals received by the user device 302. For example, these signals may include radio frequency signals (e.g., Wi-Fi signals, Bluetooth signals, etc.), sources of these radio signals (e.g., Wi-Fi access points, Bluetooth broadcasting devices, media playback devices, etc.), and / or received signal strengths of these signals. These signals may include audio signals detected by a microphone of the user device 302. For example, different locations, rooms, buildings, etc. have different ambient noise and / or acoustic characteristics. When the user device 302 detects the ambient noise and / or acoustic characteristics of a location, the ambient noise and / or acoustic characteristics may be used to define and / or identify the fixed-point location. Other signals that define the fixed-point location may include user activities performed at the location. For example, a user may typically send media content to a specific playback device (e.g., playback device 360) at or near a specific fixed-point location. The user device 302 may combine the above-mentioned various signals to define a fixed-point location or determine whether the user is located at a previously defined fixed-point location. The spot location may then be used to determine predicted scores for various playback devices based on the proximity of the user device 302 to the playback device (e.g., a device in the same room, a device in the same building, a user device located in front of or near the playback device, etc.), the user device 302's previous historical use of the playback device at that spot location, and / or based on some other combination of scoring attributes as described herein.

[0084] In some implementations, the scoring module 308 can evaluate the location of the user device 302 in conjunction with historical user behavior. For example, if the user previously sent media content from the user device 302 to the playback device 340 when located at the current fixed-point location, the scoring module 308 can generate a higher predicted score for the playback device 340 than for other playback devices. If there is a historical pattern indicating that the user typically sends media content to the playback device 360 ​​when the user device 302 is at or near a particular location, the scoring module 308 can generate a higher predicted score for the playback device 360 ​​than for other playback devices.

[0085] In some implementations, the local context attribute may include a current time context attribute. For example, the current time context attribute may include the current time of day when the scoring module generates the prediction score, the current day of the week when the scoring module generates the prediction score, or a combination thereof.

[0086] The current time context attribute may include the time of day, regardless of the day of the week or date. For example, the user of the user device 302 may have a home gym in a room at home. The user may typically exercise at 6 a.m. every day, and the user sends media content from the user device 302 to the playback device 340 while exercising. This pattern based on the time of day may be determined by the scoring module 308 and used to generate a higher predicted score for the playback device 340 at 6 a.m. than at other times of the day.

[0087] The current time scenario attribute may include a day of the week (e.g., Monday, Tuesday, etc.), regardless of the time of day. For example, the user of the user device 302 may usually send music to the playback device 330, playback device 340, and playback device 360 ​​(e.g., the entire house) when relaxing at home on the weekend. The scoring module 308 may determine the pattern based on the day of the week, and generate a higher prediction score for playback device 330, playback device 340, and playback device 360 ​​on Saturdays and Sundays than on other days of the week. In some specific implementations, the scoring module 308 may identify a user pattern that groups playback device 330, playback device 340, and playback device 360 ​​together for media playback on weekends, and automatically generate a playback group including playback device 330, playback device 340, and playback device 360. In this case, the scoring module 308 may generate a prediction score for the playback group as a single playback device.

[0088] The current time context attribute may include a combination of time and date. Continuing the example above, the user may only exercise on Monday to Friday and rest on Saturday and Sunday. Therefore, at 6 am on Monday to Friday, the scoring module 308 may generate a higher predicted score for the playback device 340 than on Saturday and Sunday.

[0089] In some specific implementations, local context attributes may include current network attributes. For example, user device 320 may be connected to or may detect various networks near user device 320. When the user is at home, these networks may include the user's home wired or wireless network. When the user is at home, these networks may include the user's neighbor's home wireless network. When the user is at work, these networks may include various wired or wireless office networks. User device 320 may detect signals (e.g., Wi-Fi signals, Ethernet signals, etc.) generated by the network, and determine the corresponding network identifier and received signal strength based on these signals. Based on the network information, user device 320 may determine the location of user device 320 and / or determine the historical user playback behavior corresponding to the combination of network signals detected by user device 320 when scoring module 308 is generating a predicted score for playback device. For example, scoring module 308 may determine that a specific combination of network signals associated with a network identifier and / or received signal strength is related to past user behavior of sending media content from user device 302 to playback device 330. Therefore, scoring module 308 may generate a higher predicted score for playback device 330 than other playback devices.

[0090] In some specific implementations, local context attributes may include scheduled activity attributes. For example, user device 320 may include a calendar application through which a user may schedule and reserve times for various events, meetings, appointments, etc., and events scheduled in the calendar application may indicate the time and / or location at which the event will occur. When the current time corresponds to the scheduled time of the event, scoring module 308 may determine the location of the event and generate prediction scores for various playback devices based on the location of the event. For example, the meeting location may be a conference room including a remote playback device 340. If the user previously sent media content from user device 302 to remote playback device 340 while in the conference room, scoring module 308 may determine that user device 302 may be in the conference room based on the location in the calendar entry, and the user may wish to send media content to remote playback device 340. Therefore, scoring module 308 may generate a higher prediction score for playback device 340 than other available playback devices.

[0091] Remote playback scenario

[0092] The current playback scenario may include a remote playback scenario associated with each available playback device. For example, the scoring module 308 may determine a remote playback device state associated with each of the available playback devices 330, 340, and / or 360, respectively, and determine the remote playback scenario based on the remote playback device state. The scoring module 308 may then generate a predicted score for each playback device based on the current remote playback scenario.

[0093] In some specific implementations, user equipment 302 can obtain the current playback state of each available remote playback device. For example, user equipment 302 can send a message to playback device 330, playback device 340 and / or playback device 360 ​​through network 320, requesting the current playback state of playback device. Each of playback device 330, playback device 340 and playback device 360 ​​can send its corresponding playback state to user equipment 302 in state message 332, state message 342 and state message 362 respectively. Then, scoring module 308 can determine the remote playback device scenario based on the playback state information received from each playback device. For example, the state information received from each playback device and / or remote playback scenario can correspond to the remote scenario attribute described below.

[0094] In some implementations, the remote context attributes may include the capabilities of each available playback device. For example, some playback devices may be able to render audio and video. Some playback devices may only be able to render audio. Some playback devices may be able to render high-definition video and / or audio, while some playback devices may only be able to render low-definition media. Some playback devices may be able to participate in a playback device group, while some playback devices may only be able to operate individually. These playback device capabilities may affect the predicted scores generated for each device. For example, if a user chooses to play a high-definition movie, a playback device that can render high-definition video will receive a higher predicted score than a playback device that can only render low-definition video or only render audio.

[0095] In some specific implementations, the remote scenario attribute may include an attribute describing the current playback state of the remote playback device. For example, the remote device playback state may indicate which playback devices are currently presenting content, and which playback devices are not currently presenting content. For example, some playback devices may currently be presenting media content (e.g., actively), while some playback devices may be idle, in low-power mode, or closed. For the playback device currently in use, the scoring module 308 may generate a lower prediction score than the playback device currently unused, because the user of the user device 302 may be less likely to want to interrupt the presentation of content on the playback device currently presenting content.

[0096] The current playback state can identify the source of the content being played on the playback device. For example, the media content presented by the playback device can be started or initiated on the playback device using the application installed on the playback device. Alternatively, the media content presented by the playback device can be started or initiated on another user device (for example, user device 370) of the playback device (for example, playback device 330) of the streaming media content or the playback of the media content has been handed over to. In some specific implementations, the scoring module 308 can generate a score lower than the playback device receiving content from another user device for the playback device that is presenting content when starting content playback on the playback device. For example, if content playback is initiated on playback device 330, the user of playback device 330 may be more likely to currently participate in enjoying (for example, watching, listening, etc.) playback device 330, and the user of user device 302 may not wish to interrupt the enjoyment of other users to playback device 330. In some implementations, the scoring module 308 can generate a higher score for a playback device that is presenting content when playback of the content is initiated on the playback device than for a playback device that received the content from another user device.

[0097] The current playback state may identify the source and / or type of media content currently presented by the playback device. For example, the source may be an application running on the playback device that provides access to the media content. The source may be a network resource (e.g., a website, a server, etc.) that streams media content externally. The type of media content may be short-form content or long-form content, as described above. For example, when the source is a short-form media source (e.g., a video sharing website or application), the scoring module 308 may generate a higher predicted score for a remote playback device that plays short-form content than a playback device that presents long-form content, because the user of the user device 302 may be more likely to interrupt the playback of short-form content (e.g., short amateur videos, social media content, etc.) presented on the playback device. When the source is a long-form media source (e.g., a TV show or movie streaming website or application) or a professionally produced media source (e.g., a music streaming website or application), the scoring module 308 may generate a lower predicted score for a remote playback device than a playback device that presents short-form content, because the user of the user device 302 may be less likely to want to interrupt another person's enjoyment of a movie, TV show, etc.

[0098] The current playback state can identify the type of output provided by the content item presented by the playback device. For example, the type of output can include pure video, pure audio, or audio and video output. Scoring module 308 can generate a predicted score for the playback device by comparing the output type currently presented by the playback device with the output type provided by the content item selected by the user of user device 302. For example, if the user of user device 302 has selected a movie with audio and video output and the playback device is presenting pure audio output, scoring module 308 can generate a higher predicted score for the playback device, because the user of user device 302 may be more willing to interrupt the pure audio playback with a movie that provides audio and video output. If the user of user device 302 has selected a song with only audio output, and the playback device is presenting audio / video output (e.g., movie, TV show, etc.), scoring module 308 can generate a lower predicted score for the playback device, because the user of user device 302 may be less willing to interrupt the audio / video content being presented by the playback device. Thus, the scoring module 308 can be configured with a hierarchy or ranking for content output types (e.g., audio-only, video-only, audio / video, etc.) that can indicate when a selected content item with a particular content output type can or should interrupt a currently playing content item with a different content output type. For example, audio-only content can interrupt video-only output, while audio / video content can interrupt audio-only content. The hierarchy or ranking of output types can affect how the scoring module 308 generates a predicted score for each playback device, as described above.

[0099] Historical user behavior

[0100] As described above, in some implementations, historical user playback behavior (e.g., historical data) may affect the predicted score generated by the scoring module 308. For example, each time a user selects and plays a content item, the user device 302 may store playback context data (e.g., context attribute values) obtained when the user plays the content item in the history database 310. The playback context attributes may correspond to the context attributes described above (e.g., local, remote, etc.). For example, the context attributes may include an identifier of the media application that provided the content item, an identifier of the selected playback device, the location of the user device when the content item was selected and / or played, the amount of time (e.g., duration) that the user used the selected content item, and / or the day of the week on which the content item was played.

[0101] In addition, the user device 302 may store the identification of the playback device selected by the user for presenting the selected content item. For example, if the user chooses to play the content item on the user device 302 (e.g., by not selecting a remote device, by selecting a local device), the user device 302 may store information identifying the local playback device and associate (e.g., map to) the playback context data obtained and stored for the current content playback. If the user chooses to send the playback of the selected content item to a remote playback device, the user device 302 may store the identifier of the remote playback device in the history database 310 and associate it with the playback context of the current content playback.

[0102] In some implementations, the scoring module 308 can use the historical playback data 310 to determine behavioral patterns (e.g., usage patterns) associated with various playback devices available in the system 300 (e.g., including the user device 302). These usage patterns can allow the scoring module 308 to generate higher scores for playback devices that are more likely to be used in specific playback scenarios by the user of the user device 302. For example, if the user's historical behavior indicates that the user typically presents media content on the playback device 330 in a specific playback scenario (e.g., when a specific combination of context attribute values ​​is present), and the specific playback scenario corresponds to or nearly corresponds to the current playback scenario, the scoring module 308 can generate a higher predicted score for the playback device 330 than for other playback devices available through the network 320.

[0103] In some implementations, the scoring module 308 can analyze the historical playback data 310 to determine the most recently used playback device and / or the most frequently used playback device. For example, the scoring module 308 can determine the most recently used playback device and / or the most frequently used playback device, and generate a higher prediction score for the most recently used playback device and / or the most frequently used playback device than for other less frequently used playback devices.

[0104] When the scoring module 308 is configured as a rule-based scoring module, historical patterns can influence how scores are generated for each context attribute. When the scoring module 308 is configured as a machine learning module, historical data 310 can be used to train a machine learning model to improve the predicted scores generated by the machine learning model. Historical usage patterns will influence the machine learning model so that the machine learning module will generate higher predicted scores for playback devices that users typically use in specific playback contexts.

[0105] In some implementations, the influence of historical data can decrease over time. For example, the scoring module 308 can generate prediction scores such that recent user behavior has a greater influence on the prediction score than earlier user behavior. Attenuation (e.g., aging) of earlier behavior data can be achieved by weighting the scores generated for each context attribute to give more weight to recent behavior than to earlier behavior. Attenuation of earlier behavior data can be achieved by retraining the machine learning model using more recent historical data 310.

[0106] Disqualify playback device

[0107] In some implementations, a remote playback device may be disqualified from participating in automatic playback selection. For example, scoring module 308 may be configured with rules that indicate when a remote playback device should be disqualified from participating in automatic selection for playback of media content. When a remote playback device is disqualified, scoring module 308 may generate a very low prediction score for the remote playback device so that the remote playback device will not be automatically selected to play back media content.

[0108] In some implementations, the disqualification rule may be based on whether the user has previously selected to send media content from the user device 302 to a remote playback device. For example, if the user always plays content selected on the user device 302 on the user device 302 and has never sent content from the user device 302 to any playback device, all remote playback devices may be disqualified from receiving media content from the user device 302. For another example, if the user has never sent any content from the user device 302 to a specific remote playback device (e.g., playback device 340), the specific remote playback device may be disqualified from receiving media content from the user device 302.

[0109] In some implementations, the disqualification rules may be based on the functionality of the remote playback device. For example, if the user has selected to play content with audio and video output and the remote playback device 360 ​​is only capable of presenting audio content, the remote playback device 360 ​​may be disqualified from participating in the automatic selection.

[0110] In some implementations, the disqualification rule may be based on the current playback state of the remote playback device. For example, if the remote playback device is currently playing media content or a specific type of media content, the remote playback device may be disqualified from participating in the automatic selection.

[0111] In some implementations, the scoring module 308 may classify playback devices based on the prediction score. For example, after the scoring module 308 generates a prediction score for each playback device (e.g., user device 302, playback device 330, playback device 340, and playback device 360) based on the playback context attributes and / or the historical data 310, the scoring module 308 may classify the playback devices into a high-scoring category, a medium-scoring category, and a low-scoring category. For example, the category may represent a confidence level (e.g., high, medium, low, etc.), corresponding to the confidence level of the scoring module 308 that the user will want the selected content item to be presented on the corresponding playback device. Playback devices with a prediction score above a first threshold (e.g., 50, 70, 81, etc.) may be grouped into a high category, corresponding to playback devices to which the user is very likely to send the selected content item. Playback devices with predicted scores below the first threshold but above a second threshold (e.g., 30, 45, etc.) may be grouped into a medium category, corresponding to playback devices to which the user may wish to send the selected content item. Playback devices with predicted scores below the second threshold may be grouped into a low category, corresponding to playback devices to which the user is unlikely to send the selected content item.

[0112] Automatically select playback device

[0113] In some specific implementations, user equipment 302 can automatically select playback devices based on the classification of playback devices. In some specific implementations, user equipment 302 will only automatically select playback devices for long-form video playback sessions. In other specific implementations, user equipment 302 can automatically select playback devices for long-form video sessions, long-form audio sessions, rich media sessions and / or non-media playback sessions. For example, after scoring module 308 generates a prediction score for each playback device (e.g., user equipment 302, playback device 330, playback device 340, playback device 360, etc.) and classifies each playback device, scoring module 308 can automatically select the playback device to which the content item selected by the user is sent. In some specific implementations, scoring module 308 can select playback devices from playback devices belonging to high-class playback devices. Since the prediction score threshold of high-class is a very high value, few (if any) playback devices will belong to high-class. If a playback device belongs to high-class, scoring module 308 can automatically select the playback device. If multiple playback devices belong to the high category, the scoring module 308 can automatically select the high category playback device with the highest predicted score. If no playback device belongs to the high category, the scoring module 308 can select the local playback device (e.g., user device 302) by default, or cause the user interface (UI) module 312 to prompt the user to select a playback device, as further described below.

[0114] When the scoring module 308 automatically selects a playback device, the scoring module 308 may send the identifier of the selected playback device to the UI module 312, and may also send the identifiers of all available playback devices and the corresponding predicted scores. Then, the UI module 312 may generate a graphical user interface (e.g., GUI 100, GUI 200) with which the user may interact to initiate playback of the selected content item, as described above. For example, the UI module 312 may generate the GUI 100 and present the identifier of the selected playback device (e.g., remote playback device 340) on the graphical element 106. Therefore, when the user selects the play button 104, the selected content item may be automatically routed to the automatically selected playback device (e.g., remote playback device 340), without the user providing other input to select the playback device.

[0115] If the user wishes to select a playback device different from the automatically selected playback device, the user may select graphical element 106 to cause the available playback devices to be presented on graphical element 202, as described above. In some implementations, this user behavior may be recorded in historical data and may be used to influence future scoring of automatically selected playback devices. For example, this behavior may cause scoring module 308 to give a lower score to the automatically selected playback device and / or give a higher score to the user selected playback device. The high category playback devices and mid category playback devices presented on graphical element 202 may be grouped and / or sorted on graphical element 202 according to their respective predicted scores. The low category playback devices presented on graphical element 202 may be grouped and / or sorted in alphabetical order, such as Figure 2 shown.

[0116] Sort playback devices to render

[0117] In some embodiments, the user device 302 may prompt the user to select a playback device. For example, if no playback devices belong to the high category and at least one playback device belongs to the medium category, the UI module 312 may prompt the user to select a playback device by presenting the graphical element 202. For example, the scoring module 308 may send identifiers of available playback devices and their corresponding predicted scores to the UI module 312. Since no high category playback device is identified, the UI module 312 may present the graphical element 202 when the user selects the play button 104. As described above, the medium category playback devices presented on the graphical element 202 may be grouped and / or sorted on the graphical element 202 according to their corresponding predicted scores. The low category playback devices presented on the graphical element 202 may be grouped and / or sorted in alphabetical order, such as Figure 2 By grouping the playback devices in the category together in a quick access location on the graphical element 202, the user can quickly and efficiently find the playback device that the user is most likely to select for playback of the selected content item.

[0118] Default Behavior

[0119] In some implementations, the user device 302 may automatically select a local playback device (e.g., user device 302) as the playback device. For example, when all available playback devices are associated with a low category (e.g., they all have a low prediction score), the user device 302 may select the user device 302 as the playback device. The UI module 312 may indicate on the graphical element 106 that the user device 302 is the selected playback device. When the user selects the play button 104 to initiate playback of the selected content item, the user device 302 may automatically begin presenting the content item on the user device 302 without prompting the user to select a playback device.

[0120] Likewise, if the user wishes to select a different playback device than the automatically selected playback device, the user may select the graphical element 106 to cause the available playback devices to be presented on the graphical element 202, as described above. In some implementations, based on the user behavior (e.g., selecting a playback device other than the automatically selected playback device), the scoring module 308 may change the scores of the user-selected playback device and the automatically selected playback device (e.g., user device 302) accordingly. In this case, since all playback devices have been classified as low-category devices, the playback devices presented on the graphical element 202 may be grouped and / or sorted in alphabetical order.

[0121] Figure 5 is a block diagram of an exemplary system 500 for generating a target playback device prompt. For example, the system 500 may correspond to the system 300 described above. The user device 302 of the system 300 / system 500 may include a selector user interface (UI) module 318. The selector UI module 318 may be configured to generate a graphical element 202 of the GUI 200 in response to a request (e.g., a selector event 502) from the UI module 312 to generate and present a graphical element 202 (e.g., a playback target selector user interface).

[0122] In some specific implementations, the selector UI module 318 can generate different selector user interfaces (e.g., graphical elements 202) for different media applications and / or media content items selected by the user. As described above, each media application and / or media content item can be associated with different types of playback sessions. When generating a selector user interface for receiving a user input for selecting a playback device, the selector UI module 318 can generate a playback device selector user interface based on the playback session type associated with the media content item and / or the media application with which the user currently interacts. For example, if the media application and / or the selected media content item corresponds to a movie, the selector UI module 318 can generate a long-form video session selector 510. If the media application and / or the selected media content item corresponds to long-form audio, the selector UI module 318 can generate a long-form audio session selector 512. If the media application and / or the selected media content item corresponds to rich media, the selector UI module 318 can generate a rich media session selector 514.

[0123] In some implementations, the selector module 318 may include other selectors 516. For example, the other selectors 516 may be selectors associated with other types of content that may be streamed to and / or presented by a playback device. For example, the other selectors 516 may be selectors that allow a user to select a playback device to mirror (e.g., simultaneously present on the playback device and the user device 302) a video presented on the user device 302. The other selectors 516 may be selectors that allow a user to select a playback device to present real-time audio and / or video associated with a communication application running on the user device 302.

[0124] In some specific implementations, selector module 318 can generate a set of playback devices presented on the selector based on the type of selector generated for media application and / or selected media content item. For example, the playback device that can be scored and / or selected for presentation on long-form video selector 510 can be limited to playback devices that can present long-form video and / or have been previously selected by the user for presentation of long-form video. Similarly, the playback device that can be scored and / or selected for presentation on long-form audio selector 512 can be limited to playback devices that can present long-form audio and / or have been previously selected by the user for presentation of long-form audio. The playback device that can be scored and / or selected for presentation on rich media selector 514 can be limited to playback devices that can present rich media content items and / or have been previously selected by the user for presentation of rich media content items. After selector UI module 318 has received a user selection of a playback device from a playback device selector, selector UI module 318 may send the selected playback device to UI module 312, and the selected content item may be routed to the user-selected playback device as described herein.

[0125] Example Process

[0126] In order to enable the reader to clearly understand the technical concepts described herein, the following process describes the specific steps performed in a specific order. However, one or more steps of a specific process can be rearranged and / or omitted while remaining within the expected scope of the technology disclosed herein. In addition, different processes and / or their steps can be combined, recombined, rearranged, omitted and / or performed in parallel to create different processing flows that are also within the expected scope of the technology disclosed herein. In addition, although for the sake of clarity, the following process can omit or briefly summarize some details of the technology disclosed herein, the details described in the above paragraphs can be combined with the process steps described below to obtain a more complete and comprehensive understanding of these processes and the technology disclosed herein.

[0127] Figure 6 6 is a flow chart of an exemplary process 600 for automatically routing media content to a predicted remote playback device. For example, process 600 may be performed by a user device 302 to automatically select a playback device for presenting content selected by a user on the user device 302. The automatically selected playback device may be a remote playback device. The automatically selected playback device may be the user device 302. When the user device 302 receives a user input to initiate playback of a content item selected by the user, the user device 302 may automatically send or route the selected content item to the automatically selected playback device without receiving a user input selecting a playback device.

[0128] At step 602, user device 302 may receive a selection of a content item. For example, user device 302 may include a software application that allows a user to browse, search, and / or find media content to view and / or listen to. The user may interact with the application and select the media content item that the user wishes to use.

[0129] At step 604, the user device 302 may obtain playback context information. For example, the user device 302 may obtain local playback context information locally on the user device 302 from metadata associated with the selected content item, applications installed on the user device 302, various components of the user device 302 (e.g., a clock), etc., as described above. The user device 302 may obtain remote playback context information from a remote playback device. For example, the user device 302 may determine which playback devices are available on the same network to which the user device 302 is connected, and send requests to these playback devices to obtain their playback states, as described above.

[0130] At step 606, the user device 302 may generate a prediction score for each available playback device. For example, the prediction score may indicate the likelihood that the user of the user device 302 will want to play the selected content item on the corresponding playback device. The prediction score may be generated based on the playback context information obtained at step 604 and / or historical user behavior data collected by the user device 302 and / or stored on the user device 302, as described above.

[0131] At step 608, the user device 302 may automatically select a playback device based on the predicted scores generated for each playback device. For example, if the playback device has not been disqualified from automatic selection and playback (as described above) and the predicted score of the playback device places it in a high category, the user device 302 may automatically select the playback device for presenting the content item selected by the user. The user device 302 may present an identifier of the automatically selected playback device on the playback user interface (e.g., GUI 100, GUI 200) so that the user can see which playback device will present the selected content item, as described above.

[0132] At step 610, the user device 302 may receive user input to play the selected content item. For example, the user device 302 may receive user input to a playback user interface for selecting a play button or control to initiate playback of the content item selected by the user. For example, the user device 302 may receive user input via the GUI 100 indicating that the user wishes to initiate playback of the selected content item.

[0133] At step 612, the user device may automatically route the content item to the automatically selected playback device. For example, the user device 302 may send (e.g., stream or hand over) the content item to the automatically selected playback device for presentation. The user device 302 may automatically send the content item to the playback device without explicitly receiving any user input or explicitly selecting a playback device.

[0134] Figure 7 Flowchart of an exemplary process 700 for efficiently presenting available playback devices based on prediction scores. For example, process 700 may be performed by user device 302 to present playback devices in a manner that allows the user to efficiently and quickly find and select the playback device that the user is most likely to use to present the currently selected content item.

[0135] At step 702, user device 302 may receive a selection of a content item. For example, user device 302 may include a software application that allows a user to browse, search, and / or find media content to view and / or listen to. The user may interact with the application and select the media content item that the user wishes to use.

[0136] At step 704, the user device 302 may obtain playback context information. For example, the user device 302 may obtain local playback context information locally on the user device 302 from metadata associated with the selected content item, applications installed on the user device 302, various components of the user device 302 (e.g., a clock), etc., as described above. The user device 302 may obtain remote playback context information from a remote playback device. For example, the user device 302 may determine which playback devices are available on the same network to which the user device 302 is connected, and send requests to these playback devices to obtain their playback states, as described above.

[0137] At step 705, the user device 302 may generate a sorted set of available playback devices based on the playback context information. For example, the user device 302 may sort the available playback devices based on steps 706 and / or 708 described below. Alternatively, the user device 302 may sort the available playback devices based on some other sorting criteria or specific playback context information without generating a prediction score for the playback device. For example, the user device 302 may sort the available playback devices based on proximity (e.g., the proximity of the playback device to the user device 302). For another example, the user device 302 may sort the available playback devices based on the recency and / or frequency of use or selection by the user of the user device 302. For another example, the user device 302 may first sort the available playback devices based on proximity, then based on recency or frequency (e.g., when two playback devices are at the same distance from the user device 302) or some other combination of context data. Thus, in some implementations, when other ranking mechanisms are used, the user device 302 may not perform step 706 and / or step 708, while in other implementations, when the playback devices are ranked using predicted scores, step 706 and / or step 708 may be used to limit step 705.

[0138] At step 706, the user device 302 may generate a prediction score for each available playback device. For example, the prediction score may indicate the likelihood that the user of the user device 302 will want to play the selected content item on the corresponding playback device. The prediction score may be generated based on the playback context information obtained at step 704 and / or historical user behavior data collected by the user device 302 and / or stored on the user device 302, as described above.

[0139] At step 708, the user device 302 may rank the playback devices based on the prediction score generated for each playback device. For example, the user device 302 may generate a ranked set (e.g., a list) of playback devices associated with a high category and a medium category (as described above), wherein the categories are ranked according to the respective prediction scores. The user device 302 may generate a ranked set of playback devices associated with a low category, wherein the category is ranked alphabetically based on the respective identifiers (e.g., names) of the playback devices.

[0140] At step 710, user device 302 may present the sorted playback devices on a playback device selector graphical user interface. For example, when user device 302 determines that there is no playback device associated with a high category, user device 302 may present a sorted set of playback devices on graphical element 202 of GUI 200 to prompt the user to select a playback device for presenting the selected content item. In response to the user selecting a play button or otherwise providing a play command or input to user device 302, the prompt may be presented, including a sorted list of playback devices. Alternatively, in response to receiving a user input selecting graphical element 106 of GUI 100, user device 302 may present a sorted set of playback devices on graphical element 202 of GUI 200. A sorted set of playback devices may be presented so that high and medium category playback devices can be presented on graphical element 202 in a prominent, easy to locate, easy to view position manner.

[0141] At step 712, the user device 302 may receive a selection of a playback device. For example, a user of the user device may select a playback device presented on the graphical element 202.

[0142] At step 714, user device 302 may route the selected content item to the playback device selected by the user. For example, in response to receiving a user input selecting a playback device, the user device may stream or hand over the selected content item to the selected playback device for presentation. Alternatively, when the selected playback device corresponds to user device 302, user device 302 may present the selected content item using a display and / or speaker of user device 302.

[0143] Figure 8 8 is a flow chart of an exemplary process 800 for routing different content items from a single computing device to different playback devices. For example, process 800 may be performed by user device 302 to route media items corresponding to different playback sessions to different playback targets. For example, a user may wish to present movie content from user device 302 on remote playback device 330 while simultaneously presenting music content on remote playback device 340. For another example, a user may wish to present a television program from user device 302 on remote playback device 360 ​​while simultaneously presenting social media audio and / or video content locally on user device 302.

[0144] At step 802, user device 302 may receive a first user input to initiate a first media playback session corresponding to a first playback session type. For example, a user may provide user input to user device 302 that instructs user device 302 to present a movie on remote playback device 330. User device 302 may generate a long-form video playback session to manage the presentation and routing of the movie on remote playback device 330.

[0145] At step 804, user device 302 may route the first media item corresponding to the first media playback session to the first playback device. For example, in response to user input selecting the first playback device and / or initiating playback of the first media item corresponding to the first media playback session, user device 302 may route the first media item corresponding to the first media playback session to the first playback device.

[0146] At step 806, user device 302 may receive a second user input to initiate a second media playback session corresponding to a second playback session type. For example, a user may provide user input to user device 302 that instructs user device 302 to present a song on remote playback device 340. User device 302 may generate a long form audio playback session to manage the presentation and routing of the song on remote playback device 340.

[0147] At step 808, user equipment 302 can route the second media item corresponding to the second media playback session to the second playback device. For example, in response to selecting the second playback device and / or initiating the user input of the playback of the second media item corresponding to the second media playback session, user equipment 302 can determine that the target playback device for the long-form audio playback session does not conflict with the target playback device for the long-form video playback session (for example, the target playback device is not the same). In response to determining that there is no conflict, user equipment 302 can route the song corresponding to the second media playback session to the second playback device. Therefore, user equipment 302 can manage two or more media playback sessions (and corresponding media items) simultaneously and route them to multiple different target playback devices.

[0148] Fig. 9 900 is a flow chart of an exemplary process 900 for determining when to interrupt a remote playback session. For example, process 900 may be performed by user device 302 to determine which media applications and / or media content items may cause playback of a currently playing media content item on a remote playback device to be interrupted. Figure 8 As described above, user device 302 can route media content items to multiple targets simultaneously. However, when two media sessions attempt to route media content items to the same playback device, user device 302 will determine whether to interrupt the content item currently playing on the remote playback device to allow playback of the newly selected content item on the remote playback device according to process 900, as described below.

[0149] In some specific implementations, process 900 can only be performed when performing automatic media routing.If the user has specified the remote playback device for presenting selected media content items, then user equipment 302 can route selected media content items to the playback device of appointment, and interrupt any media content currently played on the playback device of appointment.At step 902, user equipment 302 can receive the user input to media application program to initiate the playback of media content items.For example, the user can interact with application program 304 to select media content items.In response to receiving the call of media application program 304 and / or the selection of media content items, user equipment 302 can generate a media playback session based on the media type provided by application program 304, and automatically selects the remote playback device for presenting selected media content items.User equipment 302 can receive the user input of playing selected media content items.

[0150] At step 904 , the user device 302 may determine that a remote playback session for the selected remote playback device is currently ongoing. For example, the media session manager 136 may determine that another media playback session is currently being routed to the remote playback device automatically selected at step 902 .

[0151] At step 904, user device 302 may determine whether to interrupt the ongoing remote playback session based on various interruption criteria. For example, media session manager 136 may determine whether to interrupt the ongoing remote playback session based on a blacklist, a whitelist, an important tag, and / or other interruption criteria described above.

[0152] At step 908, when the interruption criteria indicate that the remote playback session should be interrupted, the user device 302 may route playback of the selected media content item to the automatically selected remote playback device. For example, the media session manager 136 may terminate playback of the media item corresponding to the ongoing playback session and initiate playback of the selected media content item on the automatically selected remote playback device.

[0153] At step 910, when the interruption criteria indicate that the remote playback session should not be interrupted, the user device 302 may present the selected media item on the local user device. For example, the media session manager 136 may initiate playback of the selected media content item on the user device 302. If the user device 302 is currently presenting another media content item locally, the media session manager 136 may interrupt playback of the content item currently playing on the user device 302 and initiate playback of the selected media content item.

[0154] Fig.101 is a flow diagram of an exemplary process 1000 for resuming a routed session based on location. For example, when the user device 302 returns to a particular location where the user device 302 previously routed playback to a particular remote playback device, the process 1000 can be performed by the user device 302 to resume the playback session routed to the particular remote playback device.

[0155] At step 1002, user equipment 302 may receive input to route the first media item to a remote playback device. For example, user equipment 302 may receive explicit input indicating that the media content item should be routed to a remote playback device. In response to receiving the user input of the playback media content item, user equipment 302 may automatically route the media content item to a remote playback device. User equipment 302 may generate a playback session for managing the playback of the first media content item on the remote playback device.

[0156] At step 1004, the user device 302 may determine a first current location of the user device. For example, the user device 302 may determine a first fixed-point location at which a user initiates a playback session for a remote playback device, and store the first fixed-point location in the playback session data.

[0157] At step 1006, user device 302 may determine that the connection with the playback device has been terminated. For example, user device 302 may determine that the connection with the playback device has been terminated when user device 302 is no longer on the same network as the remote playback device. For example, the user may have left home with user device 302, and user device 302 may no longer be connected to the user's home network.

[0158] At step 1008, user equipment 302 can receive input on user equipment to initiate playback of the second media item. For example, the media item can be the same media item as the media item described in step 1002. The media item can be the media item (for example, the second media item) different from the media item described in step 1002. However, the media item and / or the media application providing the media item can be associated with the same media playback session type (for example, long-form video, long-form audio, rich media, non-media, etc.). If the media playback session of step 1002 has not yet terminated (for example, less than the threshold time period passed since the last playback event of the session), the playback session can still be configured to be routed to identical remote playback device.

[0159] At step 1010, the user device 302 may determine a second current location of the user device 302. For example, at step 1008, the user device 302 may determine a second fixed-point location where the user initiated playback.

[0160] At step 1012, the user device 302 may determine a second current location that corresponds to the first current location. For example, the user device 302 may determine that the first fixed-point location determined at step 1004 is within a measurement threshold of similarity with the second fixed-point location determined at step 1010.

[0161] At step 1014, user equipment 302 can automatically route the playback of the second media item to a remote playback device. For example, because user equipment 302 has returned to the same position as the playback session initiated at step 1002, and because the playback session has never been terminated, and because the playback initiated at step 1008 is about media content items and / or media applications associated with the same playback session type of the playback session, user equipment 302 can automatically route the playback of the second media item to a remote playback device configured for the playback session. For example, media session manager 316 can compare the playback session type and fixed-point position associated with the second media item with the session type and the fixed-point position of the existing playback session. If there is an existing (e.g., not yet terminated) playback session with the same session type and the same fixed-point position, user equipment 302 can route the playback of the second media item to the same target playback device configured for the existing playback session of matching, thereby recovering an existing but possibly inactive playback session. Thus, when the user device 302 returns to the first current location, the user device 302 can resume the playback session started at step 1002.

[0162] Graphical User Interface

[0163] The present disclosure describes above various graphical user interfaces (GUIs) for implementing various features, processes, or workflows. These GUIs can be presented on various electronic devices, including but not limited to laptop computers, desktop computers, computer terminals, television systems, tablet computers, e-book readers, and smart phones. One or more of these electronic devices may include a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous input points, including processing data related to the pressure, degree, or position of each input point. Such processing can facilitate gestures performed using multiple fingers, including pinching and swiping.

[0164] When the present disclosure refers to "selecting" a user interface element in a GUI, these terms are understood to include clicking or "hovering" over a user interface element with a mouse or other input device, or touching, tapping, or gesturing on a user interface element with one or more fingers or a stylus. A user interface element may be a virtual button, menu, selector, switch, slider, brush, knob, thumbnail, link, icon, radio button, check box, and any other mechanism for receiving input from a user or providing feedback to a user.

[0165] privacy

[0166] As mentioned above, one aspect of the technology of the present invention is to collect and use the data obtained from various sources to reduce the amount of user input required for selecting the playback device for presenting media content items and improving the efficiency of user-identifiable and / or selection of playback devices. The disclosure contemplates that in some instances, these collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, twitter IDs, home addresses, data or records (e.g., vital signs measurements, medication information, exercise information) relevant to the user's health or fitness level, date of birth, or any other identifying information or personal information.

[0167] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data can be used to improve the user's enjoyment of their personal devices and various types of playback devices that the user may have around their home and / or office.

[0168] This disclosure contemplates that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for the entity's legitimate and reasonable purposes and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving the user's informed consent.

[0169] In addition, such entities should consider taking any necessary steps to safeguard and secure access to such Personal Information Data and to ensure that others who have access to the Personal Information Data comply with its privacy policies and procedures. In addition, such entities may subject themselves to third-party assessments to demonstrate compliance with widely accepted privacy policies and practices. In addition, policies and practices should be tailored to the specific types of Personal Information Data collected and / or accessed and to applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0170] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware elements and / or software elements to prevent or block access to such personal information data. For example, the present technology may be configured to allow users to “opt in” or “opt out” of participating in the collection of personal information data at any time during or after registration for a service. In addition to providing “opt in” and “opt out” options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then be reminded again just before the personal information data is accessed by the application.

[0171] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data between users), and / or other methods such as differential privacy, when appropriate.

[0172] Therefore, although the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that the various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the present technology will not fail to function properly due to the lack of all or part of such personal information data. For example, general rules for media routing that do not rely on user personal information may be used. The predictive routing mechanism described herein may use media metadata and / or other public information to predictively route selected media content items to playback devices.

[0173] Exemplary System Architecture

[0174] Fig.11 It is achievable Figure 1-Figure 10 1100. The computing device 1100 may include a memory interface 1102, one or more data processors, an image processor, and / or a central processing unit 1104, and a peripheral device interface 1106. The memory interface 1102, one or more processors 1104, and / or the peripheral device interface 1106 may be separate components or may be integrated into one or more integrated circuits. The various components in the computing device 1100 may be coupled by one or more communication buses or signal lines.

[0175] Sensors, devices, and subsystems may be coupled to the peripherals interface 1106 to facilitate a number of functions. For example, a motion sensor 1110, a light sensor 1112, and a proximity sensor 1114 may be coupled to the peripherals interface 1106 to facilitate orientation, lighting, and proximity functions. Other sensors 1116 may also be connected to the peripherals interface 1106, such as a global navigation satellite system (GNSS) (e.g., a GPS receiver), a temperature sensor, a biometric sensor, a magnetometer, or other sensing device to facilitate related functions.

[0176] Camera functions, such as taking pictures and video clips, may be facilitated using a camera subsystem 1120 and an optical sensor 1122, such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) optical sensor. The camera subsystem 1120 and the optical sensor 1122 may be used to collect images of the user to be used during authentication of the user, for example, by performing facial recognition analysis.

[0177] Communication functions may be facilitated by one or more wireless communication subsystems 1124, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of communication subsystems 1124 may depend on the communication network or networks over which computing device 1100 is intended to operate. For example, computing device 1100 may include a wireless communication subsystem designed to communicate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth® network. TM Network-operated communication subsystem 1124. Specifically, the wireless communication subsystem 1124 may include a host protocol so that the device 100 can be configured as a base station for other wireless devices.

[0178] The audio subsystem 1126 may be coupled to a speaker 1128 and a microphone 1130 to facilitate voice-enabled functions such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem 1126 may be configured to facilitate, for example, processing voice commands, voiceprint identification, and voice authentication.

[0179] I / O subsystem 1140 may include touch surface controller 1142 and / or one or more other input controllers 1144. Touch surface controller 1142 may be coupled to touch surface 1146. Touch surface 1146 and touch surface controller 1142 may detect contact and motion or interruption thereof, for example, using any of a variety of touch-sensitive technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch surface 1146.

[0180] One or more other input controllers 1144 may be coupled to other input / control devices 1148, such as one or more buttons, rocker switches, thumb wheels, infrared ports, USB ports, and / or pointer devices (such as stylus). One or more buttons (not shown) may include an up / down button for volume control of the speaker 1128 and / or microphone 1130.

[0181] In one implementation, pressing the button for a first duration can unlock the touch surface 1146; and pressing the button for a second duration that is longer than the first duration can turn the power of the computing device 1100 on or off. Pressing the button for a third duration can activate a voice control or voice command module that enables a user to speak a command into the microphone 1130 so that the device executes the spoken command. The user can customize the function of one or more buttons. For example, the touch surface 1146 can also be used to implement virtual or soft buttons and / or a keyboard.

[0182] In some implementations, computing device 1100 can present recorded audio and / or video files, such as MP3, AAC, and MPEG files. In some implementations, computing device 1100 can include the functionality of an MP3 player, such as an iPod. TM .

[0183] The memory interface 1102 may be coupled to a memory 1150. The memory 1150 may include a high-speed random access memory and / or a non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or a flash memory (e.g., NAND, NOR). The memory 1150 may store an operating system 1152, such as Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system (such as VxWorks).

[0184] The operating system 1152 may include instructions for handling basic system services and for performing hardware-related tasks. In some implementations, the operating system 1152 may be a kernel (e.g., a UNIX kernel). In some implementations, the operating system 1152 may include instructions for performing voice authentication. For example, the operating system 1152 may implement a predictive media routing feature, such as the reference Figure 1-Figure 10 described.

[0185] The memory 1150 may also store communication instructions 1154 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers. The memory 1150 may include graphical user interface instructions 1156 to facilitate graphical user interface processing; sensor processing instructions 1158 to facilitate sensor-related processing and functions; phone instructions 1160 to facilitate phone-related processes and functions; electronic messaging instructions 1162 to facilitate electronic messaging processes and functions; web browsing instructions 1164 to facilitate web browsing-related processes and functions; media processing instructions 1166 to facilitate media processing-related processes and functions; GNSS / navigation instructions 1168 to facilitate GNSS and navigation-related processes and instructions; and / or camera instructions 1170 to facilitate camera-related processes and functions.

[0186] The memory 1150 may store software instructions 1172 to facilitate other processes and functions, such as reference Figure 1-Figure 10 The predictive media processes and functions described.

[0187] The memory 1150 may also store other software instructions 1174, such as network video instructions that facilitate processes and functions related to network video; and / or network shopping instructions that facilitate processes and functions related to network shopping. In some specific implementations, the media processing instructions 1166 are divided into audio processing instructions and video processing instructions to facilitate processes and functions related to audio processing and processes and functions related to video processing, respectively.

[0188] Each of the instructions and applications identified above may correspond to an instruction set for performing one or more of the functions described above. These instructions need not be implemented as separate software programs, processes, or modules. Memory 1150 may include additional instructions or fewer instructions. Furthermore, various functions of computing device 1100 may be implemented in hardware and / or software, including in one or more signal processing and / or application specific integrated circuits.

[0189] To assist the Patent Office and any reader of any patent issuing in this application in interpreting the appended claims, Applicants wish to call attention that they do not intend for any of the appended claims or claim elements to invoke 35 U.S.C. 112(f) unless the words “means for” or “step for” are expressly used in a particular claim.

Claims

1. A method comprising: receiving, by the computing device, from a media application, a first user input initiating playback of a first media item; determining, by the computing device, a first playback device to which playback of the first media item is to be routed; In response to determining that a remote playback session to the first playback device is ongoing: determining, by the computing device, whether to interrupt the remote playback session on the first playback device based on whether the media application is allowed to interrupt the remote playback session; as well as In response to determining that the media application is allowed to interrupt the remote playback session: terminating, by the computing device, the remote playback session on the first playback device; as well as The first media item is wirelessly routed, by the computing device, to the first playback device.

2. The method according to claim 1, further comprising: Prior to wirelessly routing the first media item to the first playback device, a determination is made by the computing device that no conflict exists with the first playback device for playback of the first media item. 3 . The method of claim 2 , wherein in response to determining that there is no conflict with the first playback device for playback of the first media item, the first media item is wirelessly routed to the first playback device.

4. The method of claim 1 , wherein allowing the media application to interrupt the remote playback session is responsive to at least one of: determining that the media application participates in an operating system-level user interface feature, and determining that the media application has permission to present audio content while running as a background application.

5. The method according to claim 1, further comprising: Prior to wirelessly routing the first media item to the first playback device, determining, by the computing device, whether the remote playback session to the first playback device is ongoing, Wherein determining whether to interrupt the remote playback session on the first playback device is further based on one or more interruption criteria.

6. The method of claim 5, wherein the one or more interruption criteria comprise media type or media length, and wherein the one or more interruption criteria are analyzed according to a hierarchy of the one or more criteria.

7. The method of claim 1 , wherein determining whether to interrupt the remote playback session on the first playback device comprises: determining that the media application is not included in a blacklist of applications that are not allowed to interrupt the remote playback session; or A determination is made that the media application is included in a whitelist of applications that are allowed to interrupt the remote playback session.

8. A system comprising: one or more processors; as well as A non-transitory computer readable medium comprising a set of instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving, by the computing device, from a media application, a first user input initiating playback of a first media item; determining, by the computing device, a first playback device to which playback of the first media item is to be routed; In response to determining that a remote playback session to the first playback device is ongoing: determining, by the computing device, whether to interrupt the remote playback session on the first playback device based on whether the media application is allowed to interrupt the remote playback session; as well as In response to determining that the media application is allowed to interrupt the remote playback session: terminating, by the computing device, the remote playback session on the first playback device; as well as The first media item is wirelessly routed, by the computing device, to the first playback device.

9. The system of claim 8, wherein the operations further comprise: Prior to wirelessly routing the first media item to the first playback device, a determination is made by the computing device that no conflict exists with the first playback device for playback of the first media item.

10. The system of claim 9, wherein in response to determining that there is no conflict with the first playback device for playback of the first media item, the first media item is wirelessly routed to the first playback device.

11. The system of claim 8, wherein the media application is allowed to interrupt the remote playback session in response to at least one of: a determination that the media application participates in an operating system-level user interface feature, and a determination that the media application has permission to present audio content while running as a background application.

12. The system of claim 8, wherein the operations further comprise, prior to wirelessly routing the first media item to the first playback device: determining, by the computing device, whether the remote playback session to the first playback device is ongoing, Wherein determining whether to interrupt the remote playback session on the first playback device is further based on one or more interruption criteria.

13. The system of claim 12, wherein the one or more interruption criteria include media type or media length, and wherein the one or more interruption criteria are analyzed according to a hierarchy of the one or more criteria.

14. The system of claim 8, wherein determining whether to interrupt the remote playback session on the first playback device comprises: determining that the media application is not included in a blacklist of applications that are not allowed to interrupt the remote playback session; or A determination is made that the media application is included in a whitelist of applications that are allowed to interrupt the remote playback session.

15. A non-transitory computer readable medium comprising a set of instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising: receiving, by the computing device, from a media application, a first user input initiating playback of a first media item; determining, by the computing device, a first playback device to which playback of the first media item is to be routed; In response to determining that a remote playback session to the first playback device is ongoing: determining, by the computing device, whether to interrupt the remote playback session on the first playback device based on whether the media application is allowed to interrupt the remote playback session; as well as In response to determining that the media application is allowed to interrupt the remote playback session: terminating, by the computing device, the remote playback session on the first playback device; as well as The first media item is wirelessly routed, by the computing device, to the first playback device.

16. The non-transitory computer readable medium of claim 15, wherein the operations further comprise: Prior to wirelessly routing the first media item to the first playback device, a determination is made by the computing device that no conflict exists with the first playback device for playback of the first media item.

17. The non-transitory computer-readable medium of claim 16, wherein in response to determining that there is no conflict with the first playback device for playback of the first media item, the first media item is wirelessly routed to the first playback device.

18. The non-transitory computer-readable medium of claim 15, wherein the media application is allowed to interrupt the remote playback session in response to at least one of: a determination that the media application participates in an operating system-level user interface feature, and a determination that the media application has permission to present audio content while running as a background application.

19. The non-transitory computer-readable medium of claim 15, wherein the operations further comprise, prior to wirelessly routing the first media item to the first playback device: determining, by the computing device, whether the remote playback session to the first playback device is ongoing, Wherein determining whether to interrupt the remote playback session on the first playback device is further based on one or more interruption criteria.

20. The non-transitory computer-readable medium of claim 19, wherein the one or more interruption criteria include a media type or a media length, and wherein the one or more interruption criteria are analyzed according to a hierarchy of the one or more criteria.

Images