Dynamic Integration of Customized Supplementary Media Content

Abstract

To provide a method for providing a requested media item with a customized supplement media item to a user who requests, and provide a system and a program.SOLUTION: A method contains: a reception step 714 of receiving a request of a first user who requests a first media item of a second user; and a selection step 726 of selecting a second media item from a plurality of second media items on the basis of a regulation set provided from a second user device. In transmission steps 728 and 730, the first media item and the selected second media item are transmitted to a first user device of the first user. The regulation set provided by the second user device comprises rules for selecting the second media items to be integrated with the first media item and a condition related to the first user.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Aspects of the present disclosure generally relate to selecting customized supplementary media items for integration with requested media items.

Background Art

[0002] A viewer may request a video owner's video for presentation on the viewer's user device. The video may include a version of supplementary content for presentation to all viewers who request the video. For example, a video owner may include in the video a message that prompts the viewer to perform an action, regardless of whether the viewer is likely to complete the prompted action.

Summary of the Invention

Means for Solving the Problems

[0003] To provide a basic understanding of some aspects of the present disclosure, the following summary is a simplified summary of the present disclosure. This summary is not an extensive overview of the present disclosure. It is not intended to identify key or critical elements of the present disclosure or to delineate any scope of particular implementations of the present disclosure or any scope of the claims. Its sole purpose is to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] According to some aspects of the present disclosure, receiving, at a server, a request from a first user for a first media item of a second user, wherein the first media item includes at least one of video content or audio content; determining that the first media item of the second user corresponds to a plurality of second media items of the second user, wherein each of the plurality of second media items includes at least one of video content or audio content; identifying a set of rules for how to select, from the plurality of second media items, a second media item to be integrated with the first media item of the second user, wherein the set of rules is provided by the second user and includes one or more conditions related to the first user; selecting, based on the set of rules, a second media item from the plurality of second media items; and transmitting the first media item and the selected second media item to a first user device of the first user, wherein the selected second media item is to be integrated with the first media item for presentation to the first user, a computer-implemented method is provided.

[0005] Generally, at least one of the plurality of second media items may prompt the first user to perform an action. The action may be of any suitable type and may be part of a technical task. Thus, by transmitting the first media item and the selected second media item to the first user device of the first user together with the selected second media item to be integrated with the first media item for presentation to the first user, the method enables a human-machine interaction interface to be seamlessly incorporated within the requested first media item.

[0006] By further selecting a second media item from a plurality of second media items based on a set of rules having one or more conditions related to a first user, the selected second media item may be curated for the first user, thereby enabling the human-machine interaction interface represented by the selected second media item to have a particular relevance to the first user. In this way, the method of the present disclosure can efficiently assist the first user in performing actions related to the first user by a continuous and guided human-machine interaction process.

[0007] At least one of the plurality of second media items may include, for example, a message that suggests an action to the first user (which may comprise visual and / or acoustic elements), and / or an action element (such as a hyperlink) that can be selected by the first user to induce the execution of an action. Optionally, the actions may be associated with each respective second media item, or alternatively, may relate to a second user and not necessarily be associated with each respective second media item.

[0008] Each of the plurality of second media items may comprise unique video content or audio content.

[0009] The set of rules for how to select the second media items to be integrated with the first media items of the second user may further comprise rules for selecting the second media items based on a set of rankings provided by the second user, where each of the plurality of second media items may be associated with a ranking in the set of rankings.

[0010] Selecting a second media item based on a set of rules may comprise identifying a second media item associated with the highest ranking from among a plurality of second media items, and determining that one or more conditions relevant to the first user are satisfied with respect to the second media item.

[0011] Alternatively or additionally, selecting a second media item based on a set of rules may comprise identifying a third media item associated with the highest ranking from among a plurality of second media items, determining that one or more conditions relevant to the first user are not satisfied with respect to the third media item, identifying a second media item associated with a ranking lower than that of the third media item associated with the highest ranking from among the plurality of second media items, and determining that one or more conditions relevant to the first user are satisfied with respect to the second media item.

[0012] The first user device may render the first media item and the selected second media item as a composite media item. The selected second media item may be integrated with the first media item. The transition between the first media item and the second media item may not be visible to the first user.

[0013] The method is to determine a time-based index of the timeline of a first media item, where the time-based index indicates a location on the timeline of the first media item at which the content of a selected second media item will be presented to a first user, and to send the time-based index together with the first media item and the selected second media item to a first user device of the first user, where the selected second media item will be integrated into the first media item at the location corresponding to the time-based index on the timeline of the first media item.

[0014] The time-based index of the timeline of the first media item may be created based on an interaction of a second user with the timeline of the first media item.

[0015] Alternatively or additionally, the time-based index of the timeline of the first media item may be determined using a trained machine learning model.

[0016] The first media item may comprise a live media stream. Sending the first media item and the selected second media item to a first user device of the first user may comprise sending the live media stream to a first user device of the first user, receiving, from a second user device of a second user, a display for integrating the selected second media item with the live media stream, and, in response to receiving the display for integrating the selected second media item with the live media stream, sending the selected second media item to the first user device of the first user, where the selected second media item will be integrated into the live media stream.

[0017] One or more characteristics related to the first user may change based on actions performed by the first user. For example, if the first user performs an action prompted by a message of a selected second media item, the characteristics associated with the first user may change to reflect that the first user has previously performed an action. As a result, a third media item from a plurality of second media items may be selected based on a set of rules in response to a second request by the first user for the first media item, or a request by the second user for a second media item of the second user. The third media item may be selected based on the application of the set of rules and changes to one or more characteristics related to the first user. In this way, the method of the present disclosure can dynamically respond to a request for a first media item associated with a second user by efficiently selecting a second media item related to the first user and integrating the selected second media item with the first media item.

[0018] According to some aspects of the present disclosure, receiving, at a server, a request from a first user for a first media item of a second user, wherein the first media item includes at least one of video content or audio content; determining that the first media item of the second user corresponds to a plurality of second media items of the second user, wherein each of the plurality of second media items includes at least one of video content or audio content; identifying a set of rules for how to select a second media item to be integrated with the first media item of the second user from the plurality of second media items, wherein the set of rules is provided by the second user and includes rules for selecting the second media item based on one or more conditions related to the first user and a set of rankings provided by the second user, wherein each of the plurality of second media items is associated with a ranking in the set of rankings; selecting a second media item from the plurality of second media items based on the set of rules; determining a time - based index of a timeline of the first media item, wherein the time - based index indicates a location on the timeline of the first media item at which the content of the selected second media item will be presented to the first user; and transmitting the first media item, the selected second media item, and the time - based index to a first user device of the first user, wherein the selected second media item will be integrated with the first media item at the location corresponding to the time - based index on the timeline of the first media item. A computer - implemented method is provided that includes these steps.

[0019] Sending the first media item, the selected second media item, and the time-based index to the first user device enables the selected second media item to be easily incorporated within the first media item at the location of the first media item as defined by the time-based index. Thus, after a user request for the first media item is submitted, content integration may be performed at the first user device almost immediately (i.e., within the minimum delay caused by network conditions).

[0020] Each of the plurality of second media items may include unique video content or audio content. The first user device may render the first media item and the selected second media item as a composite media item, where the selected second media item is to be integrated with the first media item and the transition between the first media item and the selected second media item is not visible to the first user.

[0021] Optionally, selecting the second media item based on a set of rules may include identifying the second media item associated with the highest ranking from among the plurality of second media items and determining that one or more conditions related to the first user are satisfied with respect to the second media item. Selecting the second media item based on a set of rules may also include identifying the third media item associated with the highest ranking from among the plurality of second media items and determining that one or more conditions related to the first user are not satisfied with respect to the third media item, identifying the second media item associated with a ranking lower than the third media item associated with the highest ranking from among the plurality of second media items, and determining that one or more conditions related to the first user are satisfied with respect to the second media item.

[0022] Optionally, the first media item may include a live media stream. Transmitting the first media item and the selected second media item to the first user device of the first user may include transmitting the live media stream to the first user device of the first user, receiving, from the second user device of the second user, a display for integrating the selected second media item with the live media stream, and in response to receiving the display for integrating the selected second media item with the live media stream, transmitting the selected second media item to the first user device of the first user, where the selected second media item is to be integrated with the live media stream.

[0023] According to another aspect of the present disclosure, a system is provided that includes a memory and a processing device coupled to the memory. The processing device is configured to receive, at a server, a request from a first user for a first media item of a second user, where the first media item includes at least one of video content or audio content; determine that the first media item of the second user corresponds to a plurality of second media items of the second user, where each of the plurality of second media items includes at least one of video content or audio content; identify a set of rules for how to select a second media item to be integrated with the first media item of the second user from the plurality of second media items, where the set of rules is provided by the second user and includes one or more conditions related to the first user; select a second media item from the plurality of second media items based on the set of rules; and transmit the first media item and the selected second media item to the first user device of the first user, where the selected second media item is to be integrated with the first media item for presentation to the first user.

[0024] Each of the plurality of second media items may comprise unique video content or audio content.

[0025] The set of rules for how to select a second media item to be integrated with the first media item of the second user may further comprise rules for selecting a second media item based on a set of rankings provided by the second user, where each of the plurality of second media items may be associated with a ranking in the set of rankings.

[0026] Selecting a second media item based on the set of rules may comprise identifying a second media item associated with the highest ranking from among the plurality of second media items and determining whether one or more conditions related to the first user are satisfied with respect to the second media item.

[0027] Alternatively or additionally, selecting a second media item based on the set of rules may comprise identifying a third media item associated with the highest ranking from among the plurality of second media items, determining that one or more conditions related to the first user are not satisfied with respect to the third media item, identifying a second media item associated with a ranking lower than the third media item associated with the highest ranking from among the plurality of second media items, and determining that one or more conditions related to the first user are satisfied with respect to the second media item.

[0028] According to another aspect of the present disclosure, a system is provided that includes a memory and a processing device coupled to the memory. The processing device is configured to receive, at a server, a request from a first user for a first media item of a second user, where the first media item includes at least one of video content or audio content; determine that the first media item of the second user corresponds to a plurality of second media items of the second user, where each of the plurality of second media items includes at least one of video content or audio content; identify a set of rules for how to select a second media item to be integrated with the first media item of the second user from the plurality of second media items, where the set of rules is provided by the second user and includes rules for selecting a second media item based on one or more conditions related to the first user and a set of rankings provided by the second user, and where each of the plurality of second media items is associated with a ranking in the set of rankings; select a second media item from the plurality of second media items based on the set of rules; determine a time-based index of a timeline of the first media item, where the time-based index indicates a location on the timeline of the first media item at which the content of the selected second media item will be presented to the first user; and transmit the first media item, the selected second media item, and the time-based index to a first user device of the first user, where the selected second media item will be integrated with the first media item at the location corresponding to the time-based index on the timeline of the first media item. Each of the plurality of second media items may include unique video content or audio content.The first user device may render the first media item and the selected second media item as a composite media item, where the selected second media item is to be integrated with the first media item, and the transition between the first media item and the selected second media item is not visible to the first user.

[0029] According to some aspects of the present disclosure, there is provided a non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations, the operations including: requesting a first media item of a second user; receiving a request of a first user, where the first media item includes at least one of video content or audio content; determining that the first media item of the second user corresponds to a plurality of second media items of the second user, where each of the plurality of second media items includes at least one of video content or audio content; identifying a set of rules for how to select a second media item to be integrated with the first media item of the second user from the plurality of second media items, where the set of rules is provided by the second user and includes one or more conditions related to the first user; selecting a second media item from the plurality of second media items based on the set of rules; and transmitting the first media item and the selected second media item to a first user device of the first user, where the selected second media item is to be integrated with the first media item for presentation to the first user.

[0030] Each of the plurality of second media items may comprise unique video content or audio content.

[0031] A set of rules on how to select a second media item to be integrated with a first media item of a second user may further comprise rules for selecting the second media item based on a set of rankings provided by the second user, where each of the plurality of second media items may be associated with a ranking in the set of rankings.

[0032] Selecting the second media item based on the set of rules may comprise identifying the second media item associated with the highest ranking from among the plurality of second media items and determining that one or more conditions related to the first user are satisfied with respect to the second media item.

[0033] Alternatively or in addition, selecting the second media item based on the set of rules may comprise identifying the third media item associated with the highest ranking from among the plurality of second media items, determining that one or more conditions related to the first user are not satisfied with respect to the third media item, identifying the second media item associated with a ranking lower than the ranking of the third media item associated with the highest ranking from among the plurality of second media items, and determining that one or more conditions related to the first user are satisfied with respect to the second media item.

[0034] According to some aspects of the present disclosure, there is provided a non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to execute the methods described above.

[0035] Any individual feature and / or combination of features defined above in accordance with any aspect of the present disclosure, or hereinafter with respect to any particular implementation, may be utilized in any other aspect or implementation, either separately or in any combination, alone or in combination with any other features defined. Further, the present disclosure is intended to encompass an apparatus configured to perform any feature described herein with respect to a method, and / or a method of using or manufacturing using or manufacturing any feature of an apparatus described herein.

[0036] The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate like elements. Note that different references to "an" or "one" implementation in the present disclosure do not necessarily refer to the same implementation, although such references are meant to imply at least one.

Brief Description of the Drawings

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0038] Aspects of the present disclosure are directed to providing a content owner's requested media content along with integrated supplementary content customized for the requesting viewer. A content sharing platform enables a content owner to upload media content (e.g., a video item or an audio item) for presentation to various users of the content sharing platform. A content owner may wish to include supplementary content within his or her media content. For example, a content owner may wish to prompt an action to his or her viewers to achieve an objective such as causing the viewer to perform a task related to the media content. Conventional content sharing platforms enable a content owner to include one or more messages within a media item. However, each user consuming the media item is typically presented with the same message including the prompted action, regardless of whether the user is likely to complete the prompted action (e.g., whether the user has already completed the prompted action in the past) or whether the prompted action is applicable to the user. If the same message is continuously presented to the user even though the user has already completed the prompted action (or the prompted action is not applicable to the user), the user may no longer be interested in viewing and / or listening to the content owner's media item, and the content owner's objective may not be achieved.

[0039] In another example, a media item may include content that is considered handling - sensitive for some groups of viewers. That is, certain users may not be permitted access to view and / or listen to a media item that includes handling - sensitive content. Traditional content - sharing systems generally do not provide an option for content owners to supply an alternative version of the content, where the handling - sensitive part of the content is replaced with proprietary audio or video content, thus enabling a user to view and / or listen to the media item without being exposed to the handling - sensitive part of the content.

[0040] Implementations of the present disclosure address the above and other deficiencies by performing dynamic integration of supplementary content customized for the user who requested the first media content into the first media content of the content owner. As used herein, "dynamic integration" refers to content integration that occurs almost immediately (i.e., within the minimum delay caused by network conditions) after a user request for the first media item is submitted, where different supplementary content is dynamically integrated into the first media content depending on the user who requested the content and the timing of the request. Specifically, the dynamic content integration system of the present disclosure may enable a content owner to upload a first media item to a content sharing platform and also upload additional media items ("second media items") that can be used as supplementary content for the first media item (when an end-user request for the first media item is received). The dynamic content integration system may also enable the content owner to specify rules that define how one or more second media items should be selected for a particular user who requests the first media item. Thereafter, the dynamic content integration system may receive a request from a first user for the first media item (a video item or an audio item), and may respond to the request by selecting a particular second media item to be integrated with the first media item for presentation to the first user from among a plurality of second media items associated with the first media item, where the second media item is selected specifically for the first user. Each of the plurality of second media items may include unique video content and / or audio content. The second media item may be selected based on a set of rules provided by a second user.The set of rules may include one or more conditions related to a first user and may specify how to select a second media item from a plurality of supplied second media items (e.g., based on the conditions and, in some implementations, also based on a set of rankings provided by a second user). In response to the selection of the second media item, the first media item and the selected second media item may be sent to the user device of the first user. The user device may render the first media item and the second media item as a composite media item, where the second media item is integrated with the first media item for presentation to the first user.

[0041] Accordingly, the technical effect may include providing a composite media item to a first user, where the composite media item includes first media content and dynamically integrated additional content that is selected, particularly for the first user, from media content items supplied by a content owner to supplement the first media content. This prevents a portion of the content within the composite media item that is unrelated to the first user or not permitted to be consumed by the first user from being presented to the first user. For example, the first media content (e.g., a first media item) may relate to the repair process of a family of devices with various different models, and the dynamically integrated additional content (e.g., a second media item) may prompt the first user to perform specific maintenance actions depending on the model of the device owned by the first user. That is, if a third user has a different model, the third user may be presented with additional content that is different from the additional content presented to the first user while consuming the first media item and is suitable for the model owned by the third user. By providing unique additional content to the user based on one or more characteristics of the user, a more efficient human-machine interface may be provided, such a human-machine interface being seamlessly incorporated within the first media content and customized for the user. In another example, if the first user is not a subscriber to the second user's media channel, the first user may be presented with a message prompting the first user to subscribe to the media channel while consuming the second user's first media item. If a third user is a subscriber to the media channel, the third user may be presented with a message different from the message prompting the second user to subscribe to the video channel while consuming the first media item (e.g., a message prompting the third user to contribute to a fan funding campaign, purchase a product or a live event ticket, etc.).By providing a unique message to the user based on one or more characteristics, the system resources are no longer wasted in presenting messages that are irrelevant to the user, the user's trust / interest in the content sharing platform is improved, and the content owner can now promote the desired purpose without continuously presenting messages that may be ignored by many users.

[0042] FIG. 1 shows an example of a system architecture 100 for providing a requested media item along with an integrated supplementary media item customized for the requesting user, according to an implementation of the present disclosure. The system architecture 100 includes client devices 110A - 110Z (also referred to herein as user devices or endpoint devices), one or more networks 105, one or more servers 106, one or more electronic devices 170, and one or more platforms (e.g., content sharing platform 120, advertising platform 165, mobile platform 150, social network platform 160, content provider platform 195, etc.). The platforms can include one or more computing devices (such as rack - mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc.) and one or more data storage devices (e.g., hard disks, memories, and databases), and may be coupled to one or more networks 105.

[0043] One or more networks 105 can include one or more public networks (e.g., the Internet), one or more private networks (e.g., a local area network (LAN) or one or more wide area networks (WAN)), one or more wired networks (e.g., an Ethernet network), one or more wireless networks (e.g., an 802.11 network or a Wi-Fi network), one or more cellular networks (e.g., a long term evolution (LTE) network), routers, hubs, switches, server computers, and / or combinations thereof. In one implementation, some components of architecture 100 are not directly connected to each other. In one implementation, architecture 100 includes a separate network 105.

[0044] One or more data storage devices (not shown) can be present in memory (e.g., random access memory), cache, devices (e.g., hard drives), flash drives, etc., and can be part of one or more database systems, one or more file systems, or another type of component or device capable of storing data. One or more data storage devices can also include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data storage device can be a persistent storage device capable of storing data. The persistent storage device can be a local storage unit or a remote storage unit, an electronic storage unit (main memory), or a similar storage unit. The persistent storage device can be a monolithic device or a distributed set of devices. As used herein, "set" refers to any positive integer number of items.

[0045] Content item 121 (e.g., a media item) can be stored on one or more data storage devices. The data storage devices can be part of one or more platforms. Examples of content item 121 can include, but are not limited to, digital video, digital movie, animation image, digital photo, digital music, digital audio, digital video game, collaborative media content presentation, website content, social media update, electronic book (e - book), electronic magazine, digital newspaper, digital audio book, electronic journal, web blog, Real Simple Syndication (RSS) feed, electronic comic magazine, software application, etc. The content item 121 can be pre - recorded or can be a live stream. For brevity and simplicity, the media item can be used as an example of content item 121 throughout this specification. The content item 121 can be provided by a content provider. The content provider can be a user, a company, an organization, etc., and can provide the content item 121 via either the content provider platform 195 or one of the client devices 110A - 110Z. The content provider can provide the content item 121 which is a media item. For example, the content item 121 can be a video item provided by a content owner. In another example, the content item 121 can be an audio item provided by a content owner.

[0046] Client devices 110A - 110Z can include devices such as televisions, smartphones, cellular phones, personal digital assistants (PDAs), portable media players, netbooks, laptop computers, e - book readers, tablet computers, desktop computers, set - top boxes, game consoles, etc. Individual client devices 110A - 110Z can include a communication application 112. Content item 121 can be consumed via the communication application 112. As used herein, "media", "media item", "online media item", "digital media", "digital media item", "content", "media content item", and "content item" can include electronic files that can be executed or loaded using software, firmware, or hardware configured to present content item 121. In one implementation, the communication application 112 is one or more applications that enable a user to create, send, and receive content item 121 (e.g., video item and / or audio item) via a platform (e.g., content sharing platform 120, recommendation platform 157, advertising platform 165, mobile platform 150, social network platform 160, search platform 145, collaboration platform 155, and content provider platform 195), and / or a combination of platforms and / or networks.

[0047] For example, communication application 112 may be a content provider's social networking application, video sharing application, video streaming application, video game streaming application, video on demand application, photo sharing application, chat application, mobile application, or a combination of such applications. The communication applications in client devices 110A - 110Z can render, display, and / or present one or more content items 121 (e.g., video items and / or audio items) to one or more users. For example, communication application 112 can provide one or more user interfaces 114 (e.g., graphical user interfaces) to be displayed on client devices 110A - 110Z for providing, receiving, and / or playing video content. In some implementations, communication application 112 may render a first content item (e.g., a first media item) as a composite content item together with a second content item (e.g., a second media item).

[0048] In one implementation, the communication application 112 may include a user interface 114. The user interface may be used to search for content items 121 and may play the content item 121 within the content viewer 116 portion of the user interface 114. The content viewer 116 may render, display, and / or present the content item 121 (e.g., a media item) to one or more users. In one implementation, the content viewer 116 is embedded within an application (e.g., the communication application 112). In another implementation, the content viewer 116 may be a stand-alone application (e.g., a mobile application, a desktop application, a gaming console application, a television application, etc.), such as the communication application 112 that enables a user to consume (e.g., play, display) content items 121 such as videos, images, documents (e.g., web pages). For example, the content viewer 116 may be a web browser that can access, retrieve, present, and / or navigate web pages (e.g., hypertext markup language (HTML) pages, digital media items, etc.) served by a platform's web server. In another example, the content viewer 116 may display an embedded media player (e.g., a Flash® player or an HTML5 player) embedded within a document (e.g., the user interface 114 (web page)).

[0049] The content viewer 116 can be provided to the client devices 110A-110Z by the server 106 and / or the platform. For example, the content viewer 116 can be an embedded media player embedded in the user interface 114 (e.g., a document (web page), or a screen of a stand-alone application) provided by the content sharing platform 120 or the content provider platform 195. In another example, the content viewer 116 can be an application downloaded from a platform (e.g., the content sharing platform 120, the recommendation platform 157, the advertising platform 165, the mobile platform 150, the social network platform 160, the search platform 145, the collaboration platform 155, and the content provider platform 195). In another example, the content viewer 116 can be a stand-alone application pre-installed on the client devices 110A-110Z.

[0050] The client devices 110A-110Z may further include a content storage device 118. The content storage device 118 can be a storage system on the client devices 110A-110Z that stores one or more media item files that are transmitted to and received from the server 106 and / or the content sharing platform 120.

[0051] One or more servers 106 may include computing devices such as rack-mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc., and may be coupled to one or more networks 105. One or more servers 106 may be part of any of independent devices, or platforms (e.g., content sharing platform 120, recommendation platform 157, advertising platform 165, mobile platform 150, social network platform 160, search platform 145, content provider platform 195, and collaboration platform 155). The server 106 may host a dynamic content integration system 108 that can include a rule creation component 122 and a media selection component 124. The rule creation component 122 may provide a user interface that enables a content owner who uploads a first media item to a content sharing platform 120 to specify additional media items (i.e., second media items) that can be used as supplementary content for the first media item. The user interface may also enable the content owner to specify rules that define how to select one or more second media items for a particular user who requests the first media item. The rule creation component 122 stores information identifying the association between the first media item and the second media item in a data storage device together with the rules provided by the content owner.

[0052] The media selection component 124 can identify a first media item to be presented to the user on the client devices 110A-110Z in response to a user request. A user of the client devices 110A-110Z may send a request for the first media item, and such a request is received by the media selection component 124. The media selection component 124 can identify information related to the requested first media item, including the location of the first media item. In some implementations, the identified information may also include an indication (e.g., in the metadata of the first media item) that the first media item should be presented with supplementary content. The rule creation component 122 can analyze the information related to the first media item and determine that the first media item is associated with a plurality of second media items that can be used as supplementary content for the first media item.

[0053] In response to the determination that the first media item should be presented with supplementary content, the media selection component 124 can identify a plurality of second media items associated with the first media item (e.g., from a data storage device that stores associations between the first media item and the plurality of second media items that can be used as supplementary content for the first media item). The media selection component 124 can further identify a set of rules for selecting a particular second media item related to the user who requested the first media item from the plurality of second media items. According to the implementations described herein, the media selection component 124 can select a second media item from the plurality of second media items based on the identified set of rules. Upon selecting the second media item, the requested first media item and the selected second media item can be transmitted to the client devices 110A-110Z.

[0054] The content provider platform 195 can provide services of service providers. For example, the content provider can be a video streaming service provider that creates a media streaming service via the communication application 112 for users to play videos, TV shows, video clips, audio, audio clips, and movies on client devices 110A to 110Z via the content provider platform 195.

[0055] The social network platform 160 can provide an online social networking service. The social networking platform 160 can provide the communication application 112 for users to create profiles and perform activities using their profiles. Activities can include updating profiles, exchanging messages with other users, evaluating status updates, photos, videos, etc. (such as liking, commenting, sharing, recommending), and receiving notifications of other user activities.

[0056] The mobile platform 150 can be used to enable users to connect to information, share information, and / or interact with each other using one or more mobile devices (such as phones, tablet computers, laptop computers, wearable computing devices, etc.) and / or any other suitable devices. For example, the mobile platform 150 can enable telephone communication, short message service (SMS) messaging, multimedia message service (MMS) messaging, text chat, and / or any other communication between users. The mobile platform 150 can support user communication via video messaging, video chat, and / or video conferencing.

[0057] The content sharing platform 120 can be used to provide one or more users with access to content items 121 and / or to provide one or more users with content items 121. For example, the content sharing platform 120 can enable users to consume, upload, download, and / or search for content items 121. In another example, the content sharing platform 120 can enable users to evaluate content items 121, such as by liking, disliking, recommending, sharing, rating, and / or commenting on content items 121. In another example, the content sharing platform 120 can enable users to edit content items 121. The content sharing platform can also include, for example, a website (e.g., one or more web pages) and / or one or more applications (e.g., communication application 112) that can be used to provide one or more users with access to content items 121 via client devices 110A-110Z. The content sharing platform 120 can include any type of content delivery network that provides access to content items 121.

[0058] The content sharing platform 120 can include a plurality of channels (e.g., Channel A 125 to Channel Z 129). A channel can be data content available from a common source, or data content having a common topic or theme. The data content can be digital content selected by a user, digital content made available by a user, digital content uploaded by a user, digital content selected by a content provider, digital content selected by a broadcaster, etc. For example, Channel A 125 may include Videos Y and Z. A channel can be associated with an owner who is a user who can perform actions on the channel. The data content can be one or more content items 121. The data content in a channel can be pre-recorded or can be live streaming. Although a channel is described as one implementation form of the content sharing platform 120, the implementation forms of the present disclosure are not limited to content sharing platforms that provide content items 121 via a channel model.

[0059] FIG. 2 is a block diagram showing a dynamic content integration system 108 hosted by a server 106 according to some implementation forms of the present disclosure. The dynamic content integration system 108 includes a rule creation component 122 and a media selection component 124. The dynamic content integration system 108 may include a larger or smaller number of components or modules without loss of generality. For example, two or more of the components may be combined into a single component, or the features of a component or module may be divided among two or more components. In one implementation form, one or more of the components may be present on different computing devices (e.g., a server device and a client device).

[0060] Generally, the functions described in one implementation as being performed by server 106 may be performed by a plurality of different servers 106 in other implementations. For example, server 106 may execute a program that performs one or more functions of rule creation component 122, and different devices may execute one or more of the functions of media selection component 124. The functionality attributable to a particular component may be performed by different components or multiple components operating together.

[0061] The rule creation component 122 may include a user interface module 202 that generates a rule creation interface and provides it for presentation to the content owner. The rule creation user interface can enable the content owner who uploaded the first media item (identified in the first media item data 222 of the data storage device 220) to specify a second media item (identified in the second media item data 224 of the data storage device 220) that can be used as supplementary content for the first media item. The rule creation user interface may also enable the content owner to specify rules that define how one or more second media items should be selected for a particular user who requests the first media item. The rule creation component 122 stores information identifying the association between the first media item and the second media item together with the rules provided by the content owner as media item selection rules 228.

[0062] The rule creation user interface may further enable a content owner to specify a time-based index associated with a first media item. The time-based index may indicate a location on the timeline of the first media item where supplementary content is to be integrated with the first media item. In some implementations, the content owner may specify the time-based index by interacting with the timeline of the first media item via the rule creation user interface. The rule creation component 122 stores information identifying the time-based index as time-based index data 230. An exemplary rule creation user interface is described in more detail below with reference to FIG. 6.

[0063] The rule creation component 122 may further include a media item analysis module 204 that analyzes information related to the first media item that is the subject of a user request to determine that the first media item is to be associated with a plurality of second media items.

[0064] The media selection component 124 may include a media item identification module 206 that, in response to a user request, identifies a first media item to be presented to the user on the client device. The first media item may be provided by a content owner, may be stored in the data storage device 220, or may be identified as first media item data 222. The first media item data 222 may include first media item details and optionally first media item content. The first media item details may include information about the first media item such as a title, description, comment, storage location, file name, author, source, file size, duration, format, resolution, video image dimensions, edit time or creation time, other details, or a combination thereof. The first media item details may further include an indication that a second media item is associated with the first media item (e.g., an entry for the first media item in the data storage device 220 may include one or more identifiers representing the second media item). The second media item may also be provided by a content owner. The second media item can be stored in the data storage device 220 or can be identified as second media item data 224, and the second media item data 224 includes second media item details and optionally second media item content. The second media item details may include information about the second media item such as the purpose associated with the second media item, the ranking associated with the second media item, description, comment, storage location, file name, author, source, file size, duration, format, resolution, video image dimensions, edit time or creation time, other details, or a combination thereof.In response to receiving a user request for a first media item and identifying the first media item from the data storage device 220, the media item analysis module 204 of the rule creation component 122 may analyze the first media item details of the first media item data 222 to determine that the first media item is associated with a plurality of second media items.

[0065] The media selection component 124 may include a media item selection module 210. The media item selection module 210 may select one or more second media items for a user requesting the first media item based on user data 226 and media item selection rules 228. The user data 226 may include data about one or more users of a content sharing platform (e.g., the content sharing platform 120 of FIG. 1). The user data 226 may include one or more measured values for a particular user or for a group of users. In one example, the user data 226 may include characteristics of a group of users, consumption data, other data, or combinations thereof. The characteristics of the group of users may provide details about the group of users and may include, for example, the location, language, and / or other similar information of the users. The consumption data may be specific to the user or specific to the requested first media item and may include the duration of consumption, the number of users, the drop off rate, the parts watched and / or re-listened to again, the parts paused or zoomed in, other measured values, or combinations thereof.

[0066] The media selection component 124 may also include a time-based index determination module 208 that determines a time-based index of the timeline of the requested first media item indicating one or more locations (e.g., times) in the first media item that are before, after, or within the first media item at which supplemental content of the selected one or more second media items will be presented to the user. The time-based index determination module 208 may use the time-based index data 230 of the data storage device 220 to determine the time-based index. In one example, the time-based index data 230 may include a time-based index determined based on input from the content owner (e.g., based on the content owner interacting with the timeline of the first media item via a rules creation user interface).

[0067] In another example, the time-based index can be determined using a trained machine learning model. In such an example, the trained machine learning model can detect the optimal time at which to present the supplementary content of the second media item to the user. In one implementation, the trained machine learning model is trained from data regarding a plurality of users who consumed the supplementary content of one or more selected second media items presented with the first media item in response to a request for the first media item. The data may include information regarding different locations on the timeline of the first media item at which the supplementary content was presented to the user. The data may further include a success metric associated with each location, where the success metric indicates the number of users who performed an action prompted by the supplementary content. The data may also include information regarding one or more characteristics related to one or more users who performed the prompted action. The machine learning model can be trained using the location data, the success metric data, and the one or more characteristic data. As a result, the trained machine learning model can be applied to the current input data, which identifies the first media item, the selected second media item, and the user who requested the first media item, to obtain an output that can specify or indicate an optimized location on the timeline of the first media item for presentation of the supplementary content of the selected second media item. The time-based index may indicate the identified optimized location.

[0068] When one or more second media items are selected, the requested first media item, the selected second media item, and the time-based index may be sent to the client device for presentation to the user. The supplementary content of the selected one or more media items can be presented to the user at the location on the timeline of the first media item corresponding to the time-based index.

[0069] The data storage device 220 may include a memory (e.g., random access memory), a drive (e.g., hard drive, solid state drive), a database system, a cache mechanism, or other types of components or devices capable of storing data. The data storage device 220 may also include a plurality of storage components (e.g., a plurality of drives or a plurality of databases) that may span multiple computing devices (e.g., multiple server computers). In some implementations, the data storage device 220 may be cloud-based. One or more of the components may utilize the data storage device 220 to store public and private data, and the data storage device 220 may be configured to provide secure storage for private data.

[0070] In situations where the systems described in this specification may collect or use personal information about a user (such as the characteristics and / or consumption history described above), an opportunity may be provided to the user to control whether the systems, programs, or features described in this specification may enable the collection of user information, or to control whether and / or how content may be received from a server that may be more relevant to the user. For example, a control may be provided to the user that enables the user to make selections regarding both whether the systems, programs, or features described in this specification may enable the collection of user information and when it may do so, and regarding whether content or communications are sent to the user from a server. Additionally, some data may be treated in one or more ways before being stored or used so that personally identifiable information is removed. For example, a user's identity may be treated so that personally identifiable information about the user cannot be determined, or a user's geographic location may be generalized to a location (such as at the city level, zip code level, or state level) where location information is obtained so that the user's specific location cannot be determined. Accordingly, the user may have control over how information is collected about the user and used by server 106.

[0071] Figures 3-4 illustrate an exemplary method for providing a requested media item, along with a customized supplemental media item, to a user making the request, according to various aspects of the present disclosure. Figure 5 illustrates a related exemplary method for providing a live media stream, along with integrated supplemental content customized for viewers of the live media stream, according to various aspects of the present disclosure. For simplicity of explanation, the methods are illustrated and described as a series of acts. However, acts in accordance with the present disclosure can be performed in various orders and / or in parallel, and with other acts not presented and described herein. Further, acts that are not all illustrated may be performed in implementing the methods according to the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods can alternatively be represented as a series of states interrelated via a state diagram or events. Additionally, it should be understood that the methods disclosed herein can be stored on a manufacture's article to facilitate transporting and transmitting such methods to a computing device. As used herein, the term manufacture's article is intended to encompass a computer program accessible from any computer-readable device or storage medium.

[0072] Moreover, various acts are described in detail above with respect to each of the system diagrams. It should be understood that the detailed description of such acts in the previous figures may be implemented and is intended to be implemented in accordance with the following methods.

[0073] The method may be executed by processing logic that may include hardware (circuitry, dedicated logic, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. In some implementations, some or all of the operations of the method may be executed by server 106 of FIG. 1.

[0074] Next, referring to FIG. 3, the flowchart illustrates an exemplary method 300 for providing a requested media item, according to some implementations of the present disclosure, along with integrated supplementary media items customized for the user requesting the item. At block 310, a request from a first user is received that requests a first media item of a second user. In some examples, the first user may be a viewer interacting with a user interface of a content sharing platform, and the second user may be a content owner who provided the first video item to the content sharing platform. The request may include information regarding the first user who sent the request for the first media item. The request may include an identifier associated with the first user or the first user's user device. In some implementations, the identifier may correspond to a user account of the first user (e.g., on content sharing platform 120) that indicates or is associated with one or more characteristics associated with the first user. The user account may also indicate or be associated with a set of one or more prior media channels that the first user has a consumption history with or has opted into (i.e., opted to receive displays in response to the provision of additional media items by a media provider).

[0075] In block 320, it is determined that a first media item corresponds to a plurality of second media items of a second user. The plurality of second media items may be provided by the second user together with the first media item. Each of the plurality of second media items may include unique audio content and / or video content. In some implementations, each of the plurality of second media items may include a message that prompts an action for the first user. For example, the second media item may include a message that prompts the first user to complete an action associated with the second media item, such as joining the second user's media channel, engaging with the first media item (e.g., selecting a "like" button included on a user interface that displays the first media item). In another example, the message may prompt the first user to perform an action related to the second user but not associated with the second media item, such as contributing to a fan funding campaign, purchasing a product sponsored by the second user, purchasing a ticket to a live event featuring the second user, joining the second user's second media channel, purchasing a product recommended by the second user. Alternatively, the second media item may include a message from the second user expressing gratitude to the first user for being a fan of the media item provided by the second user (without prompting the first user to perform an action).

[0076] In block 330, a set of rules is identified as to how to select a particular second media item related to a user who requested a first media item from a plurality of second media items. The set of rules may be provided by a second user along with the first media item and the second media items. The set of rules may include one or more conditions related to the first user, such conditions may correspond to one or more characteristics shown with or associated with the identified user account. For example, one condition may relate to whether the user is a subscriber to the second user's media channel, which may be indicated by the first user's user account as described above.

[0077] In some implementations, the set of rules may further include rules for selecting a second media item based on a set of rankings, where each of the plurality of second media items is associated with a ranking in the set of rankings. Each ranking associated with a second media item may correspond to an objective that the second user desires to achieve. For example, the second user may desire to acquire more subscribers to his or her video channel. A second media item that encourages the first user to subscribe to the second user's video channel may thus be selected to have the highest ranking by the second user.

[0078] In block 340, a second media item is selected from a plurality of second media items based on a set of rules. In some implementations, a second media item associated with the highest ranking can be identified from the plurality of second media items. It may be determined whether one or more conditions related to the first user are satisfied with respect to the second media item ranked highest. In some implementations, the one or more conditions related to the first user may be satisfied based on the characteristics of the first user, the consumption history of the first user, and / or the subscription data associated with the first user. For example, if the second media item ranked highest prompts the first user to subscribe to the second user's video channel, the condition related to the first user may be satisfied if the first user is not a subscriber of the media channel. If one or more conditions related to the first user are satisfied, the second media item may be selected for integration with the first media item.

[0079] In other implementations, a third media item may be associated with the highest ranking that can be identified from a plurality of second media items. It may be determined whether one or more conditions related to the first user are satisfied with respect to the highest ranked third media item. If one or more conditions related to the first user are not satisfied with respect to the third media item, a second media item may be identified, where the second media item is associated with a ranking lower than the highest ranked third media item. It may be determined whether one or more conditions related to the first user are satisfied with respect to the second media item. If one or more conditions related to the first user are satisfied, the second media item may be selected for integration with the first media item. If one or more conditions related to the first user are not satisfied, a fourth media item may be identified, where the fourth media item is associated with a ranking lower than the ranking of the second media item. It may be determined whether one or more conditions related to the first user are satisfied with respect to the fourth media item. This process may be repeatedly executed until one or more conditions related to the first user are satisfied with respect to one of the plurality of second media items or a default second media item is selected. As will be described in more detail with respect to FIG. 4, if the conditions related to the first user are not satisfied with respect to any of the ranked plurality of second media items, a default second media item may be selected for integration with the first media item.

[0080] In block 350, a time-based index of the timeline of the first media item is determined. The time-based index may indicate a location on the timeline of the first media item at which supplemental content of the selected second media item is to be presented to the first user. In some implementations, the time-based index may be created based on an interaction of a second user with the timeline of the first media item via a rules creation user interface. In other implementations, the time-based index may be determined by using a trained machine learning model as previously described herein.

[0081] In block 360, the first media item, the selected second media item, and the time-based index are sent to the first user's first user device. The first user device may integrate the selected second media item with the first media item such that the supplemental content is presented at the location of the first media item timeline corresponding to the time-based index. In some implementations, the first user device may render the first media item and the selected second media item as a composite media item. The composite media item may include the selected second media item integrated with the first media item, where the transition between the first media item and the second media item is not visible to the first user.

[0082] FIG. 4 shows a flowchart of another exemplary method 400 for providing a requested media item along with an integrated supplemental media item customized for a user requesting the media item, according to some implementations of the present disclosure. In block 410, a request from a first user for a first media item of a second user is received. The request may include information regarding the first user who sent the request for the first media item. In one implementation, the request may include an identifier associated with the first user's user account.

[0083] In block 420, it is determined that a first media item corresponds to a plurality of second media items of a second user. The plurality of second media items may be provided by the second user together with the first media item. Each of the plurality of second media items may include unique video content and / or audio content. In some implementations, each of the plurality of second media items may include a message that prompts an action for a first user.

[0084] In block 430, a set of rules is identified as to how to select a particular second media item related to the user who requested the first media item from the plurality of second media items. The set of rules may also be provided by the second user together with the first media item and the second media items. The set of rules may include one or more conditions related to the first user, such conditions corresponding to one or more characteristics indicated or associated with the first user's user account. In some implementations, the set of rules may further include rules for selecting second media items based on a set of rankings, where each of the plurality of second media items is associated with a ranking in the set of rankings. Each ranking associated with a second media item may correspond to a goal that the second user desires to achieve (such as obtaining more subscribers to the second user's media channel, etc.).

[0085] In block 440, based on the identified set of rules, it is determined that the conditions related to the first user are not met. The set of rules may be applied to each of the plurality of second media items according to the implementation forms described previously herein. In an exemplary example, one or more conditions related to the first user may relate to joining the second user's media channel, contributing to the second user's fan funding campaign, and purchasing a product of an item from the second user. If the first user has previously joined the media channel, contributed to the fan funding campaign, and purchased a product of an item from the second user, it may be determined that the conditions of the first user are not met.

[0086] In block 450, in response to the determination that the conditions related to the first user are not met, a default second media item is selected from the plurality of second media items. The default second media item may be identified by the second user as the second media item to be selected in response to the determination that the conditions related to the first user are not met. In one example, the default second media item may include a message from the second user expressing gratitude to the first user for being a fan of the media item provided by the second user. In another example, the second user may specify any second media item among the plurality of second media items (including a message prompting the first user to perform an action) as the one to be selected as the default second media item.

[0087] In block 460, a time-based index of the timeline of the first media item is determined. The time-based index may indicate a location on the timeline of the first media item at which supplementary content of the selected second media item will be presented to the first user. In some implementations, the time-based index may be created based on an interaction of a second user with the timeline of the first media item via a rules creation interface. In other implementations, the time-based index may be determined by using a trained machine learning model as previously described herein.

[0088] In block 470, the first media item, the default second media item, and the time-based index may be sent to the first user's first user device. In some implementations, a signal indicating that an option to not view and / or listen to (i.e., skip) the supplementary content of the default second media item will be provided to the first user may also be sent to the first user device. The first user device may integrate the default second media item with the first media item such that the supplementary content is presented to the first user at the location on the first media item timeline corresponding to the time-based index. In some implementations, the first user device may render the first media item and the default second media item as a composite media item. When the first user consumes the composite media item, the first user device may provide an option for the first user to continue viewing and / or listening to the composite media item without viewing and / or listening to the content of the default second media item in response to detecting a transition between the first media item and the default second media item.

[0089] Figure 5 shows a flowchart of another exemplary method 500 for providing a live media stream along with integrated supplementary content customized for viewers of the live media stream, according to some implementations of the present disclosure. At block 510, a request from a first user is received that requests a live media stream of a second user. The request may include information about the first user who sent the request for the live media stream. In one implementation, the request may include an identifier associated with the first user. The identifier may correspond to a user profile of the first user that includes one or more characteristics associated with the first user. At block 520, the live media stream may be sent to a first user device of the first user.

[0090] At block 530, a display for integrating a second media item from a plurality of second media items associated with the live media stream may be received. The plurality of second media items may be provided by the second user before the first user requests the live media stream. In some implementations, the display may be received in response to an interaction by the second user with a user interface element presented via a user interface on the second user device. In other implementations, the display may be received in response to detection of a time-based index corresponding to a timeline of the live media stream, where the time-based index is provided by the second user before the start of the live media stream.

[0091] In block 540, a set of rules is identified as to how to select a particular second media item related to a user who requested a live media stream from a plurality of second media items. The set of rules may be provided by a second user before the first user requests the live media stream. The set of rules may include one or more conditions related to the first user, such conditions corresponding to one or more characteristics indicated by or associated with the first user's user profile. In some implementations, the set of rules may further include rules for selecting the second media item based on a set of rankings, where each of the plurality of second media items is associated with a ranking in the set of rankings. Each ranking associated with the second media item may correspond to an objective that the second user desires to achieve.

[0092] In block 550, the second media item may be selected from the plurality of second media items based on the identified set of rules. The second media item may be selected according to the implementations described herein. In block 560, the selected second media item may be transmitted to the first user device of the first user for integration with the live video stream. In some implementations, supplementary content of the selected second media item may be presented to the first user as part of the live media stream immediately upon the selected second media item being received by the first user device.

[0093] FIG. 6 shows a user interface 600 that facilitates dynamic integration of supplementary media content into a content owner's primary media content, according to some aspects of the present disclosure. The user interface 600 may correspond to the rule creation user interface described with respect to FIG. 2.

[0094] The user interface 600 may include a first portion 602 and a second portion 604. The main media content 606 (first media item) may be presented via the first portion 602. The main media content 606 may be associated with a timeline 608, where the timeline 608 starts at a time stamp of 0:00 and ends at a time stamp indicating the completion of the main media content 606, indicating the duration of the first media item. For example, if the first media item has a duration of 6 minutes and 53 seconds, the time stamp indicating the completion of the main media content 606 may read "6:53" as shown in FIG. 6. A user of the content sharing platform (i.e., the content owner) may select a time-based index 612 indicating a location on the timeline 608 at which the content of the second media item will be presented to the viewer and / or listener. In one implementation, the user may interact with a slider bar element 610, and the user may move the slider bar element 610 along the length of the timeline 608 to identify the time-based index 612. As the user moves the slider bar element 610 along the length of the timeline 608, the time-based index 612 may change.

[0095] The second part 604 of the user interface 600 may present a second media item 614 such as provided by the user. The second part 604 may further include one or more user interface elements to enable the user to provide a set of rules for selecting the second media item 614 for integration with the first media item. The user interface elements may include a ranking element 616, a user condition element 618, and a default selection element 620. The ranking element 616 may enable the user to select a ranking for association with the second media item 614. In some implementations, the ranking element 616 may be a text box. The user condition element 618 may enable the user to provide one or more conditions for the selection of the second media item 614. In some implementations, the user condition element 618 may be a text box. In some implementations, the user may provide conditions for selecting the second media item 614 for integration with the first media item. In other implementations, the user may select from a range of conditions provided by the content sharing platform. The default selection element 620 may enable the user to select a default second media item 614. As shown, the default selection element 620 may be a checkbox, where the user selects the checkbox to indicate that the second media item 614 is the default second media item.

[0096] The second part 604 of the user interface 600 may further include one or more delete buttons 622 for each second media item 614. The user may select the delete button 622 associated with the second media item 614 to remove the second media item 614 from the plurality of second media items. The second part 604 of the user interface 600 may further include an add button 624 to enable the user to add the second media item 614 to the plurality of second media items.

[0097] The above description includes references to some user interface elements included in either the first portion 602 or the second portion 604 of the user interface 600, but it should be understood that such references are provided for purposes of illustration and are not considered limiting. Any of the user interface elements described above may be any one of a variety of user interface elements including, but not limited to, text fields, drop-down lists, check boxes, and the like.

[0098] FIG. 7 shows an interaction diagram among a first user device 702, a second user device 706, and a dynamic content integration system 704 for performing dynamic integration of supplementary media content into a primary media content of a content owner, according to an implementation of the present disclosure. The second user device 706 provides (708) a first media item for upload to the dynamic content integration system 704, according to one implementation. The dynamic content integration system 704 may correspond to the dynamic content integration system 108 of FIG. 1. The second user device 706 may also provide (710) one or more second media items for upload to the dynamic content integration system 704. The second user device 706 may provide the first media item and the one or more second media items from a content storage device 118 on the client device 110A via the network 105.

[0099] The second user device 706 may further provide a set of rules for selecting a second media item for integration with the first media item (712). The set of rules may be provided via the user interface 600 of FIG. 6. In some implementations, the second user device may further provide a time-based index indicating the location on the timeline of the first media item at which the second media item is to be presented to the first user (not shown). In some implementations, the first media item, the second media item, the set of rules, and the time-based index may be stored in the data storage device of the dynamic content integration system 704 (e.g., the data storage device 220 of FIG. 2).

[0100] The first user device 702 may access the first media item by sending a request for the first media item to the dynamic content integration system 704 (714). The first user device 702 may access the dynamic content integration system 704 via the network 105 using the user interface 114 of the communication application 112. Accessing the first media item can be done by entering an identifier of the location of the first media item stored in the data storage device (e.g., in the user interface 114).

[0101] The dynamic content integration system 704 may receive (716) a request for a first media item from the first user device 702. The dynamic content integration system 704 may process the request and determine (718) that the first media item is to be presented to the first user device 702. The dynamic content integration system 704 may identify (720) the first media item stored in a data storage device (e.g., data storage device 220). In response to the identification 720 of the first media item, the dynamic content integration system 704 may identify (722) a plurality of second media items associated with the first media item from the data storage device. The dynamic content integration system 704 may further identify (724) a set of rules for selecting a second media item from the plurality of second media items. In response to the identification of the set of rules, the dynamic content integration system 704 may select a second media item from the plurality of second media items based on the identified set of rules.

[0102] In response to the selection of the second media item, the dynamic content integration system 704 may transmit (728) the first media item and may transmit (730) the selected second media item to the first user device 702 for presentation to the first user. In some implementations, the dynamic content integration system 704 may also transmit a time-based index indicating the location on the timeline of the first media item at which supplementary content for the selected second media item is to be presented to the first user. In response to the receipt of the first media item, the selected second media item, and the time-based index, the first user device 702 may integrate (732) the selected second media item with the first media item at the location on the first media item timeline corresponding to the time-based index.

[0103] FIG. 8 shows an exemplary block diagram of a computing device operating in accordance with one or more implementations of the present disclosure. Computing system 800 can be the server 106 or client devices 110A - 110Z in FIG. 1. The machine can operate as a server or an endpoint machine in an endpoint server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a television, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of (sequential or otherwise) instructions that specify actions to be taken by that machine. Further, although only a single machine is illustrated, the term "machine" shall also be understood to include any collection of machines that individually or jointly execute a set of (or multiple sets of) instructions to perform any one or more of the methods described herein.

[0104] Exemplary computer system 800 includes a processing device (processor) 802, main memory 804 (e.g., dynamic random access memory (DRAM) such as read-only memory (ROM), flash memory, synchronous DRAM (SDRAM), double data rate (DDR SDRAM), or DRAM (RDRAM)), static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and data storage device 818, which communicate with each other via bus 840.

[0105] The processor (processing device) 802 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More specifically, the processor 802 can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a combination of instruction sets. The processor 802 can also be one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processor 802 is configured to execute instructions 805 (e.g., of the communication application 112) to perform the operations described herein.

[0106] The computer system 800 can further include a network interface device 808. The computer system 800 can also include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an input device 812 (e.g., a keyboard, an alphanumeric keyboard, a motion sensing input device, a touch screen), a cursor control device 814 (e.g., a mouse), and a signal generation device 820 (e.g., a speaker).

[0107] The data storage device 818 can include a non-transitory machine-readable storage medium 824 (also a computer-readable storage medium) on which one or more sets of instructions 805 that embody any one or more of the methods or functions described herein (e.g., of the communication application 112) are stored. The instructions can also reside, fully or at least partially, within the main memory 804 and / or within the processor 802 during execution of the instructions by the computer system 800, and the main memory 804 and the processor 802 also constitute machine-readable storage media. The instructions can further be transmitted or received via the network interface device 808 over the network 830.

[0108] In one implementation, the instructions 805 include instructions for providing a requested media item together with supplementary media items customized for the user requesting it. The computer-readable storage medium 824 (machine-readable storage medium) is shown in an exemplary implementation as a single medium, but the terms "computer-readable storage medium" and "machine-readable storage medium" are to be understood to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The terms "computer-readable storage medium" and "machine-readable storage medium" are also to be understood to include any medium that is capable of storing, encoding, or carrying a set of instructions for machine execution and that causes a machine to execute any one or more of the methods of the present disclosure. The terms "computer-readable storage medium" and "machine-readable storage medium" are to be correspondingly understood to include, without limitation, solid-state memory, optical media, and magnetic media.

[0109] References to "one implementation" or "an implementation" throughout this specification mean that the particular features, structures, or characteristics described in connection with the implementation are included in at least one implementation. Thus, the appearances of the phrases "in one implementation" or "in an implementation" in various places throughout this specification can, depending on the context, refer to the same implementation, but not necessarily. Further, the particular features, structures, or characteristics may be combined in any suitable manner in one or more implementations.

[0110] As used in any form of the specification or claims for implementing the invention, the terms "includes", "including", "has", "contains", their variations, and other similar terms are to be construed as open - ended terms, similar to the term "comprising", without excluding any additional or other elements.

[0111] As used in this application, terms such as "component", "module", "system", etc. generally refer to computer-related entities, i.e., either hardware (e.g., circuitry), software, a combination of hardware and software, or an entity related to an operable machine having one or more specific functions. For example, a component may be, without limitation, a process operating on a processor (e.g., a digital signal processor), a processor, an object, an executable file, an execution thread, a program, and / or a computer. As an example, both an application operating on a controller and the controller can be components. One or more components may exist within a process and / or within an execution thread, and a component may be localized on one computer and / or distributed between two or more computers. Further, a "device" can take the form of specially designed hardware, generalized hardware specialized by having software, which enables the hardware to perform a specific function (e.g., generating a point of interest and / or a descriptor), execute thereon, software on a computer-readable medium, or a combination thereof.

[0112] The above-described systems, circuits, modules, etc. have been described with respect to the interactions between several components and / or blocks. It can be understood that such systems, circuits, components, blocks, etc. can include those components, or some of the specified sub-components, specified components or sub-components, and / or additional components, and can be by various permutations and combinations of the above. The sub-components can also be implemented as components communicatively coupled to other components, rather than being included (hierarchically) within the parent component. Additionally, note that one or more components may be combined into a single component that provides aggregated functionality, or may be divided into several distinct sub-components, and that any one or more intermediate layers, such as a management layer, may be provided to communicatively couple such sub-components to provide integrated functionality. Any component described herein may also interact with one or more other components, not particularly described herein but known to those skilled in the art.

[0113] In addition, as used herein, the terms "example" or "exemplary" are used to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, the use of the terms "example" or "exemplary" is intended to present concepts in a concrete fashion. The term "or" as used in this application is meant to be inclusive rather than exclusive. That is, unless otherwise specified or clear from the context, "X employs A or B" is meant to be interpreted in any of the natural inclusive permutations. That is, "X employs A or B" is satisfied under any of the following instances: X employs A; X employs B; or X employs both A and B. In addition, the articles "a" and "an" as used in this application and the appended claims are generally to be construed to mean "one or more" unless otherwise specified or the context clearly dictates that the singular form is intended.

[0114] Finally, the implementations described herein include the collection of data representing a user and / or the user's activities. In one implementation, such data is collected only when the user gives consent to the collection of this data. In some implementations, the user is prompted to explicitly permit data collection. Further, the user may opt-in or opt-out of participating in such data collection activities. In one implementation, the collected data is anonymized before any analysis is performed to obtain any statistical patterns such that the user's identity cannot be determined from the collected data.

Description of Reference Numerals

[0115] 100 System architecture 105 Network 106 Server 108 Dynamic Content Integration System 110 Client Device 112 Communication Application 114 User Interface 116 Content Viewer 118 Content Storage Device 120 Content Sharing Platform 121 Content Item 122 Rule Creation Component 124 Media Selection Component 125 Channel A 129 Channel Z 145 Search Platform 150 Mobile Platform 155 Collaboration Platform 157 Recommendation Platform 160 Social Network Platform 165 Advertising Platform 170 Electronic Device 195 Content Provider Platform 202 User Interface Module 204 Media Item Analysis Module 206 Media Item Identification Module 208 Time-Based Index Determination Module 210 Media Item Selection Module 220 Data Storage Device 222 First Media Item Data 224 Second Media Item Data 226 User Data 228 Media Item Selection Rule 230 Time-Based Index Data 600 User Interface 602 First Part 604 Second Part 606 Main Media Content 608 Timeline 610 Slide bar element 612 Time-based index 614 Second media item 616 Ranking element 618 User condition element 620 Default selection element 622 Delete button 624 Add button 702 First user device 704 Dynamic content integration system 706 Second user device 800 Computer system 802 Processing device, processor 804 Main memory 805 Instruction 806 Static memory 808 Network interface device 810 Video display unit 812 Input device 814 Cursor control device 818 Data storage device 820 Signal generation device 824 Non-transitory machine-readable storage medium, computer-readable storage medium 830 Network 840 Bus

Claims

**Claim 1** Receiving, at a server, a request from a first user for a first media item of a second user, wherein the first media item is associated with a set of rules regarding how to select one or more additional media items corresponding to the first media item, the set of rules being provided by the second user and including one or more conditions related to a user requesting access to the first media item; Determining whether one or more conditions related to the first user, defined by the set of rules, are satisfied with respect to a second media item of the one or more additional media items; Providing the second media item for presentation via a first user device associated with the first user in response to a determination that the one or more conditions are satisfied; A computer-implemented method comprising the steps above. **Claim 2** The method of claim 1, wherein the one or more additional media items comprise at least one of proprietary video content or proprietary audio content. **Claim 3** The method of claim 1 or 2, wherein the set of rules regarding how to select the one or more additional media items corresponding to the first media item comprises rules for selecting the second media item based on a set of rankings provided by the second user, wherein each of the one or more additional media items is associated with a ranking in the set of rankings. **Claim 4** Determining whether one or more conditions related to the first user, defined by the set of rules, are satisfied with respect to the second media item; Identifying the second media item associated with the highest ranking from the one or more additional media items; The method of claim 3, including the steps above. **Claim 5** The step of determining whether one or more conditions related to the first user, defined by the set of rules, are satisfied with respect to the second media item, identifying a third media item associated with the highest ranking from the one or more additional media items; determining that the one or more conditions associated with the first user are not satisfied with respect to the third media item; identifying a second media item associated with a ranking lower than the third media item associated with the highest ranking from the one or more additional media items; determining that the one or more conditions associated with the first user are satisfied with respect to the second media item, comprising: The method according to claim 3.

6. The method according to any one of claims 1 to 5, wherein the first user device renders the first media item and the second media item as a combined media item, the second media item is to be integrated with the first media item, and the transition between the first media item and the second media item is not visible to the first user.

7. determining a time-based index of the timeline of the first media item, the time-based index indicating a location on the timeline of the first media item corresponding to the second media item; transmitting the time-based index to the first user device of the first user; comprising: The method according to any one of claims 1 to 6.

8. The method according to claim 7, wherein the time-based index of the timeline of the first media item is created based on an interaction of a second user with the timeline of the first media item.

9. The method according to claim 7, wherein the time-based index of the timeline of the first media item is determined using a trained machine learning model.

10. The first media item includes a live media stream, and the step of transmitting the first media item and the second media item to the first user device of the first user is transmitting the live media stream to the first user device of the first user; receiving, from a second user device of a second user, a display for integrating the second media item with the live media stream; in response to receiving the display for integrating the second media item with the live media stream, transmitting the second media item to the first user device of the first user, wherein the second media item is to be integrated with the live media stream; The method according to any one of claims 1 to 9.

11. A system comprising: a memory; a set of one or more processing devices coupled to the memory, the set of one or more processing devices being configured to: receive a request from a first user for a first media item of a second user, the first media item being associated with a set of rules for how to select one or more additional media items corresponding to the first media item, the set of rules being provided by the second user and including one or more conditions related to a user requesting access to the first media item; determine whether one or more conditions related to the first user, defined by the set of rules, are satisfied for a second media item of the one or more additional media items; in response to a determination that the one or more conditions are satisfied, provide the second media item for presentation via a first user device associated with the first user; performing operations comprising; a system.

12. The system according to claim 11, wherein the one or more additional media items comprise at least one of proprietary video content or proprietary audio content.

13. The set of rules regarding how to select the one or more additional media items corresponding to the first media item comprises rules for selecting the second media item based on a set of rankings provided by the second user, wherein each of the one or more additional media items is associated with a ranking in the set of rankings, for the system according to claim 11 or 12.

14. Determining whether one or more conditions defined by the set of rules and related to the first user are satisfied with respect to the second media item; Identifying the second media item associated with the highest ranking from the one or more additional media items; The system according to claim 13, comprising.

15. Determining whether one or more conditions defined by the set of rules and related to the first user are satisfied with respect to the second media item can be Identifying a third media item associated with the highest ranking from the one or more additional media items; Determining that the one or more conditions related to the first user are not satisfied with respect to the third media item; Identifying the second media item associated with a ranking lower than the third media item associated with the highest ranking from the one or more additional media items; Determining that the one or more conditions related to the first user are satisfied with respect to the second media item, including The system according to claim 13.

16. A computer-readable storage medium including instructions that, when executed by a set of one or more processing devices, cause the processing devices to perform operations, the operations being Receiving a request from a first user for a first media item of a second user, wherein the first media item is associated with a set of rules on how to select one or more additional media items corresponding to the first media item, the set of rules being provided by the second user and including one or more conditions related to a user who requests access to the first media item, Determining whether one or more conditions defined by the set of rules and related to the first user are satisfied with respect to a second media item of the one or more additional media items, Providing the second media item for presentation via a first user device associated with the first user in response to a determination that the one or more conditions are satisfied, A computer-readable storage medium.

17. The computer-readable storage medium according to claim 16, wherein the one or more additional media items comprise at least one of proprietary video content or proprietary audio content.

18. The set of rules on how to select the one or more additional media items corresponding to the first media item comprises rules for selecting the second media item based on a set of rankings provided by the second user, wherein each of the one or more additional media items is associated with a ranking in the set of rankings, the computer-readable storage medium according to claim 16 or 17.

19. Determining whether one or more conditions defined by the set of rules and related to the first user are satisfied with respect to the second media item, Identifying the second media item associated with the highest ranking from the one or more additional media items, The computer-readable storage medium according to claim 18, comprising.

20. Determining whether one or more conditions defined by the set of rules and related to the first user are satisfied with respect to the second media item is identifying a third media item associated with the highest ranking from the one or more additional media items; determining that the one or more conditions relevant to the first user are not satisfied with respect to the third media item; identifying a second media item associated with a ranking lower than the third media item associated with the highest ranking from the one or more additional media items; determining that the one or more conditions relevant to the first user are satisfied with respect to the second media item, and a computer-readable storage medium according to claim 18.

Images