Method, system, and medium for presenting user comments with timed references in synchronization with a media content item - Patents.com

Abstract

To provide a method, a system, and a medium for presenting a user comment, including timed reference, in synchronization with a media content item.SOLUTION: A method includes obtaining a plurality of comments associated with a media content item, determining a subset of the plurality of comments including at least one comment from the plurality of comments, allocating a set of time windows corresponding to segments of the media content item, associating one or more comments from the subset of comments with a time window from the set of time windows, and, in response to a request to receive comments synchronized with playback of the media content item, causing a first comment associated with the first time window to be presented concurrently with a corresponding first portion of the media content item and a second comment associated with the second time window to be presented concurrently with a corresponding second portion of the media content item.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Patent Application No. 17 / 666,459, filed February 7, 2022, which is incorporated herein by reference in its entirety.

[0002] The disclosed subject matter relates to methods, systems, and media for presenting user comments, including timed references, synchronized with a media content item. [Background technology]

[0003] Users frequently use websites associated with a particular service or applications associated with a particular service to browse media content (e.g., stream or download media content such as music, television programs, movies, videos, etc.), connect with other users (e.g., via social networking services), and perform tasks (e.g., purchase items, edit documents, receive or send messages, etc.).

[0004] Such services often include a section for viewers to submit comments regarding the media content being consumed. For example, a creator who uploads media content to a media sharing service may allow viewers to submit one or more comments to the uploaded media content, and the submitted comments may be presented along with the media content. Often, the comments include the viewer's attitude or reaction to the media content. In some cases, viewers can react to a particular moment in the media content by adding a timestamp of that moment along with any other comments to their comments. However, these comments are typically provided linearly in the comments section, with comments displayed in the order in which the comments were received from viewers (e.g., most recent comments first) or in an order based on the support of the comments from other viewers (e.g., most "liked" comments first). Therefore, reviewing comments submitted to a video or any other media content can be an onerous task for viewers. Summary of the Invention [Problem to be solved by the invention]

[0005] It is therefore desirable to provide a new mechanism for presenting user comments with timed references in synchronization with a media content item. [Means for solving the problem]

[0006] According to some embodiments of the disclosed subject matter, methods, systems, and media are provided for presenting user contributions that include timed references in synchronization with media content items.

[0007] According to some embodiments of the disclosed subject matter, there is provided a method for presenting user comments during media playback, the method including the steps of: obtaining a plurality of comments related to a media content item; determining from the plurality of comments a subset of comments that include at least one of a plurality of timed references to the media content item; determining a set of time windows, each time window of the set of time windows corresponding to a segment of the media content item; associating one or more comments from the subset of comments with a time window from the set of time windows, the time window including a time referenced in the comment; receiving a request to receive comments synchronized with playback of the media content item; and in response to receiving the request to receive comments synchronized with playback of the media content item, causing a first comment associated with a first time window to be presented concurrently with a first portion of the media content item corresponding to the first time window and a second comment associated with a second time window to be presented concurrently with a second portion of the media content item corresponding to the second time window.

[0008] In some embodiments, associating one or more comments from the subset of comments with a time window from the set of time windows further includes determining that a second plurality of comments is associated with the first time window.

[0009] In some embodiments, the method further includes determining a ranking for each of the second plurality of comments, hi some embodiments, the ranking is based on a number of likes associated with each of the second plurality of comments within the first time window.

[0010] In some embodiments, a series of time windows associated with the media content item are determined based on a density of comments having timed references from a plurality of timed references, and a second plurality of comments is associated with the time window having a start time and an end time in response to determining that the density of comments between the start time and the end time is greater than a threshold.

[0011] In some embodiments, the method further includes generating a filtered set of user-generated comments by filtering the plurality of user-generated comments to remove user-generated comments that include objectionable content, and determining from the filtered set of user-generated comments a subset of user-generated comments that include at least one of the plurality of timed references to the media content item.

[0012] In some embodiments, the method further includes presenting a selectable option to switch the user-generated comments from being presented as received from one or more viewers of the media content item to being synchronized with playback of the media content item.

[0013] According to some embodiments of the disclosed subject matter, a system for presenting user comments during media playback is provided, the system including: a memory; and a hardware processor configured to: obtain a plurality of comments related to a media content item; determine from the plurality of comments a subset of comments that include at least one of a plurality of timed references to the media content item; determine a set of time windows, each time window of the set of time windows corresponding to a segment of the media content item; associate one or more comments from the subset of comments with a time window from the set of time windows, the time window including a time referenced in the comment; receive a request to receive comments synchronized with playback of the media content item; and in response to receiving the request to receive comments synchronized with playback of the media content item, cause a first comment associated with a first time window to be presented concurrently with a first portion of the media content item corresponding to the first time window and a second comment associated with a second time window to be presented concurrently with a second portion of the media content item corresponding to the second time window.

[0014] According to some embodiments of the disclosed subject matter, there is provided a non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform a method for presenting user comments during media playback, the method including: obtaining a plurality of comments related to a media content item; determining from the plurality of comments a subset of comments that include at least one of a plurality of timed references to the media content item; determining a set of time windows, each time window of the set of time windows corresponding to a segment of the media content item; associating one or more comments from the subset of comments with a time window from the set of time windows, the time window including a time referenced in the comment; receiving a request to receive comments synchronized with playback of the media content item; and in response to receiving the request to receive comments synchronized with playback of the media content item, causing a first comment associated with a first time window to be presented concurrently with a first portion of the media content item corresponding to the first time window and a second comment associated with a second time window to be presented concurrently with a second portion of the media content item corresponding to the second time window.

[0015] According to some embodiments of the disclosed subject matter, a system for presenting user comments during media playback is provided, the system including: means for obtaining a plurality of comments related to a media content item; means for determining from the plurality of comments a subset of comments that include at least one of a plurality of timed references to the media content item; means for determining a set of time windows, each time window of the set of time windows corresponding to a segment of the media content item; means for associating one or more comments from the subset of comments with a time window from the set of time windows, the time window including a time referenced in the comment; means for receiving a request to receive comments synchronized with playback of the media content item; and means for, in response to receiving the request to receive comments synchronized with playback of the media content item, causing a first comment associated with a first time window to be presented concurrently with a first portion of the media content item corresponding to the first time window and a second comment associated with a second time window to be presented concurrently with a second portion of the media content item corresponding to the second time window.

[0016] Various objects, features, and advantages of the disclosed subject matter may be more fully understood by reference to the following detailed description of the disclosed subject matter when considered in conjunction with the following drawings, in which like reference numerals identify like elements and in which: [Brief explanation of the drawings]

[0017] [Figure 1] 1 is an example flow diagram of a process for presenting comments with timed references synchronized with a media content item, according to some embodiments of the disclosed subject matter. [Figure 2] 10A-10C illustrate examples of user-submitted comments related to a video, in accordance with some embodiments of the disclosed subject matter. [Figure 3A]10A-10C illustrate exemplary display screens including comments with timed references synchronized with media content items, in accordance with some embodiments of the disclosed subject matter. [Figure 3B] 10A-10C illustrate exemplary display screens including comments with timed references synchronized with media content items, in accordance with some embodiments of the disclosed subject matter. [Figure 4] FIG. 1 is an example block diagram of a system that may be used to implement the mechanisms described herein, in accordance with some embodiments of the disclosed subject matter. [Figure 5] FIG. 5 is an example block diagram of hardware that may be used in the server and / or user device of FIG. 4 in accordance with some embodiments of the disclosed subject matter. DETAILED DESCRIPTION OF THE INVENTION

[0018] According to various embodiments of the disclosed subject matter, a mechanism (which may include a method, a system, and a medium) is provided for presenting user comments, including timed references, in synchronization with a media content item.

[0019] In general, the mechanisms described herein can identify and present comments that are relevant to content presented within a media content item. More particularly, in response to associating comments with multiple time windows generated for a media content item and selecting a comment for each of the multiple time windows (e.g., the top comment for each time window, the top three comments for each time window, etc.), the mechanism can simultaneously present the selected comment or comments along with the media content item currently at the playback position corresponding to the associated time window.

[0020] In some embodiments, the mechanism may include retrieving comments associated with the media content item. The mechanism may, for example, determine from the retrieved comments a subset of comments that include a timed reference to a portion of the media content item. In another example, the mechanism may determine that each comment in the subset of comments is relevant to the portion of the media content item. In a more detailed example, for comments that do not include a reference to a specific time or timestamp in the text portion or body of the comment, the mechanism may determine that the comments in the subset of comments are relevant to a moment in the media content item (e.g., based on the text portion or body of the comment that describes or relates to a moment that occurred within the media content item) and may assign the comment to a timestamp or time window within the media content item that corresponds to the moment.

[0021] In some embodiments, the mechanism can determine a series of time windows for a media content item and allocate comments to each of the time windows. For example, the mechanism can generate a series of time windows based on the duration of the media content item (e.g., 5-second windows for a video having a 2-minute duration). Continuing with this example, comments can be allocated to each of these time windows based on an associated time reference (e.g., a comment including text referencing the 1:32 playback position of the video can be allocated to a 5-second time window having a start time of 1:31 and an end time of 1:35). In another example, the mechanism can generate time windows based on comment density, engagement data, or any other suitable criteria. In yet another example, the mechanism can generate time windows corresponding to specified chapters, segments, or other video sections within the media content item (e.g., chapters and chapter start times specified by the creator of the media content item, interesting scenes within the media content item, or sections automatically generated by a machine learning model based on increased user engagement, etc.).

[0022] In some embodiments, the mechanism may rank and / or select comments associated with each of the time windows. For example, the mechanism may rank comments based on relevance to content presented in the time window, engagement data, etc. Continuing with this example, the mechanism may select one or more comments to associate with a particular time window based on the ranking (e.g., the top three comments for the time window between 1:00 and 1:05 based on the ranking).

[0023] The mechanism can then present the timed comments synchronized with the playback of the media content item. For example, a first comment can be suggested or otherwise presented during the presentation of the media content item in an associated first time window, and a second comment can be suggested or otherwise presented during the presentation of the media content item in an associated second time window. These timed comments can be presented, for example, in response to receiving a user-selected interface element for receiving comments synchronized with the playback of the media content item. For example, a comment interface that currently presents comments in the order in which they were received (e.g., most recent comment first) can be updated to present the timed comments selected by the mechanism described herein. In another example, a comment overlay interface can be generated to present the timed comments selected by the mechanism described herein, and the comment overlay interface can be positioned proximate to the media content item (e.g., a comment overlay interface presented below the video, a comment overlay interface positioned in on top of the video, etc.).

[0024] These mechanisms may be used in any suitable application. For example, these mechanisms may be used to automatically modify a comment area associated with a media content item to present comments around the current timestamp of the media content item without requiring the user to scroll. In another example, these mechanisms may be used to identify user-generated content that references products or services that appear within a media content item. In a more detailed example, these mechanisms may be used to present timed references to particular products or services being promoted by the creator of the media content item, with an overlay interface presenting supplemental content related to the product or service that is relevant at the moment within the media content item. In another more detailed example, these mechanisms may be used to present timed advertisements or timed sponsor information during particular portions of a media content item.

[0025] 1 , an example flow diagram of a process 100 for presenting user-submitted comments, including timed references, synchronized with a media content item is shown, in accordance with some embodiments of the disclosed subject matter. In some embodiments, process 100 may be performed on a server, such as server 402, and / or a user device, such as user device 406, described below in connection with FIG.

[0026] It should be noted that process 100 may be performed at any suitable time. For example, in some embodiments, process 100 may be performed by a server in response to receiving a request to play a media content item from a user. In a more detailed example, process 100 may be performed on a server hosting a media content item in response to receiving a request to stream the media content item to a user device. In another example, in some embodiments, process 100 may be performed periodically in connection with any suitable indexing service that indexes and / or compiles statistical data related to media content items (e.g., number of views, likes or any other suitable endorsements, new and / or modified comments, etc.). In yet another example, in some embodiments, process 100 may be performed when process 100 (or any other suitable mechanism) detects a new comment related to a given media content item.

[0027] In some embodiments, process 100 may begin at 102 by retrieving comments or any other suitable user-generated contributions associated with a media content item using any suitable technique or combination of techniques. For example, comments associated with a media content item (e.g., a video uploaded to a media sharing service) may be received at a server device (e.g., server device 402 described in connection with FIG. 4) as submitted by a user of a user device (e.g., user device 404 described in connection with FIG. 4). In such an example, process 100 may access comments at the server device, request comments from the server device, and / or retrieve comments from the server device. In a more detailed example, in response to transmitting a request including a media content identifier identifying a media content item or a portion of a media content item, the server device may access comments and associated comment information from a comment database. The comment information in the comment database may include a comment identifier identifying the comment, an author identifier identifying a user account associated with a user or group of users, an author's representation of who created, modified, and / or posted the comment, a timestamp indicating when the comment was made, the content of the comment, a media content identifier identifying the media content item or portion of the media content item to which the comment relates, etc. As another example, the comments may be included in metadata associated with the media content item. In such an example, process 100 may access, request, and / or retrieve comments associated with the media content item simultaneously with metadata associated with the media content item (e.g., the title of the media content item, a description of the media content item, the content creator of the media content item, etc.).

[0028] 2 illustrates an example 200 in which comments related to a video have been submitted by multiple users, according to some embodiments. As shown, the comment interface 200 may include any number of comments 201-207. As further shown, the comments 201-207 associated with a particular video may be presented in the order in which the comments were submitted (e.g., most recent comment first).

[0029] In some embodiments, process 100 can identify a subset of the retrieved comments that include a reference to a particular time within the media content item. For example, in some embodiments, process 100 can identify, among the retrieved comments, comments associated with the media content item that include a timestamp within the content of the comment (e.g., "I see what you did at 1:30," "3:47," etc.). Additionally or alternatively, in some embodiments, process 100 can identify, among the retrieved comments, comments that include a character that indicates a time reference to a location within the media content item, such as including an "@" symbol within the content of the comment (e.g., "@What's happening at 3:45"). Additionally or alternatively, in some embodiments, process 100 can identify, among the retrieved comments, comments that include a link to a location within the media content item. For example, some of the comments that include links can be distinguished by using one or more of the following techniques: highlighting, underlining, bolding, italics, presenting in a different color, presenting in a different size, presenting in a different font, etc.

[0030] Additionally or alternatively, in some embodiments, process 100 may identify a subset of retrieved comments that include references to particular times within the media content item by determining, using any suitable technique, whether the content of the comments matches, correlates with, or is otherwise similar to, an audio portion from the particular time within the media content item.

[0031] For example, if the comment includes text, process 100 may compare the text from the comment with the text from the media content item to determine whether there is any correlation between the text of the comment and the text of the media content item. For example, a correlation may exist if the comment includes a quote or a paragraph of a quote from the media content item. In a more detailed example, process 100 may compare the text from the comment with a transcript or closed caption data for the media content item to determine (e.g., by comparison with the transcript or caption data) whether there is a correlation between the text of the media content item and dialogue or other words (whether spoken or written) in the media content item.

[0032] In another example, if the comment includes text, process 100 may input the text from the comment into a context model that determines whether the comment references or is otherwise related to media content presented in the media content item. For example, the context model may determine that a particular entity appears at a particular time in the media content item (e.g., Michael is running in this scene), and may determine that the text from the comment references the particular entity (e.g., the comment includes the text "The best part was when he showed Michael running").

[0033] In yet another example, if a comment includes text but does not include a reference to a specific time or a timestamp within the comment text, process 100 can determine whether the comment is relevant to a moment in a media content item. In a more detailed example, a media content item can be input into a machine learning model that identifies key moments within the media content item, and process 100 can determine whether the comment corresponds to one of the identified key moments within the media content item. In another more detailed example, a media content item and user engagement data associated with the media content item can be input into a machine learning model that identified moments within the media content item that correspond to high user engagement (e.g., user engagement greater than a certain threshold amount), and process 100 can determine whether the comment corresponds to one of the identified moments within the media content item. In some embodiments, process 100 can determine a degree of correlation between a portion of the comment and a portion of the media content item based on a comparison of the comment data and the media content item. For example, the degree of correlation may represent how similar (or dissimilar) a portion of the comment is to one or more portions of the media content item. For example, if a commenter quotes a portion of a media content item but misspells a word, the quote from the comment may be considered to be closely correlated with the quoted portion of the content item. Alternatively, if a comment has one word (or several words) in common with a portion of a media content item, the portion containing the common word is unlikely to be closely correlated, except in cases where the common word is a word that is specific to the content or is otherwise rarely used. Continuing with this example, a comment may be associated with a particular portion of media content based on the degree of correlation being above a certain threshold.

[0034] In some embodiments, a correlation score may be generated for a comment based on the degree of correlation between a portion of the comment and a portion of the media content item. For example, a comment that includes a portion that directly quotes or describes a particular portion of a media content item may be given a higher score with respect to that portion of the media content item than a comment that does not include a quote from that portion of the media content item. Continuing with this example, a comment with a correlation score greater than a certain threshold may be associated with a portion of the media content item.

[0035] In yet another example, if the comment includes text, process 100 can input the text from the comment into a text model that predicts whether the comment is relevant to a moment or time position within a media content item, and in response, the text model can provide a score for the comment as to whether the comment is relevant to the media content being presented at a particular time position within the media content item.

[0036] It should be noted that while comments are described as including text, comments can include any suitable type of content, such as video or image content, that can be analyzed to extract data using techniques similar to those used to analyze content items to extract data. For example, if a comment includes an image or video snippet, process 100 can compare one or more images in the comment's image or video snippet with image content from the media content item to determine whether there is any correlation between the images in the comment and the image content in the media content item. In another example, process 100 can be performed to determine whether a product or service promoted by the creator of a media content item corresponds to image content in the media content item. In yet another example, process 100 can be performed to determine whether sponsorship information corresponds to image content in the media content item.

[0037] It should also be noted that while a subset of retrieved comments is identified that includes references to particular moments within a media content item, this is for illustrative purposes only. For example, the subset of retrieved comments may be identified to include particular keywords, particular mentions of products or services, etc. In another example, process 100 may be used to determine whether a product or service being promoted by a creator of a media content item corresponds to a moment within the media content item, and an overlay interface may present supplemental content related to the product or service that is relevant to the moment within the media content item. In yet another example, process 100 may be used to determine whether a timed advertisement or timed sponsor information corresponds to a moment within the media content item, and an overlay interface may provide the timed advertisement or timed sponsor information.

[0038] In some embodiments, process 100 can filter out inappropriate and / or objectionable comments from a subset of the retrieved comments that include references to particular times within the media content item at 104. For example, in some embodiments, process 100 can filter out comments that include abusive language, profanity, and / or violate the terms of use, community policies, etc. of any of the services that facilitate the posting of comments about the media content item. In a more detailed example, process 100 can input the subset of the retrieved comments into an objectionable content model that compares the text or other content of the comments, the objectionable content model having been trained with a training set of comments that include examples of objectionable content (e.g., foul language, abusive language, etc.). In another example, in some embodiments, process 100 can filter out comments that other users have disliked (e.g., more than a certain threshold number of "dislikes"), reported (e.g., in response to selecting a report interface element in the comments window), or otherwise marked as containing inappropriate content.

[0039] In some embodiments, at 106, process 100 may allocate the remaining comments to a time window from a plurality of time windows associated with the media content item. For example, each of the remaining comments may be allocated to a corresponding time window of the media content item based on a reference to a particular time within the media content item.

[0040] In a more detailed example, a media content item may be divided into a series of time windows based on the duration of the media content item. For example, for a video having a 90-second duration, a series of 5-second time windows may be created for the media content item, with at least one comment assigned to each of the time windows. Continuing with this example, in response to determining that a comment includes a timestamp of 0:33 (33 seconds) for a video having a 90-second duration, the comment may be associated with a 5-second time window having a start time of 0:31 (31 seconds) and an end time of 0:35 (35 seconds).

[0041] It should be noted that the media content items may be divided into time windows using any other suitable data and / or metadata associated with the media content items.

[0042] For example, a media content item may be divided into time windows based on video chapters created by a creator of the media content item, advertisement breaks inserted in the media content item, closed caption tracks, etc. This may include, for example, video chapters and chapter start times designed by the creator of the media content item. This can also include video segments or any other suitable sections that are automatically generated, for example, by inputting the media content item into a machine learning model, which can output video segments and associated timestamps based on interesting scenes, increased user engagement, and / or any other suitable criteria.

[0043] In another example, a media content item may be divided into time windows having various lengths based on the density of comments and / or based on engagement data associated with each of the comments. For example, a time window may be generated for a media content item in response to determining that a high density of comments is associated with a time window, e.g., a certain number of comments are submitted between 1:00 and 1:15 and between 1:33 and 1:55 of the media content item, and engagement data associated with these comments indicates a certain number of likes, shares, endorsements, etc. Continuing with this example, process 100 may generate time windows for a media content item based on the density of comments or based on the amount of user engagement, with time windows being generated between time windows having a high comment density and / or high user engagement.

[0044] It should be noted that process 100 may use any suitable technique for sorting, matching, ordering, pairing, and / or otherwise determining time windows within a media content item that correspond to time references within a timed comment. It should also be noted that a timed comment may be allocated to more than one time window, for example, in instances where the timed comment includes more than one timestamp, or where the timed comment relates to more than one portion of a media content item.

[0045] In response to assigning each of the comments to a time window (e.g., assigning comment A1 to a time window spanning 1:00 and 1:05 of the video based on the content of comment A1), process 100 may continue at 108 and determine which comment or comments to present when the media content item is presented in a particular time window. For example, for each time window, process 100 may rank and select comments for presentation to a viewer of the media content item in a comments section during playback of the media content item—e.g., comment {C1} for time window from T1 to T2, comment {C2} for time window from T2 to T3, top comments {C3, C4, C5} for time window from T3 to T4, etc.

[0046] In some embodiments, process 100 may determine that multiple comments are associated with a particular time window. In response, process 100 may rank the comments according to any suitable ranking criteria. For example, comments associated with a particular time window may be ranked by their relevance to the media content presented in the particular time window. In a more detailed example, as described above, each comment may be associated with a degree of correlation or correlation score with a portion of the media content, and each comment may be ranked by its degree of correlation or correlation score (e.g., from highest correlation score to lowest correlation score). Continuing with this example, a certain number of top-ranked comments may be associated with a particular time window (e.g., the comment with the highest correlation score, the top three comments with the highest correlation scores, each comment with a correlation score greater than a certain threshold, etc.).

[0047] Process 100 may associate, in a video index, a video identifier (e.g., video ID 123456) with multiple time window identifiers (e.g., timestamp ID_1, timestamp ID_2, etc.) and, for each time window identifier, one or more comments (e.g., comment ID_1 and a comment feature, comment ID_2 and a comment feature, etc.). In response to submitting a query including the video identifier to the video index, the video index may return one or more comments that may be presented during presentation of media content in each time window of the video.

[0048] It should be noted that any suitable number of comments may be selected for each time window. For example, for a time window in which the density of comments is higher than a certain threshold (e.g., high comment density), additional comments may be selected in contrast to a time window with a low density of comments within the time window (e.g., one comment may be selected for time window A with low comment density and three comments may be selected for time window B with high comment density). In another example, for a time window in which engagement data indicates that viewers supported the comments within the time window (e.g., comments within the time window have a number of "likes" higher than a certain threshold), additional comments may be selected in contrast to a time window with engagement data that indicates that viewers did not respond to the comments within the time window (e.g., one comment may be selected for time window A with low engagement comments and three comments may be selected for time window B with high engagement comments).

[0049] It should also be noted that process 100 may perform this selection of comments for each time window associated with a media content item and may determine that any suitable number of timed comments (including zero) occur in any suitable number of time windows.

[0050] At 110, process 100 may cause the selected comment to be presented during the allotted time window during the presentation of the media content item. For example, the selected comment may be presented in a comment interface concurrently with the presentation of the media content item. In another example, the selected comment or comments may be presented along with the media content item in an overlay interface (e.g., a comment overlay interface presented below the video, a comment overlay interface positioned in the top of the video, a comment teaser interface including a single selected comment positioned adjacent to the video on a video viewing page, etc.).

[0051] In some embodiments, process 100 may cause selected comments to be displayed separately from comments not identified at 102 to include a timed reference. For example, in some embodiments, selected comments corresponding to a particular playback position within a media content item may be presented in a comment interface located below the video on a video viewing page. In some embodiments, process 100 may cause the user interface to display the timed comments in the determined time window in any suitable order, including in the order determined by the ranking at 108. In some embodiments, process 100 may display only a portion of the timed comments within a given time window (e.g., the top three most relevant comments within a highly commented segment).

[0052] At 112, process 100, in some embodiments, may update the display of timed comments as the presentation of the media content item progresses. For example, after a first 5-second duration including one timed comment, process 100 may update the display of timed comments with any suitable number of selected timed comments associated with a second 5-second duration of the video.

[0053] In some embodiments, process 100 can provide a commenter with an opportunity to write a timed comment while viewing timed comments related to a media content item. For example, a commenter can create a new comment with a timed reference, and in response, the new comment can be associated with a time window of the media content item and ranked relative to comments related to the particular time window. In another example, in response to the creation of a new comment with a timed reference, process 100 can indicate that such new comment is considered a timed comment, and process 100 can determine whether the new comment is to be selected as a top comment for association with a particular time window of the media content item. In yet another example, in response to the creation of a new comment with a timed reference, process 100 can indicate that such new comment is to be selected as a top comment for association with a particular time window of the media content item based on the ranking and / or selection performed at 108.

[0054] In some embodiments, process 100 may remove the timed comment displayed at 110 and / or may determine that the timed comment should not be presented in connection with a subsequent time window. In some embodiments, process 100 may take any suitable action, such as not displaying the timed comment during the time window, continuing to present timed comments from a previous time window or windows, presenting timed comments associated with an upcoming time window, and / or any other suitable action.

[0055] It should be noted that while process 100 may cause timed comments to be displayed at 110 and 112, process 100 may, in some embodiments, cause non-timed comments to be displayed at 110 and 112, as shown and discussed below in connection with Figures 3A and 3B. In some embodiments, any other suitable mechanism may determine when and how to display comments that are not included in the timed comments identified at 102 of process 100.

[0056] 3A , an illustrative example 300 of a display screen including comments with timed references synchronized with a media content item is shown, in accordance with some embodiments of the disclosed subject matter. As shown, display 300 may, in some embodiments, include a media content item 310 at a playback time 312, closed captions 314, a comment interface 315, and a timed comment interface. In some embodiments, the comments in comment interface 315 and the comments in timed comment interface 320 may be submitted by a user with a user account, as discussed in connection with FIG. 2. In some embodiments, any suitable device, such as user device 406 discussed in FIG. 4 below, may be used to present display screen 300.

[0057] In some embodiments, the media content item 310 may be any suitable song, video, clip, and / or other media content item. In some embodiments, the media content item 310 may be streamed from a remote content source or stored locally for playback. For example, in some embodiments, the media content item 310 may be a social media post that includes a video portion and an audio portion. As another example, in some embodiments, the media content item 310 may be a podcast that is downloaded and / or streamed from a podcast feed.

[0058] In some embodiments, a play time 312 may be included on the display screen 300. For example, as shown in Figure 3A, the current play time 312 is 1 minute 30 seconds of a total media content item duration of 18 minutes 5 seconds. In some embodiments, the play time 312 may be omitted from the display screen 300 and may be included in the metadata of the media content item 310. In some embodiments, the play time 312 may be formatted in any suitable time format.

[0059] In some embodiments, the closed captions 314 may be automatically created by a text generation program and / or may be included in the media content item 310 by the media content creator. In some embodiments, the closed captions 314 may be omitted from the display screen 300. In some embodiments, the closed captions 314 may be stored in metadata included with the media content item 310 and may be used to identify timed references in user-submitted comments, as discussed above in connection with FIG.

[0060] In some embodiments, display screen 300 may include comments in comment interface 315, which may be any comment that does not include a specified timed reference. In some embodiments, comments in comment interface 315 may be displayed in any suitable order, such as by relevance, newest, oldest, etc. In some embodiments, any suitable number of comments may be included in comment interface 315. In some embodiments, comment interface 315 may be omitted.

[0061] In some embodiments, timed comment interface 320 may include any suitable comments that include timed references. In some embodiments, comments displayed in timed comment interface 320 may be identified using any suitable technique, such as process 100 described above in connection with FIG. 1. As shown, timed comment interface 320 includes comment 202, comment 201, and comment 207. As discussed above in connection with FIG. 2, comment 202 includes a timestamp of one minute and thirty seconds (1:30), comment 201 includes a timestamp of one minute and twenty-nine seconds (1:29), and comment 207 includes a reference that matches or otherwise correlates with the content presented at timestamp 312. In some embodiments, comments 202, 201, and 207 may be included in timed comment interface 320 based on a time window determined during process 100, as discussed above in FIG. 1. For example, in some embodiments, timestamp 312 may be referenced in comments 202 and 201. In another example, in some embodiments, a comment 207 may be included in a timed comment 320 based on a determination (by process 100 and / or any other suitable technique) that the user-submitted text of the comment 207 substantially overlaps with the closed caption 314 and / or audio track at the timestamp 312.

[0062] In some embodiments, the timed comment interface 320 may include any suitable number of comments. For example, the timed comment interface 320 may present a single comment for a particular amount of time, e.g., the top comment for an entire 10-second time window of a media content item. Continuing with this example, if multiple comments are ranked and selected for a particular time window, the timed comment interface 320 may be configured to present a first comment (e.g., the top-ranked comment for that time window) for five seconds, and then present a second comment (e.g., the comment with the next highest ranking) for the next five seconds.

[0063] In some embodiments in which multiple comments are presented in the timed comments interface 320, the timed comments interface 320 may display the comments in any suitable order, such as by relevance, newest, oldest, etc.

[0064] In addition to or instead of presenting comment interface 315 and timed comment interface 320, display screen 300 may present timed comment interface 320 in response to receiving a selected preference from a user of display screen 300. For example, display screen 300 may include a timed comment interface element that, when selected, replaces a comment interface such as the comment interface shown in FIG. 2 in which comments are presented in the order in which they are received with timed comment interface 320 in which selected comments associated with a particular time window are presented concurrently with the media content item.

[0065] In addition to or instead of presenting the comment interface 315 and the timed comment interface 320, the display screen 300 may present the timed comment interface 320 as an overlay interface. For example, selected timed comments associated with a time window corresponding to a particular playback position within a media content item may be presented as an overlay below the media window or on top of the media content item (e.g., when the media content item is being played in full-screen mode).

[0066] In some embodiments, the timed comment interface 320 can be updated, changed, altered, and / or remain the same as the media content item 310 progresses. For example, in some embodiments, the timed comment interface 320 can remove comments 202, 201, and 207 from the display screen 300 once the media content item 310 progresses beyond the time referenced in the comments 202, 201, and 207. In another example, as shown in display screen 350 of FIG. 3B , the media content item 310 can display a comment 204 that includes a timestamp of 6 minutes and 35 seconds in the play time 352. In some embodiments, the comment 204 can be included in the timed comment interface 320 based on a time window determined during process 100, as discussed above in FIG. 1. For example, in some embodiments, the timestamp 352 or a timestamp within a particular window (e.g., 5 seconds into the timestamp 352) can be included in the comment 204.

[0067] In some embodiments, a play time 352 may be included on the display screen 350. For example, as shown, the play time 352 is 6 minutes 34 seconds out of a total media content item duration of 18 minutes 5 seconds. In some embodiments, the play time 352 may be omitted from the display screen 350 and / or may be included in the metadata of the media content item 310. In some embodiments, the play time 352 may be formatted in any suitable time format.

[0068] In some embodiments, the closed captions 354 may be automatically created by a text generation program and / or may be included in the media content item 310 by the media content creator. In some embodiments, the closed captions 354 may be omitted from the display 350. In some embodiments, the closed captions 354 may be stored in metadata included with the media content item 310 and / or may be used to identify timed references in user-submitted comments, as discussed above in connection with FIG. 1.

[0069] 4, an example hardware 400 for presenting comments with timed references synchronized with a media content item is shown, according to some embodiments. As shown, the hardware 400 may include a server 402, a communications network 404, and / or one or more user devices 406, such as user devices 408 and 410.

[0070] Server 402 may be any suitable server for storing information, data, programs, media content, and / or any other suitable content. In some embodiments, server 402 may perform any suitable function.

[0071] The communications network 404, in some embodiments, can be any suitable combination of one or more wired and / or wireless networks. For example, the communications network may include any one or more of the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communications network. The user device 406 may be connected to the communications network 404 by one or more communications links (e.g., communications link 412), which may be linked to the server 402 via one or more communications links (e.g., communications link 414). The communications link can be any communications link suitable for conveying data between the user device 406 and the server 402, such as a network link, a dial-up link, a wireless link, a wired link, any other suitable communications link, or any suitable combination of such links.

[0072] User device 406 may include any one or more user devices suitable for use with process 100. In some embodiments, user device 406 may include any suitable type of user device, such as a speaker (with or without a voice assistant), a mobile phone, a tablet computer, a wearable computer, a laptop computer, a desktop computer, a smart television, a media player, a game console, a vehicle information and / or entertainment system, and / or any other suitable type of user device.

[0073] Although server 402 is shown as one device, the functions performed by server 402 may, in some embodiments, be performed using any suitable number of devices. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 402.

[0074] To avoid overcomplicating the diagram, two user devices 408 and 410 are shown in FIG. 4, although in some embodiments, any suitable number of user devices (including only one user device) and / or any suitable type of user device may be used.

[0075] The server 402 and the user device 406 may, in some embodiments, be implemented using any suitable hardware. For example, in some embodiments, the devices 402 and 406 may be implemented using any suitable general-purpose or special-purpose computer and may include any suitable hardware. For example, as shown in the example hardware 500 of FIG. 5, such hardware may include a hardware processor 502, memory and / or storage 504, input device controllers 506, input devices 508, display / audio drivers 510, display and audio output circuitry 512, a communication interface 514, an antenna 516, and a bus 518.

[0076] Hardware processor 502, in some embodiments, may include any suitable hardware processor, such as a microprocessor, a microcontroller, a digital signal processor, special purpose logic, and / or any other suitable circuitry for controlling the functions of a general-purpose or special-purpose computer. In some embodiments, hardware processor 502 may be controlled by a computer program stored in memory and / or storage 504. For example, in some embodiments, the computer program may cause hardware processor 502 to perform the functions described herein.

[0077] Memory and / or storage 504, in some embodiments, can be any suitable memory and / or storage for storing programs, data, documents, and / or any other suitable information. For example, memory and / or storage 504 can include random access memory, read-only memory, flash memory, hard disk storage, optical media, and / or any other suitable memory.

[0078] Input device controller 506, in some embodiments, can be any suitable circuitry for controlling and receiving input from one or more input devices 508. For example, input device controller 506 can be circuitry for receiving input from a touchscreen, from a keyboard, from a mouse, from one or more buttons, from a voice recognition circuit, from one or more microphones, from a camera, from an optical sensor, from an accelerometer, from a temperature sensor, from a near field sensor, and / or from any other type of input device.

[0079] Display / audio driver 510, in some embodiments, can be any suitable circuitry for controlling and driving output to one or more display / audio output devices 512. For example, display / audio driver 510 can be circuitry for driving a touch screen, a flat panel display, a cathode ray tube display, a projector, a speaker or speakers, and / or any other suitable display and / or presentation device.

[0080] Communications interface 514 can be any suitable circuitry for interfacing with one or more communications networks, such as network 404 shown in Figure 4. For example, interface 514 may include network interface card circuitry, wireless communications circuitry, and / or any other suitable type of communications network circuitry.

[0081] Antenna 516, in some embodiments, may be any suitable antenna or antennas for wirelessly communicating with a communications network (e.g., communications network 404). In some embodiments, antenna 516 may be omitted.

[0082] The bus 518 may, in some embodiments, be any suitable mechanism for communicating between two or more components 502, 504, 506, 510, and 514.

[0083] According to some embodiments, any other suitable components may be included in hardware 500.

[0084] In some embodiments, any suitable computer-readable medium may be used to store instructions for performing the functions and / or processes described herein. For example, in some embodiments, computer-readable media can be transitory or non-transitory. For example, non-transitory computer-readable media may include media such as magnetic media in non-transitory form (hard disks, floppy disks, etc.), optical media in non-transitory form (compact disks, digital video disks, Blu-ray disks, etc.), semiconductor media in non-transitory form (flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.), any suitable media that does not ephemeral in transmission or lack any permanence, and / or any suitable tangible media. As another example, transitory computer-readable media may include wires, conductors, optical fibers, signals in circuits, wires, conductors, optical fibers, signals in circuits, any suitable media that ephemeral in transmission and lack any permanence, and / or any suitable intangible media.

[0085] While the invention has been described and illustrated in the above exemplary embodiments, it will be understood that the disclosure is made by way of example only, and that numerous changes in the details of the implementation of the invention may be made without departing from the spirit and scope of the invention. Features of the disclosed embodiments may be combined and rearranged in various ways. [Explanation of symbols]

[0086] 100 processes 200 Comment Interface 201~207 comments 300 display, display screen 310 Media Content Items 312 playing time 314 Closed Captions 315 Comment Interface 320 Timed Comment Interface 350 display screen 352 Playback time, timestamp 354 Closed Captions 400 Hardware 402 Server 404 Communication Network 406 User Device 408 User Device 410 User Device 412 Communication Links 414 Communication Links 500 Hardware 502 Hardware Processor 504 Memory and / or Storage 506 Input Device Controller 508 Input Devices 510 Display / Audio Driver 512 Display and audio output circuits, display / audio output devices 514 Communication Interface

Claims

1. A method for producing a media content item, comprising: obtaining a plurality of comments associated with the media content item; determining a first subset of comments from the plurality of comments, each comment from the first subset of comments including at least one of a plurality of timed references to the media content item; determining a second subset of comments from the plurality of comments, wherein each comment from the second subset of comments does not include at least one of the plurality of timed references to the media content item; determining a plurality of time windows, each time window from the plurality of time windows corresponding to a respective portion of the media content item; associating one or more comments from the first subset of comments with a respective time window from the plurality of time windows, each time window being associated with a respective time-defined reference contained within the respective comment; receiving a request to present comments synchronized with playback of the media content item; In response to receiving the request to present comments synchronized with the playback of the media content item, the one or more comments from the first subset of comments associated with the respective time windows in a first interface, the one or more comments being presented in the first interface simultaneously with the respective portions of the media content item corresponding to the respective time windows; one or more comments from the second subset of comments in a second interface, the second interface being presented simultaneously with the first interface and the respective portions of the media content item; and and presenting A method comprising:

2. 2. The method of claim 1, wherein associating the one or more comments from the first subset of comments with the respective time windows from the plurality of time windows further comprises determining a ranking for each comment from the first subset of comments that includes the respective timed reference associated with the respective time window.

3. 3. The method of claim 2, wherein the ranking is based on a number of upvotes associated with each comment from a first subset of comments including the respective timed references associated with the respective time windows.

4. Each time window from the plurality of time windows is determined based on a density of comments containing the respective time-defined reference; 2. The method of claim 1, further comprising: in response to determining that the density of comments between a respective start time and a respective end time is higher than a threshold, associating the one or more comments from the first subset of comments with the respective time window having the respective start time and the respective end time.

5. further comprising filtering the plurality of user-generated comments to remove user-generated comments that include objectionable content, thereby generating a filtered set of user-generated comments; The method of claim 1 , wherein the first subset of comments is determined from the filtered set of user-generated comments.

6. 10. The method of claim 1, further comprising: presenting a selectable option to toggle comments from being presented as received from one or more viewers of the media content item to being synchronized with the playback of the media content item.

7. A system for presenting user comments during media playback, comprising: Memory and 1. A hardware processor, comprising: obtaining a plurality of comments associated with a media content item; determining a first subset of comments from the plurality of comments, each comment from the first subset of comments including at least one of a plurality of timed references to the media content item; determining a second subset of comments from the plurality of comments, wherein each comment from the second subset of comments does not include at least one of the plurality of timed references to the media content item; determining a plurality of time windows, each time window from the plurality of time windows corresponding to a respective portion of the media content item; Associating one or more comments from the first subset of comments with a respective time window from the plurality of time windows, each time window associating a respective time contained within each comment with a determined reference; receiving a request to present comments synchronized with playback of the media content item; In response to receiving the request to present comments synchronized with the playback of the media content item, the one or more comments from the first subset of comments associated with the respective time windows in a first interface, the one or more comments being presented in the first interface simultaneously with the respective portions of the media content item corresponding to the respective time windows; one or more comments from the second subset of comments in a second interface, the second interface being presented simultaneously with the first interface and the respective portions of the media content item; and To present the a hardware processor configured to: Including, the system.

8. The system described in claim 7, wherein, to associate the one or more comments from the first subset of comments with the respective time windows from the plurality of time windows, the hardware processor is further configured to determine a ranking for each comment from the first subset of comments including the respective timed references associated with the respective time windows.

9. The system described in claim 8, wherein the ranking is based on the number of upvotes associated with each comment from a first subset of comments including the respective timed references associated with the respective time windows.

10. The method of claim 1, wherein each time window from the plurality of time windows is determined based on a density of comments containing the respective timed reference, and wherein the hardware processor:

8. The system of claim 7, further configured to, in response to determining that the density of comments between a respective start time and a respective end time is higher than a threshold, associate the one or more comments from the first subset of comments with the respective time window having the respective start time and the respective end time.

11. The hardware processor, further configured to generate a filtered set of user-generated comments by filtering the plurality of user-generated comments to remove user-generated comments that include objectionable content; The system of claim 7 , wherein the first subset of comments is determined from the filtered set of user-generated comments.

12. The hardware processor:

10. The system of claim 7, further configured to present a selectable option to toggle comments from being presented as received from one or more viewers of the media content item to being synchronized with the playback of the media content item.

13. A non-transitory computer-readable recording medium having computer-executable instructions recorded thereon, which, when executed by a processor, cause the processor to: obtaining a plurality of comments associated with a media content item; determining a first subset of comments from the plurality of comments, each comment from the first subset of comments including at least one of a plurality of timed references to the media content item; determining a second subset of comments from the plurality of comments, wherein each comment from the second subset of comments does not include at least one of the plurality of timed references to the media content item; determining a plurality of time windows, each time window from the plurality of time windows corresponding to a respective portion of the media content item; Associating one or more comments from the first subset of comments with a respective time window from the plurality of time windows, each time window associating a respective time contained within each comment with a determined reference; receiving a request to present comments synchronized with playback of the media content item; In response to receiving the request to present comments synchronized with the playback of the media content item, the one or more comments from the first subset of comments associated with the respective time windows in a first interface, the one or more comments being presented in the first interface simultaneously with the respective portions of the media content item corresponding to the respective time windows; one or more comments from the second subset of comments in a second interface, the second interface being presented simultaneously with the first interface and the respective portions of the media content item; and To present the A non-transitory computer-readable recording medium that causes the

14. A non-transitory computer-readable recording medium as described in claim 13, wherein the instructions cause the processor to determine a ranking for each comment from the first subset of comments that includes the respective timed reference associated with the respective time window, in order to associate the one or more comments from the first subset of comments with the respective time window from the plurality of time windows.

15. A non-transitory computer-readable recording medium as described in claim 14, wherein the ranking is based on the number of upvotes associated with each comment from a first subset of comments including the respective timed references associated with the respective time windows.

16. The method of claim 15, wherein each time window from the plurality of time windows is determined based on a density of comments containing the respective timed reference, and wherein the instructions cause the processor to:

14. The non-transitory computer-readable storage medium of claim 13, wherein in response to determining that the density of comments between a respective start time and a respective end time is higher than a threshold, the one or more comments from the first subset of comments are associated with the respective time window having the respective start time and the respective end time.

17. The instructions to the processor: filtering the plurality of user-generated comments to remove user-generated comments containing objectionable content, thereby generating a filtered set of user-generated comments; 14. The non-transitory computer-readable medium of claim 13, wherein the first subset of comments is determined from the filtered set of user-generated comments.

18. The instructions to the processor:

14. The non-transitory computer-readable storage medium of claim 13, wherein a selectable option is presented to toggle comments from being presented as received from one or more viewers of the media content item to being synchronized with the playback of the media content item.

Images