Context-Aware Control of a User Interface Displaying Video and Associated User Text

Abstract

The disclosed technology provides a computing device that displays video content within a comments section of a user interface. When a user invokes video display by selecting a link within the comments section of the user interface, the system can control the navigation position of the user interface to simultaneously display the video and the selected comments within the comments section. In one illustrative example, the system can display a user interface having a video display area and a comments section. The user interface can be positioned to display the comments section within a viewing area of the display device, and such a position can place the video display area outside the viewing area. In such a scenario, when the system receives user input indicating a selection of a comment displayed in the comments section, the system can generate a rendering of the video content for display within the comments section.

Description

Background Art

[0001] Commenting on videos is becoming popular and ubiquitous on many social, educational, and entertainment platforms. Many video-based commentators reference the video content to contextualize and specify their message. Commentators can reference visual entities or specific sound clips in a variety of ways. For example, users can reference someone's voice or quote at a specific time or provide a timestamp. In some systems, users can include a link to their comment, allowing users to view the video starting at a specific point in time.

[0002] Although existing systems provide a platform for users to provide comments, the user interfaces currently used are inherently simplistic and do not provide tools for optimizing the user experience, both from the perspective of the commenter and from the perspective of the viewer. For example, in some existing systems, when a viewer selects a video link associated with a comment, the current system often causes the user interface to scroll away from the comments section in order to display the selected video. This feature can lead to many inefficiencies and complications. In particular, when interacting with existing systems, users are unable to maintain a view of the comments when they select a comment to view a video. This does not allow users to continuously view comments they are interested in while watching related videos. This leads to many inefficiencies by requiring users to scroll back and forth between the comments section and the video section of the user interface, which can be extremely difficult when there are hundreds or even thousands of comments. This type of navigation is very inefficient in terms of user productivity and computing resources.

[0003] It is in response to these and other technical challenges that the present disclosure is proposed. Summary of the Invention

[0004] The technology disclosed herein provides improvements to existing systems by enabling a computing device to display video content within a comments section of a user interface. When a user invokes display of a video by selecting a link to the video within the comments section of the user interface, the system can control the navigation position of the user interface to simultaneously display the selected comment within the comments section and also display the video within the comments section.

[0005] In one illustrative example, a system may display a user interface having a video display area and a comments section. In some scenarios, the user interface may be navigationally positioned to display the comments section within the viewing area of a display device, and such positioning may place the video display area outside the viewing area of the display screen. For example, the scrolling position of a webpage may cause the video to be off-screen. In such a scenario, when the system receives user input indicating a selection of a comment displayed within the comments section, the system may generate a rendering of the video content for display within the comments section.

[0006] The technology described herein can result in more efficient use of computing systems. In particular, by controlling aspects of the user interface to ensure that videos and comments are displayed simultaneously, the system can improve multiple efficiencies regarding user productivity and promote more efficient use of computing resources. The system presented herein alleviates the need for users to manually navigate through the entire user interface to view specific comments during video playback. Eliminating or alleviating the manual navigation process results in more efficient use of computing resources, such as memory usage, network usage, and processing resources, because this can reduce the time a person needs to spend on a computer to browse large pages. Additionally, the system can enhance user engagement by alleviating the need for cumbersome manual user interface navigation processes for pages containing a large number of comments.

[0007] By reading the following detailed description and viewing the associated drawings, other features and technical advantages in addition to those explicitly described above will also be apparent. This summary is provided to introduce, in a simplified form, a selection of concepts further described below in the detailed description. This summary is not intended to identify the key or important features of the claimed subject matter, nor is it intended to be used to assist in determining the scope of the claimed subject matter. For example, the term "technology" may refer to (multiple) systems, (multiple) methods, computer-readable instructions, (multiple) modules, algorithms, hardware logic, and / or (multiple) operations as allowed by the context described above and the entire document. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The same reference numbers in different drawings indicate similar or identical items. References to individual items of a plurality of items may use reference numbers with letters of the alphabetical sequence to refer to each individual item. General references to items may use specific reference numbers without the alphabetical sequence.

[0009] Figure 1 An example process involving a computing device that can control navigation of a user interface displaying a video display area and a comments section is shown.

[0010] Figure 2 An example process is shown involving a computing device that can control navigation of a user interface to display selected comments concurrently with video content associated with the comments.

[0011] Figure 3 An example process is shown involving a computing device that can control navigation of a user interface to display selected comments and related comments concurrently with video content associated with the comments and related comments.

[0012] Figure 4 An example process involving a computing device that can generate a reconfigured user interface to simultaneously display comments and video content associated with the comments is shown.

[0013] Figure 5 An example process involving a computing device that can transition from a reconfigured user interface back to an original user interface is shown.

[0014] Figure 6A An example process is shown involving a computing device that can generate a customized user interface to simultaneously display comments and video content associated with the comments.

[0015] Figure 6B An example of a customized user interface for simultaneously displaying comments and video content associated with the comments is shown.

[0016] Figure 7 An example process is shown involving a computing device that can generate different types of user interfaces depending on the type of user input.

[0017] Figure 8 An example process involving a computing device that can generate a rendering of an audio user interface element concurrently with associated commentary is shown.

[0018] Figure 9 An example process involving a computing device that can control navigation of a user interface to display still images of video content concurrently with related commentary is shown.

[0019] Figure 10 is a flow chart illustrating aspects of a routine for computationally efficiently generating spoken content for a video.

[0020] Figure 11 is a computing system diagram showing aspects of an illustrative operating environment for the techniques disclosed herein.

[0021] Figure 12 is a computing architecture diagram illustrating aspects of the configuration and operation of a computing device that may implement aspects of the techniques disclosed herein. DETAILED DESCRIPTION

[0022] Figure 1 An example process involving computing device 100 that can control and enhance the navigational location of a user interface displaying a video display area and a comments section is shown. Figure 1The example shown in Figure 1 shows two stages of a process for controlling the navigation position of the user interface 130 to render video content in the video display area 140 and the comment section 150. The first stage of the process is shown on the left side of the figure, and the second stage of the process is shown on the right side of the figure. The figure is arranged to illustrate how portions of the user interface 130 can be displayed to a user. Specifically, for illustrative purposes, portions of the user interface 130 that are within a viewing area 160 of a display device 170 (e.g., a screen) can be viewed by a user of the device. However, for illustrative purposes, portions of the user interface 130 that are not drawn within the viewing area 160 are "off-screen" and cannot be viewed by the user.

[0023] As shown in the first stage of the process, computing device 100 may display user interface 130 including video display area 140 and comments section 150. As shown, the initial navigation position of user interface 130 displays comments section 150 within viewing area 160 of display device 170, and rendered video content 116 within video display area 140. The position of user interface 130 positions video display area 140 outside of viewing area 160. Video display area 140 and comments section 150 are distinct from one another, and therefore, rendering 116 of video content is "off-screen" and cannot be viewed by the user of computing device 100. As can be appreciated, the position of user interface 130 can be manipulated by user interaction of scrolling the user interface up or down to view portions of user interface 130 within viewing area 160 of display device 170. Comment section 150 is also referred to herein as "text field 150" or "text portion 150." Comment section 150 may include any portion of the user interface that contains text. For example, the comments section 150 may be part of a word processing document, a OneNote file, a spreadsheet, a blog, or any other form of media or data that may cause a computer to render text in conjunction with the rendering of a video.

[0024] Also like Figure 1As shown in FIG, in a first stage of the process, computing device 100 may receive user input 161 indicating a selection of a comment 162 displayed within comment section 160. User input 161 may be based on user interaction with a touchscreen or any other mechanism suitable for selecting comment 162. In some configurations, user input 161 may be a voice command received by a microphone of computing device 100. In some configurations, user input 161 may be received by a camera or imaging sensor of the computing device. Such embodiments may allow a user to provide a gesture to select comment 162. For example, a user may point to a specific comment within the user interface. It will also be appreciated that selection of comment 162 may include selection of at least a portion of the comment. Thus, if a user selects a word or any other symbol or character associated with a displayed video, such user input may cause computing device 102 to perform one or more actions (e.g., display of a video, rendering of an audio output, or display of another type of media (e.g., a still image or other graphical element)) while simultaneously controlling the navigation position of the user interface to continuously display the selected word, symbol, or character.

[0025] As shown in the second stage of the process, in response to receiving user input, computing device 100 may generate a second rendering 117 of the content for display within comments section 150, wherein user interface 130 is configured to maintain a position of user interface 130 to position comments section 150 within viewing area 160 of display device 170 while displaying second rendering 117 within viewing area 160 of display device 170.

[0026] The embodiments provided herein are for illustrative purposes and should not be construed as limiting. It will be appreciated that any type of computing device may be utilized to implement the technology disclosed herein. For example, although mobile devices are utilized in some of the examples depicted herein, any computing device that communicates with a display device (e.g., a monitor) having a display area may be used to implement the disclosed technology. For illustrative purposes, the display area of a display device is considered to be the surface of the display device that can generate light for displaying a rendered image.

[0027] Figure 2 An example process is shown involving computing device 100 that can control the navigational position of user interface 130 to display selected comment 162 concurrently with the rendering of video content 117 associated with comment 162 in viewing area 160. In this example, in a second stage of the process, computing device 100 can also automatically scroll or otherwise position user interface 130 to enable computing device 100 to concurrently display rendered video content 117 such that video content 117 does not obscure or block display of selected comment 162.

[0028] exist Figure 1 In the example shown in , the user interface can be positioned so that the second rendering 117 of the video content within the comments section can block at least a portion of the selected comment 162. Figure 2 In the example shown in , the system can control the navigation position of the user interface 130 in response to user input to allow the comment to be displayed in its entirety simultaneously with the second rendering 117 in the viewing area 160. The control navigation can also select a position for the second rendering 117 so that it reduces overlap with any comments in the comments section. The system can analyze the coordinates of the comment and the coordinates of the rendered video 117 to minimize the amount of overlap in the viewing area 160. The size of the video rendering can also be adjusted to minimize the amount of overlap between the video rendering and one or more comments. The video rendering can be minimized to a predetermined size, which can depend on one or more user settings or other factors, such as the resolution of the video data.

[0029] In some configurations, the computing device 100 can scroll the user interface 130 in any direction (e.g., up, down, left, or right) to enable the computing device to simultaneously display the rendered video content 117 so that the video content 117 does not obscure or block the display of the selected comment 162. Any suitable technique for detecting the position of a user interface element (e.g., the rendering of the video content 117 and / or the comment 162) can be used in important aspects of the present disclosure. For example, the coordinates of the comment 162 can be analyzed by the computing device 100. The computing device can then determine the coordinates of the rendering of the video content 117 based on the coordinates of the comment 162. The computing device 100 can then position the rendering of the video content 117 so that the coordinates of the video content 117 have no more than a threshold level of overlap relative to the coordinates of the comment 162. The threshold level of overlap can allow for no overlap that reaches any predetermined level of overlap. Alternatively, the computing device 100 can determine the coordinates of the rendering of the video content 117 so that the video content 117 is rendered within the viewing area 160 of the display device 170.

[0030] Figure 3 An example process is shown involving computing device 100 that can control navigation of user interface 130 to display selected comment 162 and related comments 163 concurrently with video content associated with the comment 162 and related comments 163. This embodiment enables computing device 100 to analyze various comments to allow for display of comments that have a threshold level of relevance with respect to rendered video content 117. In one illustrative example, the related comments can include any replies to selected comment 162.

[0031] like Figure 3, computing device 100 may apply controlled navigation to display comment 162 and related comments 163. This may be accomplished by scrolling user interface 130 to a position that allows for display of comment 162 and related comments 163. Simultaneously, the rendering of video content 117 is positioned such that the rendering of video content 117 has a threshold level of overlap or non-overlap relative to comment 162 and related comments 163.

[0032] Figure 4 An example process involving computing device 100 is shown that may generate a reconfigured user interface 130″ to simultaneously display a selected comment 162 and a rendering of video content associated with the comment 162. In this example, computing device 100 may modify the size of comment portion 150 to allow video display area 140 to render video content 116 simultaneously with comment 162.

[0033] The reconfigured user interface 130" can be implemented in a variety of ways. In this particular example, the top border of the comments section 150 is modified so that the comments above the selected comment 162 are hidden and the selected comment 162 is displayed at the top of the comments section 150. Figure 4 As shown by the arrow in , the top border of the comment section 150 is adjusted to enable the display area 140. Although Figure 4 The example shows one example in which the size (e.g., height (H)) of the comments section 150 is modified, but it will be appreciated that the reconfigured user interface 130" can involve modification of any shape or size of graphical elements to allow simultaneous viewing of the video display area 140 and the selected comments 162.

[0034] Figure 5 An example process involving a computing device that can transition from a reconfigured user interface back to an original user interface is shown. Figure 4 For example, Figure 5 A two-step process is shown that begins with a reconfigured user interface 130". In this example, the reconfigured user interface 130" includes a selectable graphical element 164 that indicates that the comments section 150 is collapsed. The reconfigured user interface 130" is also configured to receive user input 165 at the selectable graphical element 164. In response to the user input 165, the computing device 100 can revert back to the original user interface 130, which displays the selected comment 162 and the viewing area 160 and places the video display area 140 off the screen.

[0035] While this example utilizes touch-based user input, it will be appreciated that this embodiment may involve voice commands or any other type of input for reverting the display back to the original user interface 130. It will also be appreciated that this embodiment may involve other types of graphical elements besides the selectable graphical elements 164. For example, the top border of the comments section 150 may be bolded, shaped, colored, or otherwise modified to indicate the collapsed state of the comments section 150.

[0036] In addition to providing the reconfigured user interface 130", the computing device 100 may generate a customized user interface 130'. In such an embodiment, the computing device may generate an entirely new user interface configuration that renders content within the display area 140 concurrently with at least one comment (e.g., the selected comment 162).

[0037] Figure 6A An example process involving a computing device is shown that can generate a customized user interface 130' to simultaneously display a selected comment 162 and the video content associated with the comment 162. In this illustrative example, the selected comment 162 is located below the video display area 140 within the viewing area 160. The customized user interface 130' is also shown arranged so that the display of the rendered video 116 does not obscure or otherwise block the display of the selected comment 162 or any other comments. This example is provided for illustrative purposes and should not be construed as limiting. The customized user interface 130' can have the comment 162 in any position, orientation, or size relative to the rendered video 116.

[0038] Figure 6B An example of a customized user interface 130' configured to simultaneously display comments and video content associated with the comments is shown. In general, the display area 140 of the video content and the selected comments 162 can be arranged in any manner. For example, as shown in Example 1, the selected comment 162 and the related comments 163 can be oriented proximate to the display area 140 that includes the rendering of the video content 116.

[0039] In some configurations, the display area 140 of the video content 116 can have a threshold level of overlap about the selected comment 162. For illustrative purposes, the threshold level of overlap can include zero overlap up to any predetermined level of overlap, which can be defined in a user preference file or any other contextual data based on user activity. In other embodiments, the threshold level of overlap can be based on the amount of covered text of the comment 162. For example, as long as the text can be interpreted, the threshold level of overlap can allow some overlap. Thus, as shown in Example 2, the amount of overlap can allow the video to cover or otherwise obscure some words of the comment 162 while still enabling the user to understand the essence of the comment. Thus, when certain keywords (e.g., "THE" and "A" and other connecting words or transition words) are covered by the video rendering, the user interface may have a threshold level of overlap.

[0040] The system can also adjust the size and / or position of the display elements to eliminate or minimize any overlap between the display area 140 and the comments 162. As shown in Example 3, the height (H) and width (W) of the display area 140 or at least one comment 162 can be adjusted to eliminate any overlap between the two graphical elements. In one illustrative example, the threshold level of overlap can be based on the content of the comment. If it is determined that the comment includes a threshold percentage of predetermined keywords, the system can determine that the threshold level of overlap is zero, and the system can arrange the graphical elements accordingly, as shown in Example 3.

[0041] Similar to Figure 4 and Figure 5 , the customized user interface can also be configured to revert back to the original user interface. In such an embodiment, the computing device 100 can revert the display of the customized user interface 130' back to the original user interface 130 in response to user input. In some configurations, the computing device 100 can also revert the display of the customized user interface 130' back to the original user interface 130 based on other events (e.g., a timer or other criteria).

[0042] In some embodiments, computing device 100 may take different actions depending on the type of input provided by the user. Figure 7 FIG2 shows an example process involving a computing device 100 that can generate different types of user interfaces depending on the type of user input. In this example, the computing device 100 can display a second rendering 117 of the video content within the comment section based on a first type of user input. The first type of user input can be any type of input, including passive input (e.g., hovering of a cursor 161) or any other type of suitable input. As shown, the user interface 130 can transition from the example on the left side of the figure to the example on the right side of the figure in response to the first type of user input. Figure 7 User interface 130 in the upper right corner. Figure 7 The user interface 130 in the upper right corner shows a rendering of the video content 117 within a comments section 150 .

[0043] Figure 7 1 also shows that the computing device can take other actions based on the second type of user input. In this example, based on the second type of user input (which can include more active types of input, such as hover and click or hover and double click), the computing device 100 can generate a customized user interface 130'. As shown, the user interface 130 can change from the example on the left side of the figure to the example on the left side of the figure in response to the second type of user input. Figure 7 Customized user interface 130' in the lower right corner. Figure 7 The user interface 1130 ′ in the lower right corner shows a rendering of the video content 117 along with the selected comment 162 and comment 163 .

[0044] These examples have been provided for illustrative purposes and should not be construed as limiting. It will be appreciated that any type of user interface may be generated in response to various types of user input. For example, in Figure 7 In the example of , in response to the second type of user input, computing device 100 may generate a reconfigured user interface 130 ′, or any other user interface configuration that simultaneously displays the selected comment 162 and the video associated with the selected comment 162 .

[0045] In some configurations, the computing device 100 can render video content in a variety of different formats and media. For example, a link within the comments section 150 can be configured to play an audio clip associated with the video. In such an embodiment, Figure 8 , as opposed to rendering the video in response to a user selection of comment 162, computing device 100 may display audio interface element 118 simultaneously with the associated comment. Computing device 100 may also render audio output 802 from speaker 801 in response to the user selection of comment 162. In some configurations, computing device 100 may render audio output 802 from speaker 801 in response to a user selection of at least a portion of comment 162 without displaying audio interface element 118, while also controlling the position of user interface 130 to display comment portion 150.

[0046] In another example, the link within the comments section 150 may be configured to display a still image captured from the video content. In such an embodiment, Figure 9, instead of rendering a video in response to a user selection of a comment 162, the computing device 100 may display a still image from a particular point in time of the video content. The display of the image may be displayed simultaneously with the associated comment or otherwise displayed within the comments section 150 of the user interface 130.

[0047] Figure 10 is a diagram illustrating aspects of a routine 1000 for computationally efficiently displaying a video in a comment section of a user interface. It will be understood by those skilled in the art that the operations of the methods disclosed herein are not necessarily presented in any particular order, and that it is possible and contemplated to perform some or all of the operations in an alternative order. For ease of description and illustration, the operations have been presented in the order presented. Operations may be added, omitted, performed together, and / or performed simultaneously without departing from the scope of the appended claims.

[0048] It should also be understood that the illustrated method can be terminated at any time and need not be performed in its entirety. As defined herein, some or all of the operations of the method and / or substantially equivalent operations may be performed by executing computer-readable instructions contained on a computer storage medium. The term "computer-readable instructions" and variations thereof as used in the specification and claims are used broadly herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. The computer-readable instructions may be implemented on a variety of system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, and the like.

[0049] It should be recognized, therefore, that the logical operations described herein are implemented as: (1) a sequence of computer-implemented actions or program modules running on a computing system such as those described herein; and / or (2) interconnected machine logic circuits or circuit modules within a computing system. The manner of implementation is a matter of choice depending on the performance and other requirements of the computing system. Thus, the logical operations may be implemented in software, firmware, dedicated digital logic units, and any combination thereof.

[0050] Additionally, the example presentation UI described above may be implemented in association with Figure 10 For example, the various devices and / or modules described herein may generate, send, receive, and / or display data associated with the content of a video (e.g., real-time content, broadcasted events, recorded content, etc.) and / or present a UI that includes a rendering of one or more participants of a remote computing device, avatar, channel, chat session, video stream, image, virtual object, and / or application associated with the video.

[0051] The routine 1000 begins at operation 1002 , where the system may cause display of a user interface 130 having a video display area and a comments section. Figure 1 An example of user interface 130 is shown in FIG. A video display area 140 can be configured to render video 116. A comment section can include a plurality of comments. Each comment can include a link or other associated metadata configured to display a video rendering or other media display in response to a user selection. User interface 130 can be displayed on a client device such as a tablet computer, mobile phone, desktop computer, etc.

[0052] Next, at operation 1004, the system (e.g., computing device 100) may receive user input selecting a comment within the comments section. Because individual comments can be associated with certain types of media using links or other metadata, user interface 130 may generate a rendering of video 117 in response to the user selection of comment 162. The user selection may be made by any device (e.g., a mouse, a touch surface device, a voice input device, etc.).

[0053] At operation 1006, the system may determine a navigation location for the user interface 130 or generate a customized interface or a reconfigured user interface in response to the user input. As described above, the navigation location may be based on video rendering (e.g., Figure 1 The navigation location can be selected based on the size and position of the second rendering 117 (shown in FIG. 1 ). The location of the user interface 130 can be selected to allow the second rendering 117 to be displayed within the comments section of the user interface. In some configurations, the location of the user interface (e.g., the scroll position) can be selected to allow for simultaneous, non-overlapping display of the second rendering 117 of the video and the selected comment and / or related comments.

[0054] In other configurations, a rendering of a video associated with a selected comment can be displayed along with the selected comment in a reconfigured user interface that changes the size and shape of the comment portion to enable simultaneous display of the video (e.g., the first rendering 116 and the selected comment). In another embodiment, a customized user interface can be generated to display the selected comment simultaneously with the associated rendering of the video. Such an embodiment can arrange the layout of the selected comment and the associated video to fit the screen size of the computing device, wherein the size and shape of the comment and the size and shape of the video rendering can be adjusted to allow the user to view the comment in addition to viewing a specific portion of the rendered video.

[0055] Next, at operation 1008, the system may render the selected comments concurrently with the associated rendering of the video. Several examples are disclosed herein. Figure 1 As shown in FIG, the second rendering 117 can be displayed within the comment section of the user interface. Figure 3 ), the scroll position can be selected to enable the rendered video to be displayed simultaneously with the selected comment and other related comments.

[0056] like Figure 4 , other embodiments enable the comment section to be modified so that the user interface can display the video rendering simultaneously with at least one comment (including the selected comment 162). These examples are provided for illustrative purposes and should not be construed as limiting. It will be appreciated that any location of the user interface or any shape of the rendered element within the user interface can be modified to enable the system to display the video simultaneously with the selected comment and / or other related comments.

[0057] In some aspects of operation 1008, the system may determine a specific scroll position for the user interface or reconfigure the user interface in response to different types of user input. Figure 7 As shown in FIG, in response to a first type of input (e.g., hovering), the system can automatically display the video within the comments section of the user interface. In response to a second type of input (e.g., clicking or double-clicking to select a particular comment), the system can generate a customized user interface or a reconfigured user interface to display a rendering of the video and one or more comments.

[0058] Next, at operation 1010, the system can return the user interface 130 to the original layout in response to one or more actions. For example, after the video playback has completed at operation 1008, if the system is displaying a customized user interface or a reconfigured user interface, the system can restore these displays to the original interface when the video playback ends. For example, the original interface is Figure 1 User interface 130 is shown on the left.

[0059] It should be appreciated that the subject matter described above can be implemented as a computer-controlled device, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. The operations of the example methods are shown in separate blocks and are summarized with reference to those blocks. The methods are shown as a logical flow of blocks, each of which can represent one or more operations that can be implemented in hardware, software, or a combination thereof. In the context of software, an operation represents a computer-executable instruction stored on one or more computer-readable media that, when executed by one or more processors, enables the one or more processors to perform the recited operation.

[0060] Generally, computer-executable instructions include routines, programs, objects, modules, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as limiting, and any number of the described operations may be performed in any order, combined in any order, subdivided into multiple sub-operations, and / or performed in parallel to implement the described processes. The described processes may be performed by resources associated with one or more devices (e.g., one or more internal or external CPUs or GPUs) and / or one or more hardware logic units (e.g., a field programmable gate array ("FPGA"), a digital signal processor ("DSP"), or other type of accelerator).

[0061] All of the methods and processes described above can be embodied in software code modules executed by one or more general-purpose computers or processors and fully automated via the software code modules. The code modules can be stored in any type of computer-readable storage medium or other computer storage device, such as those described below. Some or all of the methods can alternatively be embodied in specialized computer hardware, such as those described below.

[0062] Any conventional description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code, which includes one or more executable instructions for implementing the specific logical functions or elements in the routine. Alternative implementations are included within the scope of the examples described herein, wherein elements or functions may be deleted from those shown or discussed or performed in a different order, including substantially simultaneously or in reverse order, depending on the functionality involved as understood by those skilled in the art.

[0063] Figure 11 1004 .

[0064] As shown, a communication session 1104 (where N is a number having a value of two or greater) can be implemented between a plurality of client computing devices 1106(1) through 1106(N) associated with or as part of the system 1102. The client computing devices 1106(1) through 1106(N) enable users (also referred to as individuals) to participate in the communication session 1104. Although this embodiment shows a communication session 1104, it will be appreciated that a communication session 1104 is not required for every embodiment disclosed herein. It will be appreciated that a video stream can be uploaded by each client 1106 and comments can be provided by each client 1106. It will be appreciated that any client 1106 can also receive video data and audio data from the server module 1130.

[0065] In this example, a communication session 1104 is hosted by a system 1102 on one or more networks 1108. That is, the system 1102 can provide services that enable users of client computing devices 1106(1) to 1106(N) to participate in the communication session 1104 (e.g., via real-time viewing and / or recorded viewing). Thus, "participants" of the communication session 1104 can include users and / or client computing devices (e.g., multiple users can participate in a communication session in a room via the use of a single client computing device), each of which can communicate with other participants. Alternatively, the communication session 1104 can be hosted by one of the client computing devices 1106(1) to 1106(N) using peer-to-peer technology. The system 1102 can also host chat conversations and other team collaboration functionality (e.g., as part of an application suite).

[0066] In some implementations, such chat conversations and other team collaboration functions are considered to be external communication sessions distinct from the communication session 1104. The computerized agent used to collect participant data in the communication session 1104 may be able to link to such an external communication session. Thus, the computerized agent may receive information such as date, time, session-specific information, etc. that enables connectivity to such an external communication session. In one example, a chat conversation may be conducted based on the communication session 1104. Additionally, the system 1102 may host the communication session 1104, which includes at least a plurality of participants that are co-located at a meeting location (e.g., a conference room or auditorium) or located at different locations. In the examples described herein, some embodiments may not utilize the communication session 1104. In some embodiments, a video may be uploaded from at least one of the client computing devices (e.g., 1106(1), 1106(2)) to the server module 1130. When the video content is uploaded to the server module 1130, any client computing device may access the uploaded video content and display the video content within a user interface such as those described above.

[0067] In the examples described herein, client computing devices 1106(1) to 1106(N) participating in a communication session 1104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data may include a collection of various instances or streams of real-time content and / or recorded content. The collection of various instances or streams of real-time content and / or recorded content may be provided by one or more cameras (e.g., video cameras). For example, a single stream of real-time content or recorded content may include media data associated with a video feed provided by a video camera (e.g., audio data and visual data capturing the appearance and voice of users participating in the communication session). In some implementations, the video feed may include such audio data and visual data, one or more still images, and / or one or more avatars. The one or more still images may also include one or more avatars.

[0068] Another example of a single stream of real-time content or recorded content may include media data including avatars of users participating in a communication session and audio data capturing the users' voices. Yet another example of a single stream of real-time content or recorded content may include media data including a file displayed on a display screen and audio data capturing the users' voices. Thus, the various streams of real-time content or recorded content within the communication data enable facilitating remote conferencing between a group of people and sharing content within a group of people. In some implementations, the various streams of real-time content or recorded content within the communication data may originate from multiple co-located cameras located in a space (e.g., a room) for recording or streaming a presentation that includes one or more personal presentations and one or more personal consumption of the presented content.

[0069] Participants or attendees can view the content of the communication session 1104 in real time as the activity occurs, or alternatively, at a later time after the activity occurs via a recording. In the examples described herein, client computing devices 1106(1) to 1106(N) participating in the communication session 1104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data may include a collection of various instances or streams of real-time content and / or recorded content. For example, a single stream of content may include media data associated with a video feed (e.g., audio data and visual data capturing the appearance and voice of users participating in the communication session). Another example of a single stream of content may include media data including avatars of users participating in the conference session and audio data capturing the voice of the users. Yet another example of a single stream of content may include media data including content items displayed on a display screen and / or audio data capturing the voice of the users. Thus, the various streams of content within the communication data enable facilitating a conference or broadcast presentation between a group of people dispersed across remote locations. Each stream may also include text, audio, and video data, such as data transmitted within a channel, chat board, or private messaging service.

[0070] Participants or attendees of a communication session are people who are within range of a camera or other image and / or audio capture device so that the person's movements and / or sounds produced when the person is viewing and / or listening to content shared via the communication session can be captured (e.g., recorded). For example, a participant can be sitting in a crowd, viewing shared content being performed in real time at a broadcast location where a stage presentation is taking place. Alternatively, a participant can be sitting in an office conference room, viewing shared content of a communication session with other colleagues via a display screen. Even further, a participant can be sitting or standing in front of a personal device (e.g., a tablet computer, smartphone, computer, etc.) and viewing the shared content of a communication session alone in their office or at home.

[0071] System 1102 includes device(s) 1110. Device(s) 1110 and / or other components of system 1102 may include distributed computing resources that communicate with each other and / or with client computing devices 1106(1) through 1106(N) via one or more networks 1108. In some examples, system 1102 may be a standalone system that is responsible for managing aspects of one or more communication sessions (e.g., communication session 1104). As an example, system 1102 may be managed by an entity such as YOUTUBE, FACEBOOK, SLACK, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0072] Network(s) 1108 may include, for example, a public network (e.g., the Internet), a private network (e.g., an institutional and / or personal intranet), or some combination of private and public networks. Network(s) 1108 may also include any type of wired and / or wireless network, including, but not limited to, a local area network ("LAN"), a wide area network ("WAN"), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. Network(s) 1108 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as the Internet Protocol ("IP"), the Transmission Control Protocol ("TCP"), the User Datagram Protocol ("UDP"), or other types of protocols. Furthermore, network(s) 1108 may also include a plurality of devices that facilitate network communications and / or form the hardware foundation of the network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone equipment, and the like.

[0073] In some examples, network(s) 1108 may also include devices that enable connection to wireless networks, such as wireless access points (“WAPs”). Examples support connectivity via WAPs that send and receive data over various electromagnetic frequencies (e.g., radio frequencies), including WAPs that support the Institute of Electrical and Electronics Engineers (“IEEE”) 802.21 standards (e.g., 802.11g, 802.11n, 802.11ac, etc.) and other standards.

[0074] In various examples, device(s) 1110 may include one or more computing devices operating in a cluster or other grouping configuration to share resources, balance loads, improve performance, provide failover support or redundancy, or for other purposes. For example, device(s) 1110 may belong to various categories of devices, such as traditional server-type devices, desktop-type devices, and / or mobile-type devices. Thus, although device(s) 1110 are shown as a single type of device or server-type device, device(s) 110 may include a wide variety of device types and is not limited to a particular type of device. Device(s) 1110 may represent, but are not limited to, a server computer, a desktop computer, a web server computer, a personal computer, a mobile computer, a laptop computer, a tablet computer, or any other type of computing device.

[0075] The client computing device (e.g., one of client computing devices 1106(1) through 1106(N)) can belong to various classes of devices, which can be the same as or different from device(s) 1110, such as, for example, a traditional server-type device, a desktop-type device, a mobile-type device, a dedicated-type device, an embedded-type device, and / or a wearable-type device. Thus, the client computing device can include, but is not limited to, a desktop computer, a game console and / or gaming device, a tablet computer, a personal data assistant ("PDA"), a mobile phone / tablet hybrid device, a laptop computer, a telecommunications device, a computer navigation-type client computing device (e.g., a satellite-based navigation system including a Global Positioning System ("GPS") device), a wearable device, a virtual reality ("VR") device, an augmented reality ("AR") device, an implantable computing device, an automotive computer, a web-enabled television, a thin client, a terminal, an Internet of Things ("IoT") device, a workstation, a media player, a personal video recorder ("PVR"), a set-top box, a camera, an integrated component (e.g., a peripheral device) for inclusion in a computing device, a home appliance, or any other kind of computing device. Furthermore, the client computing device may include a combination of the earlier listed examples of client computing devices, such as a desktop-type device or a mobile-type device combined with a wearable device, and the like.

[0076] Client computing devices 1106(1) to 1106(N) of various classes and device types may represent any type of computing device having one or more data processing units 1192 operably connected to a computer-readable medium 1194 (e.g., via a bus 1116), which in some instances may include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of various local, peripheral, and / or independent buses.

[0077] Executable instructions stored on computer-readable medium 1194 may include, for example, operating system 1119 , client module 1120 , profile module 1122 , and other modules, programs, or applications that may be loaded and executed by data processing unit(s) 1192 .

[0078] The client computing devices 1106(1)-1106(1)N may also include one or more interfaces 1124 to enable communications between the client computing devices 1106(1)-1106(N) and other networked devices (e.g., device(s) 1110) over the network(s) 1108. Such network interfaces 1124 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. Additionally, client computing devices 1106(1) to 1106(N) may include input / output ("I / O") interface (device) 1126 that enables communication with or includes input / output devices (e.g., user input devices including peripheral input devices (e.g., game controllers, keyboards, mice, pens, sound input devices such as microphones, cameras for obtaining and providing video feeds and / or still images, touch input devices, gesture input devices, etc.), and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, touch output devices, etc.). Figure 11 The client computing device 1106 (N) is shown connected in some manner to a display device (e.g., display screen 1129 (1)) that can display a UI according to the techniques described herein.

[0079] exist Figure 11 In the example environment 1100 of FIG. 1 , client computing devices 1106 ( 1 ) through 1106 (N) can use their respective client modules 1120 to connect to each other and / or to other external devices in order to participate in a communication session 1104 or contribute activities to a collaborative environment. For example, a first user can utilize client computing device 1106 ( 1 ) to communicate with a second user of another client computing device 1106 ( 2 ). When executing client modules 1120 , the users can share data, which can result in client computing device 1106 ( 1 ) connecting to system 1102 and / or other client computing devices 1106 ( 2 ) through 1106 (N) via network(s) 1108 .

[0080] The client computing devices 1106(1) to 1106(N) (each of which is also referred to herein as a "data processing system") may use their respective profile modules 1122 to generate participant profiles ( Figure 11 1102 ), and provides the participant profile to other client computing devices and / or device(s) 1110 of system 1102. The participant profile may include one or more of the following: an identity of a user or a group of users (e.g., name, unique identifier (“ID”), etc.), user data (e.g., personal data), machine data such as location (e.g., IP address, room in a building, etc.), and technical capabilities, etc. The participant profile may be utilized to register a participant for a communication session.

[0081] like Figure 11 As shown in FIG, the device(s) 1110 of the system 1102 include a server module 1130 and an output module 1132. In this example, the server module 1130 is configured to receive media streams 1134(1) to 1134(N) from various client computing devices (e.g., client computing devices 1106(1) to 1106(N)). As described above, the media streams can include a video feed (e.g., audio and visual data associated with a user), audio data to be output along with the presentation of the user's avatar (e.g., an audio-only experience where no video data of the user is sent), text data (e.g., a text message), file data, and / or screen sharing data (e.g., a document, slide layout, image, video, etc. displayed on a display screen), etc. Thus, the server module 1130 is configured to receive a collection of various media streams 1134(1) to 1134(N) (the collection being referred to herein as "media data 1134") during the real-time viewing communication session 1104. In some scenarios, not all client computing devices participating in the communication session 1104 provide media streams. For example, a client computing device may be a consuming device or a “listening” device only, such that it only receives content associated with the communication session 1104 but does not provide any content to the communication session 1104.

[0082] In various examples, the server module 1130 can select aspects of the media stream 1134 to be shared with individual client computing devices of the participating client computing devices 1106(1)-1106(N). Accordingly, the server module 1130 can be configured to generate session data 1136 based on the stream 1134 and / or pass the session data 1136 to the output module 1132. The output module 1132 can then transmit communication data 1139 to the client computing devices (e.g., the client computing devices 1106(1)-1106(3) participating in the real-time viewing of the communication session). The communication data 1139 can include video, audio, and / or other content data provided by the output module 1132 based on content 1150 associated with the output module 1132 and based on the received session data 1136.

[0083] As shown, output module 1132 sends communication data 1139(1) to client computing device 1106(1), and sends communication data 1139(2) to client computing device 1106(2), and sends communication data 1139(3) to client computing device 1106(3), etc. The communication data 1139 sent to the client computing devices may be the same or may be different (e.g., the location of the content stream within the user interface may be different between devices).

[0084] In various implementations, the device(s) 1110 and / or the client module 1120 may include a GUI presentation module 1140. The GUI presentation module 1140 may be configured to analyze communication data 1139 for delivery to one or more of the client computing devices 1106. Specifically, the GUI presentation module 1140 at the device(s) 1110 and / or the client computing device 1106 may analyze the communication data 1139 to determine an appropriate manner for displaying video, images, and / or content on the display screen 1129 of the associated client computing device 1106. In some implementations, the GUI presentation module 1140 may provide the video, images, and / or content to a presentation GUI 1146 that is rendered on the display screen 1129 of the associated client computing device 1106. The GUI presentation module 1140 may cause the presentation GUI 1146 to be rendered on the display screen 1129. The presentation GUI 1146 may include the video, images, and / or content analyzed by the GUI presentation module 1140.

[0085] In some implementations, the presentation GUI 1146 can include multiple sections or grids that can render or include video, images, and / or content for display on the display screen 1129. For example, a first section of the presentation GUI 1146 can include a video feed of a presenter or individual, and a second section of the presentation GUI 1146 can include a video feed of an individual consuming conference information provided by the presenter or individual. The GUI presentation module 1140 can populate the first and second sections of the presentation GUI 1146 in a manner that appropriately simulates an environment experience that the presenter and individual can share.

[0086] In some implementations, the GUI presentation module 1140 can zoom in or provide a zoomed view of the individual represented by the video feed to highlight the individual's reactions to the presenter, such as facial features. In some implementations, the presentation GUI 1146 can include video feeds of multiple participants associated with a meeting (e.g., a general communication session). In other implementations, the presentation GUI 1146 can be associated with a channel such as a chat channel, an enterprise team channel, etc. Thus, the presentation GUI 1146 can be associated with an external communication session that is different from a general communication session.

[0087] Figure 12 A diagram illustrating example components of an example device 1200 (also referred to herein as a "computing device") configured to generate data for some of the user interfaces disclosed herein is shown. Device 1200 can generate data that can include one or more portions that can render or include video, images, virtual objects, and / or content for display on display screen 1129. Device 1200 can represent one of the device(s) described herein. Additionally or alternatively, device 1200 can represent one of client computing devices 1106.

[0088] As shown, device 1200 includes one or more data processing units 1202, computer-readable media 1204, and communication interface(s) 1206. The components of device 1200 are operatively connected, for example, via a bus 1209, which may include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of various local, peripheral, and / or independent buses.

[0089] As utilized herein, data processing unit(s) (e.g., data processing unit(s) 1202 and / or data processing unit(s) 1192) may represent, for example, a CPU-type data processing unit, a GPU-type data processing unit, a field programmable gate array (“FPGA”), another type of DSP, or other hardware logic component (which, in some instances, may be driven by a CPU). For example, but not limitation, illustrative types of hardware logic components that may be utilized include application specific integrated circuits (“ASICs”), application specific standard products (“ASSPs”), systems on chips (“SOCs”), complex programmable logic devices (“CPLDs”), and the like.

[0090] As used herein, computer-readable media (e.g., computer-readable media 1204 and computer-readable media 1194) can store instructions that can be executed by (multiple) data processing units. The computer-readable media can also store instructions that can be executed by an external data processing unit (e.g., by an external CPU, an external GPU) and / or by an external accelerator (e.g., an FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated into the computing device, while in some examples, one or more of the CPU, GPU, and / or accelerator is external to the computing device.

[0091] Computer-readable media (also referred to herein as multiple computer-readable media) may include computer storage media and / or communication media. Computer storage media may include one or more of volatile memory, non-volatile memory, and / or other persistent and / or secondary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Thus, computer storage media includes media in tangible and / or physical form included in a device and / or in a hardware component that is part of a device or external to a device, including but not limited to random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), phase change memory (“PCM”), read-only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, compact disk read-only memory (“CD-ROM”), digital versatile disk (“DVD”), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage devices, magnetic cards or other magnetic storage devices or media, solid-state memory devices, storage arrays, network attached storage devices, storage area networks, hosted computer storage devices, or any other storage memory, storage devices, and / or storage media that can be used to store and maintain information for access by a computing device.

[0092] In contrast to computer storage media, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism. As defined herein, computer storage media does not include communication media. That is, computer storage media does not include communication media consisting solely of a modulated data signal, a carrier wave, or a propagated signal itself.

[0093] The communication interface(s) 1206 may represent, for example, a network interface controller ("NIC") or other type of transceiver device to send and receive communications over a network. Additionally, the communication interface(s) 1206 may include one or more cameras and / or audio devices 1222 to enable the generation of video feeds and / or still images, etc.

[0094] In the example shown, computer-readable medium 1204 includes a data repository 1208. In some examples, data repository 1208 includes a data storage device, such as a database, a data warehouse, or other type of structured or unstructured data storage device. In some examples, data repository 1208 includes a corpus and / or a relational database having one or more tables, indexes, stored procedures, etc. to enable data access, including, for example, one or more of the following: Hypertext Markup Language ("HTML") tables, Resource Description Framework ("RDF") tables, Web Ontology Language ("OWL") tables, and / or Extensible Markup Language ("XML") tables.

[0095] The data repository 1208 can store data for the operation of processes, applications, components, and / or modules stored in the computer-readable medium 1204 and / or executed by the data processing unit(s) 1218 and / or accelerator(s). For example, in some examples, the data repository 1208 can store session data 1210 (e.g., session data 1136), profile data 1212 (e.g., associated with participant profiles), and / or other data. The session data 1210 can include the total number of participants (e.g., users and / or client computing devices) in a communication session, activities occurring in the communication session, a list of invitees for the communication session, and / or other data related to when and how the communication session was conducted or hosted. The data repository 1208 can also include content data 1214, for example, including video, audio, or other content for rendering and display on one or more of the display screens 1129.

[0096] Alternatively, some or all of the data referenced above may be stored on a separate memory 1181 on board one or more data processing units 1202 (e.g., on board a CPU-type processor, a GPU-type processor, an FPGA-type accelerator, a DSP-type accelerator, and / or other accelerator). In this example, the computer-readable medium 1204 also includes an operating system 1218 and (multiple) application programming interfaces 1210 (APIs) configured to expose the functionality and data of the device 1200 to other devices. Additionally, the computer-readable medium 1204 includes one or more modules (e.g., a server module 1230, an output module 1232, and a GUI presentation module 1246), but the number of modules shown is merely an example, and the number may be higher or lower. That is, the functions described herein in association with the modules shown may be performed by a smaller number of modules or a larger number of modules on one device or across multiple devices.

[0097] It should be appreciated that, unless specifically stated otherwise, conditional language used herein (e.g., "can," "could," "might," or "may") is understood within the context to convey that certain examples include and other examples do not include certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that one or more examples in any way require certain features, elements, and / or steps, or that one or more examples must include logic for deciding (with or without user input or prompting) whether certain features, elements, and / or steps are to be included or performed in any particular example. Unless specifically stated otherwise, conjunction language such as the phrase "at least one of X, Y, or Z" should be understood to mean that an item, term, etc. may be X, Y, or Z, or a combination thereof.

[0098] Other variations of the technology disclosed herein may also be within the scope of this disclosure. For example, although the examples disclosed herein relate to the selection of comments, the technology disclosed herein includes any user selection of characters, words, and images or any other graphical elements associated with a comment or text. Thus, if a user selects a particular word or a particular image within a comment or any other text, the system may respond by displaying a video rendering within the portion of the user interface that contains the selected word, image, etc. It will be appreciated that each comment or phrase within the text portion may also include multiple links. Thus, in the examples disclosed herein, a single comment may include multiple words, where each word of the single comment each has a unique link.

[0099] It should also be recognized that various variations and modifications may be made to the examples described above, and that the elements thereof should be understood as being among other acceptable examples. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the accompanying claims.

[0100] Finally, although various configurations have been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended representations is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter.

[0101] The disclosure presented herein also covers the subject matter set forth in the following clauses:

[0102] Clause 1. A method for operating to be executed by a data processing system, the method comprising: causing display of a user interface comprising a video display area and a comments portion, wherein the user interface is positioned to display the comments portion within a viewing area of a display device and the user interface is positioned to position a video display area displaying a rendering of content outside the viewing area; receiving user input indicating a selection of at least a portion of a comment displayed within the comments portion; and in response to receiving the user input, generating a second rendering of the content for display within the comments portion, wherein the user interface is configured to: maintain the position of the user interface to position the comments portion within the viewing area of the display device while displaying the second rendering of the content within the viewing area of the display device.

[0103] Clause 2. The method of clause 1, wherein the method further comprises: controlling a position of the user interface to display the comment concurrently with the second rendering of the content.

[0104] Clause 3. The method of clauses 1-2, wherein the method further comprises: controlling a position of the user interface to display the related comments concurrently with the second rendering of the comments and content.

[0105] Clause 4. A method according to clauses 1-3, wherein the method further comprises: determining that the media type associated with the comment comprises a still image of the content, and wherein the second rendering comprises the still image of the content displayed within a viewing area of the display device in response to the user input.

[0106] Clause 5. A method according to clauses 1-4, wherein the method further comprises: determining that the media type associated with the comment includes audio data of the content, and wherein the second rendering comprises a graphical user interface indicating playback of the audio data; and causing the audio device to generate an audio output of the audio data.

[0107] Clause 6. The method of clauses 1-5, wherein the metadata associated with the comment defines a time interval of video data, the video data defining the content, wherein the second rendering comprises displaying the time interval of the content within a viewing area of the display device in response to user input.

[0108] Clause 7. A method according to clauses 1-6, wherein the method further comprises: analyzing user input based on data received from an input device to determine an input type; in response to determining that the input type is the first input type, displaying a second rendering of the content within the comment section while maintaining the position of the user interface; and in response to determining that the input type is the second type, displaying a customized user interface that simultaneously displays the selected comment and at least one of the rendering of the content or the second rendering of the content.

[0109] Clause 8. The method of clauses 1-7, wherein the first input type comprises: a cursor hovering over at least a portion of the comment.

[0110] Clause 9. The method of clauses 1-8, wherein the second input type comprises: hovering of a cursor over at least a portion of the comment; and user actuation of an input device, the user actuation indicating selection of the comment.

[0111] Clause 10. A method for operating to be performed by a data processing system, the method comprising: causing display of a user interface comprising a video display area and a comments portion, wherein the user interface is positioned to display the comments portion within a viewing area of a display device and the user interface is positioned to position a video display area where content is rendered outside of the viewing area; receiving user input indicating a selection of comments displayed within the comments portion; and in response to receiving the user input, generating a customized user interface for rendering video content simultaneously with the display of the comments, wherein the customized user interface is configured to have a threshold level of overlap between the rendering of the video content and the comments.

[0112] Clause 11. The method of clause 10, wherein the method further comprises: controlling a position of the user interface to display the comment concurrently with the second rendering of the content.

[0113] Clause 12. The method of clauses 10-11, wherein the method further comprises: controlling a position of the user interface to display the related comments concurrently with the second rendering of the comments and content.

[0114] Clause 13. A method according to clauses 10-12, wherein the method further comprises: determining that the media type associated with the comment comprises a still image of the content, and wherein the second rendering comprises the still image of the content displayed within a viewing area of the display device in response to the user input.

[0115] Clause 14. A method according to clauses 10-13, wherein the method further comprises: determining that the media type associated with the comment includes audio data of the content, and wherein the second rendering comprises a graphical user interface indicating playback of the audio data; and causing the audio device to generate an audio output of the audio data.

[0116] Clause 15. The method of clauses 10-14, wherein the metadata associated with the comment defines a time interval of video data, the video data defining the content, wherein the second rendering comprises displaying the time interval of the content within a viewing area of the display device in response to the user input.

[0117] Clause 16. A system comprising: a unit for causing display of a user interface, the user interface including a video display area and a text portion, wherein a navigation position of the user interface displays the text portion within a viewing area of a display device and the navigation position of the user interface positions a video display area that displays a rendering of content outside the viewing area; a unit for receiving user input indicating a selection of selected text displayed within the text portion; and a unit for generating a second rendering of the content for display within the text portion in response to receiving the user input, wherein the user interface is configured to: maintain the navigation position of the user interface to position the text portion within the viewing area of the display device while rendering audio or a still image or video of the second rendering of the content displayed within the viewing area of the display device.

[0118] Clause 17. The system of clause 16, wherein the method further comprises controlling a navigation position of the user interface to display the commentary simultaneously with rendering the audio or displaying a second rendered still image or video of the content within a viewing area of the display device.

[0119] Clause 18. The system of clauses 16-17, wherein the method further comprises: controlling a navigation position of the user interface to display related comments concurrently with the second rendering of the comments and content.

[0120] Clause 19. The system of clauses 16-18, further comprising: means for determining that the media type associated with the comment comprises a still image of the content, and wherein the second rendering comprises the still image of the content displayed within the viewing area of the display device in response to the user input.

[0121] Clause 20. The system according to clauses 16-19 further includes: a unit for analyzing user input based on data received from an input device to determine an input type; a unit for displaying a second rendering of the content within the text portion while maintaining a navigation position of the user interface in response to determining that the input type is the first input type; and a unit for displaying a customized user interface in response to determining that the input type is the second type, the customized user interface simultaneously displaying the selected text and at least one of the rendering of content 116 or the second rendering of content 117.

Claims

1. A method for operation to be performed by a data processing system, the method comprising: causing display of a user interface, the user interface comprising a video display area and a comment section, wherein the user interface is positioned to display the comment section within a viewing area of a display device, and a first rendering of content is outside of the viewing area, wherein the outside of the viewing area is off-screen and not displayed by the display device; receiving user input indicating a selection of at least a portion of a review displayed within the review section; and In response to receiving the user input indicating a selection of the comment, generating a second rendering of the content for display within the comment portion, wherein the user interface is configured to: maintain the position of the user interface to position the comment within the viewing area of the display device, while simultaneously displaying the second rendering of the content within the viewing area of the display device, wherein the second rendering displayed in response to the user input indicating a selection of the comment is positioned within the viewing area and the first rendering is positioned outside of the viewing area.

2. The method according to claim 1, wherein The method also includes controlling the position of the user interface to display related comments simultaneously with the comments and the second rendering of the content.

3. The method according to claim 1, wherein The method also includes determining that a media type associated with the comment comprises a still image of the content, and wherein the second rendering comprises the still image of the content displayed within the viewing area of the display device in response to the user input.

4. The method according to claim 1, wherein The method further comprises: determining that the media type associated with the comment includes audio data of the content, and wherein the second rendering includes a graphical user interface indicating playback of the audio data; and An audio device is caused to generate an audio output of the audio data.

5. The method according to claim 1, wherein Metadata associated with the comment defines a time interval of video data, the video data defining the content, wherein the second rendering comprises displaying the content within the viewing area of the display device for the time interval in response to the user input.

6. The method according to claim 1, wherein The method further comprises: analyzing the user input based on data received from the input device to determine a type of input; In response to determining that the input type is the first input type, displaying the second rendering of the content within the comments section while maintaining the position of the user interface; and In response to determining that the input type is the second input type, a customized user interface is displayed that simultaneously displays the selected comment and at least one of the first rendering of the content or the second rendering of the content.

7. The method according to claim 6, wherein: The first input type includes a cursor hovering over at least a portion of the comment.

8. The method according to claim 6, wherein: The second input type includes: a cursor hovering over at least a portion of the comment; and a user actuation of an input device, the user actuation indicating a selection of the comment.

9. A method for operation to be performed by a data processing system, the method comprising: causing display of a user interface, the user interface comprising a video display area and a comment section, wherein the user interface is positioned to display the comment section within a viewing area of a display device and a first rendering of content is outside of the viewing area, wherein the outside of the viewing area is off-screen and not displayed by the display device; receiving user input indicating a selection of a comment displayed within the comment section; and In response to receiving the user input indicating a selection of the comment, generating a customized user interface for displaying a second rendering of the content concurrently with the display of the comment, wherein the customized user interface is configured to have a threshold level of overlap between the second rendering of the content and the comment, wherein the second rendering displayed in response to the user input indicating a selection of the comment is positioned within the viewing area and the first rendering is positioned outside of the viewing area.

10. The method according to claim 9, wherein: The method also includes controlling the position of the user interface to display related comments simultaneously with the comments and the second rendering of the content.

11. The method according to claim 9, wherein: The method also includes controlling the position of the user interface to display related comments simultaneously with the comments and the second rendering of the content.

12. The method according to claim 9, wherein The method also includes determining that a media type associated with the comment comprises a still image of the content, and wherein the second rendering comprises the still image of the content displayed within the viewing area of the display device in response to the user input.

13. The method according to claim 9, wherein: The method further comprises: determining that the media type associated with the comment includes audio data of the content, and wherein the second rendering includes a graphical user interface indicating playback of the audio data; and An audio device is caused to generate an audio output of the audio data.

14. The method according to claim 9, wherein Metadata associated with the comment defines a time interval of video data, the video data defining the content, wherein the second rendering comprises displaying the content within the viewing area of the display device for the time interval in response to the user input.

Images