Anchor for live streaming

Abstract

The present disclosure relates to anchors for live streaming. A stream hosting server generates anchors associated with the live stream, each anchor specifies a timestamp of the live stream that represents an appropriate moment for a user to join the live stream. When a viewer client device sends a request to join the live stream, the stream hosting server analyzes the anchor list and selects an appropriate anchor. The stream hosting server provides the live stream to the viewer client device starting from the timestamp specified by the anchor. Thus, the viewer client device can start displaying the live stream at the appropriate moment specified by the anchor. The stream hosting server also creates video on demand content, which includes the complete live stream and the anchors associated with the live stream. The viewer client device can start displaying the VOD at different anchors.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201780067750.2, filed on September 28, 2017. Technical Field

[0003] The present disclosure relates generally to live streaming, and more particularly, to generating anchors in a live stream to enable users joining or rejoining the live stream to enter at the appropriate time. Background Art

[0004] The stream hosting server serves as a platform for stream creators to provide live streams to users of the stream hosting server. In many scenarios, users use client devices to access and join the live stream. For popular streams, the creator of the stream can gain a large number of viewers who are interested in the information presented by the creator.

[0005] However, a problem unique to live streams is that viewers may join at various suboptimal times. For example, the creator may be in the middle of a discussion when a viewer joins, and therefore, the viewer may not have the proper context to fully understand the ongoing discussion. This can occur when a user newly joins a live stream or when a user rejoins a live stream after having previously been transferred. Joining a live stream at a suboptimal moment can result in a poor user experience and may even prevent the user from participating in future live streams. Summary of the invention

[0006] The stream hosting server generates an anchor in the live stream that enables a newly joined user to begin consuming the stream content at an appropriate point in time, even if the user joins at a suboptimal time. The stream hosting server provides a platform for creators to transmit live streams. At various points in time during the live stream, the server provides a user interface element that receives creator input indicating that the time point is suitable for generating a live stream anchor. For example, when the creator continues to discuss a new topic, the user interface element receives input to create an anchor. In another example, the live stream is a streaming video game, and the user interface element receives input to generate an anchor corresponding to a timestamp in the live stream when a new game starts. Each anchor specifies a timestamp of the live stream and a validity duration. The validity duration of an anchor describes the length of time that the anchor remains valid, so that a joined user can start watching the live stream at the anchor if the anchor is valid.

[0007] The viewer client device provides a request to join the live stream to the stream hosting server. The request sent by the client device is sent at a specific time and is therefore associated with the timestamp of the live stream. In one embodiment, the stream hosting server analyzes the list of generated anchors and identifies the appropriate anchor from which the viewer client device will access the live stream. The appropriate anchor is selected based on the timestamp of the live stream specified by the anchor and the validity duration associated with the anchor. The stream hosting server provides the live stream to the viewer client device starting from the timestamp specified by the identified anchor. Therefore, the present disclosure solves the technical problem of viewer navigation within the live stream, and can achieve the technical effect that the viewer can locate and watch the content of interest more quickly than when the anchor is not present. In addition, the present disclosure can achieve the following technical effects: since the viewer does not watch the immediately preceding portion of the live stream, the user does not have to view content that he or she is not interested in or even incomprehensible content.

[0008] The stream hosting server is also capable of generating video on demand (VOD) content derived from the completed live stream. In addition, the generated anchor associated with the completed live stream is also included in the created VOD. Anchors in the VOD have multiple uses. In one case, the VOD can be presented to the user by a viewer client device. Upon receiving user input, the viewer client device can display the VOD at different timestamps specified by the anchor associated with the VOD. This is desirable in game streaming, where the session scenes of the VOD can be quickly skipped. Therefore, the VOD with an associated anchor enables jumping to the start of the next game.

[0009] Additionally, anchors in the VOD may be tagged with topic tags, such as keywords. When the stream hosting server receives a query that includes keywords (or more generally, topic tags), the stream hosting server returns a list of anchors that are relevant to the provided keywords. Thus, as opposed to having to display the VOD from the beginning, the server enables the viewer client device to access the VOD starting from an anchor with a particularly relevant topic.

[0010] The present disclosure provides a computer-implemented method, a computer system for implementing the method, and a computer program product including computer program instructions for implementing the method. Each computer program product is typically a non-transitory computer-readable medium. However, in principle, the computer program product may also be a signal transmitted over a communication network. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a system environment of a stream hosting server according to one embodiment.

[0012] Figure 2 Depicted is the architecture of a live streaming module according to one embodiment.

[0013] Figure 3A Depicted is the architecture of software modules of a creator client device according to one embodiment.

[0014] Figure 3B Depicted is the architecture of a software module of a viewer client device according to one embodiment.

[0015] Figure 4 is an example flow chart of providing a live stream based on a generated anchor by a stream hosting server according to one embodiment.

[0016] Figure 5 is an example flow chart of playback of a live stream by a viewer client device based on a generated anchor, according to one embodiment.

[0017] The drawings depict various embodiments for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles of the present disclosure described herein. DETAILED DESCRIPTION

[0018] System environment overview

[0019] Figure 1 Depicted is a system environment 100 of a stream hosting server 150 for providing live streaming, according to one embodiment. The system environment 100 includes a creator client device 108, a viewer client device 110, a network 130, and a stream hosting server 150.

[0020] Steering Figure 1 , a creator client device 108 and a viewer client device 110, respectively, generate a live stream and access a live stream. As used herein, "live stream" refers to any streaming media content distributed between a creator and a viewer (or listener in the case of audio content), regardless of whether the content is broadcast, multicast, or exchanged directly between the creator and a single viewer.

[0021] Each of the creator client device 108 and the viewer client device 110 is a computing device having a software module 300A / B that sends and / or receives data via the network 130. Examples of the creator client device 108 and the viewer client device 110 include a desktop computer, a laptop computer, a tablet computer (tablet), a mobile phone, a personal digital assistant (PDA), a gaming device, or any other electronic device that includes computing functions and data communication capabilities. The creator client device 108 and the viewer client device 110 can each use a web browser such as Microsoft Internet Explorer, Mozilla Firefox, Google Chrome, Apple Safari, and / or Opera as an interface to connect to the network 130. In addition or alternatively, a dedicated application software running locally on the mobile device is used as an interface to connect to the network 130. The creator 108 and viewer client devices 110 typically include a processor, a display device (or an output to a display device), local storage such as a hard drive or flash memory device (where the creator 108 and viewer client devices 110 store data used by the user when performing tasks), and a network interface for coupling to a stream hosting server 150 via a network 130. Although Figure 1 One creator client device 108 and one viewer client device 110 are depicted, but those skilled in the art will appreciate that any number of creator client devices 108 and viewer client devices 110 may be present in the system architecture in communication with the network 130 and the stream hosting server 150 .

[0022] The creator uses the creator client device 108 to generate the live stream. The creator client device 108 receives user input through the user interface 180. For example, the creator client device 108 (e.g., a laptop computer) receives input that instructs the creator client device 108 to use software or hardware to continuously capture video and audio in real time or near real time to generate the live stream. In various embodiments, the creator client device 108 also includes a software module 300A that appropriately processes the video and audio of the live stream when recording the video and audio of the live stream. The creator client device 108 then continuously sends the stream to the stream hosting server 150 until the stream ends.

[0023] The viewer client device 110 sends a request to access the live stream and, upon receiving the live stream, begins accessing the live stream at a timestamp specified by an anchor associated with the live stream. The viewer client device 110 includes a user interface 180 and a software module 300B that together handle interaction between a user of the viewer client device 110 and content provided by the stream hosting server 150.

[0024] For example, the user interface 180 of the viewer client device 110 receives a user input (e.g., a click, a touch input) for joining the live stream. The software module 300B sends a join request for the live stream to the stream hosting server 150. At a time after sending the join request, the software module 300B receives a live stream of an identified anchor with a timestamp specifying the live stream. Upon receiving the live stream, the software module 300B appropriately decodes the video and audio of the live stream for playback on the user interface 180 of the viewer client device 110. The viewer client device 110 continues to display the live stream starting from the timestamp specified by the anchor.

[0025] The stream hosting server 150 is responsible for receiving the live stream from the creator client device 108, generating anchors associated with the live stream in various cases, such as in response to anchor generation requests from the creator client device 108, and providing the live stream for playback on multiple different viewer client devices, each viewer client device may start at a different timestamp within the stream specified by one or more anchors and proceed forward. In an example embodiment, the stream hosting server 150 includes a live streaming module 200, a content server 104, an ingest server 106, and a content storage 120.

[0026] The live stream module 200 processes the live stream. On the server side, the live stream module 200 receives a stream to be sent to the viewer client device 110, and also receives an anchor generation request. As described above, each anchor specifies a timestamp in the live stream, so that the live stream module 200 can provide the live stream to the viewer client device 110 starting from the timestamp. On the viewer side, the live stream module 200 receives a join request to join the live stream from one or more viewer client devices 110. The live stream module 200 analyzes and identifies the appropriate anchor associated with the live stream for each join request. The live stream module 200 provides the live stream and the anchor specifying the timestamp in the live stream to the viewer client device 110, so that the viewer client device 110 can play the live stream starting from the timestamp. Referring to the following Figure 2 Describe further details about the live broadcast module 200.

[0027] The live stream module 200 is also responsible for providing content items (e.g., video advertisements) to the viewer client device 110. For example, the content item may initially be provided by a third party system (e.g., an advertiser) and processed by the ingest server 106. The ingest server 106 processes the provided content item and stores the content item in the content storage 120. The content server 104 retrieves the content item located in the content storage 120 and provides the content item to the viewer client device for display on the user interface 180 of the viewer client device 110. As an example, the content item will be displayed on the viewer client device 110 to temporarily replace the live stream. Once the viewer client device 110 plays the content item for its duration, the viewer client device 110 returns to playing the live stream at the generated anchor.

[0028] The network 130 facilitates communication between one or more client devices 110 and the stream hosting server 150. The network 130 can be any wired or wireless local area network (LAN) and / or wide area network (WAN), such as an intranet, an extranet, or the Internet. In various embodiments, the network 130 uses standard communication technologies and / or protocols. Examples of technologies used by the network 130 include Ethernet, 802.11, 3G, 4G, 802.16, or any other suitable communication technology. The network 130 can use wireless, wired, or a combination of wireless and wired communication technologies. Examples of protocols used by the network 130 include Transmission Control Protocol / Internet Protocol (TCP / IP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), File Transfer Protocol (TCP), or any other suitable communication protocol.

[0029] Live streaming module

[0030] Figure 2 A live stream module 200 and its associated modules are depicted according to one embodiment. In this example embodiment, the live stream module 200 includes an anchor generation module 210, a stream communication module 220, an anchor analyzer module 230, an anchor list 240, a video on demand (VOD) generator module 250, and a search module 260. In some embodiments, the live stream module 200 has different modules and / or logical structures than those described herein, and functionality may be distributed among the modules and / or logical structures in a manner different than that described herein.

[0031] The anchor generation module 210 generates one or more anchors to be associated with the live stream. Each anchor represents an appropriate timestamp in the live stream for the viewer client device 110 to access the live stream. For example, in a game stream, an anchor may be associated with a point in time corresponding to the start of a new game. As another example, the creator may switch from a first topic to a second topic during a live stream.

[0032] The anchor generation module 210 allows the creator to create an anchor through the graphical user interface 180 presented on the creator client device 108. The creator may choose to place the anchor at the current timestamp of the ongoing live stream at his / her discretion. For example, the creator provides user input on the graphical user interface (e.g., clicks a button or drags and drops). As another example, the creator may later manually specify a timestamp that occurs earlier in the live stream to generate the anchor. This is useful for creators who may forget to generate the anchor in real time. In response to the user input for generating an anchor in the live stream, the creator client device 108 sends an anchor generation request to the anchor generation module 210, which then proceeds to generate the anchor. In various embodiments, the anchor generation module 210 may provide a notification to the viewer client device 110 that is currently accessing the live stream when a new anchor is generated.

[0033] Each anchor is associated with a live stream and specifies: 1) a timestamp within the duration of the live stream, and 2) a validity duration. For example, the timestamp of the live stream corresponds to a point in time in the live stream that coincides with the time when the creator provided input to create the anchor. As another example, the timestamp corresponds to a time manually specified by the creator.

[0034] The validity duration of an anchor describes the period of time that the anchor is valid after it is generated. Once the validity duration of an anchor expires, the viewer client device 110 can no longer access the live stream starting from that anchor. In some embodiments, the validity duration of the anchor is predetermined by the anchor generation module 210. In other embodiments, the validity duration of each anchor may be manually specified by the creator. For example, before the live stream begins, the anchor generation module 210 may prompt the creator to enter a default validity duration (e.g., 5 minutes) for each anchor that will be generated during the live stream. In addition, the anchor generation module 210 may prompt the creator to enter the validity duration of each generated anchor in real-time (e.g., immediately after the creator specifies the creation of the anchor).

[0035] The generation of anchors can also be performed automatically by the anchor generation module 210 without the intervention of the creator. The anchor generation module 210 can implement a machine learning model that analyzes the creator's preferences and / or characteristics to understand the possible locations of appropriate anchors. For example, when the creator considers what to say about the next topic, the creator may be inclined to silent moments that last longer than a threshold time. Therefore, when a silence duration longer than a threshold occurs in the stream, the machine learning model can choose to embed an anchor in the live stream. As another example, the creator can use a specific phrase indicating that a new topic is starting (e.g., "Okay, shifting gears..."). The machine learning model can be specifically trained to recognize the creator's tendencies or used phrases and generate anchors accordingly.

[0036] Each generated anchor is stored in anchor list 240. Anchor list 240 may be a readable data file or database table that stores the associated stream, an identifier of the anchor (eg, anchor 1, anchor 2, etc.), a timestamp, and a validity duration associated with each anchor.

[0037] The stream communication module 220 receives the live stream from the creator client device 108 and is responsible for processing, storing and / or buffering the stream and transmitting the stream to one or more viewer client devices 110. One example for processing the stream includes transcoding the live stream received from the creator client device 108. For example, the video and audio may have been previously encoded by the creator client device 108. The video may be in H.264 format and the audio may be in Advanced Audio Coding (AAC) format. The stream communication module 220 may transcode the video into a different format, video resolution and / or frame rate, and transcode the audio into a different format or different bit rate. The transcoded video and audio are transmitted to the viewer client device 110 by the stream communication module 220 according to a standard protocol (e.g., Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP), User Datagram Protocol (UDP), Transmission Control Protocol (TCP)).

[0038] The stream communication module 220 handles communications with the various viewer client devices 110. For example, the stream communication module 220 receives requests from the various viewer client devices 110 to access the live stream and additionally provides the live stream starting from the timestamp specified by the generated anchor. One type of request is a join request from a viewer client device 110 that is newly accessing the live stream. Another type of request is a rejoin request from a viewer client device 110 that was previously accessing the live stream, temporarily exited, and is now attempting to rejoin the stream.

[0039] In the use case where a join request is received, in response to receiving the join request from the viewer client device 110, the stream communication module 220 provides a timestamp associated with the join request to the anchor analyzer module 230. The anchor analyzer module 230 selects an anchor from the anchor list 240 based on the provided timestamp associated with the join request. Details regarding the analysis and selection of anchors will be further described below.

[0040] The stream communication module 220 receives the identified anchor from the anchor list from the anchor analyzer module 230. The identified anchor specifies a timestamp of the live stream that begins when the viewer client device 110 begins accessing the live stream. Thus, the stream communication module 220 provides the live stream to the client device 110 starting from the timestamp specified in the selected anchor.

[0041] Use cases for receiving a rejoin request include where a viewer client device 110 accesses a live stream, diverts from the live stream to play a content item, and then attempts to rejoin the live stream upon completing the content item. As an example, the content item is an advertisement that temporarily replaces the live stream displayed on the viewer client device 110. The advertisement can be a TrueView advertisement that allows a user to skip the advertisement after a threshold time period (e.g., 5 seconds). Thus, the user can complete the content item by providing user input (e.g., a click) for skipping the content item after the threshold amount of time. When the content item completes playback, the viewer client device 110 sends a rejoin request to the stream communication module 220. Therefore, anchors for live streams are advantageous because they enable the viewer client device 110 to rejoin the live stream at an appropriate timestamp after viewing a content item.

[0042] In order to enable the viewer client device 110 to properly leave the live stream and play the content item, in addition to the content item, the stream communication module 220 also provides further instructions to the viewer client device 110. As an example, the instructions include encoding / decoding information of the content item so that the viewer client device 110 can fully play back the video and audio data of the content item. As another example, the stream communication module 220 also transmits an exit time interval. The exit time interval represents the length of time that the viewer client device 110 can access the live stream before being transferred from the live stream to the content item. When the exit time interval expires, the viewer client device 110 temporarily exits the live stream to display the content item. When the viewer client device 110 exits the live stream, it records the timestamp of the live stream corresponding to its exit (e.g., the exit timestamp).

[0043] While the content item is temporarily played by the viewer client device 110 (i.e., in accordance with or after the leave timestamp), the anchor generation module 210 may generate additional anchors for the live stream. When the stream communication module 220 receives a rejoin request after terminating the playing of the content item, the stream communication module 220 also receives the leave timestamp from the viewer client device 110. The stream communication module 220 passes the received leave timestamp to the anchor analyzer module 230 so that the appropriate anchor can be identified given the rejoin request. When the stream communication module 220 receives the identified appropriate anchor from the anchor analyzer module 230, the stream communication module 220 provides a live stream starting from the timestamp specified by the appropriate anchor. Thus, the stream communication module 220 allows the viewer client device 110 to rejoin the live stream by providing a live stream starting from the identified anchor. More generally, the implementation of the generated anchors allows the user of the viewer client device 110 to not miss any live stream or to miss a very small amount of live stream or to rejoin at an appropriate anchor point when being redirected to watch the content item.

[0044] The stream communication module 220 can provide information to the viewer client device 110 indicating that the live stream is currently in a "semi-live" state. That is, the live stream (composed of content generated in real time (e.g., by being captured by a camera)) is displayed to the viewer with a (deliberate) delay that depends on the identified anchor; the delay can be selected to be greater than the delay on the live stream that the display system would use for buffering purposes. For example, the viewer client device 110 displays an indication of the semi-live state on a graphical user interface (GUI) to the viewing user on the user interface 180. The indication can be a colored button or indicator. In addition, the viewer client device 110 can also present selectable options to the user on the GUI. When the GUI receives the user input, the viewer client device 110 sends a request to the stream communication module 220 to access the live segment of the live stream. Therefore, this provides the user of the viewer client device 110 with control over viewing the live stream in a live or semi-live state. In response to the request, the stream communication module 220 provides the live segment of the live stream to the viewer client device 110. The viewer client device 110 may change the indication on the GUI to display a different indicator (eg, a different color or a different button) so that the user can easily understand that the live stream is now in a live state.

[0045] The anchor analyzer module 230 selects an appropriate anchor from the anchor list 240 so that the viewer client device 110 can join or rejoin the live stream at the appropriate timestamp specified by the anchor. The anchor analyzer module 230 selects the anchor as the appropriate anchor based on the timestamp specified by the anchor, the validity duration specified by the anchor, and the timestamp associated with the join or rejoin request sent by the client device 110. The anchor analyzer module 230 may default to the case where no anchor is selected. In this case, the anchor analyzer module 230 instructs the stream communication module 220 to provide the live stream to the viewer client device 110 at its current live time (rather than starting from the anchor).

[0046] The anchor analyzer module 230 identifies the appropriate anchor from the anchor list 240 through a process that depends on the type of request (e.g., a join or rejoin request) sent by the viewer client device 110. For example, if the viewer client device 110 is newly joining the live stream (e.g., a join request), the anchor analyzer module 230 retrieves an anchor list 240 that includes all generated anchors associated with the live stream. Alternatively, if the viewer client device is rejoining the live stream (e.g., a rejoin request), the anchor analyzer module 230 retrieves an anchor list 240 that includes only anchors that were generated while the content item was playing on the viewer client device 110 (or generated during a short predetermined time before the content item began playing on the viewer client device).

[0047] Of the anchors in the retrieved anchor list 240, the anchor analyzer module 230 determines which anchors remain valid based on the validity duration associated with each anchor. To do this, the anchor analyzer module 230 calculates the difference between the timestamp specified by the anchor and the timestamp associated with when the join or rejoin request was sent (the "current time"). The difference is compared to the validity duration associated with the identified anchor. If the difference is lower than the validity duration, indicating that the anchor is valid, the anchor analyzer module 230 marks the anchor as valid. Alternatively, if the difference between the current time and the timestamp specified by the identified anchor is higher than the validity duration, the anchor is marked as invalid. Invalid anchors are ignored and no longer considered further, while valid anchors are further analyzed. In the event that all anchors in the anchor list 240 are invalid, the anchor analyzer module 230 continues with the default case and instructs the stream communication module 220 to provide a live stream in a live state.

[0048] After identifying a valid anchor, the anchor analyzer module 230 identifies the anchor based on whether the viewer client device 110 has sent a join request or a rejoin request to the stream communication module 220. If the viewer client device 110 has newly joined the live stream, the anchor analyzer module 230 identifies and selects the latest anchor in the anchor list 240. The latest anchor specifies a timestamp in the live stream that is newer than the timestamps specified by all other anchors in the anchor list 240.

[0049] If the viewer client device 110 rejoins the live stream after being temporarily diverted (e.g., to watch a content item), the anchor analyzer module 230 identifies when the viewer client device 110 exited the live stream (e.g., a leave time). The leave time may be sent in advance by the viewer client device 110 along with the rejoin request. The anchor analyzer module 230 selects the anchor with the timestamp closest to the identified leave time. The timestamp of the anchor closest to the identified leave time may occur before the identified leave time, which means that when rejoining the live stream at the anchor, the viewer client device 110 will replay a portion of the live stream. In this case, the viewer client device 110 avoids skipping any portion of the live stream. Alternatively, the timestamp of the anchor closest to the identified leave time may occur after the identified leave time. Therefore, the viewer client device 110 can rejoin the live stream at the appropriate timestamp of the anchor while missing very little of the live stream.

[0050] Once the anchor analyzer module 230 identifies the appropriate anchor, the anchor analyzer module 230 instructs the stream communication module 220 to send a live stream starting from the selected anchor to the viewer client device 110. The stream communication module 220 may do this by sending the live stream along with the identified anchor.

[0051] After the live stream is completed, the video on demand (VOD) generator module 250 obtains the stream and generates a VOD to be stored in the content storage 120. Therefore, the viewer client device 110 can access the VOD at a later time and display the live stream, even if it is no longer live streaming. When generating the VOD, the VOD generator module 250 associates the generated anchors originally created during the live stream with the VOD. For example, the VOD generator module 250 encodes the discarded anchors with the stream. In addition, the VOD generator module 250 creates an associated data file (or database table) that includes the timestamps specified by each anchor in the stream and the tags (e.g., descriptive keywords) describing each anchor. For example, the anchor can indicate that the creator switched from talking about the topic of dogs to the topic of cats at the timestamp in the stream. Therefore, the data file can mark the anchor with a specific topic about cats at the timestamp. The data file may include multiple tags for each anchor.

[0052] The search module 260 enables a user of the viewer client device 110 to efficiently search the VOD for anchors of interest. Thus, the search module 260 provides improved search granularity so that the viewer client device 110 can access a specific anchor within a VOD without having to access the VOD from scratch. For example, the search module 260 receives a search query and parses the query to obtain keywords. The search module 260 searches the content memory 120 for anchors in the VOD that match the query or keywords in the query. For example, if the query from the client device searches for "puppies", the search module 260 searches the metadata files associated with each VOD to identify anchors tagged with the subject of "puppies". The match can be a text string match between the search query and the keywords associated with the anchor. The search module 260 returns search results for anchors tagged with matching subjects to the viewer client device 110 that sent the query. If a user of the viewer client device 110 selects an anchor, indicating that he / she is interested in viewing a VOD starting at that anchor, the search module 260 retrieves the appropriate video segment in the VOD starting at that anchor from the content storage 120 and provides it to the viewer client device 110. This enables the user to view relevant clips of the VOD file without having to view the entire VOD, which may not be wholly relevant to the provided query.

[0053] Software Modules

[0054] Figure 3A Depicted is a software module 300A and its associated modules on a creator client device 108 according to one embodiment. The software module 300A includes a stream generator module 305 and a stream processing module 310.

[0055] The stream generator module 305 generates video and audio of a stream obtained from a hardware (e.g., camera) or software (e.g., video game) source that is communicatively coupled to a portion of the viewer client device 110 or running on the device. The generated stream is stored in the stream storage 315. The stream generator module 305 sets the characteristics of the live stream including video resolution, video frame rate, and audio bit rate. Alternatively, the characteristics of the live stream can be set by the creator to provide the best live stream viewing experience for his / her viewers.

[0056] The stream generator module 305 passes portions of the live stream to the stream processing module 310 as they are captured. Thus, the stream processing module 310 can process and send the stream while the stream generator module 305 continues to record new portions of the live stream. The stream processing module 310 encodes the video and audio according to their characteristics (e.g., resolution, frame rate, bit rate). As an example, the video is encoded using an H.264 encoder while the audio is separately encoded using an AAC encoder. Once encoded, the stream processing module 310 proceeds to send the encoded video and audio to the stream hosting server 150.

[0057] The stream processing module 310 sends the encoded video and audio of the live stream using a standard protocol (e.g., RTP, RTMP, UDP, TCP) to the stream hosting server 150. For example, for live streaming, RTP is preferred over UDP to ensure that viewers can watch the stream with limited delay.

[0058] Figure 3B A software module 300B and its associated modules on a viewer client device 110 are depicted according to one embodiment. The software module 300B enables the viewer client device 110 to access a live stream. In various embodiments, the software module 300B includes a user interface manager 350, a request module 355, and a playback module 365. In some embodiments, the software module 300A / 300B of the creator client device 108 or viewer client device 110 has a different module and / or logical structure than described herein, and functionality may be distributed among the modules and / or logical structures in a manner different than described herein.

[0059] The user interface manager 350 receives information from and provides information to a user of the client device 110 through the user interface 180. For example, the client device may be a mobile client device (e.g., a cellular phone) having a touch display interface. Thus, the user interface manager 350 receives touch input and also displays the live stream for viewing through the user interface 180. The user interface manager 350 may provide information during the live stream. As previously described, the user interface manager 350 provides a GUI on the user interface 180 of the viewer client device 110. The GUI may provide an indication that the current stream is in a semi-live state (e.g., a yellow button) rather than a live state (e.g., a green button). As another example, the user interface manager 350 may provide a notification whenever a new anchor is generated in the live stream.

[0060] The user interface manager 350 presents the VOD with an anchor to the viewer client device 110. While displaying the VOD, the user interface manager 350 may present additional selectable options through the user interface 180. For example, the selectable options may be forward or backward arrows displayed on the user interface 180. When user input is received on the forward arrow, the user interface manager 350 identifies a subsequent anchor (e.g., an anchor specifying a next timestamp in the VOD) and displays the VOD at that anchor. Similarly, if user input is received on the backward arrow, the user interface manager 350 identifies a previous anchor (e.g., an anchor specifying a previous timestamp in the VOD) and displays the VOD at the previous anchor. The operation of identifying a subsequent or previous anchor may be performed by the user interface manager of the query stream hosting server 150.

[0061] The user interface manager 350 may maintain a counter that tracks the number of times a user starts from or skips an anchor in a VOD. The counter represents the popularity of the anchor in the VOD. The updated counter information may be provided to the stream hosting server 150 to further improve the search functionality. For example, a viewer client device 110 may send a search for the term "puppy" and then receive a specific anchor in the VOD associated with "puppy". If most viewer client devices 110 that have provided a "puppy" search query eventually start displaying the VOD at the anchor (e.g., without skipping the anchor), this information may be provided back to the stream hosting server 150. Alternatively, if most viewer client devices 110 that have provided a "puppy" search query briefly display the VOD starting from the anchor, but quickly stop watching, the keyword "puppy" may be a bad topic association with the anchor. The stream hosting server 150 may use this information to adjust the keyword tags associated with the anchor and further update the ranking algorithm for running a search query that identifies an anchor in the VOD content.

[0062] The user interface manager 350 identifies when the viewer client device 110 is displaying the VOD and when the viewer client device 110 stops displaying the VOD. At a later time, if the viewer client device 110 is instructed to start displaying the VOD again, the viewer client device 110 may start displaying at the relevant point of the VOD specified by the anchor. In an example scenario, the user interface manager 350 may present the VOD on a web page, and the user decides to scroll up or down on the web page so that the VOD is no longer in view. Therefore, the user interface manager 350 records a timestamp indicating when the viewer client device 110 stops displaying the VOD. If the VOD is no longer in view, the VOD will stop playing. At a later time, the user may scroll back to the VOD. Therefore, the user interface manager 350 receives an indication that the viewer client device 110 continues to display the VOD. The user interface manager 350 may start playing the VOD at the nearest anchor, which may be before or after the timestamp indicating when the viewer client device 110 stopped displaying the VOD.

[0063] The user interface manager 350 receives input indicating that a user wants to access a live stream. The user interface manager 350 provides the input to the request module 355. The request module 355 is configured to provide various different requests from the viewer client device 110 to the stream hosting server 150. As a first example, in response to receiving the user input from the user interface manager 350, the request module 355 sends a join or rejoin request to the stream hosting server 150 (depending on whether the user is newly joining or rejoining). In a second example, the request module 355 may send a user query to the stream hosting server 150 to search for anchors of VODs that may be relevant to the user query. Each of these examples is further described below.

[0064] Referring to the first example, when the viewer client device 110 desires to access the live stream, the request module 355 determines whether to send a join request or a rejoin request. For example, if the viewer client device 110 newly accesses the live stream, the request module 355 chooses to send a join request. If the viewer client device 110 has previously accessed the live stream and is temporarily transferred (e.g., 1-5 minutes), the request module 355 chooses to send a rejoin request. In some cases, even if the viewer client has previously accessed the stream, the request module 355 chooses to send a join request instead of a rejoin request. For example, the request module 355 determines that the viewer client device has previously accessed the live stream, but exited the stream before a threshold amount of time (e.g., 30 minutes, 1 hour). Therefore, the request module 355 treats the viewer client device 110 as a new join request.

[0065] The request module 355 may send a rejoin request to the stream hosting server 150 along with the timestamp retrieved from the timestamp storage 370. The included timestamp assists the stream hosting server 150 in determining when the viewer client device 110 previously left the live broadcast. For example, the viewer client device 110 may have previously accessed the live stream and was subsequently transferred to watch the content item at the leave timestamp of the live stream. Thus, when the viewer client device 110 attempts to rejoin the stream, the request module 355 sends the leave time along with the rejoin request to the stream hosting server 150. The stream hosting server 150 determines the anchor generated when the client device 110 played the content item based on the leave time.

[0066] Referring now to a second example, the request module 355 also processes the submitted user query to search the VOD for anchors associated with a particular topic of interest. The request module 355 sends the search request to the search module 260 of the stream hosting server 150. For example, the request module 355 may send a search request for "puppies." In return, the viewer client device 110 receives a list of anchors associated with the topic of "puppies." The list of anchors may be presented in an order ranked according to a search algorithm used by the search module 260. Thus, the user may select and play a VOD from anchors related to a particular topic of interest included in the search query.

[0067] In some embodiments, the software module 300B of the viewer client device 110 also includes an anchor analyzer module 230 and is responsible for identifying the appropriate anchor to start accessing the live stream. Therefore, instead of the anchor analyzer module 230 being a module of the stream hosting server 150, this embodiment includes the anchor analyzer module 230 in the viewer client device 110.

[0068] The anchor analyzer module 230 on the viewer client device 110 receives the anchor list 240 from the stream hosting server 150. As an example, the anchor analyzer module 230 automatically identifies the appropriate anchor based on the timestamp of each anchor in the anchor list 240 and the timestamp associated with the join or rejoin request. As previously described, if the request is a join request, the anchor analyzer module 230 selects the anchor that specifies the most recent timestamp. If the request is a rejoin request, the anchor analyzer module 230 selects the anchor that specifies the timestamp that is closest to the leave time.

[0069] As a second example, the anchor analyzer module 230 may query the user of the user client device 110 for user input to identify an appropriate anchor. The anchor analyzer module 230 presents the anchors in the anchor list 240 on the user interface 180 of the viewer client device 110 and receives user input specifying an appropriate anchor for joining or rejoining the stream. In this example, the anchor analyzer module 230 provides the user with ultimate control over where to start viewing in the live stream.

[0070] The playback module 365 receives the live stream from the stream hosting server 150 to be played on the viewer client device 110 based on the identified anchor. The live stream may contain additional instructions in a metadata file on how the playback module 365 plays the live stream (e.g., decoder, video resolution, and audio bitrate). For example, the playback module 365 may receive the live stream and the associated audio file and may decode them separately before playing them on the user interface manager 350 of the viewer client device 110.

[0071] The playback module 365a also receives content items from the stream hosting server 150 to be played on the client device 110. The content items may be stored in the client content storage 375. In addition, the content items may have associated instructions (e.g., stored in a metadata file associated with the content items) that instruct the client device 110 to play the content item at the departure timestamp of the live stream. Thus, the client device 110 automatically turns to play the content item at the departure timestamp. The departure timestamp may be stored in the timestamp storage 370 and used for further analysis by the anchor analyzer module 360 ​​when the client device 110 is to rejoin the live stream.

[0072] Provide live streaming

[0073] Figure 4 An example flow diagram is depicted of a stream hosting server 150 providing a live stream to be played on a client device 110 based on a generated anchor according to one embodiment. The stream hosting server 150 receives 405 a live stream from a creator client device 108. The stream hosting server 150 receives 410 a request from the creator client device 108 to generate an anchor at a timestamp of the live stream.

[0074] While the live stream is ongoing, the stream hosting server 150 generates 415 anchors. As described above, each anchor is generated with a timestamp and a validity duration. The stream hosting server 150 receives 420 a request (e.g., a join request or a rejoin request) from a client device and determines 425 valid anchors based on the validity duration associated with each anchor. From the list of valid anchors, the stream hosting server 150 identifies 430 an anchor. For example, if the received request is a join request, the stream hosting server 150 identifies the anchor that specifies the most recent timestamp. If the received request is a rejoin request, the stream hosting server 150 identifies the anchor that specifies the timestamp that is closest to the leave timestamp. Once the stream hosting server 150 identifies the anchor, the stream hosting server 150 provides 435 the live stream to the viewer client device 110 to begin playback at the timestamp specified by the identified anchor.

[0075] Requesting and playing live streams

[0076] Figure 5 An example flow chart for playback of a received live stream based on a generated anchor by a viewer client device 110 according to one embodiment is depicted. The viewer client device 110 receives 505 a user input specifying a desire to access the live stream. For example, the user may provide a touch input through the user interface 180. The viewer client device 110 determines 510 the type of request sent on behalf of the user. For example, the viewer client device 110 determines whether the viewer client device 110 is newly accessing the live stream or returning to the live stream after a transfer. Thus, the request may be a join request or a rejoin request. The viewer client device sends 515 a request to access the live stream to the stream management server 150. If the viewer client device 110 is sending a rejoin request, the request also includes a leave timestamp associated with the viewer client device 110 that previously left the live stream.

[0077] The viewer client device 110 receives 520 the live stream and the identified anchor that specifies a timestamp of the live stream. If the viewer client device 110 newly joins the live stream, the identified anchor is the most recently generated anchor associated with the live stream. If the viewer client device 110 rejoins the live stream, the identified anchor specifies a timestamp that is closest to the departure time corresponding to when the viewer client device 110 left the live stream. The viewer client device 110 begins playing 525 the live stream for the user at the timestamp specified by the identified anchor.

[0078] Overview

[0079] The foregoing description of the embodiments of the present disclosure has been presented for illustrative purposes; it is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Those skilled in the relevant art will appreciate that many modifications and variations are possible in light of the above disclosure.

[0080] Some parts of this specification describe embodiments of the present disclosure in terms of algorithms and symbolic representations of operations on information. Technicians in the field of data processing generally use these algorithmic descriptions and representations to effectively convey the essence of their work to other technicians in the field. Although these operations are described functionally, computationally or logically, they should be understood to be implemented by computer programs or equivalent circuits or microcodes, etc. In addition, without loss of generality, it is sometimes convenient to refer to these operational arrangements as modules. The described operations and their associated modules can be embodied in software, firmware, hardware or any combination thereof. In one embodiment, a software module is implemented with a computer program product. The computer program product may include a computer-readable medium containing a computer program code, which may be executed by a computer processor for performing any or all of the steps, operations or processes described. Alternatively, the computer program product may be a signal encoding the computer program code and transmitted through a signal path (e.g., a radio path, an electrical signal path or an optical path) without being recorded on a non-temporary recording medium.

[0081] Embodiments of the present disclosure may also relate to an apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purpose, and / or it may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a non-transitory tangible computer-readable storage medium or in any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. In addition, any computing system mentioned in the specification may include a single processor, or may be an architecture that employs multiple processor designs to increase computing power.

[0082] Embodiments of the present disclosure may also relate to products produced by the computing process described herein. Such products may include information produced by the computing process, wherein the information is stored on a non-transitory tangible computer-readable storage medium, and may include any embodiment of a computer program product or other data combination described herein.

[0083] Finally, the language used in the specification is selected primarily for readability and instructional purposes, and may not be selected to describe or limit the subject matter of the present invention. Therefore, the scope of the present disclosure is not intended to be limited by this detailed description, but rather by any claims that issue on this application based thereon. Therefore, the disclosure of the embodiments of the present disclosure is intended to illustrate, but not to limit, the scope of the present disclosure, which is set forth in the appended claims.

Claims

1. A computer-implemented method comprising: receiving user input by an application on a viewer client device to access the live stream; The application on the viewer client device sends a request to access the live stream; presenting, by the application on the viewer client device, information that allows a user to indicate a first anchor of a plurality of anchors of the live stream, the plurality of anchors being created in response to user input associated with the live stream, and the first anchor corresponding to a timestamp for beginning a portion of the live stream; receiving user input identifying the live stream and user input indicating the timestamp corresponding to a first anchor of the live stream; and The live stream is displayed by the application on the viewer client device starting at the timestamp corresponding to a first anchor of the live stream.

2. The computer-implemented method of claim 1 , wherein: The sent request to access the live stream is one of a join request or a rejoin request based on whether the viewer client device previously accessed the live stream for a threshold amount of time.

3. The computer-implemented method of claim 1 , wherein: The sent request to access the live stream is a join request, and wherein the first anchor corresponds to a timestamp of the live stream that is closest to a timestamp associated with the sent join request to access the live stream.

4. The computer-implemented method of claim 1 , further comprising: receiving, on the viewer client device, a content item to be presented to a user; identifying an out-of-time interval for the live stream associated with the content item; In response to expiration of the away time interval, displaying the received content item in place of the live stream; and A departure timestamp of the live stream is stored, the departure timestamp corresponding to a time when the received content item was displayed in place of the live stream.

5. The computer-implemented method of claim 4, wherein: The request to access the live stream is a rejoin request and is sent in response to completion of displaying the received content item to the user.

6. The computer-implemented method of claim 4, wherein: The sent request to access the live stream is a rejoin request, and wherein the first anchor corresponds to a timestamp that is closest to the leave timestamp of the live stream.

7. The computer-implemented method of claim 1 , further comprising: Providing an indication that the live stream is currently in a semi-live state; as well as An optional indication is provided, which, when selected, converts the live stream in a semi-live state into a live segment of the live stream.

8. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a viewer client device, cause the processor to perform operations comprising: receiving, by an application on the viewer client device, user input to access a live stream; The application on the viewer client device sends a request to access the live stream; presenting, by the application on the viewer client device, information that allows a user to indicate a first anchor of a plurality of anchors of the live stream, the plurality of anchors being created in response to user input associated with the live stream, and the first anchor corresponding to a timestamp for beginning a portion of the live stream; receiving user input identifying the live stream and user input indicating the timestamp corresponding to a first anchor of the live stream; and The live stream is displayed by the application on the viewer client device starting at the timestamp corresponding to a first anchor of the live stream.

9. The non-transitory computer readable medium of claim 8, wherein: The sent request to access the live stream is one of a join request or a rejoin request based on whether the viewer client device previously accessed the live stream for a threshold amount of time.

10. The non-transitory computer readable medium of claim 8, wherein: The sent request to access the live stream is a join request, and wherein the first anchor corresponds to a timestamp of the live stream that is closest to a timestamp associated with the sent join request to access the live stream.

11. The non-transitory computer readable medium of claim 8, wherein: The operations also include: receiving, on the viewer client device, a content item to be presented to a user; identifying an out-of-time interval for the live stream associated with the content item; In response to expiration of the away time interval, displaying the received content item in place of the live stream; and A departure timestamp of the live stream is stored, the departure timestamp corresponding to a time when the received content item was displayed in place of the live stream.

12. The non-transitory computer readable medium of claim 11, wherein: The request to access the live stream is a rejoin request and is sent in response to completion of displaying the received content item to the user.

13. The non-transitory computer readable medium of claim 11, wherein: The sent request to access the live stream is a rejoin request, and wherein the first anchor corresponds to a timestamp that is closest to the leave timestamp of the live stream.

14. The non-transitory computer readable medium of claim 11, wherein: The operations also include: Providing an indication that the live stream is currently in a semi-live state; and An optional indication is provided, which, when selected, converts the live stream in a semi-live state into a live segment of the live stream.

15. A system comprising: Memory; as well as A processor coupled to the memory is configured to perform operations, the operations comprising: receiving user input by an application on a viewer client device to access the live stream; The application on the viewer client device sends a request to access the live stream; presenting, by the application on the viewer client device, information that allows a user to indicate a first anchor of a plurality of anchors of the live stream, the plurality of anchors being created in response to user input associated with the live stream, and the first anchor corresponding to a timestamp for beginning a portion of the live stream; receiving user input identifying the live stream and user input indicating the timestamp corresponding to a first anchor of the live stream; and The live stream is displayed by the application on the viewer client device starting at the timestamp corresponding to a first anchor of the live stream.

16. The system of claim 15, wherein: The sent request to access the live stream is one of a join request or a rejoin request based on whether the viewer client device previously accessed the live stream for a threshold amount of time.

17. The system of claim 15, wherein: The sent request to access the live stream is a join request, and wherein the first anchor corresponds to a timestamp of the live stream that is closest to a timestamp associated with the sent join request to access the live stream.

18. The system of claim 15, wherein: The operations also include: receiving, on the viewer client device, a content item to be presented to a user; identifying an out-of-time interval for the live stream associated with the content item; In response to expiration of the away time interval, displaying the received content item in place of the live stream; and A departure timestamp of the live stream is stored, the departure timestamp corresponding to a time when the received content item was displayed in place of the live stream.

19. The system of claim 18, wherein: The request to access the live stream is a rejoin request and is sent in response to completion of displaying the received content item to the user.

20. The system of claim 18, wherein: The sent request to access the live stream is a rejoin request, and wherein the first anchor corresponds to a timestamp closest to the leave timestamp of the live stream.

Images