Method, apparatus, device, medium and program product for managing data streams

Abstract

The invention provides a method, device and equipment for managing data streams, a medium and a program product. In one method, at a first device, a data stream associated with a second device is received. State information is determined based on the data stream, the state information indicating a state of the second device in a time period corresponding to the data stream. Data content corresponding to the data stream is presented at the first device based on the state information. In this way, the opportunity that the anchor initiates live broadcast pause can be matched with the opportunity that the audience side client presents the pause page, live broadcast pause and live broadcast recovery can be effectively and flexibly realized, and the presentation effect of live broadcast pause and live broadcast recovery can be ensured.

Description

Technical Field

[0001] Exemplary implementations of this disclosure generally relate to media item management, and more particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for managing data streams. Background Technology

[0002] More and more applications are now offering live streaming functionality. During a live stream, the streamer can interact and communicate with viewers in real time to attract more viewers and increase their interest. However, sometimes the streamer may pause the live stream for a period of time. In this case, viewers will receive a message indicating that the live stream is paused, or they may be shown an image indicating that the live stream is paused, such as an overlay on the live stream image.

[0003] Currently, Instant Messaging (IM) messages are commonly used to pause live streams. However, the IM messages received by the viewer's client may arrive faster than the live data stream, potentially causing a mismatch between the timing of the pause overlay appearing on the viewer's client and the timing of the broadcaster initiating the pause. This affects the presentation of paused and resumed live streams. Therefore, a more efficient and accurate method for pausing and resuming live streams is desired. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for managing data streams is provided. In this method, a data stream associated with a second device is received at a first device. Status information is determined based on the data stream, indicating the status of the second device during a time period corresponding to the data stream. Based on the status information, data content corresponding to the data stream is presented at the first device.

[0005] In a second aspect of this disclosure, an apparatus for managing data streams is provided. The apparatus includes: a receiving module configured to receive, at a first device, a data stream associated with a second device; a determining module configured to determine status information based on the data stream, the status information indicating the status of the second device during a time period corresponding to the data stream; and a presenting module configured to present data content corresponding to the data stream at the first device based on the status information.

[0006] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to a first aspect of this disclosure when executed by the at least one processing unit.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method according to a first aspect of this disclosure.

[0008] In a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the implementation of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] In the following detailed description, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent, taken in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A block diagram of an application environment according to an exemplary implementation of this disclosure is shown;

[0012] Figure 2 A block diagram illustrating a traditional approach for managing data flow is shown.

[0013] Figure 3 A block diagram illustrating some implementations of this disclosure for managing data flow is shown;

[0014] Figure 4 A flowchart is shown illustrating a process for managing data flow according to some implementations of this disclosure;

[0015] Figure 5 A block diagram is shown illustrating the timing process for handling pause messages in the event of a data interruption, according to some implementations of this disclosure.

[0016] Figure 6 A block diagram is shown illustrating the timing process for handling pause messages in the event of a data interruption, according to some implementations of this disclosure.

[0017] Figure 7 A block diagram illustrating the timing process for enabling IM messages according to some implementations of this disclosure is shown;

[0018] Figure 8 A flowchart is shown of a method for managing data flow according to some implementations of this disclosure;

[0019] Figure 9A block diagram of an apparatus for managing data flow according to some implementations of this disclosure is shown; and

[0020] Figure 10 A block diagram of a device capable of implementing various implementations of the present disclosure is shown. Detailed Implementation

[0021] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.

[0029] Example Environment

[0030] In a live streaming environment (e.g., a live streaming room), a streamer can initiate a live stream, and viewers can watch the live stream and interact with the streamer. See also Figure 1 Describes an application environment based on an example implementation of this disclosure. Figure 1 A block diagram of an application environment 100 according to an exemplary implementation of this disclosure is shown. Figure 1 As shown, environment 100 includes a live streaming application 110. The live streaming application 110 for viewers can be, for example, an application capable of providing audio and / or video live streaming functionality. For instance, the live streaming application 110 can provide live sessions, such as a streamer singing, a streamer initiating a game activity, or a streamer inviting some viewers to participate as guests in a live chat, etc.

[0031] In some cases, the broadcaster can pause the live stream by operating the live streaming application. At this time, the viewer's live stream screen 120 can be frozen, and a pause overlay 130 or other pause page can appear on the live stream screen 120. Alternatively and / or additionally, when the live stream is paused, the live stream screen 120 can be the last frame before the broadcaster left, the broadcaster's avatar, or a pause screen pre-set by the broadcaster. In another implementation, to prevent viewers from leaving when the broadcaster temporarily leaves, dynamic animations, music videos, or other videos can be played.

[0032] According to one example implementation of this disclosure, overlay 130 or other pause pages can be used to indicate that the live stream is paused. For example, overlay 130 can display text such as "Live stream is currently paused, come back soon" to indicate that the live stream is currently paused. It should be understood that overlay 130 or other pause pages can be any suitable type of window, panel, stand-alone interface, or specific display area within an interface for indicating that the live stream is paused.

[0033] In traditional solutions, pausing can be implemented using IM messages; that is, a message indicating pause can be sent via IM. However, in live streaming scenarios, the live data stream has a delay, and this delay does not match the delay of the IM message. For ease of description, the IM message delay can be called the first delay, and the live data stream delay can be called the second delay. Typically, the first delay can be less than the second delay. This can cause a discrepancy between the time the client receives the pause message sent via IM and the time it receives the live stream feed.

[0034] Figure 2 A block diagram 200 for managing data flow according to a traditional scheme is shown. (e.g.) Figure 2 As shown in the diagram, block diagram 200 includes the broadcaster's screen timestamp 210 and the viewer's screen timestamp 220. Assume the first delay of the IM message is less than 1 second or other duration, and the second delay of the live data stream is 4-5 seconds or other duration. This may cause the following problems during actual push / pull streaming.

[0035] 1) When the broadcaster pauses the live stream (211), the viewers will first receive a message about the pause, such as displaying a pause overlay 130 (221), but at this time the last valid data stream is lost (that is, the data that should have been displayed is not displayed). The length of the lost data stream is equal to the difference between the second delay and the first delay.

[0036] 2) When a broadcaster pauses their live stream for an extended period, various situations may occur that cause the live stream data stream to disconnect. In this case, to restart the live stream, it is necessary to reconfirm the broadcast permission and restart the stream. However, since the message about resuming the live stream is sent to the viewer's client immediately upon clicking the resume button (212), the viewer may see a blank stream for an extended period. In the case of no interruption, the length of the blank stream is equal to the difference between the second delay and the first delay. In the case of interruption, the length of the blank stream = (the time of reconfirming permission - the time of clicking resume) + (the second delay - the first delay).

[0037] 3) When the pause time of the broadcaster is short (e.g., 1-2 seconds or other duration), the viewer will see the live content immediately after the pause resumes, i.e., after the pause overlay 130 is hidden (222). However, due to the delay of the live data stream, the viewer will be shown a screen for 1-2 seconds or other duration (e.g., a Gaussian blurred frame of the avatar) after a few seconds.

[0038] 4) When the host pauses for a long time (such as more than 10 seconds or other durations), due to the delay in the live data stream, the viewer will see a fixed avatar frame for a period of time after the pause resumes (the pause overlay 130 appears and disappears) before the live content resumes.

[0039] As mentioned above, the live stream data presented to viewers does not match the pause and resume actions of the streamer, which may lead to visual clutter. Therefore, it is desirable to implement the pause and resume of the live stream data stream in a more efficient and accurate manner.

[0040] Overview of managing data flow

[0041] To at least partially address the shortcomings of the prior art, a method for managing data flows is proposed according to an exemplary implementation of this disclosure. See also Figure 3 This describes a summary of an exemplary implementation according to the present disclosure. Figure 3 A block diagram 300 for managing data flows is shown, according to some implementations of this disclosure. For example... Figure 3 As shown, at the first device (e.g., the client device of a viewer user and / or a guest user), a data stream 310 associated with the second device (e.g., the client device of a broadcaster user) can be received. Combined with... Figure 1 According to one example implementation of this disclosure, data stream 310 can be a live data stream in live streaming application 110. Live streaming application 110 can be, for example, an application capable of providing audio and / or video live streaming functionality. For example, live streaming application 110 can provide live sessions, such as a streamer singing, a streamer initiating a game activity, a streamer inviting some audience users or other streamers as guests to participate in a live chat, etc.

[0042] According to an example implementation of this disclosure, the live streaming application 110 may include a first user and a second user. The first user may be at least one of a viewer user and a guest user, and the second user may be a broadcaster user. A broadcaster user may be the user who initiates a live stream (also referred to as a live stream session) in the live streaming application 110. Correspondingly, a viewer user may be a user who watches the live stream. Guest users may include viewer users invited by the broadcaster user, or other broadcaster users invited by the broadcaster user. According to an example implementation of this disclosure, the first device may be a device associated with the first user, and the second device may be a device associated with the second user. In other words, the first device may be a viewing device, and the second device may be a live streaming device.

[0043] Furthermore, status information 320 can be determined based on data stream 310, and status information 320 can indicate the status of the second device during the time period corresponding to data stream 310. According to an example implementation of this disclosure, the status can indicate the status of the live streaming application 110 in the second device. Alternatively and / or additionally, the status of the live streaming application 110 in the second device can include at least a working status and a paused status. A working status can, for example, indicate that the broadcaster is currently live streaming. A paused status can, for example, indicate that the live stream is paused. A working status can, for example, correspond to the binary number 1 in status information 320. Conversely, a paused status can, for example, correspond to the binary number 0 in status information 320.

[0044] According to one example implementation of this disclosure, when determining status information 320 based on the data stream, the status information 320 can be extracted from supplementary enhancement information (i.e., SEI messages) in the data stream. In other words, supplementary enhancement information containing status information 320 can be inserted into the data stream. In another example implementation, a heartbeat message can also be inserted at a predetermined position in the data stream to indicate live / pause.

[0045] Furthermore, based on the status information 320, data content 330 corresponding to the data stream can be presented on the first device. Alternatively and / or additionally, the data content 230 may include a data stream 310' and a pause page 340. The data stream 310' may be a live data stream corresponding to the working state within the data stream 310. Alternatively and / or additionally, the data stream 310' and the pause page 340 may not be presented simultaneously. In other words, the data stream 310' can be presented on the first device when the live stream is in progress, and the pause page 340 can be presented when the live stream is paused.

[0046] Alternatively and / or additionally, the pause page 340 can be any suitable type of window, panel, stand-alone interface, or specific display area within an interface used to indicate a pause in the data stream. The pause page 340 may also include... Figure 1 The pause overlay 130 in the middle. Alternatively and / or additionally, at least one identifier, such as a text identifier, an image identifier, an audio identifier, etc., may be displayed on the pause page 340 to indicate that the data stream is paused.

[0047] By utilizing the exemplary implementation method disclosed herein, the timing of the broadcaster initiating a live stream pause can be matched with the timing of the pause page being displayed on the viewer's client, avoiding misalignment in the presentation timing of the pause page. This enables effective and flexible implementation of live stream pause and resumption, ensuring the presentation effect of live stream pause and resumption.

[0048] Detailed process of managing data flow

[0049] The foregoing has described an example implementation of this disclosure; further details regarding the management of data streams will be described below. According to an example implementation of this disclosure, live streaming events in live streaming application 110 may include live singing, live lecturing, live online gaming, etc., by the streamer, or by the streamer inviting viewers or other streamers as guests to participate in interactive events, such as singing competitions, gaming competitions, and live chat sessions. In this way, richer and more diverse live streaming content can be provided in live streaming application 110. In such live streams, the live streaming data stream needs to be managed to ensure interactivity.

[0050] According to an example implementation of this disclosure, when data content 330 is presented at the first device, in response to determining status information 320 indicating that the second device is in an operational state during the time period corresponding to the data stream, data stream 310' can be presented at the first device. (See also...) Figure 3 For example, when the status information 320 in the SEI message is a binary number 1, it indicates that the broadcaster's device (i.e., the second device) is in a working state during the corresponding time period. The working state indicates that the broadcaster is currently live streaming. At this time, the live data stream 310' can be displayed on the viewer's device (i.e., the first device), meaning the viewer can watch the live stream normally.

[0051] Figure 4 A flowchart of a process 400 for managing a data stream according to some implementations of this disclosure is shown. In process 400, after a broadcaster starts a live stream (410), the broadcaster continuously sends a data stream 310, which is equivalent to continuously sending SEI messages (415) because the data stream contains SEI messages. Then, if the broadcaster pauses the live stream (420), content indicating a paused state (425) can be written to the status information 320 in the SEI message.

[0052] According to an example implementation of this disclosure, in response to determining that status information 320 indicates that the second device is in a paused state during the time period corresponding to the data stream, a pause page 340 can be presented on the first device to indicate that the data stream is paused. (Return to Reference) Figure 3 For example, when the status information 320 in the SEI message is a binary number 0, it indicates that the broadcaster's device (i.e., the second device) is in a paused state during the corresponding time period. The paused state indicates that the broadcaster has paused the current live stream. At this time, a pause page 340 can be displayed on the viewer's device (i.e., the first device) to inform the viewer that the live stream is currently paused.

[0053] Continue to refer to Figure 4 When the live stream pause ends (430), indicating the resumption of the live stream, the status information 320 indicating the resumption status can be written into the SEI message. As a result, as the data stream 310 containing the SEI message is transmitted to the viewer's client (i.e., the first device), the pause page 340 in the live stream application 110 of that client is removed. The viewer can then continue watching the live stream. When the streamer closes the live stream (440), the live stream data stream 310 is no longer transmitted to the viewer's client, and correspondingly, the sending of SEI messages to that client stops (445).

[0054] By using the exemplary implementation of this disclosure, by inserting live / pause status into the data stream, the live / pause status arrives along with the data stream, accurately conveying the corresponding messages of live pause and live resume to the viewer-side client, thereby improving the presentation effect of live pause and live resume on the viewer-side client.

[0055] According to an example implementation of this disclosure, status information 320 can be inserted into data stream 310 at a predetermined frequency, and when determining status information 320, status information 320 can be extracted at a predetermined frequency. For example... Figure 4 As shown, when continuously sending SEI messages (415), they can be sent at a predetermined frequency. In other words, SEI messages can be inserted into data stream 310 at a predetermined frequency. Correspondingly, status information 320, which can be a binary number 1 or 0, can be written into the SEI messages of data stream 310 at a predetermined frequency. Correspondingly, status information 320 can be extracted at a predetermined frequency to determine the working state or paused state indicated by status information 320.

[0056] It should be noted that the predetermined frequency can be relatively high to avoid SEI messages being lost due to network issues. For example, the predetermined frequency could be approximately once per second or other frequencies to significantly reduce the probability of SEI messages being lost and to accommodate the perceived time accuracy of live / pause for viewers.

[0057] Figure 5 A block diagram of a timing process 500 for handling pause messages in the event of a data interruption, according to some implementations of this disclosure, is shown. In timing process 500, a broadcast-side timeline 510 and a viewer-side timeline 520 are shown respectively. Due to network issues or other reasons, the viewer-side client may experience a delay in receiving data stream 310. For example, assuming the broadcast-side timeline starts live streaming at time T0, when the live streaming data stream is transmitted to the viewer-side client, the start time for the viewer is T1. Then, the viewer-side client continuously receives live streaming data stream 310-1 (which is part of data stream 310). Subsequently, due to network issues or other reasons, the viewer-side client may experience a data interruption 525 (i.e., the data stream is interrupted between T3 and T5) during the data stream reception process. In this case, four different specific scenarios can exist, which will be discussed in detail below.

[0058] The first scenario involves a pause at time 530 on the broadcaster's side. For the viewer, the pause begins at time T2 (T2 is during data stream 310-1) and ends at time T4 (T4 is during a stream interruption at time 525). In this case, at time T2, the viewer's client can learn of the pause initiation from the SEI message in data stream 310-1 and subsequently display the pause page 340 in the viewer's live streaming application 110. During the stream interruption at time 525, the viewer's client cannot receive the data stream and therefore cannot receive the SEI message indicating the end of pause 530. Thus, from T2 to T5, the pause page 340 may remain displayed in the viewer's live streaming application 110. After T5, the viewer's client can continue receiving data stream 310-2 (which is part of data stream 310). By inserting SEI messages into the data stream at a predetermined frequency as described in this disclosure, the data stream matching the live / paused state can be accurately presented in this first scenario.

[0059] The second scenario involves the broadcaster initiating a pause (540). For the viewer, the pause begins at time T2 (T2 is in data stream 310-1 stage) and ends at time T6 (T6 is in data stream 310-2 stage). In this case, at time T2, the viewer's client can detect the pause initiation from the broadcaster via the SEI message in data stream 310-1, and the pause page 340 will then appear in the live streaming application 110 on the viewer's client. During the subsequent interruption (525), although the viewer's client cannot receive the data stream, it is unaffected by the interruption because the live stream is paused (540) during this period. Later, in data stream 310-2 stage, at time T6, the viewer's client can detect the end of the pause (540) via the SEI message in data stream 310-2, and can then cancel the pause page 340 and resume watching the live stream. By inserting SEI messages into the data stream at a predetermined frequency in the implementation of this disclosure, the data stream matching the live / paused state can be accurately presented in this second case.

[0060] Figure 6 A block diagram of a timing process 600 for handling pause messages in the event of a data interruption, according to some implementations of this disclosure, is shown. The broadcast-side timeline 510 and the viewer-side timeline 520 in timing process 600 are consistent with those in timing process 500. Continuing from the above discussion, the scenario of a broadcast-side pause occurring in the event of a data interruption 525 (i.e., a data stream interruption during T3 to T5) is presented.

[0061] The third scenario involves the broadcaster initiating pause 610. From the viewer's perspective, pause 610 begins at T7 (which is during the interruption period 525) and ends at T8 (also during the interruption period 525). In this case, because the entire pause 610 phase occurs during the interruption period 525, the viewer's client cannot receive the SEI message and therefore cannot know when the broadcaster initiated or ended pause 610. In this situation, IM messages can be used to activate the pause page 340.

[0062] According to an example implementation of this disclosure, the second device can send a notification message when switching live streaming states, and the first device can receive the notification message (i.e., an IM message) from the second device. The notification message can indicate the current state of the second device. In other words, a notification message can be sent to the viewer-side client (i.e., the first device) to inform the broadcaster about pausing or resuming the live stream. The IM message can be sent independently of data stream 310. Therefore, regardless of whether the live streaming data stream is interrupted, an IM message indicating the current state can be sent whenever the broadcaster's live streaming state is switched to paused or resumed. The IM message can contain, for example, "on" (e.g., represented by 1) or "off" (e.g., represented by 0). "Off" can indicate that the broadcaster-side device (i.e., the second device) has initiated a live streaming pause, while "on" can indicate the end of the pause, i.e., the resumption of the live stream.

[0063] According to an example implementation of this disclosure, in response to a failure to extract status information 320 at a predetermined frequency, data content corresponding to the data stream can be presented on the first device based on the current status. According to an example implementation of this disclosure, when presenting data content based on the current status, in response to determining that the current status is a paused state, a pause page 340 can be presented on the first device to indicate that the data stream is paused. For example, when the IM message contains "on", the live streaming application 110 on the viewer's client displays the live content after the resumption of live streaming. When the IM message contains "off", the live streaming application 110 on the viewer's client displays the pause page 340. In this third case, the data stream matching the live / paused state can be accurately presented.

[0064] The fourth scenario involves a pause at time 620 on the broadcaster's side. For the viewer, the pause begins at time T7 (during the interruption at time 525) and ends at time T9 (during data stream phase 310-2). In this case, because the pause begins during interruption at time 525, the viewer's client cannot receive the SEI message and therefore cannot know that the broadcaster initiated the pause. In this situation, the pause page 340 can be initiated using the IM message method discussed above. Subsequently, during data stream phase 310-2, the viewer's client can learn from the SEI message in data stream 310-2 at time T9 that the broadcaster has ended the pause at time 620, and can then cancel the pause page 340 and resume watching the live stream. By inserting SEI messages into the data stream at a predetermined frequency and combining them with IM messages, the data stream matching the live / paused state can be accurately presented in this fourth scenario.

[0065] Using the exemplary implementation of this disclosure, in cases where the data stream is interrupted, IM messages can be used to assist in determining whether the pause page 340 should be displayed on the viewer's client. In cases where the stream is not interrupted, and in other cases where the data stream is interrupted, the implementation method of this disclosure, which inserts SEI messages into the data stream at a predetermined frequency, can resolve the mismatch between the timing of the appearance or disappearance of the pause overlay on the viewer's client and the timing of the broadcaster initiating a pause or resumption of the live stream. Therefore, this implementation method effectively and flexibly implements live stream pause and resumption.

[0066] Figure 7 A block diagram of a timing process 700 for enabling IM messages according to some implementations of this disclosure is shown. The broadcast-side timeline 510 and the viewer-side timeline 520 in timing process 700 are also consistent with those in timing process 500. Timing process 700 also includes a viewer-side timeline 520-1 corresponding to the viewer-side timeline 520 regarding whether SEI messages can be received, and a corresponding viewer-side timeline 520-2 regarding when and how IM messages are enabled. In viewer-side timeline 520 and the corresponding viewer-side timeline 520-1, when the viewer-side client can receive data stream 310-1, it can continuously receive SEI messages (710-1). When the stream is interrupted (525), it cannot receive SEI messages (725). Subsequently, when the viewer-side client receives data stream 310-2 again, it can continuously receive SEI messages (710-2).

[0067] As discussed above, the viewer's client does not need to use IM messages when it can receive SEI messages. In the event of a 525 interruption, if the first and second scenarios mentioned above occur, SEI messages can be used instead of IM messages. However, in the third and fourth scenarios mentioned above, IM messages can be enabled to assist in determining when to pause and resume the live stream on the broadcaster's side.

[0068] Specifically, a timer can be used to periodically check whether an SEI message has been received at a predetermined frequency. Assuming the predetermined frequency is once per second, at time T3, after one second (at time T10), the timer checks whether an SEI message has been received and determines that no SEI message has been received. For example, each time an IM message is received by the viewer's client, these IM messages can be cached. Caching can be done in one way: all messages can be cached. To save cache space, only the most recently received IM messages can be cached sequentially; that is, each time a new IM message is received, the currently first-ranked IM message can be deleted. Alternatively, only the latest IM message can be cached; that is, each time a new IM message is received, the previously received IM message is overwritten. When it is determined at time T10 that no SEI message has been received, the latest cached IM message can be synchronized to the viewer's client. Thus, the pause page 340 can be presented on the viewer's client based on this latest IM message.

[0069] By utilizing the exemplary implementation of this disclosure, a timer is used to check whether an SEI message has been received at the same frequency as a predetermined frequency. Combined with the caching method of IM messages, the timing and specific objects of IM message usage can be determined, which is beneficial for achieving effective pausing during live streaming.

[0070] According to an example implementation of this disclosure, in response to determining that the received data stream has failed, candidate data content corresponding to the second device can be presented at the first device. Specifically, in the event of a live data stream interruption, candidate media items can be presented. For example, 1) a Gaussian blurred image of the broadcaster; 2) a Gaussian blurred image of the last frame before the broadcaster paused the live stream; 3) an image indicating "downloading," such as a spinning loading screen, etc.

[0071] By utilizing the exemplary implementation of this disclosure, the corresponding live data stream can be presented to the audience in a more accurate manner according to the actual live / paused status, which helps the audience to know the current live data status and increases the accuracy of the data stream presentation.

[0072] Example process

[0073] Figure 8 A flowchart of a method 800 for managing a data stream according to some implementations of this disclosure is shown. At block 810, a data stream associated with a second device is received at a first device. At block 820, status information is determined based on the data stream, indicating the status of the second device during a time period corresponding to the data stream. At block 830, based on the status information, data content corresponding to the data stream is presented at the first device.

[0074] According to an example implementation of this disclosure, presenting data content at the first device includes at least one of the following: presenting the data stream at the first device in response to determining that the status information indicates that the second device is in an operating state during the time period corresponding to the data stream; and presenting a pause page at the first device to indicate that the data stream is paused in response to determining that the status information indicates that the second device is in a paused state during the time period corresponding to the data stream.

[0075] According to an example implementation of this disclosure, status information is inserted into the data stream at a predetermined frequency, and determining the status information includes: extracting status information at a predetermined frequency.

[0076] According to an example implementation of this disclosure, method 800 further includes: receiving a notification message from a second device, the notification message indicating the current state of the second device.

[0077] According to an example implementation of this disclosure, method 800 further includes: in response to determining that a failure has occurred in extracting state information at a predetermined frequency, presenting data content corresponding to the data stream at a first device based on the current state.

[0078] According to an example implementation of this disclosure, presenting data content based on the current state includes: in response to determining that the current state is a paused state, presenting a pause page on a first device to indicate that the data stream is paused.

[0079] According to an example implementation of this disclosure, method 800 further includes: in response to determining that receiving a data stream has failed, presenting candidate data content corresponding to the second device at the first device.

[0080] According to one example implementation of this disclosure, the data stream is a live data stream in a live streaming application, and determining state information based on the data stream includes: extracting state information from supplementary enhancement information of the data stream.

[0081] According to one example implementation of this disclosure, the live streaming application includes a first user and a second user, the first user being at least one of a viewer user and a guest user, the second user being a broadcaster user, a first device being a device associated with the first user, a second device being a device associated with the second user, and a status indicator indicating the status of the live streaming application on the second device.

[0082] Example devices and equipment

[0083] Figure 9A block diagram of an apparatus 900 for managing data streams according to some implementations of the present disclosure is shown. The apparatus 900 includes: a receiving module 99 configured to receive, at a first device, a data stream associated with a second device; a determining module 920 configured to determine status information based on the data stream, the status information indicating the status of the second device during a time period corresponding to the data stream; and a presenting module 930 configured to present data content corresponding to the data stream at the first device based on the status information.

[0084] According to an example implementation of this disclosure, the presentation module 930 is further configured to present data content at the first device based on at least one of the following: in response to determining that a status information indicates that the second device is in an active state during a time period corresponding to the data stream, presenting the data stream at the first device; and in response to determining that a status information indicates that the second device is in a paused state during a time period corresponding to the data stream, presenting a pause page at the first device to indicate that the data stream is paused.

[0085] According to one example implementation of this disclosure, status information is inserted into the data stream at a predetermined frequency, and the determination module 920 is further configured to extract the status information at the predetermined frequency.

[0086] According to one example implementation of this disclosure, the device 900 further includes a notification message receiving module configured to receive a notification message from a second device, the notification message indicating the current state of the second device.

[0087] According to one example implementation of this disclosure, the apparatus 900 further includes an extraction failure module configured to, in response to determining that an extraction of status information at a predetermined frequency has failed, present data content corresponding to the data stream at a first device based on the current status.

[0088] According to one example implementation of this disclosure, the extraction failure module is further configured to, in response to determining that the current state is paused, present a pause page at the first device to indicate that the data stream is paused.

[0089] According to one example implementation of this disclosure, the apparatus 900 further includes a reception failure module configured to present candidate data content corresponding to the second device at the first device in response to determining that a reception data stream has failed.

[0090] According to one example implementation of this disclosure, the data stream is a live data stream in a live streaming application, and the determination module 920 is further configured to extract status information from supplementary enhancement information of the data stream.

[0091] According to one example implementation of this disclosure, the live streaming application includes a first user and a second user, the first user being at least one of a viewer user and a guest user, the second user being a broadcaster user, a first device being a device associated with the first user, a second device being a device associated with the second user, and a status indicator indicating the status of the live streaming application on the second device.

[0092] Figure 10 A block diagram of a device 1000 capable of implementing various implementations of the present disclosure is shown. It should be understood that... Figure 10 The computing device 1000 shown is merely exemplary and should not be construed as limiting the functionality and scope of the implementation described herein. Figure 10 The computing device 1000 shown can be used to implement the method described above.

[0093] like Figure 10 As shown, computing device 1000 is in the form of a general-purpose computing device. Components of computing device 1000 may include, but are not limited to, one or more processors or processing units 1010, memory 1020, storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. Processing unit 1010 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1020. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 1000.

[0094] Computing device 1000 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 1000, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1020 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1030 can be removable or non-removable media and may include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within computing device 1000.

[0095] The computing device 1000 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 10As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1020 may include computer program product 1025 having one or more program modules configured to perform various methods or actions of various implementations of this disclosure.

[0096] The communication unit 1040 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 1000 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 1000 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0097] Input device 1050 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1060 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 1000 can also communicate with one or more external devices (not shown) via communication unit 1040 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with computing device 1000, or with any device (e.g., network card, modem, etc.) that enables computing device 1000 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).

[0098] According to exemplary implementations of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to exemplary implementations of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above. According to exemplary implementations of this disclosure, a computer program product is provided that stores a computer program thereon, which, when executed by a processor, implements the methods described above.

[0099] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0101] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0103] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for managing data streams, comprising: At the first device, a data stream associated with the second device is received; Status information is determined based on the data stream, and the status information indicates the status of the second device during the time period corresponding to the data stream; as well as Based on the status information, data content corresponding to the data stream is presented at the first device.

2. The method of claim 1, wherein presenting the data content at the first device comprises at least one of the following: In response to determining that the status information indicates that the second device is in an operational state during a time period corresponding to the data stream, the data stream is presented at the first device; and In response to determining that the status information indicates that the second device is in a paused state during the time period corresponding to the data stream, a pause page is presented on the first device to indicate that the data stream is paused.

3. The method according to claim 1, wherein the status information is inserted into the data stream at a predetermined frequency, and determining the status information includes: The status information is extracted at the predetermined frequency.

4. The method of claim 3, further comprising: Receive a notification message from the second device, the notification message indicating the current status of the second device.

5. The method of claim 4, further comprising: In response to a determination that extraction of status information at the predetermined frequency has failed, data content corresponding to the data stream is presented at the first device based on the current status.

6. The method of claim 5, wherein presenting the data content based on the current state includes: In response to determining that the current state is a paused state, a pause page is presented on the first device to indicate that the data stream is paused.

7. The method of claim 1, further comprising: In response to determining that receiving the data stream has failed, candidate data content corresponding to the second device is presented at the first device.

8. The method according to claim 1, wherein the data stream is a live data stream in a live streaming application, and determining the status information based on the data stream includes: The state information is extracted from the supplemental enhancement information of the data stream.

9. The method of claim 8, wherein the live streaming application includes a first user and a second user, the first user being at least one of a viewer user and a guest user, the second user being a broadcaster user, the first device being a device associated with the first user, the second device being a device associated with the second user, and the status indicating the status of the live streaming application in the second device.

10. An apparatus for managing data streams, comprising: The receiving module is configured to receive, at the first device, a data stream associated with the second device; A determination module is configured to determine status information based on the data stream, the status information indicating the status of the second device during a time period corresponding to the data stream; as well as The presentation module is configured to present data content corresponding to the data stream at the first device based on the status information.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program causing the processor to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images