Method and apparatus for managing data stream, device, medium, and program product

Abstract

The present disclosure provides a method and apparatus for managing a data stream, a device, a medium, and a program product. In one method, at a first device, a data stream associated with a second device is received; state information is determined on the basis of the data stream, the state information indicating the state of the second device in a time period corresponding to the data stream; and on the basis of the state information, data content corresponding to the data stream is presented at the first device.

Description

Method, apparatus, device, medium and program product for managing data stream

[0001] This application claims priority to the Chinese Patent Application No. 202411274230.7, filed on September 11, 2024, entitled “Method, apparatus, device, medium and program product for managing data stream”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Exemplary implementations of the present disclosure generally relate to media item management, and in particular, to a method, apparatus, device, computer-readable storage medium, and computer program product for managing data stream. BACKGROUND

[0003] More and more applications are capable of providing live streaming function. During live streaming, a host user can interact with a viewer user in real time so as to attract more viewer users and improve the viewer’s interest. However, sometimes the host user can pause for a period of time during live streaming, at which time the viewer user will receive a message that the current live streaming is paused, or an image about the live streaming pause can be displayed, for example, a mask layer is superimposed on the live streaming image.

[0004] Currently, an IM (Instant Messaging) message is commonly used to implement the live streaming pause. However, the IM message received by the viewer side client can be faster than the live streaming data stream, so it can cause the timing of the pause mask layer appearing on the viewer side client to be mismatched with the timing of the host initiating the live streaming pause, thereby affecting the presentation effect of the live streaming pause and live streaming recovery. Therefore, it is desirable to implement the live streaming pause and live streaming recovery in a more effective and accurate manner. SUMMARY

[0005] In a first aspect of the present disclosure, a method for managing data stream is provided. In the method, a data stream associated with a second device is received at a first device. State information is determined based on the data stream, the state information indicating a state of the second device at a time period corresponding to the data stream. Data content corresponding to the data stream is presented at the first device based on the state information.

[0006] In a second aspect of the present disclosure, an apparatus for managing data stream is provided. The apparatus comprises a receiving module configured to receive, at a first device, a data stream associated with a second device. A determining module configured to determine, based on the data stream, state information indicating a state of the second device at a time period corresponding to the data stream. A presenting module configured to present, at the first device, data content corresponding to the data stream based on the state information.

[0007] In a third aspect of the disclosure, an electronic device is provided. The electronic device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, cause the electronic device to perform the method according to the first aspect of the disclosure.

[0008] In a fourth aspect of the disclosure, a computer-readable storage medium is provided, having computer-executable instructions stored thereon, the computer-executable instructions, when executed by a processor, cause the processor to implement the method according to the first aspect of the disclosure.

[0009] In a fifth aspect of the disclosure, a computer program product is provided, comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to the first aspect of the disclosure.

[0010] It is to be understood that the details set forth herein are not intended to limit the key or critical features of the implementations of the disclosure, nor are they intended to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other features, advantages and aspects of the implementations of the disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings. In the drawings, the same or like reference numerals designate the same or like elements, in which:

[0012] FIG. 1 shows a block diagram of an application environment according to one example implementation of the disclosure;

[0013] FIG. 2 shows a block diagram for managing data flow according to a conventional scheme;

[0014] FIG. 3 shows a block diagram for managing data flow according to some implementations of the disclosure;

[0015] FIG. 4 shows a flowchart of a process for managing data flow according to some implementations of the disclosure;

[0016] FIG. 5 shows a block diagram of a timing process for handling a pause message in case of flow break according to some implementations of the disclosure;

[0017] FIG. 6 shows a block diagram of a timing process for handling a pause message in case of flow break according to some implementations of the disclosure;

[0018] FIG. 7 shows a block diagram of a timing process for enabling an IM message according to some implementations of the disclosure;

[0019] FIG. 8 illustrates a flowchart of a method for managing data flow according to some implementations of the present disclosure;

[0020] FIG. 9 illustrates a block diagram of an apparatus for managing data flow according to some implementations of the present disclosure; and

[0021] FIG. 10 illustrates a block diagram of a device capable of implementing a number of implementations of the present disclosure. DETAILED DESCRIPTION

[0022] Implementations of the present disclosure will be described in detail with reference to the drawings, whereupon the implementations of the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the implementations set forth herein, but rather, the implementations are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and implementations of the present disclosure are merely for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0023] In the description of implementations of the present disclosure, the term “comprising” and similar terms are to be interpreted as open-ended, i.e., “including but not limited to”. The term “based on” is to be interpreted as “based, at least in part, on”. The term “one implementation” or “the implementation” is to be interpreted as “at least one implementation”. The term “some implementations” is to be interpreted as “at least some implementations”. Other explicit and implicit definitions can also be included below. As used herein, the term “model” can represent the association between various data. For example, the above-mentioned association can be obtained based on various technical solutions known at present and / or to be developed in the future.

[0024] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0025] It can be understood that before using the technical solutions disclosed in each embodiment of the present disclosure, the type of personal information involved in the present disclosure, the scope of use, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to the relevant laws and regulations.

[0026] For example, when responding to the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require the acquisition and use of the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the electronic device, application program, server or storage medium, etc. software or hardware that performs the operation of the technical solutions of the present disclosure according to the prompt information.

[0027] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user can be, for example, a pop-up window manner, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0028] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0029] The term "in response to" used herein indicates the state in which the corresponding event occurs or the condition is met. It will be understood that the execution time of the subsequent action performed in response to the event or condition is not necessarily strongly associated with the time when the event occurs or the condition is met. For example, in some cases, the subsequent action can be performed immediately when the event occurs or the condition is met; in other cases, the subsequent action can be performed after a period of time after the event occurs or the condition is met.

[0030] Example Environment

[0031] In a live streaming environment (e.g., a live streaming room), a host user can initiate a live streaming, and a viewer user can watch the live streaming and interact with the host user in the live streaming. Referring to FIG. 1 describing an application environment according to one example implementation of the present disclosure, the figure 1 shows a block diagram of an application environment 100 according to one example implementation of the present disclosure. As shown in FIG. 1, the environment 100 includes a live streaming application 110. The live streaming application 110 of the viewer user can be, for example, an application capable of providing audio and / or video live streaming functions. For example, the live streaming application 110 can provide a live streaming session, such as the host user singing, the host user initiating a game activity, the host user inviting part of the viewer users as guests to participate in a live streaming, and the like.

[0032] In some cases, the host user can pause the live streaming by operating the live streaming application. At this time, the live streaming screen 120 of the viewer user can present a static state, and a mask 130 or other pause page about the pause of the live streaming can appear on the live streaming screen 120. Alternatively and / or additionally, when the live streaming is paused, the live streaming screen 120 can be the last frame of the screen before the host user leaves, can be the avatar of the host user, and can also be the pause screen set by the host user in advance. In another implementation, in order to avoid the viewer user from leaving when the host user temporarily leaves, a dynamic animation, a music video, or the like can be played.

[0033] According to one example implementation of the present disclosure, the layer 130 or other pause page can be used to identify the live pause. For example, the layer 130 can display a message such as "Current live pause, back soon" or the like to indicate the current live pause. It should be understood that the layer 130 or other pause page can be any suitable type of window, panel, standalone interface, or specific display area in an interface, or the like, for identifying the live pause.

[0034] In a conventional solution, the pause can be implemented based on an IM message, i.e., the message indicating the pause can be sent through an IM message. However, in the context of live streaming, the live data stream can have a delay, and the delay does not match the delay of the IM message. For ease of description, the delay of the IM message can be referred to as a first delay, and the delay of the live data stream can be referred to as a second delay. Typically, the first delay can be less than the second delay. This can cause the time at which the client receives the pause message sent through the IM message to be inconsistent with the time at which the live screen 120 is received.

[0035] FIG. 2 illustrates a block diagram 200 for managing a data stream according to a conventional solution. As shown in FIG. 2, the block diagram 200 includes a host screen timestamp 210 and a viewer screen timestamp 220. Assume that the first delay of the IM message is less than 1 second or other time length, and the second delay of the live data stream is 4-5 seconds or other time length. This can cause the following problems in actual push-pull streaming.

[0036] 1) When the host user pauses the live streaming (211), the viewer user can first receive the message about the pause, e.g., display the pause layer 130 (221), but at this time the last piece of valid data stream is lost (i.e., the data that should have been displayed is not displayed). The length of the lost data stream is equal to the difference between the second delay and the first delay.

[0037] 2) In the case of a long pause of the live streaming by the host user, various situations can occur to cause the live data stream to be disconnected. At this time, to start the live streaming again, the permission to start the live streaming needs to be reconfirmed and the live streaming needs to be pushed again. However, since the message about the resumption of the live streaming is sent to the viewer user end immediately upon clicking the resume (212), the viewer user can see a blank stream for a long time. In the case of no disconnection, the length of the blank stream is equal to the difference between the second delay and the first delay. In the case of disconnection, the length of the blank stream = (the time of reconfirming the permission - the time of clicking the resume) + (the second delay - the first delay).

[0038] 3) When the anchor user's pause time is short (e.g., the pause time is 1-2 seconds or other time length), the viewer user side sees the live content immediately after seeing the pause resume, i.e., after the pause mask 130 is hidden (222). However, in fact, due to the delay of the live data stream, the viewer user side will continue to be displayed with a picture (e.g., an avatar Gaussian blur frame) for receiving the live data amount for 1-2 seconds or other time length after a few seconds.

[0039] 4) When the anchor user's pause time is long (e.g., 10 seconds or more or other time length), due to the delay of the live data stream, the viewer user side will see a fixed avatar picture frame for a period of time before the live content is restored after seeing the pause resume (the pause mask 130 appears and disappears).

[0040] As described above, the live data presented at the viewer user does not match the pause and resume actions of the anchor user, which can cause confusion in the picture. At this time, it is desirable to implement the pause and live resume of the live data stream in a more effective and accurate manner.

[0041] Summary of managing data stream

[0042] To at least partially address the deficiencies in the prior art, according to one example implementation of the present disclosure, a method for managing a data stream is proposed. A summary of one example implementation of the present disclosure is described with reference to FIG. 3, which shows a block diagram 300 for managing a data stream according to some implementations of the present disclosure. As shown in FIG. 3, at a first device (e.g., a client device of a viewer user and / or a guest user), a data stream 310 associated with a second device (e.g., a client device of an anchor user) can be received. In conjunction with FIG. 1, according to one example implementation of the present disclosure, the data stream 310 can be a live data stream in a live application 110. The live application 110 may, for example, be an application capable of providing audio and / or video live functions. For example, the live application 110 can provide a live session, such as the anchor user singing, the anchor user initiating a game activity, the anchor user inviting part of the viewer users or other anchor users as guests to participate in a live activity, and the like.

[0043] According to one example implementation of the present disclosure, the live streaming application 110 can include a first user and a second user, the first user can be at least one of a spectator user and a guest user, and the second user can be a host user. The host user can be a user who initiates a live streaming (may also be referred to as a live streaming session) in the live streaming application 110. Accordingly, the spectator user can be a user who watches the live streaming. The guest user can include a spectator user invited by the host user, and can also include another host user invited by the host user. According to one example implementation of the present disclosure, the first device can be a device associated with the first user, and the second device can be a device associated with the second user. In other words, the first device can be a spectator-side device, and the second device can be a host-side device.

[0044] Further, the state information 320 can be determined based on the data stream 310, and the state information 320 can indicate a state of the second device in a time period corresponding to the data stream 310. According to one example implementation of the present disclosure, the state can indicate a state of the live streaming application 110 in the second device. Alternatively and / or additionally, the state of the live streaming application 110 in the second device can include at least an active state and a paused state. The active state may, for example, represent a state in which the host user is live streaming. The paused state may, for example, represent a state in which the live streaming is paused. The active state may, for example, correspond to a binary number 1 in the state information 320. Conversely, the paused state may, for example, correspond to a binary number 0 in the state information 320.

[0045] According to one example implementation of the present disclosure, when the state information 320 is determined based on the data stream, the state information 320 can be extracted from a supplemental enhancement information (i.e., SEI message) of the data stream. In other words, the supplemental enhancement information containing the state information 320 can be inserted in the data stream. In another example implementation, a heartbeat message can also be inserted at a predetermined position in the data stream in order to indicate the live streaming / paused.

[0046] Further, based on the state information 320, a data content 330 corresponding to the data stream can be presented at the first device. Alternatively and / or additionally, the data content 230 can include a data stream 310’ and a paused page 340. The data stream 310’ can be a data stream of the live streaming corresponding to the active state in the data stream 310. Alternatively and / or additionally, the data stream 310’ and the paused page 340 can not be presented at the same time. In other words, the data stream 310’ can be presented at the first device when the live streaming is proceeding normally, and the paused page 340 can be presented when the live streaming is paused.

[0047] Alternatively and / or additionally, the pause page 340 can be any suitable type of window, panel, standalone interface, or specific display area in an interface, etc. for representing the data stream pause. The pause page 340 can also include the pause curtain 130 in FIG. 1. Alternatively and / or additionally, at least one indication, such as a text indication, an image indication, an audio indication, etc. can be presented on the pause page 340 to represent the data stream pause.

[0048] With the example implementation of the present disclosure, the timing of the host initiating the live streaming pause can be matched with the timing of the client side presenting the pause page, avoiding the timing of presenting the pause page being disorderly. In this way, the live streaming pause and live streaming resume can be effectively and flexibly implemented, which is beneficial to guarantee the presentation effect of the live streaming pause and live streaming resume.

[0049] Detailed process of managing data stream

[0050] The above has described the live streaming event in the live streaming application 110 according to one example implementation of the present disclosure. In the following, more details about managing the data stream will be described. According to one example implementation of the present disclosure, the live streaming event in the live streaming application 110 can include the host user live streaming singing, live streaming teaching, live streaming network game, etc. or can include the host user inviting part of the audience users or other host users as guests to participate in interactive events, such as singing competition, game competition, live-mic activity, etc. In this way, more diverse live streaming content can be provided in the live streaming application 110. In such live streaming, the live streaming data stream needs to be managed to guarantee the interactivity of the live streaming.

[0051] According to one example implementation of the present disclosure, when the data content 330 is presented at the first device, in response to determining that the state information 320 indicates that the second device is in the working state at the time period corresponding to the data stream, the data stream 310’ can be presented at the first device. Referring to FIG. 3, for example, when the state information 320 in the SEI message is binary number 1, it can indicate that the device of the host user (i.e. the second device) is in the working state at the corresponding time period. The working state can represent the state that the host user is live streaming. At this time, the live streaming data stream 310’ can be presented at the device of the audience user (i.e. the first device), i.e. the audience user can normally watch the live streaming.

[0052] FIG. 4 shows a flowchart of a process 400 for managing the data stream according to some implementations of the present disclosure. In the process 400, after the host user starts live streaming (410), the host user continuously sends the data stream 310, which is equivalent to continuously sending the SEI message (415) because the data stream contains the SEI message. Then, if the host user pauses the live streaming (420), the state information 320 in the SEI message can be written with content that can indicate the pause state (425).

[0053] According to one example implementation of the present disclosure, in response to determining that the state information 320 indicates that the second device is in a paused state at a time period corresponding to the data stream, a pause page 340 can be presented at the first device to indicate that the data stream is paused. Referring back to FIG. 3, for example, when the state information 320 in the SEI message is binary number 0, it can indicate that the device of the host user (i.e., the second device) is in a paused state at the corresponding time period. The paused state can represent that the host user paused the current live streaming. At this time, the pause page 340 can be presented at the device of the audience user (i.e., the first device) to make the audience user aware that the current live streaming is paused.

[0054] With continued reference to FIG. 4, when the live streaming of the host user ends, i.e., the live streaming is resumed (430), the state information 320 indicating the resumed state can be written into the SEI message. As a result, when the data stream 310 containing the SEI message is transmitted to the client of the audience user (i.e., the first device), the pause page 340 in the live streaming application 110 of the client can be removed. In turn, the audience user can continue to watch the live streaming of the host. When the live streaming of the host is closed (440), the data stream 310 of the live streaming is no longer transmitted to the client of the audience user, and accordingly, the SEI message is no longer sent to the client (445).

[0055] With the example implementation of the present disclosure, by inserting the live / pause state into the data stream, the live / pause state accompanies the data stream, and the corresponding message of live streaming pause and live streaming resume is accurately delivered to the client on the audience side, which is conducive to the presentation effect of live streaming pause and live streaming resume on the client on the audience side.

[0056] According to one example implementation of the present disclosure, the state information 320 can be inserted into the data stream 310 at a predetermined frequency, and when the state information 320 is determined, the state information 320 can be extracted at the predetermined frequency. As shown in FIG. 4, when the SEI message is continuously sent (415), it can be sent at the predetermined frequency. In other words, the SEI message can be inserted into the data stream 310 at the predetermined frequency. Accordingly, the state information 320, which can be binary number 1 or 0, can be written into the SEI message of the data stream 310 at the predetermined frequency. Accordingly, the state information 320 can be extracted at the predetermined frequency to determine the working state or the paused state indicated by the state information 320.

[0057] It should be noted that the predetermined frequency can be a high frequency to avoid the SEI message being lost due to network problems. For example, the predetermined frequency can be about once per second or other frequencies to greatly reduce the probability of the SEI message being lost and adapt to the time accuracy of the audience user perceiving the live / pause.

[0058] FIG. 5 illustrates a block diagram of a timing process 500 for handling the pause message in the case of stream interruption, according to some implementations of the present disclosure. In the timing process 500, a host side timeline 510 and a viewer side timeline 520 are shown respectively. Due to network reasons or other reasons, the viewer side client can experience a delay in receiving the data stream 310. For example, assume that the time point of starting the live broadcast on the host side timeline is T0, when the data stream of starting the live broadcast is transmitted to the viewer side client, the time point of starting the live broadcast for the viewer user is T1. Then, the viewer side client continuously receives the live broadcast data stream 310-1 (which belongs to a part of the data stream 310). After that, due to network reasons or other reasons, the viewer side client can experience stream interruption 525 (i.e., the data stream is interrupted during T3 to T5) in the process of receiving the data stream. In this case, there can be four different specific cases, which will be discussed in detail below.

[0059] The first case is that the action of pausing 530 occurs on the host side, i.e., the time point of starting the pause 530 for the viewer side is at T2 (T2 is in the data stream 310-1 stage), and the time point of ending the pause 530 is at T4 (T4 is in the stream interruption 525 period). In this case, at the time point of T2, the viewer side client can learn the message of the host side initiating the pause according to the SEI message in the received data stream 310-1, and then can present the pause page 340 in the live broadcast application 110 of the viewer side client. Subsequently, during the stream interruption 525, the viewer side client cannot receive the data stream, so it also cannot receive the SEI message about the end of the pause 530. Therefore, during T2 to T5, the pause page 340 can always exist in the live broadcast application 110 of the viewer side client. After T5, the viewer side client can continue to receive the data stream 310-2 (which belongs to a part of the data stream 310). Through the implementation of inserting the SEI message in the data stream at a predetermined frequency in the present disclosure, the data stream matching the live / pause state can be accurately presented in this first case.

[0060] The second case is that the host side initiates a pause 540, i.e., the start time of the pause 540 is at T2 (T2 is in the data stream 310-1 stage) and the end time of the pause 540 is at T6 (T6 is in the data stream 310-2 stage) for the audience side. In this case, at the T2 time point, the audience side client can learn from the received SEI message in the data stream 310-1 that the host side initiates the pause, and then can present the pause page 340 in the live broadcast application 110 of the audience side client. Subsequently, during the stream interruption 525, the audience side client cannot receive the data stream, but since this period is the state of the live broadcast pause 540, the stream interruption 525 will not be affected. Subsequently, in the data stream 310-2 stage, the audience side client can learn from the SEI message in the data stream 310-2 that the host side ends the pause 540 at the T6 time point, and then can cancel the pause page 340 and return to the state of watching the live broadcast. Through the implementation of the present disclosure, the SEI message is inserted in the data stream at a predetermined frequency, so that the data stream matched to the live broadcast / pause state can be accurately presented in the second case.

[0061] FIG. 6 shows a block diagram of a timing process 600 for processing a pause message in a stream interruption case according to some implementations of the present disclosure. The host side timeline 510 and the audience side timeline 520 in the timing process 600 are consistent with those in the timing process 500. Here, the scenario of the host side pause in the stream interruption 525 (i.e., the data stream is interrupted during T3 to T5) case is continued from the above discussion.

[0062] The third case is that the host side initiates a pause 610, i.e., the start time of the pause 610 is at T7 (T7 is during the stream interruption 525) and the end time of the pause 610 is at T8 (T8 is also during the stream interruption 525) for the audience side. In this case, since the entire stage of the pause 610 is during the stream interruption 525, the audience side client cannot receive the SEI message and cannot learn that the host side initiates the pause 610 and ends the pause 610. At this time, the pause page 340 can be started in combination with the IM message.

[0063] According to one example implementation of the present disclosure, the second device can send a notification message when switching the live streaming state, and the first device can receive the notification message (i.e. IM message) from the second device, which can indicate the current state of the second device. In other words, the message about the anchor suspending or resuming the live streaming can be notified by sending the notification message to the audience-side client (i.e. the first device). The IM message can be sent separately from the data stream 310. Therefore, the IM message indicating the current switching state can be sent as long as the live streaming state of the anchor is switched to suspend or resume the live streaming, regardless of whether the data stream is interrupted. The IM message may, for example, contain on (e.g. represented by 1) or off (e.g. represented by 0), off can indicate the state of the anchor-side device (i.e. the second device) initiating the live streaming suspension, and on can indicate the state of ending the suspension, i.e. resuming the live streaming.

[0064] According to one example implementation of the present disclosure, in response to determining that the extraction of the state information 320 at the predetermined frequency fails, the data content corresponding to the data stream can be rendered at the first device based on the current state. According to one example implementation of the present disclosure, when rendering the data content based on the current state, in response to determining that the current state is the suspended state, a suspension page 340 can be rendered at the first device to indicate that the data stream is suspended. Exemplarily, when the IM message contains on, the live streaming content after the current live streaming is resumed is rendered in the live streaming application 110 of the audience-side client. When the IM message contains off, the suspension page 340 is rendered in the live streaming application 110 of the audience-side client. At this time, in this third case, the data stream matching the live streaming / suspended state can be accurately rendered.

[0065] The fourth case is that the anchor-side initiates the suspension 620, i.e. the starting point of the suspension 620 for the audience-side is at T7 (T7 is during the interruption 525), and the ending point of the suspension 620 is at T9 (T9 is in the data stream 310-2 stage). In this case, since the starting stage of the suspension 620 is during the interruption 525, the audience-side client cannot receive the SEI message and thus cannot know that the anchor-side initiates the suspension 620. At this time, the suspension page 340 can be started in combination with the IM message as discussed above. Subsequently, in the data stream 310-2 stage, the audience-side client can know that the anchor-side ends the suspension 620 at the T9 point of time through the SEI message in the data stream 310-2, and can cancel the suspension page 340 and return to the state of watching the live streaming. Through the implementation of the present disclosure, the SEI message is inserted in the data stream at the predetermined frequency, and in combination with the IM message, the data stream matching the live streaming / suspended state can be accurately rendered in this fourth case.

[0066] With the exemplary implementation of the present disclosure, the IM message can be used to assist in determining whether to present the pause page 340 at the viewer-side client in the case of partial data stream interruption. In the case of no interruption and another partial data stream interruption, the SEI message inserted in the data stream at a predetermined frequency according to the implementation of the present disclosure can solve the problem that the timing of the appearance or disappearance of the pause overlay on the viewer-side client does not match the timing of the initiation of the live broadcast pause or resumption by the host. Thus, the implementation effectively and flexibly realizes the live broadcast pause and resumption.

[0067] FIG. 7 shows a block diagram of a timing process 700 regarding the enabling of the IM message according to some implementations of the present disclosure. The host-side timeline 510 and the viewer-side timeline 520 in the timing process 700 are also consistent with those in the timing process 500. The timing process 700 also includes the viewer-side timeline 520-1 corresponding to the viewer-side timeline 520 regarding whether the SEI message can be received, and the viewer-side timeline 520-2 corresponding to the viewer-side timeline 520 regarding when and how to enable the IM message. In the viewer-side timeline 520 and the corresponding viewer-side timeline 520-1, when the viewer-side client can receive the data stream 310-1, the SEI message can be continuously received accordingly (710-1). When the interruption 525 occurs, the SEI message cannot be received accordingly (725). When the viewer-side client receives the data stream 310-2 again, the SEI message can be continuously received again accordingly (710-2).

[0068] As discussed above, the viewer-side client can not need to use the IM message when the SEI message can be received. In the case of the interruption 525, the SEI message can be used without the IM message if the first and second cases occur. In the third and fourth cases, the IM message can be enabled to assist in determining the pause and resumption of the live broadcast by the host.

[0069] Specifically, the timer can be used to check whether the SEI message is received at the same frequency as the predetermined frequency. Assuming that the predetermined frequency is once per second, at the T3 time point, the timer checks whether the SEI message is received after one second (at the T10 time point) and determines that the SEI message is not received. Illustratively, the IM messages can be buffered each time the IM message is received at the audience-side client. The buffering manner can be all buffering. In order to save the buffer space, only a number of the most recently received IM messages can be buffered in sequence, i.e., each time a new IM message is received, the IM message currently ranked first can be deleted. Alternatively, only the latest IM message can be buffered each time, i.e., each time a new IM message is received, the IM message received last time can be overwritten. When it is determined that the SEI message is not received at the T10 time point, the latest buffered IM message can be synchronized to the audience-side client. In this way, the pause page 340 can be presented on the audience-side client based on the latest IM message.

[0070] With the exemplary implementation of the present disclosure, by using the timer to check whether the SEI message is received at the same frequency as the predetermined frequency, and in combination with the buffering manner of the IM message, the usage occasion and specific object of the IM message can be determined, thereby facilitating the implementation of effective pause in the live broadcast process.

[0071] According to one exemplary implementation of the present disclosure, in response to determining that the receiving of the data stream fails, a candidate data content corresponding to the second device can be presented at the first device. Specifically, in the case of the live data stream being interrupted, a candidate media item can be presented. For example, 1) a picture of the host blurred by Gaussian blur; 2) a picture of the host blurred by Gaussian blur in the last frame before the host pauses the live broadcast; 3) a picture indicating “downloading”, such as a picture of a waiting wheel, etc.

[0072] With the exemplary implementation of the present disclosure, the corresponding live data stream can be presented at the audience user side in a more accurate manner according to the real live / pause state, thereby facilitating the audience user to know the current live data state and increasing the accuracy of the data stream presentation.

[0073] Example process

[0074] FIG. 8 illustrates a flowchart of a method 800 for managing a data stream according to some implementations of the present disclosure. At block 810, a data stream associated with a second device is received at a first device. At block 820, state information is determined based on the data stream, the state information indicating a state of the second device at a time period corresponding to the data stream. At block 830, data content corresponding to the data stream is presented at the first device based on the state information.

[0075] According to one example implementation of the present disclosure, presenting the data content at the first device comprises at least one of: presenting the data stream at the first device in response to determining that the status information indicates that the second device is in an active state at a time period corresponding to the data stream; and presenting a pause page at the first device to indicate that the data stream is paused in response to determining that the status information indicates that the second device is in a paused state at the time period corresponding to the data stream.

[0076] According to one example implementation of the present disclosure, the status information is inserted into the data stream at a predetermined frequency, and determining the status information comprises extracting the status information at the predetermined frequency.

[0077] According to one example implementation of the present disclosure, the method 800 further comprises receiving a notification message from the second device, the notification message indicating a current status of the second device.

[0078] According to one example implementation of the present disclosure, the method 800 further comprises presenting the data content corresponding to the data stream at the first device based on the current status in response to determining that the extracting the status information at the predetermined frequency fails.

[0079] According to one example implementation of the present disclosure, wherein presenting the data content based on the current status comprises presenting a pause page at the first device to indicate that the data stream is paused in response to determining that the current status is a paused status.

[0080] According to one example implementation of the present disclosure, the method 800 further comprises presenting candidate data content corresponding to the second device at the first device in response to determining that the receiving the data stream fails.

[0081] According to one example implementation of the present disclosure, the data stream is a live data stream in a live application, and determining the status information based on the data stream comprises extracting the status information from a supplemental enhancement information of the data stream.

[0082] According to one example implementation of the present disclosure, the live application comprises a first user and a second user, the first user is at least one of a spectator user and a guest user, the second user is a host user, the first device is a device associated with the first user, the second device is a device associated with the second user, and the status indicates a status of the live application in the second device.

[0083] Example apparatuses and devices

[0084] FIG. 9 illustrates a block diagram of an apparatus 900 for managing a data stream according to some implementations of the present disclosure. The apparatus 900 includes a receiving module 99 configured for receiving, at a first device, a data stream associated with a second device; a determining module 920 configured for determining, based on the data stream, state information indicating a state of the second device at a time period corresponding to the data stream; and a presenting module 930 configured for presenting, at the first device, data content corresponding to the data stream based on the state information.

[0085] According to one example implementation of the present disclosure, the presenting module 930 is further configured for presenting, at the first device, the data content based on at least one of: presenting, at the first device, the data stream in response to determining that the state information indicates that the second device is in an active state at the time period corresponding to the data stream; and presenting, at the first device, a pause page to indicate that the data stream is paused in response to determining that the state information indicates that the second device is in a paused state at the time period corresponding to the data stream.

[0086] According to one example implementation of the present disclosure, the state information is inserted into the data stream at a predetermined frequency, and the determining module 920 is further configured for extracting the state information at the predetermined frequency.

[0087] According to one example implementation of the present disclosure, the apparatus 900 further includes a notification message receiving module configured for receiving a notification message from the second device, the notification message indicating a current state of the second device.

[0088] According to one example implementation of the present disclosure, the apparatus 900 further includes a failure extracting module configured for presenting, at the first device, the data content corresponding to the data stream based on the current state in response to determining that an extraction of the state information at the predetermined frequency fails.

[0089] According to one example implementation of the present disclosure, the failure extracting module is further configured for presenting, at the first device, a pause page to indicate that the data stream is paused in response to determining that the current state is a paused state.

[0090] According to one example implementation of the present disclosure, the apparatus 900 further includes a failure receiving module configured for presenting, at the first device, candidate data content corresponding to the second device in response to determining that a reception of the data stream fails.

[0091] According to one example implementation of the present disclosure, the data stream is a live data stream in a live application, and the determining module 920 is further configured for extracting the state information from a supplemental enhancement information of the data stream.

[0092] According to one example implementation of the present disclosure, the live streaming application includes a first user and a second user, the first user is at least one of a spectator user and a guest user, the second user is a host user, the first device is a device associated with the first user, the second device is a device associated with the second user, and the status indicates a status of the live streaming application in the second device.

[0093] FIG. 10 illustrates a block diagram of a device 1000 that is capable of implementing the various implementations of the present disclosure. It should be appreciated that the computing device 1000 illustrated in FIG. 10 is merely an example and should not be construed as any limitation of the functionality and scope of the implementations described herein. The computing device 1000 illustrated in FIG. 10 can be used to implement the methods described above.

[0094] As shown in FIG. 10, the computing device 1000 is in the form of a general- purpose computing device. The components of the computing device 1000 can include, but are not limited to, one or more processors 1010, a memory 1020, a storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. The processor 1010 can be a real or virtual processor and is capable of executing various processing in accordance with programs stored in the memory 1020. In a multi-processor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 1000.

[0095] The computing device 1000 typically includes a plurality of computer storage media. Such media can be any available media that is accessible by the computing device 1000 and includes both volatile and non-volatile media, removable and non-removable media. The memory 1020 can be volatile memory (such as a register, cache, random access memory (RAM)), non-volatile memory (such as read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory), or some combination thereof. The storage device 1030 can be a removable or non-removable media and can include machine-readable media such as a flash drive, a magnetic disk drive, or any other media that can be used to store information and / or data (e.g., training data for training) and that can be accessed by the computing device 1000.

[0096] The computing device 1000 can further include additional removable / non-removable, volatile / non-volatile storage devices. Although not shown, a floppy disk drive for reading from or writing to a removable, non-removable, and volatile memory disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-removable, and volatile memory disk (e.g., a "floppy disk"), can be provided. In such instances, each drive can be connected to the bus by one or more data media interfaces. The storage device(s) 1020 can include a computer-program product 1025 that has one or more program modules configured to carry out the various methods or actions of the implementations of the present disclosure.

[0097] The communication unit 1040 enables communications with other computing devices over a communication medium. Additionally, the functionality of the components of the computing device 1000 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating over a communication connection. Thus, the computing device 1000 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0098] The input device(s) 1050 can be one or more input device(s) such as a mouse, keyboard, trackball, etc. The output device(s) 1060 can be one or more output device(s) such as a display, speakers, printer, etc. The computing device 1000 can also contain communication connection(s) 1040 that allow the device to communicate with one or more external devices not shown, such as a storage device, a display device, etc. Other external devices (not shown) can also be connected to the computing device 1000 through the communication connection(s) 1040, such as a printer, a scanner, etc. Furthermore, the computing device 1000 can also include a communication interface that can be used to enable the computing device 1000 to communicate with other devices and / or systems in a distributed environment, such as over an Intranet or the Internet, with other devices not functioning as a server, with or without a storage device or storing expansion slot. In some embodiments, the communication interface can include wired communication interfaces (e.g., a network interface card) and / or wireless communication interfaces (e.g., a wireless network interface card).

[0099] According to example implementations of the present disclosure, a computer-readable storage medium is provided having computer-executable instructions stored thereon, where the computer-executable instructions are executed by a processor to implement the methods described above. According to example implementations of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, where the computer-executable instructions are executed by a processor to implement the methods described above. According to example implementations of the present disclosure, a computer program product is provided having computer-executable instructions stored thereon, which program is executed by a processor to implement the methods described above.

[0100] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0101] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0102] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0103] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0104] Having described various implementations of the disclosure above, the descriptions are not exhaustive and do not limit the disclosure to the disclosed implementations. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The scope of the disclosure includes all the implementations of which an equivalent would be apparent to those skilled in the art from the disclosure given and the associated drawings. The selection of the terms to be used in the written description is not intended to limit the scope of the present disclosure, but rather to best describe the principles of the various implementations in preference to a careful drawing of equivalent alternatives which are to be substituted for the preferred implementations illustrated and described.

Claims

1. A method for managing a data stream, comprising: receiving, at a first device, a data stream associated with a second device; determining, based on the data stream, state information, the state information indicating a state of the second device at a time period corresponding to the data stream; and presenting, at the first device, data content corresponding to the data stream based on the state information. 2.The method of claim 1, wherein presenting, at the first device, the data content comprises at least one of the following: presenting, at the first device, the data stream in response to determining that the state information indicates that the second device is in an active state at the time period corresponding to the data stream; and presenting, at the first device, a pause page to indicate that the data stream is paused in response to determining that the state information indicates that the second device is in a paused state at the time period corresponding to the data stream. extracting the state information at a predetermined frequency.

3. The method of claim 1, wherein the status information is inserted into the data stream at a predetermined frequency, and determining the status information comprises: receiving a notification message from the second device, the notification message indicating a current state of the second device.

4. The method of claim 3, further comprising: presenting, at the first device, data content corresponding to the data stream based on the current state in response to determining that extracting state information at the predetermined frequency fails.

5. The method of claim 4, further comprising: presenting, at the first device, a pause page to indicate that the data stream is paused in response to determining that the current state is a paused state.

6. The method of claim 5, wherein presenting the data content based on the current state comprises: presenting, at the first device, candidate data content corresponding to the second device in response to determining that receiving the data stream fails.

7. The method of claim 1, further comprising: extracting the state information from supplemental enhancement information of the data stream.

8. The method of claim 1, wherein the data stream is a live data stream in a live application, determining the state information based on the data stream comprises: 9.The method of claim 8, wherein the live application comprises a first user and a second user, the first user is at least one of a spectator user and a guest user, the second user is a host user, the first device is a device associated with the first user, the second device is a device associated with the second user, and the state indicates a state of the live application in the second device. 10.An apparatus for managing a data stream, comprising: a receiving module configured to receive, at a first device, a data stream associated with a second device; a determining module configured to determine, based on the data stream, state information, the state information indicating a state of the second device at a time period corresponding to the data stream; and a presenting module configured to present, at the first device, data content corresponding to the data stream based on the state information. 11.An electronic device, comprising: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9. 12.A computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions, when executed by a processor, cause the processor to implement the method according to any one of claims 1 to 9. ​ ​ ​ 13. A computer program product comprising computer executable instructions, wherein said computer executable instructions implement the method according to any one of claims 1 to 9 when executed by a processor.

Images