Method and device for displaying stream pushing data in video livestreaming

Abstract

Embodiments of the present disclosure relate to the technical field of video livestreaming, and provide a method and device for displaying stream pushing data in video livestreaming. The method comprises: acquiring a video stream delay; when receiving stream pushing data corresponding to a first event, generating data stream pushing information carrying the stream pushing data, the video stream delay and a first timestamp, the first timestamp being used for representing the moment of occurrence of the first event; and sending the data stream pushing information to a client device, such that the client device determines a display timestamp of the stream pushing data according to the first timestamp and the video stream delay, and displays the stream pushing data according to the display timestamp.

Description

Method and device for displaying streaming data in live video broadcast

[0001] This application claims priority to the Chinese invention patent application entitled “A method and device for displaying streaming data in live video broadcasting” and application number 202310341196X, filed on March 31, 2023. The entire contents of that application are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of video live broadcast technology, and in particular to a method and device for displaying streaming data in video live broadcast. Background Art

[0003] With the rapid development of terminal technology and network technology, live video has become very common and widely used in various fields.

[0004] Generally speaking, the video stream and data stream of live video broadcasts often come from different servers. Due to network transmission delay, video encoding and decoding delay, etc., the delay of the video stream is often greater than the delay of the data stream, affecting the user's live viewing experience.

[0005] Summary of the Invention

[0006] In view of this, the embodiments of the present disclosure provide a method and device for displaying streaming data in live video broadcasting, so as to enhance the user's live viewing experience.

[0007] In order to achieve the above objectives, the embodiments of the present disclosure provide the following technical solutions:

[0008] In a first aspect, an embodiment of the present disclosure provides a method for displaying streaming data in a live video broadcast, comprising:

[0009] Get video stream delay;

[0010] When receiving streaming data corresponding to a first event, generating data streaming information carrying the streaming data, the video stream delay, and a first timestamp; the first timestamp is used to indicate the time when the first event occurs;

[0011] The data streaming information is sent to a client device, so that the client device determines a presentation timestamp of the streaming data according to the first timestamp and the video stream delay, and presents the streaming data according to the presentation timestamp.

[0012] As an optional implementation of the embodiment of the present disclosure, obtaining the video stream delay includes:

[0013] Obtaining a second timestamp and a third timestamp; wherein the second timestamp is used to represent a relative moment of a second event in the live video broadcast, the second event being an event displayed by a target video frame; and the third timestamp is used to represent an encoding moment of the target video frame;

[0014] Obtaining a fourth timestamp and a fifth timestamp; wherein the fourth timestamp is used to represent the time when the third event occurs, and the fifth timestamp is used to represent the relative time of the third event;

[0015] Determine, according to the second timestamp, the fourth timestamp, and the fifth timestamp, a sixth timestamp for representing a moment of occurrence of the second event;

[0016] Obtain a video stream delay of the video stream according to the third timestamp and the sixth timestamp.

[0017] As an optional implementation of the embodiment of the present disclosure, obtaining the second timestamp includes:

[0018] determining target object information based on an image corresponding to the target video frame;

[0019] The second timestamp is acquired according to the target object information.

[0020] As an optional implementation of the embodiment of the present disclosure, obtaining the third timestamp includes:

[0021] Determine a supplemental enhancement information SEI frame adjacent to the target video frame from the video stream of the live video broadcast;

[0022] The third timestamp is obtained according to the SEI frame.

[0023] As an optional implementation of the embodiment of the present disclosure, determining the sixth timestamp according to the second timestamp, the fourth timestamp, and the fifth timestamp includes:

[0024] Obtaining a time interval based on a difference between the second timestamp and the fifth timestamp;

[0025] The sixth timestamp is acquired based on the fourth timestamp and the time interval.

[0026] As an optional implementation of the embodiment of the present disclosure, obtaining the video stream delay according to the second timestamp and the fifth timestamp includes:

[0027] A difference between the third timestamp and the sixth timestamp is calculated to obtain the video stream delay.

[0028] As an optional implementation of the embodiment of the present disclosure, the third event is an event corresponding to the streaming data having the shortest time interval with the streaming data corresponding to the first event.

[0029] In a second aspect, an embodiment of the present disclosure provides a method for presenting data streaming in a live video broadcast, including:

[0030] Receive data streaming information sent by a data streaming server, the data streaming information including: streaming data corresponding to a first event, a first timestamp, and a video stream delay, wherein the first timestamp is used to indicate a time when the first event occurs;

[0031] Obtaining a presentation timestamp of the pushed stream data according to the first timestamp and the video stream delay;

[0032] The pushed stream data is displayed according to the display timestamp.

[0033] As an optional implementation of the embodiment of the present disclosure, obtaining the presentation timestamp of the push stream data according to the first timestamp and the video stream delay includes:

[0034] The first timestamp and the video stream delay are summed to obtain a presentation timestamp of the push stream data corresponding to the first event.

[0035] As an optional implementation of the embodiment of the present disclosure, the displaying of the streaming data according to the display timestamp includes:

[0036] Get the timestamp representing the encoding moment of the currently displayed video frame;

[0037] Determining whether a timestamp representing an encoding moment of a currently displayed video frame is greater than or equal to the presentation timestamp;

[0038] If so, the streaming data is displayed.

[0039] In a third aspect, an embodiment of the present disclosure provides a data streaming server, including:

[0040] An acquisition unit, used to acquire video stream delay;

[0041] a generating unit configured to, upon receiving streaming data corresponding to a first event, generate streaming data information carrying the streaming data, the video stream delay, and a first timestamp; the first timestamp being used to indicate a time when the first event occurs;

[0042] A sending unit is used to send the data streaming information to the client device, so that the client device determines the display timestamp of the streaming data corresponding to the first event according to the first timestamp and the video stream delay, and displays the streaming data corresponding to the first event according to the display timestamp.

[0043] As an optional implementation of the embodiment of the present disclosure, the acquisition unit is specifically used to obtain a second timestamp and a third timestamp; the second timestamp is used to represent the relative moment of the second event in the live video broadcast, the second event is the event displayed by the target video frame, and the third timestamp is used to represent the encoding moment of the target video frame; obtain a fourth timestamp and a fifth timestamp; the fourth timestamp is used to represent the occurrence moment of the third event, and the fifth timestamp is used to represent the relative moment of the third event; according to the second timestamp, the fourth timestamp and the fifth timestamp, determine a sixth timestamp used to represent the occurrence moment of the event displayed by the target video frame; according to the third timestamp and the sixth timestamp, obtain the video stream delay of the video stream.

[0044] As an optional implementation of the embodiment of the present disclosure, the acquisition unit is specifically configured to determine target object information based on an image corresponding to the target video frame; and acquire the second timestamp according to the target object information.

[0045] As an optional implementation of the embodiment of the present disclosure, the acquisition unit is specifically used to determine the SEI frame adjacent to the target video frame from the video stream of the live video broadcast; and obtain the third timestamp according to the SEI frame.

[0046] As an optional implementation of the embodiment of the present disclosure, the acquisition unit is specifically configured to acquire the time interval based on the difference between the second timestamp and the fifth timestamp; and acquire the sixth timestamp based on the fourth timestamp and the time interval.

[0047] As an optional implementation of the embodiment of the present disclosure, the acquisition unit is specifically configured to calculate a difference between the third timestamp and the sixth timestamp to acquire the video stream delay.

[0048] As an optional implementation of the embodiment of the present disclosure, the third event is an event corresponding to the streaming data having the shortest time interval with the streaming data corresponding to the first event.

[0049] In a fourth aspect, an embodiment of the present disclosure provides a client device, including:

[0050] a receiving unit, configured to receive data streaming information sent by a data streaming server, the data streaming information including streaming data corresponding to a first event, a first timestamp, and a video stream delay, wherein the first timestamp is used to indicate a time when the first event occurs;

[0051] A processing unit, configured to obtain a presentation timestamp of the pushed stream data according to the first timestamp and the video stream delay;

[0052] A display unit is configured to display the streaming data according to the display timestamp.

[0053] As an optional implementation of the embodiment of the present disclosure, the processing unit is specifically configured to sum the first timestamp and the video stream delay to obtain a presentation timestamp of the streaming data corresponding to the first event.

[0054] As an optional implementation of the embodiment of the present disclosure, the display unit is specifically used to obtain a timestamp for representing the encoding moment of the currently displayed video frame; determine whether the timestamp for representing the encoding moment of the currently displayed video frame is greater than or equal to the display timestamp; if so, display the pushed stream data.

[0055] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the method for displaying streaming data in live video broadcasting described in any of the above embodiments when executing the computer program.

[0056] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement the method for displaying streaming data in live video broadcasting described in any of the above-mentioned embodiments.

[0057] In a seventh aspect, an embodiment of the present disclosure provides a computer program product. When the computer program product is run on a computer, the computer implements the method for displaying streaming data in live video broadcasting as described in any of the above-mentioned embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0059] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] FIG1 is a scene architecture diagram of a method for displaying streaming data in a live video broadcast provided by an embodiment of the present disclosure;

[0061] FIG2 is a flowchart of one of the steps of a method for displaying streaming data in a live video broadcast provided by an embodiment of the present disclosure;

[0062] FIG3 is a second flowchart of the method for displaying streaming data in live video broadcasting provided by an embodiment of the present disclosure;

[0063] FIG4 is a schematic diagram of an image corresponding to a video frame provided by an embodiment of the present disclosure;

[0064] FIG5 is a schematic diagram of the structure of a data streaming server provided by an embodiment of the present disclosure;

[0065] FIG6 is a schematic diagram of the structure of a client device provided in an embodiment of the present disclosure;

[0066] FIG7 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0069] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0070] The method for displaying push stream data in live video broadcast provided by the embodiment of the present disclosure is that when performing live video broadcast, the data push stream server first obtains the video stream delay between the occurrence time of any event and the time when the video stream of the live video broadcast displays the event, and when receiving the push stream data corresponding to the first event, generates data push stream information carrying the push stream data, the video stream delay and the first timestamp used to characterize the occurrence time of the first event, and then sends the data push stream information to the client device so that the client device determines the display timestamp of the push stream data according to the first timestamp and the video stream delay, and displays the push stream data according to the display timestamp. Since the data push stream information sent by the data push stream server to the client device includes the video stream delay and the first timestamp, the client determines the display timestamp of the push stream data according to the first timestamp and the video stream delay, which can avoid the push stream data corresponding to the first event being displayed before the video stream displays the first event. Therefore, the embodiment of the present disclosure can enhance the user's live viewing experience.

[0071] The following first describes the scenario architecture of the method for displaying streaming data in live video broadcasting provided by the embodiment of the present disclosure.

[0072] As shown in FIG1 , the scenario architecture of the method for presenting live streaming data in a live video broadcast includes a video streaming server 11, a data streaming server 12, and a client device 13. It should be noted that FIG1 illustrates client device 13 as a mobile phone as an example, but the present disclosure is not limited thereto. Client device 13 may also be a television, computer, tablet computer, virtual reality glasses, or other devices.

[0073] Among them, the video push streaming server 11 is used to obtain the source video and send the video stream of the live video to the client device 13 based on the source video. For example: when broadcasting a ball game live, the video push streaming server 11 obtains the game screen of the ball game as the source video, and sends the video stream to the client device 13 based on the game screen of the ball game. The data push streaming server 12 is used to obtain push streaming data and send the data stream of the live video to the client device 13 based on the push streaming data. For example: when broadcasting a ball game live, push streaming data is generated according to the score changes of the ball game, and the corresponding push streaming information is sent to the client device 13. The client device 13 is used to display the live video content based on the video stream sent by the video push streaming server 11 and the data stream sent by the data push streaming server 12.

[0074] Based on the above scenario architecture, the present disclosure provides a method for displaying push stream data in a live video broadcast. As shown in FIG2 , the method for displaying push stream data in a live video broadcast includes the following steps:

[0075] S21. The data streaming server obtains the video stream delay.

[0076] The video stream delay is used to represent the time interval between the occurrence of any event and the moment when the live video stream displays the event.

[0077] When live video is being broadcast, due to reasons such as network transmission time and video encoding and decoding time, the time when the user views the event through the client device will be later than the time when the event occurs. The time interval between the time when the user views the event through the client device and the time when the event occurs is called the video stream delay in the embodiment of the present disclosure.

[0078] For example, when a ball game is live streamed, a goal is scored at 08:21:15, but the video corresponding to the goal is not displayed on the client device through the video stream until 08:22:00. Based on the time of the goal (08:21:15) and the time (08:22:00) when the video corresponding to the goal is displayed on the client device, the video stream delay can be determined to be 08:22:00 - 08:21:15 = 45 seconds.

[0079] In some embodiments, the method in which the data streaming server obtains the video stream delay may include: periodically obtaining the video stream delay with a preset time period as a period.

[0080] In some embodiments, the data push stream server may obtain the video stream delay in a manner that includes: obtaining the video stream delay once at the beginning of the live video broadcast, and using the obtained video stream delay as the video stream delay for the entire live video broadcast.

[0081] S22: When the data streaming server receives streaming data corresponding to the first event, it generates data streaming information carrying the streaming data, the video stream delay, and a first timestamp.

[0082] The first timestamp is used to represent the time when the first event occurs.

[0083] As described in the above embodiment, the video stream delay is 45 seconds. If the push stream data corresponding to the goal event generated at 08:40:25 is the push stream data that changes the score of the ball game from 1:0 to 1:1, then when the data push stream server receives the push stream data corresponding to the goal event, it can generate data push stream information carrying the push stream data (changing the score of the ball game from 1:0 to 1:1), the video stream delay (45 seconds), and the first timestamp (08:40:25) used to represent the occurrence time of the goal event.

[0084] S23: The data streaming server sends the data streaming information to the client device.

[0085] Correspondingly, the client device receives the data streaming information sent by the data streaming server, wherein the data streaming information includes: streaming data corresponding to the first event, a first timestamp, and a video stream delay, wherein the first timestamp is used to indicate the time instant when the first event occurs, and the video stream delay is used to indicate the time interval between the time instant when any event occurs and the time instant when the live video stream displays the event.

[0086] S24. The client device obtains a presentation timestamp of the pushed stream data according to the first timestamp and the video stream delay.

[0087] In some embodiments, the above step S24 (the client device obtains the presentation timestamp of the push stream data according to the first timestamp and the video stream delay) includes:

[0088] The first timestamp and the video stream delay are summed to obtain a presentation timestamp of the pushed stream data.

[0089] As described in the above embodiment, the first timestamp is 08:40:25, and the video stream delay is 45 seconds. The presentation timestamp of the data push stream obtained by summing the first timestamp and the video stream delay is 08:41:10.

[0090] S25. The client device displays the streaming data according to the display timestamp.

[0091] In some embodiments, the client device presenting the streaming data according to the presentation timestamp includes:

[0092] The streaming data starts to be displayed at the time represented by the display timestamp.

[0093] Continuing with the above embodiment, the presentation timestamp is 08:41:10. If the push stream data is displayed starting at the time indicated by the presentation timestamp, the score in the live video broadcast will be changed to 1:1 starting at 08:41:10. Since the goal event occurred at 08:40:25 and the video stream delay is 45 seconds, theoretically, the live video stream will display the video footage of the goal event at 08:41:10. Therefore, changing the score in the live video broadcast to 1:1 starting at 08:41:10 can synchronize the live video stream with the data stream, thereby improving the user experience.

[0094] The method for displaying push stream data in live video broadcast provided by the embodiment of the present disclosure is that when performing live video broadcast, the data push stream server first obtains the video stream delay between the occurrence time of any event and the time when the video stream of the live video broadcast displays the event, and when receiving the push stream data corresponding to the first event, generates data push stream information carrying the push stream data, the video stream delay and the first timestamp used to characterize the occurrence time of the first event. The data push stream information will then be sent to the client device so that the client device can determine the display timestamp of the push stream data based on the first timestamp and the video stream delay, and display the push stream data based on the display timestamp. Since the data push stream information sent by the data push stream server to the client device includes the video stream delay and the first timestamp, the client determines the display timestamp of the push stream data based on the first timestamp and the video stream delay, which can avoid the push stream data corresponding to the first event being displayed before the video stream displays the first event. Therefore, the embodiment of the present disclosure can enhance the user's live viewing experience.

[0095] As an extension and refinement of the above embodiment, the present disclosure provides another method for displaying push stream data in live video broadcasting. As shown in FIG3 , the method for displaying push stream data in live video broadcasting includes:

[0096] S31. The data streaming server obtains a second timestamp and a third timestamp.

[0097] Among them, the second timestamp is used to represent the relative moment of the second event in the live video broadcast, the second event is the event displayed by the target video frame, and the third timestamp is used to represent the encoding moment of the target video frame.

[0098] As an optional implementation of the embodiment of the present disclosure, the implementation of the above step S31 (the data streaming server obtains the second timestamp) may include the following steps 311 and 312:

[0099] Step 311: Determine target object information based on the image corresponding to the target video frame.

[0100] In some embodiments, determining the target object information based on the image corresponding to the target video frame may include: detecting the image corresponding to the target video frame to obtain the target object information.

[0101] Step 312: Acquire the second timestamp according to the target object information.

[0102] For example, as shown in FIG4 , by detecting the image 400 corresponding to the video frame shown in FIG4 , character strings such as “0:20:12”, “aaaaaa”, and “bbbbbbb” can be obtained, and the obtained character strings can be further analyzed to determine that the relative time of the event displayed by the target video frame is: 0:20:12, that is, the event (second event) displayed by the target video frame is the event that occurred at 0:20:12 of the live video broadcast.

[0103] As an optional implementation of the embodiment of the present disclosure, the data streaming server obtains the third timestamp in the following steps 313 and 314:

[0104] Step 313: Determine a Supplemental Enhancement Information (SEI) frame adjacent to the target video frame from the video stream of the live video broadcast.

[0105] Step 314: Obtain the third timestamp according to the SEI frame.

[0106] Specifically, a supplementary enhancement information frame is inserted into the video stream every preset time period (for example, 2 seconds), and the supplementary enhancement information frame contains the encoder operating system time. Therefore, it is possible to determine the SEI frame adjacent to the target video frame from the video stream of the live video broadcast, and obtain the encoding time of the target video frame based on the SEI frame.

[0107] S32: The data streaming server obtains a fourth timestamp and a fifth timestamp.

[0108] The fourth timestamp is used to represent the occurrence time of the third event, and the fifth timestamp is used to represent the relative time of the third event.

[0109] The third event in the embodiment of the present disclosure may be any event, such as a goal event in a ball game.

[0110] For example, if the third event is a goal event that occurs at 20:15:13, and the relative time of the goal event is 0:15:13, the obtained fourth timestamp and fifth timestamp are 20:15:13 and 0:15:13 respectively.

[0111] As an optional implementation of the embodiment of the present disclosure, the third event is an event corresponding to the streaming data having the shortest time interval with the streaming data corresponding to the first event.

[0112] For example, when a ball game is live-broadcasted, only the time of a goal has corresponding streaming data, and the third event is the last goal event.

[0113] S33: The data streaming server determines a sixth timestamp for representing the occurrence moment of the event displayed by the target video frame according to the second timestamp, the fourth timestamp, and the fifth timestamp.

[0114] As an optional implementation of the embodiment of the present disclosure, the above step S34 (determining a sixth timestamp used to represent the occurrence moment of the event displayed by the target video frame based on the second timestamp, the fourth timestamp, and the fifth timestamp) includes the following steps 331 and 332:

[0115] Step 331: Obtain a time interval based on a difference between the second timestamp and the fifth timestamp.

[0116] In some embodiments, obtaining the time interval based on the difference between the second timestamp and the fifth timestamp includes:

[0117] A difference between the second timestamp and the fifth timestamp is calculated to obtain the time interval.

[0118] Assume: the second timestamp is Rel_A, the fifth timestamp is Rel_Last, and the time interval is T, then: T = Rel_A - Rel_Last

[0119] For example, if the second timestamp is 0:20:12 and the fifth timestamp is 0:15:13, the time interval calculated according to T=Rel_A-Rel_Last is 4 minutes and 59 seconds.

[0120] Step 332: Acquire the sixth timestamp based on the fourth timestamp and the time interval.

[0121] In some embodiments, obtaining the sixth timestamp based on the fourth timestamp and the time interval includes:

[0122] The fourth timestamp and the time interval are summed to obtain the sixth timestamp.

[0123] Assume: the fourth timestamp is Abs_Last, the sixth timestamp is Abs_A, and the time interval is T, then: Abs_A=T+Abs_Last=Rel_A-Rel_Last+Abs_Last

[0124] According to the above embodiment, the second timestamp is 0:20:12, the fifth timestamp is 0:15:13, the time interval is 4 minutes and 59 seconds, and the sixth timestamp is 20:15:13 when the fourth timestamp is 20:15:13. The obtained sixth timestamp is 20:20:12.

[0125] S34. The data streaming server obtains the video stream delay of the video stream according to the third timestamp and the sixth timestamp.

[0126] As an optional implementation of the embodiment of the present disclosure, the above-mentioned step S35 (obtaining the video stream delay of the video stream based on the third timestamp and the sixth timestamp) includes: calculating the difference between the third timestamp and the sixth timestamp to obtain the video stream delay.

[0127] Assume that the third timestamp is SEI_A, the video stream delay is Diff, and the sixth timestamp is Abs_A, then: Diff = SEI_A - Abs_A

[0128] As described in the above embodiment, the sixth timestamp obtained is 20:20:12. If the third timestamp is 20:20:57, the delay in obtaining the video stream is 45 seconds.

[0129] S35. When the data streaming server receives the streaming data corresponding to the first event, it generates data streaming information carrying the streaming data, the video stream delay, and the first timestamp.

[0130] The first timestamp is used to represent the time when the first event occurs.

[0131] The implementation method of step S35 is the same as that of step S22, and will not be repeated here.

[0132] S36: The data streaming server sends the data streaming information to the client device.

[0133] Correspondingly, the client device receives the data streaming information sent by the data streaming information sender.

[0134] S37. The client device sums the first timestamp and the video stream delay to obtain a presentation timestamp for the pushed stream data.

[0135] Assume: the first timestamp is Abs_B, the video stream delay is Diff, and the display timestamp is Disp, then: Disp = Abs_B - Diff

[0136] For example, the first timestamp representing the occurrence time of the first event is 20:25:03, and the video stream delay is 45 seconds. Then, the display timestamp of the pushed stream data is 20:25:48.

[0137] S38. The client device displays the streaming data according to the display timestamp.

[0138] In some embodiments, the client device presents the push stream data according to the presentation timestamp, including:

[0139] The streaming data is displayed starting from the moment represented by the display timestamp.

[0140] As mentioned above, if the display timestamp is 20:25:48, the streaming data will be displayed starting from 20:25:48.

[0141] In some embodiments, the client device displays the streaming data according to the display timestamp, including the following steps 381 to 383:

[0142] Step 381: Obtain a timestamp representing the encoding moment of the currently displayed video frame.

[0143] That is, obtain the timestamp when the currently displayed video frame was encoded.

[0144] In some embodiments, the timestamp when the currently displayed video frame is encoded may be obtained through an SEI frame adjacent to the currently displayed video frame.

[0145] Step 382: Determine whether the timestamp representing the encoding moment of the currently displayed video frame is greater than or equal to the presentation timestamp.

[0146] Whether the live video stream has displayed the first event can be determined by determining whether the timestamp representing the encoding time of the currently displayed video frame is greater than or equal to the presentation timestamp. If the timestamp representing the encoding time of the currently displayed video frame is greater than or equal to the presentation timestamp, it can be determined that the live video stream has displayed the first event. If the timestamp representing the encoding time of the currently displayed video frame is less than the presentation timestamp, it can be determined that the live video stream has not displayed the first event.

[0147] In the above step 382 , if the timestamp representing the encoding moment of the currently displayed video frame is greater than or equal to the presentation timestamp, the following step 383 is executed.

[0148] Step 382: The client device displays the streaming data.

[0149] Based on the same inventive concept, as an implementation of the above method, the embodiment of the present disclosure further provides a data streaming server and a client device. This embodiment corresponds to the above method embodiment. For ease of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the image-based positioning device in this embodiment can correspond to and implement all the contents of the above method embodiment.

[0150] The embodiment of the present disclosure provides a data streaming server. FIG5 is a schematic diagram of the structure of the data streaming server. As shown in FIG5 , the image-based positioning device 500 includes:

[0151] An acquisition unit 51 is used to acquire a video stream delay;

[0152] The generating unit 52 is configured to generate data push stream information carrying the push stream data, the video stream delay, and a first timestamp when receiving the push stream data corresponding to the first event; the first timestamp is used to indicate the time when the first event occurs;

[0153] The sending unit 53 is used to send the data push streaming information to the client device, so that the client device determines the display timestamp of the push streaming data corresponding to the first event according to the first timestamp and the video stream delay, and displays the push streaming data corresponding to the first event according to the display timestamp.

[0154] As an optional implementation of the embodiment of the present disclosure, the acquisition unit 51 is specifically used to obtain a second timestamp and a third timestamp; the second timestamp is used to represent the relative moment of the second event in the live video broadcast, the second event is the event displayed by the target video frame, and the third timestamp is used to represent the encoding moment of the target video frame; the fourth timestamp is used to represent the occurrence moment of the third event, and the fifth timestamp is used to represent the relative moment of the third event; based on the second timestamp, the fourth timestamp and the fifth timestamp, a sixth timestamp used to represent the occurrence moment of the event displayed by the target video frame is determined; based on the third timestamp and the sixth timestamp, the video stream delay of the video stream is obtained.

[0155] As an optional implementation of the embodiment of the present disclosure, the acquisition unit 51 is specifically configured to determine target object information based on the image corresponding to the target video frame; and acquire the second timestamp according to the target object information.

[0156] As an optional implementation of the embodiment of the present disclosure, the acquisition unit 51 is specifically configured to determine an SEI frame adjacent to the target video frame from the video stream of the live video broadcast; and acquire the third timestamp according to the SEI frame.

[0157] As an optional implementation of the embodiment of the present disclosure, the acquisition unit 51 is specifically configured to acquire the time interval based on the difference between the second timestamp and the fifth timestamp; and acquire the sixth timestamp based on the fourth timestamp and the time interval.

[0158] As an optional implementation of the embodiment of the present disclosure, the obtaining unit 51 is specifically configured to calculate a difference between the third timestamp and the sixth timestamp to obtain the video stream delay.

[0159] As an optional implementation of the embodiment of the present disclosure, the third event is an event corresponding to the streaming data having the shortest time interval with the streaming data corresponding to the first event.

[0160] The present disclosure provides a client device. FIG6 is a schematic diagram of the structure of the client device. As shown in FIG6 , the client device 600 includes:

[0161] The receiving unit 61 is configured to receive data push stream information sent by the data push stream server, wherein the data push stream information includes: push stream data corresponding to a first event, a first timestamp, and a video stream delay, wherein the first timestamp is used to indicate the time when the first event occurs;

[0162] A processing unit 62 is configured to obtain a presentation timestamp of the pushed stream data according to the first timestamp and the video stream delay;

[0163] The display unit 63 is configured to display the streaming data according to the display timestamp.

[0164] As an optional implementation of the embodiment of the present disclosure, the processing unit 62 is specifically configured to sum the first timestamp and the video stream delay to obtain a presentation timestamp of the streaming data corresponding to the first event.

[0165] As an optional implementation of the embodiment of the present disclosure, the display unit 63 is specifically used to obtain a timestamp for representing the encoding moment of the currently displayed video frame; determine whether the timestamp for representing the encoding moment of the currently displayed video frame is greater than or equal to the display timestamp (the encoding moment is later than or equal to the moment corresponding to the display timestamp); if so, display the pushed stream data.

[0166] The data streaming server and client device provided in this embodiment can execute the streaming data display method in live video broadcast provided by the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0167] Based on the same inventive concept, the present disclosure also provides an electronic device. Figure 7 is a schematic diagram of the structure of the electronic device provided by the present disclosure. As shown in Figure 7, the electronic device provided by the present disclosure includes: a memory 701 and a processor 702. The memory 701 is used to store a computer program; the processor 702 is used to execute the method for displaying live streaming data in the live video broadcast provided by the above embodiment when executing the computer program.

[0168] Based on the same inventive concept, an embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the computing device implements the method for displaying streaming data in live video provided in the above embodiment.

[0169] Based on the same inventive concept, an embodiment of the present disclosure further provides a computer program product. When the computer program product is run on a computer, the computing device implements the method for displaying streaming data in live video broadcasting provided in the above embodiment.

[0170] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0171] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0172] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0173] Computer-readable media includes both permanent and non-permanent, removable and non-removable storage media. Storage media can implement any method or technology for storing information, which can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for displaying streaming data in a live video broadcast, comprising: Get video stream delay; When receiving the streaming data corresponding to the first event, generating data streaming information carrying the streaming data, the video stream delay and the first timestamp; The first timestamp is used to represent the time when the first event occurs; The data streaming information is sent to a client device, so that the client device determines a presentation timestamp of the streaming data according to the first timestamp and the video stream delay, and presents the streaming data according to the presentation timestamp.

2. The method according to claim 1, wherein: The obtaining of the video stream delay comprises: Obtaining a second timestamp and a third timestamp; wherein the second timestamp is used to represent a relative moment of a second event in the live video broadcast, the second event being an event displayed by a target video frame, and the third timestamp is used to represent an encoding moment of the target video frame; Obtaining a fourth timestamp and a fifth timestamp; wherein the fourth timestamp is used to represent the time when the third event occurs, and the fifth timestamp is used to represent the relative time of the third event; Determine, according to the second timestamp, the fourth timestamp, and the fifth timestamp, a sixth timestamp for representing a time when the second event occurs; A video stream delay of the video stream is acquired according to the third timestamp and the sixth timestamp.

3. The method according to claim 2, wherein: The second timestamp is obtained, including: Determining target object information based on an image corresponding to the target video frame; The second timestamp is acquired according to the target object information.

4. The method according to claim 2, wherein: Obtaining the third timestamp includes: Determine a supplemental enhancement information SEI frame adjacent to the target video frame from the video stream of the live video; The third timestamp is obtained according to the SEI frame.

5. The method according to claim 2, wherein: The determining a sixth timestamp according to the second timestamp, the fourth timestamp, and the fifth timestamp includes: Based on the second timestamp and the fifth timestamp, obtaining a time interval; Based on the fourth timestamp and the time interval, the sixth timestamp is acquired.

6. The method according to claim 2, wherein: The acquiring the video stream delay according to the second timestamp and the fifth timestamp includes: A difference between the third timestamp and the sixth timestamp is calculated to obtain the video stream delay.

7. The method according to claim 2, wherein: The third event is an event corresponding to the streaming data having the shortest time interval with the streaming data corresponding to the first event.

8. A method for displaying data streaming in a live video broadcast, comprising: Receive data streaming information, where the data streaming information includes: streaming data corresponding to a first event, a first timestamp, and a video stream delay, where the first timestamp is used to represent a time when the first event occurs; Acquire a presentation timestamp of the streaming data according to the first timestamp and the video stream delay; The streaming data is displayed according to the display timestamp.

9. The method according to claim 8, wherein: The acquiring the presentation timestamp of the streaming data according to the first timestamp and the video stream delay includes: The first timestamp and the video stream delay are summed to obtain a presentation timestamp of the pushed stream data.

10. The method according to claim 8, wherein: The displaying of the streaming data according to the display timestamp includes: Get the timestamp representing the encoding moment of the currently displayed video frame; Determine whether the timestamp used to represent the encoding time of the currently displayed video frame is greater than or is equal to the presentation timestamp; If yes, the streaming data is displayed.

11. A data streaming server, comprising: An acquisition unit, used for acquiring a video stream delay; A generating unit, configured to generate data streaming information carrying the streaming data, the video stream delay and the first timestamp when receiving streaming data corresponding to the first event; The first timestamp is used to represent the time when the first event occurs; A sending unit is used to send the data streaming information to a client device so that the client device determines a display timestamp of the streaming data corresponding to the first event according to the first timestamp and the video stream delay, and displays the streaming data corresponding to the first event according to the display timestamp.

12. A client device, comprising: A receiving unit, configured to receive data streaming information sent by a data streaming server, wherein the data streaming information includes streaming data corresponding to a first event, a first timestamp, and a video stream delay, wherein the first timestamp is used to represent a time when the first event occurs; A processing unit, configured to obtain a presentation timestamp of the push stream data according to the first timestamp and the video stream delay; A display unit is used to display the streaming data according to the display timestamp.

13. An electronic device comprising: A memory and a processor, wherein the memory is used to store a computer program; The processor is used to enable the electronic device to implement the data streaming display method in live video broadcasting as described in any one of claims 1-10 when executing a computer program.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a computing device, the computing device implements the data streaming display method in live video broadcasting as described in any one of claims 1-10.

Images