Live broadcast delay determination method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of computers, and discloses a live broadcast demonstration determination method and apparatus, a computer device and a storage medium, the method provided by the invention comprises the steps of sending a first clock synchronization request to a first server node to obtain a first calibration time difference, the first server node being used for processing live broadcast data; calibrating the local time based on the first calibration time difference to obtain calibrated time; a live broadcast code stream generated by the opposite end is obtained from the first server node to obtain a timestamp of the live broadcast code stream and a local current timestamp, the timestamp is determined after clock synchronization is carried out on the opposite end and a second server node, and clock synchronization is carried out between the second server node and the first server node; the live broadcast code stream is transmitted from the second server node to the first server node; and determining live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for determining a live broadcast delay. Background Art

[0002] Video live streaming mainly involves the processes of acquisition, preprocessing, encoding, transmission, server transcoding, and decoding. The end-to-end delay of live streaming has a great impact on the interactive experience between the audience and the anchor. Therefore, it is necessary to accurately determine the delay so as to better optimize the live streaming delay. Summary of the invention

[0003] In view of this, the present disclosure provides a method, apparatus, computer device and storage medium for determining live broadcast delay to solve the problem of determining live broadcast delay.

[0004] In a first aspect, the present disclosure provides a method for determining a live broadcast delay, the method comprising:

[0005] Sending a first clock synchronization request to a first server node to obtain a first calibration time difference, wherein the first server node is used to process live broadcast data;

[0006] Calibrate the local time based on the first calibration time difference to obtain a calibrated time;

[0007] Acquire the live stream generated by the peer end from the first server node to obtain a timestamp of the live stream and a local current timestamp, wherein the timestamp is determined after the peer end and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node;

[0008] The live broadcast delay is determined based on the current timestamp and the timestamp of the live broadcast code stream.

[0009] In a second aspect, the present disclosure provides a method for determining a live broadcast delay, the method comprising:

[0010] Sending a second clock synchronization request to a second server node to obtain a second calibration time difference, wherein the second server node is used to process the live broadcast data;

[0011] Calibrate the local time based on the second calibration time difference to obtain a calibrated time;

[0012] Obtaining live broadcast data to be sent and a timestamp corresponding to the live broadcast data to be sent;

[0013] Generate a live streaming stream based on the timestamp corresponding to the live streaming data to be sent and the live streaming data to be sent;

[0014] The live broadcast code stream is sent to the second server node to reach the opposite end, so as to determine the live broadcast delay at the opposite end, the second server node is used to transmit the live broadcast code stream to the first server node, the first server node is used to transmit the live broadcast code stream to the opposite end, the second server node is clock-synchronized with the first server node, and the opposite end is used to determine the live broadcast delay based on the time after synchronization with the first server node and the timestamp in the live broadcast code stream.

[0015] In a third aspect, the present disclosure provides a device for determining a live broadcast delay, the device comprising:

[0016] A first clock synchronization request sending module, used for sending a first clock synchronization request to a first server node to obtain a first calibration time difference, wherein the first server node is used for processing live broadcast data;

[0017] A first calibration module, configured to calibrate the local time based on the first calibration time difference to obtain a calibrated time;

[0018] A live stream acquisition module is used to acquire the live stream generated by the peer end from the first server node to obtain the timestamp of the live stream and the local current timestamp, wherein the timestamp is determined after the peer end and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node;

[0019] The live broadcast delay determination module is used to determine the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

[0020] In a fourth aspect, the present disclosure provides a device for determining a live broadcast delay, the device comprising:

[0021] A second clock synchronization request sending module, used for sending a second clock synchronization request to a second server node to obtain a second calibration time difference, wherein the second server node is used for processing live broadcast data;

[0022] A second calibration module, configured to calibrate the local time based on the second calibration time difference to obtain a calibrated time;

[0023] A module for acquiring live broadcast data to be sent, used for acquiring live broadcast data to be sent and a timestamp corresponding to the live broadcast data to be sent;

[0024] A live streaming code stream generating module, used for generating a live streaming code stream based on a timestamp corresponding to the live streaming data to be sent and the live streaming data to be sent;

[0025] The live streaming code stream sending module is used to send the live streaming code stream to the second server node to reach the opposite end, so as to determine the live streaming delay at the opposite end. The second server node is used to transmit the live streaming code stream to the first server node, and the first server node is used to transmit the live streaming code stream to the opposite end. The clocks of the second server node and the first server node are synchronized, and the opposite end is used to determine the live streaming delay based on the time after synchronization with the first server node and the timestamp in the live streaming code stream.

[0026] In a fifth aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the method for determining the live broadcast delay of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0027] In a sixth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for determining the live broadcast delay of the above-mentioned first aspect or any corresponding embodiment thereof.

[0028] The method for determining the live broadcast delay provided by the embodiment of the present disclosure uses the server node that processes the live broadcast data as a reference for clock synchronization, and the first server node and the second server node are clock synchronized. Then, after the live broadcast terminals corresponding to the first server node and the second server node are clock synchronized with the corresponding server nodes, the live broadcast terminals are under the same clock coordinates. The live broadcast delay is determined on this basis, which can ensure the accuracy of the obtained live broadcast delay. Moreover, since the above-mentioned clock synchronization is performed based on the server node that processes the live broadcast data, there is no need to introduce an additional clock synchronization server, and accurate clock delay can be obtained without increasing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 is a block diagram of an application scenario of a live broadcast delay system according to an embodiment of the present disclosure;

[0031] Figure 2 is a flow chart of a method for determining a live broadcast delay according to an embodiment of the present disclosure;

[0032] Figure 3 is a flowchart of another method for determining live broadcast delay according to an embodiment of the present disclosure;

[0033] Figure 4 is a schematic diagram of determining a first calibration time difference according to an embodiment of the present disclosure;

[0034] Figure 5 is a flowchart of another method for determining live broadcast delay according to an embodiment of the present disclosure;

[0035] Figure 6 is a schematic diagram of an application scenario of the method for determining live broadcast delay according to an embodiment of the present disclosure;

[0036] Figure 7 is a structural block diagram of a device for determining a live broadcast delay according to an embodiment of the present disclosure;

[0037] Figure 8 is a structural block diagram of another device for determining live broadcast delay according to an embodiment of the present disclosure;

[0038] Fig. 9 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0040] The live broadcast scene includes the host side and the audience side. In the relevant technology, the calculation is made with the help of the Supplemental Enhancement Information (SEI) frame in the video bitstream, that is, the overall live broadcast delay is calculated by adding the timestamp of the host's acquisition moment to the SEI frame and passing it to the audience side with the bitstream. Specifically, when the host side pushes the stream, the host's current system timestamp is recorded in the SEI information of the bitstream. When the audience pulls the stream, the terminal player parses the timestamp from the SEI information of the bitstream and subtracts the timestamp in the SEI from the current system timestamp as the live broadcast delay. The calculation method is: live broadcast delay = current system timestamp of the audience side - timestamp in the SEI information of the video bitstream.

[0041] However, in the above method, the clocks of the host's terminal device and the viewer's terminal device may not be synchronized. If the time of the host's and the viewer's devices differ by 3s at the same time, the calculation result of the live broadcast delay will be inaccurate.

[0042] In other related technologies, a clock synchronization (Network Time Protocol, or NTP) server is introduced or an existing NTP server is rented. However, this method requires additional cost. Therefore, in most scenarios, clock calibration is not performed, but the live broadcast delay is directly calculated.

[0043] Based on this, the method for determining the live broadcast delay provided by the embodiment of the present disclosure integrates clock synchronization with the server node that processes the live broadcast data, uses the existing server node for clock synchronization, and then determines the live broadcast delay. This method achieves accurate calculation of the live broadcast delay by synchronizing the clocks of the host terminal device and the audience terminal device.

[0044] As a live broadcast scenario provided by the embodiment of the present disclosure, both the host-end device and the audience-end device are referred to as live broadcast devices. If the host-end device is used as the local device, then the opposite end is the audience-end device; if the audience-end device is used as the local device, then the opposite end is the host-end device. During the live broadcast process, information can be exchanged between the host-end device and the audience-end device, that is, the host uses the host-end device to broadcast live, and the corresponding code stream is transmitted to the audience-end device, and the audience can interact with the host through the audience-end device. Figure 1 As shown, the host-side device is connected to the server node A, and the live stream of the host is pushed to the content delivery network (CDN) through the server node A. The audience-side device is connected to the server node B, and the live stream is pulled from the CDN through the server node B, and the live stream is played in the audience-side device.

[0045] According to an embodiment of the present disclosure, an embodiment of a method for determining a live broadcast delay is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] In this embodiment, a method for determining a live broadcast delay is provided, which can be used for a computer device, such as a computer, a mobile terminal, etc. In this embodiment, the computer device is a viewer terminal device. Figure 2 is a flow chart of a method for determining a live broadcast delay according to an embodiment of the present disclosure. Figure 2 As shown, the process includes the following steps:

[0047] Step S201: Send a first clock synchronization request to a first server node to obtain a first calibration time difference.

[0048] Among them, the first server node is used to process live broadcast data.

[0049] The first server node is the server node corresponding to the viewer-end device, and the viewer-end device pulls the live stream from the first server node and plays it locally. Moreover, before pulling the stream, the local needs to synchronize the clock with the first server node. Specifically, a first clock synchronization request is sent to the first server node to synchronize the clock with the first server node. The purpose of clock synchronization is to calibrate the local time. Therefore, after sending the first clock synchronization request, the first calibration time difference can be obtained by using the feedback information of the first server node. The first calibration time difference is used to represent the time difference between the local time and the first server node, and the local time is calibrated using the first calibration time difference.

[0050] Step S202: calibrate the local time based on the first calibration time difference to obtain a calibrated time.

[0051] The local time is the current system time of the viewer device. After obtaining the first calibration time difference, the local time is calibrated using the first calibration time difference to obtain the calibrated time. After calibration, the current system time of the viewer device is clock-synchronized with the system time of the first server node.

[0052] The calibration method for the local time includes but is not limited to adding the first calibration time difference to the local time, or subtracting the first calibration time difference, etc. The specific calibration method is related to the calculation method of the first calibration time difference. The specific calculation method is not limited here, and is set according to actual needs.

[0053] Step S203, obtaining the live stream generated by the opposite end from the first server node to obtain the timestamp of the live stream and the local current timestamp.

[0054] The timestamp is determined after the peer node and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node.

[0055] After the viewer-end device performs clock synchronization with the first server-end node, it obtains the live stream generated by the other end from the first server-end node. In this embodiment, the viewer-end device is local, and the other end is the host-end device. As described above, the live stream generated by the host-end device is transmitted to the CDN through the second server-end node corresponding to the host-end device, and then transmitted to the first server-end node by the CDN. The viewer-end device pulls the stream from the first server-end node to obtain the live stream.

[0056] The live stream includes a timestamp of the live stream, which is the timestamp of sending the live stream, or the timestamp of generating the live stream. It should be noted that the opposite end performs clock synchronization with the second server node before the live broadcast, and its clock synchronization method is similar to the local clock synchronization method, which will not be repeated here. Because multiple server nodes are deployed before the live broadcast application can be used, some logs or messages need to be shared between these server nodes, so these server nodes are clock synchronized. That is, the first server node and the second server node are clock synchronized.

[0057] Since the first server node and the second server node are clock-synchronized, the viewer-end device is clock-synchronized with the first server node, and the host-end device is clock-synchronized with the second server node, then the viewer-end device and the host-end device are also clock-synchronized.

[0058] The local current timestamp is the timestamp of obtaining the live stream. Specifically, after the local side pulls the live stream from the first server node, the local current timestamp is recorded. In the absence of delay, the timestamp of the live stream is consistent with the local current timestamp. However, due to the processing of push streaming, network transmission and pull streaming, delays will occur. That is, the timestamp of the live stream is inconsistent with the local current timestamp.

[0059] Step S204, determining the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

[0060] After the clock is synchronized, the local current timestamp lags behind the timestamp of the live stream. Therefore, the live delay can be determined by using the difference between the current timestamp and the timestamp of the live stream.

[0061] In some optional implementations, after obtaining the live broadcast delay, the viewer device can report the live broadcast delay with the authorization of the user, so that the server node corresponding to the live broadcast application can count the live broadcast delay, for example, calculate the average value of the live broadcast delay, the maximum live broadcast delay, etc. After obtaining the statistical result of the live broadcast delay, the live broadcast application is optimized and updated based on the statistical result to shorten the live broadcast delay.

[0062] The method for determining the live broadcast delay provided in the present embodiment uses the server node that processes the live broadcast data as a reference for clock synchronization, and the first server node and the second server node are clock synchronized. Then, after the live broadcast terminals corresponding to the first server node and the second server node are clock synchronized with the corresponding server nodes, the live broadcast terminals are under the same clock coordinates. The live broadcast delay is determined on this basis, which can ensure the accuracy of the obtained live broadcast delay. Moreover, since the above-mentioned clock synchronization is performed based on the server node that processes the live broadcast data, there is no need to introduce an additional clock synchronization server, and accurate clock delay can be obtained without increasing costs.

[0063] In this embodiment, a method for determining a live broadcast delay is provided, which can be used in computer devices, such as computers, mobile terminals, etc. Figure 3 is a flow chart of a method for determining a live broadcast delay according to an embodiment of the present disclosure. Figure 3 As shown, the process includes the following steps:

[0064] Step S301: Send a first clock synchronization request to a first server node to obtain a first calibration time difference.

[0065] Among them, the first server node is used to process live broadcast data.

[0066] Specifically, the above step S301 includes:

[0067] Step S3011: Send a first clock synchronization request to the first server node.

[0068] The first clock synchronization request includes a first timestamp for sending the clock synchronization request.

[0069] The first clock synchronization request is sent to the first server node. The format of the first clock synchronization request can be based on the format of the NTP request, or the format of the APP request, etc. There is no limitation on the specific format of the first clock synchronization request, and it is set according to actual needs. It should be noted that the first clock synchronization request can be a clock synchronization control provided on the interface of the live broadcast application, and the sending of the first clock synchronization request is triggered by interacting with the control; it can also be a method of automatically triggering the first clock synchronization request. The timing of automatic triggering can be after starting the live broadcast application, or before the live broadcast starts, etc. The specific timing is set according to actual needs.

[0070] In some optional implementations, the above step S3011 includes:

[0071] Step a1, obtaining a start instruction of a live broadcast application to start the live broadcast application.

[0072] Step a2: After the live broadcast application is started, trigger the local to send a first clock synchronization request to the first server node.

[0073] The user generates a start instruction by interacting with the viewer device, and accordingly obtains the start instruction of the live broadcast application, and starts the live broadcast application by executing the start instruction. After the live broadcast application is started, the sending of the first clock synchronization request is automatically triggered, that is, the local sends the first clock synchronization request to the first server node.

[0074] After the live broadcast application is started, the clock synchronization with the first server node is automatically triggered, which simplifies the processing process and automatically realizes the clock synchronization without the user's perception, thereby improving the user experience.

[0075] Step S3012: Receive a response message fed back by the first server node, and determine a fourth timestamp of the response message.

[0076] The response message carries a second timestamp when the first clock synchronization request arrives at the first server node and a third timestamp when the first server node sends the response message.

[0077] After sending the first clock synchronization request to the first server node, the first server node feeds back a corresponding response message based on the first clock synchronization request. Specifically, the response message carries multiple timestamps, including a second timestamp when the first clock synchronization request arrives at the first server node and a third timestamp when the first server node sends the response message. Among them, after receiving the first clock synchronization request, the first server node records the receiving time to obtain the second timestamp; the second server node sends a response message, and records the sending time of the response message to obtain the third timestamp, and both the second timestamp and the third timestamp are encapsulated in the response message, and fed back with the response message.

[0078] When receiving the response message, the viewer end device records the receiving time of the response message to obtain the fourth timestamp. Therefore, the viewer end device can obtain the first timestamp, the second timestamp, the third timestamp and the fourth timestamp.

[0079] Step S3013: determine a first calibration time difference based on the first timestamp, the second timestamp, the third timestamp and the fourth timestamp.

[0080] like Figure 4 As shown, the round-trip delay of the message from the viewer device to the first server node is delay=2t, where t is the one-way delay, i.e., delay=(t4-t1)-(t3-t2). Among them, t4 is the fourth timestamp, t3 is the third timestamp, t2 is the second timestamp, and t1 is the first timestamp.

[0081] The first calibration time difference between the viewer device and the first server node is offset, that is:

[0082]

[0083] Therefore, the first calibration time difference offset is obtained by the above formula:

[0084]

[0085] Step S302: calibrate the local time based on the first calibration time difference to obtain a calibrated time.

[0086] After obtaining the first calibration time difference offset, the sum of the first calibration time difference and the local time is determined as the calibrated time.

[0087] Since the calculation method of the first calibration time difference represents the delay difference between the local node and the first server node, the calibrated time can be obtained by adding the local time after obtaining the first calibration time difference.

[0088] Step S303: Acquire the live stream generated by the opposite end from the first server node to obtain the timestamp of the live stream and the local current timestamp.

[0089] The timestamp is determined after the peer node and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node.

[0090] Specifically, the above step S303 includes:

[0091] Step S3031, obtaining the live stream generated by the opposite end from the first server node, and determining the current timestamp.

[0092] The viewer-end device pulls the live stream from the first server-end node and records the current timestamp of the live stream, wherein the current timestamp is obtained based on the calibrated time.

[0093] Step S3032: Analyze the live stream to determine the supplementary enhancement information frame in the live stream.

[0094] The timestamp of the live stream is written into the SEI frame. After the live stream is pulled, it is parsed to determine the SEI frame.

[0095] Step S3033: extract the timestamp of the live stream from the supplementary enhancement information frame.

[0096] After the SEI frame is determined, the timestamp of the live stream is extracted from the SEI frame. The SEI frame may also include other information for other needs, which is not limited here.

[0097] Step S304: Determine the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

[0098] Specifically, the live broadcast delay is determined based on the difference between the current timestamp and the timestamp of the live broadcast code stream, that is, the live broadcast delay = the current timestamp - the timestamp of the live broadcast code stream.

[0099] The method for determining the live broadcast delay provided in this embodiment determines the first calibration time difference by sending a first clock synchronization request, receiving a feedback response message and the respective corresponding timestamps. Since the round-trip time between the local node and the first server node is fixed, the first calibration time difference can be determined quickly and accurately in combination with the timestamp. Since the supplementary enhancement information frame is a frame used to write corresponding information according to demand, the timestamp for generating the live broadcast stream is included in the supplementary enhancement information frame of the live broadcast stream, which simplifies the timestamp transmission method. Since the live broadcast delay represents the time difference between the local and the opposite end, the difference between the local current timestamp and the timestamp of the live broadcast stream is used as the live broadcast delay, which can intuitively represent the above time difference without conversion.

[0100] In this embodiment, a method for determining a live broadcast delay is provided, which can be used in computer devices, such as computers, mobile terminals, etc. Figure 5 is a flow chart of a method for determining a live broadcast delay according to an embodiment of the present disclosure. Figure 5 As shown, the process includes the following steps:

[0101] Step S501: Send a second clock synchronization request to a second server node to obtain a second calibration time difference.

[0102] The second server node is used to process live broadcast data.

[0103] The method for obtaining the second calibration time difference is similar to the method for obtaining the first calibration time difference, except that the second calibration time difference is obtained after the host end device sends a second clock synchronization request to the second server end node, while the first calibration time difference is obtained after the viewer end device sends a first clock synchronization request to the first server end node. For the method for obtaining the second calibration time difference, please refer to the detailed description of the method for obtaining the first calibration time difference above, which will not be repeated here.

[0104] Step S502: calibrate the local time based on the second calibration time difference to obtain the calibrated time.

[0105] Similar to the time calibration of the viewer end device, after obtaining the second calibration time difference, the local time of the host end device is calibrated using the second calibration time difference to obtain the local calibrated time, wherein the local time before calibration is the current system time of the host end device.

[0106] Step S503, obtaining the live broadcast data to be sent and the timestamp corresponding to the live broadcast data to be sent.

[0107] After the host device generates the live data to be sent, it obtains the timestamp of the live data to be sent. As mentioned above, the timestamp can be the timestamp of the subsequent sending of the live stream, or the timestamp of the generation of the live data to be sent.

[0108] Step S504: Generate a live stream based on the timestamp corresponding to the live data to be sent and the live data to be sent.

[0109] The timestamp corresponding to the live data to be sent is encapsulated with the live data to be sent to generate a live stream. The timestamp corresponding to the live data to be sent can be sent using a separate data frame, for example, attached to the frame header of the live data to be sent, or can be sent using an SEI frame, etc. The specific sending method of the timestamp corresponding to the live data to be sent is set according to actual needs and is not limited here.

[0110] In some optional implementations, step S504 includes:

[0111] Step b1, determining the supplementary enhancement information frame in the live broadcast data to be sent.

[0112] Step b2, writing the timestamp corresponding to the live data to be sent into the supplementary enhancement information frame to generate a live stream.

[0113] The position of the SEI frame is determined in the live data to be sent, the timestamp corresponding to the live data to be sent is written into the SEI frame, and the SEI frame is encapsulated together with the live data to be sent to generate a live stream.

[0114] Step S505, sending the live broadcast code stream to the second server node to reach the opposite end, so as to determine the live broadcast delay at the opposite end.

[0115] Among them, the second server node is used to transmit the live broadcast code stream to the first server node, the first server node is used to transmit the live broadcast code stream to the opposite end, the clock between the second server node and the first server node is synchronized, and the opposite end is used to determine the live broadcast delay based on the time after synchronization with the first server node and the timestamp in the live broadcast code stream.

[0116] The live stream is first transmitted to the second server node, reaches the CDN, and then transmitted by the CDN to the first server node. The viewer device pulls the stream from the first server node to obtain the live stream. The specific method of determining the live delay after the viewer device obtains the live stream and its corresponding timestamp is described above and will not be repeated here.

[0117] The method for determining the live broadcast delay provided in the present embodiment uses the server node that processes the live broadcast data as a reference for clock synchronization, and the first server node and the second server node are clock synchronized. Then, after the live broadcast terminals corresponding to the first server node and the second server node are clock synchronized with the corresponding server nodes, the live broadcast terminals are under the same clock coordinates. The live broadcast delay is determined on this basis, which can ensure the accuracy of the obtained live broadcast delay. Moreover, since the above-mentioned clock synchronization is performed based on the server node that processes the live broadcast data, there is no need to introduce an additional clock synchronization server, and accurate clock delay can be obtained without increasing costs.

[0118] As a specific application example of the present disclosure, Figure 6 As shown, taking the live broadcast of the anchor end as an example, the anchor end device and the audience end device are collectively referred to as live broadcast terminals. The anchor starts the live broadcast application by interacting with the live broadcast application in the anchor end device. After the live broadcast application is started, the anchor requests the application programming interface (Application Programming Interface, referred to as API) from the server node A while performing clock synchronization, and performs clock synchronization with the server node A to calibrate the time of the anchor end device to obtain the calibrated time. Before the live broadcast starts, the anchor end device and the server node A also interact to allocate the live broadcast room number, etc. The live broadcast code stream and its timestamp generated after the anchor starts the live broadcast are pushed to the server node A together, and uploaded to the CDN via the server node A. The audience of the live broadcast starts the live broadcast application by interacting with the audience end device. At this time, the audience end device and the server node B perform clock synchronization, calibrate the time of the audience end device, and obtain the calibrated time. The audience interacts with the live broadcast application and selects the live broadcast room number they want to enter. Accordingly, the stream is pulled from the server node B based on the live broadcast room number. The server node B obtains the corresponding live broadcast stream from the CDN based on the live broadcast room number and transmits it to the audience device for playback in the audience device. At this time, the audience device uses the difference between the timestamp in the live broadcast stream and the current timestamp to determine the live broadcast delay. The audience device can upload the live broadcast delay to the server node B so that the server node B can analyze the status of the live broadcast delay and obtain the analysis results, thereby optimizing the live broadcast application or giving corresponding optimization suggestions.

[0119] In this embodiment, a device for determining a live broadcast delay is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0120] This embodiment provides a device for determining a live broadcast delay, such as Figure 7 As shown, including:

[0121] The first clock synchronization request sending module 701 is used to send a first clock synchronization request to a first server node to obtain a first calibration time difference. The first server node is used to process live broadcast data.

[0122] The first calibration module 702 is configured to calibrate the local time based on the first calibration time difference to obtain a calibrated time.

[0123] The live stream acquisition module 703 is used to acquire the live stream generated by the opposite end from the first server node to obtain the timestamp of the live stream and the local current timestamp. The timestamp is determined after the clock of the opposite end is synchronized with the second server node. The clocks of the second server node and the first server node are synchronized, and the live stream is transmitted from the second server node to the first server node.

[0124] The live broadcast delay determination module 704 is used to determine the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

[0125] In some optional implementations, the first clock synchronization request sending module 701 includes:

[0126] The first request sending unit is used to send a first clock synchronization request to the first service end node, where the first clock synchronization request includes a first timestamp for sending the clock synchronization request.

[0127] A response message receiving unit is used to receive a response message fed back by the first service end node and determine a fourth timestamp of receiving the response message, wherein the response message carries a second timestamp of the first clock synchronization request arriving at the first service end node and a third timestamp of the first service end node sending the response message.

[0128] The first calibration time difference determining unit is configured to determine a first calibration time difference based on the first timestamp, the second timestamp, the third timestamp and the fourth timestamp.

[0129] In some optional implementations, the first request sending unit includes:

[0130] The start instruction acquisition subunit is used to acquire the start instruction of the live broadcast application to start the live broadcast application. The request triggering subunit is used to trigger the local to send a first clock synchronization request to the first server node after the live broadcast application is started.

[0131] In some optional implementations, the live stream acquisition module 703 includes:

[0132] The live stream acquisition unit is used to acquire the live stream generated by the opposite end from the first server node and determine the current timestamp.

[0133] The live stream analysis unit is used to analyze the live stream and determine the supplementary enhancement information frame in the live stream.

[0134] The timestamp extraction unit is used to extract the timestamp of the live stream from the supplementary enhancement information frame.

[0135] In some optional implementations, the first calibration module 702 includes:

[0136] The first calibration unit is used to determine the sum of the first calibration time difference and the local time as the calibrated time.

[0137] In some optional implementations, the live broadcast delay determination module 704 includes:

[0138] The live broadcast delay determination unit is used to determine the live broadcast delay based on the difference between the current timestamp and the timestamp of the live broadcast code stream.

[0139] This embodiment provides a device for determining a live broadcast delay, such as Figure 8 As shown, including:

[0140] The second clock synchronization request sending module 801 is used to send a second clock synchronization request to a second server node to obtain a second calibration time difference. The second server node is used to process live broadcast data.

[0141] The second calibration module 802 is used to calibrate the local time based on the second calibration time difference to obtain the calibrated time.

[0142] The module 803 for acquiring the live broadcast data to be sent is used to acquire the live broadcast data to be sent and the timestamp corresponding to the live broadcast data to be sent.

[0143] The live streaming code stream generating module 804 is used to generate a live streaming code stream based on the timestamp corresponding to the live streaming data to be sent and the live streaming data to be sent.

[0144] The live streaming code stream sending module 805 is used to send the live streaming code stream to the second server node to reach the other end, so as to determine the live streaming delay at the other end. The second server node is used to transmit the live streaming code stream to the first server node. The first server node is used to transmit the live streaming code stream to the other end. The clocks of the second server node and the first server node are synchronized. The other end is used to determine the live streaming delay based on the time after synchronization with the first server node and the timestamp in the live streaming code stream.

[0145] In some optional implementations, the live stream generation module 804 includes:

[0146] A SEI frame determination unit, configured to determine a supplementary enhancement information frame in the live broadcast data to be sent;

[0147] A timestamp writing unit is used to write the timestamp corresponding to the live broadcast data to be sent into the supplementary enhancement information frame to generate the live broadcast code stream.

[0148] The device for determining the live broadcast delay in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0149] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0150] The present disclosure also provides a computer device having the above Figure 7 or Figure 8 The device for determining the live broadcast delay is shown.

[0151] See also Fig. 9 , Fig. 9 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Fig. 9As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0152] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0153] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0154] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0155] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0156] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0157] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0158] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0159] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0160] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0161] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0162] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0163] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for determining live broadcast delay, characterized in that: The method comprises: Sending a first clock synchronization request to a first server node to obtain a first calibration time difference, wherein the first server node is used to process live broadcast data; Calibrate the local time based on the first calibration time difference to obtain a calibrated time; Acquire the live stream generated by the peer end from the first server node to obtain a timestamp of the live stream and a local current timestamp, wherein the timestamp is determined after the peer end and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node; The live broadcast delay is determined based on the current timestamp and the timestamp of the live broadcast code stream.

2. The method according to claim 1, characterized in that The sending a first clock synchronization request to the first server node to obtain a first calibration time difference includes: Sending a first clock synchronization request to the first server node, the first clock synchronization request comprising a first timestamp for sending the clock synchronization request; Receive a response message fed back by the first server node, and determine a fourth timestamp of receiving the response message, wherein the response message carries a second timestamp of when the first clock synchronization request arrives at the first server node and a third timestamp of when the first server node sends the response message; The first calibration time difference is determined based on the first timestamp, the second timestamp, the third timestamp, and the fourth timestamp.

3. The method according to claim 2, characterized in that The sending a first clock synchronization request to the first server node includes: Obtaining a start instruction of a live broadcast application to start the live broadcast application; After the live broadcast application is started, the local device is triggered to send the first clock synchronization request to the first server node.

4. The method according to claim 1, characterized in that: The acquiring the live stream generated by the opposite end from the first server node to obtain the timestamp of the live stream and the local current timestamp includes: Obtaining the live stream generated by the peer end from the first server node, and determining the current timestamp; Analyze the live code stream to determine the supplementary enhancement information frame in the live code stream; The timestamp of the live stream is extracted from the supplemental enhancement information frame.

5. The method according to claim 1, characterized in that The step of calibrating the local time based on the first calibration time difference to obtain the calibrated time includes: The sum of the first calibration time difference and the local time is determined as the calibrated time.

6. The method according to any one of claims 1 to 5, characterized in that The determining of the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream includes: The live broadcast delay is determined based on a difference between the current timestamp and the timestamp of the live broadcast code stream.

7. A method for determining live broadcast delay, characterized in that: The method comprises: Sending a second clock synchronization request to a second server node to obtain a second calibration time difference, wherein the second server node is used to process the live broadcast data; Calibrate the local time based on the second calibration time difference to obtain a calibrated time; Obtaining live broadcast data to be sent and a timestamp corresponding to the live broadcast data to be sent; Generate a live streaming stream based on the timestamp corresponding to the live streaming data to be sent and the live streaming data to be sent; The live broadcast code stream is sent to the second server node to reach the opposite end, so as to determine the live broadcast delay at the opposite end, the second server node is used to transmit the live broadcast code stream to the first server node, the first server node is used to transmit the live broadcast code stream to the opposite end, the second server node is clock-synchronized with the first server node, and the opposite end is used to determine the live broadcast delay based on the time after synchronization with the first server node and the timestamp in the live broadcast code stream.

8. The method according to claim 7, characterized in that The generating a live stream based on the timestamp corresponding to the live data to be sent and the live data to be sent includes: Determining a supplementary enhancement information frame in the live broadcast data to be sent; The timestamp corresponding to the live data to be sent is written into the supplementary enhancement information frame to generate the live stream.

9. A device for determining live broadcast delay, characterized in that: The device comprises: A first clock synchronization request sending module, used for sending a first clock synchronization request to a first server node to obtain a first calibration time difference, wherein the first server node is used for processing live broadcast data; A first calibration module, configured to calibrate the local time based on the first calibration time difference to obtain a calibrated time; A live stream acquisition module is used to acquire the live stream generated by the peer end from the first server node to obtain the timestamp of the live stream and the local current timestamp, wherein the timestamp is determined after the peer end and the second server node perform clock synchronization, the second server node and the first server node are clock synchronized, and the live stream is transmitted from the second server node to the first server node; The live broadcast delay determination module is used to determine the live broadcast delay based on the current timestamp and the timestamp of the live broadcast code stream.

10. A device for determining live broadcast delay, characterized in that: The device comprises: A second clock synchronization request sending module, used for sending a second clock synchronization request to a second server node to obtain a second calibration time difference, wherein the second server node is used for processing live broadcast data; A second calibration module, configured to calibrate the local time based on the second calibration time difference to obtain a calibrated time; A module for acquiring live broadcast data to be sent, used for acquiring live broadcast data to be sent and a timestamp corresponding to the live broadcast data to be sent; A live streaming code stream generating module, used for generating a live streaming code stream based on a timestamp corresponding to the live streaming data to be sent and the live streaming data to be sent; The live streaming code stream sending module is used to send the live streaming code stream to the second server node to reach the opposite end, so as to determine the live streaming delay at the opposite end. The second server node is used to transmit the live streaming code stream to the first server node, and the first server node is used to transmit the live streaming code stream to the opposite end. The clocks of the second server node and the first server node are synchronized, and the opposite end is used to determine the live streaming delay based on the time after synchronization with the first server node and the timestamp in the live streaming code stream.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for determining the live broadcast delay according to any one of claims 1 to 8 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for determining the live broadcast delay according to any one of claims 1 to 8.

Images