Live broadcast synchronization control method and system

Abstract

The invention discloses a live broadcast synchronization control method and system. The method comprises the following steps: a target client receives live broadcast data comprising a video frame and an audio frame sent by a server based on a CDN (Content Delivery Network); periodically determining an NTP-based coding timestamp when the coding of the current video frame is completed, and determining a self target receiving time delay according to the coding timestamp and the self current local time; the target receiving time delay is sent to a synchronization control module based on the RTC channel, a target synchronization time delay issued by the synchronization control module based on the RTC channel is received, and the target synchronization time delay is determined by the synchronization control module according to receiving time delays of multiple clients in a virtual room where the target client is located; and determining a playing strategy according to the deviation between the target synchronization time delay and the target receiving time delay, and playing the live broadcast data in the current period according to the playing strategy. According to the method and the device, the technical problem that the watching experience of a user is influenced due to asynchronous live broadcast progresses among multiple clients under different network conditions is solved.

Description

Technical Field

[0001] This application relates to the field of streaming media transmission technology, and more specifically, to a live streaming synchronization control method and system. Background Technology

[0002] With the rapid development of internet technology, users are increasingly demanding high-quality, low-latency real-time interactive experiences in interactive live streaming scenarios. Especially in social viewing scenarios like "watching together," the integrated application of CDN (Content Delivery Network) and RTC (Real-Time Communication) technologies has become a hot topic in the industry.

[0003] CDNs, with their distributed nodes and efficient caching mechanisms, can support simultaneous viewing by a large number of users, making them particularly suitable for high-bandwidth services such as live video streaming. However, the latency characteristics of CDNs (typically 3-15 seconds) mean that viewers do not receive the live stream instantly, which is insufficient for interactive scenarios requiring immediate feedback, such as real-time chat between viewers or synchronization issues in game interactions. RTC technology, with its low latency (less than 400ms) and bidirectional communication capabilities, is well-suited for enabling real-time interaction, such as online meetings, online education, and multiplayer games. However, the image quality of RTC streams is generally inferior to that of CDN streams, and its point-to-point communication method struggles to handle scenarios with large-scale concurrent viewing.

[0004] While combining these two technologies can solve the problem of balancing large-scale concurrency and low latency in interactive live streaming, it also introduces new challenges: "spoilers" and "interactive glitches." The former occurs when viewer A sees an event a few seconds before viewer B, such as the moment of a goal, and shares it via the RTC channel, ruining the latter's sense of surprise. The latter occurs when the host's questions and the viewer's answers cannot correspond in real time, resulting in a decreased interactive experience.

[0005] Among the related solutions, those relying on player progress, local system time, or video PTS (Presentation Time Stamp) / DTS (Decoding Time Stamp) information all have significant shortcomings. Player progress is affected by caching, and local time is prone to inaccuracies; video timestamps may be reset during CDN processing, losing their absolute time reference value; simple SEI (Supplemental Enhancement Information) embedding can assist in intra-stream element synchronization, but it cannot solve the frame-level synchronization problem between multiple devices.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This application provides a live streaming synchronization control method and system to at least solve the technical problem of asynchronous live streaming progress among multiple clients under different network conditions, which affects the user's viewing experience.

[0008] According to one aspect of the embodiments of this application, a live streaming synchronization control method applied to a target client is provided, comprising: receiving live streaming data sent by a server based on a content delivery network, wherein the live streaming data includes video frames and audio frames; periodically determining a first encoded timestamp of the current video frame, and determining a target reception delay of the target client based on the first encoded timestamp and the current local time of the target client, wherein the first encoded timestamp is a timestamp indicating the completion of video frame encoding based on the Network Time Protocol; sending the target reception delay to a synchronization control module based on a real-time communication channel, and receiving a target synchronization delay sent by the synchronization control module based on the real-time communication channel, wherein the target synchronization delay is determined by the synchronization control module based on the reception delays of multiple clients in a target virtual room to which the target client belongs; determining a playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, and playing the live streaming data according to the playback strategy within the current period.

[0009] Optionally, periodically determining the first encoded timestamp of the current video frame includes: if the current video frame carries supplemental enhancement information, parsing the supplemental enhancement information to obtain the first encoded timestamp of the current video frame; if the current video frame does not carry supplemental enhancement information, determining the nearest historical video frame carrying supplemental enhancement information, and predicting the first encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of the historical video frame and the first display timestamp of the current video frame.

[0010] Optionally, determining the target reception delay of the target client based on the first encoded timestamp and the current local time of the target client includes: obtaining the clock offset between the local clock of the target client and the standard network clock; determining the first sum of the current local time of the target client and the clock offset, and determining the first difference between the first sum and the first encoded timestamp, and using the first difference as the target reception delay of the target client.

[0011] Optionally, the target reception delay is sent to the synchronization control module based on the real-time communication channel, and the target synchronization delay sent by the synchronization control module based on the real-time communication channel is received. This includes: sending the target reception delay to the synchronization control module based on the real-time communication channel, wherein the synchronization control module runs on the server or any client in the target virtual room; and receiving the target synchronization delay sent by the synchronization control module based on the real-time communication channel, wherein the target synchronization delay is determined by the synchronization control module based on the maximum value or target quantile among the reception delays of multiple clients in the target virtual room, or the target synchronization delay is a preset delay set by the synchronization control module.

[0012] Optionally, the playback strategy is determined based on the delay deviation between the target synchronization delay and the target reception delay, including: if the delay deviation is less than a first preset delay threshold, determining the playback strategy for the current period as normal speed playback; if the delay deviation is not less than the first preset delay threshold and less than a second preset delay threshold, determining the playback strategy for the current period as video speed-changing combined with audio speed-changing without pitch change, and determining the playback speed adjustment coefficient based on the delay deviation; if the delay deviation is not less than the second preset delay threshold, determining the playback strategy for the current period as frame skipping playback.

[0013] Optionally, the live data is played according to a playback strategy within the current period, including: when the playback strategy is original speed playback, the live data is played at a preset playback speed within the current period; when the playback strategy is video speed-changing combined with audio speed-changing without pitch change playback, the preset playback speed is adjusted according to the playback speed adjustment coefficient to obtain the target playback speed, and the audio frames in the live data within the current period are processed using a time scaling algorithm combined with the target playback speed, and the video frames and processed audio frames in the live data are played at the target playback speed within the current period; when the playback strategy is frame skipping playback, the target video frame and the corresponding target audio frame are determined with the encoded timestamp being the first sum value and the second sum value of the target synchronization delay, and the live data is played at the preset playback speed starting from the target video frame and the target audio frame within the current period.

[0014] Optionally, predicting the first encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of historical video frames and the first display timestamp of the current video frame includes: predicting the first encoded timestamp of the current video frame according to the following formula: In the formula, , These represent the first and second encoded timestamps, respectively. , These represent the first and second display timestamps, respectively. This is the cumulative time error for periodic updates. This is a periodically updated adjustment factor related to playback speed, assuming the historical video frames are from the first period. , , This indicates the preset playback speed.

[0015] According to another aspect of the embodiments of this application, a live streaming synchronization control method applied to a server is also provided, comprising: obtaining live streaming data from a live streaming source, wherein the live streaming data includes video frames and audio frames; adding supplementary enhancement information to key video frames in the live streaming data, and adding supplementary enhancement information to non-key video frames in the live streaming data according to a preset frame interval, wherein the supplementary enhancement information includes: a first encoded timestamp based on the network time protocol when the encoding of the corresponding video frame is completed; and sending the live streaming data to multiple clients in a target virtual room via a content delivery network.

[0016] According to another aspect of the embodiments of this application, a live streaming synchronization control method is also provided, which is applied to a synchronization control module. The synchronization control module runs on a server or any client in a target virtual room. The method includes: periodically receiving reception delays sent by multiple clients in the target virtual room based on a real-time communication channel, wherein the reception delay is determined by each client based on a first encoding timestamp based on the Network Time Protocol when the video frame encoding in the received live data is completed and its own local time for receiving the video frame; determining a target synchronization delay based on the reception delays of multiple clients; and sending the target synchronization delay to multiple clients based on the real-time communication channel, wherein the target synchronization delay is used to guide each client to determine the playback strategy for the current period.

[0017] According to another aspect of the embodiments of this application, a live streaming synchronization control system is also provided, including: a server, multiple clients within a target virtual room, and a synchronization control module running on the server or any of the clients. The server is configured to: acquire live streaming data from a live streaming source, the live streaming data including video frames and audio frames; add supplementary enhancement information to key video frames in the live streaming data, and add supplementary enhancement information to non-key video frames in the live streaming data at preset frame intervals, the supplementary enhancement information including: a first encoded timestamp based on the Network Time Protocol (NTP) when the corresponding video frame encoding is completed; and send the live streaming data to multiple clients via a content delivery network (CDN). Each client... It is used to periodically determine the first encoded timestamp of the current video frame, and determine its own target reception delay based on the first encoded timestamp and its own current local time; it sends the target reception delay to the synchronization control module based on the real-time communication channel, and receives the target synchronization delay sent by the synchronization control module based on the real-time communication channel. The target synchronization delay is determined by the synchronization control module based on the reception delay of multiple clients; it determines the playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, and plays live data according to the playback strategy in the current period. The playback strategy includes one of the following: original speed playback, video speed change combined with audio speed change without pitch change playback, and frame skipping playback.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product comprising: a computer program, wherein the computer program, when executed by a processor, implements the above-described live streaming synchronization control method.

[0019] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described live streaming synchronization control method through the computer program.

[0020] In this embodiment, the server adds supplementary enhancement information to the video frames of the live stream data to carry the encoding timestamp based on the Network Time Protocol (NTP) when the video frame encoding is completed. The live stream data is then sent to multiple clients in the target virtual room via a content delivery network (CDN). Each client determines its absolute reception latency based on the encoding timestamp of the received video frame and its local time, and sends its absolute reception latency to the synchronization control module via a real-time communication channel. The synchronization control module makes collaborative decisions based on the absolute reception latency of multiple clients to determine the target synchronization latency, and then sends the target synchronization latency to each client via the real-time communication channel. All clients adjust their playback strategies based on the latency deviation between their own absolute reception latency and the target synchronization latency to achieve synchronized playback of live stream data across multiple clients, thereby ensuring a good viewing experience for users. This solution effectively solves the technical problem of asynchronous live stream progress among multiple clients under different network conditions, which affects the user's viewing experience. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a schematic diagram of an optional live streaming synchronization control system according to an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating an optional live streaming synchronization control method according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the flow of another optional live streaming synchronization control method according to an embodiment of this application;

[0025] Figure 4 This is a flowchart illustrating another optional live streaming synchronization control method according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:

[0030] Content Delivery Network (CDN): A distributed network architecture that accelerates the transmission of data and multimedia content over the internet by deploying server nodes globally, providing more efficient and lower-latency content delivery services. CDNs use caching mechanisms to pre-store popular content on nodes closest to users. When a user requests content, the CDN delivers it from the node geographically closest to the user, rather than fetching it directly from the original server, thus reducing data transmission distance and time.

[0031] Real-time communication: Technologies that enable real-time two-way communication between users over the internet, including voice calls, video calls, and instant text messaging. WebRTC is an open-source project that provides browser-level real-time communication capabilities and is one of the main implementations of RTC technology, supporting low-latency audio and video communication.

[0032] Supplemental Enhancement Information (SEI) is a type of metadata in the H.264 / MPEG-4 AVC and H.265 / HEVC video coding standards. It carries additional information unrelated to the video coding itself, such as camera motion data and copyright management information. SEI information can be read by the decoder but does not affect the video decoding process. It is used to enhance certain video characteristics or support special functions, such as timestamp synchronization in this case.

[0033] Network Time Protocol (NTP): A network protocol for clock synchronization in a distributed clock environment. NTP allows computers to correct their system clocks to obtain a consistent time baseline, which is essential for services that require time synchronization, such as financial transactions, logging, and synchronization control.

[0034] Display Timestamp: A timestamp within a multimedia file or stream indicates when the player should display the corresponding media segment. During video playback, PTS is used to ensure synchronization between video and audio, as well as time alignment between different streams within the video (such as the main video stream and secondary subtitle stream).

[0035] Keyframes: In video coding, a keyframe is an image frame that can be decoded independently. In the H.264 / AVC video coding standard, the IDR (Intra-DCt Refresh) frame is the most typical keyframe. It contains no prediction information from previous or subsequent frames and relies entirely on its own pixel data for encoding and decoding. IDR frames provide a dependency-free decoding point for the video stream, meaning that any decoder can start decoding the video from an IDR frame without needing to know the data of previous video frames.

[0036] Example 1

[0037] To address the technical problem in traditional live streaming interactive solutions where asynchronous live streaming progress among multiple clients under different network conditions negatively impacts the user viewing experience, this application provides a novel live streaming synchronization control system architecture, such as... Figure 1 As shown, the system includes: a server 11, multiple clients 12 (1~n) in the target virtual room, and a synchronization control module 13 running on the server or any of the clients.

[0038] The server can be a relay node in a content delivery network, and the client can be a user's mobile phone, tablet, computer, or other terminal device.

[0039] On the server side, live data can be obtained from the live source, which includes video frames and audio frames; supplementary enhancement information can be added to key video frames in the live data, and supplementary enhancement information can be added to non-key video frames in the live data according to preset frame intervals. The supplementary enhancement information includes: the first encoding timestamp based on the network time protocol when the corresponding video frame encoding is completed; and the live data can be sent to multiple clients based on the content delivery network.

[0040] Each client can periodically determine the first encoded timestamp of the current video frame and determine its own target reception delay based on the first encoded timestamp and its own current local time; it sends the target reception delay to the synchronization control module based on the real-time communication channel, and receives the target synchronization delay sent by the synchronization control module based on the real-time communication channel. The target synchronization delay is determined by the synchronization control module based on the reception delays of multiple clients; it determines the playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, and plays live data according to the playback strategy within the current period. The playback strategy includes one of the following: playback at original speed, playback with video speed change combined with audio speed change without pitch change, and frame skipping playback.

[0041] This application utilizes supplemental enhancement information in video coding standards as a carrier of high-precision, absolute timestamps, enabling it to penetrate the distribution chain of the content delivery network, be parsed on the client side, and then be collaboratively calculated through real-time communication channels. Finally, by dynamically adjusting the playback speed, it achieves frame-level synchronization across multiple clients.

[0042] The following section explains the functions and interaction methods of each module in the live streaming synchronization control system, based on a specific implementation process.

[0043] Typically, after obtaining live data from the live streaming source, the server needs to encode the data and then send the encoded live data to the client via edge nodes in the content delivery network. To ensure that all clients within the same virtual room play the live data synchronously, they need to be provided with the same time standard. Therefore, this application proposes that the server, during video encoding, determines the first encoding timestamp of the video frame based on the Network Time Protocol (NTP), and uses supplemental enhancement information from the H.264 / H.265 video encoding standard as the carrier of this first encoding timestamp, thereby providing the same time standard for all clients.

[0044] As an optional implementation, the format of the supplementary enhancement information can be as shown in Table 1 below.

[0045] Table 1

[0046]

[0047] Optionally, the data packet type in this embodiment is a timestamp synchronization packet, and the specific payload data format can be as shown in Table 2 below.

[0048] Table 2

[0049]

[0050] Optionally, when adding supplementary enhancement information to video frames, the server can selectively add the information instead of adding it to every single video frame to balance data accuracy and bandwidth overhead. Specifically, supplementary enhancement information is added to all critical video frames (such as IDR frames), while for non-critical video frames, it can be added once at a preset frame interval, such as once every 5 frames, ensuring at least 2-5 pieces of supplementary enhancement information per second.

[0051] After encoding is complete, the server can send the live data to multiple clients in the target virtual room via the edge nodes of the content delivery network. Each client can periodically determine its own target reception latency based on the encoding timestamp of the received video frame and its local time, and send this target reception latency to the synchronization control module via a real-time communication channel. The synchronization control module makes a collaborative decision based on the target reception latency of multiple clients to determine the target synchronization latency, and then sends the target synchronization latency to each client via the real-time communication channel. All clients adjust their playback strategies (such as variable speed playback) based on the latency deviation between their own target reception latency and the target synchronization latency to achieve synchronized playback of live data by multiple clients, thereby ensuring the user's viewing experience.

[0052] As an optional implementation, each client can periodically determine the first encoded timestamp of the current video frame. This period is the client's synchronization control period, which can be adjusted according to requirements and is not specifically limited here. Specifically, if the current video frame carries supplementary enhancement information, the client can directly parse the supplementary enhancement information to obtain the first encoded timestamp of the current video frame. If the current video frame does not carry supplementary enhancement information, the client can determine the nearest historical video frame carrying supplementary enhancement information, and predict the first encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of that historical video frame and the first display timestamp of the current video frame.

[0053] Optionally, when the current video frame does not carry supplementary enhancement information, the first coded timestamp of the current video frame can be predicted according to the following formula:

[0054]

[0055] In the formula, , These represent the first encoded timestamp and the second encoded timestamp of the historical video frame, respectively. , These represent the first display timestamp of the current video frame and the second display timestamp of the historical video frames, respectively. This represents the cumulative time error for periodic updates, with an initial value of 0. This is a periodically updated adjustment factor related to playback speed, used to absorb errors caused by the non-constant frame rate of the video. The initial value is... , This indicates the preset playback speed, which is the speed based on the video frames within the first cycle of the historical video frames. , .

[0056] Optionally, to ensure the reliability of the above timestamp prediction formula, this embodiment of the application periodically smooths and corrects the formula based on a simplified Kalman filter concept. Specifically, it periodically corrects the formula in the following manner. and Update:

[0057] Starting from the second cycle, if the current video frame carries supplemental enhancement information (i.e., a new video frame carrying supplemental enhancement information is received), the supplemental enhancement information can be parsed to obtain the first encoded timestamp of the current video frame. At the same time, according to the above formula (i.e., based on the second encoded timestamp and second display timestamp of the historical video frames and the first display timestamp of the current video frame to predict the third encoded timestamp of the current video frame), the second difference between the actual first encoded timestamp and the predicted third encoded timestamp is determined.

[0058] If the second difference is greater than the first preset threshold, it indicates that there may be a misalignment of supplementary enhancement information or severe network jitter. In this case, the above timestamp prediction formula can be directly reset and initialized. and Update , In the formula, and These represent the updated cumulative time error and adjustment factor, respectively.

[0059] If the second difference is not greater than the first preset threshold, the error can be considered to originate from the cumulative error of the formula. Therefore, the formula can be smoothed and updated according to the following formula. and :

[0060]

[0061]

[0062] In the formula, , All are preset smoothing factors. Indicates the second difference. This represents the first encoded timestamp of the current video frame obtained through parsing.

[0063] Through this closed-loop mechanism of "prediction-observation-correction", the client can maintain high-precision estimation when supplementary augmentation information is lost in the short term, and maintain the accuracy of the prediction formula when supplementary augmentation information is available in the long term, thereby achieving a seamless synchronization experience, and the user is completely unaware that timestamp signaling has been lost in the middle.

[0064] As an optional implementation, after determining the first encoded timestamp of the current video frame, the client can determine its target reception delay in the following manner:

[0065] First, obtain the clock offset between the target client's local clock and the standard network clock. Typically, when the client starts up, it can first perform a calibration based on the network time protocol and the server's time source to obtain the clock offset between the local clock and the standard network clock.

[0066] Then, the client can determine the first sum of the target client's current local time and clock offset, and determine the first difference between this first sum and the first encoded timestamp. This first difference is taken as the target reception delay of the target client, expressed by the formula:

[0067]

[0068] In the formula, and These represent the client's current local time and clock offset, respectively. This indicates the first encoded timestamp of the current video frame. This represents the target reception latency of the client. This target reception latency can be intuitively understood as the video image seen on the screen at this moment occurring before the Diff.

[0069] Afterwards, all clients can send their target receiving delay to the synchronization control module based on the real-time communication channel.

[0070] Considering the different latency requirements in different live streaming scenarios, this application provides multiple target synchronization latency decision-making methods for the synchronization control module to choose from.

[0071] In the first method, the synchronization control module can determine the maximum value among the reception delays of multiple clients and use the maximum value as the target synchronization delay.

[0072] This approach is suitable for small rooms and highly interactive scenarios (such as online quizzes and contests). It is better for everyone to be slightly slower than to maintain absolute synchronization. Since extremely slow users become the bottleneck, other users can wait by slightly slowing down for a short period of time. Because the speed change is minimal, the impact on the experience is negligible, and it can also ensure that the rhythm of the entire room is unified.

[0073] In the second method, the synchronization control module can determine the maximum value and the target quantile among the reception delays of multiple clients, and determine the third difference between the maximum value and the target quantile. If the third difference is less than the second preset threshold, the target quantile is used as the target synchronization delay. If the third difference is not less than the second preset threshold, the maximum value is used as the target synchronization delay.

[0074] The target quantile can be set to the 75th percentile, and the second preset threshold can be adjusted as needed; no specific limitation is made here. This method allows a small number of users at the top of the latency distribution (i.e., slower) to catch up with frames at a higher playback speed, while other users only need to fine-tune or maintain the current speed. From an overall experience perspective, this is better than making most people slow down and wait. This method is suitable for medium to large rooms, ensuring that the experience of most users is not negatively impacted by a few users with extremely poor network connections.

[0075] In the third method, the synchronization control module can respond to the director's instructions and use the preset delay in the director's instructions as the target synchronization delay.

[0076] Considering that in certain live streaming scenarios with control requirements (such as large-scale events, galas, etc.), all users need to be synchronized to a fixed absolute time point that is greater than the typical maximum latency, this directing mode was proposed. By providing a broadcast-level controllable latency experience, all interactions (such as raffles, red envelopes, etc.) can be triggered at the absolute time point, completely eliminating spoilers.

[0077] After determining the target synchronization delay, the synchronization control module can send the target synchronization delay to all clients based on the real-time communication channel.

[0078] As an optional implementation, the client can determine the playback strategy through the following tiered control method, and play live data according to the playback strategy within the current period:

[0079] First, the client can determine the delay deviation between the target synchronization delay and its own target reception delay;

[0080] If the latency deviation is less than the first preset latency threshold, the playback strategy for the current period can be determined as original speed playback. Within the current period, live data can be played directly at the preset playback speed.

[0081] Understandably, when the target reception latency of the client itself is very close to the target synchronization latency determined by the synchronization control module, the live broadcast image is very close to the required synchronized playback image. In this case, adjusting the playback speed is unnecessary and may even affect the user's normal viewing experience. Therefore, it is sufficient to play at the original speed.

[0082] If the latency deviation is not less than the second preset latency threshold, the playback strategy for the current period can be determined as frame skipping playback. Specifically, the target video frame and the corresponding target audio frame whose encoded timestamp is the first sum value calculated above (the sum of the client's current local time and the clock offset) and the second sum value of the target synchronization latency can be determined. In the current period, the live data is played directly from the target video frame and the target audio frame at the preset playback speed.

[0083] Understandably, when the target reception latency of the client itself differs greatly from the target synchronization latency determined by the synchronization control module, the efficiency of variable-speed frame tracking is too low. In this case, the seek function of a media player can be used to perform frame skipping. Although there may be a momentary inconsistency in the picture, it can effectively ensure the subsequent viewing experience. Compared with spending a lot of computing resources on frame tracking, it is more cost-effective.

[0084] If the delay deviation is not less than the first preset delay threshold and less than the second preset delay threshold, the playback strategy for the current period can be determined as video speed adjustment combined with audio speed adjustment without pitch change.

[0085] In this case, the playback speed adjustment coefficient can be determined first based on the time delay deviation. Usually, the two are proportional. For example, the playback speed adjustment coefficient can be calculated using the following formula:

[0086]

[0087] In the formula, Indicates time delay deviation, Represents a symbolic function. This indicates taking the minimum value. This represents the playback speed adjustment factor. To ensure a good viewing experience for users, the playback speed adjustment factor is usually limited to the range of [0.8, 1.2].

[0088] Then, the preset playback speed can be adjusted according to the playback speed adjustment coefficient to obtain the target playback speed. At the same time, the audio frames in the live data in the current period are processed using a time scaling algorithm (such as the Sonic or Sound-Touch algorithm based on WSOLA) in combination with the target playback speed. Then, in the current period, the video frames and processed audio frames in the live data are played according to the target playback speed.

[0089] By combining the aforementioned video speed-up and frame-tracking with audio speed-up and pitch-unchanged processing, the live stream can be quickly adjusted to the expected synchronized live stream while maintaining the same audio pitch, thus ensuring a good viewing experience for the user.

[0090] The live streaming synchronization control system provided in this application can achieve at least the following technical effects: 1) It breaks through the latency limitations of content delivery networks, compressing the viewing progress differences between different clients from the second level to within 200 milliseconds, achieving a synchronization level that is difficult for humans to perceive; 2) By customizing the format of supplementary enhancement information and using an interpolation prediction timestamp mechanism to resist frame loss, it ensures the continuity and reliability of synchronization logic in complex network environments; 3) Through hierarchical control and playback mechanisms such as variable speed without changing pitch, the synchronization process can be smooth and imperceptible, avoiding audio pitch distortion, video stuttering, or jumps; 4) The entire solution is based on the H.264 / H.265 standard to design supplementary enhancement information, does not depend on a specific content delivery network or player, and has wide applicability.

[0091] Example 2

[0092] Based on the live streaming synchronization control system provided in Embodiment 1, this application embodiment also provides a live streaming synchronization control method applied to a target client. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0093] Figure 2 This is a flowchart illustrating a live streaming synchronization control method according to an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps S202-S208:

[0094] Step S202: Receive live data sent by the server based on the content delivery network, wherein the live data includes video frames and audio frames;

[0095] Step S204: Periodically determine the first encoding timestamp of the current video frame, and determine the target reception delay of the target client based on the first encoding timestamp and the current local time of the target client. The first encoding timestamp is the timestamp when the video frame encoding is completed based on the network time protocol.

[0096] Step S206: Send the target reception delay to the synchronization control module based on the real-time communication channel, and receive the target synchronization delay sent by the synchronization control module based on the real-time communication channel. The target synchronization delay is determined by the synchronization control module based on the reception delay of multiple clients in the target virtual room to which the target client belongs.

[0097] Step S208: Determine the playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, and play live data according to the playback strategy within the current period.

[0098] The following section explains each step of the live streaming synchronization control method in conjunction with the specific implementation process.

[0099] Optionally, when the target client periodically determines the first encoded timestamp of the current video frame, it can do so in the following way: if the current video frame carries supplementary enhancement information, parse the supplementary enhancement information to obtain the first encoded timestamp of the current video frame; if the current video frame does not carry supplementary enhancement information, determine the nearest historical video frame carrying supplementary enhancement information, and predict the first encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of the historical video frame and the first display timestamp of the current video frame.

[0100] Optionally, when the target client determines the target reception delay based on the first encoded timestamp and the target client's current local time, it can do so in the following way: obtain the clock offset between the target client's local clock and the standard network clock; determine the first sum of the target client's current local time and the clock offset, and determine the first difference between the first sum and the first encoded timestamp, and use the first difference as the target client's target reception delay.

[0101] Optionally, the synchronization control module runs on either the server or any client within the target virtual room; the target synchronization delay is determined by the synchronization control module based on the maximum value or target quantile among the reception delays of multiple clients within the target virtual room, or the target synchronization delay is a preset delay set by the synchronization control module.

[0102] Optionally, when determining the playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, the target client can implement it in the following way: if the delay deviation is less than a first preset delay threshold, the playback strategy for the current period is determined to be normal speed playback; if the delay deviation is not less than the first preset delay threshold and less than a second preset delay threshold, the playback strategy for the current period is determined to be video speed-changing combined with audio speed-changing without pitch change, and the playback speed adjustment coefficient is determined based on the delay deviation; if the delay deviation is not less than the second preset delay threshold, the playback strategy for the current period is determined to be frame skipping playback.

[0103] Optionally, when the target client plays live data according to the playback strategy within the current period, it can achieve the following: If the playback strategy is original speed playback, play the live data at a preset playback speed within the current period; if the playback strategy is video speed-changing combined with audio speed-changing without pitch change, adjust the preset playback speed according to the playback speed adjustment coefficient to obtain the target playback speed, and use a time scaling algorithm combined with the target playback speed to process the audio frames in the live data within the current period, playing the video frames and processed audio frames in the live data at the target playback speed within the current period; if the playback strategy is frame skipping playback, determine the target video frame and corresponding target audio frame whose encoded timestamp is the sum of the first sum and the second sum of the target synchronization delay, and play the live data at the preset playback speed starting from the target video frame and target audio frame within the current period.

[0104] Optionally, when the target client predicts the first encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of historical video frames and the first display timestamp of the current video frame, it can do so in the following way: predict the first encoded timestamp of the current video frame according to the following formula: In the formula, , These represent the first and second encoded timestamps, respectively. , These represent the first and second display timestamps, respectively. This is the cumulative time error for periodic updates. This is a periodically updated adjustment factor related to playback speed, assuming the historical video frames are from the first period. , , This indicates the preset playback speed.

[0105] Optionally, starting from the second cycle, if the current video frame carries supplemental enhancement information, the target client can parse the supplemental enhancement information to obtain the first encoded timestamp of the current video frame, and predict the third encoded timestamp of the current video frame based on the second encoded timestamp and second display timestamp of historical video frames and the first display timestamp of the current video frame; determine the second difference between the first encoded timestamp and the third encoded timestamp; and update the timestamp if the second difference is greater than the first preset threshold. , In the formula, and These represent the updated cumulative time error and adjustment factor, respectively; if the second difference is not greater than the first preset threshold, the update... , In the formula, , All are preset smoothing factors. Indicates the second difference. This represents the first encoded timestamp of the current video frame obtained through parsing.

[0106] It should be noted that the process steps of the live streaming synchronization control method in this application embodiment correspond to the process steps executed by the client in the live streaming synchronization control system in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0107] Example 3

[0108] Based on the live streaming synchronization control system provided in Embodiment 1, this application embodiment also provides a live streaming synchronization control method applied to the server. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0109] Figure 3 This is a flowchart illustrating a live streaming synchronization control method according to an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps S302-S306:

[0110] Step S302: Obtain live data from the live source, wherein the live data includes video frames and audio frames;

[0111] Step S304: Add supplementary enhancement information to key video frames in the live data, and add supplementary enhancement information to non-key video frames in the live data according to a preset frame interval. The supplementary enhancement information includes: a first encoding timestamp based on the network time protocol when the corresponding video frame encoding is completed.

[0112] Step S306: The live streaming data is sent to multiple clients in the target virtual room via a content delivery network.

[0113] It should be noted that the process steps of the live streaming synchronization control method in this application embodiment correspond to the process steps executed by the server in the live streaming synchronization control system in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0114] Example 4

[0115] Based on the live streaming synchronization control system provided in Embodiment 1, this application embodiment also provides a live streaming synchronization control method applied to a synchronization control module. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0116] Figure 4 This is a flowchart illustrating a live streaming synchronization control method according to an embodiment of this application, as shown below. Figure 4 As shown, the method includes the following steps S402-S406:

[0117] Step S402: Periodically receive the reception delay sent by multiple clients in the target virtual room based on the real-time communication channel. The reception delay is determined by each client based on the first encoding timestamp of the video frame encoding in the received live data according to the network time protocol and its own local time of receiving the video frame.

[0118] Step S404: Determine the target synchronization delay based on the reception delays of multiple clients;

[0119] Step S406: The target synchronization delay is sent to multiple clients based on the real-time communication channel, wherein the target synchronization delay is used to guide each client to determine the playback strategy for the current period.

[0120] Optionally, when determining the target synchronization delay based on the reception delays of multiple clients, the synchronization control module can do so in the following ways: determine the maximum value among the reception delays of multiple clients and use the maximum value as the target synchronization delay; or, determine the maximum value and the target quantile among the reception delays of multiple clients, and determine the third difference between the maximum value and the target quantile. If the third difference is less than a second preset threshold, use the target quantile as the target synchronization delay; if the third difference is not less than the second preset threshold, use the maximum value as the target synchronization delay; or, in response to the director's instruction, use the preset delay in the director's instruction as the target synchronization delay.

[0121] It should be noted that the process steps of the live streaming synchronization control method in this application embodiment correspond to the process steps executed by the client in the live streaming synchronization control system in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0122] Example 5

[0123] According to an embodiment of this application, a computer program product is also provided, which includes a computer program, wherein when the computer program is executed by a processor, it implements the live streaming synchronization control method in embodiment 2, embodiment 3, or embodiment 4.

[0124] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the live synchronization control method in embodiment 2, embodiment 3, or embodiment 4 by running the computer program.

[0125] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the live streaming synchronization control method in embodiment 2, embodiment 3, or embodiment 4 when it runs.

[0126] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the live streaming synchronization control method of embodiment 2, embodiment 3, or embodiment 4 through the computer program.

[0127] Optionally, the computer program executes the following steps during runtime: receiving live data sent by the server based on the content delivery network, wherein the live data includes video frames and audio frames; periodically determining the first encoded timestamp of the current video frame, and determining the target reception delay of the target client based on the first encoded timestamp and the current local time of the target client, wherein the first encoded timestamp is the timestamp of the completion of video frame encoding based on the Network Time Protocol; sending the target reception delay to the synchronization control module based on the real-time communication channel, and receiving the target synchronization delay sent by the synchronization control module based on the real-time communication channel, wherein the target synchronization delay is determined by the synchronization control module based on the reception delays of multiple clients in the target virtual room to which the target client belongs; determining a playback strategy based on the delay deviation between the target synchronization delay and the target reception delay, and playing the live data according to the playback strategy within the current period.

[0128] Optionally, the computer program executes the following steps during runtime: acquiring live data from the live source, wherein the live data includes video frames and audio frames; adding supplementary enhancement information to key video frames in the live data, and adding supplementary enhancement information to non-key video frames in the live data according to a preset frame interval, wherein the supplementary enhancement information includes: a first encoded timestamp based on the network time protocol when the corresponding video frame encoding is completed; and sending the live data to multiple clients in the target virtual room via a content delivery network.

[0129] Optionally, the computer program executes the following steps during runtime: periodically receiving reception delays sent by multiple clients within the target virtual room via a real-time communication channel, wherein the reception delay is determined by each client based on the first encoding timestamp of the video frame in the received live data when encoding is completed using the Network Time Protocol and its own local time for receiving the video frame; determining the target synchronization delay based on the reception delays of the multiple clients; and sending the target synchronization delay to the multiple clients via the real-time communication channel, wherein the target synchronization delay is used to guide each client in determining the playback strategy for the current period.

[0130] As an alternative implementation, the above-mentioned electronic device may exist in the form of a mobile terminal, a computer terminal, or a similar computing device. Figure 5 A hardware block diagram of an electronic device for implementing a live streaming synchronization control method is shown. Figure 5 As shown, the electronic device 50 may include one or more (shown as 502a, 502b, ..., 502n) processors 502 (processors 502 may include, but are not limited to, processing devices such as microprocessors or programmable logic devices), a memory 504 for storing data, and a transmission device 506 for communication functions. In addition, it may also include: a display, an input / output interface, a universal serial bus port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, electronic device 50 may also include... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0131] It should be noted that the aforementioned one or more processors 502 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element of the electronic device 50. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0132] The memory 504 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the live streaming synchronization control method in this embodiment. The processor 502 executes various functional applications and data processing by running the software programs and modules stored in the memory 504, thereby realizing the aforementioned live streaming synchronization control method. The memory 504 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 504 may further include memory remotely located relative to the processor 502, and these remote memories can be connected to the electronic device 50 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] The transmission device 506 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 50. In one example, the transmission device 506 includes a network adapter that can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 506 may be a radio frequency module used for wireless communication with the Internet.

[0134] The display may be, for example, a touchscreen LCD display, which allows the user to interact with the user interface of the electronic device 50.

[0135] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0136] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. When this computer software product is stored in a storage medium, it can implement several instructions in the following manner to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0141] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A live streaming synchronization control method, applied to a target client, characterized in that, include: The receiving server sends live data based on a content delivery network, wherein the live data includes video frames and audio frames; The first encoding timestamp of the current video frame is periodically determined, and the target reception delay of the target client is determined based on the first encoding timestamp and the current local time of the target client. The first encoding timestamp is the timestamp of the completion of video frame encoding based on the Network Time Protocol. The target receiving delay is sent to the synchronization control module based on the real-time communication channel, and the target synchronization delay sent by the synchronization control module based on the real-time communication channel is received. The target synchronization delay is determined by the synchronization control module based on the receiving delays of multiple clients in the target virtual room to which the target client belongs. A playback strategy is determined based on the delay deviation between the target synchronization delay and the target reception delay, and the live data is played according to the playback strategy within the current period.

2. The method according to claim 1, characterized in that, Periodically determine the first encoded timestamp of the current video frame, including: If the current video frame carries supplementary enhancement information, the supplementary enhancement information is parsed to obtain the first encoded timestamp of the current video frame; If the current video frame does not carry supplementary enhancement information, determine the nearest historical video frame that carries supplementary enhancement information, and predict the first coding timestamp of the current video frame based on the second coding timestamp and second display timestamp of the historical video frame and the first display timestamp of the current video frame.

3. The method according to claim 1, characterized in that, Determining the target reception delay of the target client based on the first encoded timestamp and the current local time of the target client includes: Obtain the clock offset between the target client's local clock and the standard network clock; Determine the first sum of the current local time of the target client and the clock offset, and determine the first difference between the first sum and the first encoded timestamp, and use the first difference as the target reception delay of the target client.

4. The method according to claim 1, characterized in that, The target reception delay is sent to the synchronization control module based on the real-time communication channel, and the target synchronization delay sent by the synchronization control module based on the real-time communication channel is received, including: The target reception delay is sent to the synchronization control module based on the real-time communication channel, wherein the synchronization control module runs on either the server or any client within the target virtual room; The synchronization control module receives the target synchronization delay sent by the synchronization control module based on the real-time communication channel. The target synchronization delay is determined by the synchronization control module based on the maximum value or target quantile of the reception delays of multiple clients in the target virtual room, or the target synchronization delay is a preset delay set by the synchronization control module.

5. The method according to claim 3, characterized in that, Determining a playback strategy based on the delay deviation between the target synchronization delay and the target reception delay includes: If the delay deviation is less than the first preset delay threshold, the playback strategy for the current period is determined to be playback at the original speed. If the delay deviation is not less than the first preset delay threshold and less than the second preset delay threshold, the playback strategy for the current period is determined to be video speed change combined with audio speed change without pitch change, and the playback speed adjustment coefficient is determined based on the delay deviation. If the delay deviation is not less than the second preset delay threshold, the playback strategy for the current period is determined to be frame skipping playback.

6. The method according to claim 5, characterized in that, Playing the live data according to the playback strategy within the current period includes: When the playback strategy is set to original speed playback, the live data is played at a preset playback speed within the current period; When the playback strategy is video speed change combined with audio speed change without pitch change, the preset playback speed is adjusted according to the playback speed adjustment coefficient to obtain the target playback speed. The time scaling algorithm is used in combination with the target playback speed to process the audio frames in the live data in the current period. The video frames and processed audio frames in the live data are played in the current period according to the target playback speed. When the playback strategy is frame skipping, the target video frame and the corresponding target audio frame are determined with the encoding timestamp being the sum of the first sum and the second sum of the target synchronization delay. In the current period, starting from the target video frame and the target audio frame, the live data is played at a preset playback speed.

7. The method according to claim 2, characterized in that, Predicting the first encoded timestamp of the current video frame based on the second encoded timestamp and second displayed timestamp of the historical video frames and the first displayed timestamp of the current video frame includes: Predict the first encoded timestamp of the current video frame using the following formula: In the formula, , These represent the first encoded timestamp and the second encoded timestamp, respectively. , These represent the first display timestamp and the second display timestamp, respectively. This is the cumulative time error for periodic updates. This is a periodically updated adjustment factor related to playback speed, assuming the historical video frames are video frames within the first period. , , This indicates the preset playback speed.

8. A live streaming synchronization control method, applied to a server, characterized in that, include: Live data is obtained from the live streaming source, wherein the live streaming data includes video frames and audio frames; Supplementary enhancement information is added to key video frames in the live data, and supplementary enhancement information is added to non-key video frames in the live data according to a preset frame interval. The supplementary enhancement information includes: a first encoding timestamp based on the network time protocol when the encoding of the corresponding video frame is completed. The live stream data is sent to multiple clients within the target virtual room via a content delivery network.

9. A live streaming synchronization control method, applied to a synchronization control module, wherein the synchronization control module runs on a server or any client within a target virtual room, characterized in that, include: The system periodically receives reception delays sent by multiple clients within the target virtual room via a real-time communication channel. The reception delay is determined by each client based on the first encoding timestamp of the video frame in the received live data when encoding is completed using the Network Time Protocol and its own local time when the video frame is received. The target synchronization delay is determined based on the receiving delays of the multiple clients; The target synchronization delay is sent to the multiple clients via a real-time communication channel, wherein the target synchronization delay is used to guide each client to determine the playback strategy for the current period.

10. A live streaming synchronization control system, characterized in that, Includes: a server, multiple clients within the target virtual room, and a synchronization control module running on the server or any of the clients, wherein... The server is used to obtain live data from the live source, the live data including video frames and audio frames; add supplementary enhancement information to key video frames in the live data, and add supplementary enhancement information to non-key video frames in the live data according to a preset frame interval, the supplementary enhancement information including: a first encoded timestamp based on the network time protocol when the corresponding video frame encoding is completed; and send the live data to the multiple clients based on the content delivery network. Each client periodically determines the first encoded timestamp of the current video frame and determines its target reception delay based on the first encoded timestamp and its current local time. It then sends the target reception delay to the synchronization control module via a real-time communication channel and receives the target synchronization delay sent by the synchronization control module via the real-time communication channel. The target synchronization delay is determined by the synchronization control module based on the reception delays of the multiple clients. A playback strategy is determined based on the delay deviation between the target synchronization delay and the target reception delay, and the live data is played according to the playback strategy within the current period. The playback strategy includes one of the following: original speed playback, video speed adjustment combined with audio speed adjustment without pitch change, or frame skipping playback.

11. A computer program product, characterized in that, include: A computer program, wherein when executed by a processor, the computer program implements the live streaming synchronization control method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the live streaming synchronization control method according to any one of claims 1 to 9 through the computer program.

Images