Method and apparatus for synchronization between video and audio frames

Abstract

The present disclosure provides a method and apparatus for synchronization between video frames and audio frames based on high speed transmission of VoD content. A content delivery server includes a transceiver and a processor. The transceiver receives a search request for a particular playback time within the VoD content from a content requesting terminal. In response to the search request, the processor: deletes one or more audio frames existing between a video frame corresponding to the requested playback time and an audio frame corresponding to the requested playback time; and generate content data that is reconstructed to include all or part of frames that exist within a period of time from the video frame corresponding to the requested playback time to the audio frame corresponding to the requested playback time. The transceiver transmits the reconstructed content data to the content requesting terminal.

Description

Technical Field

[0001] The present disclosure relates to a method and apparatus for synchronization between video frames and audio frames. Background Art

[0002] The contents of this section merely provide background information related to the present disclosure and do not constitute prior art.

[0003] As content distribution services evolve, they provide viewers with a variety of services, including not only live content distribution services provided by broadcasters, but also video on demand (VoD) services. Unlike wireless broadcasting, VoD services can use communication lines such as the Internet to transmit VoD content to users. With VoD services, users can watch desired content at a desired time.

[0004] VoD content consists of audio data and video data. The audio data and video data include one or more audio frames and video frames respectively. As used herein, a frame is the smallest unit of video or audio output by a display for content. VoD content is played using multiple video frames and / or audio frames that change rapidly.

[0005] The audio frame is transmitted to the content requesting terminal a preset time later than the video frame having the same presentation time stamp (PTS). PTS is a timestamp indicating the playback time for synchronization of video and audio. Therefore, when playing VoD content from a specific playback time within the VoD content, there is a problem of playing audio frames before the specific playback time that do not match the video frames at the specific playback time.

[0006] In addition, when the content request terminal starts playing the VoD content from a specific playback time within the VoD content, the content request terminal cannot play the VoD content until the audio frame corresponding to the same PTS as the video frame of the specific playback time is transmitted. As a result, there is a delay problem before the video of the requested playback time is played. Summary of the invention

[0007] Technical issues

[0008] According to one embodiment of the present disclosure, the main purpose is to provide a technology that, when a search request for a specific playback time within VoD content is received, quickly performs synchronization between video frames and audio frames corresponding to the same PTS by deleting audio frames corresponding to the PTS before the requested playback time.

[0009] According to another embodiment of the present disclosure, a main object is to provide a technology for quickly transmitting content data reconstructed by deleting an audio frame corresponding to a PTS before a requested playback time to a content requesting terminal.

[0010] The problems to be solved by the present disclosure are not limited to the above-mentioned problems, and other unmentioned problems will become apparent to those skilled in the art from the following description.

[0011] Technical Solution

[0012] According to one embodiment, the present disclosure provides a content distribution server for transmitting VoD content to a content requesting terminal at high speed, the content distribution server comprising a transceiver; and a processor, wherein the transceiver receives a search request for a specific playback time within the VoD content from the content requesting terminal, wherein, in response to the search request, the processor deletes one or more audio frames existing between a video frame corresponding to the requested playback time and an audio frame corresponding to the requested playback time, wherein the processor generates content data reconstructed to include all or part of the frames existing within a time period from a video frame corresponding to the requested playback time to an audio frame corresponding to the requested playback time, and wherein the transceiver transmits the reconstructed content data to the content requesting terminal.

[0013] According to another embodiment, the present disclosure provides a method for synchronization between video frames and audio frames, the method comprising the following steps: receiving a search request for a specific playback time within VoD content from a content requesting terminal; deleting one or more audio frames existing between a video frame corresponding to the requested playback time and an audio frame corresponding to the requested playback time based on the search request; generating content data reconstructed to include all or part of the frames existing within a time period from the video frame corresponding to the requested playback time to the audio frame corresponding to the requested playback time; and transmitting the reconstructed content data to the content requesting terminal.

[0014] Effects of the Invention

[0015] According to one embodiment of the present disclosure, by performing synchronization between video frames and audio frames, inconsistency between video frames and audio frames occurring when playing video and audio corresponding to a requested playback time can be resolved.

[0016] According to another embodiment of the present disclosure, by quickly transmitting reconstructed content data to a content requesting terminal, a delay time for playing video and audio corresponding to a requested playback time within VoD content can be reduced.

[0017] The effects of the present disclosure are not limited to the above-mentioned effects, and other unmentioned effects will be clearly understood from the following description by those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1a is a conceptual diagram for showing a conventional data structure of VoD content.

[0019] Figure 1b is used to pass Figure 1a The data structure shown is divided into frames with the same time stamp to illustrate a conceptual diagram of the problem with the conventional data structure of VoD content.

[0020] Figure 2 is a diagram for illustrating a data structure of reconstruction of VoD content according to an embodiment of the present disclosure.

[0021] Figure 3 Detailed description is a flowchart illustrating a process in which a content distribution server transmits VoD content to a content requesting terminal according to an embodiment of the present disclosure.

[0022] Figure 4 is a block diagram of a content distribution server according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, similar reference numerals preferably refer to similar elements, although the elements are shown in different figures. In addition, for the sake of clarity and simplicity, the following description of some embodiments will omit the detailed description of related known components and functions when it is considered to make the subject matter of the present disclosure obscure.

[0024] Various ordinal numbers or letter codes such as first, second, i), ii), a), b), etc. as prefixes are only used to distinguish one component from other components, and do not imply or suggest the substance, order or sequence of the components. Throughout this specification, when a part "includes" or "comprises" a component, it means that the part also includes other components, but does not exclude other components, unless specifically stated otherwise.

[0025] Figure 1a is a conceptual diagram for showing a conventional data structure of VoD content.

[0026] Reference Figure 1a , VoD content includes all or some of the video data and audio data. The video data and audio data include one or more video frames and audio frames, respectively. As used herein, a frame is the smallest unit of video and / or audio constituting content data. VoD content can be played as a plurality of video frames and / or audio frames output quickly. As used herein, the term "frame" refers to a video frame and / or an audio frame.

[0027] Each frame corresponds to a specific playback time. When the frames constituting the VoD content are transmitted, the transmission of the audio frame follows the transmission of the video frame based on the same presentation time stamp (PTS). Here, PTS refers to a timestamp indicating the playback time for synchronization of video and audio. For example, assume that the VoD content starts playing from the time corresponding to the i-th (i is a natural number) video frame. In this case, when the i-th video frame is transmitted, To the i+Nth int Video Frame And transmit audio frames To Audio Frame After that, the transmission corresponds to the i-th video frame The i-th audio frame of the same PTS Therefore, when the content requesting terminal starts playing the VoD content from the time corresponding to the i-th video frame, the i-th video frame cannot be played unless an audio frame corresponding to the same PTS as the i-th video frame is received.

[0028] Figure 1b is used to pass Figure 1a The data structure shown is divided into frames with the same time stamp to illustrate a conceptual diagram of the problem with the conventional data structure of VoD content.

[0029] Reference Figure 1b , the VoD content includes all or some of the audio data 100 and the video data 110. The audio data 100 and the video data 110 are respectively composed of one or more audio frames and one or more video frames.

[0030] When a user wants to play the VoD content from a specific playback time within the VoD content (e.g., the playback time corresponding to the i-th (i is a natural number) frame), the content distribution server may receive a search request for the specific playback time from the content request terminal. In this case, the video before the i-th frame corresponding to the requested playback time is not played, but the audio before the i-th frame is played normally. In other words, there is a problem that the video and audio do not match when the VoD content is played from the playback time of the search request.

[0031] In addition, since only in audio frames to The i-th audio frame is transmitted afterwards, so there is a delay time before playing the content corresponding to the requested playback time. The delay time for playing the content corresponding to the requested playback time can be calculated using Formula 1.

[0032] (Formula 1)

[0033]

[0034] Here, D(F) is the data size of frame F (in bits), R C It is the constant transmission rate at which the content distribution server transmits data to the content requesting terminal at a constant bit rate. C Can be set arbitrarily by the configurator. The configurator can set R appropriately. CTo prevent underflow and overflow of the receiving buffer in the content request terminal. Buffer underflow occurs when a buffer used to transmit data between two devices or processes receives data at a rate lower than its reading rate. When buffer underflow occurs, the program or device reading from the buffer is temporarily suspended until the buffer is fed with data. Buffer overflow occurs when the data fed into the buffer is larger than its storage capacity and exceeds the corresponding memory space. When buffer overflow occurs, the memory space used by the program in the computer system overflows, resulting in program execution errors.

[0035] Figure 2 is a diagram for illustrating a data structure of reconstruction of VoD content according to an embodiment of the present disclosure.

[0036] Reference Figure 2 To solve the problem of traditional VoD content, the content distribution server deletes the video frames that exist at the playback time corresponding to the request. and the audio frame corresponding to the requested playback time Audio frames between to In this case, the content data is reconstructed so that (N int +1) video frame to Arranged sequentially in the video frames corresponding to the requested playback time and audio frames This achieves synchronization between video frames and audio frames, solving synchronization mismatches when playing video and audio at requested playback times within VoD content.

[0037] In addition, the content distribution server can transmit the reconstructed content data to the content request terminal at a high speed. Here, in addition to the deleted audio frame, the reconstructed content data includes video frames. To Video Frame and audio frames When the reconstructed content is transmitted to the content requesting terminal at a high speed, Equation 2 may be used to calculate the delay time for playing the content corresponding to the requested playback time.

[0038] (Formula 2)

[0039]

[0040] Here, D(F) is the data size of frame F (in bits), R Fast It is the high-speed transmission rate at which the content distribution server transmits data to the content request terminal at high speed. Fast , and using a constant transmission rate R C Compared with , the delay time can be shortened. The shortened delay time can be calculated using Formula 3.

[0041] (Formula 3)

[0042]

[0043] Based on and using a constant transmission rate R C The content requesting terminal can quickly play the video and / or audio corresponding to the requested playback time compared to the shortened delay time.

[0044] Figure 3 Detailed description is a flowchart illustrating a process in which a content distribution server transmits VoD content to a content requesting terminal according to an embodiment of the present disclosure.

[0045] Reference Figure 3 , the content distribution server receives a search request for a specific playback time within the VoD content from the content request terminal (S300). After the content request terminal sends the search request to the content distribution server, the content request terminal resets the receiving buffer. Here, resetting means clearing the storage space of the receiving buffer. Since the content request terminal receives data at a high speed after resetting the receiving buffer, buffer underflow and / or overflow does not occur in the initial time period.

[0046] Based on the received search request, the content distribution server identifies one or more audio frames existing between a video frame and an audio frame corresponding to a requested playback time (S302).

[0047] The content distribution server deletes one or more audio frames between the video frame and the audio frame corresponding to the requested playback time (S304).

[0048] The content distribution server generates content data reconstructed to include all or part of the frames from the video frame corresponding to the requested playback time to the audio frame corresponding to the requested playback time (S306).

[0049] The content distribution server transmits the reconstructed content data to the content requesting terminal at high speed. In another embodiment, when the content distribution server sends a transmission preparation notification to the content requesting terminal, the content distribution server transmits the reconstructed content data to the content requesting terminal at high speed (S308).

[0050] The content distribution server may receive a constant rate transmission request for subsequent video frames and audio frames after the reconstructed content data from the content request terminal. When the content distribution server receives the constant rate transmission request from the content request terminal, the content distribution server may send a response agreeing to the constant rate transmission to the content request terminal (S310).

[0051] Figure 4is a block diagram of a content distribution server according to one embodiment of the present disclosure.

[0052] Reference Figure 4 , the content distribution server 400 includes all or part of a transceiver 402 , a memory 404 , and a processor 406 .

[0053] The content distribution server 400 may communicate with the content requesting terminal using the transceiver 402. For example, the transceiver 402 may transmit VoD content to the content requesting terminal and receive a constant rate transmission request from the content requesting terminal.

[0054] The memory 404 may store a program that instructs the processor 406 to perform a frame synchronization method according to an embodiment of the present disclosure. For example, the program may include a plurality of instructions that can be executed by the processor 406, and the plurality of instructions can be executed by the processor 406 to perform the frame synchronization method.

[0055] The memory 404 may include at least one of a volatile memory and a nonvolatile memory. The volatile memory may include a static random access memory (SRAM) or a dynamic random access memory (DRAM), and the nonvolatile memory may include a flash memory.

[0056] The processor 406 may include at least one component capable of executing at least one instruction. The processor 406 may execute the instructions stored in the memory 404, and perform the frame synchronization method according to the present disclosure by executing the instructions.

[0057] Various illustrative implementations of the systems and methods described herein may be realized by digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof.

[0058] In the flowchart of this specification, it is described that each process is performed in sequence, but this is only an example of the technical idea of ​​one embodiment of the present disclosure. In other words, a person skilled in the art of one embodiment of the present disclosure may change and execute the order described in the flowchart / sequence diagram within the scope of not departing from the basic characteristics of the embodiment of the present disclosure, or may execute one or more processes in parallel to apply various modifications and changes, and therefore, the flowchart / sequence diagram is not limited to the time sequence.

[0059] The steps described in the flowchart may be implemented as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes any type of recording device that can record data that can be read by a computer system. Examples of computer-readable recording media include non-volatile or non-temporary media such as ROM, RAM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, optical / magnetic disk, storage device, etc. In addition, the computer-readable recording medium may be distributed in computer systems connected via a network, where the computer-readable code may be stored and executed in a distributed manner.

[0060] Although the exemplary embodiments of the present disclosure are described for illustrative purposes, it will be understood by those skilled in the art that various modifications, additions and substitutions may be made without departing from the concept and scope of the claimed invention. Therefore, for the sake of simplicity and clarity, the exemplary embodiments of the present disclosure are described. The scope of the technical ideas of the embodiments of the present disclosure are not limited by the examples. Therefore, it will be understood by those of ordinary skill in the art that the scope of the claimed invention is not limited by the embodiments explicitly described above, but by the claims and their equivalents.

[0061] (Description of label)

[0062] 400: Content distribution server

[0063] 402: Transceiver

[0064] 404: Memory

[0065] 406: Processor

[0066] CROSS-REFERENCE TO RELATED APPLICATIONS

[0067] This application claims priority to Korean Patent Application No. 10-2022-0035382, filed on Mar. 22, 2022, the disclosure of which is incorporated herein by reference in its entirety.

Claims

1. A content distribution server for transmitting VoD content to a content requesting terminal at high speed, the content distribution server comprising: Transceiver; as well as processor, wherein the transceiver receives a search request for a specific playback time within the VoD content from a content requesting terminal, wherein, in response to the search request, the processor deletes one or more audio frames existing between a video frame corresponding to the requested playback time and an audio frame corresponding to the requested playback time, wherein the processor generates content data, the content data being reconstructed to include all or part of frames existing in a time period from the video frame corresponding to the requested playback time to the audio frame corresponding to the requested playback time, and The transceiver transmits the reconstructed content data to the content requesting terminal.

2. The content distribution server according to claim 1, wherein: The transceiver transmits the reconstructed content data to the content requesting terminal at high speed.

3. The content distribution server according to claim 2, in, In response to generating the reconstructed content data, the transceiver sends a notification to the content requesting terminal that the reconstructed content data is ready for high-speed transmission, and In response to sending the notification to the content requesting terminal, the content distribution server transmits the reconstructed content data to the content requesting terminal at a high speed.

4. A method for synchronization between video frames and audio frames, the method comprising the following steps: receiving a search request for a specific playback time within the VoD content from a content requesting terminal; Based on the search request, deleting one or more audio frames existing between a video frame corresponding to the requested playback time and an audio frame corresponding to the requested playback time; generating content data reconstructed to include all or part of frames existing in a time period from the video frame corresponding to the requested playback time to the audio frame corresponding to the requested playback time; as well as The reconstructed content data is transmitted to the content requesting terminal.

5. The method according to claim 4, further comprising the steps of: The reconstructed content data is transmitted to the content requesting terminal at high speed.

6. The method according to claim 5, further comprising the steps of: In response to generating the reconstructed content data, sending a notification to the content requesting terminal that the reconstructed content data is ready for high-speed transmission, The step of transmitting the reconstructed content data includes the following steps: in response to sending the notification to the content requesting terminal, transmitting the reconstructed content data to the content requesting terminal at a high speed.

7. A computer program stored in one or more computer-readable recording media to execute each process included in the method according to any one of claims 4 to 6.

Images