Frame Synchronizer

Abstract

To provide a frame synchronization device which can perform frame synchronization at comparatively small calculation cost.SOLUTION: The frame synchronization device includes a receiver unit, a frame memory, a time generation unit, a reception time acquisition unit, a time stamp acquisition unit and a control unit. The receiver unit receives packet data which includes video data and a time stamp. The frame memory stores the packet data received by the receiver unit. The time generation unit generates a time based on a reference synchronous signal. The reception time acquisition unit acquires the reception time of packet data which satisfies a predetermined condition, based on the time generated by the time generation unit. The time stamp acquisition unit acquires the time stamp from the packet data which satisfies the predetermined condition. The control unit obtains a difference between the reception time and a time indicated by the time stamp to read out packet data from the frame memory according to the variation of the difference.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a frame synchronizer. [Background technology]

[0002] As is well known, in a system for transmitting a video signal, a slight frequency deviation may occur between the reference synchronization signal used by the source that transmits the video signal and the reference synchronization signal used by the broadcasting station that receives the video signal. The frequency deviation may cause problems such as distorting the image, particularly in real-time broadcasting. For this reason, a frame synchronization device is used on the receiving side to compensate for the frequency deviation.

[0003] Recently, content distribution systems have been realized that distribute media information such as video and audio using IP (Internet Protocol) networks. Even in systems using IP networks, frequency deviations can occur when the reference synchronization between the sending side and the receiving side differs, but because media information is transmitted as packetized data, conventional frame synchronization devices cannot be simply applied.

[0004] If the reference synchronization of both the sender and receiver is GNSS (Global Navigation Satellite System), no frequency deviation will occur. However, if either side is placed in an environment where GNSS cannot be used, a reference synchronization other than GNSS will be used, and frequency deviation may occur.

[0005] To address this issue, in systems using IP networks, a method could be considered in which the packetized data is first depacketized, then frame synchronized, and then repacketized. However, such a method has the problem of being computationally expensive. Furthermore, when the computation cost is so high, problems such as processing delays arise, impairing the real-time nature of broadcasting, or requiring an increase in the circuit scale to address this issue. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 54-52412 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a frame synchronization device that can perform frame synchronization processing quickly with relatively low computational costs. [Means for solving the problem]

[0008] A frame synchronization device according to an embodiment includes a receiving unit, a frame memory, a time generating unit, a reception time acquiring unit, a time stamp acquiring unit, and a control unit. The receiving unit receives packet data including video data and a time stamp, the frame memory stores the packet data received by the receiving unit, the time generating unit generates time based on a reference synchronization signal, the reception time acquiring unit acquires a reception time of packet data that satisfies a predetermined condition based on the time generated by the time generating unit, the time stamp acquiring unit acquires a time stamp from the packet data that satisfies the predetermined condition, and the control unit calculates the difference between the reception time and the time indicated by the time stamp and reads out the packet data from the frame memory according to the amount of change in this difference. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a configuration of a content distribution system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of a first embodiment of the frame synchronization device shown in FIG. [Diagram 3] FIG. 2 is a diagram for explaining an example of the structure of an RTP packet. [Figure 4] 3 is a flowchart for explaining the processing of the frame synchronization device shown in FIG. 2; [Diagram 5] FIG. 2 is a diagram showing the configuration of a second embodiment of the frame synchronization device shown in FIG. [Figure 6] 6 is a flowchart for explaining the processing of the frame synchronization device shown in FIG. 5; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment will be described with reference to the drawings. 1 shows the configuration of a content distribution system including a frame synchronization device and a computer program according to an embodiment of the present invention. The content distribution system includes a transmission source 100, a network NW, and a broadcast station 200.

[0011] The source 100 is a device or facility that transmits media information (video, audio, auxiliary information, etc.) through a network NW (for example, an optical public network), and is a facility at a shooting location outside a broadcasting station, or a broadcasting station other than the broadcasting station 200. The source 100 includes at least a PTP master 101 and an IP sending device (IP sender) 102.

[0012] The PTP master 101 is a device that manages the time used within the transmission source 100, and performs time synchronization (local time synchronization) using PTP (Precision Time Protocol) to generate the transmission source internal time PTP1. Here, the PTP master 101 uses a time source other than GNSS (Global Navigation Satellite System).

[0013] The IP transmission device 102 quantizes (digitizes) the video signal and audio signal based on the time PTP1 generated by the PTP master 101, and packetizes (packets) them into RTP (Real-time Transport Protocol) packets. The RTP packets are assigned a time stamp based on the time PTP1 and a sequence number that is incremented for each packet. The IP transmission device 102 then transmits the generated RTP packets to the broadcast station 200 via the network NW.

[0014] The broadcast station 200 is a facility for broadcasting content via, for example, a wireless or wired broadcast medium or communication medium (not shown), and can broadcast, as the above-mentioned content, media information obtained from an RTP packet received from the source 100 via a network NW. The broadcast station 200 includes at least a PTP master 201, a frame synchronization device 210, and an IP receiving device 220.

[0015] The PTP master 201 is a device that manages the time used within the broadcast station 200, and performs time synchronization (GNSS synchronization) using the GNSS as a time source to generate the broadcast station internal time PTP2. That is, since the above-mentioned PTP master 101 and PTP master 201 perform time synchronization using different time sources, there may be a frequency deviation between the transmission source internal time PTP1 and the broadcast station internal time PTP2.

[0016] The frame synchronizer 210 receives RTP packets transmitted from the source 100 via the network NW, and synchronizes frames of video data contained in the received RTP packets with the broadcast station internal time PTP2.

[0017] Specifically, as a first embodiment, a frame synchronizer 210 as described later with reference to Fig. 2 to Fig. 4 can be considered. That is, the frame synchronizer 210 obtains the time difference between the time indicated by the timestamp of the video data included in the RTP packet and the reception time of the RTP packet based on the broadcast station internal time PTP2, detects / monitors the amount of change (hereinafter referred to as the time difference change amount), copies or skips frames according to the time difference change amount, and updates the timestamp and sequence number of the RTP packet based on the internal time (frame synchronizer internal time described later), thereby performing frame synchronization of the video data.

[0018] As a second embodiment, a frame synchronizer 230 as described later with reference to Figures 5 and 6 is conceivable. That is, the frame synchronizer 230 is provided in the broadcasting station 200 in place of the frame synchronizer 210, and detects / monitors the amount of change in the number of packets stored in the frame memory in the frame synchronizer 230 (hereinafter, the amount of change in the number of packets stored), copies or skips frames according to the amount of change in the number of packets stored, and updates the timestamp and sequence number of the RTP packet based on the internal time, thereby performing frame synchronization of the video data.

[0019] The IP receiver 220 receives the RTP packets output from the frame synchronizer 210 based on the broadcast station internal time PTP2.

[0020] Next, a detailed configuration example of the frame synchronizer 210 according to the first embodiment will be described with reference to Fig. 2. In the following description, the configuration of the frame synchronizer 210 is shown in a functional block diagram, but it goes without saying that it can be realized by programs stored in various storage media and processors and working memories that execute the programs.

[0021] As shown in Figure 2, the frame synchronization device 210 includes a time generation unit 211, an RTP receiving unit 212, a frame memory 213, an RTP reception time acquisition unit 214, an RTP time stamp acquisition unit 215, a frame control unit 216, a read control unit 217, a time stamp stamping unit 218, and a sequence number replacement unit 219.

[0022] The time generating unit 211 receives the broadcast station internal time PTP2 from the PTP master 201, and generates a frame synchronizer internal time synchronized with the received broadcast station internal time PTP2. The RTP receiving unit 212 receives an RTP packet transmitted from the transmission source 100 via the network NW, and outputs the packet to the frame memory 213 , the RTP reception time acquiring unit 214 , and the RTP time stamp acquiring unit 215 .

[0023] For example, an RTP packet has a structure as shown in Fig. 3. That is, as shown in Fig. 3(a), one RTP packet consists of an IP header, a UDP header, meta information, and media information. The meta information also consists of an RTP header and a media payload header.

[0024] The RTP header has a structure as shown in Fig. 3(b). One frame of video data is divided and stored in several consecutive RTP packets, and the RTP packet that stores the last frame of the divided frames has a marker bit M stored in the RTP header. Therefore, the break (boundary or boundary) of frames can be determined from this marker bit M.

[0025] The RTP header also includes the lower 16 bits of a sequence number, a timestamp, etc. The upper 16 bits of the sequence number are included in a media payload header having a structure as shown in FIG.

[0026] Here, the time stamp is a time stamped based on the source internal time PTP1 of the source 100. The sequence number is a counter value that is incremented and assigned for each RTP packet.

[0027] The frame memory 213 stores (buffers) the RTP packets received by the RTP receiving unit 212 without depacketizing them, and outputs the stored RTP packets under the control of the read control unit 217 .

[0028] In addition, the frame memory 213 can store RTP packets of multiple frames, and, for example, the entire storage area is used cyclically to store and read data, and reading and writing are controlled using a read pointer indicating the read position and a write pointer indicating the write position.

[0029] The RTP reception time acquisition unit 214 monitors the RTP header of the RTP packet output from the RTP reception unit 212 to the frame memory 213, and when the RTP packet contains a marker bit M indicating the last packet of the frame, acquires the reception time of this RTP packet based on the internal time of the frame synchronization device. This reception time is output to the frame control unit 216.

[0030] The RTP timestamp acquisition unit 215 monitors the RTP header of the RTP packet output from the RTP receiving unit 212 to the frame memory 213, and if the RTP packet contains a marker bit M indicating the last packet of the frame, acquires a timestamp from the RTP header of this RTP packet. This timestamp is output to the frame control unit 216.

[0031] The frame control unit 216 obtains the time difference between the reception time output from the RTP reception time acquisition unit 214 and the time indicated by the time stamp output from the RTP time stamp acquisition unit 215. The frame control unit 216 then detects the amount of time difference change that may occur between this time difference and a previously obtained time difference. This amount of time difference change is output to the read control unit 217.

[0032] The read control unit 217 reads out the RTP packets stored in the frame memory 213 in accordance with the amount of change in the time difference. The read RTP packets are output to the time stamping unit 218.

[0033] The time stamp stamping unit 218 stamps the time stamp included in the RTP header of the read RTP packet by replacing it with a time stamp based on the frame synchronization device internal time generated by the time generation unit 211. The RTP packet with the replaced time stamp is output to the sequence number replacement unit 219.

[0034] The sequence number reassignment unit 219 includes an internal counter, and renumbers the sequence numbers included in the RTP header and media payload header of the RTP packet to values ​​incremented by the counter.

[0035] As a result, consecutive sequence numbers are assigned to the RTP packets read out from the frame memory 213. The RTP packets whose timestamps and sequence numbers have been updated in this manner are output to the IP receiver 220.

[0036] Next, the operation of the frame synchronization device 210 according to the first embodiment will be described with reference to Fig. 4. In parallel with the process described below, the RTP receiving unit 212 sequentially receives RTP packets, and the received RTP packets are stored in the frame memory 213.

[0037] In step 401, the RTP reception time acquisition unit 214 acquires the reception time t(N) of an RTP packet of a certain frame N. Here, N is an arbitrary natural number defined below to explain the continuity of frames, and the frame next to frame N is called frame N+1.

[0038] Specifically, the RTP reception time acquisition unit 214 monitors the RTP header of the RTP packet received by the RTP reception unit 212. When an RTP packet including a marker bit M indicating the last packet of N frames is detected, the RTP reception time acquisition unit 214 acquires the reception time t(N) of the detected RTP packet based on the frame synchronization device internal time, outputs the reception time t(N) to the frame control unit 216, and proceeds to step 402.

[0039] In step 402 , the RTP time stamp acquisition unit 215 acquires the time stamp ts(N) for frame N from the RTP packet in the same manner as the RTP reception time acquisition unit 214 in step 401 .

[0040] Specifically, RTP timestamp acquiring unit 215 monitors the RTP header of the RTP packet received by RTP receiving unit 212. When an RTP packet including marker bit M indicating the last packet of N frames is detected, RTP timestamp acquiring unit 215 acquires timestamp ts(N) from the RTP header of the detected RTP packet, outputs the timestamp ts(N) to frame control unit 216, and proceeds to step 403.

[0041] In step 403, the frame control unit 216 calculates the time difference Δt(N) between the reception time t(N) of frame N acquired by the RTP reception time acquisition unit 214 in step 401 and the time indicated by the timestamp ts(N) of frame N acquired by the RTP timestamp acquisition unit 215 in step 402, and proceeds to step 404.

[0042] In step 404, the RTP reception time acquisition unit 214 acquires the reception time t(N+1) of the RTP packet of frame N+1. As described above, frame N+1 is the next video frame following frame N.

[0043] Specifically, the RTP reception time acquisition unit 214 monitors the RTP header of the RTP packet received by the RTP reception unit 212. When an RTP packet including a marker bit M indicating the last packet of the N+1 frame is detected, the RTP reception time acquisition unit 214 acquires the reception time t(N+1) of the detected RTP packet based on the frame synchronization device internal time, outputs the reception time t(N+1) to the frame control unit 216, and proceeds to step 405.

[0044] In step 405, the RTP time stamp acquisition unit 215 acquires the time stamp ts(N+1) from the RTP packet for frame N+1, similarly to the RTP reception time acquisition unit 214 in step 404.

[0045] Specifically, RTP timestamp acquisition unit 215 monitors the RTP header of the RTP packet received by RTP reception unit 212. When an RTP packet including marker bit M indicating the last packet of frame N+1 is detected, RTP timestamp acquisition unit 215 acquires timestamp ts(N+1) from the RTP header of the detected RTP packet, outputs timestamp ts(N+1) to frame control unit 216, and proceeds to step 406.

[0046] In step 406, the frame control unit 216 calculates the time difference Δt(N+1) between the reception time t(N+1) of frame N+1 acquired by the RTP reception time acquisition unit 214 in step 404 and the time indicated by the timestamp ts(N+1) of frame N+1 acquired by the RTP timestamp acquisition unit 215 in step 405, and proceeds to step 407.

[0047] In step 407, the frame control unit 216 obtains the amount of time difference change that may occur between the time difference Δt(N+1) and the time difference Δt(N), and proceeds to processing according to the amount of time difference change. Specifically, the frame control unit 216 obtains the difference between the time difference Δt(N+1) and the time difference Δt(N) as the time difference change amount, and outputs this time difference change amount to the read control unit 217.

[0048] If the time difference change amount is negative, the process proceeds to step 408 , whereas if the time difference change amount is not negative, the process proceeds to step 411 .

[0049] Here, when the time difference change amount is negative, it means that PTP1 of the transmission source 100 is relatively earlier than PTP2 of the broadcast station 200. Conversely, when the time difference change amount is not negative, it means that PTP1 of the transmission source 100 is relatively later than or the same as PTP2 of the broadcast station 200.

[0050] In step 408, the read control unit 217 detects the number of RTP packets stored in the frame memory 213, and judges whether the frame memory 213 will overflow (the buffered RTP packets will exceed the specified storage capacity) when storing several consecutive RTP packets that constitute the next frame. If it is judged that the frame memory 213 will overflow, the process proceeds to step 409, whereas if it is judged that the frame memory will not overflow, the process proceeds to step 410.

[0051] In step 409, the read control unit 217 performs frame skipping and reads out the RTP packets stored in the frame memory 213. The read control unit 217 then outputs the read out RTP packets to the time stamping unit 218 and proceeds to step 413.

[0052] More specifically, the read control unit 217 skips the RTP packet of the oldest frame among the RTP packets stored in the frame memory 213 (changes the position of the read pointer to skip one frame), reads the RTP packet of the next frame, and outputs this read RTP packet to the time stamping unit 218.

[0053] The RTP packets of the skipped frames and the read RTP packets are erased from the frame memory 213. The word "erase" here is used to simplify the explanation, and does not necessarily mean that the process of erasing data is actually executed.

[0054] For example, the above erasure may be a change in the position of a write pointer indicating the start address of writing data to the storage area of ​​the frame memory 213 so that the area in which the read RTP packet is stored becomes the target for overwriting. In the following description, for the sake of simplicity, the substantial data erasure by changing the position of the write pointer is simply referred to as erasure.

[0055] On the other hand, in step 410, the read control unit 217 reads out the oldest RTP packet stored in the frame memory 213 at the normal timing. Then, the read control unit 217 outputs this read RTP packet to the time stamping unit 218, and the process proceeds to step 413. The read RTP packet is also erased from the frame memory 213.

[0056] In step 411, the read control unit 217 detects the number of RTP packets stored in the frame memory 213, and judges whether the frame memory 213 will underflow (the buffered RTP packets will fall below a specified amount) when reading out several consecutive RTP packets that make up the next frame. If it is judged that the frame memory 213 will underflow, the process proceeds to step 412, whereas if it is judged that the frame memory 213 will not underflow, the process proceeds to step 410.

[0057] In step 412, the read control unit 217 performs frame repeat and reads twice the RTP packets of the same frame stored in the frame memory 213. The read control unit 217 then outputs the read RTP packets to the time stamping unit 218 and proceeds to step 413.

[0058] More specifically, the read control unit 217 reads out all of the RTP packets that make up the oldest frame from the RTP packets stored in the frame memory 213 and outputs them to the time stamping unit 218, and then, in order to read the same frame again, it moves the read pointer back one frame, reads out the RTP packets of the same frame again, and outputs them to the time stamping unit 218, and then erases the RTP packets of the frame that were read out twice from the frame memory 213.

[0059] In step 413, the time stamp stamping unit 218 replaces the time stamp included in the RTP header of the RTP packet read by the read control unit 217 with a time stamp based on the frame synchronization device internal time generated by the time generation unit 211, and the process proceeds to step 414. The RTP packet with the replaced time stamp is output to the sequence number replacement unit 219.

[0060] In step 414, the sequence number reassignment unit 219 renumbers the sequence numbers included in the RTP header and media payload header of the RTP packet output from the time stamping unit 218 to a value counted by an internal counter.

[0061] As a result, the RTP packets read out from the frame memory 213 are given consecutive sequence numbers unique to the frame synchronizer 210. The RTP packets with updated sequence numbers are output to the IP receiver 220.

[0062] As described above, in the content distribution system configured as above, the frame synchronizer 210 provided in the broadcasting station 200 calculates the difference between the time indicated by the time stamp included in the RTP packet and the time the RTP packet was received, and adjusts the number of frames by skipping or repeating frames according to the amount of change in this difference (the amount of change in the time difference). In addition, the adjusted RTP packets are stamped with a time stamp based on the broadcasting station internal time PTP2, and the sequence numbers are renumbered.

[0063] Therefore, according to the content distribution system having the above configuration, even if the RTP packet is asynchronous with the broadcast station internal time PTP2 of the broadcast station 200, an RTP packet synchronized with the broadcast station internal time PTP2 can be generated with low computational cost without depacketizing / packetizing.

[0064] Next, a detailed configuration example of the frame synchronizer 230 according to the second embodiment will be described with reference to Fig. 5. In the following description, the configuration of the frame synchronizer 230 is shown in a functional block diagram, but it goes without saying that it can be realized by programs stored in various storage media, and processors and working memories that execute these programs.

[0065] The overall system configuration is the same as that shown in Fig. 1, and therefore the description thereof will be omitted. The structure of the RTP packet handled in the second embodiment is the same as that explained in the first embodiment using Fig. 3, and therefore the detailed description thereof will be omitted.

[0066] As shown in Figure 5, the frame synchronization device 230 includes a time generation unit 231, an RTP receiving unit 232, a frame memory 233, a packet storage count acquisition unit 234, a sequence number acquisition unit 235, a frame control unit 236, a read control unit 237, a time stamp stamping unit 238, and a sequence number replacement unit 239.

[0067] The time generating unit 231 receives the broadcast station internal time PTP2 from the PTP master 201, and generates a frame synchronizer internal time synchronized with the received broadcast station internal time PTP2. The RTP receiving unit 232 receives the RTP packets transmitted from the transmission source 100 via the network NW, and outputs them to the frame memory 233 and the sequence number acquiring unit 235 .

[0068] The frame memory 233 stores (buffers) the RTP packets received by the RTP receiving unit 232 without depacketizing them, and outputs the stored RTP packets under the control of the read control unit 237. The frame memory 233 can also store RTP packets of multiple frames, and for example, the entire storage area is cyclically used to store and read data.

[0069] The packet storage count acquisition unit 234 acquires the number of RTP packets stored in the frame memory 233 for each frame of time based on the frame synchronization device internal time generated by the time generation unit 231, and outputs that number (hereinafter referred to as the storage count) to the frame control unit 236.

[0070] A specific method for obtaining the number of packets is to obtain it from the difference between the positions of the write pointer and the read pointer of the frame memory 233. When frame skip or repeat, which will be described later, is performed, a value that compensates for that is obtained.

[0071] The sequence number acquisition unit 235 monitors the RTP header of the RTP packet output from the RTP receiving unit 232 to the frame memory 233, and if the RTP packet contains a marker bit M indicating the last packet of the frame, acquires a sequence number from the meta information of the RTP packet. The acquired sequence number is output to the frame control unit 236.

[0072] Frame control unit 236 obtains the amount of change in the sequence numbers sequentially obtained by sequence number obtaining unit 235 (the difference in sequence numbers between one frame, hereinafter referred to as the amount of change in number), and compares this amount of change in number with the amount of change in the number of stored packets obtained by stored packet number obtaining unit 234 (hereinafter referred to as the amount of change in the number of stored packets). Then, frame control unit 236 instructs read control unit 237 to execute processing according to the result of this comparison.

[0073] The read control unit 237 reads out the RTP packets stored in the frame memory 233 in accordance with an instruction from the frame control unit 236. The read RTP packets are output to a time stamping unit 238.

[0074] The time stamp stamping unit 238 stamps the time stamp included in the RTP header of the read RTP packet by replacing it with a time stamp based on the frame synchronization device internal time generated by the time generation unit 231. The RTP packet with the replaced time stamp is output to the sequence number replacement unit 239.

[0075] The sequence number reassignment unit 239 includes an internal counter, and renumbers the sequence numbers included in the RTP header and media payload header of the RTP packet to values ​​incremented by the counter.

[0076] As a result, consecutive sequence numbers are assigned to the RTP packets read out from the frame memory 233. The RTP packets whose timestamps and sequence numbers have been updated in this manner are output to the IP receiving device 220.

[0077] Next, the operation of the frame synchronizer 230 according to the second embodiment will be described with reference to Fig. 6. In parallel with the process described below, the RTP receiving unit 232 sequentially receives RTP packets, and the received RTP packets are stored in the frame memory 233.

[0078] In step 601, the frame control unit 236 compares the sequence numbers sequentially acquired by the sequence number acquisition unit 235, obtains the number change amount, and proceeds to step 602. That is, the difference between the sequence number of an RTP packet containing marker bit M of a certain frame N and the sequence number of an RTP packet containing marker bit M of the following frame N+1 is obtained as the number change amount m. Here, N is an arbitrary natural number defined below to explain the continuity of frames, and the frame following frame N is referred to as frame N+1.

[0079] In step 602, the packet storage count acquisition unit 234 acquires the number of RTP packets stored in the frame memory 233 at a certain time T, W(T), and outputs this number of stored packets W(T) and the acquisition time to the frame control unit 236, and then proceeds to step 603.

[0080] In step 603, the packet storage count acquisition unit 234 acquires the number of RTP packets stored W(T+1) stored in the frame memory 233 at the timing (T+1) when one frame's worth of time has elapsed since time T, outputs this stored number W(T+1) and the acquisition time to the frame control unit 236, and proceeds to step 604.

[0081] In step 604, the frame control unit 236 compares and determines whether the difference between the stored number W(T+1) and the stored number W(T) is greater than the number change amount m determined in step 601. If the difference between the stored number W(T+1) and the stored number W(T) is greater than the number change amount m, the process proceeds to step 605. On the other hand, if the difference is not greater than the number change amount m, the process proceeds to step 608.

[0082] Here, if the difference is greater than the number change amount m, then PTP1 of the source 100 is relatively earlier than PTP2 of the broadcast station 200. Conversely, if the difference is not greater than the number change amount m, then PTP1 of the source 100 is relatively later than or the same as PTP2 of the broadcast station 200.

[0083] In step 605, the read control unit 237 detects the number of RTP packets stored in the frame memory 233, and judges whether the frame memory 233 will overflow (the buffered RTP packets will exceed the specified storage capacity) when storing several consecutive RTP packets that constitute the next frame. If it is judged that the frame memory 233 will overflow, the process proceeds to step 606, whereas if it is judged that the frame memory will not overflow, the process proceeds to step 607.

[0084] In step 606, the read control unit 237 performs frame skipping and reads out the RTP packets stored in the frame memory 233. The read control unit 237 then outputs the read out RTP packets to the time stamping unit 238 and proceeds to step 610.

[0085] More specifically, the read control unit 237 skips packets corresponding to the difference m calculated in step 601 from among the RTP packets stored in the frame memory 233 that constitute the oldest frame, reads the next RTP packet, and outputs the read RTP packets to the time stamping unit 238.

[0086] The skipped RTP packets and the read RTP packets are deleted from the frame memory 233 (the position of the write pointer is changed to the area to be written).

[0087] On the other hand, in step 607, the read control unit 237 reads the oldest RTP packet stored in the frame memory 233 at the normal timing. Then, the read control unit 237 outputs this read RTP packet to the time stamping unit 238, and the process proceeds to step 610. The read RTP packet is also deleted from the frame memory 233.

[0088] In step 608, the read control unit 237 detects the number of RTP packets stored in the frame memory 233, and judges whether the frame memory 233 will underflow (the buffered RTP packets will fall below a specified amount) when several consecutive RTP packets constituting the next frame are read. If it is judged that the frame memory 233 will underflow, the process proceeds to step 609, whereas if it is judged that the frame memory 233 will not underflow, the process proceeds to step 607.

[0089] In step 609, the read control unit 237 performs frame repeat to read twice the RTP packets of the same frame stored in the frame memory 233. The read control unit 237 then outputs the read RTP packets to the time stamping unit 238 and proceeds to step 610.

[0090] More specifically, the read control unit 237 reads out the RTP packets that constitute the oldest frame from the RTP packets stored in the frame memory 233 and outputs them to the time stamping unit 238, and then, for example, moves the position of the packet read pointer back by the difference m calculated in step 601, reads out RTP packets again by the difference m, and outputs them to the time stamping unit 238, and then erases the RTP packets that were read out twice from the frame memory 233.

[0091] In step 610, the time stamp stamping unit 238 replaces the time stamp included in the RTP header of the RTP packet read by the read control unit 237 with a time stamp based on the frame synchronization device internal time generated by the time generation unit 231, and the process proceeds to step 611. The RTP packet with the replaced time stamp is output to the sequence number replacement unit 239.

[0092] In step 611, the sequence number reassignment unit 239 renumbers the sequence numbers included in the RTP header and media payload header of the RTP packet output from the time stamp stamping unit 238 to a value counted by an internal counter.

[0093] As a result, the RTP packets read out from the frame memory 233 are given consecutive sequence numbers unique to the frame synchronizer 230. The RTP packets with updated sequence numbers are output to the IP receiver 220.

[0094] As described above, in the content distribution system configured as above, the frame synchronizer 230 provided in the broadcasting station 200 calculates the difference in sequence numbers between one frame (the above-mentioned number change amount), calculates the change amount of the number of packets stored in the frame memory 233 within one frame time based on the broadcasting station internal time PTP2, and adjusts the number of frames by skipping or repeating frames according to the magnitude relationship between the number change amount and the change amount of the number of packets stored. In addition, the adjusted RTP packets are stamped with a time stamp based on the broadcasting station internal time PTP2, and the sequence numbers are renumbered.

[0095] Therefore, according to the content distribution system having the above configuration, even if the RTP packets are asynchronous with the broadcast station internal time PTP2 of the broadcast station 200, it is possible to generate RTP packets synchronized with the broadcast station internal time PTP2 at low computational cost without depacketizing / packetizing. Therefore, frame synchronization processing can be performed quickly.

[0096] It should be noted that the present invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, a configuration in which some components are deleted from all the components shown in the embodiment can also be considered. Furthermore, the components described in different embodiments may be appropriately combined.

[0097] For example, in the above embodiment, the transmission source 100 is described as using a time source other than GNSS, and the broadcast station 200 is described as using GNSS as its time source, but the reverse is also possible. That is, the present invention can also be applied to a case where the transmission source 100 uses GNSS as its time source, and the broadcast station 200 uses a time source other than GNSS.

[0098] This can also be applied to cases where the time source of both the transmission source 100 and the broadcast station 200 is GNSS or other than GNSS. For example, even when both use GNSS as the time source, frame synchronization can be performed by the frame synchronizer 210 or 230 even if some kind of problem occurs on the transmission source 100 side and an inaccurate timestamp is stamped on the RTP packet.

[0099] In the above embodiment, Δt(N) and W(T) are calculated for consecutive frames N and N+1, but the present invention is not limited to consecutive frames. For example, frames separated by a preset number L may be used as the comparison target, or the average value of values ​​calculated for consecutive or discontinuous L frames may be used.

[0100] Furthermore, the above L may be dynamically changed depending on the magnitude of the frequency deviation (e.g., Δt(N+L)-Δt(N) or W(T+L)-W(T)), the traffic in the network NW, the communication time zone, etc.

[0101] In the above second embodiment, the read control unit 237 skips or reads twice packets by the amount of change m in the sequence number in response to the change in the number of stored packets during a time period equivalent to one frame so as to prevent the frame memory 233 from overflowing or underflowing. However, the present invention is not limited to this. (First modified example of the second embodiment) As a first alternative variant, the frame control unit 236 calculates the difference between the sequence number of an RTP packet containing marker bit M in frame N and the sequence number of an RTP packet containing marker bit M in the next frame N+1 as the number change amount m, and calculates the change amount m' of the sequence number obtained from the packet data received at a timing based on the time generated by the time generation unit 231, and the read control unit 237 skips or reads twice the packets according to the changes m and m' to prevent the frame memory 233 from overflowing or underflowing.

[0102] Specifically, the frame control unit 236 calculates the difference m in the same manner as in step 601, and, as an alternative process to steps 602 to 603, calculates the amount of change (difference m') in the sequence number per frame of time based on the sequence number of the RTP packet received at the timing of each frame of time based on the device internal time generated by the time generation unit 231.

[0103] Then, as processing equivalent to step 604, the frame control unit 236 determines the magnitude relationship between the differences m and m', and as processing equivalent to steps 605 to 609, the read control unit 237 performs frame skip or frame repeat depending on whether the frame memory overflows or underflows, to prevent this.

[0104] Even in the first modified example, it is possible to generate RTP packets synchronized with the broadcast station internal time PTP2 with little computational cost, and to perform frame synchronization processing quickly. (Second Modification of the Second Embodiment) As a second modified example, in response to a change in the number of packets received during a time period equivalent to one frame, the read control unit 237 skips or reads twice packets by the amount of change m in the sequence number so as to prevent the frame memory 233 from overflowing or underflowing.

[0105] Specifically, a packet reception count acquisition unit 234' is provided as a control function replacing the packet storage count acquisition unit 234. The packet reception count acquisition unit 234' monitors the output of the RTP reception unit 232, and counts the number of received RTP packets.

[0106] Then, in step 602, the packet reception count acquisition unit 234' counts the number of RTP packets received up to a certain time T (the cumulative value of the number of receptions) to acquire the reception count N(T), and in step 603, counts the number of RTP packets received up to the time T+1 one frame time later (the cumulative value) to acquire the reception count N(T+1).

[0107] Then, as a process equivalent to step 604, frame control section 236 compares and determines whether the difference between reception number N(T+1) and reception number N(T) is greater than the number change amount m obtained in step 601.

[0108] Thereafter, similarly to the second embodiment, in steps 605 to 609, the read control unit 237 performs frame skip or frame repeat on packets corresponding to the difference m depending on whether the frame memory overflows or underflows in order to prevent this.

[0109] Even in the second modified example, it is possible to generate RTP packets synchronized with the broadcast station internal time PTP2 with little computational cost, and to perform frame synchronization processing quickly.

[0110] It goes without saying that various modifications can be made without departing from the spirit and scope of the present invention. The invention as described in the claims of the original application is set forth below. [C1] a receiving unit for receiving packet data including video data and a timestamp; a frame memory for storing the packet data received by the receiving unit; a time generating unit that generates a time based on a reference synchronization signal; a reception time acquisition unit that acquires a reception time of packet data that satisfies a predetermined condition based on the time generated by the time generation unit; a time stamp acquisition unit that acquires a time stamp from packet data that satisfies a predetermined condition; a control unit that calculates a difference between the reception time and the time indicated by the time stamp, and reads packet data from the frame memory in accordance with a change in this difference; A frame synchronization device comprising: [C2] The video data of one frame is divided into a plurality of consecutive packet data and stored; the packet data includes a marker bit indicating a frame boundary when subsequent packet data includes video data of another frame; the reception time acquisition unit acquires a reception time of the packet data including the marker bit based on the time generated by the time generation unit; The time stamp acquisition unit acquires a time stamp from packet data including the marker bit. The frame synchronization device according to [C1]. [C3] The frame synchronization device described in [C2] is characterized in that the control unit calculates the difference between the reception time and the time indicated by the timestamp for each of two consecutive frames, and reads packet data from the frame memory according to the amount of change in the difference between the two. [C4] The frame synchronization device described in [C2] is characterized in that the control unit calculates the difference between the reception time and the time indicated by the timestamp for each of two non-consecutive frames, and reads packet data from the frame memory according to the amount of change in the difference between the two. [C5] The frame synchronization device according to any one of [C1] to [C4], wherein the control unit reads packet data from the frame memory by frame skipping or frame repeating according to the amount of change. [C6] a receiving unit for receiving packet data including video data and a sequence number; a frame memory for storing the packet data received by the receiving unit; a time generating unit that generates a time based on a reference synchronization signal; a storage number acquisition unit that acquires the number of packet data stored in the frame memory at a predetermined timing based on the time generated by the time generation unit; a sequence number acquisition unit that acquires a sequence number from packet data that satisfies a predetermined condition; a control unit that reads packet data from the frame memory in accordance with the amount of change in the number of packet data pieces acquired by the storage number acquisition unit and the amount of change in the sequence number acquired by the sequence number acquisition unit; A frame synchronization device comprising: [C7] The video data of one frame is divided into a plurality of consecutive packet data and stored; the packet data includes a marker bit that indicates a frame boundary when the subsequent packet data includes video data of another frame; the storage number acquisition unit acquires the number of packet data stored in the frame memory at a timing corresponding to the frame based on the time generated by the time generation unit; The sequence number acquisition unit acquires a sequence number from packet data including the marker bit. The frame synchronization device according to [C6]. [C8] The control unit calculates, for two consecutive frames, the amount of change in the number of packet data acquired by the storage number acquisition unit and the amount of change in the sequence number acquired by the sequence number acquisition unit, and reads packet data from the frame memory in accordance with the amount of change in the number of packet data and the amount of change in the sequence number.The frame synchronization device described in [C7] is characterized in that [C9] The control unit calculates, for two non-consecutive frames, the amount of change in the number of packet data acquired by the storage number acquisition unit and the amount of change in the sequence number acquired by the sequence number acquisition unit, and reads packet data from the frame memory in accordance with the amount of change in the number of packet data and the amount of change in the sequence number.The frame synchronization device described in [C7] is characterized in that [C10] a receiving unit for receiving packet data including video data and a sequence number; a frame memory for storing the packet data received by the receiving unit; a time generating unit that generates a time based on a reference synchronization signal; a sequence number acquisition unit that acquires a sequence number from packet data; and a control unit that reads packet data from the frame memory in accordance with an amount of change in the sequence number acquired from packet data that satisfies a predetermined condition and an amount of change in the sequence number acquired from packet data received at a timing based on the time generated by the time generation unit. A frame synchronization device comprising: [C11] a receiving unit for receiving packet data including video data and a sequence number; a frame memory for storing the packet data received by the receiving unit; a time generating unit that generates a time based on a reference synchronization signal; a reception count acquisition unit that acquires the number of receptions of packet data at a predetermined timing based on the time generated by the time generation unit; a sequence number acquisition unit that acquires a sequence number from packet data that satisfies a predetermined condition; a control unit that reads packet data from the frame memory in accordance with a change in the number of receptions acquired by the reception number acquisition unit and a change in the sequence number acquired by the sequence number acquisition unit; A frame synchronization device comprising: [C12] The frame synchronization device described in any of [C6] to [C11], characterized in that the control unit reads packet data from the frame memory by frame skipping or frame repeating depending on the change in the number of packet data and the change in the sequence number. [C13] A frame synchronization device described in any of [C1] to [C12], further characterized in that it has a time stamp unit that replaces the time stamp included in the packet data read by the control unit with a time stamp based on the time. [C14] The frame synchronization device according to any one of [C1] to [C13], further comprising a replacement unit that renumbers the sequence number contained in the packet data read by the control unit. [Explanation of symbols]

[0111] 100...source, 101...PTP master, 102...IP transmission device, 200...broadcast station, 201...PTP master, 210...frame synchronization device, 211...time generation unit, 212...RTP receiving unit, 213...frame memory, 214...RTP reception time acquisition unit, 215...RTP time stamp acquisition unit, 216...frame control unit, 217...read control unit, 218...time stamp stamping unit, 219...sequence number replacement unit, 220...IP receiving device, 230...frame synchronization device, 231...time generation unit, 232...RTP receiving unit, 233...frame memory, 234...packet storage number acquisition unit, 235...sequence number acquisition unit, 236...frame control unit, 237...read control unit, 238...time stamp stamping unit, 239...sequence number replacement unit.

Claims

1. a receiving unit for receiving packet data including video data and a timestamp; a frame memory for storing the packet data received by the receiving unit; a time generating unit that generates a time based on a reference synchronization signal; a reception time acquisition unit that acquires a reception time of packet data that satisfies a predetermined condition based on the time generated by the time generation unit; a time stamp acquisition unit that acquires a time stamp from packet data that satisfies a predetermined condition; a control unit that calculates a difference between the reception time and the time indicated by the time stamp, and reads packet data from the frame memory in accordance with a change in this difference; Equipped with The video data of one frame is divided into a plurality of consecutive packet data and stored, the packet data includes a marker bit indicating a frame boundary when subsequent packet data includes video data of another frame; the reception time acquisition unit acquires a reception time of the packet data including the marker bit based on the time generated by the time generation unit; The time stamp acquisition unit acquires a time stamp from packet data including the marker bit.

2. The frame synchronization device according to claim 1, characterized in that the control unit calculates the difference between the reception time and the time indicated by the time stamp for each of two consecutive frames, and reads packet data from the frame memory according to the amount of change in the difference between the two.

3. The frame synchronization device according to claim 1, characterized in that the control unit calculates the difference between the reception time and the time indicated by the time stamp for each of two non-consecutive frames, and reads packet data from the frame memory according to the amount of change in the difference between the two.

4. 4. The frame synchronization device according to claim 1, wherein said control unit reads out packet data from said frame memory by frame skipping or frame repeating in accordance with said amount of change.

Images