Video delay measurement device and program

Abstract

To provide a program production system that measures a delay time of a video image with respect to audio and auxiliary data by a simple method so as to realize synchronization among the video image, the audio and the auxiliary data generated at the same time.SOLUTION: The video time difference calculating unit 11 - 1 of the video delay measuring apparatus 2 extracts the time stamp V1 from the video RTP packet VTS1 and generates the video RTP packet V2 including the timing information. The video RTP packet V2 is transmitted to the video delay time difference measuring path. A video time difference calculation section 11-1 extracts a time stamp V3 from a video RTP packet VTS2 to obtain a video time difference VT. The audio time difference calculator 11-2 obtains the audio time difference ST, and the ancillary data time difference calculator 11 - 3 obtains the ancillary data time difference AT. The delay time calculating unit 13 subtracts the audio time difference ST from the video time difference VT to obtain the audio delay time SD, and subtracts the ancillary data time difference AT from the video time difference VT to obtain the ancillary data delay time AD.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a video delay measuring device and program for measuring a delay time for synchronizing packetized video, audio and auxiliary data in the field of broadcasting technology. [Background technology]

[0002] Conventionally, in a program production system for producing broadcast programs, a method has been proposed for appropriately delaying each piece of data in order to synchronize video, audio, and auxiliary data packetized in ST2110 format (see, for example, Non-Patent Documents 1, 2, and 3).

[0003] 14 is a schematic diagram showing an example of the overall configuration of a conventional program production system. In this program production system 100, delay times between video, audio, and auxiliary data are measured manually by a user, and synchronization of these data is realized, whereby an SDI signal including each synchronized data is generated and output.

[0004] For example, synchronization is achieved between the video captured by the camera 101, the audio picked up by the microphone 106, and auxiliary data, which is a time code generated by the time code generator 105.

[0005] Figure 14 shows a video stream made up of video RTP packets (see black circles), an audio stream made up of audio RTP packets (see white triangles), and an auxiliary data stream made up of auxiliary data RTP packets (see white squares).

[0006] The camera control unit 102 assigns a timestamp to the video captured by the camera 101, and generates an RTP packet for the video. The camera control unit 102 also assigns a timestamp of the same time as the video to the audio captured by the camera 101, and generates an RTP packet for the audio.

[0007] The video RTP packets and audio RTP packets generated by the camera control unit 102 are transferred along a predetermined path in accordance with instructions from a server that performs transfer control (hereinafter referred to as the "transfer control server") not shown in Figure 14.

[0008] For example, video RTP packets are transferred so as to pass through a path from the camera control unit 102 to the network switch 108 , the video switcher 109 , another network switch 108 , and the IP gateway 114 .

[0009] 14 shows one network switch 108 for convenience, but in reality, a plurality of network switches 108 are installed. The same applies to the voice network switch 110.

[0010] Furthermore, the SDI signal (video, audio, and auxiliary SDI signal) containing the video, audio, and auxiliary data played back by the recording and playback device 103 is output to the IP gateway 104. Then, the IP gateway 104 assigns the same time stamp to each of the video, audio, and auxiliary data, and generates video RTP packets, audio RTP packets, and auxiliary data RTP packets.

[0011] The video RTP packets, audio RTP packets, and auxiliary data RTP packets generated by the IP gateway 104 are transferred through a predetermined path using the same transfer control.

[0012] Furthermore, a timestamp is added to the auxiliary data, which is a time code generated by the time code generator 105, by the time code generator 105, and an RTP packet for the auxiliary data is generated.

[0013] The ancillary data RTP packets generated by the time code generator 105 are transferred through a predetermined path using similar transfer control. For example, the ancillary data RTP packets are transferred through a path from the time code generator 105 to the network switch 108, the ancillary data delay device 113, another network switch 108, and the IP gateway 114.

[0014] Furthermore, a timestamp is added to the voice picked up by the microphone 106 by the voice IP gateway 107, and a voice RTP packet is generated.

[0015] The audio RTP packets generated by the audio IP gateway 107 are transferred through a predetermined path using similar transfer control. For example, the audio RTP packets are transferred through a path from the audio IP gateway 107 to the audio network switch 110, audio mixer 111, audio network switch 110, network switch 108, audio delay device 112, network switch 108, and IP gateway 114.

[0016] In such a program production system 100, the video switcher 109 performs video processing on the video in the video RTP packets received from the network switch 108, and when the processed video RTP packets are output, a new timestamp is added. In this case, the timestamp added by the camera control unit 102 is not carried over, but is overwritten with the new timestamp, causing delays due to buffering associated with the video processing.

[0017] Similarly, the audio mixer 111 processes the audio of the audio RTP packets received from the audio network switch 110, and adds a new timestamp when outputting the processed audio RTP packets. In this case, the timestamp added by the audio IP gateway 107 is not carried over, but is overwritten with the new timestamp, causing delays due to buffering associated with the audio processing.

[0018] 4 (not shown in Fig. 14) performs auxiliary data processing on the auxiliary data in the auxiliary data RTP packet received from the network switch 108, and a new timestamp is added when the auxiliary data RTP packet after the auxiliary data processing is output. In this case, the timestamp added by the auxiliary data processing device 118 is not carried over but is overwritten with the new timestamp, causing a delay due to buffering associated with the auxiliary data processing.

[0019] The IP gateway 114 receives video RTP packets, audio RTP packets, and auxiliary data RTP packets from the network switch 108. Then, the IP gateway 114 associates the delayed audio and auxiliary data with the video by referring to the timestamps included in each RTP packet.

[0020] That is, the IP gateway 114 identifies the video RTP packets, audio RTP packets, and auxiliary data RTP packets of the same time by selecting the audio RTP packets and auxiliary data RTP packets corresponding to the video RTP packets. Specifically, the IP gateway 114 identifies the audio RTP packets and auxiliary data RTP packets that include the same timestamp as the timestamp included in the video RTP packet (a timestamp within a predetermined threshold width of the timestamp included in the video RTP packet).

[0021] Then, the IP gateway 114 converts the identified video RTP packets, audio RTP packets, and auxiliary data RTP packets into an SDI signal containing video, audio, and auxiliary data (video, audio, and auxiliary SDI signal), and outputs this to the SDI synchronizer 115 and recording / playback devices 116 and 117.

[0022] Here, if the original timestamp is overwritten with a new timestamp by the video switcher 109, audio mixer 111, etc., the timestamp of the generation time is rewritten with the new timestamp of the processing time. This makes it difficult for the IP gateway 114 to identify and synchronize video RTP packets, audio RTP packets, and auxiliary data RTP packets that were generated at the same time, and a discrepancy occurs among the three types of RTP packets when they are converted into an SDI signal.

[0023] 14, the video stream generated by video RTP packets, the audio stream generated by audio RTP packets, and the auxiliary data stream generated by auxiliary data RTP packets are subject to delays depending on the devices and lines on the transmission paths. For example, the video is delayed by one frame in the video switcher 109, and the video is also delayed by one frame in the IP gateway 114. Furthermore, the audio is delayed by a predetermined time in the audio mixer 111, for example.

[0024] Because this delay amount varies depending on the equipment and line, when the IP gateway 114 converts these RTP packets into an SDI signal and outputs it, a delay time difference occurs among the three types of RTP packets. In other words, video generated at the same time as audio will be delayed relative to the audio, and video generated at the same time as auxiliary data will also be delayed relative to the auxiliary data. Such delays occur at multiple stages, not only due to the equipment and lines in the studio, but also due to the equipment and lines in the broadcast van, transmission equipment, etc.

[0025] FIG. 15 is a diagram illustrating the delay of video relative to audio, where (1) shows the timing of video v and audio s, which are input signals, and (2) shows the timing of video v and audio s, which are output signals.

[0026] As shown in the input signal (1), it is assumed that the microphone 106 picks up the sound s at the same timing as the frame f1 of the video v captured by the camera 101.

[0027] Video v (video RTP packets) is transferred so as to pass through a path from camera control unit 102 to network switch 108, video switcher 109, network switch 108, and IP gateway 114. A timestamp is added to video v in camera control unit 102, and a delay occurs in video switcher 109, where the timestamp is rewritten.

[0028] Furthermore, audio s (audio RTP packets) is transferred so as to pass through a path from audio IP gateway 107 to audio network switch 110, audio mixer 111, audio network switch 110, network switch 108, and IP gateway 114. A timestamp is assigned to audio s in audio IP gateway 107, and a delay occurs in audio mixer 111, where the timestamp is rewritten, but the delay is not as great as that of video v.

[0029] When the timing of the video v and audio s transferred to the IP gateway 114 is compared, as shown in the output signal (2), frame f1 of the video v relative to the audio s is delayed by, for example, one frame from the video v of the input signal. In other words, lip synchronization is lost.

[0030] FIG. 16 is a diagram illustrating the delay of video relative to audio and auxiliary data, where (1) shows the timing of the input signals, video v, audio s, and auxiliary data a, and (2) shows the timing of the output signals, video v', v'', audio s, and auxiliary data a.

[0031] As shown in the input signal (1), assume that audio s is picked up by microphone 106 at the same timing as frame f1 of video v captured by camera 101, and auxiliary data a with time code 10:00:00 is generated by time code generator 105.

[0032] For example, when video composition is performed in video switcher 109, video v (video RTP packets) is transferred as video v' so as to pass through the path from camera control unit 102 to network switch 108, video switcher 109, network switch 108, and IP gateway 114. A timestamp is assigned to video v in camera control unit 102, and a delay occurs in video switcher 109, whereby the timestamp is rewritten. Furthermore, when video composition is not performed in video switcher 109, for example, video v is transferred as video v" so as to pass through the path from camera control unit 102 to network switch 108 and IP gateway 114.

[0033] Furthermore, audio s (audio RTP packets) is transferred so as to pass through a path from audio IP gateway 107 to audio network switch 110, audio mixer 111, audio network switch 110, network switch 108, and IP gateway 114. A timestamp is assigned to audio s in audio IP gateway 107, and a delay occurs in audio mixer 111, where the timestamp is rewritten, but the delay is not as great as that of video v.

[0034] Furthermore, auxiliary data a (RTP packets for auxiliary data) is transferred so as to pass through the path from the time code generator 105 to the network switch 108 and the IP gateway 114 .

[0035] When the timing of the video v', v", audio s, and auxiliary data a transferred to the IP gateway 114 is compared, as shown in the output signal (2), frame f1 of video v' relative to audio s is delayed by, for example, one frame from the video v of the input signal. In other words, lip synchronization is out of sync. Furthermore, frame f1 of video v" relative to audio s has the same timing as the input signal, so there is no delay.

[0036] Furthermore, frame f1 of video v' corresponding to time code 10:00:00 in ancillary data a is delayed by 00:00:01 from video v in the input signal. Frame f1 of video v" corresponding to time code 10:00:00 in ancillary data a has the same timing as the input signal, so no delay occurs.

[0037] This shows that when video v is recorded using the same time code, the combination of video v' and audio s is different from the combination of video v" and audio s. This is because the amount of delay for video v differs between the path that goes through the video switcher 109 and the path that does not go through the video switcher 109.

[0038] Returning to FIG. 14, as described above, the IP gateway 114 refers to the timestamps contained in the video RTP packets, audio RTP packets, and auxiliary data RTP packets received from the network switch 108 to identify video RTP packets, audio RTP packets, and auxiliary data RTP packets of the same time, converts them into video, audio, and auxiliary SDI signals, and outputs them.

[0039] 15 and 16, the video stream is delayed relative to the audio stream and auxiliary data stream. That is, even if the video RTP packets, audio RTP packets, and auxiliary data RTP packets contain the same timestamps for their generation, the video RTP packets are delayed in video switcher 109 (larger than the audio RTP packets in audio mixer 111), causing the timestamps to be rewritten. This causes a mismatch in the reception timing of these RTP packets in IP gateway 114, making synchronization difficult.

[0040] To solve this problem, the audio delay device 112 and the auxiliary data delay device 113 perform delay processing to achieve synchronization of the video, audio, and auxiliary data among the three types of RTP packets.

[0041] Specifically, the user temporarily records a video stream made up of video RTP packets, an audio stream made up of audio RTP packets, and an auxiliary data stream made up of auxiliary data RTP packets in the recording / playback device 116, 117, or the like.

[0042] For example, to synchronize the video captured by the camera 101, the audio picked up by the microphone 106, and the auxiliary data, which is the time code generated by the time code generator 105, the video RTP packets are transferred along a path from the camera control unit 102 to the network switch 108, the video switcher 109, the network switch 108, and the IP gateway 114. The audio RTP packets are transferred along a path from the audio IP gateway 107 to the audio network switch 110, the audio mixer 111, the audio network switch 110, the network switch 108, the audio delay device 112, the network switch 108, and the IP gateway 114. The auxiliary data RTP packets are transferred along a path from the time code generator 105 to the network switch 108, the auxiliary data delay device 113, the network switch 108, and the IP gateway 114. The user records each stream that has passed through such a path.

[0043] The user then manually measures an audio delay time SD, which is the delay time between the audio and the video, and an ancillary data delay time AD, which is the delay time between the video and the ancillary data, by playing back each of the recorded streams in slow motion. The user then sets the audio delay time SD in the audio delay device 112 and the ancillary data delay time AD in the ancillary data delay device 113 by key operation or the like.

[0044] The user specifies the audio delay time SD and the ancillary data delay time AD by repeating the processes of recording each stream and measuring and setting the audio delay time SD and the ancillary data delay time AD while checking by trial and error.The specified audio delay time SD and ancillary data delay time AD are then used to perform delay processing by the audio delay device 112 and the ancillary data delay device 113.

[0045] When the audio delay device 112 receives an audio RTP packet from the network switch 108, it overwrites the timestamp included in the audio RTP packet with a new timestamp delayed by the audio delay time SD set by the user. Then, the audio delay device 112 delays (retains) the audio RTP packet by the audio delay time SD through buffering, and then transmits the delayed audio RTP packet to the network switch 108. The audio RTP packet is then transferred from the network switch 108 to the IP gateway 114.

[0046] When auxiliary data delay device 113 receives an auxiliary data RTP packet from network switch 108, it overwrites the timestamp included in the auxiliary data RTP packet with a timestamp delayed by auxiliary data delay time AD set by the user. Then, auxiliary data delay device 113 delays (retains) the auxiliary data RTP packet by auxiliary data delay time AD through buffering, and then transmits the delayed auxiliary data RTP packet to network switch 108. The auxiliary data RTP packet is then transferred from network switch 108 to IP gateway 114.

[0047] This allows the IP gateway 114 to match the reception timing of video RTP packets, audio RTP packets, and auxiliary data RTP packets that are generated at the same time, thereby achieving synchronization of the three types of RTP packets.

[0048] On the other hand, although it does not measure the audio delay time SD and ancillary data delay time AD like the program production system 100 shown in FIG. 14, a method has been proposed in which the delay time is measured as the time required for the encoding process from inputting an HD-SDI signal to outputting an IP packet (see Patent Document 1).

[0049] This method measures the delay time based on the time information of the input HD-SDI signal and the time information of the IP packet output after encoding the HD-SDI signal. [Prior art documents] [Patent documents]

[0050] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-7584 [Non-patent literature]

[0051] [Non-Patent Document 1] SMPTE ST 2110-20:2022, “Professional Media Over Managed IP Networks: Uncompressed Active Video” [Non-patent document 2] SMPTE ST 2110-30:2017, “Professional Media Over Managed IP Networks: PCM Digital Audio” [Non-patent document 3] SMPTE ST 2110-40:2023, “Professional Media Over Managed IP Networks: SMPTE ST 291-1 Ancillary Data” Summary of the Invention [Problem to be solved by the invention]

[0052] As described above, in the conventional program production system 100 shown in FIG. 14, a user operates the system to record a video stream using video RTP packets, an audio stream using audio RTP packets, and an auxiliary data stream using auxiliary data RTP packets, measure the audio delay time SD and the auxiliary data delay time AD, and set them in the audio delay device 112 and the auxiliary data delay device 113.

[0053] By repeating this process, the audio delay time SD and ancillary data delay time AD are identified, delay processing is performed using the identified audio delay time SD and ancillary data delay time AD, and the IP gateway 114 confirms that the reception timings of the three types of RTP packets match, thereby achieving synchronization. As described above, since there is no established method for automatically determining the audio delay time SD and ancillary data delay time AD, there has been a problem in that achieving synchronization is time-consuming.

[0054] Therefore, the present invention has been made to solve the above-mentioned problems, and its purpose is to provide a video delay measurement device and program that can measure the delay time of video relative to audio and auxiliary data in a program production system using a simple method, and can achieve synchronization of video, audio, and auxiliary data generated at the same time. [Means for solving the problem]

[0055] In order to solve the above problem, a video delay measuring device according to claim 1 measures the delay time of a video stream relative to an audio stream as an audio delay time SD, and delays the audio stream based on the audio delay time SD to synchronize the audio stream and the video stream, the video delay measuring device receiving a video RTP packet V1 storing a timestamp VTS1 of the time the video was generated and the video, transmitting a video RTP packet V2 storing preset timing information, receiving a video RTP packet V3 that has been delayed by processing by a video processing device present on the video path when the video RTP packet V2 passes through a predetermined video path and in which the timestamp VTS2 of the time the processing was performed has been overwritten in place of the timestamp VTS1, receiving a timestamp STS1 of the time the audio was generated and an audio RTP packet S1 storing the audio, transmitting an audio RTP packet S2 storing preset timing information, and a packet transceiver that receives an audio RTP packet S3 that has been delayed by processing by an audio processing device present on a audio path and in which a timestamp STS2 of the time when the processing was performed has been overwritten in place of the timestamp STS1; a video time difference calculation unit that, when inputting the video RTP packet V1 received by the packet transceiver, stores the timing information to generate the video RTP packet V2 and causes the packet transceiver to transmit it, and when inputting the video RTP packet V3 received by the packet transceiver, calculates a delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; and a video time difference calculation unit that, when inputting the audio RTP packet S1 received by the packet transceiver, stores the timing information to generate the audio RTP packet S2 and causes the packet transceiver to transmit it, and when inputting the audio RTP packet S3 received by the packet transceiver,The audio time difference calculation unit calculates the delay amount of the audio stream as an audio time difference ST based on the timestamp STS1 included in the audio RTP packet S1 and the timestamp STS2 included in the audio RTP packet S3, and a delay time calculation unit calculates the audio delay time SD based on the video time difference VT calculated by the video time difference calculation unit and the audio time difference ST calculated by the audio time difference calculation unit.

[0056] Furthermore, a video delay measuring device according to claim 2 measures the delay time of a video stream relative to an auxiliary data stream as an auxiliary data delay time AD, and delays the auxiliary data stream based on the auxiliary data delay time AD, thereby synchronizing the auxiliary data stream and the video stream. The video delay measuring device receives a video RTP packet V1 storing a timestamp VTS1 of the time the video was generated and the video, transmits a video RTP packet V2 storing preset timing information, and when the video RTP packet V2 passes through a predetermined video path, receives a video RTP packet V3 that has been delayed by processing by a video processing device present on the video path and in which the timestamp VTS2 of the time the processing was performed has been overwritten in place of the timestamp VTS1, receives a timestamp ATS1 of the time the auxiliary data was generated and an auxiliary data RTP packet A1 storing the auxiliary data, and transmits an auxiliary data RTP packet A2 storing preset timing information, and when the auxiliary data RTP packet A2 passes through the predetermined auxiliary data path, a packet transceiver that receives an auxiliary data RTP packet A3 that has been delayed by processing by an auxiliary data processing device present on the auxiliary data path and in which a timestamp ATS2 of the time when the processing was performed has been overwritten in place of the timestamp ATS1; a video time difference calculation unit that, when inputting the video RTP packet V1 received by the packet transceiver, stores the timing information to generate the video RTP packet V2 and causes the packet transceiver to transmit it, and when inputting the video RTP packet V3 received by the packet transceiver, calculates the delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; and a video time difference calculation unit that, when inputting the auxiliary data RTP packet A1 received by the packet transceiver, stores the timing information to generate the auxiliary data RTP packet A2 and causes the packet transceiver to transmit it, and when inputting the auxiliary data RTP packet A3 received by the packet transceiverThe RTP packet for auxiliary data A3 includes an auxiliary data time difference calculation unit that calculates the delay amount of the auxiliary data stream as an auxiliary data time difference AT based on the timestamp ATS1 included in the RTP packet for auxiliary data A1 and the timestamp ATS2 included in the RTP packet for auxiliary data A3, and a delay time calculation unit that calculates the auxiliary data delay time AD based on the video time difference VT calculated by the video time difference calculation unit and the auxiliary data time difference AT calculated by the auxiliary data time difference calculation unit.

[0057] A video delay measuring device according to claim 3 measures the delay time of a video stream relative to an audio stream as an audio delay time SD, measures the delay time of the video stream relative to an auxiliary data stream as an auxiliary data delay time AD, and delays the audio stream based on the audio delay time SD and the auxiliary data stream based on the auxiliary data delay time AD, thereby synchronizing the audio stream, the auxiliary data stream, and the video stream. The video delay measuring device receives a video RTP packet V1 in which a timestamp VTS1 of a video generation time and the video are stored, transmits a video RTP packet V2 in which preset timing information is stored, and when the video RTP packet V2 passes through a predetermined video path, receives a video RTP packet V3 that has been delayed by processing by a video processing device present on the video path and in which the timestamp VTS2 of the time when the processing was performed has overwritten the location of the timestamp VTS1, and measures a timestamp STS1 of the audio generation time and the timestamp STS2 of the audio. a packet transceiver unit that receives an audio RTP packet S1 having stored therein predetermined timing information, transmits an audio RTP packet S2 having preset timing information stored therein, and receives an audio RTP packet S3 in which the audio RTP packet S2 is delayed by processing by an audio processing device present on the audio path when the audio RTP packet S2 passes through a predetermined audio path, and in which the timestamp STS2 of the processing is overwritten in place of the timestamp STS1; receives an auxiliary data RTP packet A1 in which a timestamp ATS1 of the generation time of auxiliary data and the auxiliary data is stored, transmits an auxiliary data RTP packet A2 having preset timing information stored therein, and in which the auxiliary data RTP packet A2 is delayed by processing by an auxiliary data processing device present on the auxiliary data path when the auxiliary data RTP packet A2 passes through a predetermined auxiliary data path, and in which the timestamp ATS2 of the processing is overwritten in place of the timestamp ATS1; anda video time difference calculation unit that stores the timing information to generate the video RTP packet V2 and causes it to be transmitted by the packet transceiver unit, and that, upon inputting the video RTP packet V3 received by the packet transceiver unit, calculates a delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; and an audio time difference calculation unit that, upon inputting the audio RTP packet S1 received by the packet transceiver unit, generates the audio RTP packet S2 by storing the timing information to be transmitted by the packet transceiver unit, and causes it to be transmitted by the packet transceiver unit, and that, upon inputting the audio RTP packet S3 received by the packet transceiver unit, calculates a delay amount of the audio stream as an audio time difference ST based on the timestamp STS1 included in the audio RTP packet S1 and the timestamp STS2 included in the audio RTP packet S3. an auxiliary data time difference calculation unit that, when the auxiliary data RTP packet A1 received by the packet transceiver unit is input, stores the timing information to generate the auxiliary data RTP packet A2 and causes the packet transceiver unit to transmit it, and, when the auxiliary data RTP packet A3 received by the packet transceiver unit is input, calculates the delay amount of the auxiliary data stream as an auxiliary data time difference AT based on the timestamp ATS1 included in the auxiliary data RTP packet A1 and the timestamp ATS2 included in the auxiliary data RTP packet A3; and a delay time calculation unit that calculates the audio delay time SD based on the video time difference VT calculated by the video time difference calculation unit and the audio time difference ST calculated by the audio time difference calculation unit, and calculates the auxiliary data delay time AD based on the video time difference VT and the auxiliary data time difference AT calculated by the auxiliary data time difference calculation unit.

[0058] Furthermore, the video delay measuring device of claim 4 is characterized in that, in the video delay measuring device of claim 1, the timing information stored in the video RTP packet V2 is retained when processing is performed by the video processing device when the video RTP packet V2 passes through the video path, and the timing information stored in the audio RTP packet S2 is retained when processing is performed by the audio processing device when the audio RTP packet S2 passes through the audio path, and the packet transceiver receives the video RTP packet V3 by identifying it based on the timing information included in the video RTP packet V3, and receives the audio RTP packet S3 by identifying it based on the timing information included in the audio RTP packet S3.

[0059] Furthermore, the video delay measuring device of claim 5 is characterized in that, in the video delay measuring device of claim 2, the timing information stored in the video RTP packet V2 is retained when processing is performed by the video processing device when the video RTP packet V2 passes through the video path, the timing information stored in the auxiliary data RTP packet A2 is retained when processing is performed by the auxiliary data processing device when the auxiliary data RTP packet A2 passes through the auxiliary data path, and the packet transceiver receives the video RTP packet V3 by identifying it based on the timing information included in the video RTP packet V3, and receives the auxiliary data RTP packet A3 by identifying it based on the timing information included in the auxiliary data RTP packet A3.

[0060] A video delay measuring device according to claim 6 is the video delay measuring device according to any one of claims 1 to 5, characterized in that the timing information is set to a null value.

[0061] Furthermore, a program according to a seventh aspect of the present invention is characterized in that it causes a computer to function as the video delay measuring device according to any one of the first to third aspects of the present invention. [Effects of the Invention]

[0062] As described above, according to the present invention, in a program production system, it is possible to measure the delay time of video relative to audio and auxiliary data using a simple method, and to achieve synchronization of video, audio, and auxiliary data generated at the same time. [Brief explanation of the drawings]

[0063] [Figure 1] 1 is a schematic diagram showing an example of the overall configuration of a program production system including a video delay measuring device according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of a video delay time difference measurement path for calculating a video time difference VT. [Figure 3] 10 is a diagram illustrating an example of a path for measuring an audio delay time difference for calculating an audio time difference ST. FIG. [Figure 4] 10 is a diagram illustrating an example of an auxiliary data delay time difference measurement path for calculating an auxiliary data time difference AT. FIG. [Figure 5] 1 is a block diagram showing an example of the configuration of a video delay measuring device according to an embodiment of the present invention; [Figure 6] 10 is a flowchart illustrating an example of processing performed by the video delay measuring device according to the embodiment of the present invention. [Figure 7] 10A and 10B are diagrams illustrating in detail an example of processing by the video delay measuring device. [Figure 8] FIG. 2 is a block diagram illustrating an example of the configuration of a video time difference calculation unit. [Figure 9] 10 is a flowchart illustrating an example of processing by a video time difference calculation unit. [Figure 10] 10 is a flowchart illustrating an example of processing by a delay time calculation unit. [Figure 11] FIG. 10 is a diagram showing the structure of a video RTP packet. [Figure 12] FIG. 10 is a diagram illustrating the structure of an audio RTP packet. [Figure 13] FIG. 10 is a diagram showing the structure of an RTP packet for auxiliary data. [Figure 14] 1 is a schematic diagram showing an example of the overall configuration of a conventional program production system; [Figure 15] FIG. 1 is a diagram illustrating a delay of video relative to audio. [Figure 16] FIG. 1 is a diagram illustrating the delay of video relative to audio and auxiliary data. DETAILED DESCRIPTION OF THE INVENTION

[0064] The following describes in detail the embodiments of the present invention with reference to the drawings. The present invention is characterized by transmitting video RTP packets, audio RTP packets, and auxiliary data RTP packets, each having predetermined timing information, over a predetermined path that causes delay, thereby calculating the delay amounts for video, audio, and auxiliary data (video time difference VT, audio time difference ST, auxiliary data time difference AT), and calculating the audio delay time SD and auxiliary data delay time AD.

[0065] The audio delay time SD is the delay time of the video relative to the audio, and is used to synchronize the audio and video by adjusting the timing of the audio to that of the video. The auxiliary data delay time AD is the delay time of the video relative to the auxiliary data, and is used to synchronize the auxiliary data and video by adjusting the timing of the auxiliary data to that of the video. As a result, by adjusting the timing of the audio and auxiliary data to that of the video, synchronization of the video, audio, and auxiliary data can be achieved.

[0066] This allows a program production system to measure the delay time of video relative to audio and auxiliary data using a simple method, thereby achieving synchronization of video, audio, and auxiliary data generated at the same time.

[0067] [Program Production System] First, an example of the overall configuration for eliminating delay time differences between video, audio, and auxiliary data in a program production system that produces broadcast programs will be described. Figure 1 is a schematic diagram showing an example of the overall configuration of a program production system that includes a video delay measurement device according to an embodiment of the present invention.

[0068] This program production system 1 is composed of a camera 101, a camera control unit 102, a recording and playback device 103, an IP gateway 104, a time code generation device 105, a microphone 106, an audio IP gateway 107, a network switch 108, a video switcher (video processing device) 109, an audio network switch 110, an audio mixer (audio processing device) 111, an audio delay device 112, an auxiliary data delay device 113, an IP gateway 114, an SDI synchronization device 115, recording and playback devices 116 and 117, and a video delay measurement device 2.

[0069] In this program production system 1, the video delay measuring device 2 measures the delay time difference between the video, audio, and auxiliary data (audio delay time SD and auxiliary data delay time AD), and the audio delay device 112 and the auxiliary data delay device 113 delay the audio and auxiliary data based on the delay time difference between the data, thereby achieving synchronization of the video, audio, and auxiliary data, and generating and outputting an SDI signal containing the synchronized video, audio, and auxiliary data.

[0070] Comparing this program production system 1 with the conventional program production system 100 shown in Figure 14, both program production systems 1, 100 have in common the fact that they are equipped with cameras 101, camera control units 102, ..., IP gateway 114, SDI synchronizer 115, and recording / playback devices 116, 117. In contrast, program production system 1 differs from program production system 100, which does not have a video delay measuring device 2, in that it is equipped with a video delay measuring device 2.

[0071] As in Figure 14, Figure 1 shows a video stream made up of video RTP packets (see black circles), an audio stream made up of audio RTP packets (see white triangles), and an auxiliary data stream made up of auxiliary data RTP packets (see white squares).

[0072] When the video delay measurement device 2 measures the delay time difference between data, the video RTP packets, audio RTP packets, and auxiliary data RTP packets are transferred so as to pass through a preset path for measuring the delay time difference, in accordance with instructions from a server (transfer control server) that performs transfer control (not shown in Figure 1).

[0073] The camera control unit 102 receives the video captured by the camera 101, stores a timestamp indicating when the video was (input) and generated in the header, and stores the video (data) in the payload, thereby generating a video RTP packet. The camera control unit 102 then transmits the video RTP packet to the network switch 108. This video RTP packet is a packet conforming to the SMPTE ST 2110-20 standard shown in Non-Patent Document 1.

[0074] The camera control unit 102 also receives audio picked up by a microphone attached to the camera 101, stores a timestamp indicating when the audio was (input) generated in the header, and stores the audio (data) in the payload to generate an audio RTP packet. The camera control unit 102 then transmits the audio RTP packet to the network switch 108. This audio RTP packet is a packet conforming to the SMPTE ST 2110-30 standard shown in Non-Patent Document 2.

[0075] Video, audio, and auxiliary data are recorded in the recording and playback device 103. After playing back the video, audio, and auxiliary data, the recording and playback device 103 generates a video / audio / auxiliary SDI signal, which is an SDI signal including the video, audio, and auxiliary data, and outputs the signal to the IP gateway 104.

[0076] The IP gateway 104 receives the video, audio, and auxiliary SDI signals from the recording / playback device 103 and converts them into video RTP packets, audio RTP packets, and auxiliary data RTP packets, respectively. That is, the IP gateway 104 generates video RTP packets, audio RTP packets, and auxiliary data RTP packets by storing the same time stamp or the like in the header and storing the video, audio, and auxiliary data extracted from the video, audio, and auxiliary SDI signals in the payload, respectively. The IP gateway 104 then transmits the video RTP packets, audio RTP packets, and auxiliary data RTP packets to the network switch 108.

[0077] The video RTP packets, audio RTP packets, and auxiliary data RTP packets are packets conforming to the SMPTE ST 2110-20, SMPTE ST 2110-30, and SMPTE ST 2110-40 standards shown in Non-Patent Documents 1, 2, and 3, respectively.

[0078] The time code generator 105 generates ancillary data, which is a time code, and stores a timestamp indicating when the ancillary data was generated in the header and the ancillary data in the payload, thereby generating an ancillary data RTP packet. The time code generator 105 then transmits the ancillary data RTP packet to the network switch 108. This ancillary data RTP packet is a packet conforming to the SMPTE ST 2110-40 standard shown in Non-Patent Document 3.

[0079] The audio IP gateway 107 receives audio picked up by the microphone 106, stores a timestamp indicating when the audio was (input) generated in the header, and stores the audio in the payload to generate an audio RTP packet. The audio IP gateway 107 then transmits the audio RTP packet to the audio network switch 110. As described above, this audio RTP packet is a packet conforming to the SMPTE ST 2110-30 standard shown in Non-Patent Document 2.

[0080] The network switch 108 transfers video RTP packets, audio RTP packets, and auxiliary data RTP packets in accordance with instructions from a transfer control server (not shown in FIG. 1).

[0081] Network switch 108 receives video RTP packets and audio RTP packets from camera control unit 102. Then, in accordance with the above-mentioned instructions, network switch 108 transfers the video RTP packets to video delay measurement device 2, video switcher 109, or IP gateway 114, and transfers the audio RTP packets to video delay measurement device 2, audio network switch 110, audio delay device 112, or IP gateway 114.

[0082] Network switch 108 also receives video RTP packets, audio RTP packets, and auxiliary data RTP packets from IP gateway 104 and transfers them according to the instructions described above. Network switch 108 also receives auxiliary data RTP packets from time code generator 105 and transfers them according to the instructions described above. Network switch 108 also receives audio RTP packets from audio IP gateway 107 and transfers them according to the instructions described above.

[0083] Furthermore, network switch 108 receives video RTP packets from video switcher 109 and transfers them in the same way, receives video RTP packets, audio RTP packets, and auxiliary data RTP packets from video delay measurement device 2 and transfers them in the same way, receives audio RTP packets from audio delay device 112 and transfers them in the same way, receives auxiliary data RTP packets from auxiliary data delay device 113 and transfers them in the same way, and receives audio RTP packets from audio network switch 110 and transfers them in the same way.

[0084] Video switcher 109 receives video RTP packets from network switch 108, extracts video from the payload of the video RTP packets, and performs predetermined video processing on the video. Video switcher 109 then overwrites the header (where the timestamp is already stored) with a timestamp indicating the time at which the processed video RTP packets will be sent to network switch 108, and overwrites the payload with the processed video, thereby generating a new video RTP packet and sending it to network switch 108. In this case, the timestamp stored in the header by camera control unit 102 or IP gateway 104 is not carried over but is rewritten with the new timestamp.

[0085] Like the network switch 108, the audio network switch 110 transfers audio RTP packets in accordance with instructions from a transfer control server not shown in FIG.

[0086] The audio network switch 110 receives the audio RTP packets from the audio IP gateway 107, and transfers the audio RTP packets to the network switch 108 or the audio mixer 111 according to the above-mentioned instructions.

[0087] The audio network switch 110 also receives audio RTP packets from the network switch 108 and transfers them in the same way, and receives audio RTP packets from the audio mixer 111 and transfers them in the same way.

[0088] The audio mixer 111 receives audio RTP packets from the audio network switch 110, extracts audio from the payload of the audio RTP packets, and performs predetermined audio processing on the audio. The audio mixer 111 then overwrites the header (where the timestamp is already stored) with a timestamp indicating when the processed audio RTP packets will be sent to the audio network switch 110, and overwrites the processed audio in the payload to generate a new audio RTP packet, which it then sends to the audio network switch 110. In this case, the timestamp stored in the header by the camera control unit 102, IP gateway 104, or audio IP gateway 107 is not carried over but is rewritten with the new timestamp.

[0089] The audio delay device 112 receives the audio delay time SD from the video delay measurement device 2 and stores it. The audio delay device 112 receives the audio RTP packets from the network switch 108, extracts the timestamp from the header of the audio RTP packet, and calculates a new timestamp by adding the audio delay time SD to the timestamp.

[0090] The audio delay device 112 generates a new audio RTP packet by overwriting the header with a new timestamp, delays (retains) the new audio RTP packet by the audio delay time SD through buffering, and then transmits the delayed audio RTP packet to the network switch 108. The audio RTP packet is then transferred from the network switch 108 to the IP gateway 114 in accordance with the above-mentioned instructions.

[0091] As a result, the timestamp included in the audio RTP packet is rewritten so that the timing of the audio corresponding to the delayed video matches the video, and the timestamp included in the video RTP packet and the timestamp included in the audio RTP packet become the same (within a predetermined threshold width), and the transmission timing of the audio RTP packet is appropriately delayed. In other words, it is possible to eliminate the delay time difference between the video RTP packet and the audio RTP packet generated at the same time, and it is possible for IP gateway 114 to identify the audio RTP packet corresponding to the video RTP packet.

[0092] Ancillary data delay device 113 receives and holds the ancillary data delay time AD from video delay measurement device 2. Ancillary data delay device 113 receives ancillary data RTP packets from network switch 108, extracts a timestamp from the header of the ancillary data RTP packet, and adds the ancillary data delay time AD to the timestamp to obtain a new timestamp.

[0093] Auxiliary data delay device 113 generates a new auxiliary data RTP packet by overwriting the header with a new timestamp, delays (retains) the new auxiliary data RTP packet by auxiliary data delay time AD through buffering, and then transmits the delayed auxiliary data RTP packet to network switch 108. Then, the auxiliary data RTP packet is transferred from network switch 108 to IP gateway 114 in accordance with the above-mentioned instruction.

[0094] As a result, the timestamp included in the auxiliary data RTP packet is rewritten so that the timing of the auxiliary data corresponding to the delayed video matches the video, and the timestamp included in the video RTP packet and the timestamp included in the auxiliary data RTP packet become the same (within a predetermined threshold width), and the transmission timing of the auxiliary data RTP packet is appropriately delayed. In other words, it is possible to eliminate the delay time difference between the video RTP packet and the auxiliary data RTP packet generated at the same time, and it is possible for IP gateway 114 to identify the auxiliary data RTP packet that corresponds to the video RTP packet.

[0095] The IP gateway 114 receives video RTP packets, audio RTP packets, and auxiliary data RTP packets from the network switch 108. Then, by referencing the timestamps stored in the headers of the video RTP packets, audio RTP packets, and auxiliary data RTP packets, the IP gateway 114 identifies video RTP packets, audio RTP packets, and auxiliary data RTP packets that include timestamps within a predetermined threshold width (timestamps of the same time).

[0096] This identifies video RTP packets, audio RTP packets, and auxiliary data RTP packets that were generated at the same time (within a specified threshold range), i.e., video RTP packets, audio RTP packets, and auxiliary data RTP packets that contain synchronized video, audio, and auxiliary data.

[0097] The IP gateway 114 converts the identified video RTP packets, audio RTP packets, and auxiliary data RTP packets into video, audio, and auxiliary SDI signals, and outputs them to an SDI synchronizer 115 and recording / playback devices 116 and 117 .

[0098] The SDI synchronizer 115 receives the video, audio, and auxiliary SDI signals from the IP gateway 114, performs synchronization processing on the video, audio, and auxiliary SDI signals, and outputs the synchronized video, audio, and auxiliary SDI signals.

[0099] Recording / playback devices 116 and 117 receive the video, audio, and auxiliary SDI signals from IP gateway 114 and record the video, audio, and auxiliary data contained therein.

[0100] For example, the recording / playback device 116 records the video, audio, and auxiliary data contained in the video, audio, and auxiliary SDI signals converted using video RTP packets that have passed through a path from the camera control unit 102 via the network switch 108, video switcher 109, etc. to the IP gateway 114 (for example, the path of video v' shown in Figure 16).

[0101] Furthermore, for example, the recording / playback device 117 records the video, audio, and auxiliary data contained in the video / audio / auxiliary SDI signals converted using video RTP packets that have passed through a path (for example, the path of video v" shown in FIG. 16) from the camera control unit 102 to the IP gateway 114 via the network switch 108, etc., but that does not pass through the video switcher 109.

[0102] As shown in Figures 2, 3, and 4 described below, the video delay measurement device 2 receives a video RTP packet V1, an audio RTP packet S1, and an auxiliary data RTP packet A1 from the network switch 108 when a path for a route for delay time difference measurement that has been preset in accordance with instructions from a transfer control server (not shown in Figure 1) is set.

[0103] The video delay measurement device 2 generates video RTP packets V2, audio RTP packets S2, and auxiliary data RTP packets A2 by overwriting preset timing information into the payloads of the video RTP packets V1, audio RTP packets S1, and auxiliary data RTP packets A1. The video delay measurement device 2 then transmits the video RTP packets V2, audio RTP packets S2, and auxiliary data RTP packets A2 to the network switch 108.

[0104] When the video RTP packet V2, audio RTP packet S2, and auxiliary data RTP packet A2 return after passing through the path for measuring the delay time difference, the video delay measuring device 2 receives these packets as video RTP packet V3, audio RTP packet S3, and auxiliary data RTP packet A3.

[0105] The video delay measuring device 2 calculates the video time difference VT, the audio time difference ST, and the auxiliary data time difference AT based on the timestamps contained in the video RTP packets V1, V3, the audio RTP packets S1, S3, and the auxiliary data RTP packets A1, A3.

[0106] The video time difference VT is the delay amount of the video in the path for measuring the video delay time difference shown in Fig. 2, which will be described later. The audio time difference ST is the delay amount of the audio in the path for measuring the audio delay time difference shown in Fig. 3, which will be described later. The ancillary data time difference AT is the delay amount of the ancillary data in the path for measuring the ancillary data delay time difference shown in Fig. 4, which will be described later.

[0107] The video delay measuring device 2 calculates an audio delay time SD and an ancillary data delay time AD based on the video time difference VT, the audio time difference ST, and the ancillary data time difference AT. Then, the video delay measuring device 2 outputs the audio delay time SD to the audio delay device 112 and outputs the ancillary data delay time AD to the ancillary data delay device 113. Details of the video delay measuring device 2 will be described later.

[0108] In this case, the audio delay time SD may be manually set in the audio delay device 112 by a user operation, or the video delay measuring device 2 may transmit an IP packet including the audio delay time SD to the audio delay device 112. Furthermore, the ancillary data delay time AD may be manually set in the ancillary data delay device 113 by a user operation, or the video delay measuring device 2 may transmit an IP packet including the ancillary data delay time AD to the ancillary data delay device 113.

[0109] 2 is a diagram illustrating an example of a video delay time difference measurement path for calculating the video time difference VT. This video delay time difference measurement path (video path) is a route from camera control unit 102, through which video RTP packets are transferred, to network switch 108, video delay measurement device 2, network switch 108, video switcher 109, network switch 108, and video delay measurement device 2.

[0110] In this example, a timestamp is assigned by the camera control unit 102 and a new timestamp is assigned by the video switcher 109, so the video time difference VT reflects the delay caused by the video processing by the video switcher 109.

[0111] In the video delay time difference measurement path, the RTP packet transmitted from network switch 108 to video delay measurement device 2 is video RTP packet V1, and the RTP packet transmitted from video delay measurement device 2 to network switch 108 is video RTP packet V2. Furthermore, the RTP packet transmitted from network switch 108 to video delay measurement device 2 via video switcher 109 is video RTP packet V3.

[0112] The video delay time difference measurement path may be set on the original video stream path (for example, the path from the camera control unit 102 to the network switch 108, video switcher 109, network switch 108, and IP gateway 114) by the network switch 108 copying the video RTP packet transferred from the camera control unit 102 and transferring it to the video delay measurement device 2 as video RTP packet V1.

[0113] 3 is a diagram illustrating an example of an audio delay time difference measurement path for calculating the audio time difference ST. This audio delay time difference measurement path (audio path) is a route from audio IP gateway 107, through which audio RTP packets are transferred, to audio network switch 110, network switch 108, video delay measurement device 2, network switch 108, audio network switch 110, audio mixer 111, audio network switch 110, network switch 108, and video delay measurement device 2.

[0114] In this example, a timestamp is assigned by the audio IP gateway 107 and a new timestamp is assigned by the audio mixer 111, so the audio time difference ST reflects the delay caused by the audio processing of the audio mixer 111.

[0115] In the audio delay time difference measurement path, the RTP packet transmitted from network switch 108 to video delay measurement device 2 is audio RTP packet S1, and the RTP packet transmitted from video delay measurement device 2 to network switch 108 is audio RTP packet S2. Furthermore, the RTP packet transmitted from network switch 108 to video delay measurement device 2 via audio mixer 111 and audio network switch 110 is audio RTP packet S3.

[0116] The audio delay time difference measurement path may be set on the original audio stream path (for example, the path from the audio IP gateway 107 to the audio network switch 110, audio mixer 111, audio network switch 110, network switch 108, audio delay device 112, network switch 108, and IP gateway 114), where the audio network switch 110 copies the audio RTP packet transferred from the audio IP gateway 107 and transfers it to the network switch 108, which then transfers it to the video delay measurement device 2 as an audio RTP packet S1.

[0117] 4 is a diagram illustrating an example of an auxiliary data delay time difference measurement path for calculating the auxiliary data time difference AT. This auxiliary data delay time difference measurement path (auxiliary data path) is a route from the time code generator 105, through which the auxiliary data RTP packets are transferred, to the network switch 108, the video delay measurement device 2, the network switch 108, the auxiliary data processing device 118 (not shown in FIG. 1), the network switch 108, and the video delay measurement device 2.

[0118] In this example, a timestamp is assigned by the time code generator 105 and a new timestamp is assigned by the auxiliary data processing device 118, so the auxiliary data time difference AT reflects the delay caused by the auxiliary data processing by the auxiliary data processing device 118.

[0119] In the path for measuring the auxiliary data delay time difference, the RTP packet transmitted from the network switch 108 to the video delay measurement device 2 is the auxiliary data RTP packet A1, and the RTP packet transmitted from the video delay measurement device 2 to the network switch 108 is the auxiliary data RTP packet A2. Furthermore, the RTP packet transmitted from the network switch 108 to the video delay measurement device 2 via the auxiliary data processing device 118 is the auxiliary data RTP packet A3.

[0120] The path for measuring the auxiliary data delay time difference may be set on the path of the original auxiliary data stream (for example, the path from the time code generator 105 to the network switch 108, the auxiliary data processing device 118, the network switch 108, the auxiliary data delay device 113, the network switch 108, and the IP gateway 114), by the network switch 108 copying the auxiliary data RTP packet transferred from the time code generator 105 and transferring this as the auxiliary data RTP packet A1 to the video delay measuring device 2.

[0121] [Video Delay Measuring Device 2] Next, a detailed description will be given of the video delay measuring device 2 shown in Fig. 1. Fig. 5 is a block diagram showing an example of the configuration of the video delay measuring device 2 according to an embodiment of the present invention, and Fig. 6 is a flowchart showing an example of the processing performed by the device.

[0122] The video delay measuring device 2 includes a packet transmitting / receiving unit 10, a video time difference calculating unit 11-1, an audio time difference calculating unit 11-2, an auxiliary data time difference calculating unit 11-3, a memory 12, and a delay time calculating unit 13.

[0123] The packet transmitter / receiver 10 transmits and receives video RTP packets V1, V2, and V3 to and from the network switch 108 (step S601). Specifically, when the packet transmitter / receiver 10 receives video RTP packet V1 from the network switch 108, it outputs it to the video time difference calculation unit 11-1, and when it receives video RTP packet V2 from the video time difference calculation unit 11-1, it transmits it to the network switch 108. Furthermore, the packet transmitter / receiver 10 identifies (specifies) video RTP packet V3 from the video RTP packets received from the network switch 108 based on timing information (described later) stored in the payload, and outputs it to the video time difference calculation unit 11-1.

[0124] The header of video RTP packet V1 stores a timestamp VTS1 assigned by, for example, camera control unit 102. The header of video RTP packet V2 also stores a timestamp VTS1, and its payload stores timing information assigned by video time difference calculation unit 11-1. The header of video RTP packet V3 also stores a timestamp VTS2 assigned by video switcher 109, and its payload stores timing information.

[0125] The video time difference calculation unit 11-1 inputs a video RTP packet V1 from the packet transmission / reception unit 10, generates a video RTP packet V2 including preset timing information, and outputs this to the packet transmission / reception unit 10. The video time difference calculation unit 11-1 also inputs a video RTP packet V3 from the packet transmission / reception unit 10.

[0126] The video time difference calculation unit 11-1 calculates the video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3 (step S602), and stores it in the memory 12.

[0127] This provides the amount of delay of the video in the video delay time difference measurement path shown in Fig. 2, i.e., the video time difference VT that reflects the delay caused by the video processing of the video switcher 109. Details of the video time difference calculation unit 11-1 will be described later.

[0128] Furthermore, the packet transmitting / receiving unit 10 transmits and receives audio RTP packets S1, S2, and S3 to and from the network switch 108 (step S603). Specifically, when the packet transmitting / receiving unit 10 receives audio RTP packet S1 from the network switch 108, it outputs it to the audio time difference calculation unit 11-2, and when it receives audio RTP packet S2 from the audio time difference calculation unit 11-2, it transmits it to the network switch 108. Furthermore, the packet transmitting / receiving unit 10 identifies audio RTP packet S3 from the audio RTP packets received from the network switch 108 based on timing information, described later, stored in the payload, and outputs it to the audio time difference calculation unit 11-2.

[0129] The header of audio RTP packet S1 stores a timestamp STS1 assigned by audio IP gateway 107, for example. The header of audio RTP packet S2 also stores a timestamp STS1, and its payload stores timing information assigned by audio time difference calculation unit 11-2. The header of audio RTP packet S3 also stores a timestamp STS2 assigned by audio mixer 111, and its payload stores timing information.

[0130] The audio time difference calculation unit 11-2 inputs an audio RTP packet S1 from the packet transmission / reception unit 10, generates an audio RTP packet S2 including preset timing information, and outputs this to the packet transmission / reception unit 10. The audio time difference calculation unit 11-2 also inputs an audio RTP packet S3 from the packet transmission / reception unit 10.

[0131] The audio time difference calculation unit 11-2 calculates an audio time difference ST based on the time stamp STS1 included in the audio RTP packet S1 and the time stamp STS2 included in the audio RTP packet S3 (step S604), and stores it in the memory 12.

[0132] This provides the audio delay amount in the audio delay time difference measurement path shown in Fig. 3, i.e., the audio time difference ST that reflects the delay caused by audio processing by the audio mixer 111. Details of the audio time difference calculation unit 11-2 will be described later.

[0133] Furthermore, packet transmitter / receiver 10 transmits and receives auxiliary data RTP packets A1, A2, and A3 to and from network switch 108 (step S605). Specifically, when packet transmitter / receiver 10 receives auxiliary data RTP packet A1 from network switch 108, it outputs it to auxiliary data time difference calculation unit 11-3, and when packet transmitter / receiver 10 receives auxiliary data RTP packet A2 from auxiliary data time difference calculation unit 11-3, it transmits it to network switch 108. Furthermore, packet transmitter / receiver 10 identifies auxiliary data RTP packet A3 from the auxiliary data RTP packets received from network switch 108 based on timing information, described later, stored in the payload, and outputs it to auxiliary data time difference calculation unit 11-3.

[0134] The header of ancillary data RTP packet A1 stores a timestamp ATS1 assigned by, for example, time code generator 105. The header of ancillary data RTP packet A2 also stores a timestamp ATS1, and its payload stores timing information assigned by ancillary data time difference calculation unit 11-3. The header of ancillary data RTP packet A3 stores a timestamp ATS2 assigned by ancillary data processor 118 shown in FIG. 4, and its payload stores timing information.

[0135] Auxiliary data time difference calculation unit 11-3 inputs auxiliary data RTP packet A1 from packet transmitter / receiver 10, generates auxiliary data RTP packet A2 including preset timing information, and outputs this to packet transmitter / receiver 10. Auxiliary data time difference calculation unit 11-3 also inputs auxiliary data RTP packet A3 from packet transmitter / receiver 10.

[0136] The auxiliary data time difference calculation unit 11-3 calculates the auxiliary data time difference AT based on the timestamp ATS1 included in the auxiliary data RTP packet A1 and the timestamp ATS2 included in the auxiliary data RTP packet A3 (step S606) and stores it in memory 12.

[0137] This provides the amount of delay of the auxiliary data in the path for measuring the auxiliary data delay time difference shown in Fig. 4, i.e., the auxiliary data time difference AT reflecting the delay caused by the auxiliary data processing by the auxiliary data processing device 118. Details of the auxiliary data time difference calculation unit 11-3 will be described later.

[0138] The delay time calculation unit 13 reads the video time difference VT, the audio time difference ST, and the ancillary data time difference AT from the memory 12. Then, the delay time calculation unit 13 calculates the audio delay time SD based on the video time difference VT and the audio time difference ST, and calculates the ancillary data delay time AD based on the video time difference VT and the ancillary data time difference AT (step S607).

[0139] The delay time calculation unit 13 outputs the audio delay time SD to the audio delay device 112, and outputs the ancillary data delay time AD to the ancillary data delay device 113 (step S608). The processing by the delay time calculation unit 13 will be described in detail later.

[0140] Fig. 7 is a diagram illustrating in detail the example of processing shown in Fig. 6. When video RTP packet V1 is transmitted from network switch 108 to video delay measuring device 2 (step S701), packet transmitter / receiver 10 receives video RTP packet V1 (step S702). Video time difference calculator 11-1 extracts timestamp VTS1 from the header of video RTP packet V1 (step S703) and generates video RTP packet V2 including preset timing information (step S704).

[0141] 2, video RTP packet V2 is transmitted from video delay measurement device 2 to network switch 108 (steps S705 and S706), and is then transferred from network switch 108 to video switcher 109. Then, video RTP packet V3 is transferred from video switcher 109 to video delay measurement device 2 via network switch 108.

[0142] By passing through video switcher 109, which causes delays in the video stream due to the video RTP packets, video RTP packets V2 can be delayed in the video delay time difference measurement path shown in Fig. 2. The delayed video RTP packets V2 are then transferred to video delay measurement device 2 as video RTP packets V3.

[0143] When the video RTP packet V3 is transmitted from the network switch 108 to the video delay measuring device 2 (step S707), the packet transceiver 10 receives the video RTP packet V3 (step S708). The video time difference calculator 11-1 extracts the timestamp VTS2 from the header of the video RTP packet V3 (step S709) and calculates the video time difference VT based on the timestamps VTS1 and VTS2 (step S710).

[0144] In the same process as steps S701 to S707 described above, the audio time difference calculation unit 11-2 calculates the audio time difference ST, and the auxiliary data time difference calculation unit 11-3 calculates the auxiliary data time difference AT (step S711).

[0145] The delay time calculation unit 13 calculates an audio delay time SD based on the video time difference VT and the audio time difference ST (step S712), and calculates an ancillary data delay time AD based on the video time difference VT and the ancillary data time difference AT (step S713).

[0146] The audio delay time SD is output from the video delay measuring device 2 to the audio delay device 112 (step S714), and the ancillary data delay time AD is output from the video delay measuring device 2 to the ancillary data delay device 113 (step S715).

[0147] As a result, the audio delay device 112 performs delay processing based on the audio delay time SD, and the ancillary data delay device 113 performs delay processing based on the ancillary data delay time AD.

[0148] (Video time difference calculation unit 11-1) Next, a detailed description will be given of the video time difference calculation unit 11-1 shown in Fig. 5. Fig. 8 is a block diagram showing an example of the configuration of the video time difference calculation unit 11-1, and Fig. 9 is a flowchart showing an example of the processing performed by the unit.

[0149] The video time difference calculation unit 11-1 includes a time stamp extraction unit 20, a timing information addition unit 21, a time stamp extraction unit 22, and a delay time calculation unit .

[0150] The timestamp extraction unit 20 and timing information addition unit 21 input the video RTP packet V1 from the packet transmission / reception unit 10 (step S901). Then, the timestamp extraction unit 20 extracts the timestamp VTS1 from the header of the video RTP packet V1 (step S902) and outputs it to the delay time calculation unit 23. The header of the video RTP packet V1 stores the timestamp VTS1 added by, for example, the camera control unit 102.

[0151] When the timing information adding unit 21 receives the video RTP packet V1 from the packet transmitting / receiving unit 10, it adds preset timing information to the video RTP packet V1 to generate a video RTP packet V2 (step S903). Specifically, the timing information adding unit 21 overwrites the payload of the video RTP packet V1 with the timing information to generate a video RTP packet V2 including the timing information. Then, the timing information adding unit 21 outputs the video RTP packet V2 to the packet transmitting / receiving unit 10 (step S904).

[0152] 11 shows the structure of a video RTP packet, and indicates that timing information is stored in the payload. A video RTP packet consists of a header and a payload, each of which is a 32-bit unit.

[0153] 11, the header of a video RTP packet is composed of data such as the version of the video RTP packet, ..., time stamp, etc. The time stamp field stores time stamps assigned by the camera control unit 102, IP gateway 104, and video switcher 109.

[0154] The payload area (video valid area) of the video RTP packet stores video data by the camera control unit 102, IP gateway 104, and video switcher 109. Also, timing information is stored in this payload area by the timing information adding unit 21 of the video time difference calculation unit 11-1.

[0155] For example, consecutive null values ​​"000..." are used as timing information. The null values ​​"000..." stored in the payload of a video RTP packet are displayed in black as actual video, and are not subject to video processing by video switcher 109. For this reason, video switcher 109 does not process video using timing information of the null values ​​"000..." stored in the payload of a video RTP packet, and does not overwrite new video. In other words, the timing information stored in the payload is retained in video switcher 109.

[0156] 8 and 9, the timestamp extraction unit 22 receives the video RTP packet V3 from the packet transmission / reception unit 10 (step S905). Then, the timestamp extraction unit 22 extracts the timestamp VTS2 from the header of the video RTP packet V3 (step S906) and outputs it to the delay time calculation unit 23. The header of the video RTP packet V3 stores the timestamp VTS2 that has been overwritten by the video switcher 109, for example.

[0157] The delay time calculation unit 23 receives the timestamp VTS1 from the timestamp extraction unit 20 and receives the timestamp VTS2 from the timestamp extraction unit 22. The delay time calculation unit 23 then subtracts the timestamp VTS1 from the timestamp VTS2 to obtain the video time difference VT (step S907) and stores this in the memory 12 (step S908).

[0158] 5 also calculates the audio time difference ST by transmitting and receiving audio RTP packets S1 and S3 and audio RTP packet S2 containing predetermined timing information using the same configuration and processing examples as those in FIGS. 8 and 9. The audio time difference ST is then stored in memory 12. The audio time difference ST is obtained by subtracting the timestamp STS1 stored in the header of audio RTP packet S1 from the timestamp STS2 stored in the header of audio RTP packet S3.

[0159] 12 shows the structure of an audio RTP packet, and indicates that timing information is stored in the payload. An audio RTP packet consists of a header and a payload, each of which is a 32-bit unit.

[0160] 12, the header of an audio RTP packet is composed of data such as the version of the audio RTP packet, ..., time stamp, etc. The time stamp field stores time stamps added by the camera control unit 102, IP gateway 104, audio IP gateway 107, and audio mixer 111.

[0161] The payload area (audio valid area) of the audio RTP packet stores audio data by the camera control unit 102, IP gateway 104, audio IP gateway 107, and audio mixer 111. Also, timing information is stored in this payload area by the timing information adding unit of audio time difference calculation unit 11-2.

[0162] For example, a series of null values ​​"000..." is used as timing information. The null values ​​"000..." stored in the payload of an audio RTP packet are actually silent audio and are not subject to audio processing by the audio mixer 111. For this reason, the audio mixer 111 does not perform audio processing using timing information of the null values ​​"000..." stored in the payload of an audio RTP packet, nor does it overwrite new audio. In other words, the timing information stored in the payload is retained in the audio mixer 111.

[0163] 8 and 9, auxiliary data time difference calculation unit 11-3 shown in Fig. 5 calculates the auxiliary data time difference AT by transmitting and receiving auxiliary data RTP packets A1 and A3 and auxiliary data RTP packet A2 containing predetermined timing information. The auxiliary data time difference AT is then stored in memory 12. The auxiliary data time difference AT is obtained by subtracting the timestamp ATS1 stored in the header of auxiliary data RTP packet A1 from the timestamp ATS2 stored in the header of auxiliary data RTP packet A3.

[0164] 13 shows the structure of an RTP packet for auxiliary data, and indicates that timing information is stored in the payload. An RTP packet for auxiliary data consists of a header and a payload in 32-bit units.

[0165] 13, the header of the auxiliary data RTP packet is composed of data such as the version of the auxiliary data RTP packet, ..., time stamp, etc. The time stamp field stores time stamps added by the IP gateway 104, the time code generator 105, and the auxiliary data processor 118.

[0166] Ancillary data is stored in the payload area (UDW valid area) of the RTP packet for ancillary data by the IP gateway 104, the time code generator 105, and the ancillary data processor 118. Furthermore, timing information is stored in this payload area by the timing information assigner of the ancillary data time difference calculator 11-3.

[0167] For example, a series of null values ​​"000..." is used as timing information. The null values ​​"000..." stored in the payload of the auxiliary data RTP packet are "00:00:00:00" as actual time code auxiliary data, and are not subject to auxiliary data processing by the auxiliary data processing device 118. Therefore, the auxiliary data processing device 118 does not perform auxiliary data processing using the timing information of the null values ​​"000..." stored in the payload of the auxiliary data RTP packet, and does not overwrite new auxiliary data. In other words, the timing information stored in the payload is retained in the auxiliary data processing device 118.

[0168] (Delay time calculation unit 13) Next, a detailed description will be given of the processing of the delay time calculation unit 13 shown in Fig. 5. Fig. 10 is a flowchart showing an example of the processing of the delay time calculation unit 13.

[0169] The delay time calculation unit 13 reads the video time difference VT, the audio time difference ST, and the ancillary data time difference AT from the memory 12 (step S1001).The delay time calculation unit 13 then subtracts the audio time difference ST from the video time difference VT to obtain the audio delay time SD (step S1002).The delay time calculation unit 13 also subtracts the ancillary data time difference AT from the video time difference VT to obtain the ancillary data delay time AD (step S1003).

[0170] The delay time calculation unit 13 outputs the audio delay time SD to the audio delay device 112, and outputs the ancillary data delay time AD to the ancillary data delay device 113 (step S1004).

[0171] As described above, according to the video delay measuring device 2 of the embodiment of the present invention, the video time difference calculation unit 11-1 extracts the timestamp VTS1 of the generation time assigned by the camera control unit 102 from the received video RTP packet V1, and generates a video RTP packet V2 containing predetermined timing information. The video RTP packet V2 is sent to a video delay time difference measurement path including the video switcher 109, and returns as a video RTP packet V3 overwritten with the new timestamp VTS2.

[0172] The video time difference calculation unit 11-1 extracts the timestamp VTS2 of the video processing time assigned by the video switcher 109 from the received video RTP packet V3, and calculates the video time difference VT that reflects the delay time caused by the video switcher 109 by subtracting the timestamp VTS1 from the timestamp VTS2.

[0173] The audio time difference calculation unit 11-2 performs the same processing as the video time difference calculation unit 11-1 to calculate the audio time difference ST that reflects the delay time caused by the audio mixer 111. Furthermore, the auxiliary data time difference calculation unit 11-3 performs the same processing as the video time difference calculation unit 11-1 to calculate the auxiliary data time difference AT that reflects the delay time caused by the auxiliary data processing device 118.

[0174] The delay time calculation unit 13 calculates an audio delay time SD that reflects the delay time of the video relative to the audio by subtracting the audio time difference ST from the video time difference VT, and calculates an auxiliary data delay time AD by subtracting the auxiliary data time difference AT from the video time difference VT.

[0175] The audio delay time SD calculated in this way is used by audio delay device 112, which overwrites the timestamp included in the audio RTP packet with a timestamp delayed by the audio delay time SD, and the audio RTP packet is transmitted after being delayed by the audio delay time SD. The audio RTP packet is then forwarded to IP gateway 114. The overwritten timestamp included in the audio RTP packet will be the same as the timestamp included in the video RTP packet (the timestamp overwritten by video switcher 109) (both timestamps will fall within a predetermined threshold width).

[0176] Additionally, auxiliary data delay time AD is used by auxiliary data delay device 113, which overwrites the timestamp included in the auxiliary data RTP packet with a timestamp delayed by auxiliary data delay time AD, and the auxiliary data RTP packet is transmitted after being delayed by auxiliary data delay time AD. The auxiliary data RTP packet is then forwarded to IP gateway 114. The overwritten timestamp included in the auxiliary data RTP packet will be the same as the timestamp included in the video RTP packet (the timestamp overwritten in video switcher 109) (both timestamps will fall within a predetermined threshold width).

[0177] This allows the IP gateway 114 to match the reception timing of video RTP packets, audio RTP packets, and auxiliary data RTP packets that contain the same timestamp, thereby achieving synchronization of the three types of RTP packets.

[0178] This allows a simple method to measure the delay time of video relative to audio and auxiliary data in a program production system, enabling synchronization of video, audio, and auxiliary data generated at the same time, thereby enabling stable program production even in complex IP production systems.

[0179] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept thereof.

[0180] For example, the timing information adding unit 21 of the video time difference calculation unit 11-1 shown in Figure 8 uses a null value "000..." as timing information and generates a video RTP packet V2 including timing information of the null value "000...".

[0181] In contrast, the timing information adding unit 21 may use a value other than the null value "000..." as the timing information. In short, the timing information may be any value as long as the timing information stored in the payload of the video RTP packet V2 is retained in the device present on the video delay time difference measurement path shown in Figure 2. In the above embodiment, the timing information is sufficient as long as the video switcher 109 does not perform video processing and the processed video is not overwritten.

[0182] In addition, the timing information adding unit provided in the audio time difference calculation unit 11-2 uses a null value "000..." as timing information and generates an audio RTP packet S2 including timing information of the null value "000...".

[0183] In contrast, the timing information adding unit may use a value other than the null value "000..." as the timing information. In short, the timing information may be any value as long as the timing information stored in the payload of the audio RTP packet S2 is retained in the device present on the audio delay time difference measurement path shown in Figure 3. In the above embodiment, the timing information is sufficient as long as audio processing is not performed by the audio mixer 111 and the processed audio is not overwritten.

[0184] In addition, the timing information adding unit provided in the auxiliary data time difference calculation unit 11-3 uses a null value "000..." as timing information and generates an RTP packet A2 for auxiliary data that includes timing information of the null value "000...".

[0185] In contrast, the timing information adding unit may use a value other than the null value "000..." as the timing information. In short, the timing information may be any value as long as the timing information stored in the payload of the auxiliary data RTP packet A2 is retained in the device present on the path for measuring the auxiliary data delay time difference shown in Figure 4. In the above embodiment, the timing information is sufficient as long as the auxiliary data processing device 118 does not process the timing information and the processed auxiliary data is not overwritten.

[0186] Such timing information may be overwritten over the entire payload area of ​​the video RTP packet V2, the audio RTP packet S2, and the auxiliary data RTP packet A2, or overwritten over only a portion of the payload area. Also, the timing information may be stored in a portion of the header area.

[0187] Furthermore, in the above embodiment, the time code is described as the auxiliary data, but the auxiliary data is not limited to the time code and may be other data.

[0188] In addition, in the above embodiment, as shown in Figures 2, 3 and 4, the objects to be synchronized are the video input from the camera 101 by the camera control unit 102, the audio input from the microphone 106 by the audio IP gateway 107, and auxiliary data, which is the time code generated by the time code generator 105.

[0189] On the other hand, the video to be synchronized may be video input from the recording / playback device 103 via the IP gateway 104. The audio to be synchronized may be audio input from the camera 101 via the camera control unit 102, or audio input from the recording / playback device 103 via the IP gateway 104. The auxiliary data to be synchronized may be auxiliary data input from the recording / playback device 103 via the IP gateway 104.

[0190] Furthermore, in the above embodiment, the synchronization targets are video, audio, and auxiliary data, but they may be only video and audio, or only video and auxiliary data.

[0191] A typical computer can be used as the hardware configuration of the video delay measurement device 2 according to the embodiment of the present invention. The video delay measurement device 2 is configured by a computer equipped with a CPU, a volatile storage medium such as RAM, a non-volatile storage medium such as ROM, an interface, etc.

[0192] The functions of the packet transmitting / receiving unit 10, video time difference calculation unit 11-1, audio time difference calculation unit 11-2, auxiliary data time difference calculation unit 11-3, memory 12, and delay time calculation unit 13 provided in the video delay measuring device 2 are each realized by having a CPU execute a program that describes these functions.

[0193] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be stored in storage media such as magnetic disks (hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), semiconductor memories, etc. and distributed, or can be transmitted and received via a network. [Explanation of symbols]

[0194] 1,100 program production systems 2. Video delay measurement device 10 Packet Transmitter / Receiver 11-1 Video time difference calculation section 11-2 Audio time difference calculation section 11-3 Ancillary data time difference calculation section 12 Memory 13 Delay time calculation section 20 Time stamp extraction section 21 Timing information providing section 22 Timestamp extraction section 23 Delay time calculation section 101 Camera 102 Camera control unit 103,116,117 Recording and playback equipment 104,114 IP gateways 105 Time Code Generator 106 Mike 107 Voice IP Gateway 108 Network Switch 109 Video Switcher 110 Audio Network Switch 111 Audio Mixer 112 Audio delay device 113 Auxiliary Data Delay Device 115 SDI Synchronizer 118 Auxiliary Data Processing Device V1, V2, V3 Video RTP packets S1, S2, S3 RTP packets for audio A1, A2, A3 RTP packets for auxiliary data v,v',v'' video s Audio a. Ancillary data f1 frame VTS1,VTS2,STS1,STS2,ATS1,ATS2 timestamps VT video time difference ST audio time difference AT Auxiliary Data Time Difference SD Audio Delay Time AD auxiliary data delay time

Claims

1. A video delay measurement device that measures a delay time of a video stream relative to an audio stream as an audio delay time SD and delays the audio stream based on the audio delay time SD to synchronize the audio stream and the video stream, receiving a video RTP packet V1 storing a timestamp VTS1 of the time when the video was generated and the video; transmitting a video RTP packet V2 storing preset timing information; receiving a video RTP packet V3 in which the video RTP packet V2 has been delayed by processing by a video processing device present on the video path when the video RTP packet V2 passes through a predetermined video path, and in which the timestamp VTS2 of the time when the processing was performed has been overwritten in place of the timestamp VTS1; a packet transceiver unit that receives a timestamp STS1 of the time when the audio was generated and an audio RTP packet S1 in which the audio is stored, transmits an audio RTP packet S2 in which preset timing information is stored, and receives an audio RTP packet S3 in which the audio RTP packet S2 is delayed by processing by an audio processing device present on a predetermined audio path when the audio RTP packet S2 passes through the audio path, and in which the timestamp STS2 of the time when the processing was performed is overwritten in place of the timestamp STS1; When the video RTP packet V1 received by the packet transmitting / receiving unit is input, the timing information is stored to generate the video RTP packet V2, and the packet transmitting / receiving unit is caused to transmit the video RTP packet V2; a video time difference calculation unit that, when receiving the video RTP packet V3 received by the packet transmission / reception unit, calculates a delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; When the audio RTP packet S1 received by the packet transmitting / receiving unit is input, the audio RTP packet S2 is generated by storing the timing information and transmitted to the packet transmitting / receiving unit; an audio time difference calculation unit that, when receiving the audio RTP packet S3 received by the packet transmission / reception unit, calculates the delay amount of the audio stream as an audio time difference ST based on the time stamp STS1 included in the audio RTP packet S1 and the time stamp STS2 included in the audio RTP packet S3; a delay time calculation unit that calculates the audio delay time SD based on the video time difference VT calculated by the video time difference calculation unit and the audio time difference ST calculated by the audio time difference calculation unit; A video delay measuring device comprising:

2. 1. A video delay measurement device that measures a delay time of a video stream relative to an auxiliary data stream as an auxiliary data delay time AD and delays the auxiliary data stream based on the auxiliary data delay time AD to synchronize the auxiliary data stream and the video stream, receiving a video RTP packet V1 storing a timestamp VTS1 of the time when the video was generated and the video; transmitting a video RTP packet V2 storing preset timing information; receiving a video RTP packet V3 in which the video RTP packet V2 has been delayed by processing by a video processing device present on the video path when the video RTP packet V2 passes through a predetermined video path, and in which the timestamp VTS2 of the time when the processing was performed has been overwritten in place of the timestamp VTS1; a packet transceiver unit that receives an RTP packet A1 for auxiliary data in which a timestamp ATS1 indicating a time when auxiliary data was generated and the auxiliary data is stored, transmits an RTP packet A2 for auxiliary data in which preset timing information is stored, and receives an RTP packet A3 for auxiliary data in which the RTP packet A2 for auxiliary data is delayed by processing by an auxiliary data processing device present on a predetermined auxiliary data path when the RTP packet A2 for auxiliary data passes through the predetermined auxiliary data path, and in which a timestamp ATS2 indicating the time when the processing was performed is overwritten in place of the timestamp ATS1; When the video RTP packet V1 received by the packet transmitting / receiving unit is input, the timing information is stored to generate the video RTP packet V2, and the packet transmitting / receiving unit is caused to transmit the video RTP packet V2; a video time difference calculation unit that, when receiving the video RTP packet V3 received by the packet transmission / reception unit, calculates a delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; When the RTP packet A1 for auxiliary data received by the packet transceiver unit is input, the RTP packet A2 for auxiliary data is generated by storing the timing information and the packet transceiver unit transmits the RTP packet A2 for auxiliary data; an auxiliary data time difference calculation unit that, when receiving the auxiliary data RTP packet A3 received by the packet transceiver unit, calculates the delay amount of the auxiliary data stream as an auxiliary data time difference AT based on the timestamp ATS1 included in the auxiliary data RTP packet A1 and the timestamp ATS2 included in the auxiliary data RTP packet A3; a delay time calculation unit that calculates the auxiliary data delay time AD based on the video time difference VT calculated by the video time difference calculation unit and the auxiliary data time difference AT calculated by the auxiliary data time difference calculation unit; A video delay measuring device comprising:

3. a video delay measurement device that measures a delay time of a video stream relative to an audio stream as an audio delay time SD, measures a delay time of the video stream relative to an auxiliary data stream as an auxiliary data delay time AD, and delays the audio stream based on the audio delay time SD and delays the auxiliary data stream based on the auxiliary data delay time AD, thereby synchronizing the audio stream, the auxiliary data stream, and the video stream; receiving a video RTP packet V1 storing a timestamp VTS1 of the time when the video was generated and the video; transmitting a video RTP packet V2 storing preset timing information; receiving a video RTP packet V3 in which the video RTP packet V2 has been delayed by processing by a video processing device present on the video path when the video RTP packet V2 passes through a predetermined video path, and in which the timestamp VTS2 of the time when the processing was performed has been overwritten in place of the timestamp VTS1; receiving an audio RTP packet S1 storing a timestamp STS1 indicating the time when the audio was generated and the audio; transmitting an audio RTP packet S2 storing preset timing information; receiving an audio RTP packet S3 in which the audio RTP packet S2 has been delayed by processing by an audio processing device present on the audio path as the audio RTP packet S2 passes through a predetermined audio path, and in which the timestamp STS2 indicating the time when the processing was performed has been overwritten in place of the timestamp STS1; a packet transceiver unit that receives an RTP packet A1 for auxiliary data in which a timestamp ATS1 indicating a time when auxiliary data was generated and the auxiliary data is stored, transmits an RTP packet A2 for auxiliary data in which preset timing information is stored, and receives an RTP packet A3 for auxiliary data in which the RTP packet A2 for auxiliary data is delayed by processing by an auxiliary data processing device present on a predetermined auxiliary data path when the RTP packet A2 for auxiliary data passes through the predetermined auxiliary data path, and in which a timestamp ATS2 indicating the time when the processing was performed is overwritten in place of the timestamp ATS1; When the video RTP packet V1 received by the packet transmitting / receiving unit is input, the timing information is stored to generate the video RTP packet V2, and the packet transmitting / receiving unit is caused to transmit the video RTP packet V2; a video time difference calculation unit that, when receiving the video RTP packet V3 received by the packet transmission / reception unit, calculates a delay amount of the video stream as a video time difference VT based on the timestamp VTS1 included in the video RTP packet V1 and the timestamp VTS2 included in the video RTP packet V3; When the audio RTP packet S1 received by the packet transmitting / receiving unit is input, the audio RTP packet S2 is generated by storing the timing information and transmitted to the packet transmitting / receiving unit; an audio time difference calculation unit that, when receiving the audio RTP packet S3 received by the packet transmission / reception unit, calculates the delay amount of the audio stream as an audio time difference ST based on the time stamp STS1 included in the audio RTP packet S1 and the time stamp STS2 included in the audio RTP packet S3; When the RTP packet A1 for auxiliary data received by the packet transceiver unit is input, the RTP packet A2 for auxiliary data is generated by storing the timing information and the packet transceiver unit transmits the RTP packet A2 for auxiliary data; an auxiliary data time difference calculation unit that, when receiving the auxiliary data RTP packet A3 received by the packet transceiver unit, calculates the delay amount of the auxiliary data stream as an auxiliary data time difference AT based on the timestamp ATS1 included in the auxiliary data RTP packet A1 and the timestamp ATS2 included in the auxiliary data RTP packet A3; a delay time calculation unit that calculates the audio delay time SD based on the video time difference VT calculated by the video time difference calculation unit and the audio time difference ST calculated by the audio time difference calculation unit, and calculates the auxiliary data delay time AD based on the video time difference VT and the auxiliary data time difference AT calculated by the auxiliary data time difference calculation unit; A video delay measuring device comprising:

4. 2. The video delay measuring device according to claim 1, The timing information stored in the video RTP packet V2 is held when the video RTP packet V2 is processed by the video processing device when passing through the video path, and the timing information stored in the audio RTP packet S2 is held when the audio RTP packet S2 is processed by the audio processing device when passing through the audio path, The packet transceiver unit A video delay measuring device characterized in that the video RTP packet V3 is received by identifying it based on the timing information contained in the video RTP packet V3, and the audio RTP packet S3 is received by identifying it based on the timing information contained in the audio RTP packet S3.

5. 3. The video delay measuring device according to claim 2, The timing information stored in the video RTP packet V2 is held when the video RTP packet V2 passes through the video path and is processed by the video processing device, and the timing information stored in the auxiliary data RTP packet A2 is held when the auxiliary data RTP packet A2 passes through the auxiliary data path and is processed by the auxiliary data processing device, The packet transceiver unit A video delay measuring device characterized in that the video RTP packet V3 is received by identifying it based on the timing information contained in the video RTP packet V3, and the auxiliary data RTP packet A3 is received by identifying it based on the timing information contained in the auxiliary data RTP packet A3.

6. 6. The video delay measuring device according to claim 1, A video delay measuring device characterized in that the timing information is set to a null value.

7. A program for causing a computer to function as the video delay measuring device according to any one of claims 1 to 3.

Images