Video production device, program production system and program

Abstract

To reduce the amount of data of signals transmitted between a relay site and a broadcasting station.SOLUTION: A video creation device 100 is provided at a relay site where shooting is performed with a camera 11, and creates a return video related to use of a video captured with the camera 11 in a main line video for broadcasting. The video creation device 100 includes: a communication unit 101 that acquires, via an IP network 2, a control signal of a switcher 23 provided in the broadcasting station that produces the main line video by switching a plurality of videos including the video captured with the camera 11 transmitted to the broadcasting station; and a control unit 103 that creates the return video based on the control signal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video production device, a program production system, and a program. [Background technology]

[0002] As a new broadcast program production flow using IP (Internet Protocol), remote production has been attracting attention, in which the broadcast site and the broadcast station are connected via an IP network and the program is produced at the broadcast station away from the broadcast site (see, for example, Non-Patent Document 1). In remote production, there is a demand for low-latency program production, similar to conventional program production systems configured using SDI (Serial Digital Interface).

[0003] FIG. 12 is a diagram showing an example of the configuration of a typical program production system 1a using remote production.

[0004] 12, a relay site and a broadcast station are connected via an IP network 2. The relay site is provided with a camera 11, a monitor 12, an IP gateway 13, and a network switch 14. The broadcast station is provided with a network switch 21, an IP gateway 22, and a switcher 23a.

[0005] One or more cameras 11 are provided at the broadcast site, and are imaging devices that capture images of a subject in response to the operation of a cameraman. Camera 11 outputs an SDI signal conforming to the SDI standard, which is a video signal of the captured image, to IP gateway 13. Camera 11 has a function of displaying the return image, which will be described later, output from IP gateway 13 so that it can be viewed by the cameraman.

[0006] The IP gateway 13 converts and compresses the SDI signal output from the camera 11 into an IP signal conforming to the IP standard, and outputs the signal to the network switch 14. The IP gateway 13 also converts and decodes the IP signal output from the network switch 14 into an SDI signal. As will be described later, in the program production system 1a shown in FIG. 12, a video similar to the main line video produced and broadcast at the broadcast station is transmitted as a return video from the broadcast station to the relay site via the IP network 2. The IP gateway 13 converts and decodes the IP signal output from the network switch 14 (video signal of the return video) into an SDI signal, and outputs the signal to the camera 11 and the monitor 12.

[0007] The network switch 14 transfers the IP signal output from the IP gateway 13 to the broadcasting station side via the IP network 2. The network switch 14 also receives an IP signal (a video signal of a return video) transmitted from the broadcasting station side via the IP network 2, and transfers it to the IP gateway 13.

[0008] The network switch 21 receives an IP signal (a video signal of the video captured by the camera 11) transmitted from the relay site via the IP network 2, and outputs the signal to the IP gateway 22. The network switch 21 also transfers the IP signal output from the IP gateway 22 to the relay site via the IP network 2.

[0009] The IP gateway 22 converts and decodes the IP signal (video signal of the video shot by the camera 11) output from the network switch 21 into an SDI signal, and outputs it to the switcher 23a. The IP gateway 22 also converts and compresses the SDI signal (video signal of the returned video) output from the switcher 23a into an IP signal, and outputs it to the network switch 21.

[0010] The switcher 23a receives an SDI signal (a video signal of the video captured by the camera 11) from the IP gateway 22. The switcher 23a also receives video from a VTR and CG (Computer Graphics) used in program production. The switcher 23a also receives images such as subtitles used in program production. The switcher 23a appropriately switches between the above-mentioned multiple videos in accordance with a control signal to produce main line video for broadcast. The switcher 23a also sends back the produced main line video and outputs it to the IP gateway 22 as a return video.

[0011] 12 shows an example in which the IP gateways 13 and 22 are provided independently, but the present invention is not limited to this. The IP gateway 13 may be built into the camera 11, the monitor 12, or the network switch 14. The IP gateway 22 may be built into the network switch 21 or the switcher 23a. [Prior art documents] [Non-patent literature]

[0012] [Non-Patent Document 1] Junichiro Kawamoto, Satoshi Koyama, Masahiro Kawaraki, Kyoichi Saito, Takuya Kurakake, "Development of Lightly Compressed 8K IP Transmission Equipment for Remote Production," NHK STRL R&D, No. 179, January 2020. Summary of the Invention [Problem to be solved by the invention]

[0013] In the program production system 1a shown in Fig. 12, the return video transmitted from the broadcast station to the shooting site via the IP network 2 is displayed on the camera 11 and monitor 12, and is used, for example, by the person shooting the camera 11 as a reference for determining the operation of the camera 11. Here, if there is a large delay in the return video transmitted from the broadcast station, the difficulty of such an operation increases.

[0014] 13, the delay of the returned video is the sum of (1) the delay due to conversion and compression from SDI signals to IP signals in IP gateway 13, (2) the transmission delay of the IP signal from the relay site to the broadcast station, (3) the delay due to conversion and decoding from IP signals to SDI signals in IP gateway 22, (4) the processing delay in switcher 23a, (5) the delay due to conversion and compression from SDI signals to IP signals in IP gateway 22, (6) the transmission delay of the IP signal from the broadcast station to the relay site, (7) the delay due to conversion and decoding from IP signals to SDI signals in IP gateway 13, and the delay in the cables connecting each device. For smooth program production, the delay of the returned video needs to be kept below a certain allowable value.

[0015] Furthermore, transmitting high-quality video signals with low latency over the IP network 2 results in skyrocketing line costs. Skyrocketing line costs are not a problem when transmitting video within a local area such as within a studio, but they become a problem especially when transmitting video over a WAN (Wide Area Network).

[0016] Therefore, there is a demand for reducing the amount of data in the signals transmitted between the broadcasting site and the broadcast station.

[0017] An object of the present invention is to provide a video production device, a program production system, and a program that can solve the above-mentioned problems and reduce the amount of signal data transmitted between a broadcasting site and a broadcasting station. [Means for solving the problem]

[0018] In order to solve the above problems, the video production device of the present invention is a video production device that is installed at a broadcast site where filming is performed by a filming device, and creates return video regarding the use of the video captured by the filming device in the main line video for broadcast, and a broadcasting station connected to the broadcasting site via a network is provided with a switcher that produces the main line video by switching between multiple videos, including the filmed video captured by the filming device and transmitted to the broadcasting station via the network, and is equipped with a communication unit that acquires a control signal for the switcher via the network, and a control unit that produces the return video based on the control signal.

[0019] In addition, in the video creation device of the present invention, it is preferable that the communication unit further acquires the captured image of the shooting device and the return image in which the main line image is transmitted to the broadcast site via a transmission path different from the network, and the control unit determines whether or not to switch between the captured image and other images in the main line image based on the control signal, and switches between the acquired captured image and the return image depending on the result of the determination to create the return image.

[0020] In addition, in the video creation device of the present invention, it is preferable that the communication unit further acquires the captured image of the shooting device and the return image which is the main line image compressed and transmitted to the broadcast site via the network, and the control unit determines whether or not to switch between the captured image and other images in the main line image based on the control signal, and switches between the acquired captured image and the return image depending on the result of the determination to create the return image.

[0021] In addition, in the video creation device of the present invention, it is preferable that the control unit further includes a memory unit for storing images and videos prepared in advance as the return video, and when the control unit determines based on the control signal that the main line video has been switched from the image captured by the photographing device to another video, it creates the return video based on the images or videos prepared in advance and stored in the memory unit until the acquired return video is switched from the photographed video to the other video, and when the acquired return video is switched from the photographed video to the other video, it creates the return video based on the acquired return video.

[0022] In addition, in the image creation device of the present invention, it is preferable that the memory unit stores images and images known to be used in the main line image, and when the control unit determines based on the control signal that images and images known to be used in the main line image are used in the main line image, it creates the return image based on the images and images stored in the memory unit that are known to be used in the main line image.

[0023] In addition, the program production system of the present invention comprises a video production device installed at a broadcast site where filming is performed by a filming device, and creating return video regarding the use of the video captured by the filming device in the main line video for broadcast, and a switcher installed at a broadcasting station, and creating the main line video by switching between a plurality of videos including the video captured by the filming device and transmitted to the broadcasting station via a network, and the video production device comprises a communication unit that acquires a control signal for the switcher via the network, and a control unit that creates the return video based on the control signal.

[0024] Furthermore, a program according to the present invention causes a computer to function as any one of the image creation devices described above. Effect of the Invention

[0025] According to the video production device, program production system, and program of the present invention, it is possible to reduce the amount of data of signals transmitted between a broadcasting site and a broadcasting station. [Brief description of the drawings]

[0026] [Figure 1] 1 is a diagram showing an example of a configuration of a program production system according to an embodiment of the present invention. [Diagram 2] 2 is a diagram illustrating an example of the configuration of the video production device illustrated in FIG. 1. [Diagram 3] FIG. 11 is a diagram showing another example of the configuration of a program production system according to an embodiment of the present invention. [Figure 4] FIG. 11 is a diagram showing yet another example of the configuration of a program production system according to an embodiment of the present invention. [Diagram 5] 2 is a diagram for explaining a control signal of the switcher shown in FIG. 1. FIG. [Figure 6] 3 is a diagram showing an example of creating a return video by the control unit shown in FIG. 2. [Figure 7A] FIG. 13 is a diagram showing an example of an image or video prepared in advance as a return video. [Figure 7B] FIG. 13 is a diagram showing another example of an image or video prepared in advance as a return video. [Figure 7C] FIG. 13 is a diagram showing yet another example of an image or video prepared in advance as a return video. [Figure 8] 4 is a diagram showing another example of the creation of a return video by the control unit shown in FIG. 2. FIG. [Figure 9A] 3 is a diagram showing an example of superimposition display in return video by the control unit shown in FIG. 2. FIG. [Figure 9B] 13 is a diagram showing another example of displaying superimpositions in return video by the control unit shown in FIG. 2. FIG. [Figure 10] 3 is a diagram for explaining the creation of a return video by the control unit shown in FIG. 2 when the main line video is switched by MIX. FIG. [Figure 11]3 is a flowchart showing an example of the operation of the video production device shown in FIG. 2. [Figure 12] FIG. 1 is a diagram illustrating an example of the configuration of a conventional program production system. [Figure 13] FIG. 13 is a diagram for explaining a delay in a sent back video in the program production system shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0028] Fig. 1 is a diagram showing an example of the configuration of a program production system 1 according to an embodiment of the present invention. The program production system 1 according to this embodiment performs program production by remote production, in which a broadcast program is produced at a broadcasting station away from the broadcast site. In Fig. 1, the same components as those in Fig. 12 are given the same reference numerals, and the description thereof will be omitted.

[0029] As shown in Fig. 1, program production system 1 according to this embodiment includes camera 11 as a shooting device installed at the broadcast site, monitor 12, IP gateway 13, network switch 14, and video production device 100. Program production system 1 according to this embodiment also includes network switch 21, IP gateway 22, and switcher 23 installed at the broadcast station. Program production system 1 according to this embodiment differs from program production system 1a shown in Fig. 12 in that switcher 23a is replaced with switcher 23 and video production device 100 is added.

[0030] The switcher 23 switches between a plurality of videos (such as VTR, CG, and superimposed images) including the video captured by the camera 11 and transmitted to the broadcast station via the IP network 2 to create a main line video for broadcast. The switcher 23 switches between the plurality of videos to create a main line video in accordance with a control signal related to the switching between the plurality of videos. In this embodiment, the switcher 23 outputs a control signal related to the switching between the plurality of videos to the network switch 21 in order to create a main line video. The control signal output to the network switch 21 is transmitted to the broadcast site side via the IP network 2. The control signal transmitted to the broadcast site side is received by the network switch 14 and output to the video creation device 100 described later. That is, the switcher 23 transmits a control signal related to the switching between the plurality of videos to the video creation device 100 via the IP network 2.

[0031] The image production device 100 receives an input of a control signal from the switcher 23 from the network switch 14. The image production device 100 also receives an input of an image captured by the camera 11. When multiple cameras 11 are installed at a broadcast site, the image production device 100 receives an input of the images captured by each of the multiple cameras 11. The image production device 100 creates a return image corresponding to the main line image. The return image is displayed at the shooting site and is an image related to the use of the image captured by the camera 11 in the main line image. The return image is displayed, for example, on the camera 11 or the monitor 12 and is used by the person filming the camera 11 as a reference for operating the camera 11. The image production device 100 creates the return image based on the input control signal.

[0032] FIG. 2 is a diagram showing an example of the configuration of the video production device 100. As shown in FIG.

[0033] As shown in FIG. 2, the video production device 100 includes a communication unit 101, a storage unit 102, and a control unit 103.

[0034] The communication unit 101 includes at least one communication interface. The communication interface is, for example, a LAN (Local Area Network) interface or a Bluetooth (registered trademark) interface. The communication unit 101 is capable of wired or wireless communication with each component (such as the camera 11, the monitor 12, the network switch 14, and a broadcast tuner 15 described later) provided at the broadcasting site via the communication interface. The communication unit 101 includes, for example, an interface for SDI input to acquire the video (SDI signal) captured by the camera 11.

[0035] The storage unit 102 is, for example, a semiconductor memory, a magnetic memory, or an optical memory, but is not limited thereto. The storage unit 102 may function, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 102 stores any information used in the operation of the image creation device 100. For example, the storage unit 102 may store a system program, an application program, or the like. The storage unit 102 also stores images and images prepared in advance as return images. The images and images prepared in advance as return images may include, for example, images and images indicating that the image used in the main line image has been switched from the image captured by the camera 11 to another image. Also, images (such as superimposed images) and images (such as VTRs and CGs) that are known to be used in the main line image may be included.

[0036] The control unit 103 includes at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application specification integrated circuit (ASIC). The control unit 103 executes processes related to the operation of the image creation device 100 while controlling each part of the image creation device 100.

[0037] The control unit 103 creates a return video based on a control signal of the switcher 23 output from the network switch 14. For example, the control unit 103 determines whether or not the video captured by the camera 11 is being used as the main line video based on the control signal, and when it determines that the video captured by the camera 11 is being used as the main line video, it uses the video captured by the camera 11 as the return video until it determines that the main line video has been switched to another video. When it determines that the main line video has been switched from the video captured by the camera 11, the control unit 103 switches the return video from the video captured by the camera 11 to another video. The control unit 103 outputs the created return video to the camera 11 and the monitor 12 via the communication unit 101 to display them.

[0038] If the photographer of the camera 11 knows whether the image captured by the camera 11 is being used as the main line image (i.e., the timing of switching of the image captured by the camera 11 used as the main line image), he / she can determine whether or not it is OK to operate the camera 11, such as moving the camera 11 or stopping the image capture of the camera 11. Therefore, even if the return image is not displayed as in the conventional program production system 1a, the photographer of the camera 11 can determine the operation of the camera 11 from the return image regarding the use of the image captured by the camera 11 in the main line image, which is created by the control unit 103. Here, the control signal of the switcher 23 usually has a smaller data amount than the return image. Therefore, according to the program production system 1 of this embodiment, it is possible to reduce the data amount of the signal transmitted between the relay site and the broadcasting station.

[0039] The video production device 100 may create a return video based on the return video transmitted from the broadcast station to the live broadcast site. In this embodiment, the return video is transmitted via a transmission path other than the IP network 2, or with a reduced increase in the data amount of the signal transmitted via the IP network 2 compared to the conventional program production system 1. Therefore, even if the program production system 1 according to this embodiment creates a return video using the return video, it is possible to reduce the data amount of the signal transmitted via the IP network 2 compared to the conventional program production system 1.

[0040] As a configuration for acquiring the return video at the broadcast site, for example, as shown in FIG. 1, there is a configuration in which a broadcast tuner 15 for receiving a broadcast using a main line video is provided at the broadcast site. In this configuration, the broadcast tuner 15 receives the broadcasted main line video and inputs the received main line video (hereinafter referred to as "broadcast video") to the video creation device 100. The communication unit 101 acquires the broadcast video (main line video transmitted via a transmission path different from the IP network 2) as a return video. The control unit 103 switches between the video captured by the camera 11 and the broadcast video (return video) based on the control signal to create the return video. For example, the control unit 103 determines whether or not to switch between the video captured by the camera 11 and other video in the main line video based on the control signal, and depending on the determination result, switches between the acquired captured video and the broadcast video (return video in which the main line video is transmitted to the broadcast site via a transmission path different from the IP network 2) to create the return video. Generally, two or more lines of video can be input to camera 11, so control unit 103 may input broadcast video to camera 11 in addition to the return video. Also, broadcast video received by broadcast tuner 15 may be split into two lines, with one line of broadcast video being input to video creation device 100 and the other line of broadcast video being input directly to camera 11.

[0041] Furthermore, when the control unit 103 determines based on the control signal that the main line video uses images and videos that are known to be used in the main line video, the control unit 103 may create a return video based on the known images and videos that are pre-stored in the storage unit 102. A delay of, for example, about 2 to 3 seconds occurs from when the captured video is captured by the camera 11 until the main line video including the captured video is broadcast and received by the broadcast tuner 15. Therefore, if the return video is created using the broadcast video received by the broadcast tuner 15, a delay occurs in the return video. Therefore, the images and videos that are known to be used in the main line video are pre-stored in the storage unit 102, and when the images and videos are used in the main line video, the return video is created using the stored images and videos, thereby suppressing the delay in the return video.

[0042] FIG. 1 illustrates an example in which the image production device 100 acquires main line images (returned images) from the broadcast tuner 15 installed at the broadcast site, but the method of acquiring the return images at the broadcast site is not limited to this example.

[0043] Fig. 3 is a diagram showing another example of the configuration of the program production system 1 according to this embodiment. In Fig. 3, the same components as those in Fig. 1 are given the same reference numerals, and the description thereof will be omitted.

[0044] The program production system 1 shown in Fig. 3 includes a camera 11, a monitor 12, an IP gateway 13, a network switch 14, a video production device 100, and a return receiving device 16, which are installed at the broadcasting site. The program production system 1 shown in Fig. 3 also includes a network switch 21, an IP gateway 22, a switcher 23, and a return transmitting device 24, which are installed at the broadcasting station. The program production system 1 shown in Fig. 3 differs from the program production system 1 shown in Fig. 1 in that the return transmitting device 24 and the return receiving device 16 are added.

[0045] The return transmission device 24 receives the main line video created by the switcher 23. The return transmission device 24 converts and compresses the video signal of the input main line video into an IP signal and transmits it to the relay site as a return video (compressed return video). Specifically, the return transmission device 24 generates a video (compressed return video) by compressing the main line video to a low image quality and low frame rate, and converts the video signal of the compressed return video into IP packets and outputs them to the network switch 21.

[0046] The network switch 21 transmits the control signal of the switcher 23 input from the switcher 23 to the relay site side via the IP network 2. The network switch 21 also transmits the video signal (IP signal) of the compressed return video output from the return transmitting device 24 to the relay site side via the IP network 2.

[0047] The network switch 14 receives a control signal for the switcher 23 transmitted from the broadcast station side, and outputs it to the IP gateway 13. The network switch 14 also receives compressed return video transmitted from the broadcast station side, and outputs it to the return receiving device 16.

[0048] The return receiving device 16 converts and decodes the IP signal output from the network switch 14 into an SDI signal to obtain the compressed return video, and outputs it to the video production device 100.

[0049] The image creation device 100 (control unit 103) may create a return image based on the compressed return image (compressed main line image) output from the return receiving device 16. For example, the image creation device 100 may determine whether or not to switch between the image captured by the camera 11 and other images in the main line image based on a control signal, and may create a return image by switching between the acquired captured image and the compressed return image according to the result of the determination. The image creation device 100 (control unit 103) may output the created return image to the camera 11 and the monitor 12 via the communication unit 101 for display. As described above, two or more lines of images can be input to the camera 11. The control unit 103 may input the compressed main line image to the camera 11 in addition to the return image. The compressed return image received by the return receiving device 16 may be branched into two lines, the return image of one line is input to the image creation device 100, and the return image of the other line is directly input to the camera 11.

[0050] As described above, when the broadcasted main line video is acquired from the broadcast tuner 15, a delay of about 2 to 3 seconds occurs from when the captured video is captured by the camera 11 until the main line video including the captured video is received by the broadcast tuner 15. On the other hand, when the compressed return video is transmitted via the IP network 2, even if the delay due to compression and decoding of the main line video (e.g., several hundred ms) and the transmission delay (e.g., several tens of ms) of transmitting the main line video via the IP network 2 are combined, the amount of delay is smaller than when the main line video is acquired from the broadcast tuner 15. Therefore, by transmitting the compressed return video of the main line video (return video with reduced data amount) from the broadcast station to the broadcast site via the IP network 2, the delay of the return video can be suppressed.

[0051] Fig. 4 is a diagram showing yet another example of the configuration of the program production system 1 according to this embodiment. In Fig. 4, the same components as those in Fig. 3 are given the same reference numerals, and the description thereof will be omitted.

[0052] The program production system 1 shown in Fig. 4 includes a camera 11, a monitor 12, an IP gateway 13, a network switch 14, a video production device 100, and a return receiving device 16A, which are installed at the broadcasting site. The program production system 1 shown in Fig. 4 also includes a network switch 21, an IP gateway 22, a switcher 23, and a return transmitting device 24A, which are installed at the broadcasting station. The program production system 1 shown in Fig. 4 differs from the program production system 1 shown in Fig. 3 in that the return transmitting device 24 and the return receiving device 16 are changed to the return transmitting device 24A and the return receiving device 16A, respectively.

[0053] The return transmitting device 24A, unlike the return transmitting device 24, transmits compressed return video to the return receiving device 16A at the relay site via a mobile phone network. The return receiving device 16A receives the compressed return video transmitted via the mobile phone network and outputs it to the video creation device 100.

[0054] The image creation device 100 (control unit 103) switches between the image captured by the camera 11 and the compressed return image (compressed main line image) based on the control signal to create a return image. For example, the control unit 103 determines whether or not to switch between the image captured by the camera 11 and other images in the main line image based on the control signal, and switches between the acquired captured image and the compressed return image according to the determination result to create a return image. The image creation device 100 (control unit 103) outputs the created return image to the camera 11 and the monitor 12 via the communication unit 101 to display them. As described above, since two or more lines of images can be input to the camera 11, the control unit 103 may input the compressed return image to the camera 11 in addition to the return image. Also, the compressed return image received by the return receiving device 16 may be branched into two lines, the return image of one line may be input to the image creation device 100, and the return image of the other line may be directly input to the camera 11.

[0055] When compressed return video is transmitted via a mobile phone network, even when the delay due to compression and decoding of the main line video (e.g., several hundred ms) and the transmission delay (e.g., several hundred ms) of transmitting through the mobile phone network are combined, the amount of delay is smaller than when the main line video is obtained from broadcast tuner 15. Therefore, by transmitting the return video compressed from the main line video (return video with reduced data volume) from the broadcast station to the broadcast site via the mobile phone network, the delay in the return video can be suppressed.

[0056] Any device can be used as the return transmitting device 24, 24A as long as it has the function of compressing the main line video into a video with a low image quality and low frame rate and transmitting it via the IP network 2 or a mobile phone network. Any device can be used as the return receiving device 16, 16A as long as it has the function of decoding the compressed video signal transmitted via the IP network 2 or a mobile phone network.

[0057] Next, the operation of the video production apparatus 100 will be described in more detail. First, the control signal of the switcher 23 will be described with reference to FIG. 5. Hereinafter, it will be described assuming that a tally signal is used as the control signal of the switcher 23. The tally signal is a signal indicating whether the corresponding video is being used as the main video.

[0058] In FIG. 5, it is assumed that a plurality of videos (the captured videos of two cameras 11 (relay-site cameras 11-1 and 11-2) provided at the relay site, the captured video of a camera (studio camera) provided at the studio (broadcasting station), the VTR and CG of studio playback) are switched to produce the main video.

[0059] Specifically, as shown in FIG. 5, it is assumed that from time t11, the captured video of the relay-site camera 11-1 is used as the main video, and at time t12, the main video is switched from the captured video of the relay-site camera 11-1 to the captured video of the studio camera. In this way, the video switching method of switching the main video from one video to another video at a certain time is hereinafter referred to as "CUT". Next, it is assumed that at time t14, the main video is switched from the captured video of the studio camera to the VTR of studio playback by "CUT". Also, it is assumed that CG is superimposed on the captured video of the studio camera from time t13 (t12 < t13 < t14) to time t14. Next, it is assumed that from time t15 to time t16, the main video is switched so that the VTR of studio playback gradually disappears and the captured video of the relay-site camera 11-2 is gradually displayed. In this way, the video switching method of gradually switching the main video from one video to another video over a predetermined time is hereinafter referred to as "MIX". Next, it is assumed that at time t17, the main video is switched from the captured video of the relay-site camera 11-2 to the VTR of studio playback by "CUT".

[0060] When the main line video is switched as shown in FIG. 5, at time t12, the tally signal corresponding to the video shot by the broadcast site camera 11-1 turns from ON to OFF, and the tally signal corresponding to the video shot by the studio camera turns ON. At time t13, the tally signal corresponding to the CG turns ON. At time t14, the tally signal corresponding to the VTR for studio playback turns ON, and the tally signal corresponding to the video shot by the studio camera and the tally signal corresponding to the CG turn OFF. At time t15, the tally signal corresponding to the video shot by the broadcast site camera 11-2 turns ON. At time t15, the tally signal corresponding to the VTR for studio playback remains ON. At time t16, the tally signal corresponding to the VTR for studio playback turns OFF. At time t17, the tally signal corresponding to the video shot by the broadcast site camera 11-2 turns OFF, and the tally signal corresponding to the VTR for studio playback turns ON.

[0061] In this way, the tally signal corresponding to each image is turned ON or OFF according to the switching of the image used for the main line image. Based on the tally signal corresponding to each image, the control unit 103 can determine which image is being used as the main line image. Therefore, based on the tally signal, the control unit 103 can determine whether or not the image captured by the camera 11 is being used as the main line image, and can switch between the captured image and the main line image (broadcast image or compressed returned image) according to the result of the determination, to create the return image.

[0062] Next, the creation of return video by control unit 103 will be described in more detail with reference to Fig. 6. In the following, it is assumed that the video (main line video) after switching by switcher 23 is switched in the order of video shot by camera 11-1, video shot by camera 11-2, VTR, video shot by camera 11-2, and video shot by camera 11-1, as shown in Fig. 6. In the following, it is assumed that the broadcast video received by broadcast tuner 15 and the return video transmitted via IP network 2 or a mobile phone network are input to video creation device 100, as shown in Figs. 3 and 4.

[0063] As described above, the broadcast video and the return video are delayed by a predetermined delay amount relative to the main line video (video after switching). In the following, the delay amount of the broadcast video relative to the main line video is Ta, and the delay amount of the return video relative to the main line video is Tb. As described above, the delay amount Ta of the broadcast video is about several seconds, and the delay amount of the compressed return video is about several hundred milliseconds. Therefore, as shown in Figure 6, the delay amount Ta of the broadcast video is greater than the delay amount Tb of the return video.

[0064] The control unit 103 creates a return video based on the control signal of the switcher 23 transmitted from the broadcasting station. If the total delay amount, which is the transmission delay of transmitting the control signal of the switcher 23 via the IP network 2 and the processing delay in the video creation device 100, is smaller than the delay amount Tb of the returned video, the delay amount Tc of the return video relative to the main line video will be smaller than the delay amount Tb of the returned video. The data amount of the control signal of the switcher 23 is much smaller than that of the returned video, and the transmission delay due to transmission via the IP network 2 is very small. Therefore, as shown in FIG. 6, the delay amount Tc of the return video is usually smaller than the delay amount Tb of the returned video.

[0065] When the control unit 103 determines based on the control signal (tally signal) that the image captured by the camera 11-1 is being used as the main line image, it outputs the image captured by the camera 11-1 as the return image. Next, when the control unit 103 determines based on the control signal that the main line image has been switched from the image captured by the camera 11-1 to the image captured by the camera 11-2, it outputs the image captured by the camera 11-2 as the return image.

[0066] Next, the control unit 103 determines that the main line video has been switched from the video captured by the camera 11-2 to the VTR based on the control signal. When the main line video is switched from the video captured by the camera 11-2 at the broadcast site to the video not captured at the broadcast site, there are several possible patterns for creating the return video.

[0067] In the first pattern (Pattern A), when the control unit 103 determines based on the control signal that the main line video has been switched from the video shot by camera 11-2 to a VTR (video not available at the broadcast site), the control unit 103 switches the return video from the video shot by camera 11-2 to an image or video prepared in advance. The image or video prepared in advance is stored in advance in the storage unit 102. In Pattern A, the control unit 103 outputs the image or video prepared in advance as the return video until it determines based on the control signal that the VTR has been switched to the video shot by camera 11-2 (video available at the broadcast site) in the main line video.

[0068] In the second pattern (pattern B), when the control unit 103 determines based on the control signal that the main line video has been switched from the video captured by the camera 11-2 to the VTR, it switches the return video from the video captured by the camera 11-2 to a previously prepared image or video for a predetermined period of time. For example, as shown in Fig. 6, the control unit 103 outputs the previously prepared image or video as the return video until the return video is switched from the video captured by the camera 11-2 to the VTR, and thereafter outputs the VTR in the return video as the return video.

[0069] In the third pattern (pattern C), when the control unit 103 determines based on the control signal that the main line video has been switched from the video captured by camera 11-2 to a VTR (video not available at the broadcast site), it switches the return video from the video captured by camera 11-2 to a previously prepared image or video for a predetermined period of time. For example, as shown in Fig. 6, the control unit 103 outputs the previously prepared image or video as the return video until the broadcast video is switched from the video captured by camera 11-2 to the VTR, and thereafter outputs the VTR in the broadcast video as the return video.

[0070] In patterns B and C, when the control unit 103 determines based on the control signal that the main line video has been switched from the image captured by camera 11 to another video (video not available at the broadcast site), it creates a return video based on an image or video prepared in advance and stored in the storage unit 102 until the acquired main line video (broadcast video or compressed returned video) is switched from the video captured by camera 11 to the other video. Then, when the acquired main line video is switched from the video captured by camera 11 to the other video, the control unit 103 creates a return video based on the acquired main line video.

[0071] In this way, instead of outputting a previously prepared image or video as a return video for a certain period of time (hereinafter referred to as the "switching standby time") and then outputting the broadcast video or the returned video as the return video, it is also possible to output the video shot by camera 11-2 as the return video during the switching standby time. However, in this case, the same video shot by camera 11-2 will be repeatedly output as the return video. On the other hand, with the switching methods of Pattern B and Pattern C, it is possible to prevent the same video from being repeatedly output as the return video, as shown in FIG. 6.

[0072] The switching waiting time in pattern B is a value obtained by subtracting the delay amount Tc of the return video from the delay amount Tb of the return video. The switching waiting time in pattern C is a value obtained by subtracting the delay amount Tc of the return video from the delay amount Ta of the broadcast video. The delay amount Ta of the broadcast video is constant. The delay amount Tb of the return video fluctuates because it includes jitter in the transmission time of the IP network 2 or the mobile phone network. However, since the amount of fluctuation is small compared to the total delay amount Tb, the delay amount Tb of the return video can be considered to be approximately constant. The delay amount Tc of the return video fluctuates due to jitter in the transmission time of the control signal of the switcher 23, etc., but the amount of fluctuation is very small compared to the delay amount Tb of the return video and the delay amount Ta of the broadcast video. Therefore, the delay amount Tc of the return video can be considered to be a constant value. The switching waiting time is manually set in the video creation device 100 based on, for example, the known delay amounts Ta, Tb, and Tc. Moreover, the control unit 103 may calculate and set the switching wait time based on the known delay amounts Ta, Tb, and Tc.

[0073] As described above, the delay amount Ta of the broadcast video, the delay amount Tb of the return video, and the delay amount Tc of the return video are all constant and known. Therefore, if the control unit 103 can specify the timing at which the main line video is switched from the video captured by the camera 11-2 to the VTR based on the control signal, it can also specify the timing at which the video captured by the camera 11-2 is switched to the VTR in the return video and the broadcast video.

[0074] The image or video prepared in advance in Fig. 6 may be any image or video, so long as it is clear that the main line video has been switched from the video captured by camera 11. Furthermore, a plurality of images or videos may be prepared in advance to match the video to which the main line video will be switched. For example, an image or video (Fig. 7A) indicating that the video to be switched to is a VTR, an image or video (Fig. 7B) indicating that the video to be switched to is a replay video (a video that repeats the video captured by camera 11), an image or video (Fig. 7C) indicating that the video to be switched to is a video captured by a studio camera, and the like may be prepared in advance and stored in storage unit 102.

[0075] Fig. 8 is a diagram showing another example of the creation of return video by the control unit 103. In Fig. 8, it is assumed that images or videos known to be used for the main line video are stored in advance in the storage unit 102 as images and videos prepared in advance as return video. In the following, it is assumed that the main line video is switched in the order of the video captured by camera 11-1, the video captured by camera 11-2, VTR1, VTR2, the video captured by camera 11-2, the replay video, image 1, and image 2, as shown in Fig. 8. Here, it is assumed that VTR1, VTR2, image 1, and image 2 are known to be used for the main line video and are stored in the storage unit 102.

[0076] The control unit 103 switches the return video in the order of the video captured by the camera 11-1, the video captured by the camera 11-2, VTR1, and VTR2 in accordance with the switching of the main line video in the order of the video captured by the camera 11-1, the video captured by the camera 11-2, VTR1, and VTR2. Specifically, the control unit 103 switches the video in the return video by delaying the return video by the delay amount Tc of the return video relative to the main line video from the timing of switching of the video in the main line video. Here, when the main line video is switched from the video captured by the camera 11-2 to VTR1, the control unit 103 switches the return video from the video captured by the camera 11-2 to VTR1 stored in the storage unit 102. Also, when the main line video is switched from VTR1 to VTR2, the control unit 103 switches the return video from VTR1 to VTR2 stored in the storage unit 102.

[0077] Next, when the main line video is switched from the video captured by camera 11-2 to the replay video, the control unit 103 switches the return video from the video captured by camera 11-2 to the replay video. Here, the replay video is a video that repeats the video captured by camera 11, and therefore cannot be prepared in advance. Therefore, when the main line video is switched from the video captured by camera 11-2 to the replay video, the control unit 103 switches the return video from the video captured by camera 11-2 to an image or video prepared in advance, for example, as shown in FIG. 7B, when the return video is switched from the video captured by camera 11-2 to the replay video, the control unit 103 switches the return video from the image or video prepared in advance to the replay video in the return video.

[0078] Next, when the main line video is switched from the replay video to image 1 and image 2 in that order, the control unit 103 switches the return video from the replay video to image 1 and image 2 stored in the storage unit 102.

[0079] In this way, when the control unit 103 determines based on the control signal that images and images known to be used in the main line video are being used in the main line video, it may create a return video based on the images and images stored in the memory unit 102 that are known to be used in the main line video.

[0080] In addition, the contents, display positions, display order, etc. of the supers that are decided to be used in the main line video may be stored in the storage unit 102 in advance. In this case, when the control unit 103 determines that the supers are displayed in the main line video based on the control signal, the control unit 103 may display the supers stored in the storage unit 102 in the return video in accordance with the display mode of the supers in the main line video. For example, it is pre-determined that the super 1 is displayed in the video captured by the camera 11 as the main line video, and another super 2 is displayed in accordance with the zoom of the video captured by the camera 11. In this case, as shown in FIG. 9A, the control unit 103 may display the video captured by the camera 11 as the return video, and may display the supers 1 and 2 in the video captured by the camera 11 in accordance with the zoom of the video captured by the camera 11. In this way, the control unit 103 can display the same supers as the supers in the main line video in the return video without waiting for the reception of the broadcast video or the main line video.

[0081] In addition, there are cases where the contents, display positions, and display order of supers used in the main line video are not determined in advance, such as in sports broadcasts. In this case, as shown in FIG. 9B, when a tally signal corresponding to a super is turned on, the control unit 103 may display, in the return video, a super 3, which indicates that a super is displayed in the main line video, different from the super used in the main line video, in the video captured by the camera 11. By displaying the super 3, which indicates that a super is displayed in the main line video, as the return video, the photographer of the camera 11 can understand that a super is displayed in the main line video. If the photographer wants to know the details (display position, contents, etc.) of a super displayed in the main line video, if the return video is input to the camera 11 in addition to the return video, the photographer can grasp the details of the super by switching the video displayed in the viewfinder of the camera 11 from the return video to the return video. Therefore, when a super is displayed in the main line video, the video creation device 100 keeps displaying the super 3 as the return video without displaying the return video.

[0082] As described above, one method of switching images in the main line images is to gradually switch the main line images from one image to another image over a predetermined period of time (MIX). In the following, the creation of return images by the control unit 103 when the main line images are switched by MIX will be described with reference to Fig. 10. In Fig. 10, it is assumed that the main line images are gradually switched from the images captured by camera 11-1 to the images captured by camera 11-2 by MIX from time t21 to time t23, which is a time Td later.

[0083] When video switching by MIX is performed, at time t21, the tally signal corresponding to the video shot by camera 11-1 remains ON, while the tally signal corresponding to the video shot by camera 11-2 turns ON. Then, at time t23, the tally signal corresponding to the video shot by camera 11-1 turns OFF.

[0084] The control unit 103 can grasp that the main line video is being switched by MIX, based on the control signal (tally signal) of the switcher 23. Specifically, the control unit 103 can grasp that the video shot by the camera 11-1 is being switched to the video shot by the camera 11-2 by MIX, because the tally signal corresponding to the video shot by the camera 11-2 is turned ON while the tally signal corresponding to the video shot by the camera 11-1 remains ON.

[0085] When the control unit 103 grasps that the main line video is switched by MIX from the video captured by camera 11-1 to the video captured by camera 11-2 at time t22 (time t21<time t22<time t23), the control unit 103 gradually switches the return video from the video captured by camera 11 to the video captured by camera 11-2 from time t22 to time t24, which is a predetermined time Te later. Here, the time Td at which the video is switched in the main line video by MIX differs depending on the operation of the switcher 23 and is not constant. In addition, the time Td at which the video is switched in the main line video by MIX cannot be grasped at the broadcast site. Therefore, the time Td at which the video is switched in the main line video by MIX does not necessarily match the time Te at which the control unit 103 switches the return video by MIX.

[0086] Next, a description will be given of the operation of the image creation device 100 according to this embodiment. Fig. 11 is a flowchart showing an example of the operation of the image creation device 100 according to this embodiment, and is a diagram for explaining a method of creating an image by the image creation device 100.

[0087] The communication unit 101 acquires a control signal for the switcher 23 transmitted via the IP network 2 (step S11).

[0088] The control unit 103 creates a return image based on the acquired control signal (step S12). For example, when the control unit 103 determines that the image used for the main line image has been switched from the image captured by the camera 11 based on the control signal, the control unit 103 switches the return image from the image captured by the camera 11 to another image.

[0089] Thus, the video creation device 100 according to this embodiment includes a communication unit 101 and a control unit 103. The communication unit 101 acquires, via the IP network 2, a control signal for a switcher 23 provided at the broadcasting station that switches between multiple videos including the captured video captured by the camera 11 and transmitted to the broadcasting station via the IP network 2 to create a main line video. Based on the control signal from the switcher 23, the control unit 103 creates a return video regarding the use of the captured video in the main line video.

[0090] The control signal of the switcher 23 has a much smaller data volume than the return video (main line video). Therefore, by receiving the control signal of the switcher 23 from the broadcasting station and creating the return video based on the control signal, it is possible to reduce the data volume of the signal transmitted between the broadcasting site and the broadcasting station compared to the case of transmitting the return video.

[0091] By reducing the amount of data of signals transmitted between the relay site and the broadcast station, if the amount of delay in the return video is to be the same as in a conventional system, the allowable amount of delay due to signal conversion and compression in the IP gateways 13, 22 can be increased. Also, by reducing the amount of data of signals transmitted between the relay site and the broadcast station, if the amount of delay in the return video is to be the same as in a conventional system, the allowable amount of transmission delay in the network switches 14, 21 can be increased. Also, by reducing the amount of data of signals transmitted between the relay site and the broadcast station, the line costs for the IP network 2 can be reduced.

[0092] Although not specifically mentioned in the embodiment, a program for causing a computer to function as the video creation device 100 may be provided. The program may be recorded in a computer-readable medium. The computer-readable medium can be used to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0093] Alternatively, a chip may be provided that is configured with a memory that stores programs for executing each process performed by the image creation device 100, and a processor that executes the programs stored in the memory, and is mounted on the image creation device 100.

[0094] Although the above-mentioned embodiment has been described as a representative example, it is clear to those skilled in the art that many modifications and substitutions are possible within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple building blocks shown in the configuration diagram of the embodiment into one, or to divide one building block. [Explanation of symbols]

[0095] 1,1a Video production system 2 IP Network 11 Camera (photography device) 12 Monitors 13,22 IP Gateway 14,21 Network Switch 15 Broadcast Tuner 16,16A Return receiving device 23 Switcher 24,24A Return transmitter 100 Video Production Equipment 101 Communications Department 102 Storage section 103 Control section

Claims

1. A video production device is provided at a broadcasting site where a video is shot by a video camera, and produces a return video regarding the use of the video shot by the video camera in a main line video for broadcasting, A broadcasting station connected to the broadcasting site via a network is provided with a switcher that switches between a plurality of images including the images captured by the imaging device and transmitted to the broadcasting station via the network to produce the main line image; A communication unit that acquires a control signal for the switcher via the network; a control unit that creates the return video based on the control signal.

2. 2. The image creation device according to claim 1, The communication unit further acquires the captured image of the imaging device and a return image of the main line image transmitted to the relay site via a transmission path different from the network, The control unit determines whether or not to switch between the captured image and other images in the main line image based on the control signal, and switches between the acquired captured image and the returned image depending on the result of the determination, to create the return image.

3. 2. The image creation device according to claim 1, The communication unit further acquires the captured image of the imaging device and a return image obtained by compressing the main line image and transmitting it to the relay site via the network, The control unit determines whether or not to switch between the captured image and other images in the main line image based on the control signal, and switches between the acquired captured image and the returned image depending on the result of the determination, to create the return image.

4. 4. The image creation device according to claim 2, A storage unit is further provided for storing images and videos prepared in advance as the return video, When the control unit determines based on the control signal that the main line image has been switched from the image captured by the photographing device to another image, it creates the return image based on a pre-prepared image or image stored in the memory unit until the acquired return image is switched from the photographed image to the other image, and when the acquired return image is switched from the photographed image to the other image, it creates the return image based on the acquired return image.

5. 5. The image creation device according to claim 4, The storage unit stores images and videos that are known to be used for the main line video, When the control unit determines based on the control signal that images and videos known to be used in the main line video are used in the main line video, the control unit creates the return video based on the images and videos known to be used in the main line video and stored in the memory unit.

6. an image creation device that is provided at a broadcasting site where the image capturing device is used to capture the image, and that creates a return image related to the use of the image captured by the image capturing device in the main line image for broadcast; A switcher is provided in a broadcasting station connected to the broadcasting site via a network, and switches between a plurality of images including the images captured by the image capturing device and transmitted to the broadcasting station via the network to produce the main line image; The image creation device includes: A communication unit that acquires a control signal for the switcher via the network; a control unit that creates the return video based on the control signal.

7. A program for causing a computer to function as the image creation device according to any one of claims 1 to 5.

Images