Video synchronization device, video synchronization system, video synchronization method, and program

Abstract

To decentralize the timing of receiving imaging data from each of a plurality of other locations connected via a network.SOLUTION: A video synchronization device includes an acquisition unit that acquires first imaging data, which is imaging data captured by a first imaging device, and acquires from the first imaging data a first imaging signal that indicates the timing at which the first imaging device captures an image, a generation unit that uses the first imaging signal and the number of multiple second imaging devices to generate multiple second imaging signals that indicate the timing at which each of the multiple second imaging devices captures an image, and a transmission unit that transmits the multiple second imaging signals generated by the generation unit to each control device that controls the second imaging devices.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a video synchronization device, a video synchronization system, a video synchronization method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for synchronizing the vertical synchronization signals of the imaging system and the playback system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-060965 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not consider, for example, receiving video from each of a plurality of other locations connected via a network.

[0005] An object of the present invention is to provide a technique that can decentralize the timing of receiving imaging data from each of a plurality of other locations connected via a network. [Means for solving the problem]

[0006] In one aspect of the present invention, there is provided a video synchronization device having an acquisition unit that acquires first imaging data, which is imaging data captured by a first imaging device, and acquires from the first imaging data a first imaging signal that indicates the timing at which the first imaging device captures an image; a generation unit that uses the first imaging signal and the number of second imaging devices to generate a plurality of second imaging signals that indicate the timing at which each of the second imaging devices captures an image; and a transmission unit that transmits the plurality of second imaging signals generated by the generation unit to each control device that controls the second imaging devices. [Effects of the Invention]

[0007] According to the present invention, the timing of receiving imaging data from each of a plurality of other locations connected via a network can be decentralized. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of a video synchronization system according to an embodiment. [Figure 2] 1 is a diagram illustrating an example of the configuration of a video synchronization device and a control device according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer of a video synchronization device and a control device according to an embodiment. [Figure 4] FIG. 2 is a sequence diagram illustrating an example of processing of the video synchronization system according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating an example of image capturing timings by image capturing devices at locations other than the location KA and an example of reception timings at the location KA according to the embodiment. [Figure 6] 10A and 10B are diagrams illustrating an example of image capturing timings by image capturing devices at locations other than the location KA and an example of reception timings at the location KA according to the embodiment. [Figure 7] 10A and 10B are diagrams illustrating an example of image capturing timings by image capturing devices at locations other than the location KA and an example of reception timings at the location KA according to the embodiment. [Figure 8] FIG. 4 is a diagram illustrating an example of a display screen according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The principles of the present invention will be described with reference to several exemplary embodiments. It will be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the invention, without suggesting any limitation on the scope of the invention. The invention described herein may be implemented in a variety of ways other than those described below.

[0010] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <Configuration> The configuration of a video synchronization system 1 according to an embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram illustrating an example of the configuration of the video synchronization system 1 according to the embodiment. In the example of FIG. 1, the video synchronization system 1 includes a video synchronization device 10, an imaging device 20A, and a display device 30A, which are installed at a location KA. The video synchronization system 1 also includes a control device 40B, an imaging device 20B, and a display device 30B, which are installed at a location KB. The video synchronization system 1 also includes a control device 40C, an imaging device 20C, and a display device 30C, which are installed at a location KC. The video synchronization system 1 also includes a control device 40D, an imaging device 20D, and a display device 30D, which are installed at a location KD. The video synchronization system 1 also includes a control device 40E, an imaging device 20E, and a display device 30E, which are installed at a location KE.

[0012] Hereinafter, when there is no need to distinguish between them, each of the imaging devices 20A-E will also be simply referred to as "imaging device 20." Furthermore, when there is no need to distinguish between them, each of the display devices 30A-E will also be simply referred to as "display device 30." Furthermore, when there is no need to distinguish between them, each of the control devices 40B-E will also be simply referred to as "control device 40." The number of video synchronizers 10, imaging devices 20, display devices 30, and control devices 40 is not limited to the example of FIG.

[0013] 1, the video synchronizer 10 and the control device 40 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include the Internet, a mobile communication system, a local area network (LAN), a wireless LAN, short-range wireless communication such as Bluetooth (registered trademark) Low Energy (BLE), and a bus. Examples of the mobile communication system include a fifth-generation mobile communication system (5G), a fourth-generation mobile communication system (4G), a third-generation mobile communication system (3G), and the like.

[0014] The video synchronizer 10, imaging device 20A, and display device 30A installed at the site KA are connected by a video transmission cable such as HDMI (High-Definition Multimedia Interface) (registered trademark). The control device 40, imaging device 20, and display device 30 installed at the sites KB to KE are also connected by a video transmission cable such as HDMI.

[0015] The imaging device 20 is, for example, a video camera that captures video of a user at the own base, etc. The display device 30 is a display (monitor) that displays at least one of the video captured by the imaging device 20 at the own base and the video captured by one or more imaging devices 20 at other bases.

[0016] The location KA may be a location that monitors the video of each of the other locations KB to KE. In this case, for example, a user at the location KA who is a conductor of an orchestra can monitor the video of the performers at each of the remote locations KB to KE.

[0017] The video synchronizer 10 may encode video captured by the imaging device 20 at its own site and transmit packets of the encoded data to one or more control devices 40 at other sites via the network N. The video synchronizer 10 may also decode data received from one or more control devices 40 at other sites and display the decoded data on the display device 30 at its own site. This allows the video captured by each camera to be displayed in real time in each area on a single display screen. The video synchronizer 10 and the control device 40 may also display the video captured by the imaging device 20 at its own site in a specific area on the display screen of the display device 30 at its own site.

[0018] Based on the video signal received from the video synchronizer 10, the display device 30A displays the video captured by at least one of the imaging devices 20B to 20E (each imaging device 20 other than the imaging device 20A).

[0019] Each control device 40 at the locations KB to KE encodes the video captured by the imaging device 20 at its own location and transmits packets of the encoded data to the video synchronizer 10 via the network N. Furthermore, each control device 40 may transmit packets of the encoded data to a control device 40 at another location. The control device 40 decodes the data received from the video synchronizer 10 at the other site and displays it on the display device 30 at its own site. The control device 40 may also decode the data received from the control device 40 at the other site and display it on the display device 30 at its own site.

[0020] <Configuration> The configurations of the video synchronizer 10 and the control device 40 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configurations of the video synchronizer 10 and the control device 40 according to the embodiment. In the example of Fig. 2, the video synchronizer 10 has an acquisition unit 11, a reception unit 12, a display cycle acquisition unit 13, a generation unit 14, a transmission unit 15, and a display unit 16. Each of these units may be realized by cooperation between one or more programs installed in the video synchronizer 10 and hardware such as a processor and memory of the video synchronizer 10.

[0021] The acquisition unit 11 acquires first imaging data, which is imaging data captured by the imaging device 20A, and acquires a first imaging signal indicating the timing of imaging by the imaging device 20A from the imaging data. The first imaging signal is, for example, a vertical synchronization signal of the imaging device 20A. The first imaging signal may also be a signal for synchronizing the imaging timing of the imaging device 20A with the display timing of the display device 30A.

[0022] The receiving unit 12 receives second imaging data, which is data captured by each imaging device 20 (second imaging device) other than the imaging device 20A, from the control device 40 that controls each imaging device 20. The display cycle obtaining unit 13 obtains from the display device 30A a display cycle indicating a cycle for updating the display of the display device 30A that displays the imaging data of the second imaging device.

[0023] The generating unit 14 generates a plurality of second imaging signals indicating the timing at which each of the plurality of second imaging devices captures an image, using the first imaging signal acquired by the acquiring unit 11 and the number of the plurality of second imaging devices. The generating unit 14 may also acquire, from the second imaging data received by the receiving unit 12, the period at which the second imaging device captures the second imaging data. Specifically, the generating unit 14 may acquire, for example, the period of the vertical synchronization signal of the second imaging device included in the second imaging data. The generating unit 14 may also acquire information indicating the period at which the second imaging device captures the second imaging data from information set in advance in the video synchronizer 10 by an operator or the like. The generating unit 14 may also generate a second imaging signal indicating the timing at which the second imaging device captures an image, using the first imaging signal, the imaging period, and the display period. A specific method for generating the second imaging signal will be described later.

[0024] The transmitter 15 transmits the second imaging signal generated by the generator 14 to each control device 40 that controls the second imaging device. The display unit 16 displays an image based on the second imaging data received from another location on the display device 30A.

[0025] 2, the control device 40 includes a receiving unit 41, a control unit 42, and a transmitting unit 43. Each of these units may be realized by cooperation between one or more programs installed in the control device 40 and hardware such as a processor and memory of the control device 40.

[0026] The receiving unit 41 receives from the video synchronizer 10 a second imaging signal indicating the imaging timing of the imaging device 20 (for example, the imaging device 20B in the case of the control device 40B) that is the control target of the control device 40. The transmitting unit 43 encodes the imaging data captured by the imaging device 20 and transmits it to the video synchronizer 10 or another control device 40. The transmitting unit 43 also transmits the second imaging signal to the imaging device 20. The control unit 42 controls the transmitting unit to transmit the second imaging signal received by the receiving unit 41 to the imaging device 20.

[0027] <Hardware configuration> 3 is a diagram showing an example of the hardware configuration of a computer 100 of the video synchronizer 10 and the control device 40 according to the embodiment. In the example of FIG. 3, the computer 100 includes a processor 101, a memory 102, and a communication interface 103. These components may be connected by a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0028] When the program 104 is executed by the processor 101, memory 102, and the like in cooperation with each other, the computer 100 performs at least some processing of an embodiment of the present invention. The memory 102 may be of any type. As a non-limiting example, the memory 102 may be a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. Although only one memory 102 is shown in the computer 100, several physically different memory modules may exist in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture, as a non-limiting example. The computer 100 may have multiple processors, such as application-specific integrated circuit chips that are time-slaved to a clock that synchronizes the main processor.

[0029] Embodiments of the present invention may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0030] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present invention. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0031] The program code for executing the methods of the present invention may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0032] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0033] <Processing> Next, an example of the processing of the video synchronization system 1 according to the embodiment will be described with reference to Fig. 4 to Fig. 8. Fig. 4 is a sequence diagram showing an example of the processing of the video synchronization system 1 according to the embodiment. Figs. 5 to 7 are diagrams showing an example of the timing of imaging by the imaging device 20 at each site other than the site KA according to the embodiment, and the timing of reception at the site KA. Fig. 8 is a diagram showing an example of a display screen according to the embodiment. Note that the order of the following processes may be changed as appropriate within the scope of not causing any contradiction.

[0034] In step S101, the acquisition unit 11 of the video synchronizer 10 acquires first imaging data, which is imaging data captured by the imaging device 20A, and acquires a first imaging signal from the first imaging data, which indicates the timing at which the imaging device 20A captures an image. The first imaging signal is, for example, a vertical synchronizing signal (VSYNC) that indicates the timing at which the imaging device 20A captures one frame. Note that the imaging device 20A and the display device 30A may be vertically synchronized using the VSYNC, thereby matching the frame rate of the video output by the imaging device 20A with the display cycle (refresh rate) of the display device 30A.

[0035] Next, the receiving unit 12 of the video synchronizer 10 receives second imaging data (video frames), which are data captured by each imaging device 20 other than the imaging device 20A, from the control device 40 that controls each imaging device 20 (steps S102-1 to S102-3). Next, the display cycle acquiring unit 13 of the video synchronizer 10 acquires a display cycle indicating a cycle for updating the display of the display device 30A that displays the imaging data of each imaging device 20 other than the imaging device 20A (step S103). Here, the display cycle acquiring unit 13 may acquire information about the display cycle from, for example, the display device 30A. Alternatively, the display cycle acquiring unit 13 may acquire information about the display cycle from setting data that is set in advance in the video synchronizer 10 by, for example, an operator (administrator). Note that if the display cycle information is set in advance in the video synchronizer 10, step S103 may be omitted.

[0036] Next, the generation unit 14 of the video synchronization device 10 generates a second imaging signal indicating the timing at which the second imaging device captures the second imaging data, using the first imaging signal, the period at which the second imaging device captures the second imaging data, and the display period acquired by the display period acquisition unit 13 (step S104).

[0037] Here, the generation unit 14 may determine the imaging timing of each imaging device 20 other than the imaging device 20A so that the timing of receiving frames of video captured by the imaging devices 20 other than the imaging device 20A is dispersed. For example, by assigning a first imaging signal of the imaging device 20A, which is generated at a constant cycle, as a second imaging signal for each location, the timing of receiving frames of video captured by the imaging devices 20 other than the imaging device 20A is dispersed. This distributes the timing of receiving data at the location KA, for example, and can reduce pressure on the reception bandwidth at the location KA.

[0038] 5 shows an example of the imaging timing of each imaging device 20 according to the embodiment and the reception timing at the location KA. In the example of FIG. 5, the generation unit 14 sets the first imaging signal at time 500, which is one of the first imaging signals that the imaging device 20A always generates at regular intervals during imaging, as the second imaging signal 511, and sets frame f at the location KB between time 501 and time 502. kb The generation unit 14 also processes the first imaging signal at time 501 as a second imaging signal 512, and generates a frame f kc As a result, at the base KA, frame f kb The reception timing of frame f is between time 502 and time 503. kc The reception timing of the frames is between time 503 and time 504. Therefore, the timing of receiving frames of the images captured by the image capturing device 20B and the image capturing device 20C is dispersed. Furthermore, for each cycle in which the display device 30A displays the screen, the display screen can be updated using frames that have been received from either the site KB or the site KC.

[0039] A more detailed description will be given. Figures 5 to 7 show processing related to the transmission and reception of the second imaging signal and the transmission and reception of the captured frames, and the following processing is commonly performed. Here, a series of processing starting from the second imaging signal 511 in Figure 5 will be described as an example. The imaging device 20A at site KA generates the second imaging signal 511, and the transmitting unit 15 of the video synchronizer 10 transmits the second imaging signal 511 at time point 500. The receiving unit 41 of the control device 40 at site KB receives the second imaging signal 511 at time point 501. In other words, the transmission time from site KA to site KB is the length of one square in the example of Figure 5. The same applies when transmitting from site KA to site KC, and conversely, the transmission time when sites KB and KC transmit video frames to site KA is also the length of one square.

[0040] When the base KB receives the second image pickup signal 511, the base KB kb The frame processing mentioned here refers to three processes: imaging, compression, and transmission to another location. The second imaging signal 511 received by the receiving unit 41 is transmitted to the imaging device 20B by the transmitting unit 43. The imaging device 20B starts imaging in response to the second imaging signal 511, and the captured image is compressed and transmitted to the transmitting unit 43. The compression may be performed by the imaging device 20B or the control unit 42. The transmitting unit 43 kb The frame is transmitted to the base station KA. Note that the frame is transmitted, for example, by transmitting pixel data constituting the frame line by line.

[0041] To perform the above processing, a certain amount of time is required to process the frame. kb The processing of frame f starts at time 501, and at time 502 kb This indicates that all processing, including image capture, compression, and transmission, is complete.

[0042] At base KA, frame f kbThe image capture device 20A and the display device 30A at the location KA are synchronized with each other by a vertical synchronization signal (first image capture signal) of the image capture device 20A, so that the frame rate of the image output by the image capture device 20A matches the display cycle (refresh rate) of the display device 30A. That is, the frame f kb The display indicates that the process is completed between time 502 and time 503. The above process is performed in common in FIGS.

[0043] Returning to FIG. 4, the generation unit 14 may use the number of second imaging devices to allocate each first imaging signal of the imaging device 20A at the site KA to a second imaging device at a site other than the site KA, and generate the corresponding second imaging signal. In this case, if there are two sites other than the site KA, site KB and site KC, the generation unit 14 may alternately allocate each first imaging signal of the imaging device 20A to the imaging device 20B and the imaging device 20C, and generate the second imaging signal for the imaging device 20B and the second imaging signal for the imaging device 20C. Furthermore, if there are three or more sites other than the site KA, the generation unit 14 may allocate the first imaging signal in order to each imaging device other than the imaging device 20A. In the example of FIG. 5, for convenience, the frame f kb2、 Frame F kc2、 Although the description of frame f is omitted, since there are two imaging devices 20 at other locations, it is shown that the second imaging signal for each location is generated every two squares. That is, at time 502, a second imaging signal 513 is transmitted from location KA to location KB, and at time 503, a second imaging signal 514 is transmitted to location KC. In the example of FIG. 6, for convenience, frame f kb2、 Frame F kc2、 Frame F kd2Although the description of the second imaging signal for processing is omitted, since there are three imaging devices 20 at other locations, it is shown that the second imaging signal for each location is generated every three squares. In other words, at time 603, the second imaging signal 612 is transmitted from location KA to locations KB, KC, and KD. Note that the imaging devices 20 at each location other than location KA may capture images only when they receive the second imaging signal from location KA.

[0044] The generation unit 14 may further determine the timing of imaging by each imaging device 20 based on the communication time with each location. The communication time with each location may be measured in advance using, for example, ping. In this case, the generation unit 14 may determine the communication time as half the time elapsed from the time specific data is sent to the other party using an echo command of the Internet Control Message Protocol (ICMP) to the time a response is received from the other party. For example, in FIG. 5, the communication time from location KA to location KB is one square from time 500 to time 501, the communication time from location KA to location KC is two squares from time 500 to time 502, and the communication time from location KB to location KA and the communication time from location KC to location KA are both one square. In this case, the generation unit 14 may generate the first imaging signal at time 500 as the second imaging signal for location KB and location KC. In other words, the generation unit 14 generates each second imaging signal using the same first imaging signal. This distributes the timing of receiving data from each location.

[0045] FIG. 6 shows an example of the imaging timing of each imaging device 20 and the reception timing at location KA according to the embodiment. In the example of FIG. 6, there are three locations other than location KA, and the frame rates of the imaging devices 20 other than imaging device 20A are different, with the frame rates of the imaging devices 20 decreasing in the order of location KB, location KC, and location KD. In the example of FIG. 6, the generation unit 14 generates the second imaging signal so that the reception timing becomes later as the frame rate decreases. Specifically, the generation unit 14 generates the second imaging signal 611 for each location using the first imaging signal at time 600, and generates frame f at location KB between time 601 and time 602. kb1 , and frame f kc1 , and frame f kd1 is being processed.

[0046] As a result, at the base KA, frame f kb1 The reception timing of frame f is between time 602 and time 603. kc1 The reception timing of frame f is between time 603 and time 604. kd1 The reception timing of the frames is between time 604 and time 605. Therefore, the timing of receiving frames of the images captured by the image capturing device 20B, the image capturing device 20C, and the image capturing device 20D is dispersed. Furthermore, for each cycle in which the display device 30A displays the screen, the display screen can be updated using a frame that has been received from any one of the locations KB, KC, and KD.

[0047] FIG. 7 shows an example of the imaging timing of each imaging device 20 and the reception timing at the location KD according to the embodiment. In the example of FIG. 7, similar to the example of FIG. 6, the frame rates of the imaging devices 20 other than the imaging device 20A are different. In the example of FIG. 7, the generation unit 14 generates the second imaging signal so that the reception timing becomes later as the frame rate becomes higher. Specifically, the generation unit 14 generates the second imaging signal using the first imaging signal at time 700, and receives frame f at the location KD between time 701 and time 704.kd1 The generation unit 14 also generates a second imaging signal using the first imaging signal at time 702, and processes a frame f kc1 The generation unit 14 also generates a second imaging signal using the first imaging signal at time 704, and processes a frame f kb1 is being processed.

[0048] As a result, at the base KA, frame f kd1 The reception timing of frame f is between time 703 and time 704. kc1 The reception timing of frame f is between time 704 and time 705. kb1 The reception timing of the frames falls between time 705 and time 706. Therefore, similar to the example of Fig. 6, the timing of receiving frames of the images captured by the image capturing device 20B, the image capturing device 20C, and the image capturing device 20D is dispersed. Also, similar to the example of Fig. 6, the display screen can be updated using frames that have been received from any one of the locations KB, KC, and KD at each cycle in which the display device 30A displays the screen.

[0049] Note that, for example, when the communication time from the location KA to each of the other locations KB to KE differs, or when the frame rates of the imaging devices 20B to E at each of the locations KB to KE differ, the generation unit 14 may cause some or all of the imaging times of the imaging devices 20B to E to overlap in time. In this case, a first imaging signal capturing a certain frame out of the 240 frames per second of the imaging device 20A is used for synchronization with all of the imaging devices 20B to E. Alternatively, some of the first imaging signals out of the 240 frames per second of the imaging device 20A are used for synchronization with each of the imaging devices 20B to E.

[0050] The generation unit 14 may determine the frame rate of the video captured by each imaging device 20 other than the imaging device 20A based on video data from each imaging device 20 received via the network N. The generation unit 14 may also acquire information indicating the frame rate of the video captured by each imaging device 20 other than the imaging device 20A from the control device 40 at each location. The information indicating the frame rate of the video captured by each imaging device 20 other than the imaging device 20A may be set in advance in the video synchronizer 10 by an operator (administrator) or the like.

[0051] Next, the transmitting unit 15 of the video synchronizer 10 transmits the second imaging signal generated by the generating unit 14 to each control device 40 that controls each imaging device 20 other than the imaging device 20A (steps S105-1 to S105-3). As a result, the receiving unit 41 of each control device 40 receives, from the video synchronizer 10, information indicating the imaging timing of the imaging device 20 (e.g., imaging device 20B in the case of control device 40B) that is the control target of the control device 40. Note that the information indicating the imaging timing may be, for example, a part of each imaging data captured by the imaging device 20A. In this case, the transmitting unit 15 may transmit different imaging data from the imaging data captured by the imaging device 20A to the control devices 40 at each of the bases KB to E.

[0052] Next, the control unit 42 of each control device 40 controls the imaging timing of the imaging device 20 that is the control target of the control device 40, based on the information received by the receiving unit 41 (steps S106-1 to S106-3). Here, the control device 40 of each of the bases KB to KE may, for example, decode packets of imaging data captured by the imaging device 20A that have been encoded by the video synchronizer 10. Then, each control device 40 may control the imaging timing (vertical synchronization signal) of the imaging device 20 at its own base, based on the vertical synchronization signal (VSYNC) of the imaging data. This allows, for example, each of the bases KB to KE to synchronize with the base KA without using devices such as PTP (Precision Time Protocol) or GPS (Global Positioning System).

[0053] Next, the transmitting unit 43 of each control device 40 transmits packets obtained by encoding the imaging data captured by each imaging device 20 to the video synchronizer 10 (steps S107-1 to S107-3). Here, the length of time from when the imaging device 20 captures each piece of imaging data to when the imaging data is transmitted is approximately the same for each piece. Therefore, once the imaging timing of the imaging device 20 is specified in step S104, the timing at which each piece of imaging data is transmitted thereafter is approximately at a constant cycle. Therefore, the location KA can receive imaging data from each of the locations KB to KE at a distributed cycle.

[0054] Next, the display unit 16 of the video synchronizer 10 combines the imaging data captured by the imaging devices 20B-E at the locations KB-KE into regions on one display screen and displays the combined data on the display device 30A (steps S108-1-3).

[0055] An example of a display screen according to the embodiment is shown in Fig. 8. In the example of Fig. 8, on a display screen 801 of the display device 30A, an area 811 displays imaging data of the location KB, an area 812 displays imaging data of the location KC, an area 813 displays imaging data of the location KD, and an area 814 displays imaging data of the location KE.

[0056] When display unit 16 receives new imaging data from a certain location, it may update the image of the area corresponding to that location with the new imaging data, while maintaining the images of the areas corresponding to the other locations. In this case, by controlling the imaging timing of each imaging device 20, each of areas 811-814 is updated at a distributed timing for each specific period. For example, if the specific period is 60 fps (every 16.6 ms) and the frame rate of display device 30A is 240 fps, one of areas 811-814 is updated every 4.4 ms.

[0057] (Example of each location receiving video from multiple other locations) In the above example, each of the bases KB to KE transmits and receives video only to and from the base KA. However, the technology of the present disclosure is not limited to this, and can be applied to a case where each of the bases KB to KE transmits and receives video to and from a plurality of bases.

[0058] In this case, the video synchronizer 10 may specify the timing of imaging at each imaging device 20 by the same processes as in steps S101 to S105 in Fig. 4. As a result, similar to the above-mentioned example, at the site KA, the timing of receiving imaging data from each of the sites KB to KE is decentralized.

[0059] 4, each control device 40 may control the timing of imaging at each imaging device 20. Each control device 40 may transmit packets obtained by encoding imaging data captured by each imaging device 20 not only to the video synchronizer 10 but also to other control devices 40.

[0060] 4, each control device 40 may combine the imaging data captured by the imaging devices at the other locations into areas on a single display screen and display the combined data on the display device 30 at its own location. This allows, for example, the timing of receiving imaging data from the other locations to be further dispersed at locations KB to KE.

[0061] <Other> In recent years, remote music lessons and sessions have become common, and GPS and PTP are used to synchronize the video clock and Vsync between distant locations. However, using GPS and PTP requires specialized equipment, which increases costs and increases the size of the system.

[0062] Furthermore, when transmitting video data with low latency for remote lessons, etc., a line-based compression codec such as JPEG-XS can be used. However, while this achieves low latency, the image size increases. Therefore, when receiving video from multiple locations, such as in a remote orchestra, the receiving bandwidth may be constrained, resulting in delays.

[0063] On the other hand, according to the present disclosure, for example, in a remote two-way video communication system, it is possible to synchronize images even when a PTP or GPS device is not used. Also, when receiving video from multiple locations, it is possible to reduce overlapping of the timing of receiving images from each location without incurring delays due to waiting.

[0064] <Modification> The video synchronization device 10 and the control device 40 may each be devices contained in a single housing, but the video synchronization device 10 and the control device 40 of the present disclosure are not limited to this. The video synchronization device 10 and the control device 40 may each be realized by cloud computing consisting of one or more computers, for example. The video synchronization device 10 and the control device 40 may also be the same device. Such a video synchronization device 10 and control device 40 are also included as examples of the "video synchronization device" and "control device" of the present disclosure, respectively.

[0065] The present invention is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the invention. [Explanation of symbols]

[0066] 1. Video synchronization system 10 Video Synchronizer 11 Acquisition Department 12 Receiving section 13 Display cycle acquisition section 14 Generation part 15 Transmitter 16 Display section 20 Imaging device 30 Display device 40 Control device 41 Receiving unit 42 Control Unit 43 Transmitter

Claims

1. an acquisition unit that acquires first imaging data that is imaging data captured by a first imaging device, and acquires a first imaging signal that indicates timing at which the first imaging device captures an image from the first imaging data; a generation unit that generates a plurality of second imaging signals indicating timings at which each of the plurality of second imaging devices captures an image, using the first imaging signal and the number of the plurality of second imaging devices; a transmitter that transmits the plurality of second imaging signals generated by the generator to respective control devices that control the second imaging devices; A video synchronizer having:

2. a receiving unit that receives second imaging data, which is data captured by the second imaging device, from a control device that controls the second imaging device; the generation unit generates a plurality of second imaging signals indicating timings at which each of the plurality of second imaging devices captures an image, using the first imaging signal, a period at which the second imaging device captures the second imaging data, and the number of the second imaging devices.

2. The video synchronizer according to claim 1.

3. a video synchronizer and a control device, The video synchronizer comprises: an acquisition unit that acquires first imaging data that is imaging data captured by a first imaging device, and acquires a first imaging signal that indicates timing at which the first imaging device captures an image from the first imaging data; a generation unit that generates a plurality of second imaging signals indicating timings at which each of the plurality of second imaging devices captures an image, using the first imaging signal and the number of the plurality of second imaging devices; a transmitter that transmits the plurality of second imaging signals generated by the generator to respective control devices that control the second imaging devices; and The control device a receiving unit that receives the second imaging signal indicating imaging timing of the second imaging device; a control unit that controls an imaging timing of the second imaging device based on the second imaging signal, Video synchronization system.

4. acquiring first imaging data that is imaging data captured by a first imaging device, and acquiring a first imaging signal that indicates timing at which the first imaging device captures an image from the first imaging data; generating a plurality of second imaging signals indicating timings at which each of the plurality of second imaging devices captures an image using the first imaging signal and the number of the plurality of second imaging devices; transmitting the generated second imaging signals to respective control devices that control the second imaging devices; Video synchronization method.

5. acquiring first imaging data that is imaging data captured by a first imaging device, and acquiring a first imaging signal that indicates timing at which the first imaging device captures an image from the first imaging data; generating a plurality of second imaging signals indicating timings at which each of the plurality of second imaging devices captures an image using the first imaging signal and the number of the plurality of second imaging devices; transmitting the generated second imaging signals to respective control devices that control the second imaging devices; A program that causes a computer to perform a process.

Images