Communication device, server system, and control method and program for communication device

Abstract

To properly control output of synchronization signals when multiple clocks in a communication apparatus complete time synchronization at different timings.SOLUTION: A communication apparatus includes: a plurality of communication interfaces connected to a time server (server); a plurality of clocks associated with the communication interfaces, respectively; a plurality of output units associated with the clocks, respectively, to output synchronization signals; means for synchronizing the clocks with the server; means for checking whether the clocks are in synchronization with the server; and means for selecting, as a synchronization signal supplied to another apparatus, a synchronization signal output from an output unit associated with the synchronized clock, from among the synchronization signals output from the output units. Each output unit outputs a synchronization signal when clock time reaches a start time designated by an external apparatus, and outputs the synchronization signal when the clock time reaches the start time, even if the clock associated with the output unit is out of synchronization with the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication device, a server system, a control method for a communication device, and a program. [Background technology]

[0002] Recently, virtual viewpoint image generation systems have become popular. These systems synchronize multiple viewpoints captured by multiple cameras installed at different locations, and then synthesize virtual viewpoint content using the multiple viewpoint images obtained by the capture. These systems require highly accurate synchronization of the capture timing to synthesize high-quality virtual viewpoint images. Precision Time Protocol (PTP) is widely used to synchronize the time among multiple communication devices connected to a network.

[0003] In PTP, PTP slave devices communicate with a PTP GM device, which is a time server with an accurate clock, to synchronize time, and the clock attached to the network interface of the PTP slave device is synchronized with the clock of the time server. GM stands for Grand Master. In a virtual viewpoint image generation system, a synchronization signal is generated to synchronize multiple cameras based on the synchronized clocks of multiple communication devices.

[0004] Patent Document 1 discloses a method for reducing jitter in time information in a system that uses a time synchronization protocol such as PTP. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-191226 Summary of the Invention [Problem to be solved by the invention]

[0006] In a virtual viewpoint image generation system in which multiple communication devices are time-synchronized and virtual viewpoint content is synthesized using multiple viewpoint images via a network, time synchronization may become impossible due to failures in network devices, lines, etc. In particular, the probability of failures occurring may increase when building a large system with an increased number of communication devices or when installing the system in a vast location. In response to this, it is possible to have each communication device have multiple network interfaces and connect each network interface independently to a time server, which makes the system redundant and reduces the probability of failure. However, in this case, sufficient consideration has not been given to how to control the generation of synchronization signals based on multiple clocks within each communication device (based on which clock the synchronization signal should be generated). In particular, when the timing at which time synchronization is completed for multiple clocks differs, there is a problem in that the generation of the synchronization signal is delayed if time synchronization is waited for the completion of time synchronization for all clocks.

[0007] The problem to be solved by the present invention is to suitably control the output of a synchronization signal when the timings at which time synchronization of a plurality of clocks in a communication device is completed differ. [Means for solving the problem]

[0008] A communication device according to one aspect of the present invention comprises a plurality of communication interfaces connected to a time server, a plurality of clocks each associated with the communication interfaces, a plurality of output units each associated with the clocks and outputting a synchronization signal based on the associated clock, a synchronization means for synchronizing the plurality of clocks with the time server, a confirmation means for confirming whether the plurality of clocks are synchronized with the time server, and a selection means for selecting, from a plurality of synchronization signals output from the plurality of output units, a synchronization signal output from the output unit associated with the clock synchronized with the time server as a synchronization signal to be supplied to another device, wherein each of the plurality of output units outputs the synchronization signal when the time of the clock reaches a start time specified by an external device, and each of the plurality of output units outputs the synchronization signal when the time of the clock reaches the start time even if the clock associated with that output unit is not synchronized with the time server. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration and a functional configuration of a communication device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing an image communication system including a plurality of communication devices according to an embodiment. [Figure 3] 10 is a flowchart of a PPS signal selection process executed by a communication device according to an embodiment. [Figure 4] 6 is a flowchart showing the operation of the control device in FIG. 2 for transmitting a PPS signal output command to a plurality of communication devices. [Figure 5] 6 is a flowchart showing an operation performed by a PHC synchronization confirmation process of a communication device according to the first embodiment of the present invention when the communication device receives a synchronization state inquiry command from a control device. [Figure 6] 6 is a flowchart showing operations executed by a PPS signal control process of the communication device according to the first embodiment of the present invention when the communication device receives a PPS signal output command transmitted by the control device. [Figure 7]A time chart showing an example of the time of a time server, the time and PPS signal of one PHC, and the time and PPS signal of another PHC. [Figure 8] 10 is a flowchart showing operations executed by a PPS signal control process of a communication device according to a second embodiment of the present invention, which has received a PPS signal output command transmitted by a control device. [Figure 9] 10 is a flowchart showing an operation performed by a PHC synchronization confirmation process of a communication device according to a third embodiment of the present invention when the communication device receives a synchronization state inquiry command from a control device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments.

[0011] First embodiment 1 shows the hardware configuration and functional configuration of a communication device 10 according to a first embodiment of the present invention. The communication device 10 is a PTP slave that uses PTP to communicate with a PTP GM device, which is a time server, and synchronizes the times of multiple clocks inside the communication device 10. The communication device 10 is connected to a camera 28, and synchronizes the imaging timing of the camera 28 with the imaging timing of another camera (see FIG. 2) based on the synchronized time. The communication device 10 includes a system bus 11, a control unit 12, a storage device 13, a DRAM 15, a storage device 14, communication I / F units 16A and 16B, and a PPS signal switching unit 19. The control unit 12 is connected to the storage device 13, the DRAM 15, the storage device 14, the communication I / F units 16A and 16B, and the PPS signal switching unit 19 via the system bus 11.

[0012] The control unit 12 is configured with one or more processors such as a CPU or an MPU, and controls the entire communication device 10 by executing a computer program stored in the storage device 13. The CPU stands for Central Processing Unit, and the MPU stands for Micro Processing Unit. The CPU and the MPU function as a computer. The control unit 12 may control the entire communication device 10 in cooperation with a computer program stored in the storage device 13 or 14 and an OS (Operating System) 20. Alternatively, the control unit 12 may be equipped with multiple processors such as a multi-core processor, and the entire communication device 10 may be controlled by the multiple processors.

[0013] The storage device 13 is configured with one or more memories and stores computer programs for performing various operations described below, and various information such as communication parameters for wireless communication. The storage device 13 stores a program for booting a system program such as the OS 20 from the storage device 14. Storage media such as ROM, flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, non-volatile memory card, or DVD can be used as the storage device 13. ROM stands for Read Only Memory, CD stands for Compact Disc, and DVD stands for Digital Versatile Disc.

[0014] System programs such as an OS and application programs are stored in the storage device 14. The storage device 14 is composed of one or more memories. The storage device 14 may be a storage medium such as a ROM, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, or a DVD. The DRAM (Dynamic Random Access Memory) 15 is a memory used as a working memory. Instead of or in addition to the DRAM, a Static Random Access Memory (SRAM) or other RAM may be used.

[0015] The communication I / F (interface) units 16A and 16B are connectors connected to a local area network such as Ethernet (registered trademark), and perform wired communication conforming to the Ethernet standard, for example. Cables 25A and 25B (e.g., Ethernet cables) are connected to the communication I / F units 16A and 16B, respectively, and the communication device 10 communicates with the image distribution server system 30 (see FIG. 2) via both the communication I / F units 16A and 16B. Furthermore, communication I / F unit 16A is associated with PHC 17A and PPS signal generator 18A. For example, communication I / F unit 16A includes PHC 17A and PPS signal generator 18A. Similarly, communication I / F unit 16B is associated with PHC 17B and PPS signal generator 18B. For example, communication I / F unit 16B includes PHC 17B and PPS signal generator 18B. The PHC (PTP Hardware Clock) is a clock used in PTP. The communication device 10 has two internal clocks, namely, PHC 17A and 17B, and either PHC 17A or 17B is used as the reference clock of the communication device 10. PHC 17A and 17B are also used to record the sending and receiving times of PTP packets (the receiving time of a Sync packet and the sending time of a Delay Request packet). The sending and receiving times of PTP packets sent and received by communication I / F unit 16A (the receiving time of a Sync packet and the sending time of a Delay Request packet) are recorded by PHC 17A. The sending and receiving of PTP packets by communication I / F unit 16B is recorded by communication I / F unit 16B.

[0016] The PPS (Pulse Per Second) signal is a synchronization signal that synchronizes the imaging timing of the camera 28 with the imaging timing of another camera 28 (see FIG. 2). In other words, the PPS signal is used as a GenLock signal for the camera 28. The PPS signal generation unit 18A is a counter that generates a PPS signal from the time of the PHC 17A, and generates and outputs a PPS signal at a frequency specified at a time specified by the PPS signal control process 24. On the other hand, the PPS signal generation unit 18B is a counter that generates a PPS signal from the time of the PHC 17B, and generates and outputs a PPS signal at a frequency specified at a time specified by the PPS signal control process 24.

[0017] The control unit 12 has, as functional blocks, an OS 20, PTP processes 21A and 21B, a PHC synchronization confirmation process 22, a PPS signal selection process 23, and a PPS signal control process 24. These are executed by operating in accordance with computer programs stored in storage devices 13 and 14. The PTP process 21A uses the communication I / F unit 16A to communicate with a time synchronization server, which is a PTP GM, and adjusts the time of the PHC 17A according to a known PTP method. Specifically, the time synchronization server transmits a Sync packet including a transmission timestamp of the time synchronization server, which is received by the communication I / F unit 16A. The communication I / F unit 16A records the time (compliant with the PHC 17A) at which the Sync packet is received. The communication I / F unit 16A transmits a Delay Request packet including a transmission timestamp of the communication I / F unit 16A, which is received by the time synchronization server. The time synchronization server returns a Delay Response packet to the communication I / F unit 16A, notifying the time at which the Delay Request packet was received. The process 21A adjusts the PHC 17A based on the transmission timestamp of the Sync packet, the reception time of that packet at the communication I / F unit 16A, the transmission timestamp of the Delay Request packet, and the reception time of that packet at the time synchronization server. That is, the PTP process 21A adjusts the frequency of the PHC 17A to adjust the time of the PHC 17A and synchronize it with the time of the time synchronization server. Similarly, the PTP process 21B communicates with the time synchronization server using the communication I / F unit 16B, and adjusts the time of the PHC 17B according to a known PTP method. The PTP processes 21A and 21B are repeated periodically to periodically synchronize the times of the PHCs 17A and 17B with the time synchronization servers 34A and 34B.

[0018] The PHC synchronization confirmation process 22 operates in response to a synchronization status inquiry command received by the communication I / F units 16A and 16B via the network from the image distribution server system 30 (see FIG. 2). In response to this inquiry command, the PHC synchronization confirmation process 22 inquires of the PTP processes 21A and 21B about the synchronization status of the PHCs 17A and 17B, and returns to the image distribution server system 30 whether the PHCs 17A and 17B are synchronized.

[0019] The PPS signal selection process 23 inquires about the synchronization state of the PTP processes 21A and 21B, and, depending on the synchronization state, selects the PPS signal generation unit 18A or 18B as the output source of the PPS signal for synchronizing the camera 28, and issues a command to the PPS signal switching unit 19. Specifically, the PPS signal selection process 23 selects the PPS signal generation unit 18A or 18B corresponding to the PHC 17A or 17B synchronized with the time synchronization server. The PPS signal switching unit 19 is connected to the PPS signal generators 18A and 18B and the camera 28. In accordance with a command from the PPS signal selection process 23, the PPS signal switching unit 19 switches the PPS signal output for synchronizing the camera 28 between the output from the PPS signal generator 18A and the output from the PPS signal generator 18B. The PPS signal switching unit 19 is, for example, an electronic switch. The PPS signal from the PPS signal generator 18A or 18B selected by the PPS signal switching unit 19 is supplied to the camera 28 via a PPS signal line 26.

[0020] The PPS signal control process 24 instructs the PPS signal generators 17A and 17B to output a PPS signal at a specified time and to stop outputting a PPS signal, in accordance with instructions from the image distribution server system 30 (see FIG. 2).

[0021] The camera 28 captures images based on the PPS signal supplied from the PPS signal switching unit 19. The camera 28 can capture images without using a PPS signal, but when capturing images synchronously with other cameras 28, the camera 28 uses the PPS signal to establish time synchronization with the other cameras 28. The camera 28 may capture either moving or still images and may record audio along with the images. The images captured by the camera 28 may be stored in storage (e.g., storage device 14 or 15) built into the camera 28 or the communication device 10, in an image distribution server system 30 connected to the communication device 10, or in cloud storage. Virtual viewpoint content can be synthesized from multi-viewpoint images captured synchronously and accumulated by multiple cameras 28. However, the images captured by the camera 28 may be used to synthesize virtual viewpoint content in real time without being stored in storage.

[0022] 2 is a block diagram showing an image communication system having a plurality of communication devices 10 according to an embodiment. The communication system has a plurality of communication devices 10, namely, communication devices (10a to 10r). Cameras 28a to 28r are connected to the communication devices 10a to 10r, respectively, via PPS signal lines 26. The communication I / F unit 16A of each communication device 10 is connected to a switching hub 32A of the image distribution server system 30 via a cable 25A, and the communication I / F unit 16B is connected to a switching hub 32B via a cable 25B. Therefore, each communication device 10 can transmit image data captured by a camera 28 connected to the communication device 10 itself to the image distribution server system 30. If necessary, each communication device 10 may transmit image data captured by a camera 28 connected to the communication device 10 itself to another communication device 10.

[0023] The image communication system includes an image distribution server system 30. The image distribution server system 30 includes an image distribution server 31, switching hubs 32A and 32B, a control device 33, and time synchronization servers 34A and 34B. The image distribution server 31 stores and processes image data received from the communication devices 10a to 10r via the switching hubs 32A and 32B. The image distribution server 31 also receives a request from any of the end user devices 42 via the Internet 40, synthesizes virtual viewpoint content from the stored image data in response to the request, and transmits the virtual viewpoint content to the end user device 42 that originated the request.

[0024] The control device 33 controls the image distribution server 201. The control device 33 also instructs each communication device 10, via switching hubs 32A and 32B, to output and stop outputting a PPS signal for synchronizing each camera 28, and to transmit and stop transmitting camera images to the image distribution server system 30. The control unit 33 is composed of one or more processors such as a CPU or MPU, and performs these processes by executing a computer program stored in a storage device (not shown).

[0025] Each of the time synchronization servers 34A, 34B is a PTP GM. The time synchronization servers 34A, 34B multicast PTP Sync packets, receive Delay Request packets from the communication devices 10 that are PTP slaves, and transmit Delay Response packets. As a result, each communication device 10 synchronizes the time of the PHCs 17A, 17B. Specifically, the time synchronization server 34A transmits Sync packets to the communication I / F units 16A of each communication device 10, receives Delay Request packets from the communication I / F units 16A, and transmits Delay Response packets to the communication I / F units 16A. A switching hub 32A is connected to the time synchronization server 34A, and the time synchronization server 34A transmits and receives PTP packets via the switching hub 32A. The time synchronization server 34B transmits Sync packets to the communication I / F unit 16B of each communication device 10, receives Delay Request packets from the communication I / F unit 16B, and transmits Delay Response packets to the communication I / F unit 16B. A switching hub 32B is connected to the time synchronization server 34B, and the time synchronization server 34B transmits and receives PTP packets via the switching hub 32B. Each of the time synchronization servers 34A and 34B synchronizes its own time using time information distributed by a GPS (Global Positioning System), so that the times of the time synchronization servers 34A and 34B are synchronized with each other.

[0026] Next, the operation of the PPS signal selection process 23 of the communication device 10 will be described with reference to FIG. First, the PHC 17A is provisionally set in the PPS signal selection flag (S1), and the PPS signal switching unit 19 is set so as to supply the PPS signal of the PPS signal generating unit 18A to the camera 28 (S2).

[0027] Next, the PPS signal selection process 23 uses the PHC synchronization confirmation process 22 to confirm the synchronization status between the time synchronization server 34A and the PHC 17A from the PTP process 21A, and to confirm the synchronization status between the time synchronization server 34B and the PHC 17B from the PTP process 21B (S3).Then, the PPS signal selection process 23 checks the PPS signal selection flag (S4). The case where the PPS signal selection flag is set to PHC 17A in S4 will be described. If the synchronization status is confirmed in S3 and the PHC 17A is synchronized with the time synchronization server 34A, and the determination result in S5 is affirmative, the PPS signal selection process 23 returns to S3. This is because there is no need to switch the PPS signal supplied from the PPS signal switching unit 19 to the camera 28. On the other hand, if the PHC 17A is not synchronized and the determination result in S5 is negative, the PPS signal selection process 23 checks whether the PHC 17B is synchronized with the time synchronization server 34B (S6). If the PHC 17B is not synchronized with the time synchronization server 34B (if the determination result in S6 is negative), neither the PHCs 17A nor 17B are synchronized. In this case, the PPS signal switching unit 19 is not switched, and the PPS signal selection process 23 returns to S3. As a result, the PPS signal from the PPS signal generation unit 18A continues to be supplied to the camera 28. If PHC 17B is synchronized and the determination result in S6 is affirmative, PPS signal selection process 23 assigns PHC 17B to the PPS signal selection flag (S7), thereby setting PPS signal switching unit 19 to supply the PPS signal from PPS signal generation unit 18B to camera 28 (S8). Then, PPS signal selection process 23 returns to S3.

[0028] The case where the PPS signal selection flag is set to PHC 17B in S4 will be described. If the synchronization status is confirmed in S3 and PHC 17B is synchronized with the time synchronization server 34B, and the determination result in S9 is affirmative, the PPS signal selection process 23 returns to S3. This is because there is no need to switch the PPS signal supplied from the PPS signal switching unit 19 to the camera 28. On the other hand, if PHC 17B is not synchronized and the determination result in S9 is negative, the PPS signal selection process 23 checks whether PHC 17A is synchronized with the time synchronization server 34A (S10). If PHC 17A is not synchronized with the time synchronization server 34A (if the determination result in S10 is negative), neither PHC 17B nor PHC 17A is synchronized. In this case, the PPS signal switching unit 19 is not switched, and the PPS signal selection process 23 returns to S3. As a result, the PPS signal from PPS signal generation unit 18B continues to be supplied to the camera 28. If PHC 17A is synchronized and the determination result in S10 is affirmative, PPS signal selection process 23 assigns PHC 17A to the PPS signal selection flag (S11), thereby setting PPS signal switching unit 19 to supply the PPS signal from PPS signal generation unit 18A to camera 28 (S12). Then, PPS signal selection process 23 returns to S3.

[0029] As described above, the PPS signal selection process 23 selects a PPS signal based on a PHC synchronized with the time synchronization server, and switches the PPS signal switching unit 19. Therefore, a PPS signal synchronized with the time synchronization server can be output from the PPS signal line 26. If a device (such as the time synchronization servers 34A and 34B or the hubs 32A and 32B) in the image distribution server system 30 fails, the PPS signal selection process 23 selects a PPS signal generator corresponding to a PHC synchronized with one of the time synchronization servers. Therefore, even if one of the PHCs becomes out of synchronization, a PPS signal synchronized with one of the time synchronization servers can be supplied to the camera 28. When the PPS signal generators 18A and 18B are outputting PPS signals but neither of the PHCs 17A and 17B is synchronized, the PPS signal generated by the PHC in use can be supplied to the camera 28 by not switching the PPS signal switcher 19. In this case, there is a risk that the imaging timing of the camera 28 will be out of sync with the imaging timing of the other cameras 28. However, the PTP processes 21A and 21B periodically synchronize the time of the PHCs 17A and 17B with the time synchronization servers 34A and 34B, so that the PPS signal supplied to the camera 28 will eventually be synchronized with the time synchronization servers 34A and 34B. Furthermore, the operations in FIGS. 4 and 5, which will be described later, prevent the occurrence of a situation in which neither of the PHCs 17A and 17B is synchronized.

[0030] Fig. 4 is a flowchart showing the operation of the control device 33 in Fig. 2 for transmitting a PPS signal output command to the communication devices 10a to 10r in order to synchronize the cameras 28a to 28r. In response to this PPS signal output command, the communication devices 10a to 10r output PPS signals to the corresponding cameras 28a to 28r, thereby synchronizing the cameras 28a to 28r. First, the image distribution server system 30 multicasts a synchronization status inquiry command to the communication devices 10a to 10r via the network (S21). Upon receiving the synchronization status inquiry command, each communication device 10 operates as described below in accordance with the flowchart of FIG. 5 and replies to the image distribution server system 30 about whether the communication device 10 itself is synchronized. Here, "synchronized" means that at least one of the PHCs 17A and 17B of the communication device 10 is synchronized with the time synchronization servers 34A and / or 34B. "Synchronized" also means that the communication device 10 is "in a state where it can output a PPS signal."

[0031] Next, the control device 33 refers to the replies from the communication devices 10a to 10r and determines whether all of the communication devices 10a to 10r are synchronized (S22). If any of the communication devices 10a to 10r are not synchronized and the determination result in S22 is negative, the control device 33 waits for a predetermined period (for example, one second) (S23), returns to S21, and transmits the synchronization status inquiry command again.

[0032] If the communication devices 10a to 10r are all synchronized and the determination result in S22 is affirmative, the control device 33 acquires the current time of the time synchronization server 34A or 34B from the server (S24). Next, the control device 33 determines the time at which the communication devices 10a to 10r should output a PPS signal from the obtained current time (S25). The time at which the PPS signal should be output is a time after the current time that takes into account the time at which the communication devices 10a to 10r will receive a PPS signal output command via the network (for example, 10 seconds after the current time). The control device 33 then transmits the PPS signal output command to the communication devices 10a to 10r (S26). The PPS signal output command specifies the time at which the communication devices 10a to 10r should start outputting a PPS signal and the frequency of the PPS signal.

[0033] FIG. 5 is a flowchart showing the operation performed by the PHC synchronization confirmation process 22 of the communication device 10 upon receiving a synchronization state inquiry command from the control device 33. This operation starts when the communication device 10 receives a synchronization status inquiry command from the control device 33 at the communication I / F unit 16A and / or 16B.

[0034] First, the PHC synchronization confirmation process 22 inquires of the PTP process 21A about the synchronization status of the PHC 17A (S31). If the PHC 17A is synchronized with the time synchronization server 34A and the determination result in S32 is affirmative, the PHC synchronization confirmation process 22 replies to the control device 33 that "the communication device 10 is synchronized" (S36). If the PHC 17A is not synchronized with the time synchronization server 34A and the determination result in S32 is negative, the PHC synchronization confirmation process 22 inquires of the PTP process 21B about the synchronization status of the PHC 17B (S33). If the PHC 17B is synchronized with the time synchronization server 34B and the determination result in S34 is positive, the PHC synchronization confirmation process 22 replies to the control device 33 that "the communication device 10 is synchronized" (S36). If the determination result in S34 is negative, the PHC synchronization confirmation process 22 replies to the control device 33 that "the communication device 10 is not synchronized" (S35).

[0035] In PTP, synchronization (time adjustment) of a PHC with a time synchronization server (a PTP GM) is performed by frequency adjustment of the PHC, as long as the PHC's time is not significantly out of sync with the time synchronization server's time. In this case, the PHC's time will not jump or regress. Also, although the crystal installed in the PHC tends to gain or lose time, frequency adjustment can reduce the time discrepancy of the PHC, even during periods when time information is not being received from the time synchronization server. However, in time synchronization by frequency adjustment, the time it takes for the PHCs to synchronize varies depending on factors such as the accuracy of the crystals installed in the PHCs and the network conditions. As shown in Fig. 5, if either PHC 17A or 17B is in a synchronized state, it replies "communication device 10 is synchronized." If the PHC that took the shortest time to synchronize synchronizes, it can reply "communication device 10 is synchronized" to control device 33. This shortens the time required for the loop of S21, S22, and S23 in Fig. 4, and enables communication devices 10a to 10r to start outputting PPS signals earlier.

[0036] FIG. 6 is a flowchart showing the operation executed by the PPS signal control process 24 of the communication device 10 upon receiving a PPS signal output command transmitted by the control device 33. This operation starts when the communication device 10 receives a PPS signal output command (S26 in FIG. 4) from the control device 33 at the communication I / F unit 16A and / or 16B. Upon receiving the PPS signal output command, the PPS signal control process 24 instructs the PPS signal generation unit 18A to start outputting a PPS signal at the specified frequency at the start time specified in the PPS signal output command (S41). When the time on the PHC 17A reaches the specified start time, the PPS signal generation unit 18A outputs a PPS signal at the specified frequency. Similarly, the PPS signal control process 24 instructs the PPS signal generation unit 18B to start outputting a PPS signal at the specified frequency at the start time specified in the PPS signal output command (S42). When the time on the PHC 17B reaches the specified start time, the PPS signal generation unit 18B starts outputting a PPS signal at the specified frequency.

[0037] As shown in FIG. 5, even if PHC 17A or 17B is not synchronized, PHC synchronization confirmation process 22 replies to control device 33 that "communication device 10 is synchronized" as long as the other PHC 17A or 17B is synchronized. Also, as shown in FIG. 4, upon receiving a response from communication device 10a to 10r indicating that "communication device 10 is synchronized," control device 33 transmits a PPS signal output command to communication device 10a to 10r. Then, as shown in FIG. 6, PPS signal control process 24 of communication device 10 instructs both PPS signal generators 18A and 18B to output PPS signals. Therefore, each of PPS signal generators 18A and 18B outputs a synchronization signal when the PHC's time reaches the designated start time, even if the PHC associated with the PPS signal generator is not synchronized with the time synchronization server. When PPS signal generators 18A and 18B start outputting PPS signals, even if PHC 17A or 17B is not synchronized, PPS signal selection process 23 performs the process shown in Fig. 3 and selects the PPS signal corresponding to the synchronized PHC. Therefore, a synchronized PPS signal is supplied from PPS signal line 26 to camera 28.

[0038] Upon receiving a PPS signal output command from the control device 33, the PPS signal generators 18A and 18B of each communication device 10 begin outputting PPS signals at the same time based on the time of the PHCs 17A and 17B, even if the PHCs 17A and 17B are not synchronized. The PTP processes 21A and 21B periodically synchronize the time of the PHCs 17A and 17B with the time synchronization servers 34A and 34B, so that the time of the PHCs 17A and 17B will eventually be synchronized with the time synchronization servers 34A and 34B. In other words, the frequencies and phases of the PPS signals output from the two PPS signal generators 18A and 18B will eventually match. The PPS signal selection process 23 supplies one of the PPS signals to the camera 28, but after the frequencies and phases of the PPS signals output from the two PPS signal generators 18A and 18B match, either PPS signal can be supplied to the camera 28.

[0039] A specific example will be described using Fig. 7. Fig. 7 shows examples of the time of the time server (PTP GM), the time of the PHC 17A, the PPS signal output by the PPS signal generation unit 18A, the time of the PHC 17B, and the PPS signal output by the PPS signal generation unit 18B. 7, PHC 17A synchronizes with the PTP GM at 10:11:00, and PHC 17B synchronizes with the PTP GM at 10:11:18. PPS signal generation unit 18A starts outputting a PPS signal at 10:11:02, when PHC 17A is already synchronized with the PTP GM. PPS signal generation unit 18B also starts outputting a PPS signal at the same time, but because PHC 17B is not synchronized with the PTP GM, the output of the PPS signal actually starts at a different time. However, after 10:11:18, when PHC 17B synchronizes with the PTP GM, the phase and frequency of the PPS signal output from PPS signal generation unit 18A and the phase of the PPS signal from PPS signal generation unit 18B are aligned with the frequency. If the communication device 10 is configured so that the PPS signal generators 18A and 18B output PPS signals when both PHCs 17A and 17B are synchronized, the control device 33 must specify the output times of the two PPS signals, which are synchronized to different times. The PPS signals used in capturing moving images have frequencies of 25 Hz and 29.97 Hz, but 29.97 Hz is an odd number, making it difficult to calculate the specified time. Starting the output of the PPS signals at the same time based on the time of each PHC is advantageous because it simplifies the calculation and specification of the specified time.

[0040] Second embodiment The hardware configuration and functional configuration of the communication device 10 according to the second embodiment of the present invention, as well as the configuration of the image communication system, are the same as those of the first embodiment described with reference to FIGS. 8 shows the operation executed by the PPS signal control process 24 of the communication device 10 according to the second embodiment when it receives a PPS signal output command transmitted by the control device 30. The PPS signal control process 24 (FIG. 6) of the first embodiment instructs both PPS signal generators 18A and 18B to start outputting PPS signals at the time specified in the PPS signal output command, even if the PHC 17A or 17B is not synchronized. Therefore, if the current time in the out-of-synchronization PHC 17A or 17B is later than the specified PPS signal output start time, the PPS signal generator 18A or 18B corresponding to that PHC cannot output a PPS signal. The PPS signal control process 24 according to the second embodiment can solve this problem.

[0041] The operation of FIG. 8 starts when the communication device 10 receives a PPS signal output command (S26 in FIG. 4) from the control device 33 at the communication I / F unit 16A and / or 16B. Upon receiving the PPS signal output command, the PPS signal control process 24 uses the PHC synchronization confirmation process 22 to inquire of the PTP processes 21A and 21B about the synchronization state of the PHCs 17A and 17B (S51). If the PHC 17A is synchronized with the time synchronization server 34A and the determination result in S52 is affirmative, the PPS signal control process 24 further checks whether the PHC 17B is synchronized with the time synchronization server 34B (S53). If both the PHCs 17A and 17B are synchronized and the determination result in S53 is affirmative, the PPS signal control process 24 proceeds to S41 and then to S42. S41 and S42 are as described in relation to FIG. 6. Therefore, when both PHCs 17A and 17B are synchronized, both PPS signal generators 18A and 18B start outputting PPS signals at the specified frequency at the time specified by the PPS signal output command.

[0042] If the PHC 17B is not synchronized and the determination result in S53 is negative, the PPS signal control process 24 acquires the current time of the PHC 17B (S54). Then, the current time of the PHC 17B is compared with the output start time of the PPS signal specified in the PPS signal output command (S55). If the current time of the PHC 17B is after the output start time of the PPS signal specified in the PPS signal output command and the determination result in S55 is positive, the PPS signal control process 24 proceeds to S56. In S56, the PPS signal control process 24 sets the current time of the PHC 17B to a time earlier than the PPS signal output start time specified in the PPS signal output command. The PPS signal control process 24 then proceeds to S41 and then to S42. Therefore, the PPS signal generation unit 18B associated with the PHC 17B that is not synchronized with the time synchronization server 34B outputs a PPS signal when the time of the PHC 17B, for which the current time has been set as described above, reaches the output start time. In other words, if the current time of the PHC 17B is at or has passed the PPS signal output start time, the current time of the PHC 17B is set back so that the PPS signal generation unit 18B can output a PPS signal. If the determination result in S55 is negative (the current time of the PHC 17B is before the output start time of the PPS signal), the PPS signal control process 24 proceeds to S41 and then to S42.

[0043] If the PHC 17A is not synchronized in S52 (if the determination result in S52 is negative), the PPS signal control process 24 acquires the current time of the PHC 17A (S57). Then, the current time of the PHC 17A is compared with the output start time of the PPS signal specified in the PPS signal output command (S58). If the current time of the PHC 17A is after the output start time of the PPS signal specified in the PPS signal output command and the determination result in S58 is positive, the PPS signal control process 24 proceeds to S59. In S59, the PPS signal control process 24 sets the current time of the PHC 17A to a time earlier than the PPS signal output start time specified in the PPS signal output command. After this, the PPS signal control process 24 proceeds to S41 and then to S42. Therefore, the PPS signal generation unit 18A associated with the PHC 17A that is not synchronized with the time synchronization server 34A outputs a PPS signal when the time of the PHC 17A, for which the current time has been set as described above, reaches the output start time. In other words, if the current time of the PHC 17A is at or has passed the PPS signal output start time, the current time of the PHC 17A is set back so that the PPS signal generation unit 18A can output a PPS signal. If the determination result in S58 is negative (the current time of the PHC 17A is before the output start time of the PPS signal), the PPS signal control process 24 proceeds to S41 and then to S42.

[0044] In this embodiment, even if the PPS signal output command specifies a PPS signal output start time that is earlier than the current time of the unsynchronized PHC 17A or 17B, it is possible to start outputting PPS signals from both PPS signal generation units 18A and 18B.

[0045] Third embodiment The hardware configuration and functional configuration of the communication device 10 according to the third embodiment of the present invention, as well as the configuration of the image communication system, are the same as those of the first embodiment described with reference to FIGS. FIG. 9 shows the operation executed by the PHC synchronization confirmation process 22 of the communication device 10 according to the third embodiment when the control device 30 has received a synchronization state inquiry command. In the third embodiment, in the PHC synchronization confirmation process 22, if PHC 17A or 17B is not synchronized with time synchronization server 34A or 34B, the current time of the unsynchronized PHC is set to the same time as or earlier than the current time of the synchronized PHC.

[0046] The operation of FIG. 9 starts when the communication device 10 receives a synchronization state inquiry command from the control device 33 at the communication I / F unit 16A and / or 16B. S31 to S36 are the same as in FIG. However, if the determination result in S32 or S34 is affirmative, that is, if at least one of the PHCs 17A and 17B is synchronized with the time synchronization server, the PHC synchronization confirmation process 22 proceeds to S61. In S61, it is checked which PHCs are synchronized. If both the PHCs 17A and 17B are synchronized with the time synchronization servers 34A and 34B, the PHC synchronization confirmation process 22 replies to the control device 33 that "the communication device 10 is synchronized" (S36).

[0047] If only PHC 17A is synchronized, the PHC synchronization confirmation process 22 proceeds to S62. In S62, the PHC synchronization confirmation process 22 acquires the current times of PHCs 17A and 17B. Then, in S63, the current time of PHC 17A is compared with the current time of PHC 17B. If the current time of PHC 17B is ahead of the current time of PHC 17A and the determination result in S63 is positive, the PHC synchronization confirmation process 22 sets the current time of PHC 17B to a time that is the same as or earlier than the current time of PHC 17A (S64). If the current time of PHC 17B is not ahead of the current time of PHC 17A and the determination result in S63 is negative, the PHC synchronization confirmation process 22 proceeds to S36. If only PHC 17B is synchronized, the PHC synchronization confirmation process 22 proceeds to S65. In S65, the PHC synchronization confirmation process 22 acquires the current times of PHCs 17A and 17B. Then, in S66, the current time of PHC 17B is compared with the current time of PHC 17A. If the current time of PHC 17A is ahead of the current time of PHC 17B and the determination result in S66 is positive, the PHC synchronization confirmation process 22 sets the current time of PHC 17A to a time that is the same as or earlier than the current time of PHC 17B (S67). If the current time of PHC 17A is not ahead of the current time of PHC 17B and the determination result in S66 is negative, the PHC synchronization confirmation process 22 proceeds to S36.

[0048] In this way, in the PHC synchronization confirmation process 22, if the PHC 17A or 17B is not synchronized with the time synchronization server 34A or 34B, the current time of the unsynchronized PHC is set to the same time as or earlier than the current time of the synchronized PHC. Therefore, when the control device 33 sends a PPS signal output command (S26 in FIG. 4), it is possible to prevent the current time of the unsynchronized PHC from being behind the PPS signal output start time specified in the PPS signal output command. Therefore, in the communication device 10 that receives the PPS signal output command, by executing the PPS signal control process 24 (FIG. 6) of the first embodiment, both PPS signal generation units 18A and 18B can output PPS signals.

[0049] Other embodiments Although the embodiments of the present invention have been described above, the above description does not limit the present invention, and various modifications including deletion, addition, and substitution of components are possible within the technical scope of the present invention.

[0050] For example, in the above embodiment, the image distribution server system 30 has two time synchronization servers 34A and 34B. The time synchronization servers 34A and 34B are connected to the communication I / F units 16A and 16B of the communication devices 10a to 10r via switching hubs 32A and 32B and cables 25A and 25B, respectively. Therefore, even if one of the two time synchronization servers 34A and 34B fails, the PPS signal generation unit corresponding to the other time synchronization server outputs a PPS signal and supplies it to the camera 28. However, the image distribution server system 30 may have a single time synchronization server, and the time synchronization server may be connected to the communication I / F units 16A and 16B of the communication devices 10a to 10r via the switching hubs 32A and 32B and cables 25A and 25B. In the above embodiment, the communication device 10 has two communication I / F units 16A and 16B, two PHCs 17A and 17B, and two PPS signal generators 18A and 18B. However, the communication device 10 may have three or more communication I / F units, three or more PHCs, and three or more PPS signal generators, and the image distribution server system 30 may also have three or more switching hubs.

[0051] In the above embodiment, the PPS signal is used as a synchronization signal for multiple cameras 28 that capture multiple viewpoint images in a virtual viewpoint image generation system that synthesizes virtual viewpoint content. However, the PPS signal may also be used as a synchronization signal for multiple cameras 28 in a system that switches between images captured by multiple cameras 28. The PPS signal may also be used as a synchronization signal for multiple audio recording devices, video recording devices, or other devices.

[0052] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a recording medium, and having one or more processors in the computer of the system or device read and execute the program. In this case, the program (program code) itself read from the recording medium realizes the functions of the embodiments. Furthermore, the recording medium on which the program is recorded can constitute the present invention. Furthermore, based on the instructions of the program read by the computer, an operating system (OS) running on the computer may perform some or all of the actual processing, and the functions of the above-mentioned embodiments may be realized through that processing. At least some of the functional blocks shown in Figure 1 may be implemented by hardware. When implementing by hardware, for example, a specific compiler may be used to automatically generate a dedicated circuit on an FPGA from a program for implementing each step. Alternatively, a gate array circuit may be formed in the same way as an FPGA and implemented as hardware. Alternatively, it may be implemented by an ASIC. FPGA stands for Field-Programmable Gate Array. ASIC stands for Application Specific Integrated Circuit. [Explanation of symbols]

[0053] 10. Communications equipment 12 Control Unit 16A,16B Communication I / F section 17A, 17BPHC 18A,18B PPS signal generation section 21A, 21B PTP process 22 PHC Synchronization Confirmation Process 23 PPS signal selection process 28 Camera

Claims

1. a plurality of communication interfaces connected to a time server; a plurality of clocks each associated with said communication interface; a plurality of output units each associated with a corresponding one of the clocks, and each output unit outputs a synchronization signal based on the associated clock; a synchronization means for synchronizing the plurality of clocks with the time server; a checking means for checking whether the plurality of clocks are synchronized with the time server; a selection means for selecting, from among the plurality of synchronization signals output from the plurality of output units, a synchronization signal output from the output unit associated with the clock synchronized with the time server as a synchronization signal to be supplied to another device; and each of the plurality of output units outputs the synchronization signal when the time of the clock reaches a start time designated by an external device; Each of the plurality of output units outputs the synchronization signal when the time of the clock associated with the output unit reaches the start time, even if the clock associated with the output unit is not synchronized with the time server. A communication device comprising:

2. a comparison means for comparing the start time designated by the external device with the current time of the clock that is not synchronized with the time server; a setting unit that sets the current time of the clock that is not synchronized with the time server to a time that is earlier than the start time when the current time is later than the start time, The output unit associated with the clock that is not synchronized with the time server outputs the synchronization signal when the time of the clock to which the current time is set by the setting means reaches the start time.

2. The communication device according to claim 1.

3. The confirmation means, in response to an inquiry from an external device, confirms whether the plurality of clocks are synchronized with the time server, and if at least one of the plurality of clocks is synchronized with the time server, returns a message to the external device indicating that the communication device is synchronized.

3. The communication device according to claim 1 or 2.

4. The confirmation means, in response to an inquiry from an external device, confirms whether the plurality of clocks are synchronized with the time server, and if at least one of the plurality of clocks is synchronized with the time server, replies to the external device that the communication device is synchronized, and sets the current time of the clock not synchronized with the time server to the same time as or earlier than the current time of the clock synchronized with the time server.

4. The communication device according to claim 1, wherein the first and second communication terminals are connected to each other.

5. 5. A server system that operates as the external device communicating with a plurality of communication devices according to claim 1, the time server; an inquiry means for inquiring of each of the plurality of communication devices whether at least one of a plurality of clocks included in each of the plurality of communication devices is synchronized with the time server; an instruction means for specifying a start time at which the plurality of output units will output the synchronization signal when all of the plurality of communication devices are synchronized with the time server, and instructing the plurality of communication devices to output the synchronization signal; A server system comprising:

6. A plurality of time servers are connected to the plurality of communication interfaces of each communication device, respectively, and are synchronized with each other.

6. The server system according to claim 5.

7. a plurality of communication interfaces connected to a time server; a plurality of clocks each associated with said communication interface; a plurality of output units each associated with a clock and outputting a synchronization signal based on the associated clock, the communication device synchronizing the plurality of clocks with the time server; the communication device checking whether the plurality of clocks are synchronized with the time server; the communication device selects, from among the plurality of synchronization signals output from the plurality of output units, a synchronization signal output from the output unit associated with the clock synchronized with the time server as a synchronization signal to be supplied to another device; and each of the plurality of output units outputs the synchronization signal when the time of the clock reaches a start time designated by an external device; A control method for a communication device, characterized in that each of the plurality of output units outputs the synchronization signal when the time of the clock associated with that output unit reaches the start time, even if the clock is not synchronized with the time server.

8. A program for causing a computer to function as each of the means of the communication device according to any one of claims 1 to 4.

9. A program for causing a computer to function as each means of the server system according to claim 5 or 6.

Images