Communication device, method for synchronizing communication device, and program

Abstract

This enables rapid time correction, preventing the time difference between the source and destination from widening. [Solution] The communication device sends and receives control messages with the source communication device to obtain the average delay time during communication. Based on statistical information of multiple average delay times obtained through multiple transmissions and receptions of control messages, it sets a threshold time range for the average delay time. If the most recently obtained average delay time is within the threshold time range, the communication device corrects its own time based on the time difference between the most recently obtained average delay time and the time of the source communication device to synchronize with the time of the source communication device.

Description

Technical Field

[0001] The present invention relates to a technique for synchronizing the time with the time of a communication device as a synchronization source that communicates via a network.

Background Art

[0002] Time synchronization between a plurality of communication devices connected via a network can be realized, for example, by using the function of PTP (Precision Time Protocol) of the IEEE1588 standard. First, a time server as a synchronization source communication device and a synchronization destination communication device transmit and receive time synchronization packets including time information to each other according to a time synchronization sequence defined by PTP to synchronize the time. By implementing such a time synchronization sequence between the time server and all other communication devices, the times of each communication device are synchronized. However, when the communication path between the time server as the synchronization source and the synchronization destination communication device is wireless communication, the fluctuation of the time required for transmitting the time synchronization packet tends to increase. And this fluctuation of the transmission time becomes a factor for reducing the time synchronization accuracy.

[0003] On the other hand, Patent Document 1 discloses a method for preventing a decrease in time synchronization accuracy when wireless communication is used for a communication path. In the time synchronization system of Patent Document 1, the time synchronization sequence defined by PTP is executed a predetermined number of times, and each time the time synchronization sequence is executed, the average delay time (time required for transmission) is calculated and stored, and a time correction value for the synchronization destination is also calculated and stored. Then, in the time synchronization system, the minimum average delay time is selected from the stored average delay times, and the time of the synchronization destination is corrected using the time correction value corresponding to the minimum average delay time.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] As mentioned above, in the time synchronization system of Patent Document 1, the time synchronization sequence must be performed a predetermined number of times, and time correction of the synchronized time cannot be performed until the predetermined number of time synchronization sequences is completed. As a result, the time difference between the time of the synchronization source and the time of the synchronized time may widen while the predetermined number of time synchronization sequences are being performed. And when the time difference between the synchronization source and the synchronized time widens, there is a risk that the time synchronization accuracy will decrease.

[0006] Therefore, the present invention aims to enable rapid time correction and suppress the widening of the time difference between the synchronization source and the synchronization destination. [Means for solving the problem]

[0007] The communication device of the present invention is characterized by comprising: an acquisition means for sending and receiving control messages with a source communication device to acquire the average delay time during communication; a setting means for setting a threshold time range for the average delay time based on statistical information of multiple average delay times acquired by sending and receiving the control messages multiple times; and a synchronization means for correcting the time of the device itself and synchronizing it with the time of the source communication device based on the time difference between the latest average delay time and the time of the source communication device, if the latest average delay time acquired by the acquisition means is within the threshold time range. [Effects of the Invention]

[0008] The present invention aims to enable rapid time correction and suppress the widening of the time difference between the synchronization source and the synchronization destination. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of a synchronized imaging system. [Figure 2]This is a diagram showing the time synchronization sequence. [Figure 3] This is a diagram showing an example of the internal configuration of a communication device. [Figure 4] This is a flowchart of the time synchronization process according to this embodiment. [Figure 5] This figure shows an example of a management table. [Figure 6] This figure shows the average delay time for 1 to 6 time synchronization sequences. [Figure 7] This figure shows the average delay time for time synchronization sequences ranging from 1 to 12 times. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are not limiting to the present invention, and not all combinations of features described in these embodiments are essential to the solutions of the present invention. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications of the apparatus to which the present invention is applied and various conditions (usage conditions, usage environment, etc.). In the following embodiments, redundant explanations of the same or similar configurations or processing steps will be omitted.

[0011] In this embodiment, we will describe an example in which multiple communication devices are connected, for example, via a wireless network, and time synchronization is performed using the PTP (Precision Time Protocol) function of the IEEE 1588 standard. In particular, in this embodiment, we will describe a synchronized shooting system in which each of the multiple communication devices is equipped with a camera unit (imaging unit), and these multiple communication devices are installed in different locations to synchronize shooting from multiple viewpoints, and 3D image content is generated using the multiple viewpoint images obtained from this synchronized shooting. This 3D image content generation technology makes it possible to view highlight scenes of sports and other events from various angles, providing viewers with a high sense of realism. In this way, in order to generate 3D image content based on images captured by the camera units of multiple communication devices, it is necessary to synchronize the shooting timing of each camera unit. Since the shooting timing is generated based on the time information of the camera unit, synchronized shooting can be achieved by synchronizing the time of each camera unit.

[0012] In this embodiment, time synchronization between the camera units of multiple communication devices is achieved using the PTP function of the IEEE 1588 standard, with one of these communication devices designated as the time server that will be used for synchronization. Between the time server and the other communication devices that will be used for synchronization, time synchronization packets containing time information are sent and received according to the time synchronization sequence defined by PTP, thereby synchronizing the time between these communication devices. By performing this time synchronization sequence between the time server's communication device and all other communication devices, the time of the camera unit of each communication device becomes synchronized. Then, each camera unit with synchronized time generates a shooting timing from its respective time, enabling synchronized shooting at each camera unit.

[0013] Figure 1 shows an example configuration of the synchronous shooting system according to this embodiment. The synchronous shooting system of this embodiment is composed of a plurality of communication devices 101 to 103, each of which is equipped with a camera unit. In the synchronous shooting system according to this embodiment, the time of each communication device 100 to 103 is synchronized, and synchronous shooting is achieved by having each camera unit take a picture at a shooting timing generated using the synchronized time. Although not shown in the figure, the image information captured by the camera units of each communication device 100 to 103 is sent to an image processing device (information processing device) that generates, for example, 3D image content. As a result, the image processing device can perform image processing to generate desired 3D image content using the multiple images captured in synchronous order.

[0014] In the synchronized imaging system shown in Figure 1, we will give an example in which communication device 100 acts as the time server (time distribution server) among the four communication devices 100 to 103. The other communication devices 101 to 103, excluding the time server communication device 100, act as client devices that synchronize to the time of the time server communication device 100. Hereafter, the communication device that acts as the time server will be referred to as the "source communication device," and the communication devices that act as client devices will be referred to as the "destination communication devices." The source communication device 100 acts as a wireless LAN access point, and the three destination communication devices 101 to 103 are wirelessly connected to the source communication device 100. Furthermore, it is assumed that each of the destination communication devices 101 to 103 performs time synchronization with the source communication device 100 using PTP. Therefore, the source communication device 100 is assumed to have the function of a GMC (Grand Master Clock) or BC (Boundary Clock) that acts as the time synchronization source for PTP.

[0015] Next, the time synchronization sequence used during PTP time synchronization will be explained with reference to Figure 2. Figure 2 shows the PTP time synchronization sequence performed between the source communication device 100 and the destination communication devices 101 to 103. In the following explanation, communication device 101 will be used as a representative example of a destination communication device.

[0016] First, as a message communication 201, the source communication device 100 sends a communication packet of a synchronization message (Sync) to the destination communication device 101 as a control message to start the time synchronization sequence. At this time, the source communication device 100 internally stores timestamp information T1 of when it sent the synchronization message (Sync) in message communication 201. The destination communication device 101 also internally stores timestamp information T2 of when it received the communication packet of the synchronization message (Sync) sent from the source communication device 100 in message communication 201.

[0017] Next, as message communication 202, the source communication device 100 sends a follow-up message (Follow_Up) communication packet to the destination communication device 101 as the next control message in the time synchronization sequence. This follow-up message (Follow_Up) contains the timestamp information T1 from when the synchronization message (Sync) was sent in message communication 201. The destination communication device 101 receives the follow-up message (Follow_Up) communication packet sent from the source communication device 100 in message communication 202 and internally stores the timestamp information T1 contained in that communication packet.

[0018] Next, as message communication 203, the communication device 101 at the synchronization destination transmits a communication packet of a delay request message (Delay_Req) to the communication device 100 serving as the synchronization source as the next control message in the time synchronization sequence. The communication device 101 at the synchronization destination at this time holds the timestamp information T3 at the time when the delay request message (Delay_Req) is transmitted in message communication 203 internally. Also, the communication device 100 serving as the synchronization source holds the timestamp information T4 at the time when the communication packet of the delay request message (Delay_Req) is received in message communication 203 internally.

[0019] Subsequently, as message communication 204, the communication device 100 serving as the synchronization source transmits a communication packet of a delay response message (Delay_Resp) to the communication device 101 serving as the synchronization destination as the next control message in the time synchronization sequence. The delay response message (Delay_Resp) at this time includes the timestamp information T4 at the time when the delay request message (Delay_Req) was received in message communication 203. The communication device 101 at the synchronization destination receives the communication packet of the delay response message (Delay_Resp) in message communication 204 and holds the timestamp information T4 included in the communication packet internally. The above is the time synchronization sequence used when performing time synchronization by PTP.

[0020] In this way, the communication device 100 serving as the synchronization source and the communication device 101 at the synchronization destination implement a time synchronization sequence using the PTP function of the IEEE1588 standard and transmit and receive control messages to each other. Also, the communication device 101 at the synchronization destination acquires and holds the time information (T2, T3) when transmitting and receiving the control message and the time information (T1, T4) included in the control message transmitted from the communication device 100 serving as the synchronization source.

[0021] Next, an outline of the method for the communication device 101 at the synchronization destination to synchronize its own time with the time of the communication device 100 serving as the synchronization source will be described. As explained in Figure 2, the PTP time synchronization sequence is completed when the synchronization destination communication device 101 receives the delayed response message (Delay_Resp) via message communication 204. At this time, the synchronization destination communication device 101 internally stores the time (T2) when it received the synchronization message (Sync) via message communication 201 and the time (T3) when it sent the delayed response message (Delay_Req) ​​via message communication 203. Furthermore, the synchronization destination communication device 101 obtains and stores the time (T1) when the source communication device 100 sent the synchronization message (Sync) via message communication 201 from the follow-up message (Follow_Up) received via message communication 202. In addition, the synchronization destination communication device 101 obtains and stores the time (T4) when the source communication device 100 received the delayed response message (Delay_Resp) via message communication 203 from the delayed response message (Delay_Resp) received via message communication 204.

[0022] In PTP, formulas are defined to calculate the average delay time during communication and the time difference between the time of the source communication device and the time of the destination communication device, using the times of the timestamp information T1 to T4. The average delay time is defined as the time it takes for communication packets to be transmitted between the source communication device and the destination communication device, and is defined as being the same time for both the outbound and return paths. According to the PTP definition, the transmission time (average delay time) between the source communication device 100 and the destination communication device 101, using the times of the timestamp information T1, T2, T3, and T4 shown in Figure 2, can be calculated using the formula shown in equation (1) below. Also, according to the PTP definition, the time difference between the time of the source communication device 100 and the time of the destination communication device 101 can be calculated using the formula shown in equation (2) below.

[0023] ((T4-T1)-(T3-T2)) / 2 Equation (1) ((T2-T1)-(T4-T3)) / 2 Equation (2)

[0024] As described above, the destination communication device 101 can calculate the average delay time and time difference during communication with the destination communication device 100 by performing a PTP time synchronization sequence with the source communication device 100 and performing calculations defined in PTP. Therefore, the destination communication device 101 can synchronize its own time with the time of the source communication device by adjusting its own clock based on the calculated time difference.

[0025] However, when the communication path between the source and destination communication devices is wireless, as mentioned above, fluctuations in the transmission time of time synchronization packets can become large, and these fluctuations in transmission time can degrade the accuracy of time synchronization. In the technology described in Patent Document 1, a predetermined number of time synchronization sequences are performed, and the time of the destination is corrected using a time correction value corresponding to the minimum average delay time. However, the time difference between the source and destination may widen during the execution of the predetermined number of time synchronization sequences.

[0026] Therefore, in the synchronization destination communication device according to this embodiment, a threshold time range for the average delay time is set based on statistical information of multiple average delay times obtained by performing the time synchronization sequence multiple times. Furthermore, if the average delay time when the latest control message is sent and received is within the threshold time range, the synchronization destination communication device calculates the time difference between itself and the synchronization source communication device using that latest average delay time. Then, the synchronization destination communication device corrects its own time based on that time difference to synchronize its own time with the synchronization source communication device's time.

[0027] Figure 3 shows an example of the internal configuration of each communication device 100 to 103 included in the synchronized imaging system of Figure 1 of this embodiment. The internal configuration of the source communication device 100 and the destination communication devices 101 to 103 is the same. The operation as a wireless LAN access point and the operation as a client device are realized by the software executed by each communication device.

[0028] As shown in Figure 3, the communication devices 100 to 103 include a control unit 301, a storage unit 302, a clock unit 303, a time correction unit 304, a communication I / F unit 305, and a camera unit 306, and each of these units is interconnected via a system bus 308. In the communication devices 100 to 103, each unit connected via the system bus 308 is controlled by the control unit 301.

[0029] The storage unit 302 includes a non-volatile storage unit that stores system programs and application programs including the time synchronization program according to this embodiment, and a temporary storage unit for when those programs are executed.

[0030] The control unit 301 executes system programs and application programs read from the non-volatile memory unit and deployed in the temporary memory unit to generate and control the transmission and reception of control messages for the aforementioned time synchronization sequence, and to perform various processes and controls related to time synchronization, as described later. As will be described in detail later, the control unit 301 sends and receives control messages with the source communication device via the communication I / F unit 305 to obtain the average delay time during communication. The control unit 301 also sets a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving control messages multiple times. Then, if the latest average delay time obtained by the control unit 301 is within the value time range, it corrects the time of its own device via the time correction unit 304, described later, based on the time difference between the latest average delay time and the time of the source communication device, and synchronizes it with the time of the source communication device. The control unit 301 is assumed to be implemented by a CPU or the like.

[0031] The clock unit 303 measures the system time within the communication devices 100-103. The clock unit 303 also has the function of generating pulse signals of frequencies such as 1Hz, 25Hz, and 29.97Hz based on the measured time and outputting them to the camera unit 306. Furthermore, the clock unit 303 also has the function of directly outputting time information to the communication interface unit 305 without going through the system bus 308. The clock unit 303 also corrects and updates time information according to instructions from the time correction unit 304, which will be described later. The clock unit 303 may be configured to be included within the communication interface unit 305, or it may be implemented as a software clock executed by the control unit 301.

[0032] The time correction unit 304 has the function of directly adjusting the time of the clock unit 303 without going through the system bus 308, in accordance with instructions from the control unit 301. The communication interface unit 305 includes a wireless communication controller IC and an antenna. The communication interface unit 305 establishes a wireless communication link 307 with other communication devices and sends and receives communication packets. The communication interface unit 305 also has a function to stamp (time stamp) the time information notified by the clock unit 303 when sending and receiving communication packets. In this embodiment, wireless communication is assumed to be wireless LAN (Wi-Fi®), but other wireless communication standards such as private LTE (Long Term Evolution) and local 5G may also be used.

[0033] The camera unit 306 includes hardware units necessary for shooting, such as a lens group, an imaging processing unit, an image processing unit, and a storage media unit. When the camera unit 306 receives a pulse signal output by the aforementioned clock unit 303, it generates a shooting timing at a frame rate such as 59.94 fps or 50 fps based on that pulse signal and performs shooting. By generating a shooting timing based on a time synchronized with other communication devices, the camera unit 306 can perform shooting synchronized with the camera units of other communication devices.

[0034] Next, the flow of the time synchronization process using PTP according to this embodiment, which is performed in the synchronized communication device 101, will be explained with reference to the flowchart shown in Figure 4. In the flowchart shown in Figure 4, the flow of the time synchronization process will be explained using communication device 101 as a representative example among the synchronized communication devices 101 to 103 of the synchronized shooting system shown in Figure 1.

[0035] In step S401, the communication device 101 that is the synchronization destination has established wireless communication with the communication device 100 that is the synchronization source and is ready to perform the PTP time synchronization sequence. The initial values ​​of the first default value [m] and the second default value [n], which will be described later, have been set, the counter value, and the first and second flags, which will be described later, have also been reset. The reset value of the counter value is "0", and the reset values ​​of the first and second flags are also "0". The first default value [m] and the second default value [n] can each be set to any value, but the second default value [n] is set to a value greater than the first default value [m]. In this embodiment, the first default value [m] is set to, for example, "5", and the second default value [n] is set to, for example, "10".

[0036] Next, in step S402, the control unit 301 of the communication device 101 determines whether the communication packet received by the communication I / F unit 305 from the source communication device 100 is a synchronization message (Sync) as defined in PTP. If the control unit 301 determines that the communication packet is a synchronization message (Sync), it proceeds to step S403. On the other hand, if the control unit 301 determines that the communication packet is not a synchronization message (Sync), the control unit 301 maintains the determination process in step S402.

[0037] When the process proceeds to step S403, the control unit 301 performs a counter update process that increments the counter value. The counter value is the number of times the subsequent series of steps S404 to S406 have been performed in the communication device 101. After step S403, the control unit 301 proceeds to step S404.

[0038] When the process proceeds to step S404, the communication device 101 performs the aforementioned PTP time synchronization sequence processing with the original communication device 100. In this case, the time synchronization sequence processing is the process from receiving the synchronization message (Sync) of message communication 201 in Figure 2 to receiving the delayed response message (Delay_Resp) of message communication 204. The time synchronization sequence processing in the communication device 101 is performed under the control of the control unit 301. The control unit 301 then obtains timestamp information T2 and T3, which are time stamps (timestamps) of the clock unit 303 when a communication packet is sent or received, and also obtains timestamp information T1 and T4 from the received communication packet according to the communication protocol. After step S404, the control unit 301 proceeds to step S405.

[0039] When the process proceeds to step S405, the control unit 301 calculates the average delay time by substituting the time information (timestamp information T1, T2, T3, T4) obtained during the time synchronization sequence processing in step S404 into the aforementioned equation (1). Then, as the next step S406 following step S405, the control unit 301 stores the average delay time calculated in step 405 in the storage unit 302. After step S406, the control unit 301 proceeds to step S407.

[0040] When the process proceeds to step S407, the control unit 301 determines whether the counter value stored in the memory unit 302 is equal to the first specified value [m]. The first specified value [m] is a value predetermined to indicate the number of times the series of processes in steps S404 to S406, that is, the series of processes related to the time synchronization sequence, have been performed. In this embodiment, for example, "5" is set as the first specified value [m]. Hereinafter, the execution of the series of processes related to the time synchronization sequence will be described as "execution of the time synchronization sequence," and the number of times the series of processes of the time synchronization sequence have been performed will be described as "number of time synchronization sequence executions." If the control unit 301 determines that the counter value is equal to the first specified value [m], it proceeds to step S408; otherwise, it proceeds to step S409. That is, when the number of time synchronization sequence executions in steps S404 to S406 reaches [m] times, the control unit 301 proceeds to step S408; otherwise, it proceeds to step S409.

[0041] If the process proceeds to step S408, the control unit 301 performs the process of setting a first flag to be stored in the memory unit 302. The first flag indicates that the number of times the time synchronization sequence in steps S404 to S406 has been performed is [m] times, and the control unit 301 sets the first flag to "1" when the number of times the time synchronization sequence in steps S404 to S406 has been performed is [m] times. After step S408, the control unit 301 proceeds to step S409.

[0042] If the process proceeds to step S409, the control unit 301 determines whether the counter value stored in the memory unit 302 is equal to the second specified value [n]. The second specified value [n] is a value predetermined to indicate the number of times the time synchronization sequence process in steps S404 to S406 has been performed. In this embodiment, for example, "10" is set as the second specified value [n]. If the control unit 301 determines that the counter value is equal to the second specified value [n], it proceeds to step S410. If it determines that the values ​​are not equal, it proceeds to step S411. That is, if the number of times the time synchronization sequence in steps S404 to S406 has been performed reaches [n], the control unit 301 proceeds to step S410; otherwise, it proceeds to step S411.

[0043] If the process proceeds to step S410, the control unit 301 performs a process to set a second flag to be stored in the memory unit 302. The second flag indicates that the number of times the time synchronization sequence in steps S404 to S406 has been performed is [n] times, and the control unit 301 sets the second flag to "1" when the number of times the time synchronization sequence in steps S404 to S406 has been performed is [n] times. After step S410, the control unit 301 proceeds to step S411.

[0044] If the process proceeds to step S411, the control unit 301 determines whether the first flag stored in the memory unit 302 is set to "1". If it determines that the first flag is set to "1", the control unit 301 proceeds to step S412; otherwise, it proceeds to step S413.

[0045] If the process proceeds to step S412, the control unit 301 determines the minimum value from the multiple average delay times stored in the memory unit 302 and updates it as the minimum average storage time. As described above, each time the time synchronization sequence in step S404 is performed, the control unit 301 calculates the average delay time in step S405 and stores it in the memory unit 302 in step S406. Therefore, the control unit 301 updates the minimum average delay time by selecting the minimum average delay time from the multiple average delay times stored in the memory unit 302. After step S412, the control unit 301 proceeds to step S413.

[0046] When the process proceeds to step S413, the control unit 301 sets a threshold time range for the average delay time based on statistical information of multiple average delay times stored in the memory unit 302, and determines whether the latest average delay time is within the threshold time range. However, if the number of time synchronization sequence executions after the start of communication with the synchronization source communication device 100 has not reached the number of times specified by the first predetermined value [m], the control unit 301 does not determine whether the latest average delay time is within the threshold time range. Then, when the number of time synchronization sequence executions after the start of communication with the synchronization source communication device 100 reaches the number of times specified by the first predetermined value [m], the control unit 301 sets a threshold time range based on the minimum value among the multiple average delay times resulting from the execution of those time synchronization sequences.

[0047] Here, the control unit 301 uses the minimum value among multiple average delay times stored in the storage unit 302 as statistical information when setting the threshold time range. The control unit 301 then sets the threshold time range by adding a predetermined time to the minimum average delay time. In this embodiment, the threshold time range is used to determine whether the state of the wireless communication channel is affected by disturbances, etc., based on the time information (T1, T2, T3, T4) used when calculating the latest average delay time. In other words, if the minimum average delay time is within the threshold time range set based on the minimum value of multiple average delay times stored in the storage unit 302, it can be determined that the wireless communication channel was not significantly affected by disturbances, etc., when the time synchronization sequence in step S404 was performed. If it is determined in step S413 that the latest average delay time is within the threshold time range, the control unit 301 proceeds to step S417; on the other hand, if it is determined that it is outside the threshold time range, it proceeds to step S414.

[0048] If the process proceeds to step S414, the control unit 301 determines whether the second flag stored in the memory unit 302 is set to "1". If it determines that the second flag is set, the control unit 301 proceeds to step S415; otherwise, it returns to step S402.

[0049] If the process proceeds to step S415, the control unit 301 uses the time information (timestamp information T1, T2, T3, T4) at the time the minimum average delay time was calculated to calculate the time difference with the original communication device 100 using the aforementioned equation (2). The control unit 301 then stores the calculated time difference in the storage unit 302. After step S415, the control unit 301 proceeds to step S416.

[0050] In step S416, the control unit 301 performs a process to reset the counter value stored in the memory unit 302. The process of resetting the counter value sets the counter value to "0". At this time, the control unit 301 also performs a process to return the second flag to the reset value of "0". After step S416, the control unit 301 proceeds to step S418.

[0051] If the process proceeds to step S417, the control unit 301 uses the time information (timestamp information T1, T2, T3, T4) from when the latest average delay time was calculated to calculate the time difference with the communication device 100 using the aforementioned formula (2). The control unit 301 then stores the calculated time difference in the storage unit 302. After step S417, the processing of the control unit 301 proceeds to step S418.

[0052] When the process proceeds to step S418, the control unit 301 adjusts the clock information of its own device using the time difference calculated in step S415 or step S417. That is, the control unit 301 controls the time correction unit 304 based on the newly calculated time difference to correct the time of the clock unit 303, thereby synchronizing the time of its own device with the time of the communication device 100 that was the source of synchronization. At this time, the control unit 301 also stores the value indicating the time difference in the storage unit 302. After step S418, the processing of the control unit 301 proceeds to step S419.

[0053] When the process proceeds to step S419, the control unit 301 determines whether there is an instruction from the user or the communication device 100 that is the source of synchronization, such as to terminate the time synchronization process shown in Figure 4. If there is no termination instruction, the control unit 301 returns to step S402; however, if there is a termination instruction, the process proceeds to step S420. When the process proceeds to step S420, the control unit 301 terminates the processing shown in the flowchart of Figure 4. Upon completion of the processing shown in the flowchart of Figure 4, the time synchronization process between the synchronization destination communication device 101 and the synchronization source communication device 100 is completed.

[0054] In this embodiment, the control unit 301 of the synchronized communication device 101 manages the information stored in the storage unit 302 in the time synchronization process shown in the flowchart of Figure 4, or the information that has been stored, in a table format as shown in Figure 5. Figure 5 is a diagram showing an example of a management table handled by the control unit 301. The management table shown in Figure 5(a) is mainly for managing information obtained by performing the time synchronization sequence, and the table shown in Figure 5(b) is a table for storing management information when using the information in the management table of Figure 5(a).

[0055] In the management table shown in Figure 5(a), column 501 stores the counter value for the number of times the time synchronization sequence was executed in step S404. As described above, the control unit 301 updates (increments) the counter value in step S403 and resets the counter value to its initial value of "0" in step S416. In addition, in the management table shown in Figure 5(a), the counter value in column 501 is used as information to specify a row. In this embodiment, since the second default value [n] of the counter value is "10", the rows specified by the counter value are 10 rows (rows 1-508 to 10-511). For example, counter value "1" specifies row 1-508, counter value "2" specifies row 2-509, counter value "3" specifies row 3-510, and so on, with counter value "10" being used as information to specify row 10-511. The control unit 301 manages the information in columns 502 to 505, described later, by corresponding it to the counter values ​​in rows 508 to 511 of column 501.

[0056] Column 502 in Figure 5(a) is the column that stores the time information (timestamp information T1, T2, T3, T4) obtained when the time synchronization sequence in step S404 is performed. As mentioned above, rows are specified by the counter value, so the control unit 301 manages the time information by associating each row of column 502 with the counter value in column 501. That is, the control unit 301 manages the time information for the counter value "1" in the first row 508 of column 502, the time information for the counter value "2" in the second row 509 of column 502, and the time information for the counter value "3" in the third row 510 of column 502. The control unit 301 continues to manage the time information in the same manner thereafter, and manages the time information for the counter value "10" in the tenth row 511 of column 502.

[0057] Column 503 in Figure 5(a) is the column that stores the average delay time calculated in step S405. The control unit 301 manages the average delay time in column 503 in the same way as described above, corresponding to the counter value in column 501. That is, the control unit 301 manages the average delay time when the counter value is "1" in the first row 508 of column 503, the average delay time when the counter value is "2" in the second row 509, and the average delay time when the counter value is "3" in the third row 510. The control unit 301 continues to manage the average delay time in the same way thereafter, managing the average delay time when the counter value is "10" in the tenth row 511 of column 503.

[0058] Column 504 in Figure 5(a) is a column that stores the time correction value corresponding to the time difference calculated using equation (2) in step S415. The control unit 301 manages the time correction value in column 504 in the same way as described above, corresponding to the counter value in column 501. That is, the control unit 301 manages the time correction value for counter value "1" in the first row 508 of column 504, the time correction value for counter value "2" in the second row 509, and the time correction value for counter value "3" in the third row 510. The control unit 301 continues to manage the time correction values ​​in the same way thereafter, and manages the time correction information for counter value "10" in the tenth row 511.

[0059] Column 505 in Figure 5(a) is a column that stores a flag indicating which time synchronization sequence execution the minimum average delay time determined in step S412 corresponds to. The control unit 301 manages the minimum average delay time flag in column 505 in the same way as described above, corresponding to the counter value in column 501. Specifically, the control unit 301 manages the minimum average delay time flag for counter value "1" in the first row 508 of column 505, the minimum average delay time flag for counter value "2" in the second row 509, and the minimum average delay time flag for counter value "3" in the third row 510. The control unit 301 continues to manage the minimum average delay time in the same manner thereafter, managing the minimum average delay time flag for counter value "10" in the tenth row 511 of column 505. The control unit 301 sets the flag for the minimum average delay time corresponding to the counter value where the average delay time is minimized to "1", and to "0" otherwise. Therefore, in each row of column 505, only one flag for the minimum average delay time will be set to "1".

[0060] The management information table shown in Figure 5(b) consists of the first flag setting value 521, the first flag 522, the second flag setting value 523, and the second flag 524. The control unit 301 manages the aforementioned first default value [m], which is used in step S412 to determine the minimum average delay time, as the first flag setting value 521. In this embodiment, the control unit 301 sets the first flag setting value 521 to "5", which is the initial value of the first default value [m], before the start of the time synchronization process in Figure 4. The determination process in step S407 described above is performed by referring to this first default value [m] of the first flag setting value 521. Note that the first default value [m] of the first flag setting value 521 is set to a value smaller than the second default value [n], and may be "0".

[0061] The control unit 301 sets the initial value of the first flag 522 in Figure 5(b) to "0". Then, in the first flag setting process in step S408 of Figure 4, the control unit 301 sets the first flag 522 to "1". Furthermore, in step S411 of Figure 4, the control unit 301 determines whether the value of this first flag 522 is "1". This first flag 522 is a flag used to confirm that the number of time synchronization sequence executions has reached a first specified value [m], which is necessary to determine the minimum average delay time in step S412. The first flag 522 is used to investigate the state of the wireless communication path in advance when communication is started between the source communication device 100 and the destination communication device 101 when the time synchronization process in Figure 4 is performed.

[0062] The control unit 301 manages a second default value [n] as the second flag setting value 523 in Figure 5(b), which indicates the number of average delay times (parameter) used to determine the minimum average delay time used to calculate the time difference in step S415. In this embodiment, the control unit 301 sets the second flag setting value 523 to "10", which is the initial value of the second default value [n], before the start of the time synchronization process in Figure 4. The determination process in step S409 described above is performed by referring to the second default value [n] of the second flag setting value 523. In this embodiment, the value of the second default value [n] of the second flag setting value 523 may be set according to the state of the wireless communication path between the source communication device 100 and the destination communication device 101. For example, it may be set to a smaller value if the fluctuation of the average delay time in the wireless communication path is small, and conversely, to a larger value if it is small. As the fluctuation of the average delay time in the wireless communication path, for example, the fluctuation of the average delay time estimated from the state of the wireless communication path can be used. In other words, for example, the less likely it is that the average delay time estimated from the state of the wireless communication channel will fluctuate significantly, the smaller the value of the second default value [n] of the second flag setting value 523 may be set to; otherwise, it may be set to a larger value. This makes it possible to prevent a decrease in time synchronization accuracy. Also, the second default value [n] of the second flag setting value 523 may be changed at the same time that the counter value is reset in step S416. In other words, the second default value [n] may be set to a different value before and after the counter value is reset. As mentioned above, the second default value [n] of the second flag setting value 523 is used as the value to determine the maximum value of the counter value in column 501 of Figure 5.

[0063] The control unit 301 sets the initial value of the second flag 524 in Figure 5(b) to "0". Then, in the second flag setting process of step S410 in Figure 4, the control unit 301 sets the second flag 524 to "1". Furthermore, in step S414 in Figure 4, the control unit 301 determines whether the value of this second flag 524 is "1". This second flag 524 is a flag used to confirm that the number of time synchronization sequence executions has reached the number of average delay times (determinant) required to determine the minimum average delay time used in step S415.

[0064] Figures 6 and 7 are graphs showing the change in average delay time when the time synchronization process in Figure 4 is performed using the management table in Figure 5 in the synchronized shooting system in Figure 1. Below, the timing at which the synchronized communication device 101 performs time correction in the case where the average delay time fluctuates due to the use of a wireless communication channel will be explained with reference to Figures 6 and 7. The vertical axis of the graphs in Figures 6 and 7 shows the average delay time (ms) calculated by the synchronized communication device 101 in step S405 in the time synchronization process in Figure 4. The horizontal axis of the graphs in Figures 6 and 7 shows the number of times the time synchronization sequence is performed. The initial value of the first specified value [m] in the communication device 101 is assumed to be "5", and the initial value of the second specified value [n] is assumed to be "10".

[0065] Figure 6 is a graph plotting the average delay time obtained for each time synchronization sequence when the time synchronization sequence is performed 1 to 5 times. Figure 7 is a graph plotting the average delay time obtained for each time synchronization sequence when the time synchronization sequence is performed 6 to 12 times. Note that the predetermined time used in step S413 in Figure 4 is defined as, for example, 0.4 ms.

[0066] In Figures 6 and 7, point 600 represents the point where the average delay time obtained from the first (1st) time synchronization sequence performed in the time synchronization process shown in Figure 4 is plotted. Points 601 to 604 represent the points where the average delay time was plotted when the time synchronization sequence was performed 2 to 5 times. In Figure 7, points 700 to 706 represent the points where the average delay time was plotted when the time synchronization sequence was performed 6 to 12 times. Points 602, 700, 703, 704, and 706, etc., indicate that their average delay times are larger than others because the communication path between the source communication device 100 and the destination communication device 101 is a wireless communication path and is therefore affected by disturbances.

[0067] The following describes the execution of the time synchronization sequence in order, with reference to Figures 6, 7, and 5. In this embodiment, as mentioned above, the first specified value [m] indicating the number of times the time synchronization sequence is executed is set to "5", and the second specified value [n] is set to "10".

[0068] If the time synchronization sequence has been performed 1 to 4 times, the control unit 301 stores the counter values ​​"1" to "4" in the order in which the time synchronization sequence was performed in the first to fourth rows of column 501 of the management table. The control unit 301 also stores the time information acquired each time the time synchronization sequence is performed in the first to fourth rows of column 502. Similarly, the average delay time calculated each time the time synchronization sequence is performed is stored in the first to fourth rows of column 503.

[0069] If the time synchronization sequence has been executed for the fifth time, the control unit 301 stores the counter value "5" in the fifth row of column 501 of the management table. Here, since the first specified value [m] is "5", in step S408 the first flag is set to "1", in step S411 it is determined that the first flag is present, and in step S412 the minimum average delay time is determined and updated. At this time, the average delay time obtained up to the fifth time the time synchronization sequence has been executed is shown as points 600 to 604 in the example of Figure 6, and the minimum average delay time among these points 600 to 604 is the time shown as point 600. Also, since the predetermined time is defined as 0.4ms, in step S413 the control unit 301 sets the time range from time 605 to time 606 as the threshold time range, which is obtained by adding the predetermined time of 0.4ms to time 605 at point 600, which is the minimum average delay time. The control unit 301 then stores the time information obtained during the fifth time synchronization sequence in the fifth row of column 502, and also stores the average delay time in the fifth row of column 503. Furthermore, the control unit 301 sets "1" as the minimum average delay time flag in the first row 508 of column 505. This means that point 600 has been registered as having the minimum average delay time. At this point, the time correction value has not yet been calculated, so the time correction value is not stored in column 504, and therefore no time correction is performed. Note that the threshold time range may be a fixed time range that includes the minimum average delay time. That is, the threshold time range may be a time range obtained by adding a predetermined time to a value (shorter time) that is even smaller than the minimum average delay time of point 600. In this case, the minimum average delay time of point 600 will fall within the threshold time range.

[0070] If the time synchronization sequence has been executed for the sixth time, the average delay time calculated in step S405 is the time indicated by point 700. In this case, the control unit 301 stores the time information acquired in the sixth execution of the time synchronization sequence in the sixth row of column 502, and also stores the average delay time in the sixth row of column 503. However, point 700, the average delay time, is outside the threshold time range set up to the fifth execution of the time synchronization sequence. Therefore, in step S413, the control unit 301 determines that it is outside the threshold time range. Also, if the time synchronization sequence has been executed for the sixth time, the second flag in step S410 is "0", so in step S414, the control unit 301 determines that the second flag is not set. Therefore, the control unit 301 returns to step S402.

[0071] If the time synchronization sequence has been executed for the seventh time, the average delay time calculated in step S405 is assumed to be the time indicated by point 701. Since the average delay time at point 701 is shorter (smaller) than the time at point 600, the control unit 301 updates the minimum average delay time to the average delay time at point 701 in step S412. In step S413, the control unit 301 sets the time range from time 708 to time 709 as a new threshold time range, obtained by adding a predetermined time (0.4 ms) to time 708, which is the updated minimum average delay time at point 701. In this case as well, the threshold time range may be a fixed time range that includes the minimum average delay time. For example, the threshold time range may be a time range obtained by adding a predetermined time to an even smaller value (shorter time) than the minimum average delay time at point 701. In this case, the minimum average delay time at point 701 will fall within the threshold time range.

[0072] Then, in step S413, the control unit 301 determines that point 701, which is the most recent average delay time, is within the threshold time range, and calculates the time difference in step S417. Subsequently, in step S418, the time correction processing is performed by the time correction unit 304, thereby correcting the time of the communication device 101. Here, the second specified value [n], which is set as the number of average delay times (determinant) required to determine the minimum average delay time, is "10", and point 701, obtained in the 7th execution of the time synchronization sequence, is the minimum average delay time. In other words, the communication device 101 has obtained a good value for the average delay time in the 7th execution of the time synchronization sequence, and can perform time correction at the timing when this average delay time is obtained. Thus, in the communication device 101 of this embodiment, time correction can be performed as soon as a sufficiently small value for the average delay time is obtained, without having to wait until 10 executions of the time synchronization sequence corresponding to "10" set as the second specified value [n] has been performed.

[0073] If the time synchronization sequence has been performed for the eighth time, the average delay time calculated in step S405 is assumed to be point 702. Furthermore, if the time synchronization sequence has been performed for the ninth time, the average delay time calculated in step S405 is assumed to be point 703. Both of these average delay times calculated for the eighth and ninth time synchronization sequence executions are outside the threshold time range. Therefore, in the determination in step S413, the control unit 301 determines that the latest average delay time is outside the threshold time range, and the control unit 301 proceeds to step S414. In this case, the number of time synchronization sequence executions has not reached 10 as defined by the second specified value [n], and the second flag is not set to "1" in step S410, so the control unit 301 returns to step S402. In other words, no time correction is performed in the communication device 101 in this case.

[0074] If the time synchronization sequence has been executed 10 times, the average delay time calculated in step S405 is assumed to be point 704. This average delay time at point 704 is outside the threshold time range. Therefore, in the determination in step S413, the control unit 301 determines that the latest average delay time is outside the threshold time range, and the control unit 301 proceeds to step S414. At this time, the number of time synchronization sequence executions has reached 10 times, which is the second specified value [n], so in step S410 the second flag is set to "1", and in step S414 it is determined that the second flag is present, and the control unit 301 proceeds to step S415. Then, in step S415, the minimum average delay time at point 701 is selected. In the next step S416, the counter value is reset, and in the following step S418, time correction is performed.

[0075] If the time synchronization sequence has been performed 11 times, the average delay time calculated in step S405 is assumed to be point 705. This average delay time at point 705 is within the threshold time range. Therefore, in the determination in step S413, the control unit 301 determines that the latest average delay time is within the threshold time range, and calculates the time difference in step S417. Subsequently, in step S418, time correction is performed by the time correction unit 304. In other words, even when the time synchronization sequence has been performed 11 times, a good average delay time was obtained, so time correction can be performed at the timing when that average delay time was obtained. Thus, in the communication device 101, even in the 11th time synchronization sequence immediately after the counter was reset in the 10th time synchronization sequence, a sufficiently small average delay time was obtained, so time correction is performed.

[0076] If the number of time synchronization sequence executions is 12, the average delay time calculated in step S405 is point 706, and this average delay time at point 706 is outside the threshold time range. Therefore, in the determination in step S413, the control unit 301 determines that the latest average delay time is outside the threshold time range, and the control unit 301 proceeds to step S414. In this case, the number of time synchronization sequence executions after the reset of the counter value mentioned above has not reached 10, and the second flag is not set in step S410, so the control unit 301 returns to step S402.

[0077] As shown in Figure 7, if a time synchronization sequence using wireless communication, which is highly likely to have fluctuating average delay times, is performed multiple times, the average delay time may fluctuate. In the communication device 101 according to this embodiment, the smallest average delay time is selected from the average delay times obtained within a first specified number of time synchronization sequence executions, and a threshold time range for the average delay time is set based on the selected average delay time. Then, if the most recent average delay time obtained within a second specified number of time synchronization sequence executions is within the threshold time range, the communication device 101 uses that average delay time to obtain the time difference with the original communication device 100, and performs time correction of its own device based on that time difference. In the example of Figure 7 described above, the communication device 101 is able to perform time correction three times during the process of performing 12 time synchronization sequences.

[0078] As described above, in the communication device that is the synchronization target of the synchronized shooting system according to this embodiment, if the latest average delay time obtained from the execution of the time synchronization sequence is within the threshold time range set based on the minimum average delay time, time correction can be performed immediately. Therefore, according to this embodiment, time synchronization can be performed without waiting for a predetermined number of time synchronization sequences, such as 10 times, to be completed in order to perform time correction, thus preventing the time difference between the time of the synchronization source and the time of the synchronization target from widening.

[0079] In the embodiments described above, an example was given of setting a threshold time range based on the minimum average delay time. However, for example, an upper limit of time may be set based on the minimum average delay time. Then, if the latest average delay time is less than (or less than or equal to) the upper limit of time, the same time correction as described above may be performed. An example of the upper limit of time set based on the minimum average delay time is the minimum average delay time plus the predetermined time described above.

[0080] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more of the functions. The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features.

[0081] This embodiment includes the following configurations, methods, and programs. (Composition 1) An acquisition means that obtains the average delay time during communication by sending and receiving control messages with the source communication device, A setting means for setting a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving the control message multiple times, If the latest average delay time obtained by the acquisition means is within the threshold time range, the synchronization means corrects the time of the device itself based on the time difference between the latest average delay time and the time of the source communication device to synchronize it with the time of the source communication device. A communication device characterized by having the following features. (Configuration 2) The communication device according to configuration 1, characterized in that the acquisition means calculates the average delay time based on the time information at the time the control message was sent and received with the synchronization source communication device and the time information contained in the control message transmitted from the synchronization source communication device. (Composition 3) The system has a storage means for storing a plurality of average delay times obtained by the acquisition means, The communication device according to configuration 1 or 2, characterized in that the setting means sets the threshold time range based on statistical information of the plurality of average delay times stored in the storage means. (Composition 4) The communication device according to any one of configurations 1 to 3, characterized in that the setting means sets the threshold time range using the minimum value of the plurality of average delay times as statistical information of the plurality of average delay times. (Composition 5) The communication device according to configuration 4, characterized in that the setting means sets the time range between the minimum average delay time and the time obtained by adding a predetermined time to the minimum average delay time as the threshold time range. (Composition 6) The communication device according to configuration 4, characterized in that the setting means sets a certain time range including the minimum average delay time as the threshold time range. (Composition 7) The setting means counts the number of times the control message is sent and received after communication with the synchronization source communication device has started, The communication device according to any one of configurations 1 to 6, characterized in that the synchronization means does not determine whether the latest average delay time is within the threshold time range until the transmission and reception of the control message reaches a first predetermined number of times after the communication has started. (Composition 8) The communication device according to configuration 7, characterized in that the setting means resets the count of the number of times the control message has been sent or received when the number of times the control message has been sent or received is greater than the number of times the first specified value has been reached. (Composition 9 places) The communication device according to configuration 8, characterized in that the second specified value is a value set according to the state of the communication path with the original communication device. (Composition 10) The communication device according to configuration 9, characterized in that the second specified value is set to a value that becomes smaller as the state of the communication channel does not fluctuate as much as the average delay time. (Composition 11) A communication device according to any one of configurations 8 to 10, characterized in that, after resetting the count of the number of times the control message has been sent and received, the second specified value is set to a value different from the value before the count was reset. (Composition 12) The aforementioned communication device that synchronizes with the device itself is a communication device equipped with a camera. A communication device according to any one of configurations 1 to 11, characterized in that it generates the shooting timing of the camera of its own device based on the synchronized time. (Method 1) The acquisition process involves sending and receiving control messages with the source communication device to obtain the average delay time during communication, A setting step of setting a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving the control message multiple times, If the latest average delay time obtained by the acquisition step is within the threshold time range, a synchronization step is performed to correct the time of the device itself based on the time difference between the latest average delay time and the time of the source communication device and synchronize it with the time of the source communication device. A method for synchronizing communication devices, characterized by having the following features. (Program 1) A program for causing a computer to function as a communication device described in any one of configurations 1 to 12. [Explanation of Symbols]

[0082] 100-103: Communication device, 201: Control unit, 202: Memory unit, 203: Clock unit, 204: Time correction unit, 205: Communication I / F unit, 206: Camera unit

Claims

1. An acquisition means that obtains the average delay time during communication by sending and receiving control messages with the source communication device, A setting means for setting a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving the control message multiple times, If the latest average delay time obtained by the acquisition means is within the threshold time range, the synchronization means corrects the time of the device itself based on the time difference between the latest average delay time and the time of the source communication device to synchronize it with the time of the source communication device. A communication device characterized by having the following features.

2. The communication device according to claim 1, wherein the acquisition means calculates the average delay time based on the time information at the time the control message was sent and received with the synchronization source communication device and the time information contained in the control message transmitted from the synchronization source communication device.

3. The system has a storage means for storing a plurality of average delay times obtained by the acquisition means, The communication device according to claim 1, wherein the setting means sets the threshold time range based on statistical information of the plurality of average delay times stored in the storage means.

4. The communication device according to claim 1, characterized in that the setting means sets the threshold time range using the minimum value of the plurality of average delay times as statistical information of the plurality of average delay times.

5. The communication device according to claim 4, characterized in that the setting means sets the time range between the minimum average delay time and the time obtained by adding a predetermined time to the minimum average delay time as the threshold time range.

6. The communication device according to claim 4, characterized in that the setting means sets a certain time range including the minimum average delay time as the threshold time range.

7. The setting means counts the number of times the control message is sent and received after communication with the synchronization source communication device has started, The communication device according to claim 1, characterized in that the synchronization means does not determine whether the latest average delay time is within the threshold time range until the transmission and reception of the control message reaches a first predetermined number of times after the communication has started.

8. The communication device according to claim 7, characterized in that the setting means resets the count of the number of times the control message has been sent or received if the number of times the control message has been sent or received is greater than the first specified value (a second specified value).

9. The communication device according to claim 8, characterized in that the second specified value is a value set according to the state of the communication path with the communication device that is the source of synchronization.

10. The communication device according to claim 9, characterized in that the second specified value is set to a value that becomes smaller as the state of the communication channel does not fluctuate as much as the average delay time.

11. The communication device according to claim 8, characterized in that, after resetting the count of the number of times the control message has been sent and received, the second specified value is set to a value different from the value before the count was reset.

12. The aforementioned communication device that synchronizes with the device itself is a communication device equipped with a camera. The communication device according to any one of claims 1 to 11, characterized in that it generates the shooting timing of the camera of the device based on the synchronized time.

13. The acquisition process involves sending and receiving control messages with the source communication device to obtain the average delay time during communication, A setting step of setting a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving the control message multiple times, If the latest average delay time obtained by the acquisition step is within the threshold time range, a synchronization step is performed to correct the time of the device itself based on the time difference between the latest average delay time and the time of the source communication device and synchronize it with the time of the source communication device. A method for synchronizing communication devices, characterized by having the following features.

14. Computers, An acquisition means that obtains the average delay time during communication by sending and receiving control messages with the source communication device, A setting means for setting a threshold time range for the average delay time based on statistical information of multiple average delay times obtained by sending and receiving the control message multiple times, If the latest average delay time obtained by the acquisition means is within the threshold time range, the synchronization means corrects the time of the device itself based on the time difference between the latest average delay time and the time of the source communication device to synchronize it with the time of the source communication device. A program that makes a device function as a communication device.

Images