Data transmission / reception device and method for controlling the same

Abstract

To provide a data transmitting / receiving device capable of synchronizing transmitted and received image data.SOLUTION: A communication time measurement unit 35 measures and stores a communication response time between a data transmission / reception device 101S and another data transmission / reception device. A transmission time storage unit 32 stores the generation time of the first packet of a frame in first image data generated by a camera 200S as a transmission time. A reception time storage unit 34 stores the generation time of the first packet of a frame in second image data transmitted from another data transmission / reception device as the reception time. A vertical synchronization deviation calculation unit 36 calculates a time obtained by subtracting a communication delay time based on the communication response time from the difference between the transmission time and the reception time as the time deviation of the vertical synchronization signals of the first and second image data. A vertical synchronization position change control unit 37 controls the camera 200S to shift the time position of the vertical synchronization signal of the first image data by the time corresponding to the deviation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a data transmission / reception device and a control method for the data transmission / reception device. [Background technology]

[0002] In recent years, image data for, for example, music lessons or sessions has been transmitted and received between distant locations using data transmission / reception devices. Two data transmission / reception devices at different locations transmit and receive image data via a wide area network (WAN). At this time, it is necessary to synchronize the image data between the two data transmission / reception devices. Patent Document 1 describes clock synchronization between a master device and a slave device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-31334 A Summary of the Invention [Problem to be solved by the invention]

[0004] One method for synchronizing image data is the Precision Time Protocol (PTP), but PTP cannot be used in a WAN. Another method for synchronizing image data is to use time information transmitted from a satellite for a Global Navigation Satellite System (GNSS), such as the Global Positioning System (GPS). However, adopting a method using time information transmitted from a satellite for a GNSS is not preferable because it results in high costs for data transmission / reception devices.

[0005] An object of the present invention is to provide a data transmission / reception device and a control method for a data transmission / reception device that can synchronize image data transmitted and received by two data transmission / reception devices with a simple configuration. [Means for solving the problem]

[0006] The present invention includes a communication time measurement unit that measures and stores a communication response time between a data transmission / reception device of the own and another data transmission / reception device, a packet transmission unit that transmits a packet of first image data generated by a camera connected to the data transmission / reception device of the own to the other data transmission / reception device, a transmission time storage unit that stores a generation time of a first packet of a frame of the first image data as a transmission time at which the packet transmission unit transmitted the frame to the other data transmission / reception device, a packet reception unit that receives a packet of second image data transmitted from the other data transmission / reception device, and a transmission time storage unit that stores a generation time of a first packet of a frame of the second image data as a transmission time at which the packet transmission unit transmitted the frame to the other data transmission / reception device. a vertical synchronization offset calculation unit that calculates a time obtained by subtracting a communication delay time from the data transmission / reception device to the other data transmission / reception device based on the communication response time measured by the communication time measurement unit from the difference between the transmission time and the reception time, as a time offset between a vertical synchronization signal of the first image data and a vertical synchronization signal of the second image data; and a vertical synchronization position change control unit that controls the camera to shift the temporal position of the vertical synchronization signal of the first image data by a time equivalent to the offset.

[0007] The present invention measures a communication response time between a data transmission / reception device of the own and another data transmission / reception device, stores the measured communication response time in a storage unit in advance, and stores in the storage unit a generation time of a first packet of a frame in first image data generated by a camera connected to the data transmission / reception device of the own as a transmission time when a packet transmission unit transmitted the frame to the other data transmission / reception device; The present invention provides a method for controlling a data transmission / reception device, which stores in a memory unit the generation time of the first packet of a frame in second image data transmitted from the other data transmission / reception device as the reception time at which a packet receiving unit received the frame, calculates the time obtained by subtracting a communication delay time from the own data transmission / reception device to the other data transmission / reception device based on the communication response time from the difference between the transmission time and the reception time, as the time deviation between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data, and controls the camera to shift the temporal position of the vertical synchronization signal of the first image data by a time equivalent to the deviation. Effect of the Invention

[0008] According to the data transmission / reception device and the control method for the data transmission / reception device of the present invention, image data transmitted and received by two data transmission / reception devices can be synchronized with a simple configuration. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a data transmission / reception system in which a data transmission / reception device as a master and a data transmission / reception device according to the first or second embodiment as a slave are connected via a network. [Diagram 2] FIG. 2 is a block diagram showing the data transmitting / receiving device according to the first embodiment. [Diagram 3] FIG. 3 is a conceptual diagram showing a method for synchronizing image data in the data transmitting / receiving device according to the first embodiment. [Figure 4] FIG. 4 is a conceptual diagram showing a collision between data transmission and data reception that may occur when a half-duplex communication system is used as a data transmission / reception system. [Diagram 5] FIG. 5 is a block diagram showing a data transmitting / receiving device according to the second embodiment. [Figure 6] FIG. 6 is a conceptual diagram showing a state in which a collision between data transmission and data reception is resolved by the data transmitting / receiving device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a data transmission / reception device and a control method for a data transmission / reception device according to the first and second embodiments will be described with reference to the accompanying drawings. First, a data transmission / reception system in which two data transmission / reception devices are connected via a network will be described with reference to FIG. 1. In FIG. 1, a data transmission / reception device 100M and a data transmission / reception device 101S or 102S are connected via a network 50. Typically, the network 50 is the Internet. The data transmission / reception device 101S is the data transmission / reception device according to the first embodiment, and the data transmission / reception device 102S is the data transmission / reception device according to the second embodiment.

[0011] A camera 200M and a monitor 300M are connected to the data transmission / reception device 100M. A camera 200S and a monitor 300S are connected to the data transmission / reception device 101S or 102S. The data transmission / reception device 100M, the camera 200M, and the monitor 300M shown in Fig. 1 are masters in the data transmission / reception system, and the data transmission / reception device 101S or 102S, the camera 200S, and the monitor 300S are slaves in the data transmission / reception system. The data transmission / reception device 100M and the data transmission / reception device 101S or 102S are data transmission / reception devices with the same configuration, and the camera 200M and the camera 200S are cameras with the same configuration.

[0012] First Embodiment The configuration and operation of the data transmission / reception device 101S according to the first embodiment will be described with reference to Fig. 2. The data transmission / reception device 101S includes an encoder 11, data buffers 12 and 22, a packet transmission unit 13, a packet reception unit 21, and a decoder 23. The encoder 11 compresses and encodes the captured image supplied from the camera 200S. The data buffer 12 stores the encoded data supplied from the encoder 11. The packet transmission unit 13 packetizes the encoded data stored in the data buffer 12 and transmits the packetized data to the data transmission / reception device 100M via the network 50. The monitor 300M shown in Fig. 1 displays a decoded image obtained by decoding the packets received by the data transmission / reception device 100M.

[0013] The packet receiving unit 21 receives packets transmitted from the data transmitting / receiving device 100M via the network 50. The data buffer 22 stores the packets received by the packet receiving unit 21. The decoder 23 decodes the packets stored in the data buffer 22 to generate a decoded image. The monitor 300S displays the decoded image.

[0014] The encoder 11 and the decoder 23 can be configured with hardware. The encoder 11 and the decoder 23 may be configured with an FPGA (Field Programmable Gate Array). The packet transmitter 13 and the packet receiver 21 can be configured with a network interface. The data buffers 12 and 22 can be configured with a memory such as a RAM.

[0015] The data transmitting / receiving device 101S further includes leading packet detection units 31 and 33, a transmission time storage unit 32, a reception time storage unit 34, a communication time measurement unit 35, a vertical synchronization deviation calculation unit 36, and a vertical synchronization position change control unit 37. The leading packet detection unit 31 to the vertical synchronization position change control unit 37 enclosed by the dashed dotted line can be configured by a central processing unit (CPU) of a microcomputer. The CPU executes software to realize functions equivalent to the leading packet detection unit 31 to the vertical synchronization position change control unit 37. The leading packet detection unit 31 to the vertical synchronization position change control unit 37 are not limited to being configured by a CPU, and may be configured by hardware such as an FPGA.

[0016] The operations of the head packet detection unit 31 to the vertical synchronization position change control unit 37 will be described with reference to Fig. 3. In Fig. 3, the time required for the camera 200M to photograph an object and generate a photographed image, and for the data transmitter / receiver 100M to encode the photographed image and transmit a packet is set to T1. Since the camera 200S and the data transmitter / receiver 101S have the same configurations as the camera 200M and the data transmitter / receiver 100M, respectively, the time required for the camera 200S to photograph an object and generate a photographed image, and for the data transmitter / receiver 101S to encode the photographed image and transmit a packet is also T1. In Fig. 3, AxM indicates the time axis on the master side, and AxS indicates the time axis on the slave side.

[0017] The communication time measurement unit 35 measures and stores the communication response time T0 between the data transmitter / receiver 101S and the data transmitter / receiver 100M in advance using a ping command. The communication time measurement unit 35 includes a storage unit. The communication response time T0 is, for example, the time required from when the data transmitter / receiver 100S transmits a message to the data transmitter / receiver 101M by the ping command until the data transmitter / receiver 100S receives a response packet returned from the data transmitter / receiver 101M. A solid arrow line from the time axis AxS to the time axis AxM indicates communication from the data transmitter / receiver 101S to the data transmitter / receiver 100M, and a dashed arrow line from the time axis AxM to the time axis AxS indicates communication from the data transmitter / receiver 100M to the data transmitter / receiver 101S. Preferably, the communication response time T0 is an average communication response time when transmitting and receiving data between the data transmitter / receiver 101S and the data transmitter / receiver 100M.

[0018] 2, the leading packet detection unit 31 detects the leading packet of each frame in the image data of the captured image (hereinafter, first image data) generated by the camera 200S. The transmission time storage unit 32 stores the occurrence time of the leading packet detected by the leading packet detection unit 31 as the transmission time at which the first image data is transmitted to the data transmission / reception device 100M (another data transmission / reception device). The leading packet detection unit 33 detects the leading packet of each frame in the image data of the captured image (hereinafter, second image data) generated by the camera 200M and transmitted from the data transmission / reception device 100M. The reception time storage unit 34 stores the occurrence time of the leading packet detected by the leading packet detection unit 33 as the reception time at which the data transmission / reception device 101S (its own data transmission / reception device) receives the second image data.

[0019] In FIG. 3, the vertical synchronization signal in the first image data is generated at time tVS, and the vertical synchronization signal in the second image data is generated at time tVM. The time required from when the camera 200S generates a captured image until the data transmitter / receiver 101S transmits the first image data, and the time required from when the camera 200M generates a captured image until the data transmitter / receiver 100M transmits the second image data is T1. Therefore, the data transmitter / receiver 101S transmits the first image data at time tsS, which is the time T1 elapsed from time tVS, which is the start timing of the frame, and the data transmitter / receiver 100M transmits the second image data at time tsM, which is the time T1 elapsed from time tVM. The data transmitter / receiver 101S receives the second image data at time trS. The data transmitter / receiver 100M and the data transmitter / receiver 101S transmit and receive image data based on their respective vertical synchronization signals.

[0020] The time from time tsS to time trS is T3. The deviation between the vertical synchronization signal in the first image data and the vertical synchronization signal in the second image data is the time TdV between time tVS and time tVM. Since the communication response time T0 is the response time when data is transmitted from the data transmission / reception device 101S to the data transmission / reception device 100M, half the communication response time T0 can be regarded as the communication delay time from the data transmission / reception device 101S to the data transmission / reception device 101M. Therefore, the time TdV can be obtained by subtracting half the communication response time T0 from the time T3. Therefore, the vertical synchronization deviation calculation unit 36 ​​calculates the time obtained by subtracting half the communication response time T0 stored in the communication time measurement unit 35 from the difference between the transmission time stored in the transmission time storage unit 32 and the reception time stored in the reception time storage unit 34 as the time deviation between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data.

[0021] The vertical synchronization position change control unit 37 controls the camera 200S to shift the temporal position of the vertical synchronization signal of the first image data by an amount of time corresponding to the temporal shift between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data calculated by the vertical synchronization deviation calculation unit 36.

[0022] According to the data transmission / reception device 101S shown in FIG. 2 and the control method executed by the data transmission / reception device 101S, image data transmitted and received by the two data transmission / reception devices, the data transmission / reception device 100M and the data transmission / reception device 101S, can be synchronized with a simple configuration.

[0023] <Second embodiment> In order to transmit images between the data transmission / reception device 100M and the data transmission / reception device 101S with low latency, a line compression codec such as JPEG XS may be used. In this case, the images transmitted and received have low latency but a high bit rate and a large image size. Also, half-duplex wireless communication such as Wi-Fi may be used in communication between the data transmission / reception device 100M and the data transmission / reception device 101S via the network 50. When transmitting and receiving high-bit-rate image data using half-duplex wireless communication, a delay occurs in receiving the image data because transmission and reception must be processed in a time-division manner.

[0024] As in the first embodiment, it is assumed that the image data transmitted and received by the two data transmitter / receivers 100M and 101S are synchronized, and the communication time of the packets is the same as the period (vertical period) of the vertical synchronization signal. In the case of progressive scanning (60p) at 60 frames per second, the vertical period is about 16.6 ms. In this case, as shown in FIG. 4, the timing at which the data transmitter / receiver 100M receives the first image data and the timing at which the data transmitter / receiver 101S receives the second image data and the timing at which the data transmitter / receiver 101S transmits the first image data overlap and collide.

[0025] In Fig. 4, the length of time T1 is shown as a different length from the length of time T1 in Fig. 3, but the two are the same length of time. The interval between the two dashed lines perpendicular to the time axes AxM and AxS is the vertical period. Half the communication response time T0, T0 / 2, is the vertical period.

[0026] In the second embodiment, when the timing of transmission and reception of image data overlaps between the data transmission and reception device 100M and the slave data transmission and reception device, a collision caused by the overlapping transmission and reception timing is avoided. Therefore, the slave data transmission and reception device is the data transmission and reception device 102S shown in FIG. 5 instead of the data transmission and reception device 101S.

[0027] 5, parts common to the data transmitting / receiving device 101S are denoted by the same reference numerals, and their description may be omitted. The data transmitting / receiving device 102S includes a transmitting / receiving collision determination unit 41. The transmitting / receiving collision determination unit 41 receives the communication response time T0 stored in the communication time measurement unit 35, the transmission times of two consecutive frames stored in the transmission time storage unit 32, and the time corresponding to the time difference between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data calculated by the vertical synchronization deviation calculation unit 36. The transmitting / receiving collision determination unit 41 calculates the vertical period of the first image data based on the difference between the transmission times of the two frames.

[0028] The transmission / reception collision determination unit 41 determines whether or not a transmission / reception collision will occur based on a time corresponding to a time difference between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data, the vertical period of the first image data, and a time T0 / 2 that is half the communication response time T0 stored in the communication time measurement unit 35. For example, if there is no time difference between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data, the transmission / reception collision determination unit 41 determines that a collision will occur when the vertical period and the time T0 / 2 are less than a predetermined time difference. Note that the occurrence of a collision includes a case where there is a high possibility of a collision occurring.

[0029] If the transmission / reception collision determination unit 41 does not determine that a transmission / reception collision will occur, the vertical synchronization position change control unit 37 controls the camera 200S to shift the temporal position of the vertical synchronization signal of the first image data by a time corresponding to the temporal shift between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data calculated by the vertical synchronization deviation calculation unit 36, as in the first embodiment. If the transmission / reception collision determination unit 41 determines that a transmission / reception collision will occur, the vertical synchronization position change control unit 37 controls the camera 200S to shift the temporal position of the vertical synchronization signal of the first image data by a predetermined time so as not to cause a collision.

[0030] For example, in the case of progressive scanning (120p) at 120 frames per second, the vertical period is approximately 8.3 ms. If the distance between the master and slave is the distance between Tokyo and Osaka, the communication response time T0 measured using the ping command is approximately 16 ms. Therefore, there is a possibility of a transmission / reception collision occurring, as shown in Figure 4.

[0031] If the time difference between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data is approximately ±0 ms, the vertical synchronization position change control unit 37 controls the camera 200S to shift the vertical synchronization signal of the first image data by, for example, +4.15 ms, which is half of the vertical period of approximately 8.3 ms. By shifting the vertical synchronization signal by half the vertical period, it is possible to almost certainly avoid collisions even if the vertical synchronization signal is shifted due to jitter or the like. However, the time for shifting the vertical synchronization signal is not limited to half the vertical period.

[0032] Furthermore, if the time lag between the vertical synchronization signal of the first image data and the vertical synchronization signal of the second image data is, for example, +3 ms, the vertical synchronization position change control unit 37 controls the camera 200S to shift the signals by +1.15 ms, which is obtained by subtracting 3 ms from 4.15 ms.

[0033] 6 shows a state in which the vertical synchronization signal of the first image data is shifted by the vertical synchronization position change control unit 37 by the time T4. By shifting the vertical synchronization signal by the time T4, the vertical synchronization position change control unit 37 avoids a collision between the transmission of the first image data and the reception of the second image data in the data transmission / reception device 102S.

[0034] 5, the data transmitter / receiver 102S and the control method executed by the data transmitter / receiver 102S can synchronize image data transmitted and received by the two data transmitter / receivers, the data transmitter / receiver 100M and the data transmitter / receiver 102S, with a simple configuration. Also, the data transmitter / receiver 102S and the control method executed by the data transmitter / receiver 102S can avoid a collision when the transmission of the first image data and the reception of the second image data collide.

[0035] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0036] 11 Encoders 12,22 Data Buffer 13 Packet transmitter 21 Packet receiver 23 Decoder 31,33 First packet detector 32 Transmission time memory section 34 Reception time memory section 35 Communication Time Measurement Unit 36 Vertical sync offset calculation unit 37 Vertical sync position change control section 41 Transmission / reception collision determination unit 50 Network 100M, 101S, 102S data transmission and reception device 200M, 200S Camera 300M, 300S Monitor

Claims

1. a communication time measurement unit that measures and stores a communication response time between the data transmission / reception device itself and another data transmission / reception device; a packet transmission unit that transmits a packet of first image data generated by a camera connected to the data transmission / reception device to the other data transmission / reception device; a transmission time storage unit that stores a generation time of a first packet of a frame in the first image data as a transmission time at which the packet transmission unit transmitted the frame to the other data transmission / reception device; a packet receiving unit that receives a packet of second image data transmitted from the other data transmitting / receiving device; a reception time storage unit that stores a generation time of a first packet of a frame in the second image data as a reception time at which the packet receiving unit received the frame; a vertical synchronization deviation calculation unit that calculates a time obtained by subtracting a communication delay time from the data transmission / reception device to the other data transmission / reception device based on the communication response time measured by the communication time measurement unit from a difference time between the transmission time and the reception time, as a time deviation between a vertical synchronization signal of the first image data and a vertical synchronization signal of the second image data; a vertical synchronization position change control unit that controls the camera to shift a time position of the vertical synchronization signal of the first image data by a time corresponding to the shift; A data transmitting and receiving device comprising:

2. the communication time measurement unit measures and stores an average communication response time between the data transmission / reception device and the other data transmission / reception device; The vertical synchronization deviation calculation unit calculates a half time of the average communication response time as the communication delay time.

2. The data transmitting / receiving device according to claim 1.

3. Measure a communication response time between the data transmitting / receiving device itself and another data transmitting / receiving device and store the measured time in a storage unit in advance; storing in a storage unit a generation time of a first packet of a frame in first image data generated by a camera connected to said own data transmission / reception device as a transmission time at which a packet transmission unit transmitted said frame to said other data transmission / reception device; storing in a storage unit a generation time of a first packet of a frame in the second image data transmitted from the other data transmitting / receiving device as a reception time at which the packet receiving unit received the frame; calculating a time obtained by subtracting a communication delay time from the data transmission / reception device to the other data transmission / reception device based on the communication response time from a difference between the transmission time and the reception time as a time deviation between a vertical synchronization signal of the first image data and a vertical synchronization signal of the second image data; The camera is controlled so as to shift the time position of the vertical synchronization signal of the first image data by a time period corresponding to the shift. A method for controlling a data transmitting / receiving device.

Images