Timing adjustment method

Abstract

To provide a timing adjustment method that can adjust to match the timings in displaying, at a reference site, pieces of image data obtained by photographing subjects carrying out predetermined movements that are individually carried out at a transmission-side site and the reference site.SOLUTION: A timing adjustment method includes: transmitting image data generated by a camera 11 included in an image transmitting device arranged at a transmission-side site photographing a subject who carries out a predetermined movement to an image receiving device arranged at a reference site via a network 30, where the image receiving device measures a delay time when displaying the received image data on a display 22; setting a delay time correction value based on the delay time to a timing instruction unit 29 arranged at the transmission-side site; and indicating, by a timing indication unit 29, a timing to start the predetermined movement, to the subject, at a time earlier than a time when the subject is instructed to start the predetermined movement by a time indicated by the delay time correction value.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to a timing adjustment method. [Background technology]

[0002] Image data is transmitted from an image transmitting device located at at least one base to an image receiving device via a network, and the image transmitted from the image transmitting device is displayed on a display device provided in the image receiving device. In this case, the practical application of the fifth generation mobile communication system (5G) makes it possible to transmit image data with low latency, so the delay due to the transmission of image data is negligible. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-88305 Summary of the Invention [Problem to be solved by the invention]

[0004] However, delays due to internal processing in the image transmitting device and the image receiving device cannot be ignored. In addition, when image transmitting devices are installed at multiple locations and image data is transmitted from the multiple image transmitting devices to the image receiving device, delay times due to internal processing in the multiple image transmitting devices vary, which may cause timing of images displayed on the display device to vary, resulting in an uncomfortable feeling. It is required to grasp the delay time including delays due to internal processing in the image transmitting device and the image receiving device when transmitting image data from the image transmitting device to the image receiving device via a network.

[0005] The present invention aims to provide a timing adjustment method that can adjust the timing when image data captured of a subject performing a specified action performed separately at a sending location and a reference location is displayed at the reference location to match. [Means for solving the problem]

[0006] The present invention relates to a camera provided in an image transmission device disposed at a transmission side base, Play an instrument Image data generated by photographing a subject performing an action is transmitted to an image receiving device disposed at a reference base via a network, a delay time when the image receiving device displays the received image data on a display device is measured, a delay time correction value based on the delay time is set in a timing instruction unit disposed at the transmitting side base, and the subject is caused to perform the Play an instrument The delay time correction value is set to the time before the time when the operation is instructed to start. Play an instrument To provide a timing adjustment method for signaling a subject to inform the subject of the timing of starting an action. Effect of the Invention

[0007] According to the timing adjustment method of the present invention, it is possible to adjust the timing when image data captured of a subject performing a specified action, which is performed separately at the sending location and the reference location, is displayed at the reference location. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram conceptually showing an image transmission system in which image data is transmitted from a plurality of image transmitting devices to an image receiving device via a network. [Diagram 2] FIG. 2 is a block diagram showing an example configuration of an image transmitting device and an image receiving device configured to measure the delay time from the time when a camera in the image transmitting device starts capturing a moving image to the time when a display device in the image receiving device starts displaying the moving image. [Diagram 3] FIG. 3 is a diagram showing the 1 PPS signal and the 10 MHz clock generated by the GNSS satellite wave clock 13 in FIG. 2, and the 148.5 MHz clock generated by the clock generating unit 14. In FIG. [Figure 4] FIG. 4 is a diagram showing the 1 PPS signal and the 10 MHz clock generated by the GNSS satellite wave clock 23 in FIG. 2, and the 148.5 MHz clock generated by the clock generating unit 24. [Diagram 5] FIG. 5 is a flowchart showing the process executed by the delay time measurement unit 28 in FIG. [Figure 6] FIG. 6 is a flowchart showing the detailed process of step S1 in FIG. [Figure 7] FIG. 7 is a flowchart showing the detailed process of step S2 in FIG. [Figure 8] FIG. 8 is a block diagram showing a delay time management server that manages delay times when image data is transmitted and received bidirectionally between image transmitting / receiving devices arranged at three locations. [Figure 9] FIG. 9 is a diagram showing an example of the delay time stored in the delay time management server shown in FIG. [Figure 10] FIG. 10 is a block diagram showing an image transmitting / receiving device that adjusts the timing of giving a signal to start a predetermined operation based on the delay time stored in a delay time management server. [Figure 11] FIG. 11 is a block diagram showing a specific example of the configuration of the timing instruction section 29 in FIG. [Figure 12] FIG. 12 is a flowchart showing a process executed by the image transmitting / receiving devices arranged at the three locations shown in FIG. 10 to synchronize the reception timing of image data. [Figure 13] FIG. 13 is a block diagram showing an image transmitting device configured to correct the timing shift of image data frames, and a delay time management server suitable for correcting the timing shift of image data frames. [Figure 14]FIG. 14 is a diagram showing an example of the delay time stored in the delay time management server shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an image transmitting device, an image receiving device, a delay time measuring device, a delay time management server, and a timing adjustment method according to one or more embodiments will be described with reference to the accompanying drawings.

[0010] First, an image transmission system that transmits image data from multiple image transmitting devices to an image receiving device via a network will be conceptually explained using Fig. 1. The image data is moving image data. The image transmission system may transmit audio data in addition to the image data. The process of transmitting and receiving the audio data is not shown in the figure.

[0011] 1, image transmitting devices 10A to 10C are installed at three different locations. An image receiving device 20 is installed at a location different from the locations where the image transmitting devices 10A to 10C are installed. The image transmitting devices 10A to 10C each include a camera 11 that captures an image of a subject, and a transmitting unit 12 that transmits image data generated by the camera 11 capturing an image of the subject.

[0012] Image data transmitted from the image transmitting devices 10A to 10C is transmitted to the image receiving device 20 via a network 30. The network 30 is typically the Internet. The image receiving device 20 includes a receiving unit 21 that receives image data, and a display device 22 that displays a moving image based on the image data. The display device 22 may simultaneously display the moving images from the image transmitting devices 10A to 10C, or may display the moving images from the image transmitting devices 10A to 10C while switching between them. An image transmitting device that does not specify which of the image transmitting devices 10A to 10C is referred to as an image transmitting device 10.

[0013] <Specific examples of configurations of an image transmitting device, an image receiving device, and a delay time measuring device> Fig. 2 shows an example of the configuration of the image transmitting device 10 and the image receiving device 20 configured to measure the delay time from the time when the camera 11 in the image transmitting device 10 starts capturing a moving image to the time when the display device 22 in the image receiving device 20 starts displaying the moving image. The image transmitting device 10 and the image receiving device 20 shown in Fig. 2 have a suitable configuration for grasping the delay time including the delay due to their internal processing.

[0014] The delay time measurement unit 28 included in the image receiving device 20 constitutes a delay time measurement device of one or more embodiments, and measures a more accurate delay time including delays due to internal processing in the image transmitting device 10 and the image receiving device 20 as described below. The image transmitting device 10 is an image transmitting device of one or more embodiments. The image receiving device 20 is an image receiving device of one or more embodiments.

[0015] 2, the image transmitting device 10 includes a GNSS satellite radio clock 13, a clock generating unit 14, an image memory 15, and a communication unit 16 in addition to the camera 11. The camera 11 has a control unit 111, an image sensor 112, and an image processing circuit 113. The image memory 15 includes a write / read control unit 150. The communication unit 16 functions as the transmission unit 12 in FIG.

[0016] In addition to the display device 22, the image receiving device 20 includes a GNSS satellite radio clock 23, a clock generating unit 24, a communication unit 25, an image processing circuit 26, an image memory 27, and a delay time measuring unit 28. The display device 22 has a control unit 221, a drive circuit 222, and a liquid crystal panel 223. The image memory 27 includes a write / read control unit 270. The communication unit 25 functions as the receiving unit 21 in FIG.

[0017] The GNSS satellite radio clock 13 receives radio waves from a satellite 40 for the Global Navigation Satellite System (GNSS), and outputs a 1PPS signal which is a pulse with a one-second interval as shown in (a) of Fig. 3, and a 10MHz clock (hereinafter, 10MHz clock) as shown in (b) of Fig. 3. One example of the GNSS is the Global Positioning System (GPS).

[0018] The clock generating unit 14 generates a 148.5 MHz clock (hereinafter referred to as the 148.5 MHz clock) shown in Fig. 3(c) based on the input 1 PPS signal and 10 MHz clock, and supplies it to the camera 11, image memory 15, and communication unit 16. For convenience of illustration, the 148.5 MHz clock shown in Fig. 3(c) is shown with one clock period significantly extended. Note that the clock generating unit 14 may generate clocks of other video synchronization frequencies, such as 297 MHz, 74.25 MHz, 27 MHz, etc., in addition to 148.5 MHz.

[0019] The image transmitting device 10 may include a clock generating unit that generates a clock including time information based on radio waves received from a GNSS satellite 40. A 10 MHz clock may be supplied to the camera 11, image memory 15, and communication unit 16 to operate the camera 11, image memory 15, and communication unit 16 on the 10 MHz clock. In this case, the GNSS satellite radio clock 13 serves as the clock generating unit that generates a clock including time information. The frequency of the clock supplied to the camera 11, image memory 15, and communication unit 16 is not limited.

[0020] Assume that the GNSS satellite radio clock 13 generates a pulse of the 1PPS signal shown in FIG. 3(a) at time 0:00:00, and generates the next pulse at time 0:00:01. The clock generating unit 14 generates clock numbers 0 to 148.5×10 based on the time 0:00:00. -6 -1 clock, clock number 0 to 148.5 x 10 based on time 0:00:01 -6 Like the -1 clock, it outputs a 148.5MHz clock that includes time information.

[0021] A control unit 111 of the camera 11 controls an electronic shutter in the image sensor 112, and the image sensor 112 captures an image of a subject. The control unit 111 may be a central processing unit provided in the camera 11. Image data generated by the image sensor 112 capturing an image of a subject is subjected to various types of image processing by an image processing circuit 113 and supplied to an image memory 15.

[0022] The image processing in the image processing circuit 113 includes one or more of defective pixel interpolation processing, black level processing, white balance processing, demosaic processing, shading correction processing, and compression encoding processing. The image processing in the image processing circuit 113 includes at least compression encoding processing.

[0023] The control unit 111 controls so that the clock number of the 148.5 MHz clock indicating the time t0 when the image sensor 112 released the electronic shutter of a predetermined reference pixel to generate pixel data is superimposed as metadata on the blanking period of the image data. The reference pixel is preferably the first pixel in the first row in the frame. That is, the clock number included in the image data output by the camera 11 indicates the time when the camera 11 started generating image data for each frame. As shown in (c) of FIG. 3, the clock number indicating the time t0 included in the image data output by the camera 11 is assumed to be Cn0, which is clock number 0, as an example.

[0024] The write / read control unit 150 writes image data into the image memory 15. At this time, the write / read control unit 150 writes image data into the image memory 15 in a state in which the clock number of the 148.5 MHz clock indicating the time t1 at which pixel data of the reference pixel of each frame was written into the image memory 15 is superimposed as metadata onto the blanking period of the image data. As shown in FIG. 3(c), it is assumed that the clock number indicating the time t1 contained in the image data written into the image memory 15 is Cn1. The image data written into the image memory 15 has the clock numbers Cn0 and Cn1 superimposed thereon.

[0025] The write / read control unit 150 reads out the image data stored in the image memory 15 and supplies it to the communication unit 16. The communication unit 16 packetizes the image data read out from the image memory 15 and transmits it to the image receiving device 20 via the network 30. At this time, the communication unit 16 packetizes the image data in a state in which the clock number of the 148.5 MHz clock indicating the time t2 at which pixel data of the reference pixel of each frame is transmitted is superimposed as metadata on the blanking period of the image data. As shown in FIG. 3(c), the clock number indicating the time t2 at which pixel data of the reference pixel is transmitted is assumed to be Cn2. As a result, the clock number Cn2 indicates the time t2 at which the pixel data of the reference pixel is transmitted.

[0026] Furthermore, communication unit 16 adds clock frequency information to image data and packetizes the image data to inform image receiving device 20 that camera 11, image memory 15, and communication unit 16 are operating at a 148.5 MHz clock. The packet data transmitted from image transmitting device 10 to image receiving device 20 via network 30 includes image data and clock frequency information, and clock numbers Cn0, Cn1, and Cn2 are superimposed on the image data.

[0027] Like the GNSS satellite radio clock 13, the GNSS satellite radio clock 23 in the image receiving device 20 receives radio waves from the satellite 40 and outputs a 1PPS signal, which is a pulse at 1-second intervals as shown in (a) of Figure 4, and a 10 MHz clock as shown in (b) of Figure 4.

[0028] 2, the satellite 40 that transmits radio waves received by the GNSS satellite radio clock 13 and the GNSS satellite radio clock 23 is a single common satellite 40. The satellite 40 that transmits radio waves received by the GNSS satellite radio clock 13 may be different from the satellite 40 that transmits radio waves received by the GNSS satellite radio clock 23. The GNSS satellite radio clocks 13 and 23 may receive radio waves from any satellite 40 among multiple satellites 40 operated by one global navigation satellite system.

[0029] The communication unit 25 receives packet data including image data and clock frequency information, supplies the clock frequency information to the clock generation unit 24, and supplies the image data to the image processing circuit 26. Since the clock frequency information indicating 148.5 MHz is input to the clock generation unit 24, it generates the 148.5 MHz clock shown in Fig. 4(c) based on the input 1 PPS signal and 10 MHz clock. The 148.5 MHz clock is supplied to the display device 22, the communication unit 25, the image memory 27, and the delay time measurement unit 28.

[0030] The 148.5 MHz clock shown in FIG. 4(c) is the same clock (same phase and same frequency) as the 148.5 MHz clock shown in FIG. 3(c).

[0031] The clock supplied from the clock generating unit 24 to the display device 22, the communication unit 25, the image memory 27, and the delay time measuring unit 28 is a 148.5 MHz clock including time information, as in the above.

[0032] 4(c), the communication unit 25 superimposes, as metadata, the clock number of the 148.5 MHz clock indicating the time t3 at which pixel data of the reference pixel of each frame was received from the network 30 onto the blanking period of the image data. The clock number indicating the time t3 at which pixel data of the reference pixel was received is assumed to be Cn3. Therefore, the image data output from the communication unit 25 has the clock numbers Cn0, Cn1, Cn2, and Cn3 superimposed thereon.

[0033] The image data output from the communication unit 25 is subjected to various image processing by the image processing circuit 26, and is supplied to the image memory 27. The image processing in the image memory 27 is at least a compression decoding process. The image processing circuit 26 supplies the clock numbers Cn0, Cn1, Cn2, and Cn3 included in the image data to the delay time measurement unit 28.

[0034] The write / read control unit 270 writes the image data supplied to the image memory 27 into the image memory 27, and reads out the image data stored in the image memory 27 and supplies it to the drive circuit 222 of the display device 22. At this time, the write / read control unit 270 supplies the clock number of the 148.5 MHz clock indicating the time t4 at which the pixel data of the reference pixel of each frame is read out to the delay time measurement unit 28. The time t4 is also the time at which the pixel data of the reference pixel is supplied to the drive circuit 222. As shown in (c) of Fig. 4, the clock number indicating the time t4 at which the image data is read out from the image memory 27 is assumed to be Cn4.

[0035] The control unit 221 of the display device 22 controls the drive circuit 222 so as to display the image data supplied to the drive circuit 222 on the liquid crystal panel 223. The control unit 221 may be a central processing unit included in the display device 22. The control unit 221 supplies the clock number of the 148.5 MHz clock indicating the time t5 at which the pixel data of the reference pixel of each frame is displayed on the liquid crystal panel 223 to the delay time measurement unit 28. As shown in (c) of Fig. 4, it is assumed that the clock number indicating the time t5 at which the pixel data of the reference pixel is displayed on the liquid crystal panel 223 is Cn5.

[0036] It is not necessary for the display device 22 to operate on the 148.5 MHz clock. The 148.5 MHz clock is supplied to the display device 22 in order to supply the delay time measurement unit 28 with the clock number Cn5 indicating the time t5 in the 148.5 MHz clock.

[0037] When the display device 22 is not a direct-view display device but a projection-type display device, the liquid crystal panel 223 is a liquid crystal display element for a projection-type display device. The display device 22 may include a display panel (display element) other than a liquid crystal panel.

[0038] The delay time measurement unit 28 obtains the delay time between the image transmitting device 10 and the image receiving device 20 according to the processes shown in the flowcharts of FIGS. 5 to 7, and determines a delay time correction value for correcting the delay due to the delay time.

[0039] In Fig. 5, the delay time measurement unit 28 sets the shooting time in the image transmitting device 10 in step S1, and sets the display time in the image receiving device 20 in step S2. Fig. 6 shows detailed processing of step S1. In Fig. 6, the delay time measurement unit 28 determines whether or not clock number Cn0 (i.e., time t0) exists in step S11. If clock number Cn0 exists (YES), the delay time measurement unit 28 sets the time t0 indicated by clock number Cn0 as the shooting time in step S13.

[0040] If the clock number does not exist in step S11 (NO), the delay time measurement unit 28 determines whether or not the clock number Cn1 (i.e., time t1) exists in step S12. If the clock number Cn1 exists (YES), the delay time measurement unit 28 sets the time t1 indicated by the clock number Cn1 as the shooting time in step S14. If the clock number Cn1 does not exist (NO), the delay time measurement unit 28 sets the time t2 indicated by the clock number Cn2 as the shooting time in step S15.

[0041] Fig. 7 shows the detailed process of step S2. In Fig. 7, the delay time measurement unit 28 judges whether or not the clock number Cn5 (i.e., the time t5) exists in step S21. If the clock number Cn5 exists (YES), the delay time measurement unit 28 sets the time t5 as the display time in step S23.

[0042] If clock number Cn5 does not exist in step S21 (NO), the delay time measurement unit 28 determines whether clock number Cn4 (i.e., time t4) exists in step S22. If clock number Cn4 exists (YES), the delay time measurement unit 28 sets time t4 as the display time in step S24. If clock number Cn4 does not exist (NO), the delay time measurement unit 28 sets time t3 indicated by clock number Cn3 as the display time in step S25.

[0043] 6, the presence or absence of clock number Cn0 is determined because the camera 11 may not be configured to superimpose clock number Cn0, which indicates time t0 when the camera 11 starts generating image data for each frame, on the image data.The presence or absence of clock number Cn1 is determined because the camera 11 may not be configured to superimpose clock number Cn1, which indicates time t1 when the image memory 15 writes the image data, on the image data.

[0044] 7, the presence or absence of clock number Cn5 is determined because there may be cases where the display device 22 is not configured to supply the delay time measuring unit 28 with clock number Cn5 indicating time t5 at which image data was displayed on the liquid crystal panel 223. Also, the presence or absence of clock number Cn4 is determined because there may be cases where the display device 22 is not configured to supply the delay time measuring unit 28 with clock number Cn4 indicating time t4 at which image memory 27 reads out image data.

[0045] Returning to Fig. 5, in step S3, delay time measurement unit 28 calculates the delay time between image transmitting device 10 and image receiving device 20, which is the difference between the shooting time and the display time. In step S4, delay time measurement unit 28 determines whether the shooting time is time t2 and the display time is time t3. If the shooting time is time t2 and the display time is time t3, the time from time 2 to time t3 is the transmission time of the image data over network 30 itself. Therefore, it does not represent the delay time from when image transmitting device 10 starts generating image data to when image receiving device 20 starts displaying the image data.

[0046] Therefore, if the shooting time is time t2 and the display time is time t3 in step S4 (YES), the delay time measurement unit 28 determines in step S5 whether or not a pre-measured fixed correction value is set in the delay time measurement unit 28. The fixed correction value is the sum of either the average time from time t0 to time t2 or the average time from time t3 to time t5, or both.

[0047] If a fixed correction value measured in advance is set in step S5 (YES), the delay time obtained in step S3 is added with the fixed correction value, and the delay time correction value is determined as the delay time correction value, and the process ends.

[0048] If in step S4 the shooting time is not time t2 and the display time is not time t3 (NO), or if in step S5 a fixed correction value measured in advance has not been set (NO), the delay time measurement unit 28 determines in step S7 the delay time obtained in step S3 as the delay time correction value as is, and ends the process.

[0049] As described above, the image transmitting device 10 includes the camera 11, the clock generating unit 14, the image memory 15, and the communication unit 16. The clock generating unit 14 generates a first clock (for example, a 148.5 MHz clock) including time information based on radio waves received from a GNSS satellite 40. The camera 11 operates according to the first clock, and generates image data by the image sensor 112 capturing an image of a subject. The image memory 15 operates according to the first clock, and writes and reads image data under the control of the write / read control unit 150.

[0050] The communication unit 16 transmits image data that is read out from the image memory 15 and onto which at least one of the first to third clock numbers is superimposed. The first to third clock numbers are clock numbers Cn0 to Cn2.

[0051] The first clock number indicates the time when the image sensor 112 of the camera 11 generated pixel data for a reference pixel in a frame. If the camera 11 is configured to execute a process of superimposing the first clock number on the image data, the image data including the first clock number is transmitted.

[0052] The second clock number indicates the time when the write / read control unit 150 wrote the pixel data of the reference pixel into the image memory 15. If the write / read control unit 150 is configured to execute a process of superimposing the second clock number on the image data, image data including the second clock number is transmitted. The third clock number indicates the time when the communication unit 16 transmitted the pixel data of the reference pixel. If the communication unit 16 is configured to execute a process of superimposing the third clock number on the image data, image data including the third clock number is transmitted.

[0053] The image receiving device 20 includes a communication unit 25 that receives image data on which at least one of the first clock number, the second clock number, and the third clock number is superimposed. If the communication unit 25 is considered to be a first communication unit, the communication unit 16 is a second communication unit. If the communication unit 16 is considered to be the first communication unit, the communication unit 25 is a second communication unit.

[0054] The first clock number is a clock number in the first clock generated by clock generation unit 14 (first clock generation unit) and indicates the time when image sensor 112 of camera 11 generated pixel data of a reference pixel in a frame. The second clock number is a clock number in the first clock and indicates the time when write / read control unit 150 (first write / read control unit) wrote the pixel data of the reference pixel to image memory 15 (first image memory). The third clock number is a clock number in the first clock and indicates the time when communication unit 16 (second communication unit) transmitted the pixel data of the reference pixel.

[0055] The image receiving device 20 further includes a clock generating unit 24 (second clock generating unit), an image memory 27 (second image memory), a display device 22, and a delay time measuring unit 28. The clock generating unit 24 generates a second clock that is the same as the first clock. The image memory 27 operates according to the second clock, and writes and reads image data based on the control of a write / read control unit 270 (second write / read control unit). The display device 22 displays the image data read from the image memory 27.

[0056] The delay time measurement unit 28 measures a delay time indicating the difference between any one of the first to third clock numbers and any one of the fourth to sixth clock numbers. The fourth clock number is a clock number in the second clock, and indicates the time when the communication unit 25 received the pixel data of the reference pixel. The fifth clock number is a clock number in the second clock, and indicates the time when the write / read control unit 270 reads out the pixel data of the reference pixel from the image memory 27. The sixth clock number is a clock number in the second clock, and indicates the time when the display device 22 displays the pixel data of the reference pixel.

[0057] When the delay time measurement unit 28 acquires the first clock number (clock number Cn0), it is preferable for the delay time measurement unit 28 to measure the delay time using the first clock number. When the delay time measurement unit 28 is unable to acquire the first clock number and acquires the second clock number (clock number Cn1), it is preferable for the delay time measurement unit 28 to measure the delay time using the second clock number. When the delay time measurement unit 28 acquires the sixth clock number (clock number Cn5), it is preferable for the delay time measurement unit 28 to measure the delay time using the sixth clock number. When the delay time measurement unit 28 is unable to acquire the sixth clock number and acquires the fifth clock number (clock number Cn4), it is preferable for the delay time measurement unit 28 to measure the delay time using the fifth clock number.

[0058] The delay time measurement unit 28 is a delay time measurement device according to one or more embodiments. As described above, the delay time measurement unit 28 obtains the difference between any one of the first to third clock numbers and any one of the fourth to sixth clock numbers. In this way, the delay time measurement unit 28 measures the delay time when the image receiving device 20 receives and displays image data generated by the image transmitting device 10 and transmitted to the image receiving device 20.

[0059] According to one or more embodiments of the image transmitting device, image receiving device, and delay time measuring device, it is possible to grasp a more accurate delay time when transmitting image data from the image transmitting device 10 to the image receiving device 20 via the network 30.

[0060] <Delay time management server> 8, image transmitting / receiving devices 120A to 120C are disposed at locations B1 to B3, respectively. The image transmitting / receiving devices 120A to 120C communicate with each other bidirectionally via a network 30. An image transmitting / receiving device without specifying which of the image transmitting / receiving devices 120A to 120C is referred to as an image transmitting / receiving device 120. The image transmitting / receiving device 120 has the configuration of both the image transmitting device 10 and the image receiving device 20 shown in FIG.

[0061] As shown in Fig. 8, the image transmitting / receiving device 120 generally includes the camera 11, the display device 22, a delay time measuring unit 28, a control unit 122, and a transmitting / receiving unit 121. Although not shown in Fig. 8, the image transmitting / receiving device 120 includes a GNSS satellite radio clock 13 (or 23) and a clock generating unit 14 (or 24), and a 148.5 MHz clock is supplied to the camera 11 and the display device 22. The transmitting / receiving unit 121 corresponds to the communication units 16 and 25. The control unit 122 corresponds to the control units 111 and 221.

[0062] The image transmitting device 10 shown in Fig. 2 is a detailed description of the configuration when the image transmitting / receiving device 120 shown in Fig. 8 operates as an image transmitting device. The image receiving device 20 shown in Fig. 2 is a detailed description of the configuration when the image transmitting / receiving device 120 shown in Fig. 8 operates as an image receiving device.

[0063] A delay time management server 50 is connected to the network 30. The delay time management server 50 communicates bidirectionally with the image transmitting and receiving devices 120A to 120C. As shown in Fig. 9, the delay time management server 50 stores delay times when the image transmitting and receiving devices 120A to 120C arranged at the bases B1 to B3 are the transmitting side and the other image transmitting and receiving devices 120 are the receiving side. In Fig. 9, information indicating the delay time from which time among times t0 to t2 to which time among times t3 to t5 the delay time extends is written together with the delay time.

[0064] The delay time management server 50 may update the delay time shown in Fig. 9 every time the delay time measurement unit 28 of the image transmitting and receiving device 120A to 120C measures the delay time. The delay time management server 50 may update the delay time shown in Fig. 9 to the latest delay time measured by the delay time measurement unit 28 every time a predetermined time has elapsed. The image transmitting and receiving device 120 can read out the delay time stored in the delay time management server 50.

[0065] As described above, one or more delay time management servers, ie, delay time management servers 50, are connected to the network 30. At least two image transmitting / receiving devices 120, ie, the first and second image transmitting / receiving devices, are connected to the network 30 so as to communicate with each other bidirectionally. Any two image transmitting / receiving devices 120 among the image transmitting / receiving devices 120A to 120C are the first and second image transmitting / receiving devices.

[0066] The first image transmitting / receiving device includes a first delay time measurement unit (delay time measurement unit 28) that measures a first delay time when the first image transmitting / receiving device receives and displays image data generated by the second image transmitting / receiving device and transmitted to the first image transmitting / receiving device. The second image transmitting / receiving device includes a second delay time measurement unit (delay time measurement unit 28) that measures a second delay time when the second image transmitting / receiving device receives and displays image data generated by the first image transmitting / receiving device and transmitted to the second image transmitting / receiving device.

[0067] The delay time management server 50 stores a first delay time transmitted from a first image transmitting / receiving device via a network 30 and a second delay time transmitted from a second image transmitting / receiving device via the network 30 .

[0068] According to one or more embodiments of the delay time management server 50, it is possible to more accurately manage the delay time when transmitting image data from an image transmitting / receiving device 120 operating as an image transmitting device to an image transmitting / receiving device 120 operating as an image receiving device via a network 30.

[0069] <Timing adjustment method when receiving image data> As an example, in Fig. 10, a musical instrument is played at each of the locations B1 to B3, and the image transmitting / receiving device 120B at the location B2 where the performance supervisor is located receives image data transmitted from the image transmitting / receiving devices 120A and 120C. Playing a musical instrument is an example of a predetermined action performed by a person who is a subject. The subject who plays the musical instrument is a performer.

[0070] The camera 11 of the image transmitting / receiving device 120A photographs the performers at location B1, and the transmitting / receiving unit 121 transmits the image data to the image transmitting / receiving device 120B. The camera 11 of the image transmitting / receiving device 120C photographs the performers at location B3, and the transmitting / receiving unit 121 transmits the image data to the image transmitting / receiving device 120B. The camera 11 of the image transmitting / receiving device 120B photographs the performers at location B2, and the display device 22 displays a video. The display device 22 of the image transmitting / receiving device 120B may simultaneously display the video generated by the image transmitting / receiving devices 120A to 120C, or may switch between the video generated by the image transmitting / receiving devices 120A to 120C.

[0071] The image transmitting / receiving devices 120A to 120C each include a timing instruction unit 29 for signaling the start of performance to the performers located at the locations B1 to B3, respectively. As shown in Fig. 11, the timing instruction unit 29 has a control unit 291, a clock 292, an operation unit 293, a delay time correction value storage unit 294, and a display unit 295. The clock 292 may be the GNSS satellite radio clock 13 (or 23). The display unit 295 may be a liquid crystal panel or an LED.

[0072] The timing instruction unit 29 is configured such that, when the time to start performance is set in the control unit 291 by the operation unit 293, the control unit 291 monitors the time measured by the clock 292 and displays a signal notifying the timing to start performance on the display unit 295. For example, if performance starts at each of the locations B1 to B3 at 13:00:00, there is the above-mentioned delay time until the image data is transmitted from the locations B1 and B3 to the location B2 and displayed on the display device 22, so the timing of the performances at all of the locations B1 to B3 displayed at the location B2 will not be the same.

[0073] 9 from the delay time management server 50 under the control of the control unit 122, and supplies the delay time to the control unit 291 of the timing instruction unit 29. The control unit 291 causes the delay time correction value storage unit 294 to store a delay time correction value based on the delay time. Here, a case where the delay time is directly used as the delay time correction value is taken as an example.

[0074] 10, the delay time correction value storage unit 294 of the timing instruction unit 29 included in the image transmitting / receiving device 120A stores a delay time of 10.3 ms with location B1 as the transmitting side and location B2 as the receiving side. The delay time correction value storage unit 294 of the timing instruction unit 29 included in the image transmitting / receiving device 120C stores a delay time of 10.2 ms with location B3 as the transmitting side and location B2 as the receiving side.

[0075] There is no need to store a delay time correction value in the delay time correction value storage unit 294 of the timing instruction unit 29 included in the image transmitting / receiving device 120B. In Fig. 10, since a delay time is not supplied from the transmitting / receiving unit 121 to the timing instruction unit 29, the connection between the transmitting / receiving unit 121 and the timing instruction unit 29 is indicated by a dashed arrow line.

[0076] The control unit 291 of the timing indication unit 29 provided in the image transmitting / receiving device 120A displays on the display unit 295 a cue notifying the timing to start performance at a delay time of 10.3 ms before the performance start time of 13:00:00. The control unit 291 of the timing indication unit 29 provided in the image transmitting / receiving device 120C displays on the display unit 295 a cue notifying the timing to start performance at a delay time of 10.2 ms before the performance start time of 13:00:00.

[0077] The control unit 291 of the timing instruction unit 29 included in the image transmitting / receiving device 120B displays on the display unit 295 a cue informing the user of the timing to start playing at 13:00:00, which is the time to start playing.

[0078] The performers at each of the locations B1 to B3 start playing in sync with the cues displayed on the display units 295 of the timing indicators 29 at each of the locations B1 to B3. As a result, the image data displayed on the display device 22 at the location B2 is displayed in a state where the performance timings at all of the locations B1 to B3 are synchronized.

[0079] The process executed by the image transmitting / receiving devices 120A to 120C to synchronize the reception timing of image data will be described with reference to the flowchart shown in Fig. 12. In Fig. 12, the control unit 122 of the image transmitting / receiving device 120B sets the image transmitting / receiving device 120B as a reference location in step S31. The control units 122 of the image transmitting / receiving devices 120A and 120C, which are the transmitting side locations, acquire the corresponding delay times from the delay time management server 50 in step S32.

[0080] In step S33, the control unit 291 of the image transmitting / receiving devices 120A and 120C sets the delay time as a delay time correction value in the delay time correction value storage unit 294. Here, the delay time is used as the delay time correction value as it is. In step S34, the control unit 291 of the image transmitting / receiving devices 120A to 120C sets the performance start time based on the operation input by the operation unit 293.

[0081] In step S351, the control unit 122 of the image transmitting / receiving devices 120A and 120C, which are the transmitting side bases, sets the performance start timing to a time earlier than the performance start time by the delay time correction value. In step S352, the control unit 122 of the image transmitting / receiving device 120B, which is the reference base, sets the performance start timing to the performance start time.

[0082] In step S361, the control unit 122 of the image transmitting / receiving devices 120A and 120C determines whether the current time has reached the performance start timing. If the current time has not reached the performance start timing (NO), the control unit 122 repeats the process of step S361. If the current time has reached the performance start timing (YES), the control unit 122 shifts the process to step S371.

[0083] In step S362, the control unit 122 of the image transmitting / receiving device 120B determines whether the current time has reached the performance start timing. If the current time has not reached the performance start timing (NO), the control unit 122 repeats the process of step S362. If the current time has reached the performance start timing (YES), the control unit 122 shifts the process to step S372.

[0084] In step S371, the control unit 122 of the image transmitting / receiving device 120A and 120C controls the display unit 295 to display a cue notifying the start of performance at the performance start timing, and ends the process. In step S372, the control unit 122 of the image transmitting / receiving device 120B controls the display unit 295 to display a cue notifying the start of performance at the performance start timing, and ends the process.

[0085] 10, the image transmitting device (image transmitting / receiving device 120A or 120C) located at the transmitting side base includes the camera 11. The image transmitting device transmits image data generated by the camera 11 capturing an image of a subject performing a predetermined action to the image receiving device (image transmitting / receiving device 120B) located at the reference base via the network 30.

[0086] The timing adjustment method executed in the image transmission system measures the delay time when the image receiving device displays the received image data on the display device 22. The timing adjustment method sets a delay time correction value based on the delay time in the timing instruction unit 29 arranged at the transmitting side base. The timing adjustment method causes the timing instruction unit 29 to signal the subject to inform the subject of the timing to start a predetermined action at a time indicated by the delay time correction value before the time instructed for the subject to start the predetermined action.

[0087] According to the timing adjustment method of one or more embodiments, it is possible to adjust the timing when image data captured of a subject performing a specified action performed separately at a transmitting location and a reference location is displayed at the reference location to match.

[0088] <Frame timing adjustment method when receiving image data> Even if the timing of the start of performance is adjusted using the configuration shown in FIG. 10, the timing of the frames of image data generated by the image transmitting / receiving devices 120A to 120C is shifted, and image noise may occur when switching between the moving images generated by the image transmitting / receiving devices 120A to 120C for display.

[0089] Fig. 13 shows an image transmitting / receiving device 120 configured to correct a timing shift of image data frames, and a delay time management server 50 suitable for correcting the timing shift of image data frames. In Fig. 13, similarly to Fig. 10, an image transmitting / receiving device 120B receives image data transmitted from image transmitting / receiving devices 120A and 120C.

[0090] The image transmitting / receiving device 120B has a vertical synchronization delay circuit 123. The vertical synchronization delay circuit 123 receives the image data transmitted from the image transmitting / receiving devices 120A and 120C and the image data output from the camera 11. It is assumed that the vertical synchronization signals of the image data transmitted from the image transmitting / receiving devices 120A and 120C are delayed with respect to the vertical synchronization signal of the image data output from the camera 11.

[0091] The vertical synchronization delay circuit 123 delays the vertical synchronization of the image data output from the camera 11 so as to reduce the deviation of the vertical synchronization signal of the image data transmitted from the image transmitting / receiving devices 120A and 120C from the vertical synchronization signal of the image data output from the camera 11. When the deviation amounts of the vertical synchronization signals of the image data transmitted from the image transmitting / receiving devices 120A and 120C are different, the vertical synchronization delay circuit 123 delays the vertical synchronization of the image data with the smaller deviation amount to match the image data with the larger deviation amount. The vertical synchronization delay circuit 123 delays the image data from the camera 11 to match the deviation amount, thereby matching the vertical synchronization of the three image data.

[0092] The vertical synchronization delay circuit 123 supplies to the transmission / reception unit 121 a first delay time of the vertical synchronization signal of the image data transmitted from the image transmission / reception device 120A and a second delay time of the vertical synchronization signal of the image data transmitted from the image transmission / reception device 120C, relative to the vertical synchronization signal of the image data output from the camera 11. The vertical synchronization delay circuit 123 functions as a vertical synchronization deviation amount measuring unit that measures the first and second delay times.

[0093] The transmitting / receiving unit 121 transmits the first and second delay times to the delay time management server 50 via the network 30 under the control of the control unit 122. It is assumed that the first delay time is 3.2 ms and the second delay time is 0.3 ms.

[0094] 14, the delay time management server 50 stores a delay time obtained by adding a first delay time of 3.2 ms to a delay time of 10.3 ms when the location B1 is the sending side and the location B2 is the receiving side. The delay time management server 50 also stores a delay time obtained by adding a second delay time of 0.3 ms to a delay time of 10.2 ms when the location B3 is the sending side and the location B2 is the receiving side.

[0095] The delay time correction value holding units 294 of the timing instruction units 29 in the image transmitting / receiving devices 120A and 120C respectively hold the delay times (10.3+3.2) ms and (10.2+0.3) ms as delay time correction values.

[0096] The control unit 291 of the timing instruction unit 29 in the image transmitting / receiving device 120A displays on the display unit 295 a cue notifying the user of the timing to start performance at a time before the performance start time, which is the delay time correction value of (10.3+3.2) ms. The control unit 291 of the timing instruction unit 29 in the image transmitting / receiving device 120C displays on the display unit 295 a cue notifying the user of the timing to start performance at a time before the performance start time, which is the delay time correction value of (10.2+0.3) ms.

[0097] As a result, the image data displayed on the display device 22 at location B2 is not only displayed with the timing of the performances at all locations B1 to B3 synchronized, but also the timing discrepancies of the frames of image data generated by the image transmitting / receiving devices 120A to 120C are corrected.

[0098] If, over time, the vertical synchronization signal of the image data transmitted from the image transmission / reception device 120A or 120C becomes out of sync with the vertical synchronization signal of the image data output from the camera 11, the vertical synchronization of the image data output from the camera 11 can be delayed by the vertical synchronization delay circuit 123 to reduce the deviation of the vertical synchronization signal.

[0099] Furthermore, a first delay time of the vertical synchronizing signal of the image data transmitted from the image transmitting / receiving device 120A and a second delay time of the vertical synchronizing signal of the image data transmitted from the image transmitting / receiving device 120C relative to the vertical synchronizing signal of the image data output from the camera 11 are transmitted to the delay time management server 50 and updated to the latest first and second delay times. Therefore, the timing of the frames of the image data generated by the image transmitting / receiving devices 120A to 120C is always corrected to an optimal state at every predetermined time.

[0100] 13, an image transmitting device (image transmitting / receiving device 120A or 120C) including a first camera (camera 11) and an image receiving device including a second camera (camera 11) and a display device 22 are connected via a network 30. The image receiving device (image transmitting / receiving device 120B) includes a delay time measuring unit 28 and a vertical synchronization delay circuit 123 that functions as a vertical synchronization deviation measuring unit.

[0101] The delay time measurement unit 28 measures the delay time when the first image data generated by the first camera and transmitted by the image transmitting device to the image receiving device is received by the image receiving device and displayed on the display device 22. The vertical synchronization delay circuit 123 measures the vertical synchronization deviation amount indicating the time by which vertical synchronization is deviated between the first image data and the second image data generated by the second camera and displayed on the display device 22. The delay time management server 50 stores the delay time and the vertical synchronization deviation amount transmitted from the image receiving device via the network 30.

[0102] 13, a first camera provided in an image transmitting device located at a transmitting side site captures an image of a first subject performing a predetermined action, and the first image data is generated by the first camera and transmitted to an image receiving device located at a reference site via a network 30. A timing adjustment method executed in the image transmission system measures a delay time when the first image data received by the image receiving device is displayed on a display device 22.

[0103] The timing adjustment method executed in the image transmission system measures a vertical synchronization deviation amount indicating a time when vertical synchronization is deviated between the first image data and the second image data generated by a second camera of the image receiving device by photographing a second subject performing a predetermined action. The timing adjustment method executed in the image transmission system sets a delay time correction value based on a delay time obtained by adding the delay time and the vertical synchronization deviation amount to a timing instruction unit 29 arranged at a transmitting side base.

[0104] The timing adjustment method executed in the image transmission system has a timing instruction unit 29 issuing a signal to the first subject to inform the first subject of the timing to start a predetermined action at a time indicated by a delay time correction value before the time at which the first subject is instructed to start the predetermined action.

[0105] According to the delay time management server of one or more embodiments, it is possible to more accurately manage the delay time and the vertical synchronization deviation amount when transmitting image data from the image transmitting / receiving device 120 operating as an image transmitting device to the image transmitting / receiving device 120 operating as an image receiving device via the network 30. According to the timing adjustment method of one or more embodiments, it is possible to adjust the timing, including the timing of the frames of the image data, to match when image data obtained by capturing a subject performing a predetermined action performed separately at the transmitting base and the reference base is displayed at the reference base.

[0106] The present invention is not limited to one or more of the embodiments described above, and various modifications can be made without departing from the gist of the present invention. [Explanation of symbols]

[0107] 10, 10A~10C Image transmitting device 11 Camera 12 Transmitter 13,23 GNSS satellite radio clock 14,24 Clock generation unit 15,27 Image memory 16,25 Communications Department 20 Image receiving device 21 Receiving section 22 Display device 26,113 Image processing circuit 28 Delay time measurement section 29 Timing indicator 30 Network 40 satellites 50 Delay Time Management Server 111,122,221 Control section 112 Image sensor 120A~120C Image transmission / reception device 121 Transmitter / receiver 123 Vertical sync delay circuit (vertical sync offset measurement section) 150,270 Write / read control section 222 Drive circuit 223 Liquid crystal panel

Claims

[Claim 1] a camera provided in an image transmitting device arranged at a transmitting side site captures an image of a subject performing an action of playing a musical instrument, the image data being generated by the camera being transmitted to an image receiving device arranged at a reference site via a network, and a delay time when the image receiving device displays the received image data on a display device is measured; A delay time correction value based on the delay time is set in a timing instruction unit disposed in the transmitting side base, The timing instruction unit signals the subject to inform the subject of the timing to start the action of playing the musical instrument at a time that is a time indicated by the delay time correction value before a time at which the subject is instructed to start the action of playing the musical instrument. How to adjust the timing.

Images