Image receiving device and delay time measuring device

Abstract

To provide an image receiving device suitable for grasping a more accurate delay time when image data is transmitted from an image transmitting device to the image receiving device via a network.SOLUTION: A clock generation unit 14 generates a clock including time information based on radio waves received from a satellite 40 for a global navigation satellite system. A communication unit 16 transmits image data on which at least one of first to third clock numbers read from an image memory is superimposed. The first clock number indicates a time when an imaging element 112 generated pixel data for a reference pixel in a frame. The second clock number indicates a time when a write / read control unit 150 wrote the pixel data for the reference pixel into an image memory 15. The third clock number indicates a time when the communication unit 16 transmitted the pixel data for the reference pixel.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image receiving device and a delay time measuring device. [Background technology]

[0002] An image transmission device is installed at at least one location and receives images via a network. The image data is transmitted to the image receiving device, and the image transmitted from the image transmitting device is set in the image receiving device. In this case, the fifth generation mobile communication system (5G) With the practical application of this technology, image data can be transmitted with low latency, making it possible to The delay due to this is negligible. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-88305 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the delay due to the internal processing in the image transmitting device and the image receiving device is ignored. In addition, image transmission devices are installed at multiple locations, and multiple image transmission devices are used. When transmitting image data from a device to an image receiving device, the internal The timing of the image displayed on the display device varies due to variations in the delay time caused by processing. The image may be transmitted from the image transmission device via the network, causing a sense of incongruity. The internal structure of the image transmitting device and the image receiving device when transmitting image data to the image receiving device It is necessary to understand the delay time, including delays due to sub-processing.

[0005] The present invention relates to a method for transmitting image data from an image transmitting device to an image receiving device via a network. The present invention aims to provide an image receiving device and a delay time measuring device suitable for grasping a more accurate delay time when transmitting a video signal. [Means for solving the problem]

[0006] The present invention relates to a method for transmitting a first clock number, a second clock number, and At least one of the third clock numbers is a first clock number that receives the superimposed image data. a first clock generating unit provided in the image transmitting device, It includes time information generated by the GPS receiver based on radio waves received from satellites for the Global Navigation Satellite System. the clock number of the first clock including the image data of the camera of the image transmitting device; the second clock indicating the time at which the element generated pixel data for the reference pixel in the frame; The number is a clock number in the first clock, and the number is a clock number in the first clock provided in the image transmitting device. When the write / read control unit 1 writes the pixel data of the reference pixel into the first image memory, the third clock number is a clock number in the first clock; the time when the second communication unit of the image transmitting device transmits the pixel data of the reference pixel; and includes time information based on radio waves received from a satellite for the global navigation satellite system, a second clock generating unit that generates a second clock identical to the first clock; The image data is read and written under the control of a second write / read control unit. a second image memory into which the image data is written and read; a display device that displays the image data; and one of the first to third clock numbers. and a time when the first communication unit received the pixel data of the reference pixel. a fourth clock number in the second clock, and the second write / read control unit The second clock signal indicates the time when the pixel data of the reference pixel is read out from the second image memory. a fifth clock number in the clock signal, and the display device displays pixel data of the reference pixel; and the sixth clock number in the second clock indicating the time when The image receiving device further includes a delay time measuring unit for measuring a delay time indicating a difference between the image receiving device and the block number. Provide a place.

[0007] In the present invention, the image data transmitted from the image transmitting device includes a first clock number, a second clock number, at least one of the first clock number, the second clock number, and the third clock number is superimposed on the first clock number; The clock number is a global clock number generated by a first clock generating unit included in the image transmitting device. In the first clock containing time information based on radio waves received from satellites for a navigation satellite system the clock number in the frame, and the image sensor of the camera of the image transmission device The second clock number indicates the time when the pixel data of the sub-pixel is generated, and the first clock number indicates the time when the pixel data of the sub-pixel is generated. a clock number in the clock, and a first write / read control unit included in the image transmitting device indicates the time when the pixel data of the reference pixel is written into the first image memory, and the third The clock number is a clock number in the first clock, and the image transmission device a time when the first communication unit transmits the pixel data of the reference pixel; The image receiving device that receives the fourth clock number, the fifth clock number, and the sixth clock number and generating at least one of the fourth clock number and the fourth clock number. the same as the first clock containing time information based on radio waves received from a satellite for the system a clock number in a second clock, and a second communication unit included in the image receiving device the fifth clock number indicates the time when the pixel data of the reference pixel is received, and the second clock number indicates the time when the pixel data of the reference pixel is received. The clock number of the second write / read control unit included in the image receiving device is a clock number of the second write / read control unit included in the image receiving device. The control unit writes the image data into a second image memory, and transfers the image data from the second image memory to the The sixth clock number indicates the time when the pixel data of the reference pixel is read out. the display device provided in the image receiving device is the second indicates the time when the pixel data of the reference pixel read from the image memory is displayed, any one of the clock numbers from the first to the third clock numbers and the fourth to sixth clock numbers The image transmitting device generates the signal by calculating the difference between the clock number of the image transmitting device and any one of the clock numbers. The image data is generated and transmitted to the image receiving device, and the image receiving device receives and displays the image data. The present invention provides a delay time measuring device that measures the delay time when [Effects of the Invention]

[0008] According to the image receiving device and delay time measuring device of the present invention, it is possible to grasp a more accurate delay time when transmitting image data from an image transmitting device to an image receiving device via a network. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram conceptually showing an image transmission system in which image data is transmitted from a plurality of image transmitting devices to an image receiving device via a network. [Figure 2]FIG. 2 is a block diagram showing an example configuration of an image transmitting device and an image receiving device configured to measure the delay time from the time when a camera in the image transmitting device starts capturing a moving image to the time when a display device in the image receiving device starts displaying the moving image. [Figure 3] FIG. 3 is a diagram showing the 1 PPS signal and 10 MHz clock generated by the GNSS satellite wave clock 13 in FIG. 2, and the 148.5 MHz clock generated by the clock generating unit 14. In FIG. [Figure 4] FIG. 4 is a diagram showing the 1 PPS signal and the 10 MHz clock generated by the GNSS satellite wave clock 23 in FIG. 2, and the 148.5 MHz clock generated by the clock generating unit 24. [Figure 5] FIG. 5 is a flowchart showing the process executed by the delay time measurement unit 28 in FIG. [Figure 6] FIG. 6 is a flowchart showing the detailed processing of step S1 in FIG. [Figure 7] FIG. 7 is a flowchart showing the detailed process of step S2 in FIG. [Figure 8] FIG. 8 is a block diagram showing a delay time management server that manages delay times when image data is transmitted and received bidirectionally between image transmitting and receiving devices located at three locations. [Figure 9] FIG. 9 is a diagram illustrating an example of delay times stored in the delay time management server illustrated in FIG. [Figure 10] FIG. 10 is a block diagram showing an image transmitting / receiving device that adjusts the timing of giving a signal to start a predetermined operation based on the delay time stored in the delay time management server. [Figure 11] FIG. 11 is a block diagram showing a specific example of the configuration of the timing instruction section 29 in FIG. [Figure 12] FIG. 12 is a flowchart showing the process executed by the image transmitting / receiving devices arranged at the three locations shown in FIG. 10 to synchronize the reception timing of image data. [Figure 13]FIG. 13 is a block diagram showing an image transmitting device configured to correct the timing deviation of image data frames, and a delay time management server suitable for correcting the timing deviation of image data frames. [Figure 14] FIG. 14 is a diagram showing an example of delay times stored in the delay time management server shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an image receiving device, a delay time measuring device, a delay time management server, and a timing adjustment method according to one or more embodiments will be described with reference to the accompanying drawings.

[0011] First, using FIG. 1, we will explain how to send and receive images from multiple image sending devices via a network to an image receiving device. This section explains the concept of an image transmission system that transmits image data. Note that the image transmission system may transmit audio data in addition to image data. The process of transmitting and receiving audio data is not shown in the figures.

[0012] In FIG. 1, image transmission devices 10A to 10C are installed at three different locations. At a site other than the site where the image transmitting devices 10A to 10C are installed, The image transmitting devices 10A to 10C are provided with a receiving device 20. The camera 11 transmits image data generated by capturing an image of a subject. It has a signal section 12.

[0013] The image data transmitted from the image transmitting devices 10A to 10C is transmitted via the network 30. The image is transmitted to the image receiving device 20. The network 30 is typically the Internet. The image receiving device 20 includes a receiving unit 21 that receives image data and a video signal based on the image data. The display device 22 is provided with a display device 22 for displaying an image from the image transmission devices 10A to 10C. The moving images from the image transmitting devices 10A to 10C may be displayed simultaneously. The images may be displayed while being switched. An image transmission device without specifying which one is used will be referred to as an image transmission device 10.

[0014] <Specific configuration examples of the image transmitting device, the image receiving device, and the delay time measuring device> FIG. 2 shows the time from when the camera 11 in the image transmission device 10 starts capturing a moving image to when the image The delay time until the display device 22 in the image receiving device 20 starts displaying the moving image is measured. 1 shows an example of the configuration of an image transmitting device 10 and an image receiving device 20 configured to determine the number of images to be transmitted. The image transmitting device 10 and the image receiving device 20 shown in FIG. 2 include delays due to their internal processing. The system has a suitable configuration for grasping delay times, including delay times.

[0015] The delay time measurement unit 28 included in the image receiving device 20 measures the delay time of one or more embodiments. A time measurement device is configured, and the time measurement device is configured to measure the time by the internal processing in the image transmitting device 10 and the image receiving device 20. The image transmitting device 10 measures the delay time, including the delay, more accurately as described below. The image receiving device 20 may be implemented in one or more embodiments. It is an image receiving device of the form.

[0016] As shown in FIG. 2, the image transmission device 10 includes a camera 11 and a GNSS satellite wave clock 1 3, a clock generating unit 14, an image memory 15, and a communication unit 16. The camera 11 is a control unit 111, an image sensor 112, and an image processing circuit 113. The image memory 15 is The communication unit 16 includes a control unit 150. The communication unit 16 functions as the transmission unit 12 in FIG.

[0017] The image receiving device 20 includes a display device 22, a GNSS satellite wave clock 23, a clock generator a communication unit 25, an image processing circuit 26, an image memory 27, and a delay time measurement unit 28. The display device 22 includes a control unit 221, a drive circuit 222, and a liquid crystal panel 223. The memory 27 includes a write / read control unit 270. The communication unit 25 is the same as the receiving unit 21 in FIG. It functions as such.

[0018] GNSS satellite-wave clock 13 is a Global Navigation Satellite System (GNSS) The GPS receiver receives radio waves from a satellite 40 for the Global Navigation System (GNSS) at one-second intervals as shown in (a) of FIG. The 1PPS signal is a pulse of 10 MHz, and the 10 MHz clock shown in Figure 3(b) (hereafter referred to as 10 MHz clock). GNSS is an example of a GPS (Global Positioning System) system).

[0019] The clock generating unit 14 generates the following signal based on the input 1 PPS signal and 10 MHz clock: The 148.5MHz clock shown in Figure 3(c) (hereinafter referred to as the 148.5MHz clock) The generated image data is supplied to the camera 11, the image memory 15, and the communication unit 16. For convenience of illustration, the image data shown in FIG. The 148.5MHz clock shown in (c) is shown with a significantly longer clock period. In addition to 148.5 MHz, the clock generation unit 14 can also generate 297 MHz and 74. A clock having another frequency for video synchronization, such as 25 MHz or 27 MHz, may also be generated.

[0020] The image transmitting device 10 includes time information based on radio waves received from a GNSS satellite 40. A clock generating unit that generates a clock is provided. A 10 MHz clock is supplied to the unit 16, and the camera 11, the image memory 15, and the communication unit 16 are operated in a single In this case, the GNSS satellite wave clock 13 may be operated with a clock of 0 MHz. The camera 11 and the image memory 1 are clock generators that generate clocks containing time information. 5. The frequency of the clock supplied to the communication unit 16 is not limited.

[0021] The GNSS satellite wave clock 13 transmits a pulse of the 1PPS signal shown in FIG. 3(a) at time 0:0 Suppose a clock pulse is generated at 0:00 and the next pulse is generated at 0:00:01. The generating unit 14 generates clock numbers 0 to 148.5×10 based on the time 0:00:00. -6 - Clock number 0 to 148.5 x 10, based on the time 0:00:01 -6 Like the -1 clock, it outputs a 148.5MHz clock that includes time information.

[0022] The control unit 111 of the camera 11 controls the electronic shutter of the image sensor 112. The control unit 111 is a central processing unit provided in the camera 11. The image data generated by the image sensor 112 capturing an image of a subject may be The image undergoes various image processing by the processing circuit 113 and is then supplied to the image memory 15 .

[0023] The image processing in the image processing circuit 113 includes defective pixel interpolation processing, black level processing, white level processing, and the like. One of the following processes: image balance processing, demosaic processing, shading correction processing, and compression encoding processing The image processing in the image processing circuit 113 is at least compression encoding. Includes processing.

[0024] The control unit 111 controls the image sensor 112 to release the electronic shutter at a predetermined reference pixel and to capture pixel data. The clock number of the 148.5MHz clock that indicates the time t0 at which the The reference pixel is controlled so that it is superimposed on the blanking period of the image data. It is preferable that the pixel is the first pixel in the first row. The clock number indicates the time when the camera 11 starts generating image data for each frame. As shown in FIG. 3(c), the time t0 included in the image data output by the camera 11 is As an example, the clock number is assumed to be Cn0, which is clock number 0.

[0025] The write / read control unit 150 writes the image data into the image memory 15. The read control unit 150 writes pixel data of the reference pixel of each frame into the image memory 15. The clock number of the 148.5MHz clock that indicates the time t1 is stored as metadata in the image data. The image data superimposed during the blanking period of the image data is written to the image memory 15. As shown in (c) of FIG. 3, the time t1 included in the image data written in the image memory 15 The clock number indicating the image data written in the image memory 15 is Cn1. In this example, clock numbers Cn0 and Cn1 are superimposed.

[0026] The write / read control unit 150 reads out the image data stored in the image memory 15 and transmits the image data. The communication unit 16 then transmits the image data read from the image memory 15 to the communication unit 16. The image data is then sent to the image receiving device 20 via the network 30. 16 is a 148.5 MHz clock that indicates the time t2 at which pixel data of the reference pixel of each frame is transmitted. The clock number of the clock is superimposed as metadata on the blanking period of the image data. As shown in Figure 3(c), the pixel data of the reference pixel is The clock number indicating the time t2 at which the data is transmitted is Cn2. The code Cn2 indicates the time t2 when the pixel data of the reference pixel is transmitted.

[0027] The camera 11, the image memory 15, and the communication unit 16 are connected to a 148.5 MHz clock. In order to notify the image receiving device 20 that it is operating in clock mode, a clock is added to the image data. The image is transmitted from the image transmitting device 10 via the network 30. The packet data transmitted to the image receiving device 20 contains image data, clock frequency information, and The image data includes clock numbers Cn0, Cn1, and Cn2 superimposed thereon.

[0028] The GNSS satellite radio clock 23 in the image receiving device 20 is also the same as the GNSS satellite radio clock 13. Similarly, when radio waves are received from the satellite 40, the 1 second pulses shown in FIG. It outputs a PPS signal and a 10 MHz clock shown in FIG. 4(b).

[0029] In FIG. 2, the radio waves received by the GNSS satellite radio clock 13 and the GNSS satellite radio clock 23 are The transmitting satellite 40 is a common one 40. The GNSS satellite wave clock 13 receives and a satellite 40 that transmits radio waves received by the GNSS satellite wave clock 23. The GNSS satellite wave clocks 13 and 23 may be different from the star 40. The satellite 40 is operated by the legal satellite system. You can believe it.

[0030] The communication unit 25 receives packet data including image data and clock frequency information, Clock frequency information is supplied to a clock generating unit 24, and image data is supplied to an image processing circuit 26. The clock generating unit 24 receives clock frequency information indicating 148.5 MHz. Therefore, based on the input 1PPS signal and 10MHz clock, The 148.5 MHz clock is generated by the display device 2. 2, the signal is supplied to a communication unit 25, an image memory 27, and a delay time measurement unit 28.

[0031] The 148.5MHz clock shown in Figure 4(c) is the same as the 148.5MHz clock shown in Figure 3(c). It is a clock that is identical (same phase and frequency) to the z clock.

[0032] The clock generation unit 24 is connected to the display device 22, the communication unit 25, the image memory 27, and the delay time measurement unit 2 The clock supplied to 8 is a 148.5MHz clock that includes time information, as described above. do.

[0033] As shown in FIG. 4(c), the communication unit 25 receives the reference image of each frame from the network 30. The clock number of the 148.5 MHz clock indicating the time t3 when the raw pixel data was received is The pixel data of the reference pixel is received and superimposed on the blanking period of the image data as metadata. The clock number indicating the received time t3 is Cn3. The image data to be displayed has clock numbers Cn0, Cn1, Cn2, and Cn3 superimposed on it. .

[0034] The image data output from the communication unit 25 is subjected to various image processing by the image processing circuit 26. The image data is then supplied to the image memory 27. The image processing in the image memory 27 includes at least The image processing circuit 26 processes the clock number C included in the image data. n0, Cn1, Cn2, and Cn3 are supplied to the delay time measurement unit 28.

[0035] The write / read control unit 270 writes the image data supplied to the image memory 27 to the image memory 27. The image data stored in the image memory 27 is read out and the drive circuit of the display device 22 is At this time, the write / read control unit 270 supplies the image of the reference pixel of each frame to the line 222. The clock number of the 148.5MHz clock that indicates the time t4 when the raw data was read is At time t4, the pixel data of the reference pixel is supplied to the drive circuit 222. As shown in FIG. 4(c), the image data is read from the image memory 27. The clock number indicating the time t4 is assumed to be Cn4.

[0036] The control unit 221 of the display device 22 drives the liquid crystal panel 221 to convert the image data supplied to the drive circuit 222 into The control unit 221 controls the drive circuit 222 so that the display device 22 displays the The control unit 221 may be a central processing unit that controls the pixel data of the reference pixel of each frame. The clock number of the 148.5 MHz clock indicating the time t5 displayed on the LCD panel 223 is The pixel data of the reference pixel is supplied to the delay time measurement unit 28. As shown in FIG. 4(c), It is assumed that the clock number indicating the time t5 displayed on the LCD panel 223 is Cn5.

[0037] It should be noted that the display device 22 does not need to operate at a 148.5 MHz clock. The 148.5MHz clock is supplied to the This is to supply the clock number Cn5 indicating 5 to the delay time measurement unit 28.

[0038] If the display device 22 is a projection type display device rather than a direct-view type display device, the liquid crystal panel The display device 22 is a liquid crystal panel. The display panel (display element) may be other than the above.

[0039] The delay time measurement unit 28 measures the delay time of the image transmission device in accordance with the processing shown in the flowcharts of FIGS. The delay time between the image receiving device 10 and the image receiving device 20 is calculated, and a delay due to the delay time is corrected. Determine the delay time correction value for

[0040] In FIG. 5, the delay time measurement unit 28 measures the delay time of the image transmission device 10 in step S1. The photographing time is set, and in step S2, the display time in the image receiving device 20 is set. FIG. 6 shows the detailed process of step S1. In FIG. 6, the delay time measurement unit 28 In step S11, it is determined whether or not clock number Cn0 (that is, time t0) exists. If the clock number Cn0 exists (YES), the delay time measurement unit 28 performs the following in step S13. The time t0 indicated by the clock number Cn0 is set as the shooting time.

[0041] If the clock number does not exist in step S11 (NO), the delay time measurement unit 28 In step S12, it is determined whether clock number Cn1 (i.e., time t1) exists. If the clock number Cn1 exists (YES), the delay time measurement unit 28 proceeds to step S14. The time t1 indicated by the clock number Cn1 is set as the shooting time. If there is no clock (NO), the delay time measurement unit 28 calculates the clock number Cn2 in step S15. The time t2 indicated by is set as the shooting time.

[0042] FIG. 7 shows the detailed process of step S2. In FIG. 7, the delay time measurement unit 28 In step S21, it is determined whether clock number Cn5 (i.e., time t5) exists. If the clock number Cn5 exists (YES), the delay time measurement unit 28 proceeds to step S23. Then, time t5 is set as the display time.

[0043] If the clock number Cn5 does not exist in step S21 (NO), the delay time measurement unit 28 In step S22, it is determined whether clock number Cn4 (i.e., time t4) exists. If the clock number Cn4 exists (YES), the delay time measurement unit 28 proceeds to step S In step 24, the time t4 is set to the display time. If the clock number Cn4 does not exist (NO), In step S25, the delay time measurement unit 28 displays the time t3 indicated by the clock number Cn3. Set it to the time shown.

[0044] In FIG. 6, it is the camera 11 that determines whether the clock number Cn0 exists. The clock number Cn0 indicating the time t0 at which the image data of each frame starts to be generated is used as the image data. This is because there may be cases where the clock number is not configured to be superimposed on the data. Whether Cn1 exists or not is determined when the image memory 15 writes image data. The clock number Cn1 indicating the time t1 may not be configured to be superimposed on the image data. Because it is possible.

[0045] In FIG. 7, it is the display device 2 that determines whether the clock number Cn5 exists. 2 delays the clock number Cn5 indicating the time t5 at which the image data is displayed on the liquid crystal panel 223. This is because there may be cases where the clock is not configured to be supplied to the time measurement unit 28. The presence or absence of lock number Cn4 is determined by the image memory 27 storing image data. The clock number Cn4 indicating the read time t4 is supplied to the delay time measurement unit 28. This is because there may be cases where the information has not been provided.

[0046] Returning to FIG. 5, in step S3, the delay time measurement unit 28 calculates the difference between the shooting time and the display time. The delay time between the image transmitting device 10 and the image receiving device 20 is calculated as follows. In step S4, the unit 28 determines whether the image capture time is time t2 and the display time is time t3. If the shooting time is time t2 and the display time is time t3, In this case, the time from time t2 to time t3 is the time required for transmitting the image data through the network 30. Therefore, after the image transmitting device 10 starts generating image data, It does not represent the delay before device 20 begins displaying image data.

[0047] Therefore, in step S4, if the photographing time is time t2 and the display time is time t3 ( If YES, the delay time measurement unit 28 determines in step S5 that the pre-measured fixed correction value is equal to the delay time. It is determined whether or not the fixed correction value is set in the time measurement unit 28. one of the average time to time t2 and the average time from time t3 to time t5; Or the sum of both.

[0048] If a fixed correction value measured in advance is set in step S5 (YES), The delay time obtained in step 3 is added to the fixed correction value to determine the delay time correction value. End the process.

[0049] In step S4, if the photographing time is not time t2 and the display time is not time t3 (NO), Or, if the fixed correction value measured in advance is not set in step S5 (NO), the delay In step S7, the time measurement unit 28 directly delays the delay time obtained in step S3. The delay time correction value is determined, and the process is terminated.

[0050] As described above, the image transmission device 10 includes the camera 11, the clock generation unit 14, the image memory 1 The clock generating unit 14 receives a signal from a GNSS satellite 40. A first clock (for example, a 148.5 MHz clock) containing time information based on the wave is generated. The camera 11 operates according to a first clock, and the image sensor 112 captures an image of a subject. The image memory 15 is driven by the first clock. The image data is written and read under the control of the write / read control unit 150 .

[0051] The communication unit 16 reads out from the image memory 15 and outputs the first to third clock numbers. At least one superimposed image data is transmitted. The first to third clock numbers are the clocks The numbers are Cn0 to Cn2.

[0052] The first clock number is the number of pixels in the frame that the camera 11 captures. The camera 11 superimposes the first clock number on the image data. If the image data including the first clock number is will be sent.

[0053] The second clock number is the number of clocks that the write / read control unit 150 writes the pixel data of the reference pixel to the image memory. The write / read control unit 150 writes the second clock number to the image data. If the image processing device is configured to perform a process of superimposing the second clock number on the image data, The third clock number is the clock number at which the communication unit 16 transmits pixel data of the reference pixel. The communication unit 16 executes the process of superimposing the third clock number on the image data. If configured to do so, image data including the third clock number is transmitted.

[0054] The image receiving device 20 receives a first clock number, a second clock number, and a third clock number. At least one of the communication units 25 receives the superimposed image data. If the communication unit 25 is the first communication unit, the communication unit 16 is the second communication unit. If the communication unit 21 is a communication unit, the communication unit 25 is a second communication unit.

[0055] The first clock number is the first clock number generated by the clock generating unit 14 (first clock generating unit). The clock number in the clock, and the image sensor 112 of the camera 11 is the base number in the frame. The second clock number indicates the time when the pixel data of the sub-pixel is generated. The write / read control unit 150 (first write / read control unit) This indicates the time when the pixel data of the sub-pixel is written into the image memory 15 (first image memory). The clock number of 3 is the clock number in the first clock, and the communication unit 16 (second Indicates the time when the communication unit (or the RX1100) transmitted the pixel data of the reference pixel.

[0056] The image receiving device 20 includes a clock generating unit 24 (second clock generating unit), an image memory 27 (second image memory), a display device 22, and a delay time measurement unit 28. The image generation unit 24 generates a second clock that is the same as the first clock. The write / read control unit 270 (second write / read control unit) operates according to a second clock. The display device 22 writes and reads image data based on the control of the image memory 27. The image data read out from the memory is displayed.

[0057] The delay time measurement unit 28 determines whether the delay time is equal to one of the first to third clock numbers. , a delay time indicating the difference between the fourth clock number and any one of the sixth clock numbers The fourth clock number is the clock number in the second clock. The fifth clock number indicates the time when the pixel data of the reference pixel is received by the second clock. The write / read control unit 270 writes the pixel data of the reference pixel to the clock number. The sixth clock number indicates the time when the image is read from the image memory 27. This is a clock number in the time series and indicates the time at which the display device 22 displays the pixel data of the reference pixel.

[0058] When the delay time measurement unit 28 acquires the first clock number (clock number Cn0), It is preferable to measure the delay time using the first clock number. If the first clock number cannot be obtained and the second clock number (clock number Cn1) is obtained, In this case, it is preferable to measure the delay time using the second clock number. If the sixth clock number (clock number Cn5) is acquired, the sixth clock number is The delay time measurement unit 28 acquires the sixth clock number. If the fifth clock number (clock number Cn4) is not obtained, the fifth clock number It is advisable to measure the delay time using a signal.

[0059] The delay time measurement unit 28 is a delay time measurement device according to one or more embodiments. In this way, the delay time measurement unit 28 measures the delay time of any one of the first to third clock numbers. The difference between the clock number and any one of the fourth to sixth clock numbers is calculated. As a result, the delay time measurement unit 28 detects the delay time generated by the image transmitting device 10 and sent to the image receiving device 20. The delay time when the transmitted image data is received by the image receiving device 20 and displayed is measured.

[0060] In one or more embodiments of the image transmitting device, the image receiving device, and the delay time measuring device According to this, image data is transmitted from an image transmitting device 10 to an image receiving device 20 via a network 30. This allows for a more accurate understanding of the delay time when transmitting data.

[0061] <Delay time management server> In FIG. 8, image transmitting / receiving devices 120A to 120C are provided at locations B1 to B3, respectively. The image transmitting / receiving devices 120A to 120C communicate with each other via the network 30. The image transmitting / receiving device 120A to 120C is not specified. The receiving device is referred to as an image transmitting / receiving device 120. The image transmitting / receiving device 120 is the image transmitting device shown in FIG. The image receiving device 10 has the configuration of both the image receiving device 10 and the image receiving device 20 .

[0062] As shown in FIG. 8, the image transmitting and receiving device 120 generally comprises a camera 11, a display device 22, and a The system includes a delay time measurement unit 28, a control unit 122, and a transmission / reception unit 121. However, the image transmitting and receiving device 120 is connected to the GNSS satellite wave clock 13 (or 23) and the clock The camera 11 and the display device 22 are provided with a clock generator 14 (or 24). The transmitter / receiver 121 corresponds to the communication units 16 and 25. The unit 122 corresponds to the control units 111 and 221 .

[0063] The image transmitting device 10 shown in FIG. 2 is the same as the image transmitting device 120 shown in FIG. The image receiving device 20 shown in FIG. 8 shows in detail the configuration when the image transmitting / receiving device 120 shown in FIG. 8 is operating as an image receiving device. This is described in.

[0064] A delay time management server 50 is connected to the network 30. The image transmitting / receiving device 50 communicates with the image transmitting / receiving devices 120A to 120C in two-way communication. As described above, the delay time management server 50 manages the image transmission / reception devices 120A installed at the locations B1 to B3. Delay when the image transmitting / receiving device 120C is the transmitting side and another image transmitting / receiving device 120 is the receiving side In FIG. 9, the delay time is set to a value from any of the times t0 to t2 to the time t Information indicating which of the times t3 to t5 the delay time is is also written.

[0065] The delay time management server 50 is configured to measure the delay time of the image transmitting / receiving devices 120A to 120C. The delay time shown in FIG. 9 may be updated every time the delay time is measured. 50, the delay time measurement unit 28 measures the delay time shown in FIG. 9 every time a predetermined time has elapsed. The image transmitting and receiving device 120 may update the delay time to the latest delay time determined by the delay time management server. The delay time stored in 50 can be read out.

[0066] As described above, one or more delay time management servers, namely, delay time management server 50 are connected to a network 30. The network 30 includes at least two images. The first and second image transmitting / receiving devices 120 communicate with each other bidirectionally. Any two of the image transmitting and receiving devices 120A to 120C are connected. The devices 120 are the first and second image transmitting / receiving devices.

[0067] The first image transmitting / receiving device receives the image data generated by the second image transmitting / receiving device and sent to the first image transmitting / receiving device. a first delay time when the first image transmitting / receiving device receives and displays the image data transmitted from the first image transmitting / receiving device; The second image transmitting / receiving device is provided with a first delay time measuring unit (delay time measuring unit 28) for determining The image data generated by the first image transmitting / receiving device and transmitted to the second image transmitting / receiving device is transmitted to the second image transmitting / receiving device. A second delay time measurement for measuring a second delay time when the image transmitting / receiving device receives and displays the image. The delay time measuring unit 28 is provided.

[0068] The delay time management server 50 receives the image data transmitted from the first image transmitting / receiving device via the network 30. The first delay time and the second image transmission / reception device transmitted via the network 30 The second delay time is stored.

[0069] According to one or more embodiments of the delay time management server 50, as an image transmission device The image transmitting / receiving device 120 in operation is connected to the image receiving device via the network 30. In this way, the delay time when transmitting image data to the image transmitting / receiving device 120 can be more accurately managed. It is possible.

[0070] <Timing adjustment method when receiving image data> As an example, in FIG. 10, musical instruments are played at each of the locations B1 to B3, and the supervisor of the performance The image transmitting and receiving device 120B at the site B2 where the image transmitting and receiving device 120A and the image transmitting and receiving device 120B are located is Consider the case where image data sent from C is received. The performance of the instrument is This is an example of a predetermined action performed by a subject. The subject playing a musical instrument is a musician.

[0071] The camera 11 of the image transmitting / receiving device 120A takes an image of the performer at the location B1, and the transmitting / receiving unit 121 The image data is transmitted to the image transmitting / receiving device 120B. takes a picture of the performer at the location B3, and the transmitting / receiving unit 121 sends the image data to the image transmitting / receiving device 120B. The camera 11 of the image transmitting / receiving device 120B takes an image of the performer at the site B2, and transmits the image to the display device The display device 22 of the image transmitting / receiving device 120B displays a moving image. The moving images generated by 20A to 120C may be displayed simultaneously, or the images may be transmitted and received. The moving images generated by the devices 120A to 120C may be displayed while being switched. .

[0072] The image transmitting / receiving devices 120A to 120C transmit the images to the performers located at the locations B1 to B3, respectively. As shown in FIG. The timing instruction unit 29 includes a control unit 291, a clock 292, an operation unit 293, and a delay time correction value storage unit. The clock 292 has a GNSS satellite wave clock 13 (or 2 The display unit 295 may be a liquid crystal panel or an LED. good.

[0073] The timing instruction unit 29 controls the control unit 291 to indicate the time when the performance should start by using the operation unit 293. Once set, the control unit 291 monitors the time measured by the clock 292 and determines the time to start playing. The display unit 295 is configured to display a timing signal. For example, at 13:00 If the performance starts at each of the locations B1 to B3 at 00 minutes 00 seconds, The delay time from when the image data is transmitted to the site B2 until it is displayed on the display device 22 is Therefore, the timing of all performances at locations B1 to B3 displayed at location B2 will not match. stomach.

[0074] Therefore, the transmitting / receiving unit 121, under the control of the control unit 122, transmits the delay time to the delay time management server 5. 9 is read from the timing instruction unit 29 and supplied to the control unit 291 of the timing instruction unit 29. The control unit 291 stores a delay time correction value based on the delay time in the delay time correction value storage unit 294. Here, an example is taken in which the delay time is used as the delay time correction value as is.

[0075] In the example shown in FIG. 10, the delay time of the timing instruction unit 29 included in the image transmitting / receiving device 120A is The delay correction value storage unit 294 calculates a delay time of 10.3 seconds between the sending point B1 and the receiving point B2. The delay time correction of the timing instruction unit 29 included in the image transmitting / receiving device 120C is performed in ms. The value storage unit 294 stores a delay time of 10.2 ms between the point B3 as the sending side and the point B2 as the receiving side. Hold.

[0076] The delay time correction value holding unit 294 of the timing instruction unit 29 included in the image transmitting / receiving device 120B In FIG. 10, the transmission / reception unit 121 does not need to store a delay time correction value. Since the delay time is not supplied to the timing instruction unit 29, the transmission / reception unit 121 and the timing instruction unit The dashed arrow indicates the connection between the display unit 29 and the display unit 29.

[0077] The control unit 291 of the timing instruction unit 29 included in the image transmitting / receiving device 120A starts playing. The performance will start at a delay time of 10.3 ms before 13:00:00, which is the time to A signal informing the timing is displayed on the display unit 295. The control unit 291 of the timing instruction unit 29 starts the performance at 13:00:00. The display shows a signal indicating when the performance will start 10.2 ms before the delay time. Display at 295.

[0078] The control unit 291 of the timing instruction unit 29 included in the image transmitting / receiving device 120B starts playing. The display will show a cue to indicate when the performance will start at 13:00:00, the time set for the performance. Display at 295.

[0079] The performers at each of the locations B1 to B3 are in charge of the timing instruction unit 29 at each of the locations B1 to B3. The performance starts in accordance with the signal displayed on the display unit 295 of the location B2. The image data displayed on the display device 22 is displayed so that the timing of the performances at all the locations B1 to B3 is synchronized. It will be displayed in the

[0080] Using the flowchart shown in FIG. 12, the timing for receiving image data is adjusted. The process executed by the image transmitting / receiving devices 120A to 120C will be described. In step S31, the control unit 122 of the receiving device 120B executes the following steps based on the image transmitting / receiving device 120B: The control unit 12 of the image transmitting / receiving devices 120A and 120C, which are the transmitting side bases, is set as the reference point. In step S32, the process proceeds to step S33. 2 acquires the corresponding delay time from the delay time management server 50.

[0081] In step S33, the control unit 291 of the image transmitting / receiving device 120A and 120C determines whether the delay time is The delay time is set as a delay time correction value in the delay time correction value holding unit 294. is used as the delay time correction value. In step S34, the performance start time is set based on the operation input from the operation unit 293.

[0082] The control unit 122 of the image transmitting / receiving devices 120A and 120C, which are the transmitting side bases, performs step S 351, the time that is earlier than the performance start time by the delay time correction value is set as the performance start timing. The control unit 122 of the image transmitting / receiving device 120B, which is the reference site, performs step S352. The performance start time is set as the performance start timing.

[0083] In step S361, the control unit 122 of the image transmitting / receiving device 120A and 120C Determine whether the time has reached the performance start timing. If it has not yet reached (NO), the control unit 122 repeats the process of step S361. If the time has reached the performance start timing (YES), the control unit 122 returns to step S Transition to 371.

[0084] In step S362, the control unit 122 of the image transmitting / receiving device 120B determines whether the current time is the time when the performance starts. Determine whether the current time has reached the performance start timing. If not (NO), the control unit 122 repeats the process of step S362. If the start timing has been reached (YES), the control unit 122 proceeds to step S372. To make.

[0085] In step S371, the control unit 122 of the image transmitting / receiving device 120A and 120C At the start timing, a signal informing the start of the performance is displayed on the display unit 295. In step S372, the control unit 122 of the image transmitting / receiving device 120B ends the performance. At the start timing, a signal informing the start of the performance is displayed on the display unit 295. Terminate the process.

[0086] As described above, in the image transmission system shown in FIG. The image transmitting device (image transmitting / receiving device 120A or 120C) includes a camera 11. The transmitting device receives an image generated by the camera 11 by capturing an image of a subject performing a predetermined action. The image data is transmitted to an image receiving device (image transmitting device) located at a reference point via a network 30. The received signal is transmitted to the receiving device 120B.

[0087] The timing adjustment method performed in the image transmission system is a method for adjusting timing of an image data received by an image receiving device. The delay time when the data is displayed on the display device 22 is measured. A delay time correction value based on the delay time is set in the timing instruction unit 29 arranged at the side site. The timing adjustment method is to adjust the timing by the timing instruction unit 29 when the subject performs a predetermined action. The specified operation will start at the time indicated by the delay time correction value before the time instructed to start. Cue the subject to let them know when to start shooting.

[0088] According to one or more embodiments of the timing adjustment method, a transmitting site and a reference site When image data of a subject performing a specific action that is performed individually is displayed at a reference point, The timing can be adjusted to match.

[0089] <Frame timing adjustment method when receiving image data> Even if the timing of the start of the performance is adjusted by the configuration shown in FIG. The timing of the image data frames generated by 0A to 120C is out of sync, When the moving images generated by the devices 120A to 120C are displayed while being switched, image noise occurs. This may occur.

[0090] FIG. 13 illustrates an image processing system configured to correct for timing discrepancies between frames of image data. The transmitter / receiver 120 and a delay circuit suitable for correcting the timing difference between the frames of image data are 10, the image transmitting / receiving device 50 is shown. 120B receives image data transmitted from the image transmitting / receiving devices 120A and 120C. .

[0091] The image transmitting / receiving device 120B includes a vertical synchronization delay circuit 123. The image data transmitted from the image transmitting / receiving devices 120A and 120C received by the transmitting / receiving unit 121 is shown in FIG. The image data output from the camera 11 and the image data output from the image transmitting / receiving device 12 are input. The vertical synchronization signal of the image data transmitted from 120A and 120C is output from the camera 11. It is assumed that the image data is delayed with respect to the vertical synchronization signal.

[0092] The vertical synchronization delay circuit 123 delays the vertical synchronization signal of the image data output from the camera 11. The vertical synchronization signal of the image data transmitted from the image transmitting / receiving devices 120A and 120C is To reduce the deviation, the vertical synchronization of the image data output from the camera 11 is delayed. Deviation amount of vertical synchronization signal of image data transmitted from the image transmitting / receiving devices 120A and 120C If the difference is not the same, the vertical synchronization delay circuit 123 delays the vertical synchronization of the image data with the smaller deviation. The vertical synchronization delay circuit 123 delays the signal from the camera 11 to match the signal with the larger deviation. By delaying the image data from these three images to match the amount of shift, Adjust vertical sync.

[0093] The vertical synchronization delay circuit 123 delays the vertical synchronization signal of the image data output from the camera 11. The first delay time of the vertical synchronization signal of the image data transmitted from the image transmitting / receiving device 120A is and a second delay time of the vertical synchronization signal of the image data transmitted from the image transmitting / receiving device 120C. The vertical synchronization delay circuit 123 supplies the first and second delay times to the transmitting / receiving unit 121. It functions as a vertical synchronization deviation measurement unit that measures the vertical synchronization deviation.

[0094] The transmitting / receiving unit 121, under the control of the control unit 122, sets the first and second delay times to the network. The first delay time is transmitted to the delay time management server 50 via the network 30. s, and the second delay time is 0.3 ms.

[0095] As shown in FIG. 14, the delay time management server 50 is configured to have the site B1 as the sending side and the site B2 as the receiving side. The delay time is 10.3 ms when using the first delay time plus 3.2 ms. The delay time management server 50 stores the time. The site B3 is the sending site and the site B2 is the receiving site. The delay time is 10.2 ms when using the first side and 0.3 ms as the second delay time. Remember the time.

[0096] Delay time correction value of the timing instruction unit 29 in the image transmitting / receiving devices 120A and 120C The holding unit 294 holds the delay times (10.3+3.2) ms and (10.2+0.3 ) ms is stored as the delay time correction value.

[0097] The control unit 291 of the timing instruction unit 29 in the image transmitting / receiving device 120A starts playing. The playback will start at a time (10.3 + 3.2) ms before the time of the A signal informing the timing is displayed on the display unit 295. The control unit 291 of the timing instruction unit 29 calculates a delay time correction value from the time when the performance starts. A signal indicating the timing to start playing (10.2 + 0.3) ms before the time is displayed on the display. Display at 295.

[0098] As a result, the image data displayed on the display device 22 at the location B2 is the same as that displayed on the display device 22 at all the locations B1 to B3. Not only are the performances of the image transmitting and receiving devices 120A to 120D displayed in a synchronized manner, but the image transmitting and receiving devices 120A to 120D are also displayed in a synchronized manner. The timing difference between the frames of image data generated by 120C is also corrected.

[0099] As time passes, the image data output from the camera 11 changes in accordance with the vertical synchronization signal. The vertical synchronization signal of the image data transmitted from the image transmitting / receiving device 120A or 120C is out of sync. If the vertical synchronization signal is being output, the vertical synchronization delay circuit 123 is used to reduce the deviation of the vertical synchronization signal. The vertical synchronization of the image data output from the camera 11 can be delayed.

[0100] Furthermore, the image transmitting and receiving device is configured to receive the vertical synchronization signal of the image data output from the camera 11. a first delay time of the vertical synchronization signal of the image data transmitted from the image transmitting and receiving device 120A; The second delay time of the vertical synchronization signal of the image data transmitted from the device 120C is the delay time management The image data is transmitted to the server 50 and updated to the latest first and second delay times. The timing of the frames of image data generated by the receiving devices 120A to 120C is set to a predetermined time. It is constantly adjusted to the optimum state every time.

[0101] As described above, in the image transmission system shown in FIG. 13, An image transmitting device (image transmitting / receiving device 120A or 12) equipped with a first camera (camera 11) 0C) is connected to an image receiving device equipped with a second camera (camera 11) and a display device 22. The image receiving device (image transmitting / receiving device 120B) is a device for measuring the delay time 28 and the vertical synchronization. The vertical synchronization delay circuit 123 functions as a time offset measurement unit.

[0102] The delay time measurement unit 28 measures the delay time generated by the first camera and transmitted by the image transmitting device to the image receiving device. The delay time between when the image receiving device receives the first image data and when the image data is displayed on the display device 22 is set to The vertical synchronization delay circuit 123 measures the first image data and the second image data generated by the second camera. A vertical synchronization error indicating a time when the vertical synchronization with the second image data displayed on the display device 22 is out of sync. The delay time management server 50 receives the image from the image receiving device via the network 30 and measures the amount of delay. The delay time and vertical synchronization deviation amount transmitted are stored.

[0103] The image transmission system shown in FIG. 13 includes an image transmission device installed at a transmission side site. A first camera captures a first subject performing a predetermined action. The image data is transmitted to an image receiving device located at a reference site via a network 30. The timing adjustment method performed in the image transmission system is The delay time when the first image data is displayed on the display device 22 is measured.

[0104] The timing adjustment method performed in the image transmission system includes: The second camera of the device captures a second subject performing a predetermined action. and measure a vertical synchronization deviation amount indicating a time during which vertical synchronization with the generated second image data is deviated. The timing adjustment method performed in the image transmission system is a timing adjustment method that uses a timer located at the transmitting site. The timing instruction unit 29 receives a delay time based on the delay time obtained by adding the delay time and the vertical synchronization deviation amount. Set the time correction value.

[0105] The timing adjustment method performed in the image transmission system is performed by the timing instruction unit 29. The delay time correction value is displayed from the time when the first subject is instructed to start a predetermined action. The timing to start the predetermined action is set to be informed to the first subject at a time before the predetermined time. Figure.

[0106] According to one or more embodiments of the delay time management server, the delay time management server operates as an image transmission device. The image transmitting / receiving device 120 transmits the image data to the image receiving device 100 via the network 30. More accurate delay time and vertical synchronization when transmitting image data to the image transmitting / receiving device 120 By using the timing adjustment method of one or more embodiments, If the image is captured, the image of the subject performing a predetermined action performed separately at the transmitting site and the reference site is Timing, including the timing of frames of image data when the data is displayed at a reference point You can adjust the settings to match.

[0107] The present invention is not limited to one or more of the above-described embodiments. Various modifications are possible within the scope of the present invention. [Explanation of symbols]

[0108] 10, 10A~10C Image transmitting device 11 Camera 12 Transmitter 13,23 GNSS satellite radio clock 14,24 Clock generation unit 15,27 Image memory 16,25 Communications Department 20 Image receiving device 21 Receiving unit 22 Display device 26,113 Image processing circuit 28 Delay time measurement unit 29 Timing indicator 30 Network 40 satellites 50 Delay Time Management Server 111,122,221 Control unit 112 Image sensor 120A~120C Image transmission / reception device 121 Transmitter / Receiver 123 Vertical sync delay circuit (vertical sync deviation measurement unit) 150,270 Write / read control section 222 Drive circuit 223 LCD panel

Claims

1. The first clock number, the second clock number, the third clock number, and the a first communication unit that receives image data on which at least one of the block numbers is superimposed; 、 The first clock number is generated by a first clock generating unit included in the image transmitting device. a first clock including time information based on radio waves received from a satellite for a global navigation satellite system; The image sensor of the camera of the image transmission device is a clock number in the frame. indicates the time when the pixel data of the reference pixel in the frame was generated, The second clock number is a clock number in the first clock, and the image a first write / read control unit provided in the image transmitting device for writing pixel data of the reference pixel to a first image; Indicates the time when the data was written to memory. The third clock number is a clock number in the first clock, and the image a time when a second communication unit included in the image transmitting device transmitted pixel data of the reference pixel; time information based on radio waves received from a satellite for the global navigation satellite system, a second clock generating unit that generates a second clock identical to the first clock; The second clock operates, and the second write / read control unit controls the second write / read control unit. a second image memory into which the image data is written and from which the image data is read; a display device that displays the image data read from the second image memory; any one of the first to third clock numbers and the first communication unit a fourth clock in the second clock indicating the time when the pixel data of the reference pixel is received by the The second write / read control unit writes the pixel data of the reference pixel to the second pixel. a fifth clock number in the second clock indicating the time of readout from the image memory; The second clock indicates the time when the display device displays the pixel data of the reference pixel. and the sixth clock number in the second clock signal. a delay time measurement unit for determining The image receiving device further comprises:

2. The image data transmitted from the image transmitting device includes a first clock number, a second clock number, and a at least one of the first clock number, the second clock number, and the third clock number is superimposed; The first clock number is generated by a first clock generating unit included in the image transmitting device. a first clock including time information based on radio waves received from a satellite for a global navigation satellite system; The image sensor of the camera of the image transmission device is a clock number in the frame. indicates the time when the pixel data of the reference pixel in the frame was generated, The second clock number is a clock number in the first clock, and the image a first write / read control unit provided in the image transmitting device for writing pixel data of the reference pixel to a first image; Indicates the time when the data was written to memory. The third clock number is a clock number in the first clock, and the image a time when a first communication unit included in an image transmitting device transmitted pixel data of the reference pixel; The image receiving device that received the image data receives the fourth clock number, the fifth clock number, , sixth clock number; The fourth clock number is determined by radio waves received from a satellite for the global navigation satellite system. a clock number of a second clock identical to the first clock containing time information based on the first clock; a second communication unit provided in the image receiving device receives pixel data of the reference pixel; Shows the time, The fifth clock number is a clock number in the second clock, and the image A second write / read control unit provided in the image receiving device writes the image data to a second image memory. and indicates the time when the pixel data of the reference pixel is read out from the second image memory. The sixth clock number is a clock number in the second clock, and the sixth clock number is a clock number in the second clock. A display device provided in the image receiving device displays the image of the reference pixel read from the second image memory. Indicates the time when the raw data was displayed, Any one of the first to third clock numbers and any one of the fourth to sixth clock numbers The difference between the clock numbers and any one of the clock numbers is calculated, and the image transmission The image data generated by the image receiving device and transmitted to the image receiving device is received by the image receiving device. Measure the delay time when Delay time measuring device.

Images