Video delay measuring system

Abstract

The present disclosure is a control device that changes the light-emitting state of a light-emitting element, acquires a video signal obtained by imaging a change in the light-emitting state of the light-emitting element, detects the change in the light-emitting state in the video signal, and measures a delay from a change time point, at which the light-emitting state of the light-emitting element was changed, to a detection time point, at which the change in the light-emitting state in the video signal was detected.

Description

Video Delay Measurement System

[0001] The present disclosure relates to video delay measurement techniques.

[0002] In recent years, with the trend toward remote work, low-latency video transmission systems have become increasingly important. Low-latency video transmission systems are expected to enable remote collaboration, such as remote ensembles, where highly accurate synchronization is required, requiring low latency on the order of milliseconds across the entire system. For example, a 120 BPM (beats per minute) rhythm has a beat of approximately 500 ms. If a deviation of approximately 5% is allowed, the entire synchronized system must have a latency of approximately 25 ms. Designing and building such systems requires accurate measurement of the system's latency, both overall and for each individual component.

[0003] As such a measurement system, for example, a method is known in which a light-emitting device such as an LED (light-emitting diode) is placed in front of a camera, which is the input of the system, and a light-receiving element is placed at the end of a monitor or the like of the system's output, and the delay of the system is calculated from the time difference between the light emitted by the light-emitting device and the light received by the light-receiving element [see, for example, Non-Patent Document 1].

[0004] This type of measurement system has the problem that measurement errors are large, and in order to obtain accurate values, multiple sample measurements are required for averaging. This is partly due to the difference between the timing of the LED placed in front of the camera to emit light and the timing of the camera to capture the image, as well as the length of the exposure time.

[0005] For example, with a 30 fps (frames per second) camera, the imaging interval is 1 / 30 = approximately 33 ms, so the image may be captured as early as 0 ms after the LED is turned on, or as late as 33 ms after the LED is turned on. This contributes to the variation in measurement values, making accurate system delay measurement difficult. As a result, to improve measurement accuracy, long-term measurements are required, which significantly reduces the simplicity of the method. Furthermore, short-term measurements do not provide high accuracy. Furthermore, such measurements can determine the delay of the entire system, but cannot determine the delay of each component, such as the delay of the camera alone.

[0006] Therefore, there is a need to provide a system that can measure video delays more quickly and accurately, especially for delay measurements of imaging devices such as cameras, which are prone to large errors.

[0007] Bachhuber, Christopher, and Eckehard Steinbach. “A system for high precision glass-to-glass delay measurements in video communication.”2016 IEEE international conference on image processing (ICIP). IEEE, 2016.

[0008] An object of the present disclosure is to enable accurate measurement of delay in a video signal in a short time.

[0009] The time from when a light-emitting element is instructed to emit light to when the light-emitting element emits light is much shorter than the delay in an imaging device. Therefore, the present disclosure measures the delay of a video signal in an imaging device or the like based on the timing at which the light-emitting element emits light.

[0010] The control device of the present disclosure executes the video delay measurement method of the present disclosure, which includes: changing a light-emitting state of a light-emitting element; acquiring a video signal capturing the change in the light-emitting state of the light-emitting element; detecting the change in the light-emitting state in the video signal; and measuring a delay from a point in time at which the control device changes the light-emitting state of the light-emitting element to a point in time at which the control device detects the change in the light-emitting state in the video signal.

[0011] The control device may detect a change in the light emission state of a particular pixel in the video signal.

[0012] The control device may vary a difference between the change time point and the detection time point and measure a change in the delay corresponding to the difference. The control device may vary the difference by a first interval and measure the change in the delay by varying the difference by a second interval narrower than the first interval between the times when the change in the delay switches from a linear change.

[0013] Specifically, the video delay measurement system of the present disclosure includes an imaging device that captures the change in the light-emitting state of the light-emitting element, and a control device of the present disclosure that acquires the video signal captured by the imaging device.

[0014] The video delay measurement system of the present disclosure may further include a projection device that displays the video captured by the imaging device, and a light receiving element that detects a change in the light-emitting state of the light-emitting element displayed on the projection device. In this embodiment, the control device may measure a delay from the time point at which the change in the light-emitting element occurs to the time point at which the change in the light-emitting state is detected by the light receiving element.

[0015] The video delay measurement system of the present disclosure may include a first network device that transmits the video signal captured by the imaging device to the projection device, and a second network device that receives the video signal transmitted from the first network device, in which the imaging device and the projection device are connected via a communication network. In this embodiment, the projection device may display the video signal received by the second network device, and the control device may measure the delay from the time point at which the light-emitting element changes to the time point at which the light-emitting element is displayed.

[0016] The above disclosures can be combined as much as possible.

[0017] According to the present disclosure, it is possible to accurately measure delay in a video signal in a short time.

[0018] 1 shows an example embodiment of a video delay measurement system; 2 shows an example embodiment of a video delay measurement method; 3 shows an explanatory diagram illustrating an example of delay measurement; 4 shows an example embodiment of a video delay measurement system; 5 shows an example embodiment of a video delay measurement method; 6 shows an example setting of a deviation width δ in step S14; 7 shows an example setting of a deviation width δ in step S14; 8 shows an example of a delay profile; 9 shows an example embodiment of a video delay measurement system; 10 shows an example arrangement of light receiving elements; 11 shows an example embodiment of a video delay measurement system; 12 shows an example embodiment of a video delay measurement system.

[0019] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0020] 1 shows an example of a system configuration according to the present disclosure. The video delay measurement system of this embodiment includes a light-emitting element 11, an imaging device 12, and a control device 13. These devices may be directly connected, or may be indirectly connected via a network or the like.

[0021] The control device 13 is any device capable of performing calculations, such as a computer equipped with a processor and a memory. The light-emitting element 11 is any light-emitting body controllable by the control device 13, such as an LED. The imaging device 12 is any device capable of capturing images, such as a camera.

[0022] 2 shows an embodiment of a video delay measuring method. The video delay measuring method of this embodiment includes the following steps: S11: The control device 13 transmits a control signal to the light emitting element 11 to change the light emitting state of the light emitting element 11. In the present disclosure, the timing of changing the light emitting state is referred to as a change time t ｃ S12: The imaging device 12 captures an image of a change in the light-emitting state of the light-emitting element 11. The image signal captured by the imaging device 12 is input to the control device 13. The imaging of the light-emitting element 11 by the imaging device 12 may be started before step S11. S13: The control device 13 detects a change in the light-emitting state in the image signal captured in step S12. The time when the control device 13 receives the detected image signal as an input is called the detection time t ｓｅｎｓｅ The control device 13 determines the time of change t ｃ From the detection time t ｓｅｎｓｅ The delay D until

[0023] In this embodiment, the test target is only the imaging device 12, so by measuring the delay D, the delay D in the imaging device 12 ｃａｍｅｒａ Here, the detection time of the change in the light emission state in the control device 13 requires a detection processing time, so t ｓｅｎｓｅ However, if the detection processing time is sufficiently short for the required accuracy, the detection time after the detection processing is set to t ｓｅｎｓｅ If the detection processing time is long, the time itself can be measured and subtracted to obtain t ｓｅｎｓｅ It may also be possible to use the following.

[0024] The change in the light-emitting state of the light-emitting element 11 can be exemplified by, for example, changing the light-emitting element 11 from a non-light-emitting state to a light-emitting state, or changing the light-emitting element 11 from a light-emitting state to a non-light-emitting state. ｃThe timing at which the light emitting element 11 emits light is set as the light emission time t ｏｎ The timing at which the light emitting element 11 is turned off is referred to as the light-off time t ｏｆｆ It may be written as follows.

[0025] For example, as shown in FIG. 3, in step S11, the control device 13 causes the light emitting element 11 to emit light, in step S12 the imaging device 12 captures the light emitted from the light emitting element 11, and in step S13 the control device 13 detects the light emission time t ｏｎ and detection time t ｓｅｎｓｅ This allows the delay D to be measured.

[0026] For example, in step S11, the control device 13 turns off the light emitting element 11, in step S12 the imaging device 12 captures an image of the light emitting element 11 being turned off, and in step S13 the control device 13 captures the light emitting element 11 at the time t ｏｆｆ and detection time t ｓｅｎｓｅ This allows the delay D to be measured.

[0027] The time from when the control device 13 transmits a control signal to the light-emitting element 11 until the light-emitting state of the light-emitting element 11 changes is significantly shorter than the delay in the imaging device 12. Therefore, in step S13, the control device 13 can measure the delay in the imaging device 12.

[0028] The change in the light-emitting state in step S11 is not limited to ON / OFF of brightness, but may be a change in brightness of three or more stages. Also, the change in the light-emitting state in step S11 may be not only brightness but also a change in other color perception parameters such as color. This reduces the delay D ｃａｍｅｒａ This will enable measurements to be taken more quickly and with higher accuracy.

[0029] Second Embodiment Fig. 4 shows an example system configuration of the present disclosure. The processing delay D may vary depending on the imaging position in the imaging device 12. Furthermore, by reducing the amount of calculation required for the sensing process, the delay D can be measured more accurately. Therefore, the control device 13 designates specific pixels in the video signal captured by the imaging device 12 as sensing target pixels, and detects changes in the light emission state of the sensing target pixels.

[0030] For example, the control device 13 adds a guide frame around a sensing target pixel in the imaging device 12 and detects changes in the sensing target pixel surrounded by the guide frame. The guide frame can have any shape that allows the sensing target pixel to be identified. The guide frame can be added, for example, by overlaying it on the captured image in the video signal and combining it. The guide frame can make it easier to align the imaging device 12.

[0031] The sensing target pixel is typically located in the center of the screen, but it can also be located at the top or bottom of the screen, etc. The sensing target pixel can be measured accurately by making the width of the sensing target pixel between several lines and several tens of lines in the vertical direction.

[0032] In many cases, the delay D changes little in the horizontal direction of the screen, allowing the sensing pixel position to be wider in the horizontal direction. For example, for a 30 Hz video signal, ignoring the overhead of the video signal, one screen frame is 1 / 30 = 33.3 ms. If the video signal pixels are 1920 x 1080p, the time per line is 33.3 ms / 1080 = 30.9 μs. A vertical shift of 100 lines can result in a deviation of approximately 3 ms in the measurement value. The deviation per dot in the horizontal direction is 30.9 μs / 1920 = 16.1 ns, and even if there is a horizontal shift of 100 dots, it is only about 1.6 μs, which is sufficiently small relative to the measurement accuracy.

[0033] 3, a video signal is composed of a plurality of screen frames, and in this embodiment, the period of one screen frame is set to T. In this embodiment, the change in the light emitting state of the light emitting element 11 occurs at the light emitting time t ｏｎ Here is an example:

[0034] The delay D measured in step S13 is calculated based on the light emission time t ｏｎ and detection time t ｓｅｎｓｅ Therefore, the video delay measuring method of this embodiment includes step S14 before step S11, as shown in FIG.

[0035] In step S14, the control device 13 adjusts the light emission time t ｏｎ For example, the control device 13 adjusts the deviation width δ shown in FIG. ３ , and the deviation width δ shown in FIG. ４ At this time, the deviation width δ ３ Deviation width δ ４ The controller 13 varies the deviation δ by the first interval and measures the change in the delay D.

[0036] Here, since the frame period is T, the delay D is also switched at the period T. Therefore, the control device 13 sets the delay width δ to t ｒｏｕｇｈ −T<δ<t ｒｏｕｇｈ +T. As a result, the control device 13 changes the deviation width δ within the range of δ +T, as shown by the black dots in FIG. １ ~δ １０ The delay D when the ｒｏｕｇｈ If the measurement accuracy is poor, this δ width can be increased.

[0037] The change in delay D switches from a linear change to a linear change in accordance with the frame period T. For example, in FIG. ３ and δ ４ The delay D switches between ７ and δ ８ Therefore, the control device 13 measures the change in the delay D by changing the delay D between the deviation width δ when the change in the delay switches from a linear change to a second interval that is narrower than the first interval. For example, ３ and δ ４ and the offset width δ ７ and δ ８ 8, the control device 13 changes the delay D to the maximum value D. ｍａｘ and the minimum value D ｍｉｎ It is also possible to control δ more finely in n steps and change it at n-th intervals. By narrowing down the measurement delay range in this way, it is possible to perform measurements with high precision and high speed.

[0038] In the system configuration shown in FIG. ｍｉｎ The control device 13 measures the maximum value D of the delay D. ｍａｘ and minimum value D ｍｉｎ The average value of the delay D may be calculated using the following formula: ｍａｘ , minimum value D ｍｉｎ , average value D ａｖｅ The method of deriving t is not limited to this embodiment. For example, the deviation width δ may be decreased at regular intervals. ｒｏｕｇｈ −T<δ<t ｒｏｕｇｈ The shift width δ in the entire range of +T may be set to the n-th interval from the beginning.

[0039] The deviation width δ can be based on an arbitrary signal within one screen frame of the video signal from the imaging device 12. For example, as shown in FIG. ｓｅｎｓｅ However, if a pixel to be sensed is determined, the pixel to be sensed can be used as a reference.

[0040] In step S14, the control device 13 may calibrate a threshold value for determining a change in the light-emitting state of the light-emitting element 11. For example, the control device 13 may calibrate the average intensity I of the luminance of the light-emitting element 11 in the extinction state. ｏｆｆ , and the average brightness intensity I of the light-emitting element 11 in the light-emitting state. ｏｎ Difference I ｄｉｆｆ The threshold value is set based on the calculated value. For example, the threshold value can be as follows: First threshold value: I ｏｆｆ +0.2*I ｄｉｆｆ (20% intensity) Second threshold: I ｏｆｆ +0.5*I ｄｉｆｆ (50% intensity) Third threshold: I ｏｆｆ +0.8*I ｄｉｆｆ (80% strength)

[0041] The brightness of the sensing pixel can be calculated as the overall average value, but if the sensing pixel range is wide, pixels with a brightness below a certain level can be excluded from the average value. This makes it possible to detect changes in the light emission state with high sensitivity, particularly when the light-emitting area of ​​the light-emitting element 11 is narrower than the sensing pixel.

[0042] The time for which the light emitting element 11 continues to emit light when it is turned on in step S11 and the time for which the light emitting element 11 continues to be extinguished when it is turned off in step S11 are t ｒｏｕｇｈ It is possible to set the value to +2T or more, which makes it possible to shorten the measurement time.

[0043] 9 shows an example of a system configuration according to the present disclosure. The system of this embodiment further includes a projection device 21 and a light receiving element 22. In this embodiment, the projection device 21 displays a change in the light emitting state of the light emitting element 11 at a display time t ｄ is the detection time t ｓｅｎｓｅ In this embodiment, the change in the light emitting state of the light emitting element 11 occurs at the light emitting time t ｏｎ Here is an example:

[0044] The projection device 21 is any device capable of displaying an image captured by the imaging device 12, and examples thereof include a display and a projector. In this embodiment, an example is shown in which a video signal output to the control device 13 is displayed on the projection device 21. The light-receiving element 22 is any device capable of detecting a change in the light-emitting state of the light-emitting element 11 displayed on the projection device 21, and a photodiode, for example, can be used. A photodiode is a semiconductor that can immediately detect received light as a current, so by monitoring that current (or the converted voltage), the light-emitting state of the projection device 21 can be monitored more directly and with less delay.

[0045] 10 shows an example of the arrangement of the light receiving element 22. The projection device 21 is a device that displays part or all of the image captured by the imaging device 12. The light receiving element 22 is attached at a position where it can capture the part of the pixel on the screen of the projection device 21 that corresponds to the light emitting element 11. When the projection device 21 and the light receiving element 22 are separated from each other, partial light that corresponds to the pixel to be sensed on the screen of the projection device 21 can be collected through an optical system and received by the light receiving element 22.

[0046] The light receiving element 22 detects a change in the light emission state of the pixel corresponding to the light emitting element 11 captured by the imaging device 12. In this embodiment, the light receiving element 22 detects that the light emission of the light emitting element 11 is displayed in the projection device 21. As a result, the display time t when the change in the light emission state of the light emitting element 11 is displayed in the projection device 21 is detected. ｄ can be detected.

[0047] The control device 13 determines the display time t ｄ Thus, the control device 13 directly or indirectly acquires the display time t ｄ The control device 13 can acquire the light emitting time t ｏｎ From the display time t ｄ Measure the delay until

[0048] This embodiment makes it possible to measure the delay of the entire system, including the delay of the image capture device 12, the projection device 21, and other delays. In other words, it is possible to measure the delay of the entire system more accurately in a shorter time. Furthermore, by subtracting the delay of the image capture device 12, it is possible to measure the delay of the remaining measurement targets.

[0049] 11 shows an example of a system configuration according to the present disclosure. In the system of this embodiment, an image capturing device 12 and a projection device 21 are connected via a communication network 31. This embodiment includes a first network device 32 that connects the image capturing device 12 to the communication network 31, and a second network device 33 that connects the projection device 21 to the communication network 31. In this embodiment, too, a change in the light emitting state of the light emitting element 11 occurs at a light emitting time t ｏｎ and the display time t when the change in the light-emitting state of the light-emitting element 11 is displayed on the projection device 21. ｄ is the detection time t ｓｅｎｓｅ This becomes:

[0050] In this embodiment, a video signal captured by the imaging device 12 is branched and output to a first network device 32 and a control device 13. The first network device 32 transmits the video signal captured by the imaging device 12 to a second network device 33. The second network device 33 receives the video signal from the first network device 32 and outputs it to a projection device 21. The projection device 21 displays the video signal received by the second network device 33. The control device 13 determines the light emitting time t of the light emitting element 11. ｏｎ From the display time t ｄ Measure the delay until

[0051] In this embodiment, it is possible to measure delays including the communication network 31 in addition to the image capturing device 12 and the projection device 21 .

[0052] Sixth Embodiment Fig. 12 shows an example of a system configuration according to the present disclosure. In the system of this embodiment, a control device 13 and the like are incorporated into the test subject. For example, as shown in the figure, the first network device 32 may have the functions of the control device 13. The figure shows an example of this embodiment, and the control device 13 may be incorporated into another device. Furthermore, a light receiving element 22 or a light emitting element 11 may be incorporated into the test subject.

[0053] Other Embodiments The delay measurement system and control device 13 of the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0054] 11: Light emitting element 12: Image capturing device 13: Control device 14, 21: Projection device 22: Light receiving element 31: Communication network 32: First network device 33: Second network device

Claims

1. A control device that changes a light-emitting state of a light-emitting element, acquires a video signal capturing the change in the light-emitting state of the light-emitting element, detects the change in the light-emitting state in the video signal, and measures a delay from the point in time when the light-emitting state of the light-emitting element is changed to the point in time when the change in the light-emitting state in the video signal is detected.

2. The control device according to claim 1, wherein a change in the light emission state of a specific pixel in the video signal is detected.

3. The control device according to claim 1, wherein the deviation between the change time point and the detection time point is changed, and the change in the delay corresponding to the deviation is measured.

4. The control device according to claim 3, wherein the change in delay is measured by changing the deviation width at a first interval and changing the deviation width at a second interval narrower than the first interval when the change in delay switches from a linear change.

5. A video delay measurement system comprising: an imaging device that captures images of changes in the light-emitting state of the light-emitting element; and a control device according to any one of claims 1 to 4 that acquires a video signal captured by the imaging device.

6. The video delay measurement system according to claim 5, further comprising: a projection device that displays the image captured by the imaging device; and a light-receiving element that detects a change in the light-emitting state of the light-emitting element displayed on the projection device, wherein the control device measures the delay from the time point at which the change in the light-emitting element occurs to the time point at which the change in the light-emitting state is detected by the light-receiving element.

7. The video delay measurement system of claim 6, wherein the imaging device and the projection device are connected via a communication network, the system comprises: a first network device that transmits the video signal captured by the imaging device to the projection device; and a second network device that receives the video signal transmitted from the first network device, the projection device displays the video signal received by the second network device, and the control device measures the delay from the time point at which the light-emitting element changes to the time point at which it is displayed.

8. A video delay measurement method, comprising: a control device changing a light-emitting state of a light-emitting element; the control device acquiring a video signal capturing the change in the light-emitting state of the light-emitting element; the control device detecting the change in the light-emitting state in the video signal; and measuring a delay from the point in time at which the control device changes the light-emitting state of the light-emitting element to the point in time at which the control device detects the change in the light-emitting state in the video signal.

Images