System, method, apparatus and program for measuring delay in a device

A system using a remote device and user terminal over a communication network measures device delays in video and audio devices without special equipment, addressing the impracticality of existing methods and enabling universal delay assessment.

JP7578196B2Active Publication Date: 2024-11-06NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023529408
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-06-25
Publication Date
2024-11-06
Estimated Expiration
2041-06-25

AI Technical Summary

Technical Problem

Existing methods for measuring device delays in video and audio devices require special equipment like LEDs and optical sensors, making widespread implementation impractical.

Method used

A system and method that uses a remote device connected via a communication network to a device processing video or audio data, with a user terminal transmitting and receiving signals to calculate device delays without requiring special equipment or precise time synchronization.

Benefits of technology

Enables measurement of device delays in video and audio devices without the need for specialized equipment or precise inter-device synchronization, facilitating universal and practical delay time assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578196000001
    Figure 0007578196000001
  • Figure 0007578196000002
    Figure 0007578196000002
  • Figure 0007578196000003
    Figure 0007578196000003
Patent Text Reader

Abstract

The purpose of the present disclosure is to enable measurement of a delay time having occurred in a device for inputting or outputting video or audio, without the need of a special apparatus or accurate inter-apparatus clock time synchronization. A system according to the present disclosure is provided with: a remote apparatus connected via a communication network to a device for executing a process for outputting or inputting data of video or audio; and a user terminal that transmits / receives a signal to / from the remote apparatus through the communication network. The system performs a process for outputting or inputting video or audio of a predetermined pattern in the device and the remote apparatus, detects output or input of the video or audio of the pattern in the user terminal or the remote apparatus, and calculates a delay time in the device using the detected time as a reference and a time at which the device and the remote apparatus have transmitted the video or audio of the pattern through the communication network.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for measuring delay time in a device that inputs or outputs video or audio. [Background technology]

[0002] With the spread of remote work, communication using video and audio is becoming more important. To facilitate smooth communication using video and audio, it is important to reduce video and audio delays. Video and audio delays are not only caused by communication delays that occur on the network, but also by device delays that occur in devices such as cameras, microphones, speakers, and displays.

[0003] One method for measuring device delay is to use an LED and an optical sensor (see, for example, Non-Patent Document 1). With this method, it is possible to measure the delay time generated by the camera by using an oscilloscope to measure the time difference between when the LED is turned on and when the light is observed by the optical sensor.

[0004] In order to utilize the technology of Non-Patent Document 1, dedicated equipment such as an LED, an optical sensor, and an oscilloscope for measurement is required, so it is not realistic to implement it widely. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "2. How to measure latency", Technologies, Optical Path Communications, connecting high-definition images with optical technology, https: / / h-path.co.jp / technologies / 2-how-to-measure-latency / Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present disclosure is to make it possible to measure delay times occurring in devices that input or output video or audio, without requiring special equipment or precise time synchronization between devices. [Means for solving the problem]

[0007] The system of the present disclosure comprises: A remote device connected to a device that executes a process of inputting or outputting video or audio data through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, performing a predetermined pattern of video or audio output or input processing in the device and the remote device; detecting a visual or audio output or input of said pattern at said user terminal or said remote device; The delay time in the device is calculated based on the detected time and using the time when the device and the remote device transmitted the video or audio data of the pattern through the communication network.

[0008] The method of the present disclosure comprises: A system including a device that executes a process of inputting or outputting video or audio data, a remote device connected to the device through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, performing a predetermined pattern of video or audio output or input processing in the device and the remote device; detecting a visual or audio output or input of said pattern at said user terminal or said remote device; The delay time in the device is calculated based on the detected time and using the time when the device and the remote device transmitted the video or audio data of the pattern through the communication network.

[0009] The device of the present disclosure comprises: A device that functions as a user terminal and is provided in a system including a remote device connected to a device that executes a process of outputting or inputting video or audio data through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, A predetermined pattern of video or audio output or input processing is performed in the device and the remote device; when a video or audio output or input of said pattern is detected at said user terminal or said remote device, The delay time in the device is calculated based on the detected time and using the time when the device and the remote device transmitted the video or audio data of the pattern through the communication network.

[0010] The apparatus of the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network. Effect of the Invention

[0011] According to the present disclosure, it becomes possible to measure the delay time that occurs in a video device without requiring special equipment or precise time synchronization between devices. [Brief description of the drawings]

[0012] [Figure 1] 1 illustrates an example system configuration according to an embodiment of the present disclosure. [Diagram 2] 1 illustrates an example system configuration according to an embodiment of the present disclosure. [Diagram 3] 1 shows an example of a delay database. [Figure 4] 1 shows an example of functions provided in a user terminal. [Diagram 5] 1 illustrates an example system configuration according to an embodiment of the present disclosure. [Figure 6] 1 illustrates an example system configuration according to an embodiment of the present disclosure. [Figure 7] 1 shows an example of a delay database. [Figure 8]1 shows an example of functions provided in a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings are assumed to indicate the same components.

[0014] The delay time in a communication network can be measured. Therefore, the present disclosure includes a device that executes a process of inputting or outputting video or audio data and a remote device connected via the communication network. Then, the device responds via the communication network, and the delay time in the communication network is subtracted from the response time of the device via the communication network, thereby making it possible to measure the response time of the device alone at the user terminal.

[0015] In this disclosure, a device is any device that executes a process of inputting or outputting video or audio media data, such as a camera, a monitor, a microphone, or a speaker. A camera converts video into video data and outputs the video data. A monitor receives video data and outputs video. A microphone converts audio into audio data and outputs the audio data. A speaker receives audio data and outputs audio. In this disclosure, a method for measuring the response time of a device alone when the media data is video data will be described.

[0016] The present disclosure provides a method for processing the input or output of a predetermined pattern of video in a device and in the remote device, Detecting an output or input of an image of the pattern at the user terminal or the remote device; A delay time in the device is calculated based on the detected time and using the time when the device and the remote device transmitted the image data of the pattern through the communication network.

[0017] For example, if the device is a camera, the camera and the response device are connected through a communication network. In this state, the screen of the user terminal is changed in a predetermined pattern, the change in the screen is captured by the camera, and the response from the response device is received by the user terminal. The user terminal detects the time when the screen is changed in the pattern. The user terminal then measures the time from when the received screen is changed to when the response signal is received, and subtracts the delay time in the communication network from the measured time, thereby making it possible to measure the delay time in the camera.

[0018] For example, if the device is a display, the display and the control device are connected through a communication network. In this state, a request signal is sent from the user terminal to the control device, and the image on the display is changed in a predetermined pattern. The user terminal detects the change in the pattern of the image on the display. The user terminal then measures the time from sending the request signal to the time the image on the display changes, and subtracts the delay time in the communication network from the measured time, thereby making it possible to measure the delay time on the display.

[0019] The video data is any data that can be processed by a device, for example, a High-Definition Multimedia Interface (HDMI) signal (HDMI is a registered trademark). An example in which the video data is an HDMI signal will be described in detail below.

[0020] (First embodiment) 1 shows an example of a system configuration according to this embodiment. The video delay measurement system according to this embodiment includes a user terminal 11, a camera 21 which is a device to be measured, a video analysis unit 12, and a response unit 13. The user terminal 11 includes an application that measures the delay time of an HDMI signal output from the camera 21.

[0021] The camera 21 is connected to the video analysis unit 12 and the response unit 13 through a communication network 24. In this embodiment, MCs (Media Converters) 22 and 26 for transmitting an HDMI signal from the camera 21 through the communication network 24 are provided.

[0022] The video delay measurement system of this embodiment executes the following steps. S101: Using Ping or the like, the delay time T N is measured in advance. S102: The user terminal 11 displays a predetermined video pattern on the screen. The video pattern is any video in which a change in the screen can be detected by the video analysis unit 12. The camera 21 captures the changing screen. The HDMI signal from the camera 21 is transmitted through the communication network 24 using the MCs 22 and 26. S103: The video analysis unit 12 acquires the HDMI signal from the MC 26, and transmits a signal to the response unit 13 at the timing when it detects that the screen has become white. This allows the communication network 24 to detect the change in the screen. S104: The response unit 13 returns a response signal upon receiving the signal from the video analysis unit 12. The destination of the response signal can be set to any destination, for example, the user terminal 11. S105: The user terminal 11 measures the round trip time from when the screen is changed to when the response signal is received, and uses this to measure the delay in the camera 21. For example, the delay time T N The delay at the camera 21 is measured by subtracting

[0023] In step S102, the user terminal 11 displays a screen according to the control of the measurement application. For example, the image pattern is one in which the screen display is switched in the order of black, black, white, white, white at regular intervals. The response unit 13 detects the black, black, white, white, white pattern and generates a response signal at the time when the screen changes from black to white.

[0024] Furthermore, in step S104, response unit 13 may transmit the time of receiving the signal from video analysis unit 12 to user terminal 11 using a response signal. In this case, in step S105, user terminal 11 measures the delay in camera 21 using the time from displaying the video pattern on the screen to the time notified by the response signal, instead of the round-trip time from changing the screen to receiving the response signal.

[0025] In this embodiment, an example is shown in which the delay time is measured using the camera 21, but the present disclosure is not limited to this. For example, a microphone may be used instead of the camera 21. In this case, an audio analysis unit is provided instead of the video analysis unit 12. Then, in step S102, the user terminal 11 changes the output audio, in step S103 the microphone outputs audio data, and in step S104 the audio analysis unit detects the change in the audio.

[0026] Second embodiment An example of a system configuration according to this embodiment is shown in Fig. 2. The video delay measurement system according to this embodiment includes a video encoder 22E as MC 22, a video decoder 26D as MC 26, the video encoder 22E is connected to a communication network 24 via a router 23, the video decoder 26D is connected to the communication network 24 via a router 25, and a delay database in which delay times in the communication network 24 are recorded. Also, the video analysis unit 12 and the response unit 13 are provided in a response device.

[0027] An example of the delay database is shown in FIG. 3. The delay database includes the delay time T N , the delay time T E , the delay time T E , the delay time at the answering device T R Includes.

[0028] The HDMI signal from the camera 21 is input to the video encoder 22E using an HDMI cable. The video encoder 22E converts the HDMI signal into a format that can be transmitted through the communication network 24. For example, the HDMI signal is converted into an IP packet. As a result, the HDMI signal from the video encoder 22E is input to the router 23 using the LAN, transmitted from the router 23 to the router 25 using the communication network 24, and input from the router 25 to the video decoder 26D using the LAN.

[0029] The video decoder 26D converts the IP packets into an HDMI signal and outputs it to the video analysis unit 12. The video analysis unit 12 detects that the color has changed from black to white using the HDMI signal. When the video analysis unit 12 detects that the color has changed from black to white, the response unit 13 transmits a response signal to the router 25 at that timing. The router 25 transmits the response signal to the router 23, and the router 23 transmits the response signal to the user terminal 11. Here, since the response signal does not include a video signal, any data transmission method can be used; for example, a LAN can be used between the response unit 13 and the router 25, and a wireless LAN such as WiFi can be used between the router 23 and the user terminal 11.

[0030] 4 shows an example of functions provided in the user terminal 11. The applications provided in the user terminal 11 include a screen control function 111, a response receiving function 112, an other delay acquisition function 113, and a delay calculation function 114. The screen control function 111 performs any display detectable by the video analysis unit 12, and notifies the time when the display is performed as the control time. For example, the display switches the screen display in the order of black, black, white, white, and white at regular intervals. The response receiving function 112 receives a response signal from the response unit 13, and notifies the delay calculation function 114 of the time when the response signal was received. The other delay acquisition function 113 acquires delay time information from a delay database, and notifies the delay calculation function 114 of the delay time information. The delay calculation function 114 subtracts the delay time acquired by the other delay acquisition function 113 from the time when the response signal is received, to obtain the delay time in the camera 21 .

[0031] (Third embodiment) An example of a system configuration according to this embodiment is shown in Fig. 5. The video delay measurement system according to this embodiment includes a user terminal 31, a display 41 as a measurement target, a video output unit 32, and a control unit 33. The user terminal 11 includes an illuminance sensor that detects the display on the display 41, and an application that measures the time from when a request signal for changing the output video is transmitted until a change in illuminance is detected.

[0032] The display 41 is connected to the video output unit 32 and the control unit 33 through a communication network 44. In this embodiment, MCs (Media Converters) 46 and 42 for transmitting an HDMI signal from the video output unit 32 through the communication network 44 are provided.

[0033] The video delay measurement system of this embodiment executes the following steps. S201: Using Ping or the like, measure the delay time T N is measured in advance. S202: The user terminal 11 transmits a request signal to change the output video. S203: When the video output unit 32 receives the request signal, it outputs an HDMI signal that changes the output video. The HDMI signal from the video output unit 32 is transmitted through the communication network 44 using the MCs 46 and 42. S204: The display 41 receives the HDMI signal from the MC 42 and displays it. S205: The user terminal 31 detects a change in the image on the display 41 that occurs over the network. Then, the user terminal 31 measures the time from when the request signal is transmitted until when the change in the image on the display 41 is detected, and measures the delay time on the display 41 using this time.

[0034] In step S203, the video output unit 32 outputs a video to be turned to white under the control of the control unit 33. For example, the video output unit 32 outputs an HDMI signal that switches in the order of black, black, white, white, and white at regular intervals. As a result, in step S203, a video that switches in the order of black, black, white, white, and white at regular intervals is displayed on the display 41.

[0035] In this embodiment, an example is shown in which the delay time is measured on the display 41, but the present disclosure is not limited thereto. For example, a speaker may be used instead of the display 41. In this case, an audio output unit is provided instead of the video output unit 32. Then, in step S203, the audio output unit changes the output audio, in step S204 the speaker outputs audio data, and in step S205 the user terminal 31 detects the change in the audio.

[0036] (Fourth embodiment) An example of a system configuration according to this embodiment is shown in Fig. 6. The video delay measurement system according to this embodiment includes a video encoder 46E as an MC 46, a video decoder 42D as an MC 42, the video encoder 46E is connected to a communication network 44 via a router 45, the video decoder 42D is connected to the communication network 44 via a router 43, and a delay database in which delay times in the communication network 44 are recorded. Furthermore, the video output unit 32 and the control unit 33 are provided in the control device.

[0037] An example of the delay database is shown in FIG. 7. The delay database includes the delay time T N , the delay time T E , the delay time T D , delay in the controller T C Includes.

[0038] Video encoder 46E converts the HDMI signal into a format that can be transmitted through communication network 44. As a result, the HDMI signal from video encoder 46E is input to router 45 using the LAN, transmitted from router 45 to router 43 using communication network 44, and input from router 43 to video decoder 42D using the LAN.

[0039] The video decoder 42D converts the signal from a format that can be transmitted through the communication network 44 into an HDMI signal and outputs it to the display 41. The display 41 displays an image that changes in a certain period of time in the order black, black, white, white, and white. The user terminal 31 detects the change in the image on the display 41 that occurs through the communication network 44.

[0040] 8 shows an example of functions included in the user terminal 31. The applications included in the user terminal 11 include an illuminance change detection function 211, a request transmission function 212, an other delay acquisition function 213, and a delay calculation function 214. The illuminance change detection function 211 acquires illuminance information from the illuminance sensor, detects when the illuminance changes from black to white, and notifies the delay calculation function 214 of the time when the illuminance change was detected. The request transmission function 212 transmits a request signal to the control unit 33, and notifies the delay calculation function 214 of the transmission time of the request signal. The other delay acquisition function 213 acquires delay time information from the delay database, and notifies the delay calculation function 214 of the information. The delay calculation function 214 calculates the delay in the display 41 by using the time when the illuminance change is detected, the time when the request signal is transmitted, and the delay time information acquired by the other delay acquisition function 213 .

[0041] (Effects of the present disclosure) According to the present disclosure, it becomes possible for the user terminal 11 to measure delays occurring in devices such as the camera 21 and the display 41 without requiring special equipment or precise time synchronization between devices.

[0042] In the above embodiment, an example is shown in which the delay time is measured using the camera 21 or the display 41, but the present disclosure is not limited to this. For example, the camera 21 may be replaced with a microphone, the display 41 may be replaced with a speaker, or a combination of these may be used.

[0043] Further, although an example is shown in which the remote device is a response device or a control device, the remote device may include both a response function equivalent to a response device and a control function equivalent to a control device.

[0044] In addition to the applications provided in the user terminals 11 and 31, any functions provided in the camera 21, video analysis unit 12, response unit 13, MCs 22, 26, 42, 46, video encoders 22E, 46E, video decoders 26D, 42D, display 41, video output unit 32, and control unit 33 can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided through any communication network. [Industrial Applicability]

[0045] The present disclosure can be applied to the information and communications industry. [Explanation of symbols]

[0046] 11, 31: User terminal 12: Video Analysis Department 13: Response section 21: Camera 22, 26, 42, 46:MC 22E, 46E: Video Encoder 23, 25, 43, 45: Router 26D, 42D: Video decoder 24, 44: Communication network 32: Video output section 33: Control unit 41: Display 111: Screen control function 112: Response reception function 113: Other delayed acquisition functions 114: Delay calculation function 211: Illuminance change detection function 212: Request sending function 213: Other delayed acquisition functions 214: Delay calculation function

Claims

1. A remote device connected to a device that executes a process of inputting or outputting video or audio data through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, performing a predetermined pattern of video or audio output or input processing in the device and the remote device; detecting a visual or audio output or input of said pattern at said user terminal or said remote device; Calculating a delay time in the device based on the detected time and using a time when the device and the remote device transmitted the image or audio data of the pattern through the communication network; system.

2. The device has a function of converting video or audio into media data and outputting the media data, the remote device has a response function for detecting a change in the image or sound included in the pattern from the media data; outputting an image or sound of the pattern on the user terminal; The video or audio of the pattern is converted into media data by the device; transmitting the media data to a remote device over the communications network; transmitting detection of the change in media data at the remote device as a response signal to the user terminal; Measure the time from when the user terminal outputs the image or sound of the pattern to when the response signal is received by the user terminal; Using the measured time to measure the delay at the device; The system of claim 1 .

3. The device has a function of outputting video or audio corresponding to media data, the remote device has a control function for outputting media data for outputting video or audio on the device; transmitting a request signal from said user terminal to said remote device over said communications network; The remote device that receives the request signal outputs the image or sound of the pattern from the device; The user terminal detects a change in video or audio included in the pattern output from the device; measuring a time from when the user terminal transmits the request signal to when the user terminal detects the change; Using the measured time to measure the delay at the device; 3. A system according to claim 1 or 2.

4. measuring a delay in the device by subtracting a delay time in the communication network from the measured time; 4. A system according to claim 2 or 3.

5. A system including a device that executes a process of inputting or outputting video or audio data, a remote device connected to the device through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, performing a predetermined pattern of video or audio output or input processing in the device and the remote device; detecting a visual or audio output or input of said pattern at said user terminal or said remote device; Calculating a delay time in the device based on the detected time and using a time when the device and the remote device transmitted the image or audio data of the pattern through the communication network; method.

6. A device that functions as a user terminal and is provided in a system including a remote device connected to a device that executes a process of outputting or inputting video or audio data through a communication network, and a user terminal that transmits and receives signals to and from the remote device through the communication network, A predetermined pattern of video or audio output or input processing is performed in the device and the remote device; when a video or audio output or input of said pattern is detected at said user terminal or said remote device, Calculating a delay time in the device based on the detected time and using a time when the device and the remote device transmitted the image or audio data of the pattern through the communication network; Device.

7. A program for implementing a computer as a user terminal in a system including a device that executes a process of inputting or outputting video or audio data, a remote device connected via a communication network, and a user terminal that transmits and receives signals to and from the remote device via the communication network, A predetermined pattern of video or audio output or input processing is performed in the device and the remote device; when a video or audio output or input of said pattern is detected at said user terminal or said remote device, Calculating a delay time in the device based on the detected time and using a time when the device and the remote device transmitted the image or audio data of the pattern through the communication network; A program that causes a computer to execute steps.

Citation Information

Patent Citations

  • System for accurately testing video monitoring time delay

    CN210518643U

  • Delay time measurement system and camera

    JP2016116107A

  • Information processing device, program, and information processing method

    JP2017112398A

  • Propagation delay time estimating device and program, as well as echo canceler, echo canceling program and echo canceling method

    JP2021036634A

  • Method and system for automatically synchronizing audio-video inputs in a multi-camera environment

    US20210014385A1