Measurement method, program, and measurement system

Abstract

A measurement method according to one embodiment of the present disclosure is a measurement method executed by a computer, and includes: a first display step of repeatedly displaying a first image (S10) on a first display device while updating a plurality of pieces of time information one by one in order with respect to the first image including the plurality of pieces of time information each showing a time; a first imaging step in which a first imaging device captures a first image displayed on the first display device, thereby generating a second image (S20); a second display step in which a second image is displayed on a second display device (S30); and a measurement step in which, on the basis of the first image and the second image, a delay time (S40), which is the time from the time when the first image is captured by the first imaging device to the time when the second image is displayed by the second display device, is measured.

Description

Technical Field

[0001] The present disclosure relates to a measurement method, a program, and a measurement system. Background Art

[0002] In recent years, a system for remotely operating robots such as vehicles has been developed. In this system, an image obtained from a camera mounted on the robot is displayed on a display device used by an operator who remotely operates the robot. The operator remotely operates the robot while confirming the image displayed on the display device. At this time, the timing of the image being displayed on the display device is slightly later than the timing of the camera shooting, that is, the timing of the image being generated by the camera shooting. In other words, a delay time (also called glass-to-glass delay time) is generated, which is the time from the time the camera shoots to the time the generated image is displayed on the display device.

[0003] For example, Patent Document 1 discloses a system for remotely operating an industrial vehicle in response to time delays.

[0004] (Prior art literature)

[0005] (Patent Document)

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2019-83462 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] The present disclosure provides a measurement method and the like capable of suppressing the inability to measure delay time.

[0009] Means for solving problems

[0010] A measurement method according to one embodiment of the present invention is a measurement method executed by a computer, the measurement method comprising the following steps: a first display step, for a first image including a plurality of moment information each indicating a moment, updating the plurality of moment information one by one in sequence while causing a first display device to repeatedly display the first image; a first camera step, generating a second image by causing a first camera device to capture the first image displayed on the first display device; a second display step, causing a second display device to display the second image; and a measurement step, measuring a delay time based on the first image and the second image, the delay time being the time from causing the first camera device to capture the first image to causing the second display device to display the second image.

[0011] A program according to one embodiment of the present disclosure is a program for causing a computer to execute the above-mentioned measurement method.

[0012] A measuring system according to one embodiment of the present disclosure comprises: a first display control unit, which updates a plurality of moment information including a first image, each of which indicates a moment, one by one in sequence while causing a first display device to repeatedly display the first image; a camera control unit, which generates a second image by causing a first camera device to capture the first image displayed on the first display device; a second display control unit, which causes a second display device to display the second image; and a measuring unit, which measures a delay time based on the first image and the second image, the delay time being the time from causing the first camera device to capture the first image to causing the second display device to display the second image.

[0013] Effects of the Invention

[0014] According to the present disclosure, it is possible to provide a measurement method and the like that can suppress the inability to measure delay time. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram for explaining an overview of a delay measurement system according to an embodiment.

[0016] Figure 2 This is a block diagram showing a characteristic functional configuration of a delay measurement system according to an embodiment.

[0017] Figure 3 A diagram showing a specific example of an image displayed on a display device according to an embodiment.

[0018] Figure 4 A diagram showing a specific example of an image displayed on a display device according to an embodiment.

[0019] Figure 5 A diagram showing a specific example of an image displayed on a display device according to an embodiment.

[0020] Figure 6 It is a diagram for explaining the imaging timing and display timing according to the embodiment.

[0021] Figure 7 A diagram showing a specific example of an image displayed on a display device according to an embodiment.

[0022] Figure 8 This is a flowchart for explaining the frame display process executed by the control device according to the embodiment.

[0023] Figure 9This is a flowchart for explaining the delay time measurement process executed by the measurement device according to the embodiment.

[0024] Figure 10 This is a sequence diagram showing the processing procedure of the delay measurement system according to the embodiment.

[0025] Figure 11 This is a sequence diagram showing the processing procedure of the delay measurement system according to the embodiment.

[0026] Figure 12 This is a flowchart showing the processing procedure of the measurement system according to the embodiment. DETAILED DESCRIPTION

[0027] (Knowledge that forms the basis of this disclosure)

[0028] As mentioned above, systems for remotely operating robots such as vehicles have been developed in recent years. This system enables, for example, store-to-home delivery services by utilizing multiple autonomous robots remotely monitored by a single operator. Furthermore, in the case of remotely operated buses, for example, a single operator can remotely monitor and control multiple buses, thereby reducing labor and costs.

[0029] In such a system, an image obtained by an imaging device mounted on the robot is displayed on a display device used by an operator who remotely operates the robot. As an evaluation index of the performance of such a system, the above-mentioned delay time can be given as an example.

[0030] The delay time can be measured, for example, by measuring the time between an image generated by capturing an object with a camera and being displayed on a display device used by an operator, as in actual system operation. The object is, for example, an image including the current time displayed on an arbitrary display device. Thus, the delay time can be measured based on the time included in the image generated by the camera to be displayed on the display device used by the operator (the current time at the time of capture) and the time when the image is displayed on the display device used by the operator (the current time at the time of image display).

[0031] When a display device displays time, the time included in the image (e.g., the current time) is updated as the image changes (i.e., for each displayed frame). However, if the camera captures an image at the timing of the image change, the camera may generate an image in which the pre-change image and the post-change image are superimposed. This image cannot accurately read the time included in the image generated by the camera, making it impossible to measure delay time.

[0032] The inventors of the present application created the present disclosure in view of the above-mentioned problems.

[0033] Below, with reference to the attached Figure 1 The embodiments are described below. In addition, the embodiments to be described below are all general or specific examples. The numerical values, shapes, materials, components, configuration positions of components, connection methods, steps, order of steps, etc. shown in the following embodiments are all examples, and their purpose is not to limit the present disclosure. Moreover, the components of the following embodiments that are not recorded in the independent technical solutions will be described as arbitrary components.

[0034] In addition, each figure is a schematic diagram and is not a strict illustration. Therefore, for example, the scales of the various figures are not necessarily the same. Moreover, in the various figures, substantially the same components are given the same reference numerals, and repeated descriptions may be omitted or simplified.

[0035] Furthermore, in the following description, although there may be instances where terms such as "above" or "less than" are used, these are not strictly speaking the same. For example, "above" may be used to indicate a greater value. Furthermore, "less than" may be used to indicate the following meanings. Furthermore, when contrasting terms such as "above a specified value" or "less than a specified value" are used, these terms may be used to indicate a distinction based on the specified value, meaning "greater than a specified value" or "less than a specified value," respectively.

[0036] (Implementation Method)

[0037] [constitute]

[0038] First, the configuration of the delay measurement system 10 will be described.

[0039] Figure 1 It is a diagram for explaining an outline of the delay measurement system 10 according to the embodiment.

[0040] The delay measurement system 10 is a system for measuring delay time, which is the time from the time the camera device 400 captures the image of any position when the operator remotely operates the robot 300 until the image is displayed on the display device used by the operator (for example, the display device 220), that is, the so-called lens-to-glass delay time.

[0041] The operator operates operating devices such as a steering wheel and brakes (not shown) connected to the remote operating device 120 while checking an image displayed on the display device 220. The remote operating device 120 transmits information related to the operations received through the operating devices to the robot 300. In response, the robot 300 operates according to the operator's operations.

[0042] In the delay measurement system 10 , in order to measure the delay time, the imaging device 400 captures an image displayed on the display device 200 .

[0043] The delay measurement system 10 includes a control device 100, a measuring device 110, a remote operation device 120, a display device 200, a display device 210, a display device 220, a display device 230, a display device 240, a robot 300, a camera device 400, a camera device 410, a base station 500, a hub 510, a hub 530, a router 540, a distributor 550 and a terminal 600.

[0044] The control device 100 is a computer that is communicatively connected to the display device 200 , the display device 210 , the display device 220 , the display device 230 , the display device 240 , the imaging device 400 , and the imaging device 410 and controls these devices.

[0045] The measurement device 110 is a computer that is communicatively connected to the imaging device 410 and obtains an image (also referred to as a third image) generated by the imaging device 410 from the imaging device 410 .

[0046] The remote operation device 120 is a computer that is communicatively connected to the robot 300 and is used by an operator to remotely operate the robot 300 .

[0047] Each of these computers includes, for example, a communication interface for communicating with various devices, a nonvolatile memory storing programs, a volatile memory serving as a temporary storage area for executing programs, input and output ports for transmitting and receiving signals, and a processor for executing programs. The computer may also be implemented as a so-called microcomputer.

[0048] Display devices 200, 210, 220, 230, and 240 are displays that display images. Each of display devices 200, 210, 220, 230, and 240 is communicatively connected to a computer such as control device 100, measurement device 110, or remote control device 120 via, for example, an HDMI (registered trademark) (High-Definition Multimedia Interface) cable.

[0049] The robot 300 is a machine that is remotely operated by an operator. The robot 300 is, for example, a vehicle, but may be any machine.

[0050] The imaging device 400 and the imaging device 410 are cameras that capture images of an object and generate images. The imaging device 400 is communicatively connected to the control device 100 via, for example, the robot 300, the base station 500, and the hub 510. The imaging device 410 is communicatively connected to the control device 100 and the measuring device 110 via, for example, a communication line such as an HDMI cable.

[0051] The base station 500 is a base station that performs wireless communication with the robot 300 and communicates with the control device 100 via the hub 510 .

[0052] The hub 510 is a repeater that connects the base station 500 and the control device 100 .

[0053] The hub 530 is a repeater that connects the control device 100, the measuring device 110, and the router 540. The hub 530 is connected to a server device (not shown) and the like via the Internet 520 so as to be communicable.

[0054] The router 540 is a repeater that connects the hub 530 and the terminal 600 .

[0055] The distributor 550 is connected to the control device 100 , the display device 200 , and the display device 210 via a communication line such as an HDMI cable, and transmits the first image output from the control device 100 to the display device 200 and the display device 210 .

[0056] Terminal 600 is a computer that receives user operations. Information indicating user operations received by terminal 600 is transmitted to control device 100 and / or measuring device 110 via router 540 and hub 530. In this embodiment, terminal 600 is a tablet terminal, but it can also be implemented as any device such as a personal computer or smartphone.

[0057] In addition, as a communication standard used for the above-mentioned communication, Ethernet (registered trademark) etc. can be cited, for example, but any communication standard may be used.

[0058] The user operates terminal 600, such as a tablet, to send an instruction to start delay time measurement to control device 100 and measuring device 110 via router 540 and hub 530. Upon receiving the instruction, control device 100 and measuring device 110 start processing for delay time measurement.

[0059] For example, upon receiving this instruction, the control device 100 causes the display devices 200 and 210 to display the same image (also referred to as the first image) via the distributor 550 connected via a communication line such as an HDMI cable. The display devices 200 and 210 are, for example, controlled to be synchronized so that the same first image is displayed on the display devices 200 and 210 at the same timing. For example, the display device 200 is located at the location where the robot 300 is located, and the display device 210 is located at the location where the remote control device 120 is located. The first images displayed on the display devices 200 and 210, for example, include information indicating the current time. The information indicating the current time included in the first images displayed on the display devices 200 and 210 is updated each time the first images displayed on the display devices 200 and 210 are switched. In other words, the current time is displayed on the display devices 200 and 210 while being repeatedly updated.

[0060] The control device 100 then causes the imaging device 400 mounted on the robot 300 to capture the first image displayed on the display device 200. In other words, the imaging device 400 captures the screen of the display device 200 displaying the first image. The resulting image (also referred to as the second image) is captured by the robot 300 and transmitted to the control device 100 via the base station 500 and hub 510, which are capable of wireless communication with the robot 300. For example, the imaging device 400 repeatedly captures the first image displayed on the display device 200 and repeatedly transmits the resulting second image to the control device 100.

[0061] Alternatively, the image may be transmitted from the robot 300 to the control device 100 via a server device (not shown) that is communicable with the base station 500 and the control device 100 via the Internet 520 and the hub 510 .

[0062] The control device 100 then causes the display device 220 to display the acquired second image. For example, the control device 100 transmits the second image to the remote operating device 120, causing the remote operating device 120 to cause the display device 220 to display the second image. Hereinafter, the process of the control device 100 transmitting the second image to the remote operating device 120 and the remote operating device 120 causing the display device 220 to display the second image will also be simply referred to as the control device 100 causing the display device 220 to display the acquired second image. The control device 100 may also directly cause the display device 220 to display the acquired second image without intervening through the remote operating device 120. For example, the control device 100 repeatedly causes the display device 220 to display the acquired second image each time the second image is acquired.

[0063] Furthermore, the control device 100 causes the imaging device 410 to capture the first image displayed on the display device 210 and the second image displayed on the display device 220. The image (third image) generated thereby and including the first and second images is obtained by the measurement device 110. For example, the imaging device 410 repeatedly captures the first image displayed on the display device 210 and the second image displayed on the display device 220, and repeatedly transmits the third image generated thereby to the measurement device 110.

[0064] The measuring device 110 measures the delay time based on the obtained third image. For example, the measuring device 110 measures the delay time every time the third image is obtained.

[0065] Furthermore, for example, by having two synchronized imaging devices capture the first image displayed on the display device 210 and the second image displayed on the display device 220, respectively, a third image including the first image and a third image including the second image may be generated.

[0066] Furthermore, for example, if the camera 400 and the camera 410 are in an environment where they can capture the same image included in the display device 200 , the display device 200 and the display device 210 may be implemented by a single device.

[0067] Figure 2 1 is a block diagram showing a characteristic functional configuration of the delay measurement system 10 according to the embodiment. Figure 2 In FIG, some components of the delay measurement system 10 , such as the robot 300 and the remote operation device 120 , are omitted from illustration.

[0068] The control device 100 includes an imaging control unit 101 , a display control unit 102 , a bandwidth control unit 103 , and a storage unit 104 .

[0069] The imaging control unit 101 is a processing unit that controls the imaging of the imaging devices 400 and 410. For example, the imaging control unit 101 controls the imaging device 400 so that the imaging device 400 captures the first image displayed on the display device 200 to generate a second image. Furthermore, for example, the imaging control unit 101 controls the imaging device 400 so that the imaging device 400 captures the second image displayed on the display device 220 and the first image displayed on the display device 210 to generate a third image.

[0070] The imaging conditions such as the imaging timing and exposure time of the imaging devices 400 and 410 can be arbitrarily set. In this embodiment, the imaging devices 400 and 410 each capture images at a frame rate of 60 fps (frames per second), or approximately every 16.6 msec.

[0071] The display control unit 102 is a processing unit that controls the display of images on the display devices 200, 210, and 220. The display control unit 102 is an example of a first display control unit and a second display control unit. For example, the display control unit 102 causes the display device 200 to repeatedly display a first image that includes multiple pieces of time information, each indicating a time. Specifically, the display control unit 102 causes the display device 200 to repeatedly display the first image while sequentially updating the multiple pieces of time information.

[0072] The time information is information indicating the time. For example, in the first image, the time information is represented by a number indicating the time.

[0073] Figure 3 1 is a diagram showing a specific example of an image (first image) displayed on the display device 200 according to the embodiment. Figure 3 For example, the display control unit 102 causes the display device 210 to repeatedly display the first image while sequentially updating the time indicated by any one of the plurality of time information.

[0074] like Figure 3 As shown, for example, in the first image displayed on the display device 200 , “Column 1” includes time information 700 , and “Column 2” includes time information 710 .

[0075] The time information 700 is “12:34:56:016”, which indicates that the current time when the time information 700 is updated is 12:34:56:016.

[0076] The time information 710 is “12:34:56:033”, which indicates that the current time when the time information 710 is updated is 12:34:56:033.

[0077] For example, each time the first image is repeatedly displayed on the display device 200, one of the time information 700 and the time information 710 is alternately updated. Therefore, for example, one of the time information 700 and the time information 710 may indicate the latest current time included in the first image, while the other may indicate the latest current time when the first image immediately before the first image was displayed.

[0078] For example, the display control unit 102 calculates a count representing the number of times the display device 200 displays the first image after updating the time indicated by any one of the multiple time information. Here, for example, the multiple time information includes the first time information and the second time information, such as time information 700 and time information 710. In other words, the multiple time information may also be two time information. In this case, for example, the display control unit 102 updates only one of the first time information and the second time information for the first image to be displayed on the display device 200 when the count is an even number, and updates only the other of the first time information and the second time information for the first image to be displayed on the display device 200 when the count is an odd number.

[0079] For example, the time information 700 displayed in "Column 1" is updated in the first image (also referred to as an odd frame) displayed on the display device 200 when the count reaches an odd number. Therefore, the time information 710 displayed in "Column 2" is not updated in the odd frame.

[0080] On the other hand, for example, the time information 710 displayed in "Column 2" is updated in the first image (also referred to as an even-numbered frame) displayed on the display device 200 when the count reaches an even number. Therefore, the time information 700 displayed in "Column 1" is not updated in the even-numbered frame.

[0081] Then, for example, the display control unit 102 causes the display device 220 to display the second image.

[0082] Figure 4 as well as Figure 5 : is a diagram showing a specific example of an image (second image) displayed on the display device 220 according to the embodiment. Specifically, Figure 4 This is a specific example of a second image captured when an odd-numbered frame is displayed on the display device 200. Figure 5 This is a specific example of a second image when an even-numbered frame is captured and displayed on the display device 200 .

[0083] like Figure 4 As shown in FIG. 1 , if the timing at which the camera device 400 captures the first image displayed on the display device 200 is inappropriate, a second image is generated in which the first image before the update is overlapped with the first image after the update. Figure 4As shown in FIG. 1 , the time information 701 displayed in “Column 1” is included in the second image after the two numbers of the time information of the first image before and after the update are superimposed. Similarly, for example, if the timing of shooting is not appropriate, such as Figure 5 As shown, the time information 712 displayed in "Column 2" is included in the second image after the two numbers of the time information of the first image before and after the update are superimposed. As for such time information 701 and time information 712, the time information cannot be correctly read from them. However, for example, Figure 4 As shown in the time information 711 displayed in "Column 2", even if the numbers of the two time information of the first image before and after the update overlap, if the numbers are the same, that is, the same time information, the correct time information can be included in the second image. Figure 5 As shown in the time information 702 displayed in "Column 1", even if the numbers of the two time information of the first image before and after the update overlap, if the numbers are the same, that is, the time information is the same, the correct time information can be included in the second image.

[0084] Figure 6 1 is a diagram for explaining the shooting timing and display timing involved in the embodiment. Specifically, Figure 6 (a) is a diagram showing the timing at which the camera device 400 captures the first image. Figure 6 (b) is a diagram showing the timing of updating the time indicated by the time information in the first image. Figure 6 (c) is a diagram showing the timing of refreshing the first image displayed on the display device 200. In addition, as described above, although the camera device 400 shoots at 60 fps, since the two frames shot are reduced to one frame, the measurement accuracy is reduced to about 30 fps. Figure 6 In (a), the shooting timing is recorded as 30 fps. Figure 6 "Odd 1" and "Odd 2" shown in (b) show, for example, the update display in Figure 3 The timing of the time indicated by the time information of "Column 1" shown in the above table. Figure 6 "Even 1" and "Even 2" shown in (b) show, for example, the update Figure 3 For example, when the time information displayed in "Column 2" changes from "Odd 1" to "Odd 2", the time information displayed in "Column 1" is updated. Figure 6 The shaded lines shown in (c) are refreshed at the same timing.

[0085] In this example, the time indicated by the time information in the first image is updated at 30 fps. That is, in this example, the time indicated by the time information in the first image is updated approximately every 33.3 msec. Furthermore, in this example, the refresh rate of the display device 200 is 60 Hz. Therefore, in this example, the time indicated by the time information in the first image is updated approximately every 16.6 msec. Furthermore, every time two refreshes are performed, the time indicated by the time information in the first image is updated.

[0086] In addition, the timing for updating the time indicated by the time information in the first image and the timing for refreshing the first image may be arbitrary.

[0087] like Figure 6 As shown, even if the refresh timing and the shooting timing overlap, any one of the multiple time information is continuously displayed, thereby reducing the occurrence of measurement errors in the measuring device 110 described later, such as reading errors of time information caused by overlapping display of numbers.

[0088] In this way, the display control unit 102 causes the display device 200 to repeatedly display the first image including the plurality of pieces of time information each indicating a time, while sequentially updating the time indicated by any one of the plurality of pieces of time information. In other words, when the first image is changed (updated), the time information that is not updated is also included in the first image. Therefore, the plurality of pieces of time information included in the second image generated by capturing the first image also includes readable time information.

[0089] Furthermore, the control device 100 may include a timekeeping unit such as an RTC (Real Time Clock) to obtain the time indicated by the time information. This time may also be obtained from, for example, a server device (not shown).

[0090] And, for example, in Figures 3 to 5 In the example shown, the number of time information included in the first image is two, but the number of time information included in the first image may be three or more.

[0091] Alternatively, the camera device 400 or the robot 300 may generate compressed information with a smaller data size than the second image and indicating the time indicated by each of the multiple time information included in the second image, and transmit the compressed information to the control device 100. For example, the camera control unit 101 may cause the camera device 400 or the robot 300 to generate compressed information with a smaller data size than the second image and indicating the time indicated by each of the multiple time information included in the second image, based on the transmission rate between the camera device 400 and the display device 210 (e.g., the transmission rate between the camera device 400 and the control device 100), and transmit the generated compressed information. For example, the display control unit 102 transmits the compressed information to the remote control device 120, which reconstructs the second image based on the compressed information and displays it on the display device 220. Alternatively, the display control unit 102 may reconstruct the second image based on the compressed information and display it on the display device 220. In this manner, the second image may be encoded and transmitted, and then decoded when displayed on the display device 220. For example, if the transmission rate is above a predetermined transmission rate, the second picture does not need to be encoded and transmitted. If the transmission rate is below the predetermined transmission rate, the second picture is encoded and transmitted. The predetermined transmission rate can be set arbitrarily and is not particularly limited.

[0092] Furthermore, each of the multiple moment information may be a number or not. For example, each of the multiple moment information may be a QR code (registered trademark) indicating the time. Furthermore, each of the multiple moment information may be a barcode indicating the time or an arbitrary symbol. Furthermore, each of the multiple moment information may be represented not as a number indicating a specific time, but as a serial number or symbol associated with the time.

[0093] Furthermore, for example, the display control unit 102 changes the size of the plurality of time information included in the first image to be displayed on the display device 200, based on the transmission rate at which the second image is transmitted from the imaging device 400 to the display device 220. The display control unit 102 changes the display form of the time information in the first image, for example, so that the time information is displayed larger as the transmission rate is lower.

[0094] For example, the display control unit 102 may transmit the image with the area displaying the time information cut out as the first image, based on the transmission rate at which the second image is transmitted from the camera device 400 to the display device 220. Alternatively, the display control unit 102 may transmit a compressed first image based on the transmission rate at which the second image is transmitted from the camera device 400 to the display device 220. For example, the display control unit 102 may compress the image with the area displaying the time information cut out, based on the transmission rate at which the second image is transmitted from the camera device 400 to the display device 220, and then transmit the compressed image. For example, the display control unit 102 performs the aforementioned cropping and / or compression processing when the transmission rate at which the second image is transmitted from the camera device 400 to the display device 220 is lower than a predetermined transmission rate. Consequently, even if the transmission rate at which the second image is transmitted from the camera device 400 to the display device 220 is low, the time information in the first image is less likely to be distorted when displayed on the display device 200.

[0095] The bandwidth control unit 103 is a processing unit that controls the transmission rate between the second image transmitted from the imaging device 400 and the display device 220. For example, the delay measurement system 10 measures the delay time while changing the transmission rate to calculate the delay time relative to the transmission rate.

[0096] Each processing unit of the imaging control unit 101 , the display control unit 102 , and the bandwidth control unit 103 is implemented by, for example, a processor that executes a control program stored in the storage unit 104 .

[0097] The storage unit 104 is a storage device that stores control programs executed by each processing unit, various conditions, threshold values, etc. The storage unit 104 is implemented by, for example, a semiconductor memory or an HDD (Hard Disk Drive).

[0098] The measuring device 110 includes a measuring unit 111 and a storage unit 112 .

[0099] The measurement unit 111 is a processing unit that measures the delay time. Specifically, the measurement unit 111 measures the delay time, which is the time from when the camera 400 captures the first image to when the display device 220 displays the second image, based on the first and second images. More specifically, the measurement unit 111 measures the delay time based on a third image that includes the first and second images. For example, the camera 410 captures the first image displayed on the display device 210 and the second image displayed on the display device 220, and sends the third image generated by capturing it to the measurement unit 110. The measurement unit 111 measures the delay time based on the third image obtained from the camera 410.

[0100] Figure 7: is a diagram showing a specific example of an image (third image) according to the embodiment. Specifically, Figure 7 Specific examples of time information 703 and time information 713 included in a first image captured by imaging device 410 and displayed on display device 210 , and time information 704 and time information 714 included in a second image displayed on display device 220 are shown.

[0101] Having obtained Figure 7 In the case of the third image shown, for example, the measuring unit 111 measures the delay time by calculating the difference between the time indicated by the readable time information 713 and the time indicated by the readable time information 714. In this example, the delay time is 0 seconds 105.

[0102] Furthermore, for example, when multiple time information is available for reading, the latest time among the multiple time information is used to measure the delay time. Therefore, for example, the measuring unit 111 measures the delay time based on the difference between the latest time indicated by the readable time information among the multiple time information included in the second image displayed on the display device 220 and the latest time indicated by the readable time information among the multiple time information included in the first image displayed on the display device 210.

[0103] The measuring unit 111 is realized by, for example, a processor that executes a control program stored in the storage unit 112 .

[0104] The storage unit 112 is a storage device that stores a control program, various conditions, threshold values, etc. executed by the measurement unit 111. The storage unit 112 is implemented by, for example, a semiconductor memory or an HDD.

[0105] [Processing order]

[0106] Next, the processing procedure of the delay measurement system 10 will be described.

[0107] Figure 8 1 is a flowchart for explaining the frame display process performed by the control device 100 according to the embodiment. Specifically, Figure 8 This is a flowchart showing a processing procedure for the display control unit 102 to display the first image on the display device 200 .

[0108] The display control unit 102 starts a frame display process, which is a process of repeatedly executing steps S120 to S170 (S110). The number of times this process is repeated can be set arbitrarily and is not particularly limited.

[0109] The display control unit 102 counts the number of frames (S120). Step S120 can be executed at any timing as long as it is appropriately determined whether the count is an even number or an odd number at the timing of the next update of the first image.

[0110] The display control unit 102 waits until the frame display time, which is the timing for updating the time information in the first image, arrives (S130). If the display control unit 102 has already displayed the first image on the display device 200, it causes the display device 200 to continue displaying the already displayed first image. The frame display time can be set arbitrarily and is not particularly limited.

[0111] When the frame display time arrives, the display control unit 102 obtains the time at that time (S140). In addition, the display control unit 102 can also set a certain offset time in consideration of the refresh timing of the display (for example, the display device 200). For example, in the case where the display is refreshed at intervals of 16.6ms, it is also possible to offset 8.4ms only at the first refresh, and this value is between the intervals. That is, the display does not necessarily wait for 16.6ms from a specific moment to display the next moment, but can wait only until the moment becomes 16.6ms+8.4ms at the first time, and wait for 16.6ms from the next time.

[0112] The display control unit 102 determines whether the count is an odd number ( S150 ).

[0113] When the display control unit 102 determines that the count is an odd number (Yes in S150), for example, Figure 3 The first image is updated so that the time indicated by the time information in "Column 1" is the time obtained in step S130 (S160).

[0114] In addition, when the display control unit 102 determines that the count is not an odd number (ie, an even number) (No in S150), for example, Figure 3 The first image is updated so that the time indicated by the time information in "Column 2" is the time obtained in step S130 (S170).

[0115] The display control unit 102 repeatedly executes the processing of steps S120 to S170 a predetermined number of times, and then ends the frame display processing ( S180 ).

[0116] Figure 9 1 is a flowchart for explaining the delay time measurement process performed by the measurement device 110 according to the embodiment. Specifically, Figure 9This is a flowchart showing the processing sequence from obtaining the third image to measuring the delay time, performed by the measurement unit 111. Display device A is, for example, one of display device 210 and display device 220, and display device B is, for example, the other of display device 210 and display device 220.

[0117] The measuring unit 111 starts the delay time measurement process, which is to repeatedly perform the process of steps S220 to S280 (S210). The number of times of this repetitive process can be set arbitrarily and is not particularly limited. For example, the measuring unit 111 performs this repetitive process during a specified time period. The specified time period can be set arbitrarily and is not particularly limited. The measuring device 110 may also have a timing unit such as an RTC for measuring time. For example, the measuring unit 111 repeatedly obtains the third image from the camera device 410 during the specified time period and repeatedly performs the process of steps S220 to S280 on the obtained third image.

[0118] The measuring unit 111 performs (i) reading processing of the time indicated by the time information of "column 1" included in the image displayed by the display device A and included in the third image, and performs (ii) reading processing of the time indicated by the time information of "column 2" included in the image displayed by the display device A and included in the third image (S220).

[0119] Furthermore, the measuring unit 111 performs (iii) reading processing of the time indicated by the time information of "column 1" included in the image displayed by the display device B and included in the third image, and performs (iv) reading processing of the time indicated by the time information of "column 2" included in the image displayed by the display device B and included in the third image (S230).

[0120] Based on the results of the reading processes in steps S220 and S230, the measurement unit 111 determines whether no time was read in either of the reading processes (i) and (ii), or whether no time was read in either of the reading processes (iii) and (iv) (S240). In other words, the measurement unit 111 determines whether no readable time is included in the time information included in the image displayed by display device A and included in the third image, or whether no readable time is included in the time information included in the image displayed by display device B and included in the third image.

[0121] If the answer is “Yes” in step S240 , the measuring unit 111 does not perform further delay time measurement processing using the third image used in the iterative processing, and does not store the measurement result in the storage unit 112 ( S280 ).

[0122] In addition, for example, when the measuring unit 111 answers "No" in step S240, that is, when there are readable times for both the time indicated by the time information included in the image displayed by display device A and included in the third image, and the time indicated by the time information included in the image displayed by display device B and included in the third image, the latest time indicated by the readable time information among the multiple time information included in the image displayed by display device A and included in the third image is determined as Amax (S250).

[0123] Then, for example, the measuring unit 111 determines the latest time indicated by the readable time information among the plurality of time information included in the image displayed on the display device B and included in the third image as Bmax ( S260 ).

[0124] The measuring unit 111 calculates the delay time by subtracting Bmax from Amax, and stores the calculated result as a measurement result of the delay time in, for example, the storage unit 112 ( S270 ).

[0125] The measuring unit 111 repeatedly executes the processing of steps S220 to S280 for a predetermined period of time, and then ends the delay time measurement processing ( S290 ).

[0126] Figure 11 as well as Figure 12 It is a sequence diagram showing the processing procedure of the delay measurement system 10 according to the embodiment.

[0127] First, the control device 100 transmits the first image to each of the display devices 200 and 210 ( S310 ).

[0128] In response to this, the display device 200 displays the first image ( S320 ).

[0129] Similarly, the display device 210 displays the first image ( S330 ).

[0130] In this manner, the control device 100 controls the display device 200 and the display device 210 in steps S310 to S330 , thereby causing the display device 200 and the display device 210 to display the same first image.

[0131] Next, the control device 100 updates the time information of one of the two columns included in the first image transmitted in step S310 (S340). For example, the control device 100 generates a new first image by changing the time indicated by the time information of one of the two columns included in the first image transmitted in step S310 to the current time, where the one of the two columns corresponds to the count of the number of images displayed on each of the display devices 200 and 210 (i.e., the aforementioned count).

[0132] Next, the control device 100 transmits the new first image generated in step S340 to each of the display devices 200 and 210 ( S350 ).

[0133] In response to this, the display device 200 displays the new first image generated in step S340 ( S360 ).

[0134] Similarly, the display device 210 displays the new first image generated in step S340 ( S370 ).

[0135] In this manner, the control device 100 controls the display device 200 and the display device 210 in steps S350 to S370 , thereby causing the display device 200 and the display device 210 to display the same new first image.

[0136] Next, the control device 100 updates the time information of the other of the two columns included in the first image transmitted in step S350 (S380). For example, the control device 100 generates a new first image by changing the time information indicated by the other column, which was not changed in step S340, to the current time. The other of the two columns also corresponds to the count of the number of images to be displayed on each of the display devices 200 and 210.

[0137] Next, the control device 100 transmits the new first image generated in step S380 to each of the display devices 200 and 210 ( S390 ).

[0138] In response to this, the display device 200 displays the new first image generated in step S380 ( S400 ).

[0139] Similarly, the display device 210 displays the new first image generated in step S380 ( S390 ).

[0140] In this manner, the control device 100 controls the display device 200 and the display device 210 in steps S380 to S410 , thereby causing the display device 200 and the display device 210 to display the same new first image.

[0141] Thereafter, the control device 100 repeatedly executes the processing of steps S340 to S390 to repeatedly and alternately update the two columns included in the first image so that the time indicated by the two columns becomes the current time, and displays them on the display devices 200 and 210 .

[0142] In the middle of repeatedly executing such a process, Figure 11 The processing shown.

[0143] The control device 100 transmits an instruction to the imaging device 400 to capture the first image displayed on the display device 200 ( S510 ). Specifically, the control device 100 controls the imaging device 400 to capture the first image displayed on the display device 200 .

[0144] In response to this, the imaging device 400 captures the first image being displayed on the display device 200 to thereby generate a second image ( S520 ).

[0145] Next, the imaging device 400 transmits the second image generated by imaging the first image being displayed on the display device 200 to the control device 100 ( S530 ).

[0146] Next, the control device 100 transmits the second image to the display device 220 ( S540 ).

[0147] In response to this, the display device 200 displays the second image ( S550 ).

[0148] Thereafter, the control device 100 repeatedly executes the processing of steps S510 to S540 to cause the imaging device 400 to repeatedly capture the first image displayed on the display device 200 to repeatedly generate the second image, thereby causing the display device 220 to repeatedly display the second image.

[0149] As a result, the time indicated in one of the two columns of the first image included in the second image displayed on the display device 220 is repeatedly updated.

[0150] While such processing is being repeatedly executed, the processing after step S560 is executed.

[0151] The control device 100 transmits an instruction to the camera 410 to cause the camera 410 to simultaneously capture both the first image being displayed on the display device 210 and the second image being displayed on the display device 220 (S560). Specifically, the control device 100 controls the camera 410 to simultaneously capture both the first image being displayed on the display device 210 and the second image being displayed on the display device 220. For example, the display devices 210 and 220 are pre-configured to be included in the viewing angle of the camera 410 so that the camera 410 can simultaneously capture both the first image being displayed on the display device 210 and the second image being displayed on the display device 220.

[0152] Accordingly, the camera 410 captures the first image being displayed on the display device 210 and the second image being displayed on the display device 220, thereby generating a third image (S570). Figure 7 The 3rd image is shown.

[0153] Next, the imaging device 410 transmits the generated third image to the measuring device 110 ( S580 ).

[0154] Next, the measuring device 110 measures the delay time based on the third image ( S590 ).

[0155] For example, the measurement device 110 notifies the user of the measured delay time by displaying the measured delay time on a display device (not shown).

[0156] Figure 12 A flowchart illustrating the processing sequence of the measurement system according to the embodiment. The measurement system is implemented by at least some of the components of the delay measurement system 10. For example, the measurement system includes: an imaging control unit 101 included in the control device 100; a display control unit 102 included in the control device 100; and a measurement unit 111 included in the measurement device 110.

[0157] First, a first display control unit causes a first display device to repeatedly display a first image including a plurality of time information items each indicating a time, while sequentially updating the plurality of time information items (S10). The first display control unit is, for example, display control unit 102. The first display device is, for example, display device 200.

[0158] Next, the imaging control unit 101 generates a second image by causing the first imaging device to capture the first image displayed on the first display device ( S20 ). The first imaging device is, for example, the imaging device 400 .

[0159] Next, the second display control unit causes the second display device to display the second image (S30). The second display control unit is, for example, the display control unit 102. The first display control unit and the second display control unit may be implemented by a common processor or by separate processors.

[0160] Next, the measuring unit 111 measures the delay time from when the first image is captured by the first imaging device to when the second image is displayed by the second display device based on the first and second images ( S40 ). The second display device is, for example, the display device 220 .

[0161] [Effects, etc.]

[0162] Hereinafter, the technologies obtained from the disclosure of this specification will be exemplified, and the effects and the like obtained from the exemplified technologies will be described.

[0163] Technology 1 is a measurement method executed by a computer, comprising the following steps: a first display step, for a first image including a plurality of moment information each indicating a moment, causing a first display device to repeatedly display the first image while sequentially updating the moment indicated by any one of the plurality of moment information (S10); a first camera step, causing a first camera device to capture the first image displayed on the first display device, thereby generating a second image (S20); a second display step, causing a second display device to display the second image (S30); and a measurement step, measuring a delay time based on the first image and the second image, the delay time being the time from causing the first camera device to capture the first image to causing the second display device to display the second image (S40).

[0164] Thus, even if the first image is captured by the first camera device at the time when the first image is changed, that is, when the time indicated by the time information included in the first image is updated, thereby changing the time information, it is possible to prevent the time information that has not been changed at that time from being captured in a state where it cannot be read due to overlap between the first image before and after the change. Therefore, by utilizing the readable time information among the multiple time information included in the second image, the possibility of measuring the delay time can be increased. Therefore, it is possible to prevent the delay time from being unmeasurable.

[0165] Technology 2 is a measuring method as described in Technology 1, which also includes a counting step, in which a count is calculated, and the count is the number of times the first display device displays the first image after the time indicated by any one of the multiple time information is updated. The multiple time information includes the first time information and the second time information. In the first display step, for the first image to be displayed on the first display device when the count becomes an even number, only one of the first time information and the second time information is updated, and for the first image to be displayed on the first display device when the count becomes an odd number, only the other of the first time information and the second time information is updated.

[0166] With this, the times indicated by the two pieces of time information can be alternately updated through simple processing.

[0167] Technology 3 is a measurement method as described in Technology 1 or 2, which also includes: a third display step, while sequentially updating the time indicated by any one of the multiple time information, repeatedly displaying the first image on a third display device; and a second camera step, by causing the second camera device to capture the second image displayed on the second display device and the first image displayed on the third display device, thereby generating a third image, and in the measurement step, measuring the delay time based on the third image.

[0168] The third display device is, for example, the display device 210. The first display device and the third display device are, for example, controlled to be synchronized so that the same first image is displayed at the same timing.

[0169] With this configuration, the delay time can be measured based on the plurality of pieces of time information included in each of the first image and the second image included in the third image.

[0170] Technology 4 is a measurement method as described in Technology 3, wherein in the measurement step, the delay time is measured based on the difference between the latest moment indicated by the readable moment information among the multiple moment information included in the second image displayed on the second display device and the latest moment indicated by the readable moment information among the multiple moment information included in the first image displayed on the third display device.

[0171] With this, the latest time among the times indicated by the plurality of time information is the most recently updated time, and therefore the delay time can be measured more accurately.

[0172] Technology 5 is a measurement method as described in any one of Technologies 1 to 4, wherein in the first camera step, compressed information having a smaller data volume than the second image is generated based on the transmission rate between the second image and the second display device, and the compressed information shows the time indicated by each of the multiple time information included in the second image. In the second display step, the second image is reconstructed based on the compressed information and displayed on the second display device.

[0173] When the transmission rate decreases, a less clear image must be transmitted to reduce the amount of data being transmitted. Therefore, for example, when the transmission rate is above a specified transmission rate, the second image is transmitted from the first camera to the second display device. When the transmission rate is lower than the specified transmission rate, compressed information compressing the second image is generated and outputted by the first camera. For example, the control device 100 or remote control device 120, which controls the display device 220 as an example of the second display device, obtains this compressed information, reconstructs the second image based on the compressed information, and displays the reconstructed second image on the display device 220. This prevents image distortion caused by a decrease in the transmission rate.

[0174] Technique 6 is the measurement method according to any one of Techniques 1 to 5, wherein each of the plurality of time information is a QR code (registered trademark) indicating a time.

[0175] With this, compared to a case where the time information is included in an image in the form of numbers, for example, the time indicated by the time information can be easily obtained with simple processing such as image analysis.

[0176] Technology 7 is a measurement method as described in any one of technologies 1 to 6, wherein in the first display step, the size of multiple moment information included in the first image to be displayed on the first display device is changed according to the transmission rate between the second image sent from the first camera device to the second display device.

[0177] As described above, when the transmission rate decreases, a less clear image needs to be transmitted to reduce the amount of data to be transmitted. Therefore, for example, the lower the transmission rate, the larger the time information is displayed, making it easier to read the time information even if the second image is not clear.

[0178] Technique 8 is a program for causing a computer to execute the measurement method described in any one of Techniques 1 to 7.

[0179] This can achieve the same effects as those of the measurement method according to one embodiment of the present disclosure.

[0180] Technology 9 is a measuring system comprising: a first display control unit, which updates the plurality of moment information, each indicating a moment, one by one in sequence, while causing the first display device to repeatedly display the first image; a camera control unit 101, which generates a second image by causing the first camera device to capture the first image displayed on the first display device; a second display control unit, which causes the second display device to display the second image; and a measuring unit 111, which measures a delay time based on the first image and the second image, the delay time being the time from when the first camera device captures the first image to when the second display device displays the second image.

[0181] This can achieve the same effects as those of the measurement method according to one embodiment of the present disclosure.

[0182] In addition, the measurement system involved in one embodiment of the present disclosure can be implemented by multiple devices (for example, multiple computers) such as the control device 100 and the measurement device 110 of the above-mentioned embodiment, or by a single device (for example, a computer) that implements each function.

[0183] (Other Embodiments)

[0184] Although the embodiments have been described above, the present disclosure is not limited to the above embodiments.

[0185] For example, while the above embodiment displays the time corresponding to two frames, an even-numbered frame and an odd-numbered frame, on the screen, the method is not limited to this. This method can also be extended to display more than three frames. For example, when three frames are to be displayed, these three frames can be displayed on the same screen, such as the display area for displaying when the frame count is a multiple of 3, the display area for displaying when the frame count is a multiple of 3 plus 1, and the display area for displaying when the frame count is a multiple of 3 plus 2. Similarly, a display area for displaying multiple frames, such as four or five frames, can also be set.

[0186] Through such work, the delay time can be accurately measured if the camera's shooting capability (frame rate) or the display capability (frame rate) of the monitor is improved in the future, or even if the exposure time of the camera taking two pictures is delayed due to photography in a dark place, etc.

[0187] Furthermore, in the delay measurement of the above embodiment, although the evaluation image captured by the camera device 400 mounted on the robot 300 is an image displayed at the time of multiple frames, as a glass-to-glass delay measurement method, a more realistic image can also be added as an evaluation image. In other words, an image with a large change in code size, such as an increase in the amount of code when the image is encoded, can also be added as an evaluation image. In this case, the delay can also be measured before and after the change in code size. Furthermore, the amount of code encoded in the image can be changed, and the delay can be measured for each image after the change. Furthermore, the data size of the image can be changed without changing the amount of code.

[0188] Furthermore, the control device 100 may cause a delay when transmitting the image data packets from the robot 300 to the remote control device 120. This delay may be a delay that reproduces the delay observed in a wireless network or the Internet.

[0189] Furthermore, color codes may be used instead of QR codes for encoding.

[0190] Furthermore, in addition to displaying time, a frame counter can be embedded in the image to display the frame counter. Furthermore, the frame counter can be associated with the amount of generated code through other analysis means, and a means can be provided to link the frame counter with the delay measurement timing. This allows for detailed delay measurement, such as when the delay increases due to frames with increased code amounts being transmitted.

[0191] Furthermore, for example, in the above-described embodiment, a process executed by a specific processing unit may be executed by another processing unit. Furthermore, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0192] Furthermore, in the above-described embodiments, each component (each processing unit) can be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0193] Furthermore, each component can also be implemented by hardware. Each component can also be a circuit (or integrated circuit). These circuits can constitute a circuit as a whole, or they can be separate circuits. Furthermore, these circuits can be general circuits or dedicated circuits.

[0194] Furthermore, the general or specific aspects of the present disclosure may be implemented by a system, apparatus, method, integrated circuit, computer program, or non-transitory recording medium such as a computer-readable CD-ROM. Furthermore, the present disclosure may be implemented by any combination of the system, apparatus, method, integrated circuit, computer program, and recording medium.

[0195] For example, the present disclosure may be implemented as a measurement method executed by a computer. Furthermore, the present disclosure may be implemented as a program for causing a computer to execute the measurement method, or as a computer-readable non-transitory recording medium storing such a program.

[0196] In addition, various modifications that can be conceived by those skilled in the art to the embodiments, or any combination of components and functions in the embodiments without departing from the spirit of the present disclosure are also included in the present disclosure.

[0197] Industrial applicability

[0198] The present disclosure can be applied to a device for calculating glass-to-glass delay time.

[0199] Explanation of symbols

[0200] 10 Delay Measurement System

[0201] 100 Control Device

[0202] 101 Camera Control Unit

[0203] 102 Display control unit

[0204] 103 Bandwidth Control Department

[0205] 104, 112 Storage Department

[0206] 110 Measuring device

[0207] 111 Measurement Department

[0208] 120 Remote Operation Device

[0209] 200, 210, 220, 230, 240 display devices

[0210] 300 robots

[0211] 400, 410 Camera Device

[0212] 500 base stations

[0213] 510, 530 hubs

[0214] 520 Internet

[0215] 540 router

[0216] 550 Distributor

[0217] 600 Terminal

[0218] 700, 701, 702, 703, 704, 710, 711, 712, 713, 714 time information

Claims

1. A measurement method is a measurement method performed by a computer, The measuring method comprises the following steps: a first display step of causing a first display device to repeatedly display a first image including a plurality of time information each indicating a time while sequentially updating the plurality of time information one by one; a first imaging step of generating a second image by causing a first imaging device to capture the first image displayed on the first display device; a second display step of causing a second display device to display the second image; and The measuring step measures a delay time from when the first image is captured by the first imaging device to when the second image is displayed by the second display device, based on the first image and the second image.

2. The measuring method according to claim 1, The measurement method further includes a counting step of calculating a count of the number of times the first display device displays the first image after the time indicated by any one of the plurality of time information is updated. The plurality of time information includes first time information and second time information, In the first display step, for the first image to be displayed on the first display device when the count becomes an even number, only one of the first moment information and the second moment information is updated; and for the first image to be displayed on the first display device when the count becomes an odd number, only the other of the first moment information and the second moment information is updated.

3. The measuring method according to claim 1, The measuring method further comprises the following steps: a third display step of causing a third display device to display the first image while sequentially updating the time indicated by any one of the plurality of time information; and a second imaging step of causing a second imaging device to capture the second image displayed on the second display device and the first image displayed on the third display device, thereby generating a third image; In the measuring step, the delay time is measured based on the third image.

4. The measuring method according to claim 3, In the measurement step, the delay time is measured based on the difference between the latest moment indicated by the readable moment information among the multiple moment information included in the second image displayed on the second display device and the latest moment indicated by the readable moment information among the multiple moment information included in the first image displayed on the third display device.

5. The measuring method according to claim 1, Furthermore, in the first imaging step, compressed information having a smaller data volume than the second image is generated based on a transmission rate at which the second image is transmitted from the first imaging device to the second display device, and the compressed information indicates the time indicated by each of the plurality of time information included in the second image. In the second display step, the second image is reconstructed based on the compression information and displayed on the second display device.

6. The measuring method according to claim 1, Each of the plurality of time information is a QR code indicating a time, and the QR code is a registered trademark.

7. The measuring method according to claim 1, In the first display step, the size of the plurality of time information included in the first image to be displayed on the first display device is changed in accordance with a transmission rate between the second image sent from the first camera device to the second display device. 8 . A program for causing a computer to execute the measurement method according to claim 1 .

9. A measurement system comprising: a first display control unit for causing the first display device to repeatedly display the first image while sequentially updating the plurality of time information items, each of which indicates a time; an imaging control unit configured to generate a second image by causing a first imaging device to capture the first image displayed on the first display device; a second display control unit configured to cause a second display device to display the second image; as well as The measuring unit measures a delay time from when the first image is captured by the first imaging device to when the second image is displayed by the second display device, based on the first image and the second image.

Images