System with calibration function, program, and calibration method

Abstract

Provided are a system, a program, and a method that make it possible to more easily perform calibration processing of a display device and a sensor.　This system comprises: a display device; one or more sensors in which a detection region is set so as to overlap a display region of the display device; and a control device connected to the display device and the sensor. The control device includes: a display control unit that displays a target T at a predetermined display position in the display region; and an association unit that, on the basis of a detection position detected by the sensor during the display of the target T and the display position of the target T, associates the display position of the display device with the detection position by the sensor. The display control unit displays, together with the target T, a guidance area G larger than the target T in a range including the target T, with a convergence point set on the target T or in the vicinity thereof, reduces the guidance area G toward the convergence point, and when the detection position by the sensor is stopped, displays information I relating to the remaining time before the end of detection processing by the sensor in the vicinity of the target T.

Description

System with calibration function, program, and calibration method

[0001] The present invention relates to a calibration technique for a system including a display device and a sensor that detects a touch position on the display area, particularly for associating the display coordinate system of the display device with the sensor coordinate system of the sensor, and for correcting any misalignment between the two coordinate systems.

[0002] Conventionally, a system has been known in which an optical range sensor is placed near a display area of ​​a display, projector, or the like, and the sensor's inspection area is adjusted to overlap with the display area of ​​the display device. When a user touches the display area, position information (coordinates, etc.) on the display area is acquired. This system can, for example, display a predetermined object or the like at the contact position on the display area (see, for example, Patent Document 1). It is also possible to realize interactive video expression, in which a user's touch on a video work displayed on the display device causes some kind of reaction in the video work.

[0003] Incidentally, when a sensor is installed near the display area of ​​a display device, it is necessary to accurately associate the position detected by the sensor with the display position of the display device. If there is a misalignment between the position detected by the sensor and the display position of the display device, there is a concern that a problem may occur, such as a reaction occurring at a position on the display area different from the position touched by the user. Therefore, when using a system combining a display device and a range measurement sensor, it is known to perform a process of calculating parameters for performing coordinate conversion between the display coordinate system of the display device and the sensor coordinate system of the sensor by associating the position of a predetermined point on the display area with the position detected by touching that point, which is called a calibration process (Patent Document 2, etc.).

[0004] JP 2023-128843 A JP 2012-027769 A

[0005] In the calibration process described in Patent Document 2, targets (points) are displayed in areas including the four corners of the display area, and the user is asked to touch (indicate) each target with a finger, a touch pen, or the like. Patent Document 2, as an example, sequentially displays nine targets on the display area, and the user is asked to touch each target. However, because each target is displayed as a small circular dot, it is difficult for the user to find the sequentially displayed targets. If the user misses even one target during the calibration process, an error occurs in the coordinate transformation parameters, making it impossible to perform accurate calibration. In particular, when a display or screen is installed on a wall surface of a commercial facility or exhibition facility, a large display area is required. In this case, it becomes even more difficult for the user to find the target to touch within the large display area.

[0006] In addition, after a target is displayed in the display area and the user touches it, the sensor detects the location where the user touched it. However, if the user touches the target for a short time, the sensor's detection process cannot be completed in time, and the location where the user touched cannot be accurately detected. For this reason, the user is required to touch the target for a certain period of time, but if the user does not know how long the process will take to complete, the user will not know how long they should touch the target and may continue to touch the target longer than necessary, which makes the calibration process more difficult. In particular, this process is often cumbersome for those who are not familiar with calibrating display devices and sensors.

[0007] Therefore, a main object of the present invention is to make it possible to more easily perform the calibration process of the display device and the sensor.

[0008] A first aspect of the present invention relates to a system with a calibration function. The system according to the present invention includes a display device, one or more sensors, and a control device. The display area of ​​the display device may be formed by a display device such as a liquid crystal display or an organic electroluminescence (EL) display, or by a projector that projects an image onto a screen. The display device may also be, for example, a device that connects multiple displays (a so-called multi-display). One or more sensors have detection areas that overlap the display area of ​​the display device. The sensors are optical sensors such as infrared or range sensors, and emit inspection light along the display area of ​​the display device. When multiple sensors are provided, it is preferable that the detection areas of each sensor overlap the display area of ​​the display device. However, it is sufficient that at least the combined detection areas of the sensors overlap the display area of ​​the display device. The control device is a computer connected to the display device and the sensors via a wired or wireless connection. The control device includes a display control unit and a correspondence unit (e.g., a coordinate conversion unit). The display control unit causes the display device to display a target at a predetermined display position within the display area. The association unit associates the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is displayed and the display position of the target. Specifically, the association unit calculates parameters for coordinate conversion between the display coordinate system of the display device and the sensor coordinate system of the sensor by associating the display position with the detection position of the sensor. In the system according to the present invention, the display control unit displays, together with the target, a guidance area larger than the target within a range including the target. The guidance area is displayed on the display area so as to surround the target. The shape of the guidance area is not particularly limited and may be a circle, ellipse, triangle, rectangle, pentagon, or other polygon. Furthermore, the display control unit uses a convergence point on or near the target and shrinks the guidance area toward the convergence point. In other words, the guidance area shrinks toward the target in a continuous or stepwise manner. Note that the guidance area does not necessarily have to ultimately disappear toward the convergence point; it is sufficient for it to surround the target even in its minimized state.The case where the convergence point of the guidance area is near the target specifically means that the guidance area overlaps with the target even in the minimized state.

[0009] As in the above configuration, by displaying a guidance area around a target that the user should touch (point) with a finger, a touch pen, etc., and gradually shrinking this guidance area toward the target, the position of the target can be presented to the user in an easy-to-understand manner. Furthermore, if the target itself is displayed large so that the user can easily see the target, the target and the detection point cannot be accurately associated, which hinders the calibration process. However, by displaying a large guidance area around the target as in the present invention, the target itself can be kept small while ensuring the visibility of the target, and the accuracy of the calibration process can be improved.

[0010] In the system according to the present invention, the guidance area is preferably a circle (including a perfect circle and an ellipse) centered on the target, and is displayed in a manner that is visually distinguishable from the target. By making the guidance area a circle centered on the target, the user can intuitively understand the position of the target. In particular, it is preferable that the guidance area be a perfect circle. Note that when the guidance area is an ellipse, its center can be defined as the intersection of the major and minor axes.

[0011] In the system according to the present invention, it is preferable that the display control unit displays information about the progress of the detection process by the sensor near the target when the detection position detected by the sensor stops. By displaying information about the progress of the detection process in the display area in this way, it becomes easier for the user to understand that the detection process is progressing while touching the target. Note that the vicinity of the target specifically refers to a range that is within the user's field of vision together with the target when the user is touching the target. In other words, it is sufficient if the user can simultaneously view the target and information about the progress of the detection process.

[0012] In the system according to the present invention, the information about the progress of the detection process is preferably information about the time remaining until the detection process by the sensor is completed. For example, the time remaining until the detection process is completed may be displayed as a countdown. This allows the user to easily understand how long they need to continue touching the target.

[0013] In the system according to the present invention, after the detection position detected by the sensor has stopped and the detection process by the sensor has ended, the display control unit preferably displays both the target and / or the guidance area in a manner that is visually distinguishable from the state during the detection process by the sensor. For example, the color of the guidance area during the detection process by the sensor may be different from the color of the guidance area after the detection process has ended. This allows the user to intuitively understand that the detection process has ended.

[0014] In the system according to the present invention, it is preferable that the display control unit gradually reduces the guidance area toward a convergence point on or near the target, and after the reduction of the guidance area starts, determine whether the detection position detected by the sensor has stopped. In this way, by gradually reducing the guidance area toward the convergence point before the sensor detects the user's touch position, etc., the user can easily recognize the position of the target within the display area.

[0015] A second aspect of the present invention relates to a computer program. The program according to the present invention causes a general-purpose computer to function as a control device in the system according to the first aspect. The program according to the present invention may be downloadable via the Internet or stored on a non-transitory recording medium such as a CD-ROM.

[0016] A third aspect of the present invention relates to a calibration method. The calibration method according to the present invention is performed by a system including a display device, one or more sensors having detection areas set to overlap the display area of ​​the display device, and a control device connected to the display device and the sensor. Calibration is a process for correlating the display position of the display device with the detection position of the sensor. The calibration method includes a display step and a correction step. The display step is a step in which the control device causes the display device to display a target at a predetermined display position within the display area. The correction step is a step in which the control device correlates the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is displayed and the display position of the target. Here, the display step first displays the target together with a guidance area larger than the target in a range including the target. The display step then sets a convergence point on or near the target and shrinks the guidance area toward the convergence point.

[0017] According to the present invention, the calibration process for the display device and the sensor can be performed more easily.

[0018] FIG. 1 is a schematic overall view of an embodiment of a system according to the present invention. FIG. 2 is a block diagram showing an example of the functional configuration of a system according to the present invention. FIG. 3 is a flow diagram showing an example of a calibration process. FIG. 4 is a schematic diagram showing the relationship between the detection area of ​​a sensor and the display area of ​​a display device. FIG. 5 is a schematic diagram showing the relationship between the sensor coordinates of a sensor and the display coordinates of a display device. FIG. 6 is a schematic diagram showing the process of correcting a detection point. FIG. 7 is a schematic diagram showing the process of displaying an object by a display device. FIG. 8 is a flow diagram showing an example of the process of displaying a target. FIG. 9 shows an example of a state in which a guidance area is displayed around a target. FIG. 10 shows an example of a state in which a progress status is displayed around a target.

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes appropriate modifications of the embodiments described below within the scope obvious to those skilled in the art.

[0020] FIG. 1 illustrates an embodiment of a system 100 according to the present invention. As illustrated in FIG. 1, the system 100 includes a control device 10, two sensors 21 and 22, and a display device 30. The system 100 includes the display device 30, which includes a large display 31 mounted on a wall, for example, and displays predetermined video content on the display device 30. In the embodiment illustrated in FIG. 1, the display device 30 includes the display 31 and a display generating device 32 that generates video to be output to the display 31. The display 31 is also composed of multiple display devices 31(a) to (f). The system 100 can detect a position on the display area of ​​the display device 30 touched by a user with a finger, a touch pen, or the like using sensors 21 and 22, and reflect the information on the detected position in the video content, etc., displayed on the display device 30. For example, when the user touches the display area, a predetermined object is displayed at the touched position, or the video at the touched position changes.

[0021] The control device 10 is responsible for overall control of the system 100. Specifically, the control device 10 is connected to the display device 30 and the sensors 21 and 22 via a known bus, hub, or the like. The control device 10 receives detection information from the sensors 21 and 22 and controls the display device 30 to display video content, objects, and the like. The control device 10 can be implemented by a general-purpose computer. In the example shown in FIG. 1 , the control device 10 is configured by a computer installed locally together with the display device 30 and the sensors 21 and 22. However, this is not limited to this, and the control device 10 may also include a cloud server connected to the local computer via the Internet. In other words, the control device 10 is not limited to being implemented by a single computer, but may be implemented by sharing its functions among multiple computers or servers.

[0022] The sensors 21 and 22 are used to detect the position of a user's touch on the display area of ​​the display device 30. Each sensor 21 and 22 is installed, for example, above the display area of ​​the display device 30, and its installation position is adjusted so that the entire display area is the detection area. More specifically, the system 100 according to this embodiment includes a first sensor 21 and a second sensor 22. The first sensor 21 is installed in the upper left of the display area of ​​the display device 30, and the second sensor 22 is installed in the upper right of the display area. By installing multiple sensors 21 and 22, the detection accuracy of a user's finger or touch pen can be improved. For example, even if multiple users are touching the display area of ​​the display device 30 simultaneously, the installation of multiple sensors 21 and 22 allows for accurate detection of these multiple touch positions.

[0023] Furthermore, the sensors 21 and 22 are preferably range sensors (area sensors). The range sensor is a two-dimensional scanning optical distance measuring sensor that emits inspection light (laser) and measures the distance and angle to an object to be inspected (such as a user's finger or a touch pen) by scanning the inspection light. Information detected by the range sensor is transmitted to the control device 10. The range sensor emits inspection light along the surface area of ​​the display device 30. If the position coordinates of the range sensor in real space are known, the position coordinates of the object to be inspected in real space can be detected by measuring the distance and angle to the object that comes into contact with the light emitted from the range sensor. Note that each of the sensors 21 and 22 is not limited to a range sensor, and known infrared sensors can also be used.

[0024] The display device 30 displays video content under the control of the control device 10. In the example shown in FIG. 1 , the display device 30 is composed of a display 31 and a display generation device 32. The display 31 is composed of multiple display devices 31(a) to 31(f) connected in a daisy chain. In this display device 30, multiple display devices 31(a) to 31(f) form a single display area, allowing video content to be displayed across the boundaries between the display devices 31(a) to 31(f). Each of the display devices 31(a) to 31(b) may be a known liquid crystal display or organic EL display. While this embodiment uses six display devices 31(a) to 31(f), the number of display devices 31 is not limited to this number and may be two to five, seven or more, or only one display device 31. The display generation device 32 may be a known computer capable of outputting video to the display 31.

[0025] FIG. 2 shows an example of the functional configuration of the system 100, particularly the control device 10. As shown in FIG. 2, the control device 10 mainly includes a processing unit 11 and a storage unit 12. The processing unit 11 includes, for example, a processor and a main memory, and the processor executes predetermined arithmetic processing according to a program loaded in the main memory. The main memory is used to write and read data such as the progress and results of the processor's arithmetic processing. The processor may be a central processing unit (CPU) or a graphics processing unit (GPU). The main memory may be a volatile memory such as a RAM or DRAM. The storage unit 12 mainly stores data such as video content and objects to be displayed on the display device 30. The processing unit 11 reads video content and the like from the storage unit 12 and outputs the read content to the display device 30. The storage unit 12 may be a nonvolatile memory such as an HDD or SSD. Although not shown, the control device 10 is provided with a known input / output interface for transmitting and receiving information to and from the first sensor 21, the second sensor 22, and the display device 30.

[0026] 2, the processing unit 11 has various functional blocks, such as a display control unit 11a, a detection coordinate measurement unit 11b, a coordinate conversion unit (corresponding to a correspondence unit) 11c, and a correction unit 11d. These functional blocks are functions obtained by the processing unit 11 executing a predetermined program, and are actually realized by cooperation between a processor and a main memory. In particular, the functional blocks shown here relate to a calibration process for matching the detection positions of the sensors 21 and 22 with the display positions of the display device 30. Details of these functional blocks will be described together with the calibration process flow shown in FIG. 3.

[0027] As shown in FIG. 3 , the calibration process first confirms the installation positions of the sensors 21 and 22 relative to the display area of ​​the display device 30 (step S1). Specifically, targets are displayed at positions including the four corners of the display area, and the user touches each target. The touch position is detected by each sensor 21 and 22, and the coordinates of the touch position are confirmed in the sensor coordinate system of each sensor 21 and 22. An overview of step S1 is shown in FIG. 4 . In FIG. 4 , the detection areas of each sensor 21 and 22 are indicated by semicircles. As shown in FIG. 4 , the display area of ​​the display device 30 overlaps with the detection areas of the two sensors 21 and 22. In other words, the display area of ​​the display device 30 is included in the overlapping range of the detection areas of the two sensors 21 and 22. Therefore, each sensor 21 and 22 can detect touch positions throughout the entire display area of ​​the display device 30. First, it is necessary to install each sensor 21 and 22 in this manner.

[0028] 4, the display area of ​​the display device 30 is defined by a two-dimensional display coordinate system (XY coordinate system). The sensor area of ​​the first sensor 21 is defined by a two-dimensional sensor coordinate system (U Ｌ V Ｌ The sensor area of ​​the second sensor 22 is also defined in a two-dimensional sensor coordinate system (U Ｒ V Ｒ In this way, in the initial state, the display device 30, the first sensor 21, and the second sensor 22 each have an independent coordinate system.

[0029] In step S1, targets T are displayed in order at the four corners of the display area of ​​the display device 30, and the user is prompted to touch each target T. At this time, the display control unit 11a of the display device 30 controls the display device 30 to display the targets T at the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display area. Note that one of the features of the present invention is the user interface used when displaying the targets T, the details of which will be described later.

[0030] When the user touches a target T displayed at one of the four corners of the display area of ​​the display device 30, each of the sensors 21 and 22 detects the touch position of the user. As described above, the first sensor 21 and the second sensor 22 each have their own coordinate system. Therefore, depending on the first sensor 21, the first sensor coordinate system U Ｌ , V Ｌ The coordinate P1 (U Ｌ , V Ｌ ) ~ P4 (U Ｌ , V Ｌ ) is detected, and the second sensor 22 detects the second sensor coordinate system U Ｒ , V Ｒ The coordinate P1 (U Ｒ , V Ｒ ) ~ P4 (U Ｒ , V Ｒ ) are detected. The calculation process for determining the coordinate values ​​of each coordinate system based on the detection signals from the sensors 21 and 22 is performed by the processing unit 11 of the control device 10, specifically the detection coordinate measurement unit 11b. In other words, when the detection signals from the sensors 21 and 22 are input to the control device 10, the detection coordinate measurement unit 11b measures the coordinate values ​​of each coordinate system based on the detection signals.

[0031] Next, in the calibration process, the coordinate values ​​detected by the sensors 21 and 22 are converted to obtain coordinate values ​​in a sensor coordinate system U, V that integrates the coordinate systems of the sensors 21 and 22 (step S2). This coordinate conversion is performed by the processing unit 11 of the control device 10, specifically, by the coordinate conversion unit 11c. An overview of step S2 is shown in FIG. 5. In other words, if the coordinate values ​​detected by the sensors 21 and 22 remain defined in separate coordinate systems, they will be difficult to handle in subsequent calculation processes, so they need to be expressed in the same coordinate system as the coordinate values ​​detected by the sensors 21 and 22. Specifically, in the example shown in FIG. 5, new sensor coordinate systems U, V are provided, and the coordinate conversion unit 11c converts the coordinate values ​​detected by the first sensor 21 into the first sensor coordinate system U, V. Ｌ , V Ｌ Coordinates P1 (U Ｌ , V Ｌ ) ~ P4 (U Ｌ , V Ｌ) into coordinate values ​​in the sensor coordinate system U, V. Similarly, the coordinate conversion unit 11c converts the coordinates of the second sensor coordinate system U Ｒ , V Ｒ Coordinates P1 (U Ｒ , V Ｒ ) ~ P4 (U Ｒ , V Ｒ ) is converted into coordinate values ​​in the sensor coordinate system U, V. As a result, the coordinate values ​​(U Ｌ , V Ｌ ) in the coordinate values ​​(U, V) of the sensor coordinate system, and the coordinate values ​​(U Ｌ , V Ｌ ) in the coordinate values ​​(U, V) of the sensor coordinate system. Thereafter, by using these parameters, the coordinate conversion unit 11c can convert the coordinate values ​​(U Ｌ , V Ｌ ) and the coordinate value of the second sensor coordinate system (U Ｒ , V Ｒ ) can be converted into coordinate values ​​(U, V) in the sensor coordinate system. In the example shown in FIG. 5, new sensor coordinate systems U and V are set, but this is not limiting. Ｌ , V Ｌ and the second sensor coordinate system U Ｒ , V Ｒ Alternatively, one of the coordinate values ​​may be converted into the other coordinate value by utilizing either one of the coordinate values.

[0032] In this way, the coordinate conversion unit 11c represents the detection points of each sensor 21, 22 corresponding to the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display area of ​​the display device 30 as coordinates P1(U,V) to P4(U,V) in the sensor coordinate system. Furthermore, the coordinate conversion unit 11c calculates parameters for converting the coordinate values ​​(X,Y) in the display coordinate system to the coordinate values ​​(U,V) in the sensor coordinate system based on the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display coordinate system and the coordinates P1(U,V) to P4(U,V) in the sensor coordinate system. This associates the coordinate values ​​(X,Y) in the display coordinate system of the display device 30 with the coordinate values ​​(U,V) in the sensor coordinate system of each sensor 21, 22. In FIG. 5, the rectangular area (indicated by the dotted line) surrounded by the coordinates P1 to P4 of the four corners in the sensor coordinate system U, V is the interaction area. This interaction area corresponds to the display area of ​​the display device 30. In other words, when each sensor 21, 22 detects that the user has touched within this interaction area, the control device 10 can display, for example, video content at the position where the user touched the display area of ​​the display device 30. Note that in the example shown in FIG. 5 and other figures, targets T are displayed only at the four corners of the display area for the user to touch. However, this is not limiting. The interaction area may be defined by displaying targets at multiple positions along the outer edge of the display area in addition to the four corners and having the user touch them. The more targets displayed, the more accurately the interaction area can be defined.

[0033] Next, in the calibration process, the detection points detected by the sensors 21 and 22 are corrected (step S3). This correction process is performed by the processing unit 11 of the control device 10, specifically the correction unit 11d. An overview of step S3 is shown in FIG. 6. As shown in FIG. 6, the display control unit 11a of the control device 10 controls the display device 30 to display a target T at an arbitrary position P5 within the display area (interaction area) and prompts the user to touch the target T. At this time, the coordinate conversion unit 11c converts the coordinates P5(X,Y) of the arbitrary position P5 in the display coordinate system X,Y to obtain theoretical coordinates P5(U,V) in the sensor coordinate system U,V. Furthermore, when the user touches the target T at the arbitrary position P5, the sensors 21 and 22 detect that the user has touched the target T at the arbitrary position P5. As shown in FIG. 6 , the coordinates P5′(U,V) of the arbitrary position P5 actually detected by the sensors 21 and 22 may deviate from the theoretical coordinates P5(U,V) previously calculated. In this case, the correction unit 11d of the control device 10 locally corrects the position by modifying the grid in the sensor coordinate system U, V so that the actually detected coordinates P5′(U,V) coincide with the theoretical coordinates P5(U,V) (warping process). Specifically, the correction unit 11d corrects the parameters used by the coordinate conversion unit 11c to convert the coordinate values ​​(X,Y) in the display coordinate system calculated in step S2 to the coordinate values ​​(U,V) in the sensor coordinate system, depending on the deviation between the actually detected coordinates P5′(U,V) and the theoretical coordinates P5(U,V). Furthermore, in this correction process (step S3), similar processing is repeatedly performed for multiple arbitrary positions other than the arbitrary position P5. This optimizes the parameters for coordinate conversion.

[0034] Finally, in the calibration process, the operation of the calibrated sensors 21 and 22 and the display device 30 is confirmed (step S4). An overview of step S4 is shown in FIG. 7. For example, as shown in FIG. 7, a user touches a desired position within the display area of ​​the display device 30. Then, each of the sensors 21 and 22 detects the user's touch position. The coordinate conversion unit 11c of the control device 10 converts the detected coordinates (U, V) in the sensor coordinate system into coordinates (X, Y) in the display coordinate system. The display control unit 11a of the control device 10 then controls the display device 30 to display a predetermined object (video content) at the coordinates (X, Y) in the display coordinate system thus determined. If the position where the user touched the display device 30 and the position where the object is displayed in the real world visually match, calibration can be confirmed as complete. That is, in this case, the coordinates (U, V) in the sensor coordinate system and the coordinates (X, Y) in the display coordinate system match. If there is a discrepancy between the position where the user touches the display device 30 and the position where the object is displayed, the calibration process may be restarted from step S1, or if the amount of discrepancy is small, step S3 may be performed additionally.

[0035] Next, a user interface (UI) for displaying the target T in the display area of ​​the display device 30 will be described in detail with reference to Figures 8 to 10. The user interface described here can be applied to the case where the target T is displayed in a location including the four corners of the display area in step S1 in the calibration process described above, or the case where the target T is displayed at an arbitrary position within the display area in step S3. In the examples shown in Figures 9 and 10, the target T is displayed in a position other than a corner of the display area to make the user interface easier to understand, but in the above-mentioned step S1, this target T is displayed in a corner of the display area.

[0036] As shown in step S5 of FIG. 8 and FIG. 9(a), first, the display control unit 11a of the control device 10 displays a target T and a guidance area G at a predetermined position in the display area of ​​the display device 30. The guidance area G is intended to guide the user's line of sight to the target T and present the position of the target T on the display area in an easy-to-understand manner to the user. First, the target T is displayed as a dot of a size that allows the coordinate values ​​of the display coordinate system and the sensor coordinate system to be uniquely identified. The shape of the target T is not particularly limited, but examples include a circle, a triangle, a square, other polygons, a cross, etc. The size of the target T is preferably 5 to 30 mm in diameter, and particularly preferably 5 to 15 mm.

[0037] Furthermore, the guidance area G overlaps with the target T and is displayed over a larger area than the target T. In particular, it is preferable that the guidance area G is displayed so that its center overlaps with the center of the target T. The shape of the guidance area G is preferably circular, particularly a perfect circle, as shown in FIG. 8 . However, it can also be elliptical, triangular, rectangular, or any other polygonal shape as long as it is effective in guiding the user's line of sight. The area (range) in which the guidance area G is initially displayed is preferably 20 to 90% or more, and more preferably 30 to 85% or 40 to 80%, of the area of ​​the display region taken as 100%. Furthermore, it is preferable that the guidance area G be displayed in a manner that is visually distinguishable from the target T. For example, it is effective to display the target T and the guidance area G in different colors. In particular, it is preferable that the color of the guidance area G be set to a color that makes the target T stand out even when it overlaps with the target T.

[0038] Next, as shown in step S6 of FIG. 8 and FIG. 9B, the display control unit 11a gradually reduces the size of the guidance area G while keeping the position and size of the target T fixed. At this time, it is preferable that the convergence point of the guidance area G coincides with the target T. The convergence point is the point where all vectors of the outer edge of the guidance area G intersect while the guidance area G is being reduced, and means the point where the guidance area G will eventually disappear if the guidance area G is reduced indefinitely. In this way, reducing the guidance area G toward the target T (convergence point) more effectively guides the user's gaze to the target T.

[0039] Next, as shown in step S7 of FIG. 8 and FIG. 9(c), the display control unit 11a determines whether the detection point S of the user's touch position detected by each sensor 21, 22 has stopped. If the detection point S of the user's touch position has stopped, it is assumed that the user is touching the target T while referring to the line of sight guidance of the guidance area G, and the process proceeds to the next step. Meanwhile, this determination process is repeated until the detection point S has stopped. Whether the detection point S has stopped can be determined by whether the detection point S has stopped for a predetermined time (approximately 0.5 to 1 second) or more. Furthermore, the size of the guidance area G is preferably larger than the target T, even when minimized as shown in FIG. 9(c). Because the user may miss the target T if the guidance area G disappears, it is preferable to stop the reduction of the guidance area G before it disappears and maintain the guidance area G slightly larger than the target T.

[0040] When the detection point S stops, as shown in step S8 of FIG. 8 and FIG. 10(a), the display control unit 11a displays information (progress information I) regarding the progress of the detection process in the display area of ​​the display device 30. This progress information I is displayed within a range visible to the user touching the target T, along with the target T. For example, the distance from the portion of the progress information I closest to the target T to the target T is preferably within 500 mm, and particularly preferably within 50 to 200 mm. In the example shown in FIG. 10(a), this progress information I includes the remaining time until the end of the detection process, the progress of the detection process of the first sensor 21 (0 to 100%), and the progress of the detection process of the second sensor 22 (0 to 100%). This indicates that the user should continue touching the target T until the remaining time reaches 0. Furthermore, as shown in FIG. 10(b), when the remaining time of the detection process reaches 0, the progress of each sensor 21, 22 becomes 100%. As shown in step S9 of FIG. 8, the display process of this progress information I continues until the detection process by each of the sensors 21 and 22 is completed.

[0041] When the detection process is completed, as shown in step S10 of FIG. 8 and FIG. 10(c), the display control unit 11a displays information (completion information) indicating the completion of the detection process in the display area of ​​the display device 30. As one example of this completion information, the display control unit 11a may display the guidance area G after the detection process by the sensors 21 and 22 is completed in a manner that is visually distinguishable from the state in which the detection process is currently being performed. In the example shown in FIG. 10(c), the color of the guidance area G is changed after the detection process is completed. In addition to changing the color of the guidance area G, the color of the target T may also be changed. This allows the user touching the target T to intuitively understand that the detection process has been completed. In addition to changing the color of the guidance area G as shown in FIG. 10(c), text information (e.g., "Complete") informing the user that the detection process has been completed may also be displayed in the display area of ​​the display device 30.

[0042] 8, the processes from steps S5 to S10 are performed for all targets T. For example, if it is necessary to display targets T at the four corners of the display area and allow the user to view each target T, the processes from steps S5 to S10 are performed for each target T one by one in order. This makes it less likely that the user will miss a target T displayed in the display area. As a result, the user will be able to touch the targets T displayed on the display device 30 in order without missing them, thereby improving the accuracy of the calibration process.

[0043] In the above, in order to express the contents of the present invention, the present specification has described the embodiments of the present invention with reference to the drawings. However, the present invention is not limited to the above embodiments, and includes modifications and improvements that are obvious to those skilled in the art based on the matters described in the present specification.

[0044] REFERENCE SIGNS LIST 10... control device 11... processing unit 11a... display control unit 11b... detection coordinate measurement unit 11c... coordinate conversion unit 11d... correction unit 12... storage unit 21... first sensor 22... second sensor 30... display device 31... display 32... display generation device 100... system G... guidance area S... detection point T... target I... progress information

Claims

1. A system comprising a display device, one or more sensors whose detection areas are set to overlap a display area of ​​the display device, and a control device connected to the display device and the sensor, wherein the control device includes: a display control unit that causes the display device to display a target at a predetermined display position within the display area; and a matching unit that matches the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is being displayed and the display position of the target, wherein the display control unit displays, together with the target, a guidance area larger than the target in a range that includes the target, reduces the guidance area toward a convergence point on or near the target, and displays information regarding the remaining time until the detection process by the sensor is completed near the target when the detection position detected by the sensor stops.

2. The system according to claim 1, wherein the guidance area is a circle centered on the target and is displayed in a manner that is visually distinguishable from the target.

3. The system described in claim 1, wherein the display control unit gradually reduces the guidance area toward a convergence point on or near the target, and determines whether the detection position detected by the sensor has stopped after the guidance area starts to reduce.

4. The system described in claim 1, wherein the display control unit displays both or either of the target and the guidance area in a manner that is visually distinguishable from when the detection process by the sensor is being performed, after the detection position detected by the sensor has stopped and the detection process by the sensor has ended.

5. A program for causing a computer to function as the control device in the system according to claim 1.

6. A calibration method for matching a display position of the display device with a detection position of the sensor in a system including a display device, one or more sensors whose detection areas are set to overlap a display area of ​​the display device, and a control device connected to the display device and the sensor, comprising: a display step in which the control device causes the display device to display a target at a predetermined display position within the display area; and a correction step in which the control device matches the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is being displayed and the display position of the target, the display step including: a step of displaying, together with the target, a guidance area larger than the target in a range including the target; a step of reducing the guidance area toward a convergence point on or near the target; and a step of displaying, near the target, information regarding the remaining time until the detection process by the sensor is completed when the detection position detected by the sensor stops.

Images