System with calibration function, program and calibration method

Abstract

To provide a system with calibration function facilitating calibration processing of a display device and a sensor.SOLUTION: A system includes a display device, one or more sensors with detection areas set so as to overlap a display area of the display device, and a control device connected to the display device and the sensors. The control device includes a display control part for displaying a target T at a predetermined display position within the display area, and an association part for associating the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target T is displayed, and the display position of the target T. The display control part displays together with the target T, a guidance area G larger than the target T in a range including the target T, and reduces the guidance area G toward a convergence point, with the convergence point set on or near the target T.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a calibration technique for a system including a display device and a sensor that detects a touch position on the display area thereof, in particular for matching the display coordinate system of the display device with the sensor coordinate system of the sensor and correcting the misalignment between the two coordinate systems. [Background technology]

[0002] Conventionally, a system has been known in which an optical range sensor is placed near a display area of ​​a display or projector, the inspection area of ​​the sensor is adjusted to overlap with the display area of ​​the display device, and when a user touches the display area, position information (coordinates, etc.) on the display area is acquired. With this system, for example, a predetermined object or the like can be displayed at the contact position on the display area (Patent Document 1, etc.). In addition, it is also possible to realize interactive video expression in which some kind of reaction occurs in a video work displayed by a display device when the user touches the video work.

[0003] Incidentally, when a sensor is installed near the display area of ​​a display device, it is necessary to accurately match the position detected by the sensor with the display position of the display device. If there is a misalignment between the position detected by the sensor and the display position of the display device, there is a concern that a problem may occur, such as a reaction occurring at a position on the display area other than the position touched by the user. Therefore, when using a system combining a display device and a side area sensor, it is known to perform a process of calculating parameters for performing coordinate conversion between the display coordinate system of the display device and the sensor coordinate system of the sensor by matching the position of a specific point in the display area with the position detected by touching that point, that is, a so-called calibration process (Patent Document 2, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-128843 A [Patent Document 2] JP 2012-027769 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, in the calibration process described in Patent Document 2, targets (points) are displayed in areas including the four corners of the display area, and the user is made to touch (instruct) each target with a finger, a touch pen, or the like. In Patent Document 2, as an example, nine targets are displayed in sequence on the display area, and the user is made to touch each target. However, since each target is displayed as a small circular dot, there is a problem that the user has difficulty finding the targets displayed in sequence. If the user misses even one target during the calibration process, an error occurs in the parameters of the coordinate conversion, and accurate calibration cannot be performed. In particular, when a display or screen is installed on the wall of a commercial facility or an exhibition facility, it is necessary to make the display area large. In that case, it becomes more difficult for the user to find the target to be touched in the large display area.

[0006] In addition, after displaying a target in the display area and having the user touch it, the sensor detects the position where the user touched it, but if the user touches the target for a short time, the sensor cannot detect the position where the user touched it accurately, which is a problem. For this reason, the user needs to touch the target for a certain period of time or more, but if the user does not know how long it will take to complete the process, the user will not understand how long he or she should touch the target, and may continue to touch the target longer than necessary, which makes the calibration process more difficult. In particular, such work is often cumbersome for those who are not familiar with the calibration of display devices and sensors.

[0007] Therefore, a main object of the present invention is to make it possible to more easily perform the calibration process of the display device and the sensor. [Means for solving the problem]

[0008] A first aspect of the present invention relates to a system with a calibration function. The system according to the present invention includes a display device, one or more sensors, and a control device. The display area of ​​the display device may be formed by a display device such as a liquid crystal display or an organic EL display, or may be formed by a projector that projects an image on a screen. The display device may be, for example, a display device in which multiple displays are connected (so-called a multi-display). The one or more sensors have a detection area set so as to overlap with the display area of ​​the display device. The sensor is an optical sensor such as an infrared or side-area sensor, and emits an inspection light along the display area of ​​the display device. In addition, when multiple sensors are provided, it is preferable that the detection area of ​​each sensor overlaps with the display area of ​​the display device, but it is sufficient that at least the combined area of ​​the detection areas of each sensor overlaps with the display area of ​​the display device. The control device is a computer connected to the display device and the sensor by wire or wirelessly. This control device includes a display control unit and a correspondence unit (for example, a coordinate conversion unit). The display control unit causes the display device to display a target at a predetermined display position within the display area. The matching unit matches the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is displayed and the display position of the target. Specifically, the matching unit calculates a parameter for performing coordinate conversion between the display coordinate system in the display device and the sensor coordinate system in the sensor by matching the display position with the detection position of the sensor. Here, in the system according to the present invention, the display control unit displays a guidance area larger than the target in a range including the target together with the target. The guidance area is displayed on the display area so as to surround the target. The shape of the guidance area is not particularly limited, and may be a circle, an ellipse, a triangle, a rectangle, a pentagon, or other polygon. In addition, the display control unit gradually reduces the guidance area toward the convergence point on or near the target. In other words, the guidance area becomes smaller steplessly or stepwise toward the target. Note that the guidance area does not need to finally disappear toward the convergence point, and it is preferable that the guidance area surrounds the target even in a minimized state.When the convergence point of the guidance area is near the target, specifically, it means that the guidance area overlaps with the target even in a minimized state. After that, the display control unit determines whether the detection position detected by the sensor has stopped after the guidance area starts to be reduced.

[0009] As in the above configuration, the guidance area is displayed around the target that the user should touch (point) with a finger, a touch pen, etc., and the guidance area is gradually reduced toward the target, so that the position of the target can be presented to the user in an easy-to-understand manner. Also, if the target itself is displayed large so that the user can easily see the target, the target and the detection point cannot be accurately associated, which hinders the calibration process. However, by displaying a large guidance area around the target as in the present invention, the target itself can be kept as a small point while ensuring the visibility of the target, and therefore the accuracy of the calibration process can be improved.

[0010] In the system according to the present invention, the guidance area is preferably a circle (including a perfect circle and an ellipse) centered on the target, and is displayed in a manner that is visually distinguishable from the target. By making the guidance area a circle centered on the target, the user can intuitively understand the position of the target. In particular, it is preferable that the guidance area is a perfect circle. Note that when the guidance area is an ellipse, its center can be defined as the intersection of the major and minor axes.

[0011] In the system according to the present invention, it is preferable that the display control unit displays information about the progress of the detection process by the sensor near the target when the detection position detected by the sensor stops. By displaying the information about the progress of the detection process in the display area in this way, it becomes easier for the user to understand that the detection process is progressing while the user is touching the target. Note that the vicinity of the target may specifically be within the range that is within the user's field of vision together with the target when the user is touching the target. In other words, it is sufficient that the user can simultaneously view the target and the information about the progress of the detection process.

[0012] In the system according to the present invention, the information on the progress of the detection process is preferably information on the remaining time until the detection process by the sensor is completed. For example, the remaining time until the detection process is completed may be displayed as a countdown. This allows the user to easily know how long they need to keep touching the target.

[0013] In the system according to the present invention, it is preferable that the display control unit displays both or either one of the target and the guidance area in a manner that is visually distinguishable from the state during the execution of the detection process by the sensor after the detection position detected by the sensor stops and the detection process by the sensor is completed. For example, the color of the guidance area during the detection process by the sensor may be different from the color of the guidance area after the detection process is completed. This allows the user to intuitively understand that the detection process is completed.

[0014] A second aspect of the present invention relates to a computer program. The program according to the present invention causes a general-purpose computer to function as a control device in the system according to the first aspect. The program according to the present invention may be downloadable via the Internet, or may be stored in a non-transitory recording medium such as a CD-ROM.

[0015] The third aspect of the present invention relates to a calibration method. The calibration method according to the present invention is executed by a system including a display device, one or more sensors whose detection areas are set so as to overlap with the display area of ​​the display device, and a control device connected to the display device and the sensor. The calibration is a process for associating the display position of the display device with the detection position of the sensor. The calibration method includes a display step and a correction step. The display step is a step in which the control device causes the display device to display a target at a predetermined display position within the display area. The correction step is a step in which the control device associates the display position of the display device with the detection position of the sensor based on the detection position detected by the sensor while the target is being displayed and the display position of the target. Here, the display step first displays a guidance area larger than the target in a range including the target together with the target. In the display step, next, the guidance area is gradually reduced toward the convergence point on or near the target. In the display step, after the start of the guidance area, it is determined whether or not the detection position detected by the sensor has stopped. Effect of the Invention

[0016] According to the present invention, the calibration process for the display device and the sensor can be performed more easily. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is an overall view showing a schematic diagram of an embodiment of a system according to the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of the functional configuration of the system according to the present invention. [Diagram 3] FIG. 3 is a flow diagram showing an example of the calibration process. [Figure 4] FIG. 4 is a schematic diagram showing the relationship between the detection area of ​​the sensor and the display area of ​​the display device. [Diagram 5] FIG. 5 shows a schematic diagram of the relationship between the sensor coordinates of the sensor and the display coordinates of the display device. [Figure 6]FIG. 6 shows a schematic diagram of the process of correcting the detection points. [Figure 7] FIG. 7 is a schematic diagram showing the display process of an object by a display device. [Figure 8] FIG. 8 is a flow diagram showing an example of a target display process. [Figure 9] FIG. 9 shows an example of a state in which a guidance area is displayed around a target. [Figure 10] Figure 10 shows an example of progress displayed around a target. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes appropriate modifications of the embodiments described below within the scope obvious to those skilled in the art.

[0019] FIG. 1 shows an embodiment of a system 100 according to the present invention. As shown in FIG. 1, the system 100 includes a control device 10, two sensors 21 and 22, and a display device 30. In the system 100, the display device 30 is provided, which includes a large display 31 attached to a wall surface, and a predetermined video content is displayed on the display device 30. In the embodiment shown in FIG. 1, the display device 30 includes a display 31 and a display generating device 32 that generates a video to be output to the display 31. The display 31 is also composed of a plurality of display devices 31(a)-(f). The system 100 can detect a position on the display area of ​​the display device 30 that is touched by a user with a finger, a touch pen, or the like, using the sensors 21 and 22, and can reflect information on the detected position in the video content, etc., displayed on the display device 30. For example, when a user touches the display area, a predetermined object is displayed at the touched position, or the video at the touched position changes.

[0020] The control device 10 is responsible for the overall control of the system 100. Specifically, the control device 10 is connected to the display device 30 and the sensors 21, 22 via a known bus, hub, or the like. The control device 10 receives detection information from the sensors 21, 22 and controls the display device 30 to display video content, objects, and the like. The control device 10 can be realized by a general-purpose computer. In the example shown in FIG. 1, the control device 10 is configured by a computer installed locally together with the display device 30 and the sensors 21, 22, but is not limited to this, and may include a cloud server connected to this local computer via the Internet. In other words, the control device 10 is not limited to a single computer, and may be realized by sharing its functions among multiple computers or servers.

[0021] The sensors 21 and 22 are for detecting the position touched by the user in the display area of ​​the display device 30. Each of the sensors 21 and 22 is installed, for example, above the display area of ​​the display device 30, and the installation position is adjusted so that the entire display area becomes the detection area. More specifically, the system 100 according to this embodiment includes a first sensor 21 and a second sensor 22, and the first sensor 21 is installed in the upper left of the display area of ​​the display device 30, and the second sensor 22 is installed in the upper right of the display area. By installing multiple sensors 21 and 22, the detection accuracy of the user's finger or touch pen can be improved. For example, even if multiple users are touching the display area of ​​the display device 30 at the same time, the multiple touch positions can be detected with high accuracy by providing multiple sensors 21 and 22.

[0022] Moreover, the sensors 21 and 22 are preferably range sensors (area sensors). The range sensor is a two-dimensional scanning type optical distance measuring sensor that emits an inspection light (laser) and measures the distance and angle to an object to be inspected (such as a user's finger or a touch pen) by scanning the inspection light. Information detected by the range sensor is transmitted to the control device 10. The range sensor emits the inspection light along the surface area of ​​the display device 30. If the position coordinates of the range sensor in the real space are known, the position coordinates of the object to be inspected in the real space can be detected by measuring the distance and angle to the object touched by the light emitted from the range sensor. Note that each of the sensors 21 and 22 is not limited to a range sensor, and a known infrared sensor can also be used.

[0023] The display device 30 displays video content under the control of the control device 10. In the example shown in FIG. 1, the display device 30 is composed of a display 31 and a display generating device 32, and the display 31 is composed of a plurality of display devices 31(a) to 31(f) connected in a daisy chain manner. In this display device 30, one display area is composed of the plurality of display devices 31(a) to 31(f), and video content can be displayed across the boundaries between the display devices 31(a) to 31(f). A known liquid crystal display or an organic EL display may be adopted as each of the display devices 31(a) to 31(b). In this embodiment, six display devices 31(a) to 31(f) are used, but the number of display devices 31 is not limited to this, and may be two to five or seven or more, or only one display device 31 may be used. In addition, a known computer capable of outputting video to the display 31 may be adopted as the display generating device 32.

[0024] FIG. 2 shows an example of the functional configuration of the system 100, particularly the control device 10. As shown in FIG. 2, the control device 10 mainly includes a processing unit 11 and a storage unit 12. The processing unit 11 includes, for example, a processor and a main memory, and the processor executes a predetermined arithmetic processing according to a program deployed in the main memory. The main memory is used to write or read the progress of the arithmetic processing by the processor, the arithmetic results, and the like. The processor may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The main memory may be a volatile memory such as a RAM or a DRAM. The storage unit 12 mainly stores data such as video content and objects to be displayed on the display unit 30. The processing unit 11 reads out the video content and the like from the storage unit 12, and outputs the read out content to the display device 30. The storage unit 12 may be a non-volatile memory such as an HDD or an SDD. Although not shown in the figure, the control device 10 is provided with a known input / output interface for transmitting and receiving information between the first sensor 21, the second sensor 22, and the display device 30.

[0025] 2, the processing unit 11 has various functional blocks, such as a display control unit 11a, a detection coordinate measurement unit 11b, a coordinate conversion unit (corresponding to a correspondence unit) 11c, and a correction unit 11d. These functional blocks are functions obtained by the processing unit 11 executing a predetermined program, and are actually realized by cooperation between a processor and a main memory. In particular, the functional blocks shown here relate to a calibration process for matching the detection positions of the sensors 21, 22 with the display positions of the display device 30. Details of these functional blocks will be described together with the flow of the calibration process shown in FIG. 3.

[0026] As shown in FIG. 3, in the calibration process, first, the installation positions of the sensors 21 and 22 with respect to the display area of ​​the display device 30 are confirmed (step S1). Specifically, targets are displayed at positions including the four corners of the display area, and the user touches each target. The touch positions are detected by the sensors 21 and 22, and it is confirmed as what coordinates the touch positions are detected as in the sensor coordinate system of the sensors 21 and 22. An overview of this step S1 is shown in FIG. 4. In FIG. 4, the detection areas of the sensors 21 and 22 are shown as semicircles. As shown in FIG. 4, the display area of ​​the display device 30 overlaps with the detection areas of the two sensors 21 and 22. In other words, the display area of ​​the display device 30 is included in the range where the detection areas of the two sensors 21 and 22 overlap. Therefore, each of the sensors 21 and 22 can detect touch positions throughout the entire display area of ​​the display device 30. First, it is necessary to install each of the sensors 21 and 22 in this way.

[0027] 4, the display area of ​​the display device 30 is defined in a two-dimensional display coordinate system (XY coordinate system). The sensor area of ​​the first sensor 21 is defined in a two-dimensional sensor coordinate system (U L V L The sensor area of ​​the second sensor 22 is also defined in a two-dimensional sensor coordinate system (U R V R In this way, in the initial state, the display device 30, the first sensor 21, and the second sensor 22 each have an independent coordinate system.

[0028] In step S1, targets T are displayed in order at the four corners of the display area of ​​the display device 30, and the user is prompted to touch each target T. At this time, the display control unit 11a of the display device 30 controls the display device 30 to display the targets T at the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display area. Note that one of the features of the present invention is the user interface used when displaying the targets T, which will be described in detail later.

[0029] When a user touches a target T displayed at one of the four corners of the display area of ​​the display device 30, each of the sensors 21 and 22 detects the touch position of the user. As described above, the first sensor 21 and the second sensor 22 each have their own coordinate system. Therefore, depending on the first sensor 21, the first sensor coordinate system U L ,V L The coordinate P1(U L ,V L )~P4(U L ,V L ) is detected, and the second sensor 22 detects the second sensor coordinate system U R ,V R The coordinate P1(U R ,V R )~P4(U R ,V R ) is detected. The calculation process for determining the coordinate values ​​of each coordinate system based on the detection signals from the sensors 21, 22 is performed by the processing unit 11 of the control device 10, specifically, the detection coordinate measurement unit 11b. That is, when the detection signals from the sensors 21, 22 are input to the control device 10, the detection coordinate measurement unit 11b measures the coordinate values ​​of each coordinate system based on the detection signals.

[0030] Next, in the calibration process, the coordinate values ​​detected by the sensors 21 and 22 are converted to obtain coordinate values ​​in a sensor coordinate system U, V that combines the coordinate systems of the sensors 21 and 22 (step S2). This coordinate conversion is performed by the processing unit 11 of the control device 10, specifically, the coordinate conversion unit 11c. An overview of this step S2 is shown in FIG. 5. In other words, if the coordinate values ​​detected by the sensors 21 and 22 remain defined in separate coordinate systems, they will be difficult to handle in subsequent calculation processes, so they need to be expressed in the same coordinate system as the coordinate values ​​detected by the sensors 21 and 22. Specifically, in the example shown in FIG. 5, new sensor coordinate systems U, V are provided, and the coordinate conversion unit 11c converts the first sensor coordinate system U detected by the first sensor 21 into L ,V L Coordinates P1(U L ,V L )~P4(U L ,V L) into coordinate values ​​in the sensor coordinate system U, V. Similarly, the coordinate conversion unit 11c converts the coordinates of the second sensor coordinate system U R ,V R Coordinates P1(U R ,V R )~P4(U R ,V R ) is converted to coordinate values ​​in the sensor coordinate system U, V. As a result, the coordinate values ​​(U L ,V L ) in the coordinate values ​​(U, V) of the sensor coordinate system, and the coordinate values ​​(U L ,V L ) in the coordinate values ​​(U, V) of the sensor coordinate system is obtained. Thereafter, the coordinate conversion unit 11c uses this parameter to convert the coordinate values ​​(U L ,V L ) and the coordinate value of the second sensor coordinate system (U R ,V R ) can be converted to coordinate values ​​(U, V) in the sensor coordinate system. In the example shown in FIG. 5, new sensor coordinate systems U and V are set, but this is not limiting. The first sensor coordinate system U L ,V L and the second sensor coordinate system U R ,V R Alternatively, one of the above may be utilized, and the coordinate values ​​of one may be converted into the coordinate values ​​of the other.

[0031] In this way, the coordinate conversion unit 11c expresses the detection points of the sensors 21, 22 corresponding to the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display area of ​​the display device 30 as coordinates P1(U,V) to P4(U,V) of the sensor coordinate system. The coordinate conversion unit 11c also obtains parameters for converting the coordinate values ​​(X,Y) of the display coordinate system into the coordinate values ​​(U,V) of the sensor coordinate system based on the coordinates P1(X,Y) to P4(X,Y) of the four corners of the display coordinate system and the coordinates P1(U,V) to P4(U,V) of the sensor coordinate system. As a result, the coordinate values ​​(X,Y) in the display coordinate system of the display device 30 correspond to the coordinate values ​​(U,V) of the sensors 22, 22 in the sensor coordinate system. In addition, in FIG. 5, a rectangular area (dotted line) surrounded by the coordinates P1 to P4 of the four corners in the sensor coordinate system U,V is the interaction area. This interaction area corresponds to the display area of ​​the display device 30. That is, when each of the sensors 21 and 22 detects that the user has touched within this interaction area, the control device 10 can display, for example, video content at the position where the user touched the display area of ​​the display device 30. Note that in the example shown in FIG. 5 and the like, the targets T are displayed only at the four corners of the display area to be touched by the user, but this is not limited thereto, and the interaction area may be defined by displaying targets at a plurality of other positions along the outer edge of the display area in addition to the four corners and having the user touch them. The more targets are displayed, the more accurately the interaction area can be defined.

[0032] Next, in the calibration process, the detection points by the sensors 21 and 22 are corrected (step S3). This correction process is performed by the processing unit 11 of the control device 10, specifically, the correction unit 11d. An overview of this step S5 is shown in FIG. 6. As shown in FIG. 6, the display control unit 11a of the control device 10 controls the display device 30 to display a target T at an arbitrary position P5 in the display area (interaction area) and has the user touch the target T. At this time, the coordinate conversion unit 11c converts the coordinates P5(X,Y) of the arbitrary position P5 in the display coordinate system X,Y to obtain theoretical coordinates P5(U,V) in the sensor coordinate system U,V. In addition, when the user touches the target T at the arbitrary position P5, each of the sensors 21 and 22 detects that the user has touched the target T at the arbitrary position P5. At this time, as shown in FIG. 6, the coordinates P5'(U,V) of the arbitrary position P5 actually detected by each sensor 21, 22 may deviate from the theoretical coordinates P5(U,V) previously obtained. In that case, the correction unit 11d of the control device 10 corrects the grid in the sensor coordinate system U,V so that the actually detected coordinates P5'(U,V) coincide with the theoretical coordinates P5(U,V), and locally corrects the position (warping process). Specifically, the correction unit 11d may correct the parameters for converting the coordinate values ​​(X,Y) of the display coordinate system obtained by the coordinate conversion unit 11c in step S2 into the coordinate values ​​(U,V) of the sensor coordinate system according to the amount of deviation between the actually detected coordinates P5'(U,V) and the theoretical coordinates P5(U,V). In addition, in this correction process (step S3), the same process is repeatedly performed for a plurality of arbitrary positions other than the arbitrary position P5. As a result, the parameters for coordinate conversion are optimized.

[0033] Finally, in the calibration process, the operation of each of the calibrated sensors 21, 22 and the display device 30 is confirmed (step S4). An overview of this step S4 is shown in FIG. 7. For example, as shown in FIG. 7, the user touches a desired position in the display area of ​​the display device 30. Then, each of the sensors 21, 22 detects the user's touch position. The coordinate conversion unit 11c of the control device 10 converts the detected coordinates (U, V) in the sensor coordinate system into coordinates (X, Y) in the display coordinate system. Then, the display control unit 11a of the control device 10 controls the display device 30 to display a predetermined object (video content) at the coordinates (X, Y) in the display coordinate system obtained here. At this time, if the position where the user touched the display device 30 and the position where the object is displayed in the real world match with each other visually, it can be confirmed that the calibration is completed. That is, in this case, the coordinates (U, V) in the sensor coordinate system and the coordinates (X, Y) in the display coordinate system match. If there is a discrepancy between the position where the user touches the display device 30 and the position where the object is displayed, the calibration process may be repeated from step S1, or if the amount of discrepancy is small, step S3 may be additionally performed.

[0034] Next, a user interface (UI) for displaying the target T in the display area of ​​the display device 30 will be described in detail with reference to Fig. 8 to Fig. 10. The user interface described here can be applied to a case where the target T is displayed in a location including the four corners of the display area in step S1 in the above-mentioned calibration process, or a case where the target T is displayed at an arbitrary position within the display area in step S3. In the example shown in Fig. 9 and Fig. 10, the target T is displayed in a position other than the corners of the display area in order to clearly show the user interface, but in the above-mentioned step S1, this target T is displayed in a corner of the display area.

[0035] As shown in step S5 of FIG. 8 and FIG. 9(a), first, the display control unit 11a of the control device 10 displays the target T and the guidance area G at a predetermined position in the display area of ​​the display device 30. The guidance area G is intended to guide the user's gaze to the target T and present the position of the target T on the display area in an easy-to-understand manner to the user. First, the target T is displayed as a point of a size that allows the coordinate values ​​of the display coordinate system and the sensor coordinate system to be uniquely specified. The shape of the target T is not particularly limited, but examples include a circle, a triangle, a square, other polygons, a cross mark, and the like. The size of the target T is preferably 5 to 30 mm in diameter, and particularly preferably 5 to 15 mm.

[0036] Moreover, the guidance area G overlaps with the target T and is displayed in a range larger than the target T. In particular, the guidance area G is preferably displayed so that its center overlaps with the center of the target T. The shape of the guidance area G is preferably a circle, particularly a perfect circle, as shown in FIG. 8, but it can also be an ellipse, a triangle, a rectangle, or a polygon thereof, as long as it has the effect of guiding the user's line of sight. The area (range) in which the guidance area G is initially displayed is preferably 20 to 90% or more, and particularly preferably 30 to 85% or 40 to 80%, when the area of ​​the display area is 100%. Moreover, the guidance area G is preferably displayed in a manner that is distinguishable from the target T in appearance. For example, it is effective to display the target T and the guidance area G in different colors. In particular, the color of the guidance area G is preferably set to a color that makes the target T stand out even when it overlaps with the target T.

[0037] Next, as shown in step S6 of FIG. 8 and FIG. 9(b), the display control unit 11a gradually reduces the size of the guidance area G while keeping the position and size of the target T fixed. At this time, it is preferable that the convergence point of the guidance area G coincides with the target T. The convergence point is an intersection point where all vectors of the outer edge of the guidance area G intersect while the guidance area G is being reduced, and means a point where the guidance area G will eventually disappear if the guidance area G is reduced indefinitely. In this way, by reducing the guidance area G toward the target T (convergence point), the effect of guiding the user's gaze to the target T is strengthened.

[0038] Next, as shown in step S7 of FIG. 8 and FIG. 9(c), the display control unit 11a judges whether or not the detection point S of the user's touch position detected by each sensor 21, 22 has stopped. If the detection point S of the user's touch position has stopped, it is presumed that the user is touching the target T with reference to the line of sight guidance of the guidance area G, and the process proceeds to the next step. Meanwhile, this judgment process is repeated until the detection point S has stopped. Whether or not the detection point S has stopped may be judged based on whether or not the detection point S has stopped for a predetermined time (about 0.5 to 1 second). In addition, the size of the guidance area G is preferably larger than the target T even when minimized as shown in FIG. 9(c). If the guidance area G disappears, there is a risk that the user will overlook the target T, so it is preferable to stop the reduction of the guidance area G before it disappears and maintain the guidance area G in a state one size larger than the target T.

[0039] When the detection point S stops, as shown in step S8 of FIG. 8 and FIG. 10(a), the display control unit 11a displays information (progress information) on the progress of the detection process in the display area of ​​the display device 30. This progress information is displayed within a range that is visible to the user who is touching the target T together with the target T. For example, the distance from the part of the progress information closest to the target T to the target T is preferably within 500 mm, and particularly preferably within 50 to 200 mm. In the example shown in FIG. 10(a), the progress information includes the remaining time until the end of the detection process, the progress degree (0 to 100%) of the detection process of the first sensor 21, and the progress degree (0 to 100%) of the detection process of the second sensor 22. Therefore, it is understood that the user should continue to touch the target T until the remaining time becomes 0. Also, as shown in FIG. 10(b), when the remaining time of the detection process becomes 0, the progress degree of each sensor 21, 22 becomes 100%. As shown in step S9 in FIG. 8, the display process of this progress information continues until the detection process by each of the sensors 21 and 22 is completed.

[0040] When the detection process is completed, as shown in step S10 of FIG. 8 and FIG. 10(c), the display control unit 11a displays information (completion information) on the completion of the detection process in the display area of ​​the display device 30. As one of the completion information, the display control unit 11a may display the guidance area G in a manner that is distinguishable in appearance from the state during the execution of the detection process after the detection process by the sensors 21 and 22 is completed. In the example shown in FIG. 10(c), the color of the guidance area G is changed after the detection process is completed. In addition to changing the color of the guidance area G, the color of the target T may be changed. This allows the user who is touching the target T to intuitively understand that the detection process is completed. In addition to changing the color of the guidance area G, as shown in FIG. 10(c), text information (e.g., Complete) that informs the user that the detection process is completed may be displayed in the display area of ​​the display device 30.

[0041] As shown in Fig. 8, the processes from steps S5 to S10 are performed for all targets T. For example, when it is necessary to display targets T at the four corners of the display area and have the user display each target T, the processes from steps S5 to S10 are performed for each target T one by one in order. This makes it difficult for the user to overlook the target T displayed in the display area. As a result, the user can touch the targets T displayed on the display device 30 in order without overlooking them, thereby improving the accuracy of the calibration process.

[0042] In the above, in order to express the contents of the present invention, the present specification has described the embodiments of the present invention with reference to the drawings. However, the present invention is not limited to the above-mentioned embodiments, and includes modifications and improvements that are obvious to a person skilled in the art based on the matters described in the present specification. [Explanation of symbols]

[0043] 10...Control device 11...Processing unit 11a... Display control unit 11b... Detection coordinate measurement unit 11c... Coordinate conversion section 11d... Correction section 12...Memory unit 21...First sensor 22... second sensor 30... display device 31...display 32...display generating device 100…System G…Guidance area S: Detection point T: Target

Claims

1. A system including a display device, one or more sensors having a detection area set to overlap a display area of ​​the display device, and a control device connected to the display device and the sensor, The control device includes: a display control unit that causes the display device to display a target at a predetermined display position within the display area; a correlation unit that correlates a display position of the display device with a detection position of the sensor based on a detection position detected by the sensor while the target is being displayed and a display position of the target, The display control unit is Displaying a guidance area larger than the target in a range including the target together with the target; The guidance area is gradually reduced toward a convergence point on or near the target, After the guide area starts to shrink, it is determined whether the detection position detected by the sensor has stopped. system.

2. The guidance area is a circle centered on the target, and is displayed in a manner that is visually distinguishable from the target. The system of claim 1 .

3. The display control unit displays information about a progress status of a detection process by the sensor near the target when the detection position detected by the sensor stops. A system according to claim 1 or claim 2.

4. The information about the progress is information about the remaining time until the detection process by the sensor is completed. The system of claim 3.

5. The display control unit displays both or either one of the target and the guidance area in a manner that is visually distinguishable from the manner in which the detection process by the sensor is being performed after the detection position detected by the sensor has stopped and the detection process by the sensor has ended. A system according to claim 1 or claim 2.

6. A program for causing a computer to function as the control device in the system according to claim 1 or 2.

7. 1. A calibration method for associating a display position of the display device with a detection position of the sensor in a system including a display device, one or more sensors having a detection area set to overlap a display area of ​​the display device, and a control device connected to the display device and the sensor, the method comprising: a display step in which the control device causes the display device to display a target at a predetermined display position within the display area; a correction step in which the control device causes a display position of the display device to correspond to a detection position of the sensor based on a detection position detected by the sensor while the target is being displayed and a display position of the target; The display step includes: displaying a guidance area larger than the target in a range including the target together with the target; a step of gradually reducing the induction area toward a convergence point on or near the target; and determining whether the detection position detected by the sensor has stopped after the start of the guidance area. Calibration method.

Images