Wearable device and input system

Abstract

To provide a wearable device and an input system capable of improving input operability by a user in an apparatus configuration in which a detection device for detecting an instruction position of an instruction device and a display unit for displaying a space are separately provided.SOLUTION: A wearable device (20) is provided with a processor (24) and a memory for storing an instruction for calculating an instruction position based on data transmitted from an instruction device (30) when executed by the processor (24). The data includes a pair of a first state quantity indicating the state of the instruction device (30) or a time variation of the state, and a second state quantity indicating the state of a detection device (40) or a time variation of the state.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a wearable device and an input system. [Background technology]

[0002] Digital ink systems have been known in the past that allow users to input ink data describing a collection of strokes through writing operations using a pointing device, such as a stylus. For example, it is conceivable that a user's writing operations are followed by displaying handwritten content at a location separate from the writing location.

[0003] Patent Document 1 discloses a technology in which a two-dimensional virtual plane is set in a three-dimensional space, and then strokes written on this virtual plane are instantly displayed on a display device that can be worn by the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-125487 Summary of the Invention [Problem to be solved by the invention]

[0005] By displaying a virtual space where various objects exist, a user can have a simulated experience of leaving handwriting in that space. However, in the technology disclosed in Patent Document 1, the virtual plane is set and then fixed, which can lead to a problem that the handwriting content is displayed in a location different from the user's intention, impairing the user's input operability.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide an input system and an input method that can improve input operability for users in an apparatus configuration in which a detection device that detects the pointing position of a pointing device and a display unit that displays the space are provided separately. [Means for solving the problem]

[0007] The input system of the first invention comprises a pointing device, a detection device that detects the pointing position of the pointing device, a display unit that is provided separately from the detection device and displays a space, a position detection unit that detects the user's gaze position within the space, and a processor, wherein the processor sets a target area within the space that includes a reference position of the pointing device based on the gaze position detected by the position detection unit at the time a predetermined operation is received by the pointing device, and controls the display unit to display the pointing position of the pointing device within the target area by superimposing it on the space.

[0008] The input method of the second invention is a method using an input system comprising: a pointing device; a detection device that detects the pointing position of the pointing device; a display unit that is provided separately from the detection device and displays a space; and a position detection unit that detects the user's gaze position within the space, in which one or more processors execute the steps of setting a target area within the space that includes a reference position of the pointing device based on the gaze position detected by the position detection unit at the time a predetermined operation is received by the pointing device; and controlling the display unit to display the pointing position of the pointing device within the target area by overlaying it on the space. [Effects of the Invention]

[0009] According to the present invention, in an apparatus configuration in which a detection device that detects the pointing position of a pointing device and a display unit that displays a space are provided separately, input operability by a user is further improved. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating the overall configuration of an input system according to an embodiment of the present invention. [Figure 2] 2 is a flowchart showing an input operation of the input system shown in FIG. [Figure 3] FIG. 10 is a diagram illustrating an example of a virtual space displayed on a wearable device. [Figure 4] FIG. 10 is a diagram illustrating an example of a data structure of an area definition table. [Figure 5] 10A and 10B are diagrams showing the transition of a 3D image before and after a pen-down operation. [Figure 6] FIG. 2 is a diagram illustrating the correspondence between a pen coordinate system and a virtual coordinate system. [Figure 7] 10A and 10B are diagrams showing the transition of a 3D image before and after a pen-up operation. [Figure 8] FIG. 10 is a diagram showing the correspondence between a pen coordinate system and a detection coordinate system. [Figure 9] 10A and 10B are schematic diagrams relating to a correction operation taking into consideration the state quantity of the tablet. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. To facilitate understanding of the description, the same components and steps in the drawings will be denoted by the same reference numerals as much as possible, and duplicated descriptions may be omitted.

[0012] [Configuration of Input System 10] 1 is a diagram illustrating the overall configuration of an input system 10 according to one embodiment of the present invention. This input system 10 is a "digital ink system" that allows input of ink data describing a collection of strokes through writing operations using a pointing device. Specifically, the input system 10 includes a wearable device 20, which is a type of display device, a stylus 30, which is a type of pointing device, and a tablet 40, which is a type of detection device.

[0013] Wearable device 20 is a portable display device that can be worn by user U. Specifically, wearable device 20 includes a housing 21, a display panel 22 (corresponding to a "display unit"), a gaze sensor 23 (corresponding to a "position detection unit"), a processor 24 (corresponding to a "processor"), and a communication module 25.

[0014] The housing 21 includes a frame that holds each electronic component and a fixing member that fixes the frame to the head of the user U. The display panel 22 is capable of displaying images or videos, and may be, for example, a liquid crystal panel, an organic EL (Electro-Luminescence) panel, electronic paper, etc. The gaze sensor 23 is, for example, an infrared sensor, and detects a physical quantity (for example, eye movement) that correlates with the gaze of the user U.

[0015] The processor 24 is configured by a processing operation device including a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). The processor 24 reads and executes programs stored in a memory (not shown) to control the display of images, perform various calculations required for the display control, and control the transmission and reception of data.

[0016] The communication module 25 is a communication interface for transmitting and receiving electrical signals to and from an external device, which enables the wearable device 20 to exchange various data with the stylus 30 or the tablet 40.

[0017] The stylus 30 is an electronic pen configured to be capable of one-way or two-way communication with the tablet 40. The electronic pen may be of either an electromagnetic induction type (EMR) or an active electrostatic coupling type (AES). For example, in the case of the AES type, the stylus 30 includes a pen pressure sensor 31, an inertial measurement unit (hereinafter referred to as a first IMU 32), a micro control unit (hereinafter referred to as an MCU 33), and a communication chip 34.

[0018] The writing pressure sensor 31 is, for example, a pressure sensor using a variable capacitance capacitor that detects changes in capacitance caused by pressure on the pen tip. This writing pressure sensor 31 can detect not only writing pressure but also pen events including pen-down and pen-up of the stylus 30.

[0019] The first IMU 32 is a measurement unit formed by combining, for example, a three-axis gyro sensor and a three-directional acceleration sensor. This allows the first IMU 32 to measure state quantities that indicate the state of the device itself or changes in that state over time on a pen coordinate system 60 (see FIGS. 6 and 8), which will be described later. These state quantities include various physical quantities used to identify the position and orientation, such as position, velocity, acceleration, jerk, angle, and angular velocity.

[0020] The MCU 33 is a control unit including a processor capable of controlling the operation of the stylus 30. For example, the MCU 33 performs various calculations related to the calculation of the position indicated by the stylus 30, as well as controls the transmission and reception of data including the results of these calculations.

[0021] The communication chip 34 is an integrated circuit for wireless communication with external devices in accordance with various wireless communication standards, including Bluetooth (registered trademark), which allows the stylus 30 to exchange various data with the wearable device 20 or the tablet 40 via the communication chip 34.

[0022] Tablet 40 is a device capable of detecting the position indicated by stylus 30, and may or may not have a display function. Various detection devices, including a smartphone or a personal computer, may be used instead of tablet 40. User U can write pictures and characters on the display surface of wearable device 20 by holding stylus 30 in one hand and moving the pen tip against touch surface 41 of tablet 40.

[0023] The tablet 40 includes a sensor electrode 42, a touch IC (Integrated Circuit) 43, a CPU 44, a communication chip 45, and a second IMU 46. The second IMU 46 may be omitted as necessary.

[0024] The sensor electrode 42 is a collection of electrodes capable of detecting a change in capacitance caused by the approach or contact of a conductor. The capacitance detection method may be either a mutual capacitance method or a self-capacitance method. For example, in the case of the mutual capacitance method, the sensor electrode 42 includes a plurality of X-line electrodes for detecting the position of the Xd axis of a detection coordinate system 80 (see FIG. 8) and a plurality of Y-line electrodes for detecting the position of the Yd axis.

[0025] The touch IC 43 is an integrated circuit that controls the driving of the sensor electrodes 42. The touch IC 43 drives the sensor electrodes 42 based on a control signal supplied from the CPU 44. This allows the touch IC 43 to perform a "pen detection function" that detects the state of the stylus 30 and a "touch detection function" that detects a touch by the user U's finger or the like.

[0026] The CPU 44 can execute various functions, including generating ink data and controlling data transmission and reception, by reading and executing programs stored in a memory (not shown). Note that various processing and calculation devices, including an MPU and a GPU, may be used instead of the CPU 44.

[0027] Like the communication chip 34, the communication chip 45 is an integrated circuit for wirelessly communicating with an external device in accordance with various communication standards including Bluetooth (registered trademark). This allows the tablet 40 to exchange various data with the wearable device 20 or the stylus 30 via the communication chip 45.

[0028] The second IMU 46 is a measurement unit formed by combining, for example, a three-axis gyro sensor and a three-direction acceleration sensor, similar to the first IMU 32. As a result, the second IMU 46 is configured to be able to measure state quantities related to the position and orientation of the device itself on a detection coordinate system 80 (see FIG. 8), which will be described later.

[0029] [Input System 10 Operation] The input system 10 of this embodiment is configured as described above. Next, the input operation by the input system 10 will be described with reference to the flowchart of FIG.

[0030] <Basic operation> 2, the input system 10 displays the virtual space 50 in a manner that is visible to the user U. Specifically, the processor 24 of the wearable device 20 performs display control on the display panel 22, whereby an image or video showing the virtual space 50 is displayed.

[0031] 3 is a diagram showing an example of a virtual space 50 displayed on the wearable device 20. This virtual space 50 shows a three-dimensional imaginary room interior. Various objects are provided in this room, including structures such as a ceiling, floor, walls, and closet, and furniture such as a bed, table, and chairs.

[0032] In step S2 of FIG. 2, the input system 10 checks whether a pen-down operation by the stylus 30 has been received. Specifically, the MCU 33 of the stylus 30 determines whether the pen pressure state has changed from "OFF" to "ON" based on the detection signal of the pen pressure sensor 31. If a pen-down operation is not detected (step S2: NO), the process remains in step S1 until the operation is detected. On the other hand, if a pen-down operation is detected (step S2: YES), the process proceeds to the next step S3. The wearable device 20 may obtain the detection result by the stylus 30 directly or indirectly via the tablet 40.

[0033] In step S3, the input system 10 identifies an object (hereinafter referred to as a visual object) that the user U is gazing at when the pen-down operation detected in step S2 is made. First, the processor 24 of the wearable device 20 detects the gaze position 52 of the user U based on the detection signal of the gaze sensor 23, and then identifies an object in the virtual space 50 that is in the path of this gaze. When identifying the visual object, the processor 24 may, for example, refer to the area definition table shown in FIG.

[0034] Fig. 4 is a diagram showing an example of the data structure of an area definition table for virtual space 50 in Fig. 3. This area definition table is data in a table format that shows the correspondence between [1] an "object ID" that is identification information of an object in virtual space 50, [2] an "occupation range" that the object occupies, [3] a "base vector" of a coordinate system formed by the object surface, and [4] a "reference coordinate" that indicates the origin of the coordinate system.

[0035] First, processor 24 detects gaze position 52 of user U based on the detection signal of gaze sensor 23, and calculates a gaze vector in virtual space 50. Then, processor 24 refers to the area definition table shown in FIG. 4 and determines whether the gaze vector intersects with the occupied range associated with each object. If a corresponding object is found, the object is identified as a "visual object." Note that, although an ID is assigned to each object in the example of FIG. 4, an ID may be assigned to each surface of an object having multiple surfaces.

[0036] 2, the input system 10 (more specifically, the processor 24) determines whether the visual object identified in step S3 is the same as the visual object identified previously. If they are the same visual object (step S4: YES), the process proceeds to step S5, whereas if they are different visual objects (step S4: NO), the process proceeds to step S6.

[0037] When the process proceeds to step S5, the input system 10 (more specifically, the processor 24) sets a reference area 74 ( FIG. 6 ) corresponding to the visual object identified in step S3, i.e., the current line of sight. Specifically, the processor 24 references the area definition table of FIG. 4 and sets a planar reference area 74 in the virtual coordinate system 70 ( FIG. 6 ). For example, when the visual object has ID=OB0001, the processor 24 sets the reference position 72 ( FIG. 6 ) with coordinate values ​​(Xv1, Yv1, Zv1) as the “center point” of the reference area 74, the two base vectors (Ex1↑, Ey1↑) as the “plane vectors” of the reference area 74, and the remaining base vector Ez1↑ as the “normal vector” of the reference area 74.

[0038] When the process proceeds to step S6, the input system 10 (more specifically, the processor 24) sets the same referent area 74 as in step S5 or S6 of the previous execution of the flowchart, i.e., the previous case. At the same time, the processor 24 displays the virtual space 50 with the viewpoint switched so that the referent area 74 faces forward.

[0039] 5 is a diagram showing the transition between 3D images 50a and 50b before and after a pen-down operation. More specifically, the upper diagram shows 3D image 50a before the pen-down operation, and the lower diagram shows 3D image 50b after the pen-down operation. For example, when a pen-down operation is performed while user U's line of sight 52 is at the closet door, 3D image 50a transitions to 3D image 50b. This 3D image 50b corresponds to a two-dimensional image in which the viewpoint is switched so that the closet door (i.e., pointing target area 74) is directly in front.

[0040] 2, the input system 10 performs a drawing process in the virtual space 50 in response to a pen move operation by the user U. First, the stylus 30 sets the position of the stylus 30 at the time of the pen-down operation as the origin Op of the pen coordinate system 60, and then sequentially acquires state quantities measured by the first IMU 32. Then, the wearable device 20 directly acquires data including these state quantities from the stylus 30. Alternatively, the wearable device 20 may indirectly acquire the data from the stylus 30 via the tablet 40.

[0041] Then, processor 24 converts the state quantities of stylus 30 in accordance with a predetermined conversion rule, and sequentially adds the converted state quantities to calculate the pointed position in virtual space 50. An example of calculating the pointed position in virtual coordinate system 70 using a time series of the movement amount will be described below with reference to FIG.

[0042] 6 is a three-dimensional Cartesian coordinate system consisting of Xp, Yp, and Zp axes, and is uniquely defined by the stylus 30. For example, if the pen-down position 62 of the stylus 30 is the origin Op, then m↑(t) indicates the amount of movement (three-dimensional vector) of the stylus 30 on the pen coordinate system 60 at the tth time point.

[0043] On the other hand, the virtual coordinate system 70 in the lower part of Fig. 6 is a three-dimensional Cartesian coordinate system consisting of Xv, Yv, and Zv axes, and is defined independently by the wearable device 20. Here, the rectangular area indicated by the dashed line corresponds to the pointing target area 74 set based on the reference position 72. If the pointing position on the virtual coordinate system 70 at the t-th time point is Pv(t), Pv(t) is calculated sequentially according to Pv(t) = Pv(t-1) + A·m↑(t).

[0044] Here, A corresponds to an affine matrix (3 rows x 3 columns) for converting from the pen coordinate system 60 to the virtual coordinate system 70. This affine matrix A can be uniquely determined if the pen coordinate system 60 and the virtual coordinate system 70 are known. For example, the affine matrix A may be stored in advance in association with an object or an object surface, or may be calculated each time the pointing target area 74 is set.

[0045] Then, processor 24 controls display panel 22 to display a mark superimposed on the calculated designated position in currently displayed virtual space 50. By sequentially executing this operation, the locus (i.e., stroke) of a mark such as a dot is superimposed and displayed on virtual space 50.

[0046] In step S8 of Fig. 2, the input system 10 checks whether a pen-up operation has been detected. Specifically, the MCU 33 of the stylus 30 determines whether the pen pressure state has changed from "ON" to "OFF" based on the detection signal of the pen pressure sensor 31. If a pen-up operation has not been detected (step S8: NO), steps S7 and S8 are repeated in sequence until the operation is detected. On the other hand, if a pen-up operation has been detected (step S8: YES), the input system 10 proceeds to the next step S9.

[0047] In step S9, the input system 10 (more specifically, the processor 24) cancels the pointing target area 74 that has been set since the execution of steps S5 and S6. At the same time, the processor 24 displays the virtual space 50 with the viewpoint returned to its original position.

[0048] 7 shows the transition of 3D images 50c and 50d before and after a pen-up operation. More specifically, the upper diagram shows 3D image 50c before the pen-up operation, and the lower diagram shows 3D image 50d after the pen-up operation. For example, when a pen-up operation is performed after one stroke has been completed, 3D image 50c transitions to 3D image 50d. This 3D image 50d corresponds to an image in which one stroke has been written on 3D image 50a in FIG. 5.

[0049] Thereafter, the process returns to step S1, and any of steps S1 to S9 is executed in sequence. By continuing the operation of the input system 10 in this manner, the user U can have a simulated experience of leaving handwriting in the virtual space 50.

[0050] <Effects of basic movements> As described above, the input system 10 in this embodiment includes the stylus 30, the tablet 40 that detects the pointing position of the stylus 30, the display panel 22 that is provided separately from the tablet 40 and displays a space (here, virtual space 50), the gaze sensor 23 that detects the gaze position 52 of the user U in the virtual space 50, and one or more processors (here, processor 24). Then, based on the gaze position 52 detected by the gaze sensor 23 at the time of receiving a predetermined operation by the stylus 30, the processor 24 sets a pointing target area 74 including the reference position 72 of the stylus 30 within the virtual space 50 (steps S5 and S6 in FIG. 2), and controls the display panel 22 to display the pointing position of the stylus 30 within the pointing target area 74, superimposed on the virtual space 50 (step S7 in FIG. 2).

[0051] In this way, the pointing target area 74 including the reference position 72 of the stylus 30 is set in the virtual space 50 based on the detected gaze position 52, so that the pointing position can be displayed in front of the user U's current gaze, that is, in a location closer to the user U's intention to write. This further improves the input operability for the user U in a device configuration in which the tablet 40 that detects the pointing position of the stylus 30 and the display panel 22 that displays the virtual space 50 are provided separately.

[0052] In particular, when the specific operation described above is a "pen-down operation" in which the tip of the stylus 30 is pressed against the touch surface 41, there is no need to perform any special operation in the same way as when performing a normal writing operation, which further improves usability for the user U.

[0053] Furthermore, the pointing target area 74 may be set for each object or each object surface in the virtual space 50. This allows the user U to get the feeling that he or she is selecting an object in the virtual space 50 and leaving a mark on it, thereby increasing the realism of the simulated experience.

[0054] Furthermore, when the previous gaze position 52 and the current gaze position 52 are on the same object or object surface, the processor 24 may set the same reference position 72 and pointing target area 74 as those set previously. This makes it easier to reflect the intention of the user U to continue writing on the same object or object surface.

[0055] Conversely, when the previous gaze position 52 and the current gaze position 52 are on different objects or object surfaces, the processor 24 may set a reference position 72 and a designated area 74 that are different from those set previously. This makes it easier to reflect the user U's intention that the object or object surface of interest has changed.

[0056] Furthermore, when the instruction target area 74 is set, the processor 24 may switch to a viewpoint in which the instruction target area 74 is in front and display the virtual space 50. This makes it easier for the user U to perform a writing operation.

[0057] Alternatively, the stylus 30 or tablet 40 may sequentially output the amount of movement of the stylus 30 from the time a predetermined operation is received, and the processor 24 may convert the amount of movement of the stylus 30 according to a predetermined conversion rule and sequentially add the converted amount of movement to the coordinate values ​​of the reference position 72 to calculate the indicated position in the virtual space 50. By configuring the processor 24 to calculate the indicated position in the virtual space 50, the load of calculation processing by the stylus 30 is reduced.

[0058] [Stylus 30 status correction] <Problems with basic movements> The basic operation described above assumes that the tablet 40 is placed horizontally without moving. However, if the position and orientation of the tablet 40 change over time, writing may not be performed correctly in the virtual space 50.

[0059] 8 is a diagram showing the correspondence between the pen coordinate system 60 and the detection coordinate system 80. The pen coordinate system 60 in the upper part of the drawing is a three-dimensional Cartesian coordinate system consisting of Xp, Yp, and Zp axes, and is uniquely defined by the stylus 30. On the other hand, the detection coordinate system 80 in the lower part of the drawing is a three-dimensional Cartesian coordinate system consisting of Xd, Yd, and Zd axes, and is uniquely defined by the tablet 40. Here, the rectangular area indicated by the dashed line corresponds to the area defined by the touch surface 41.

[0060] When the tablet 40 is placed in an ideal state (for example, horizontally), the Xp-Yp plane is parallel to the Xd-Yd plane. That is, while the user U is writing (pen move operation), the stylus 30 moves horizontally. However, when the tablet 40 is placed at an angle to the horizontal plane, the stylus 30 moves along the inclined touch surface 41 (Xd-Yd plane) while the user U is writing. That is, a deviation from the ideal placement state occurs, and the affine transformation process executed by the wearable device 20 no longer matches the actual behavior. Therefore, the above problem can be solved by correcting the state quantities of the stylus 30 using the state quantities of the tablet 40.

[0061] <Explanation of correction operation> 9 is a schematic diagram of a correction operation that takes into account the state of the tablet 40. Only the main components of the wearable device 20, the stylus 30, and the tablet 40 are shown. Hereinafter, the state quantity of the device itself measured by the stylus 30 using the first IMU 32 will be referred to as the "first state quantity," and the state quantity of the device itself measured by the tablet 40 using the second IMU 46 will be referred to as the "second state quantity."

[0062] The MCU 33 of the stylus 30 sequentially acquires the first state quantity measured by the first IMU 32 in the first period T1. Then, the MCU 33 aggregates this first state quantity and the most recently acquired second state quantity, and then controls the transmission of data including the detection result of the writing pressure by the writing pressure sensor 31 and the aggregated state quantity (hereinafter referred to as "first data") to the wearable device 20. Hereinafter, "aggregation" means obtaining a relative state quantity by subtracting the second state quantity from the first state quantity, or linking the first state quantity and the second state quantity. Hereinafter, communication between the stylus 30 and the wearable device 20 will be referred to as "first communication."

[0063] On the other hand, the MCU 33 of the stylus 30 controls to receive data including the second state quantity (hereinafter referred to as "second data") from the tablet 40. This second state quantity is sequentially measured by the second MCU 46 of the tablet 40 at the second period T2. Hereinafter, the communication between the stylus 30 and the tablet 40 is referred to as "second communication". When transmitting and receiving the second state quantity, for example, a relay communication function implemented according to a communication standard such as Bluetooth (registered trademark) is used.

[0064] By the way, since the stylus 30 on the writing side is frequently moved by the user U, the time change of the state becomes relatively large. On the other hand, since the tablet 40 on the written side is hardly moved by the user U, the time change of the state becomes relatively small. That is, even if the update frequency of the second state quantity is lowered, it is considered that a sufficient correction effect can be obtained. Therefore, by making the execution frequency of the second communication relatively lower than the execution frequency of the first communication, the number of communications can be reduced by that amount, and as a result, power consumption can be saved.

[0065] Specifically, when the first communication and the second communication are performed synchronously, the number of communications per unit time may be set to N:1 (N is an integer of 2 or more). Alternatively, when the first communication and the second communication are performed asynchronously, T1 and T2, which are the execution periods of the first and second communications, may be set respectively so as to satisfy the relationship of T1 < T2.

[0066] The processor 24 of the wearable device 20 calculates the indicated position on the virtual space 50 using the corrected state quantity. When the processor 24 acquires the first data including the relative state quantity, the processor 24 calculates the indicated position using this relative state quantity as it is. On the other hand, when the processor 24 acquires the first data including the pair of the first state quantity and the second state quantity, the processor 24 calculates the indicated position using the relative state quantity obtained by subtracting the second state quantity from the first state quantity.

[0067] As described above, the input system 10 in this embodiment includes the stylus 30, the tablet 40 that detects the position indicated by the stylus 30, and one or more processors 24. The stylus 30 and the tablet 40 are capable of communicating with each other, the stylus 30 is capable of measuring a first state quantity that indicates the state of the device itself or a change in the state over time, and the tablet 40 is capable of measuring a second movement quantity that indicates the state of the device itself or a change in the state over time. The stylus 30 receives second data including the second state quantity from the tablet 40 through the second communication, and transmits first data that aggregates the first state quantity and the second state quantity to the processor 24 through the first communication.

[0068] Furthermore, the aggregation may be calculating a relative state quantity by subtracting the second state quantity from the first state quantity, or linking the first state quantity and the second state quantity. Furthermore, the second communication may be communication using a means different from the first communication (for example, relay communication) that is not performed with the processor 24. Furthermore, the second communication may be performed less frequently than the first communication.

[0069] [Variations] The present invention is not limited to the above-described specific examples. In other words, designs that are appropriately modified by a person skilled in the art from the above-described specific examples are also included within the scope of the present invention as long as they include the features of the present invention. Furthermore, the elements of the above-described embodiments and the modifications described below can be combined to the extent technically possible, and such combinations are also included within the scope of the present invention as long as they include the features of the present invention.

[0070] In the above embodiment, an example has been described in which wearable device 20 displays a virtual reality space using virtual reality (VR) technology, but instead, a space using augmented reality (AR) technology or mixed reality (MR) technology may be displayed. For example, in the case of AR technology, after various information about objects in real space is acquired, the area definition table shown in FIG. 4 may be created in advance.

[0071] In the above-described embodiment, the processor 24 sets the instruction target area 74 in response to a pen-down operation, but the setting may be triggered by another operation, for example, by operation of a side switch of the stylus 30. Also, in the above-described embodiment, the processor 24 performs display control to switch the viewpoint of the virtual space 50 in response to a pen-down operation / pen-up operation, but instead, display control may be performed that does not switch the viewpoint. [Explanation of symbols]

[0072] 10...input system, 20...wearable device, 22...display panel (display unit), 23...gaze sensor (position detection unit), 24...processor, 30...stylus (pointing device), 40...tablet (detection device), 50...virtual space (space), 52...gaze position, 72...reference position, 74...pointing target area

Claims

1. A wearable device operable using a pointing device and a detecting device, a processor; a memory storing instructions that, when executed by the processor, cause a pointing position to be calculated based on data transmitted from the pointing device; the data includes a pair of a first state quantity indicating a state of the indicating device or a change in the state over time, and a second state quantity indicating a state of the detecting device or a change in the state over time; Wearable device.

2. the instruction, when executed by the processor, calculates a relative state quantity from the first state quantity and the second state quantity included in the data, and calculates the pointed position using the relative state quantity. The wearable device of claim 1 .

3. The instructions, when executed by the processor, converting the relative state quantity in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding the converted relative state quantities; The wearable device of claim 2 .

4. The instructions, when executed by the processor, calculating a plurality of relative state quantities from a plurality of first state quantities indicating a plurality of states of the indicating device or time changes in the states and a plurality of second state quantities indicating a plurality of states of the detecting device or time changes in the states; converting each of the relative state quantities in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding up the converted relative state quantities; The wearable device of claim 1 .

5. the predetermined transformation rule is a rule that uses an affine matrix, The wearable device of claim 3 .

6. The affine matrix is ​​stored in association with an object or object surface. The wearable device of claim 5 .

7. The affine matrix is ​​calculated each time a target region is set. The wearable device of claim 5 .

8. A wearable device operable using a pointing device and a detecting device, a processor; a memory storing instructions that, when executed by the processor, cause a pointing position to be calculated based on data transmitted from the pointing device; the data includes a relative state quantity obtained based on a first state quantity and a second state quantity; Wearable device.

9. The instructions, when executed by the processor, converting the relative state quantity in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding the converted relative state quantities; The wearable device of claim 8 .

10. the predetermined transformation rule is a rule that uses an affine matrix, The wearable device of claim 9 .

11. an indicating device; a detection device that detects a pointing position of the pointing device during operation; a wearable device provided separately from the detection device, The wearable device is a display unit for displaying an image of the virtual space during operation; a processor; a memory storing therein instructions that, when executed by the processor, cause the instruction to calculate the indicated position based on a first state quantity indicating a state of the indicating device or a change in the state over time and a second state quantity indicating a state of the detecting device or a change in the state over time; Input system.

12. the indicating device, during operation, transmits a pair of the first state quantity and the second state quantity; The input system of claim 11.

13. The instructions, when executed by the processor, converting a relative state quantity calculated from the first state quantity and the second state quantity in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding the converted relative state quantities; 13. The input system of claim 12.

14. the instruction device transmits a relative state quantity calculated from the first state quantity and the second state quantity during operation. The input system of claim 11.

15. The instructions, when executed by the processor, converting the relative state quantity in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding the converted relative state quantities; 15. The input system of claim 14.

16. The instructions, when executed by the processor, calculating a plurality of relative state quantities from a plurality of first state quantities indicating a plurality of states of the indicating device or time changes in the states and a plurality of second state quantities indicating a plurality of states of the detecting device or time changes in the states; converting each of the relative state quantities in accordance with a predetermined conversion rule; calculating the pointed position in the virtual space by sequentially adding up the converted relative state quantities; The input system of claim 11.

17. the predetermined transformation rule uses an affine matrix, 14. The input system of claim 13.

18. The affine matrix is ​​stored in association with an object or object surface.

18. The input system of claim 17.

19. The affine matrix is ​​calculated each time a target region is set.

18. The input system of claim 17.

Images