Input system and input method

Abstract

The present application provides an input system and an input method, in a device configuration in which a detection device that detects a pointing position of a pointing device and a display unit of a display space are provided separately, the input operability of a user can be improved. A processor (24) of an input system (10) sets a pointing target region (74) that contains a reference position (72) of a pointing device (30) from among a space (50) based on a line-of-sight position (52) detected by a position detection section (23) at a time point at which a prescribed operation of the pointing device (30) is accepted, and controls a display unit (22) in a manner in which the pointing position of the pointing device (30) is displayed within the pointing target region (74) on the space (50) in superposition.

Description

Technical Field

[0001] This invention relates to an input system and an input method. Background Technology

[0002] Previously, digital ink systems were known to input ink data describing a collection of strokes through writing operations using a pointer device including a stylus. For example, it is envisioned that the written content is displayed in a different area than the writing area, following the user's writing operation.

[0003] Patent document 1 discloses the following technology: after setting a two-dimensional virtual plane in three-dimensional space, the strokes written on the virtual plane are displayed instantly on a display device that the user can wear.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2013-125487 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, by displaying a virtual space containing various objects, users can obtain a simulated experience as if they were leaving writing in that space. However, in the technology disclosed in Patent Document 1, since the virtual plane is fixed directly after being set, the writing content may be displayed in a location different from the user's intention, depending on the situation, which may impair the user's input operability.

[0009] The present invention was made in view of the above-mentioned problems, and its object is to provide an input system and input method that improves the user's input operability in a device configuration in which the detection device for detecting the indicated position of the indicator device and the display unit for display space are separately arranged.

[0010] Technical solutions for solving the problem

[0011] The input system of the first solution includes: an indicating device; a detection device for detecting the indicating position of the indicating device; a display unit, which is separately disposed from the detection device and displays a space; a position detection unit for detecting the user's line of sight position within the space; and a processor, which, based on the line of sight position detected by the position detection unit at a predetermined time point when the indicating device has performed a specified operation, sets an indicating target area containing a reference position of the indicating device from the space, and controls the display unit to display the indicating position of the indicating device overlapping in the space within the indicating target area.

[0012] The second approach uses an input system comprising: an indicating device; a detection device for detecting the indicating position of the indicating device; a display unit separately disposed from the detection device and displaying a space; and a position detection unit for detecting the user's gaze position within the space. One or more processors perform the following steps: based on the gaze position detected by the position detection unit at a predetermined time point when the indicating device has performed a specified operation, setting an indicating target area within the space that includes a reference position of the indicating device; and controlling the display unit to display the indicating position of the indicating device overlapping within the indicating target area in the space.

[0013] Invention Effects

[0014] According to the present invention, in the configuration of the device for detecting the indicated position of the detection indicator and the display unit of the display space are separately arranged, the user's input operability is further improved. Attached Figure Description

[0015] Figure 1 This is an overall structural diagram of the input system according to one embodiment of the present invention.

[0016] Figure 2 It is about Figure 1 The flowchart shown is a flowchart of the input actions of the input system.

[0017] Figure 3 This is a diagram illustrating an example of a virtual space displayed on a wearable device.

[0018] Figure 4 This is a diagram illustrating an example of the data structure of a region definition table.

[0019] Figure 5 It is a diagram showing the transformation of the 3D image before and after the pen stroke.

[0020] Figure 6 It is a diagram showing the correspondence between the pen coordinate system and the virtual coordinate system.

[0021] Figure 7 It is a diagram showing the transformation of the 3D image before and after the pen-lifting operation.

[0022] Figure 8 It is a diagram showing the correspondence between the pen coordinate system and the detection coordinate system.

[0023] Figure 9 This is a schematic diagram regarding the correction actions that take into account the state variables of the input board.

[0024] Label Explanation

[0025] 10…Input system, 20…Wearable device, 22…Display panel (display unit), 23…Looking sensor (position detection unit), 24…Processor, 30…Stylus pen (indicating device), 40…Input pad (detection device), 50…Virtual space (space), 52…Looking position, 72…Reference position, 74…Indicating object area. Detailed Implementation

[0026] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. For ease of understanding, the same reference numerals will be used as much as possible to denote the same constituent elements and steps in the drawings, and repeated descriptions will sometimes be omitted.

[0027] [Structure of Input System 10]

[0028] Figure 1 This is an overall structural diagram of an input system 10 according to one embodiment of the present invention. The input system 10 is a "digital ink system" capable of inputting ink data describing a collection of strokes through writing operations using an indicator device. Specifically, the input system 10 is configured to include a wearable device 20 as a display device, a stylus 30 as an indicator device, and an input pad 40 as a detection device.

[0029] Wearable device 20 is a portable display device that can be worn by user U. Specifically, wearable device 20 is configured to include housing 21, display panel 22 (equivalent to "display unit"), gaze sensor 23 (equivalent to "position detection unit"), processor 24 (equivalent to "processor"), and communication module 25.

[0030] The housing 21 includes a frame for holding the various electronic components and a fixing member for securing the frame to the head of the user U. The display panel 22 is capable of displaying images or videos, and may be, for example, a liquid crystal panel, an organic EL (Electro-Luminescence) panel, electronic paper, etc. The gaze sensor 23, for example, is composed of an infrared sensor, which detects physical quantities related to the user U's gaze (e.g., eye movement).

[0031] The processor 24 comprises a processing and computing device including a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). The processor 24 reads and executes programs stored in a memory (not shown) to perform image display control, various calculations required for this display control, and data transmission and reception control.

[0032] The communication module 25 is a communication interface for receiving and transmitting signals to external devices. Thus, the wearable device 20 can exchange various types of data with the stylus 30 or the input pad 40.

[0033] The stylus 30 is an electronic pen configured to communicate with the input panel 40 in one or both directions. The electronic pen can be either electromagnetic induction (EMR) or active electrostatic coupling (AES). For example, in the AES case, the stylus 30 is configured to include a pen pressure sensor 31, an inertial measurement unit (hereinafter referred to as the first IMU 32), a micro control unit (hereinafter referred to as the MCU 33), and a communication chip 34.

[0034] The pen pressure sensor 31 is, for example, a pressure sensor that uses a variable capacitor to capture changes in electrostatic capacitance caused by pressing the pen tip. This pen pressure sensor 31 can detect not only pen pressure but also pen events including the pressing or lifting of the stylus 30.

[0035] The first IMU 32 is, for example, a measurement unit consisting of a 3-axis gyroscope sensor and a 3-direction accelerometer. Thus, the first IMU 32 is configured to measure the pen coordinate system 60 (see reference 60) described later. Figure 6 and Figure 8 The state quantity on the device represents the state of the device or the time-varying state. This state quantity includes various physical quantities used to determine position / attitude, such as position, velocity, acceleration, jerk, angle, angular velocity, etc.

[0036] MCU33 is a control unit that includes a processor capable of controlling the movements of stylus 30. For example, in addition to various calculations related to determining the indicated position of stylus 30, MCU33 also performs data transmission and reception control, including the results of these calculations.

[0037] The communication chip 34 is an integrated circuit used for wireless communication with external devices according to various wireless communication standards, including Bluetooth (registered trademark). Thus, the stylus 30 can exchange various types of data with the wearable device 20 or the input pad 40 via the communication chip 34.

[0038] The input pad 40 is a device capable of detecting the indicated position of the stylus 30, regardless of whether a display function is present. Alternatively, various detection devices, including smartphones and personal computers, can be used instead of the input pad 40. The user U holds the stylus 30 with one hand and moves it while pressing the pen tip against the touch surface 41 of the input pad 40, thereby writing pictures or text onto the display surface of the wearable device 20.

[0039] The input board 40 is configured to include sensor electrodes 42, a touch IC (Integrated Circuit) 43, a CPU 44, a communication chip 45, and a second IMU 46. The second IMU 46 can be omitted if necessary.

[0040] Sensor electrode 42 is an assembly of electrodes capable of detecting changes in electrostatic capacitance caused by the proximity or contact of a conductor. The electrostatic capacitance can be detected using either mutual capacitance or self-capacitance. For example, in the case of mutual capacitance, the sensor electrode 42 includes a detection coordinate system 80 (see reference 80). Figure 8 Multiple X-ray electrodes for detecting the position of the Xd-axis and multiple Y-ray electrodes for detecting the position of the Yd-axis.

[0041] Touch IC 43 is an integrated circuit that drives and controls the sensor electrode 42. Touch IC 43 drives the sensor electrode 42 based on control signals supplied from CPU 44. Thus, Touch IC 43 performs a "pen detection function" to detect the state of the stylus 30 and a "touch detection function" to detect touches from the user U's finger, etc.

[0042] CPU44 can perform various functions, including ink data generation and data transmission / reception control, by reading and executing programs stored in memory (not shown). Alternatively, various processing units, including MPUs and GPUs, can be used instead of CPU44.

[0043] Similar to communication chip 34, communication chip 45 is an integrated circuit used for wireless communication with external devices according to various communication standards, including Bluetooth (registered trademark). Thus, input board 40 can exchange various types of data with wearable device 20 or stylus 30 via communication chip 45.

[0044] Similar to the first IMU 32, the second IMU 46 is, for example, a measurement unit consisting of a 3-axis gyroscope sensor and a 3-direction accelerometer sensor. Thus, the second IMU 46 is configured to measure the detection coordinate system 80 (see reference 80) described later. Figure 8 The state variables on the device that are related to its position / or orientation.

[0045] [Input System 10 Actions]

[0046] The input system 10 of this embodiment is configured as described above. Next, referring to... Figure 2 Flowchart and Figures 3-7 The input actions of the input system 10 are described.

[0047] <Basic Movements>

[0048] exist Figure 2 In step S1, the input system 10 displays the virtual space 50 in a manner that the user U can visually confirm. Specifically, the processor 24 of the wearable device 20 controls the display panel 22 to display an image or video representing the virtual space 50.

[0049] Figure 3 This diagram illustrates an example of a virtual space 50 displayed on a wearable device 20. The virtual space 50 represents a three-dimensional, elevated interior. Within this interior, various objects are arranged, including structures such as ceilings, floors, walls, and closets, as well as furniture such as beds, tables, and chairs.

[0050] exist Figure 2 In step S2, the input system 10 confirms whether the stylus 30 has accepted the pen stroke operation. Specifically, the MCU 33 of the stylus 30 determines whether the pen pressure state has changed from "OFF" to "ON" based on the detection signal from the pen pressure sensor 31. If no pen stroke operation is detected (step S2: No), the system remains in step S1 until the operation is detected. On the other hand, if a pen stroke operation is detected (step S2: Yes), the system proceeds to the next step S3. Furthermore, the wearable device 20 can directly obtain the detection result of the stylus 30, or indirectly obtain the detection result of the stylus 30 via the input board 40.

[0051] In step S3, the input system 10 determines the object that user U was visually confirming at the time of the pen stroke detected in step S2 (hereinafter referred to as the visual confirmation object). First, after the processor 24 of the wearable device 20 detects the user U's gaze position 52 based on the detection signal of the gaze sensor 23, it determines the object located in the virtual space 50 in front of that gaze. When determining the visual confirmation object, for example, the processor 24 may also refer to... Figure 4 The area definition table shown.

[0052] Figure 4 It means and Figure 3 The diagram shows an example of the data structure of the region definition table related to the virtual space 50. The region definition table is a tabular data representing the correspondence between [1] the “object ID” as identification information of an object located in the virtual space 50, [2] the “occupation range” occupied by the object, [3] the “base vector” of the coordinate system formed by the object’s surfaces, and [4] the “reference coordinates” representing the origin of the coordinate system.

[0053] First, processor 24 detects the user U's gaze position 52 based on the detection signal from gaze sensor 23, and calculates the gaze vector on virtual space 50. Then, processor 24 refers to... Figure 4The shown area definition table determines whether the gaze vector intersects with the corresponding occupancy range of each object. If a corresponding object exists, that object is identified as a "visual confirmation object." Furthermore, in... Figure 4 In the example, an ID is assigned to each object, but it is also possible to assign an ID to each face of an object with multiple faces.

[0054] exist Figure 2 In step S4, the input system 10 (more specifically, the processor 24) determines whether the visual confirmation object determined in step S3 is the same as the previously determined visual confirmation object. If it is the same visual confirmation object (step S4: Yes), proceed to step S5; otherwise, if it is a different visual confirmation object (step S4: No), proceed to step S6.

[0055] Upon proceeding to step S5, the input system 10 (more specifically, the processor 24) sets the visual confirmation object determined in step S3, i.e., the indicator object area 74 corresponding to the current line of sight. Figure 6 Specifically, processor 24 references... Figure 4 The region definition table sets the virtual coordinate system to 70 ( Figure 6 The planar indicator area 74 in the ) . For example, in the case of visually confirming an object's ID = OB0001, the processor 24 will assign a reference position 72 with coordinates (Xv1, Yv1, Zv1) to the reference position 72. Figure 6 The point is set as the "center point" of the target area 74, the two basis vectors (Ex1↑, Ey1↑) are set as the "plane vectors" of the target area 74, and the remaining one basis vector Ez1↑ is set as the "normal vector" of the target area 74.

[0056] Upon entering step S6, the input system 10 (more specifically, the processor 24) sets the target area 74 to be the same as in step S5 or S6 of the previous flowchart execution, i.e., the previous situation. At the same time, the processor 24 displays a virtual space 50 with the viewpoint switched so that the target area 74 is facing forward.

[0057] Figure 5 This diagram illustrates the transformation of 3D images 50a and 50b before and after a pen stroke. More specifically, the upper diagram represents 3D image 50a before the pen stroke, and the lower diagram represents 3D image 50b after the pen stroke. For example, when a pen stroke is performed with the user U's line of sight 52 facing the closet door, 3D image 50a transforms into 3D image 50b. This 3D image 50b is equivalent to a two-dimensional image with the closet door (i.e., the indicated object area 74) facing forward.

[0058] exist Figure 2 In step S7, the input system 10 follows the pen movement of the user U and performs drawing processing on the virtual space 50. First, after setting the position of the device at the time of the pen stroke to the origin Op of the pen coordinate system 60, the stylus 30 sequentially acquires the state quantities measured by the first IMU 32. Then, the wearable device 20 directly acquires the data containing the state quantities from the stylus 30. Alternatively, the wearable device 20 can also indirectly acquire the data from the stylus 30 via the input panel 40.

[0059] Then, the processor 24 converts the state variables of the stylus 30 according to the prescribed conversion rules, and adds the converted state variables sequentially to calculate the indicated position on the virtual space 50. See below for reference. Figure 6 An example is given of using a time series of movement to calculate the indicated position on virtual coordinate system 70.

[0060] Figure 6 The upper pen coordinate system 60 is a three-dimensional orthogonal coordinate system composed of the Xp axis, Yp axis, and Zp axis, which is uniquely defined by the stylus 30. For example, when the pen position 62 of the stylus 30 is the origin Op, m↑(t) represents the amount of movement (three-dimensional vector) of the stylus 30 on the pen coordinate system 60 at the t-th time point.

[0061] on the other hand, Figure 6 The virtual coordinate system 70 on the lower side is a three-dimensional orthogonal coordinate system composed of the Xv axis, Yv axis, and Zv axis, which is uniquely defined by the wearable device 20. Here, the quadrilateral area represented by the dashed line corresponds to the indicator area 74 set with the reference position 72 as the reference. When the indicator position on the virtual coordinate system 70 at the t-th time point is Pv(t), Pv(t) is calculated sequentially according to Pv(t) = Pv(t-1) + A·m↑(t).

[0062] Here, A is equivalent to an affine matrix (3 rows × 3 columns) used for transforming from the pen coordinate system 60 to the virtual coordinate system 70. If the pen coordinate system 60 and the virtual coordinate system 70 are known, then this affine matrix A is uniquely determined. For example, the affine matrix A can be pre-stored with a correspondence established with an object or object surface, or it can be calculated each time the indicated object area 74 is set.

[0063] Then, the processor 24 controls the display panel 22 to overlay markers at calculated indicated positions for the currently displayed virtual space 50. By performing this action sequentially, the trajectory (i.e., strokes) of markers such as dots are overlaid on the virtual space 50.

[0064] exist Figure 2In step S8, the input system 10 confirms whether a pen lift operation has been detected. Specifically, the MCU 33 of the stylus 30 determines whether the pen pressure state has changed from "ON" to "OFF" based on the detection signal from the pen pressure sensor 31. If no pen lift operation is detected (step S8: No), steps S7 and S8 are repeated sequentially until the operation is detected. On the other hand, if a pen lift operation is detected (step S8: Yes), the process proceeds to the next step, S9.

[0065] In step S9, the input system 10 (more specifically, the processor 24) removes the indicated object area 74 set during steps S5 and S6. Simultaneously, the processor 24 displays a virtual space 50 that restores the viewpoint.

[0066] Figure 7 This diagram illustrates the transformation of 3D images 50c and 50d before and after a pen-lifting operation. More specifically, the upper image represents 3D image 50c before the pen-lifting operation, and the lower image represents 3D image 50d after the pen-lifting operation. For example, when a pen-lifting operation is performed after one stroke, 3D image 50c transforms into 3D image 50d. This 3D image 50d is equivalent to... Figure 5 The 3D image 50a contains an image with one stroke written on it.

[0067] Then, return to step S1, and proceed with any one of steps S1 through S9 in sequence. In this way, by continuously inputting actions into system 10, user U can obtain a simulated experience as if leaving writing in virtual space 50.

[0068] <Effects of basic movements>

[0069] As described above, the input system 10 of this embodiment is configured to include: a stylus 30; an input panel 40 for detecting the indicated position of the stylus 30; a display panel 22, which is separately disposed from the input panel 40 and displays a space (here, a virtual space 50); a gaze sensor 23 for detecting the gaze position 52 of the user U within the virtual space 50; and one or more processors (here, processor 24). Furthermore, the processor 24, based on the gaze position 52 detected by the gaze sensor 23 at the time when a predetermined operation of the stylus 30 is received, sets an indicated object area 74 from the virtual space 50 that includes the reference position 72 of the stylus 30. Figure 2 Steps S5 and S6), and control the display panel 22 in such a way that the indicated position of the stylus 30 is displayed superimposed on the virtual space 50 within the indicated object area 74. Figure 2 Step S7).

[0070] In this way, based on the detected gaze position 52, an indication area 74 including the reference position 72 of the stylus 30 is set in the virtual space 50. Therefore, the indication position can be displayed in front of the user U's gaze at the current time, that is, closer to the user U's intention to write. As a result, in the device configuration where the input panel 40 for detecting the indication position of the stylus 30 and the display panel 22 for displaying the virtual space 50 are separately set, the user U's input operability is further improved.

[0071] In particular, when the aforementioned operation is a "pen-pressing operation" in which the tip of the stylus 30 is pressed against the touch surface 41, the ease of use is further improved for the user U because no special operation is required to feel like performing a normal writing operation.

[0072] Additionally, the target area 74 can be set for each object or each object surface located within the virtual space 50. Thus, the user U can experience the feeling of selecting an object within the virtual space 50 and leaving a writing mark, thereby increasing the realism of the simulation experience.

[0073] Furthermore, the processor 24 can also set the same reference position 72 and the indicated object area 74 as before, if the previous viewing position 52 and the current viewing position 52 are on the same object or object surface. This easily reflects the user U's intention to continue writing on the same object or object surface.

[0074] Conversely, if the previous viewing position 52 and the current viewing position 52 are located on different objects or on different object surfaces, the processor 24 can also set a different reference position 72 and an indicated object area 74 than the previous setting. This makes it easier to reflect the user U's intention to indicate that the object or object surface being viewed has been changed.

[0075] Furthermore, when the target area 74 is set, the processor 24 can also switch the target area 74 to become the front viewpoint and display the virtual space 50. This makes it easier for the user U to perform writing operations.

[0076] Alternatively, the stylus 30 or input pad 40 can sequentially output the movement amount of the stylus 30 starting from the time point when the specified operation is received. The processor 24 converts the movement amount of the stylus 30 according to a specified conversion rule and sequentially adds the converted movement amount to the coordinate value of the reference position 72, thereby calculating the indicated position on the virtual space 50. By employing a structure where the processor 24 calculates the indicated position on the virtual space 50, the computational processing load of the stylus 30 can be reduced.

[0077] [Stylus 30 status correction]

[0078] <Problems with basic movements>

[0079] In the basic actions described above, we assume that the input panel 40 remains stationary and is configured in a horizontal position. However, when the position / pose of the input panel 40 changes over time, it may sometimes fail to write correctly in the virtual space 50.

[0080] Figure 8 This diagram illustrates the correspondence between the pen coordinate system 60 and the detection coordinate system 80. The pen coordinate system 60 on the upper side of the diagram is a three-dimensional orthogonal coordinate system composed of the Xp, Yp, and Zp axes, uniquely defined by the stylus 30. Conversely, the detection coordinate system 80 on the lower side of the diagram is a three-dimensional orthogonal coordinate system composed of the Xd, Yd, and Zd axes, uniquely defined by the input panel 40. Here, the quadrilateral area indicated by the dashed lines corresponds to the area formed by the touch surface 41.

[0081] When the input pad 40 is configured in an ideal state (e.g., horizontal state), the Xp-Yp plane is parallel to the Xd-Yd plane. That is, during writing (pen movement operation) by the user U, the stylus 30 moves in the horizontal direction. However, when the input pad 40 is configured at an angle to the horizontal plane, during writing by the user U, the stylus 30 moves along the angled touch surface 41 (Xd-Yd plane). That is, a deviation from the ideal configuration occurs, and the affine transformation processing performed by the wearable device 20 does not match the actual action. Therefore, by correcting the state of the stylus 30 using the state of the input pad 40, the above-mentioned problem can be solved.

[0082] <Explanation of Correction Actions>

[0083] Figure 9 This is a schematic diagram regarding the correction action taking into account the state of the input board 40. Only the main structures in the wearable device 20, stylus 30, and input board 40 are marked. Hereinafter, the state quantity of this device measured by the stylus 30 using the first IMU 32 will be referred to as the "first state quantity", and the state quantity of this device measured by the input board 40 using the second IMU 46 will be referred to as the "second state quantity".

[0084] The MCU 33 of the stylus 30 sequentially acquires the first state quantity measured by the first IMU 32 in the first period T1. Then, the MCU 33 aggregates the first state quantity and the most recently acquired second state quantity, and performs control to transmit data including the pen pressure detection result of the pen pressure sensor 31 and the aggregated state quantity (hereinafter referred to as "first data") to the wearable device 20. Here, the term "aggregate" refers to obtaining a relative state quantity obtained by subtracting the second state quantity from the first state quantity, or concatenating the first state quantity and the second state quantity. Hereinafter, the communication between the stylus 30 and the wearable device 20 is referred to as "first communication".

[0085] On the other hand, the MCU 33 of the stylus 30 performs control to receive data including the second state quantity (hereinafter referred to as "second data") from the tablet 40. The second state quantity is sequentially measured by the second MCU 46 of the tablet 40 in the second period T2. Hereinafter, the communication between the stylus 30 and the tablet 40 is referred to as "second communication". When transmitting and receiving the second state quantity, for example, a relay communication function implemented according to a communication standard such as Bluetooth (registered trademark) is used.

[0086] However, since the writing-side stylus 30 is frequently moved by the user U, the temporal change of its state is relatively large. In contrast, the writing-receiving-side tablet 40 is hardly moved by the user U, so the temporal change of its state is relatively small. That is, it is considered that a sufficient correction effect can be obtained even if the update frequency of the second state quantity is reduced. Therefore, by making the execution frequency of the second communication relatively lower than the execution frequency of the first communication, the number of communication times can be reduced accordingly, and as a result, power consumption can be saved.

[0087] Specifically, in the case where the first communication and the second communication are performed synchronously, the number of communication times per unit time may be set to N:1 (N is an integer of 2 or more). Alternatively, in the case where the first communication and the second communication are performed asynchronously, the execution periods of the first and second communications, that is, T1 and T2, may be respectively set so as to satisfy the relationship of T1<T2.

[0088] The processor 24 of the wearable device 20 calculates the indicated position on the virtual space 50 using the corrected state quantity. When the processor 24 acquires the first data including the relative state quantity, it directly uses the relative state quantity to calculate the indicated position. On the other hand, when the processor 24 acquires the first data including the pair of the first state quantity and the second state quantity, it calculates the indicated position using the relative state quantity obtained by subtracting the second state quantity from the first state quantity.

[0089] As described above, the input system 10 of this embodiment is configured to include: a stylus 30; an input pad 40 for detecting the indicated position of the stylus 30; and one or more processors 24. The stylus 30 and the input pad 40 are capable of communicating with each other. The stylus 30 is capable of measuring a first state quantity representing the state of the device or a time-varying change in state, and the input pad 40 is capable of measuring a second movement quantity representing the state of the device or a time-varying change in state. Furthermore, the stylus 30 receives second data including the second state quantity from the input pad 40 via a second communication, and sends the first data summarizing the first and second state quantities to the processor 24 via a first communication.

[0090] Additionally, summarization can be performed by subtracting the second state value from the first state value to calculate the relative state value, or by concatenating the first and second state values. Furthermore, the second communication can also be communication that is not conducted with the processor 24 and uses a different method than the first communication (e.g., relay communication). Additionally, the execution frequency of the second communication can be lower than the execution frequency of the first communication.

[0091] [Variation Example]

[0092] Furthermore, the present invention is not limited to the specific examples described above. That is, in the specific examples described above, those skilled in the art can appropriately make design changes, and as long as they possess the features of the present invention, they are included within the scope of the present invention. In addition, the elements of the above-described embodiments and the modified examples described below can be combined as long as they are technically feasible, and the combination of these elements, as long as it contains the features of the present invention, is also included within the scope of the present invention.

[0093] In the above embodiments, the example described is a wearable device 20 displaying a virtual reality (VR) space using virtual reality (VR) technology. Alternatively, it can also display a space using augmented reality (AR) or composite reality (MR) technology. For example, in the case of AR technology, after obtaining various information related to objects located in real space, a virtual reality space can be pre-created. Figure 4 The area definition table shown.

[0094] In the above embodiment, the processor 24 sets the target area 74 based on a pen-dropping operation, but it can also be based on other operations, such as the operation of the side switch of the stylus 30. Furthermore, in the above embodiment, the processor 24 switches the display control of the virtual space 50's viewpoint based on a pen-dropping / pen-lifting operation; alternatively, it can also perform display control without switching the viewpoint.

Claims

1. An input system, comprising: Indicating device; The detection device detects the indicated position of the indicating device; The display unit is separately configured from the detection device, and has a display space; The position detection unit detects the user's line-of-sight position within the space; and processor, The processor, based on the gaze position detected by the position detection unit at the specified time point when the instruction device performs a specified operation, sets an instruction target area in the space that includes the reference position of the instruction device, and controls the display unit to display the instruction position of the instruction device overlapping in the space within the instruction target area. The indicating device and the detection device are able to communicate with each other. The indicating device is capable of measuring a first state quantity that represents the state of the device or a time-varying change in state. The detection device is capable of measuring a second state quantity that represents the state of the device or the time-varying changes in the state. The indicating device receives second data containing the second state quantity from the detection device via a second communication, and sends first data summarizing the first state quantity and the second state quantity to the processor via a first communication.

2. The input system according to claim 1, wherein, The indicated object area can be set for each object or each object surface located within the space.

3. The input system according to claim 2, wherein, If the previous line of sight and the current line of sight are on the same object or on the same surface, the processor sets the reference position and the indicated object area to be the same as the previous setting.

4. The input system according to claim 2, wherein, If the processor sets a reference position and the indicated object area that are different from the previous setting when the previous viewing position is on a different object or surface than the current viewing position, the processor will set a reference position and an indicated object area that are different from the previous setting.

5. The input system according to any one of claims 1 to 4, wherein, The space in question is a virtual reality space. When the indicated object area is set, the processor switches to the indicated object area as the frontal viewpoint to display the virtual reality space.

6. The input system according to any one of claims 1 to 4, wherein, The indicating device or the detection device sequentially outputs the movement amount of the indicating device from the time point when the specified operation is accepted. The processor converts the movement of the indicating device according to a prescribed conversion rule, and adds the converted movement to the coordinate value of the reference position in sequence, thereby calculating the indicating position in space.

7. The input system according to claim 1, wherein, The summarization refers to calculating the relative state quantity by subtracting the second state quantity from the first state quantity, or by concatenating the first state quantity and the second state quantity.

8. The input system according to claim 7, wherein, The second communication is a communication that is not conducted with the processor and uses a different method than the first communication.

9. The input system according to claim 8, wherein, The second type of communication is relay communication.

10. The input system according to claim 9, wherein, The execution frequency of the second communication is lower than that of the first communication.

11. An input method that uses an input system. The input system is configured to include: Indicating device; The detection device detects the indicated position of the indicating device; The display unit is separately configured from the detection device, and has a display space; and The position detection unit detects the user's line-of-sight position within the space. in, One or more processors perform the following steps: The steps are: first, defining an indication target area encompassing the reference position of the indication device from the space based on the line-of-sight position detected by the position detection unit at the specified time point when the indication device is operated; and second, defining an indication target area from the space including the reference position of the indication device. The step of controlling the display unit in a manner that overlaps the indication position of the indication device in the indicated object area in the space. The indicating device and the detection device are able to communicate with each other. The indicating device is capable of measuring a first state quantity that represents the state of the device or a time-varying change in state. The detection device is capable of measuring a second state quantity that represents the state of the device or the time-varying changes in the state. The indicating device receives second data containing the second state quantity from the detection device via a second communication, and sends first data summarizing the first state quantity and the second state quantity to the processor via a first communication.

Images