Instruction input processing system and input processing method

Abstract

An electronic pen is used to input positional information in a VR (Virtual Reality) space. [Solution] The device includes a digitizer 20 for detecting the position indicated by an electronic pen 10, a tracking device for detecting the position of a tracker of the electronic pen 10 and for performing position input by the electronic pen 10 in a VR (Virtual Reality) space set to a first coordinate system associated with the tracking device, and a computer that, when the device with the digitizer detects the position indicated by the electronic pen 10, converts the coordinates in the first coordinate system corresponding to the position indicated by the electronic pen 10 in the three-dimensional space where the electronic pen 10 exists to coordinates in a second coordinate system associated with the device with the digitizer that corresponds to the position indicated by the electronic pen 10 detected by the device with the digitizer.

Description

Technical Field

[0001] This invention relates to an instruction input processing system and an instruction input processing method suitable for using an electronic pen in space.

Background Art

[0002] There is known a drawing system for creating an animation image or the like by continuously indicating a position with an electronic pen for a coordinate input device called a digitizer and drawing.

[0003] Conventionally, various methods have been proposed as methods for detecting the inclination and rotation of an electronic pen. Patent Document 1 (Japanese Patent Application Laid-Open No. 2016-126503) provides an electronic pen capable of detecting an inclination angle and a rotation angle with respect to an input surface of a sensor unit of a position detection device. In the case of this Patent Document 1, between an electronic pen and a position detection device including a sensor unit, by transmitting and receiving signals by electromagnetic induction coupling, electrostatic capacitance coupling, or the like, the position detection device detects the position indicated by the electronic pen, and also detects the position coordinates of the indicated position of the electronic pen, as well as the inclination angle and the rotation angle.

[0004] Further, Patent Document 2 (U.S. Patent No. 9,329,703 B2) describes that a sensor capable of measuring the movement and orientation of an electronic pen is built in or attached to the electronic pen, and by transmitting the detection output of the sensor to a position detection device, the movement and state (inclination and rotation) of the electronic pen can be detected by the position detection device.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the case of Patent Document 1 mentioned above, the position coordinates and tilt of the electronic pen's point of reference are detected using multiple signal levels obtained from multiple loop coils surrounding the point of reference indicated by the electronic pen. However, in the area surrounding the sensor, there was a problem in that the accuracy of the position coordinates of the electronic pen's point of reference decreased, and it became difficult to detect the tilt of the electronic pen.

[0007] According to the method described in Patent Document 2, the problems described in Patent Document 1 do not occur. However, under certain conditions, there are problems such as a slow reaction rate or a decrease in the accuracy of the position coordinates.

[0008] The purpose of this invention is to provide an input system that can solve the above-mentioned problems. [Means for solving the problem]

[0009] To solve the above problems, A device having a digitizer that detects the position indicated by an electronic pen, A tracking device for detecting the position of the tracker of the electronic pen, and a tracking device for performing position input by the electronic pen in a VR (Virtual Reality) space set in a first coordinate system associated with the tracking device, When the device having the digitizer detects the position indicated by the electronic pen, a computer converts the coordinates in the first coordinate system corresponding to the position indicated by the electronic pen in the three-dimensional space where the electronic pen being tracked by the tracking device exists, to coordinates in a second coordinate system associated with the device having the digitizer, which corresponds to the position indicated by the electronic pen detected by the device having the digitizer, The present invention provides an instruction input processing system.

[0010] In the instruction input processing system with the above configuration, when the position indicated by the electronic pen is detected by a device having a digitizer, the coordinates in the first coordinate system corresponding to the position indicated by the electronic pen in the three-dimensional space where the electronic pen being tracked by the tracking device exists are converted to coordinates in the second coordinate system associated with the device having a digitizer, which corresponds to the position indicated by the electronic pen detected by the device having a digitizer. [Brief explanation of the drawing]

[0011] [Figure 1] This figure illustrates an example configuration of a spatial position indication system including a first embodiment of the input device according to this invention. [Figure 2] This is a block diagram illustrating an example of the configuration of each part of the spatial position indication system shown in Figure 1. [Figure 3] This is a diagram illustrating an embodiment of the input device according to this invention. [Figure 4] This is a block diagram illustrating an example of the configuration of each part of a second embodiment of the input device according to this invention. [Figure 5] This diagram illustrates the operation of detecting the tilt of an electronic pen through the exchange of signals between the electronic pen and a digitizer. [Figure 6] This diagram is used to explain that the detection accuracy of the digitizer when it detects the position and tilt of the electronic pen through signal exchange with the electronic pen differs between the peripheral area of ​​the input surface and other areas. [Figure 7] This figure shows a portion of a flowchart illustrating the main operation of the second embodiment of the input device according to this invention. [Figure 8] This figure shows a portion of a flowchart illustrating the main operation of the second embodiment of the input device according to this invention. [Figure 9] This figure illustrates an example configuration of a third embodiment of the input device according to this invention. [Figure 10] This is a block diagram illustrating an example of the configuration of each part of a third embodiment of the input device according to this invention.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the input device according to this invention will be described with reference to the drawings.

[0013] [First Embodiment] The embodiment of the input device described below includes a digitizer (tablet) having a sensor unit that performs at least one of transmitting and receiving signals with an electronic pen (hereinafter referred to as "signal transmission and reception"), and a spatial position detection unit that detects the indicated position information by the electronic pen in space. It relates to a spatial position indication system. Note that the position includes not only position coordinates but also posture (tilt and rotation), etc. In the following description, for the sake of clarity, the information on position coordinates and posture will be described separately.

[0014] The spatial position indication system according to this embodiment has a display unit composed of a head-mounted display, smart glasses, etc., and uses the 3D drawing space as a virtual reality (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc. Hereinafter abbreviated as VR) space. FIG. 1 is a diagram showing an overview of the overall configuration of a spatial position indication system including the input device of the first embodiment with the VR space as the 3D drawing space. FIG. 2 is a block diagram showing a detailed configuration example of the functions of each part of the spatial position indication system including the input device of the first embodiment.

[0015] That is, as shown in FIG. 1, the spatial position indication system of this example embodiment includes an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generation device 40, and a head-mounted display (hereinafter referred to as HMD) 50. As shown in FIG. 2, the spatial drawing information generation device 40 has the functions of an input information processing unit 41 and a display image generation unit 42 in this example, and is configured by, for example, a computer.

[0016] Note that the input device of this first embodiment includes a first detection unit that detects the indicated position by the electronic pen 10 in response to the transmission and reception of signals with the electronic pen 10, a second detection unit that detects the indicated position by the electronic pen 10 in the three-dimensional space where the electronic pen 10 exists, and a control unit that generates the position information of the electronic pen 10 based on these indicated positions. In this first embodiment, the first detection unit is constituted by the digitizer 20, the second detection unit is constituted by the spatial position detection unit 30 and the spatial information processing unit 410 of the input processing unit 41, and the control unit is constituted by the input information processing unit.

[0017] In this first embodiment, the electronic pen 10 uses an electromagnetic induction type electronic pen as an example, but it is not limited to the electromagnetic induction method, and an electrostatic coupling method or the like may be used. The digitizer 20 includes a thin rectangular parallelepiped housing 21, and its surface is used as the input surface 21S for position indication by the electronic pen 10. And the digitizer 20 includes a sensor unit 22 and a position detection unit 23 (see FIG. 2).

[0018] Although not shown in the figure, the sensor unit 22 is configured by arranging a plurality of loop coils in the lateral direction (X-axis direction) and the longitudinal direction (Y-axis direction) of the housing of the digitizer 20, respectively. In this example, the digitizer 20 is of the electromagnetic induction type according to the electronic pen 10, but it is not limited to the electromagnetic induction method.

[0019] Although not shown in the figure, the electronic pen 10 includes a resonance circuit (not shown) composed of a coil and a capacitor on the pen tip side, and electromagnetic induction coupling is performed between the loop coil of the sensor unit 22 of the digitizer 20 and the resonance circuit of the electronic pen 10, so as to perform signal transmission and reception between the electronic pen 10 and the sensor unit 22 of the digitizer 20.

[0020] The position detection unit 23 of the digitizer 20 supplies a signal to the electronic pen 10 through the loop coil of the sensor unit 22, and also receives a signal returned from the electronic pen 10 through the loop coil. Based on the received signal, it detects the position indicated by the electronic pen 10 within the detection area of ​​the sensor unit 22. In this embodiment, the digitizer 20 is configured to detect the position indicated by the tip of the electronic pen 10 as the indicated position of the electronic pen 10.

[0021] In this example of the digitizer 20, multiple loop coils of the sensor unit 22 are arranged to cover almost the entire area of ​​the input surface 21S.

[0022] In this embodiment, the position detection region in which the digitizer 20 can detect the indicated position of the electronic pen 10 includes not only the planar region when the tip of the electronic pen 10 is in contact with the input surface 21S of the digitizer 20, but also a spatial region (hover region in the hover state of the electronic pen 10) in which the tip of the electronic pen 10 is not in contact with the input surface 21S of the digitizer 20, and is spaced apart from the input surface 21S in directions perpendicular to the input surface 21S (the Z-axis direction perpendicular to the X-axis direction and the Y-axis direction), but in which the indicated position of the electronic pen 10 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0023] In this example, the spatial position detection unit 30 is configured to set a three-dimensional spatial region in which the digitizer 20 exists, and to detect the orientation of the electronic pen 10 (referred to as pen orientation), such as the position indicated by the tip of the electronic pen 10 and the tilt angle and rotation angle of the electronic pen 10, as well as the position of the digitizer 20 and the tilt angle and tilt direction from the horizontal plane.

[0024] The spatial position detection unit 30 is composed of two light-emitting tracking devices 31A and 31B and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. In this embodiment, as will be described later, the HMD 50 displays a 3D drawing image drawn in the motion detection spatial region MD, which includes the position detection region DT of the digitizer 20, as a virtual display image, as well as a virtual display image of the electronic pen 10.

[0025] The two light-emitting tracking devices 31A and 31B have the same configuration and each includes a laser-emitting unit for infrared laser light, a search means for searching within the motion detection spatial region MD using the emitted infrared laser light, and an optical position detection means for detecting the emission of light from the light-emitting units of trackers 32A, 32B, 32C, and 32D that have received the infrared laser light.

[0026] In this embodiment, the objects to which the trackers 32A, 32B, 32C, and 32D are attached (objects to be searched in the search space area of ​​the spatial position detection unit 30) are, as described above, the electronic pen 10 and the digitizer 20. Specifically, in this example, in order to notify the position of the digitizer 20 and its tilt angle and direction relative to the horizontal plane, trackers 32A and 32B are attached to the upper left and lower right corners of the housing of the thin, rectangular digitizer 20. Furthermore, in order to notify the position and pen orientation (tilt angle and rotation angle) of the electronic pen 10, tracker 32C is attached to the pen tip side of the electronic pen 10, and tracker 32D is attached to the rear end side opposite to the pen tip side in the axial direction of the housing of the electronic pen 10.

[0027] The light-emitting tracking devices 31A and 31B control the laser light-emitting section by a search means to emit infrared laser light and perform a search by scanning within the motion detection space area MD to detect the tracker position. Each of the trackers 32A, 32B, 32C, and 32D monitors the reception of infrared laser light with a sensor and lights up an LED light-emitting section when the sensor detects the reception of infrared laser light.

[0028] The light emission tracking devices 31A and 31B detect the position of the object to which the trackers 32A, 32B, 32C, and 32D are attached within the motion detection spatial area MD by detecting the light emission of the light-emitting parts of the trackers 32A, 32B, 32C, and 32D. The light emission tracking devices 31A and 31B are configured to also detect the elapsed time from the time the infrared laser emitted was emitted to the point in time when the light emission of the light-emitting parts of the trackers 32A, 32B, 32C, and 32D is detected. In this case, each of the trackers 32A, 32B, 32C, and 32D emits a different light according to its own identification information.

[0029] The two light-emitting tracking devices 31A and 31B are connected to the spatial drawing information generation device 40 by wire or wirelessly, and notify the spatial drawing information generation device 40 of the spatial position information of the detected trackers 32A, 32, 32C, and 32D in the motion detection spatial area MD.

[0030] In this example, the spatial position information in the motion detection spatial region MD of the trackers 32A, 32B, 32C, and 32D, detected by the two light emission tracking devices 31A and 31B, is supplied to the spatial information processing unit 410 of the input information processing unit 41 of the spatial drawing information generation device 40, as shown in Figure 2.

[0031] In this example, the spatial information processing unit 410 consists of a spatial position detection unit 4101, a pen posture detection unit 4102, and a digitizer posture detection unit 4103. The spatial position detection unit 4101 detects the position of the digitizer 20 in the motion detection spatial region MD from trackers 32A and 32B, and also detects the position of the electronic pen 10's tip (the position indicated by the electronic pen 10) and the position of the rear end of the electronic pen 10 from trackers 32C and 32D. In this example, the pen posture detection unit 4102 detects the pen posture of the electronic pen 10, including its tilt angle and rotation angle, from trackers 32C and 32D. The digitizer posture detection unit 4103 detects the position indicated by the electronic pen 10 as the tip position.

[0032] In this embodiment, the input information processing unit 41 generates information to be supplied to the display image generation unit 42 from the information of the position indicated by the electronic pen 10 in the position detection area DT detected by the digitizer 20, and the information of the position (indication position and rear end position) of the electronic pen 10 in the motion detection spatial area MD, pen orientation information, and digitizer orientation information detected by the spatial information processing unit 410 of the input information processing unit 41. The input information processing unit 41 then supplies the generated information to the display image generation unit 42.

[0033] In this embodiment, the display image generation unit 42 of the spatial drawing information generation device 40 includes a drawing image generation unit 421 for generating 3D drawing images and a VR image generation unit 422 for generating VR images to be displayed on the HMD 50, as shown in Figure 2.

[0034] The image generation unit 421 generates a 3D image based on position indications from the electronic pen 10, and also performs processing such as deformation, rotation, and movement of the 3D image based on gestures performed by the operator of the electronic pen 10. In this embodiment, the image generation unit 421 also reflects the pen orientation, including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 with respect to the input surface 21S of the digitizer 20, in the generation of the 3D image. For example, when the electronic pen 10 exchanges signals with the digitizer 20, the tilt angle and tilt direction of the digitizer 20 with respect to the input surface 21S are used. Hereinafter, processing related to the 3D image will be referred to as 3D drawing system processing.

[0035] In this embodiment, the trackers 32C and 32D attached to the electronic pen 10 can be used by the spatial position detection unit 30 to detect the pen orientation, including the tilt angle, tilt direction, and rotation angle of the electronic pen 10. Therefore, in this embodiment, the input information processing unit 41 is configured to supply the information on the pen orientation of the electronic pen 10 detected by the spatial position detection unit 30 to the display image generation unit 42 as information for 3D drawing system processing.

[0036] In other words, as will be described later, the input information processing unit 41 is configured to supply the display image generation unit 42 with information on the position of the electronic pen 10 (information on the position of the pen tip) and information on the pen posture of the electronic pen 10, so that the 3D drawing image reflects information on the pen posture of the electronic pen 10, such as the tilt and rotation of the electronic pen 10 during drawing input based on the position indication by the electronic pen 10.

[0037] Furthermore, as shown in Figure 2, the display image generation unit 42 is equipped with a gesture detection processing unit 423 for detecting gestures performed by the operator of the electronic pen 10. In this embodiment, the input information processing unit 41 is configured to supply the gesture detection processing unit 423 with information regarding the position of the pen tip of the electronic pen 10 (corresponding to the information regarding the indicated position) and the pen posture, as detected by the spatial position detection unit 30.

[0038] As described above, in this embodiment, not only the information on the pointing position of the electronic pen 10 detected by the digitizer 20 is used as information for 3D drawing processing, but also the information on the pointing position (pen tip position) and pen orientation of the electronic pen 10 detected by the spatial position detection unit 30 is used.

[0039] The VR image generation unit 422 of the display image generation unit 42 generates a VR image to be displayed on the HMD 50. In this embodiment, the VR image includes a VR image of the electronic pen 10 and a VR image of the 3D drawing generated by the drawing image generation unit 421. On the HMD 50, the VR image of the 3D drawing generated by the drawing image generation unit 421 and the VR image of the electronic pen 10 are displayed in 3D. The VR image generation unit 422 may also generate a VR image of the digitizer 20. Hereinafter, the processing related to the generation of VR images in the VR image generation unit 422 will be referred to as the VR image system processing.

[0040] In this embodiment, the VR image generation unit 422 is supplied with information from the input information processing unit 41 for generating a VR image of the electronic pen 10, and also with information on the generated 3D image from the drawing image generation unit 421. The VR image information generated by the VR image generation unit 422 is then supplied to the HMD 50 via the display drive unit 424 and displayed on the HMD 50's display screen, for example, an LCD (Liquid Crystal Display).

[0041] In this case, when the electronic pen 10 is in the position detection area DT, the input information processing unit 41 supplies the VR image generation unit 422 of the display image generation unit 42 with information on the position indicated by the electronic pen from the digitizer 20, which can detect the position (pen tip position) of the electronic pen 10 with higher accuracy than the spatial position detection unit 30, and information on the pen's orientation, such as tilt and rotation, which is detected using the spatial position detection unit 30.

[0042] Furthermore, when the electronic pen 10 is in the motion detection spatial region MD, the digitizer 20 cannot detect the indicated position of the electronic pen 10. Therefore, the spatial position detection unit 30 detects the indicated position of the electronic pen, and the pen orientation information is supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0043] As described above, in this embodiment, the information used for VR image processing includes not only the information on the indicated position (pen tip position) and pen orientation of the electronic pen 10 detected by the spatial position detection unit 30, but also the information on the indicated position by the electronic pen 10 detected by the digitizer 20.

[0044] The input information processing unit 41 generates information for drawing processing and information for VR image processing as described above from the information from the digitizer 20 and the information from the spatial position detection unit 30, and supplies these to the display image generation unit 42. It also generates a switching control signal SE for selecting whether the electronic pen 10 is in the position detection area DT or the motion detection spatial area MD.

[0045] As described above, in this embodiment, the spatial coordinate system information of the position detection area DT of the digitizer 20 and the spatial coordinate system information of the motion detection spatial area MD of the spatial position detection unit 30 can be mutually utilized in both the 3D rendering system processing and the VR image system processing. In this case, the two spatial coordinate systems can be set independently, but in this embodiment, the information of one of the two spatial coordinate systems is converted to the information of the other spatial coordinate system so that it can be treated as information in a common coordinate space. In this embodiment, the spatial coordinate system information of the motion detection spatial area MD of the spatial position detection unit 30 is converted to the spatial coordinate system information of the position detection area DT of the digitizer 20.

[0046] In this embodiment of the spatial position indication system, the information on the pen orientation of the electronic pen 10 detected in the motion detection spatial region MD of the spatial position detection unit 30 can be used in 3D drawing processing as well as in VR image processing.

[0047] In this case, in VR image processing, the pen orientation information of the electronic pen 10 is reflected in the orientation of the electronic pen 10 in the VR image. This pen orientation information of the electronic pen 10 includes information on the tilt angle of the electronic pen 10, but in 3D drawing processing, this tilt angle of the electronic pen 10 is the tilt angle relative to the input surface of the sensor unit 22 of the digitizer 20. On the other hand, the pen orientation information of the electronic pen 10 detected in the motion detection spatial region MD of the spatial position detection unit 30 is the tilt angle in the motion detection spatial region MD of the spatial position detection unit 30, for example, with reference to the direction of Earth's gravity or the horizontal plane.

[0048] If the digitizer 20 can be positioned such that the direction perpendicular to the input surface 21S of its sensor unit 22 precisely coincides with the Z-axis direction of the motion detection spatial region MD of the spatial position detection unit 30, then the tilt angle of the electronic pen 10 detected in the space of the motion detection spatial region MD of the spatial position detection unit 30 will coincide with the tilt angle of the digitizer 20 relative to the input surface of its sensor unit 22. However, in practice, the digitizer 20 may be positioned at an angle to the motion detection spatial region MD of the spatial position detection unit 30.

[0049] Therefore, in this embodiment, the absolute tilt angle of the electronic pen 10 detected in the spatial motion detection area MD of the spatial position detection unit 30 is converted into a relative tilt angle of the sensor unit 22 of the digitizer 20 with respect to the input surface, as described below.

[0050] Figure 3 is a diagram used to explain the process of transforming this inclination angle. Figure 3 shows a spherical coordinate system with the tip of the electronic pen 10 as point O, the center of the sphere. In Figure 3, the XY plane (a plane containing the mutually orthogonal X-axis and Y-axis directions) is the horizontal plane in the spatial region MD of the motion detection area of ​​the spatial position detection unit 30.

[0051] In Figure 3, the point indicating the position of the rear edge of the electronic pen 10's housing, opposite to the pen tip in the axial direction, is in spherical coordinates. In this case, the radius r of the sphere is the axial length of the electronic pen 10's housing. In the spherical coordinate system of Figure 3, when the rear edge of the electronic pen 10's housing is at point P, the tilt angle and tilt direction of the electronic pen 10 in the motion detection spatial region MD can be detected using the spatial position detection unit 30, with the tilt angle of the electronic pen 10 being δ and the tilt direction being α. Furthermore, using the spatial position detection unit 30, in the example of Figure 3, the tilt angle of the digitizer 20's housing in the motion detection spatial region MD can be detected as δ' and the tilt direction as α'.

[0052] Then, as shown in Figure 3, if point Q is the position of the rear edge of the electronic pen 10 when the axial direction of the electronic pen 10's housing coincides with the vertical direction within the motion detection space region MD, then point R is the position of the rear edge of the electronic pen 10 when the axial direction of the electronic pen 10's housing is perpendicular to the input surface 21S of the sensor section 22 of the digitizer 20.

[0053] Therefore, by determining the angle between the spatial vector OR of point R from point O and the spatial vector OP of point P from point O, it becomes possible to use this angle to convert the tilt angle and tilt direction of the electronic pen 10 detected by the spatial position detection unit 30 into the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0054] Furthermore, by performing the inverse conversion described above, the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor unit 22 of the digitizer 20 can be converted into the tilt angle and tilt direction of the electronic pen 10 detected using the spatial position detection unit 30.

[0055] Next, an example of the configuration of the input information processing unit 41 shown in Figure 2, which is configured to achieve the above, will be described. Specifically, the position detection unit 23, which constitutes the digitizer 20, supplies the detection output of the indicated position of the electronic pen 10 to the selection unit 411 as one of its input signals, and also supplies it to the selection unit 412 as one of its input signals. The information supplied from the position detection unit 23 to the selection units 411 and 412 includes not only the detection output of the indicated position of the electronic pen 10, but also the pressure information applied to the electronic pen 10.

[0056] Furthermore, the spatial position detection unit 4101 of the spatial information processing unit 410 supplies the detection output of the spatial position of the pen tip of the electronic pen 10 (the indicated position of the electronic pen) to the coordinate transformation unit 413. This coordinate transformation unit 413 converts the spatial coordinate system information of the motion detection spatial region MD of the spatial position detection unit 30 to the spatial coordinate system information of the position detection region DT of the digitizer 20. The coordinate transformation unit 413 then supplies the converted coordinate output to the selection units 411 and 412 as the other input signal.

[0057] In this embodiment, the position detection unit 23 of the digitizer 20 supplies information on the signal level of the received signal from the electronic pen 10 to the selection control signal generation unit 414. The selection control signal generation unit 414 detects the distance between the tip of the electronic pen 10 and the input surface 21S of the digitizer 20 from the signal level of the received signal from the electronic pen 10, and generates a selection control signal SE based on the detected distance from the input surface 21S.

[0058] In this case, the selection control signal SE controls the system to select the position detection output from the position detection unit 23 of the digitizer 20 when the distance between the pen tip of the electronic pen 10 and the input surface 21S of the digitizer 20 is less than or equal to the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen, and to select the detection output from the spatial position detection unit 4101 when the distance is greater than the critical height Lz.

[0059] The selection unit 411 switches one of the inputs according to the selection control signal SE and supplies it to the association unit 415 for 3D rendering processing. The selection unit 412 also switches one of the inputs according to the selection control signal SE and supplies it to the association unit 416 for VR image processing.

[0060] In the example shown in Figure 2, two selection units 411 and 412 are provided for 3D rendering and VR image processing, respectively, to clearly distinguish between the two. However, in practice, a single common selection unit is sufficient, and the output of this common selection unit can be supplied to the association units 415 and 416, respectively.

[0061] The information on the pen posture detected by the pen posture detection unit 4102 of the spatial information processing unit 410 using the spatial position detection unit 30 is directly supplied to the association unit 416 for VR image processing.

[0062] The association unit 416 for VR image processing associates the information on the indicated position of the electronic pen 10 from the selection unit 412 with the pen posture information from the pen posture detection unit 4102 of the spatial information processing unit 410, and supplies them as a pair to the VR image generation unit 422 of the display image generation unit 42. In this case, the association in the association unit 416 means associating the information on the pen posture detected at the indicated position of the electronic pen 10, output from the selection unit 412, as a pair.

[0063] If the information regarding the indicated position of the electronic pen 10 and the information regarding the pen's orientation, which are to be associated by the association unit 416, are both from the spatial information processing unit 410, the spatial information processing unit 410 adjusts the output timing of that information, allowing the association unit 416 to simply associate the information by pairing the information at the same timing.

[0064] In contrast, if the information regarding the indicated position of the electronic pen 10 to be associated by the association unit 416 is information from the digitizer 20, the association unit 416 associates the information regarding the indicated position of the electronic pen 10 with the information regarding the pen's orientation by considering the timing difference (processing delay) between the output of the detection results from the digitizer 20 and the spatial position detection unit 30.

[0065] The VR image generation unit 422 of the display image generation unit 42 generates a VR image of the electronic pen 10 using the information on the indicated position of the electronic pen 10 and the pen orientation information from the association unit 416. In this case, the VR image of the electronic pen 10 is generated to be displayed at a position based on the information on the indicated position of the electronic pen 10.

[0066] In this case, the information of the indicated position of the electronic pen 10 from the selection unit 412 is switched and selected by the selection control signal SE, but the information of the indicated position from the spatial position detection unit 4101 is transformed by the coordinate transformation unit 413 so that it becomes a common coordinate space for the digitizer 20 and the spatial position detection unit 30. Therefore, even when the selection unit 412 is switched, there is no display misalignment such as the display position of the VR image of the electronic pen 10 jumping.

[0067] The pen orientation information from the pen orientation detection unit 4102 of the spatial information processing unit 410 is also supplied to the association unit 415 for 3D drawing system processing via the orientation conversion unit 417. The orientation conversion unit 417 is supplied with the pen orientation information from the pen orientation detection unit 4102 and the orientation information of the digitizer 20 detected by the digitizer orientation detection unit 4103. The orientation conversion unit 417 performs a process to convert the pen orientation of the electronic pen 10 in the spatial coordinate system detected by the spatial position detection unit 30, as explained with reference to Figure 3, into the pen orientation relative to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0068] The association unit 415 for 3D drawing processing associates the information of the indicated position of the electronic pen 10 from the selection unit 411 with the information of the pen posture detected by the pen posture detection unit 4102 from the posture conversion unit 417, which has been converted into information of the pen posture relative to the input surface 21S of the digitizer 20, and supplies them as a pair to the drawn image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42. In this case, the association in the association unit 415 also means associating the information of the pen posture detected at the indicated position of the electronic pen 10, output from the selection unit 412, as a pair with the information of the pen posture detected at the indicated position of the electronic pen 10, similar to the association unit 416.

[0069] The image generation unit 421 includes a pen drawing function that draws fine line drawings and the like based on the detection output of the instruction position of the electronic pen 10 from the digitizer 20 and, in this embodiment, information on the pen posture detected by the spatial position detection unit 30, and a gesture processing function that performs drawing processing based on the movement (gesture) detected by the gesture detection processing unit 423 based on the spatial position and pen posture of the electronic pen 10 detected by the spatial position detection unit 30.

[0070] The image generation unit 421 and the gesture detection processing unit 423 are supplied with a selection control signal SE from the selection control signal generation unit 414. The gesture detection processing unit 423 is controlled to operate when the indicated position of the electronic pen 10 is outside the spatial area of ​​the position detection region DT of the digitizer 20. The image generation unit 421 is controlled to execute a pen drawing function when the indicated position of the electronic pen 10 is within the spatial area of ​​the position detection region DT of the digitizer 20, and to switch to execute gesture processing when the indicated position of the electronic pen 10 is outside the spatial area of ​​the position detection region DT of the digitizer 20.

[0071] In this case, in this embodiment, the coordinate transformation unit 413 is used to transform the coordinates so that the digitizer 20 and the spatial position detection unit 30 share a common coordinate space. Therefore, even if the selection unit 411 switches the indicated position of the electronic pen 10, the indicated position of the electronic pen 10 will not jump or shift. Furthermore, even if the pen posture is detected by the spatial position detection unit 30, it is converted into a pen posture relative to the input surface 21S of the digitizer 20, and is therefore appropriately reflected in the 3D drawing image.

[0072] As described above, the 3D drawing image information generated by the drawing image generation unit 421 is supplied to the VR image generation unit 422 to be converted into a VR image, which is then supplied to the HMD 50 via the display drive unit 424 for display. In this embodiment, the position indicated by the electronic pen 10 is made to have the same spatial coordinates as the digitizer 20 and the spatial position detection unit 30 by using the coordinate transformation unit 413, so that the spatial coordinate system of the 3D drawing image and the spatial coordinate system of the VR image such as the electronic pen 10 are the same. Therefore, the position indicated by the electronic pen 10 relative to the 3D drawing image does not differ between the 3D drawing system processing and the VR image system processing, so the VR image generation unit 422 does not need to correct the spatial coordinates of the two.

[0073] As can be seen from the above explanation, in this first embodiment, even if the electronic pen 10 and the digitizer 20 do not have a function to detect the pen orientation of the electronic pen 10, such as the tilt angle and rotation angle of the electronic pen 10, the pen orientation of the electronic pen 10 detected by the spatial position detection unit 30 can be used for 3D image generation. In addition, in this first embodiment, the information on the pen orientation of the electronic pen 10 detected by the spatial position detection unit 30 in a spatial area other than the position detection area DT of the digitizer 20 can be used for gesture detection for 3D image generation. Furthermore, in this first embodiment, the spatial coordinates of the detection area of ​​the digitizer 20 and the spatial coordinates of the detection area of ​​the spatial position detection unit 30 can be treated as common coordinates, so that no coordinate shift occurs when switching between the output from the digitizer 20 and the output from the spatial position detection unit 30 in the 3D drawing system.

[0074] Therefore, the operator can seamlessly perform operations ranging from detailed drawing to gesture-based control simply by spatially moving the electronic pen 10 on the digitizer 20, without being aware of switching between the digitizer 20 and the spatial position detection unit 30.

[0075] Furthermore, in the first embodiment described above, in VR image processing, the position information of the electronic pen 10 (pen tip position information) for generating the VR image of the electronic pen 10 can be used in the spatial region of the position detection area DT of the digitizer 20, as the position information of the electronic pen 10. This is because the position detection output of the digitizer 20 is more accurate and has a faster response speed than the detection output of the spatial position detection unit 30. As a result, a VR image that accurately responds to the operation of the electronic pen by the actual user can be obtained.

[0076] Furthermore, since the spatial drawing information generation device 40 described above is composed of a computer, it goes without saying that each part of the input information processing unit 41 and the display image generation unit 42 can be configured as a software function unit executed by a software program.

[0077] [Second Embodiment] In the first embodiment described above, the electronic pen 10 and digitizer 20 were examples in which there was no function to detect the pen's orientation, such as the tilt angle and rotation angle of the electronic pen. However, it is possible to configure the electronic pen to have a function to detect the pen's orientation, such as the tilt angle and rotation angle of the electronic pen.

[0078] The input device of the second embodiment described below is a system similar to the spatial position indication system described in the first embodiment above, in which the electronic pen and digitizer have a function for detecting the pen's orientation, such as the tilt angle and rotation angle of the electronic pen.

[0079] Figure 4 shows an example of the configuration of the main components of the spatial position indication system of this second embodiment. The spatial position indication system of this second embodiment has the same configuration as the first embodiment, but instead of the electronic pen 10 and digitizer 20, an electronic pen 10A (not shown in Figure 4) and a digitizer 20A are provided, which are equipped with a configuration for detecting the pen's orientation. Furthermore, instead of the input information processing unit 41 of the spatial drawing information generation device 40 in the first embodiment, an input information processing unit 41A with the configuration shown in Figure 4 is provided.

[0080] In this second embodiment of the spatial position indication system, the spatial position detection unit 30, display image generation unit 42, and HMD 50 described in the first embodiment are also included, but since they are the same as in the first embodiment, they are not shown in Figure 4. Furthermore, in the digitizer 20A and input information processing unit 41A shown in Figure 4, the same reference numerals are used for parts that are the same as those for the digitizer 20 and input information processing unit 41 shown in Figure 2, and their detailed explanation is omitted.

[0081] As shown in Figure 5(A), when the electronic pen 10A is perpendicular to the input surface of the sensor unit, the signal levels of a predetermined number of loop coil positions, for example, seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3, exhibit a symmetrical shape with respect to the indicated position.

[0082] In contrast, when the electronic pen 10A is tilted, the signal levels of these seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3 change according to the direction and angle of tilt of the electronic pen 10A, as shown in Figure 5(B). Based on this predetermined number of signal levels, the position coordinates of the indicated position of the electronic pen 10A, as well as the tilt angle and tilt direction, can be detected.

[0083] However, in the input surface (detection area) INS of the sensor unit as shown in Figure 6, in the peripheral area PE indicated by the diagonal lines outside the position of the dotted line, for example, only a number of signal levels less than the predetermined number mentioned above can be obtained. For this reason, in this second embodiment, the detection accuracy of the indicated position (pen tip position) and pen posture of the electronic pen 10A is low in the peripheral area PE of the digitizer 20A, and in this second embodiment, it is important to note that the detection accuracy of the indicated position (pen tip position) and pen posture of the electronic pen 10A detected by the spatial position detection unit 30 is higher.

[0084] In other words, in this second embodiment, even if the tip position of the electronic pen 10A is within the spatial region of the position detection area DT in which the digitizer 20A can detect the indicated position and pen orientation of the electronic pen 10A, if it is in the surrounding region PE and the spatial region above it, the information on the indicated position and pen orientation of the electronic pen 10A detected by the spatial position detection unit 30 is used instead of the information on the indicated position and pen orientation of the electronic pen 10A detected by the digitizer 20A.

[0085] As shown in Figure 4, the digitizer 20A of this second embodiment includes an attitude detection unit 24. This attitude detection unit 24 detects the pen attitude of the electronic pen 10A, such as the tilt angle, tilt direction, and rotation angle, based on the exchange of signals between the electronic pen 10A (not shown) and the sensor unit 22 of the digitizer 20A. The information on the pen attitude of the electronic pen 10A detected by this attitude detection unit 24 is supplied to one input terminal of the selection unit 418 of the input information processing unit 41A. The other input terminal of this selection unit 418 is supplied with information on the pen attitude of the electronic pen 10A detected using the spatial position detection unit 30 from the pen attitude detection unit 4102 of the spatial information processing unit 410.

[0086] The pen orientation information of the electronic pen 10A from the selection unit 418 is supplied to the association unit 415, where it undergoes association processing with the instruction position information of the electronic pen 10A from the selection unit 411, as described above, and is supplied to the drawing image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42.

[0087] In this second embodiment, the pen orientation information of the electronic pen 10A detected by the pen orientation detection unit 4102 of the spatial information processing unit 410 is supplied to one of the input terminals of the selection unit 419. The pen orientation information of the electronic pen 10A from the orientation detection unit 24 of the digitizer 20A is supplied to the orientation conversion unit 417R, and the digitizer orientation information, including the tilt angle and tilt direction of the digitizer 20A detected by the digitizer orientation detection unit 4103 of the spatial information processing unit 410, is also supplied to the orientation conversion unit 417R.

[0088] In the attitude conversion unit 417R, the pen attitude information of the electronic pen 10A from the attitude detection unit 24 of the digitizer 20A and the attitude information of the digitizer 20A from the digitizer attitude detection unit 4103 of the spatial information processing unit 410 are converted in the reverse direction from the attitude conversion unit 417. That is, the relative pen attitude of the electronic pen 10A with respect to the input surface 21S, as detected by the digitizer 20A, is converted into the absolute attitude of the digitizer 20A in the spatial coordinate system of the motion detection spatial region MD detected by the spatial position detection unit 30.

[0089] The pen orientation information of the electronic pen 10A, whose orientation has been transformed by the orientation transformation unit 417R, is then supplied to the other input terminal of the selection unit 419. The pen orientation information of the electronic pen 10A, which is selected and output by the selection unit 419 as described later, is supplied to the association unit 416, where it undergoes the same association process as described above with the information of the indicated position of the electronic pen 10A from the selection unit 412, and is then supplied to the VR image generation unit 422 of the display image generation unit 42.

[0090] In this second embodiment, the selection control signal generation unit 414A generates selection control signals SEA for selection units 411, 412, 418, and 419. The selection control signal generation unit 414A also generates selection control signals SE similar to those of the selection control signal generation unit 414 in the first embodiment described above, and controls the switching of the processes executed by the drawing image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42 in the same manner as in the first embodiment described above.

[0091] In this second embodiment, the selection control signal SEA from the selection control signal generation unit 414A not only selects and controls the selection units 411, 412, 418, and 419 based on whether the indicated position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection area DT of the digitizer 20A or in another spatial region, but also selects and controls the selection units 411, 412, 418, and 419 based on whether the position is in the peripheral region PE (see Figure 6) of the input surface 21S of the digitizer 20A or in the central region inside the peripheral region PE of the input surface 21S.

[0092] Therefore, in this second embodiment, the selection control signal generation unit 414A is supplied with information on the signal level of the received signal from the electronic pen 10A, as well as information on the indicated position of the electronic pen 10A, from the position detection unit 23 of the digitizer 20A. The selection control signal generation unit 414A then generates a selection control signal SEA that selects and controls the selection units 411, 412, 418, and 419 depending on whether the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and within the central spatial area inside the peripheral area PE, or is in any other spatial area.

[0093] The selection unit 411, based on the selection control signal SEA, selects the information on the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 415 if the indicated position of the electronic pen 10A (pen tip position) is within the spatial area of ​​the position detection area DT and within the central spatial area inside the peripheral area PE. If the indicated position of the electronic pen 10A (pen tip position) is in a spatial area other than the above, it selects the information on the indicated position of the electronic pen 10A from the spatial position detection unit 4101 of the coordinate transformation unit 413 and supplies it to the association unit 415.

[0094] Furthermore, the selection unit 412, based on the selection control signal SEA, selects the information on the electronic pen 10A's indicated position (pen tip position) from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 416 when the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection region DT and within the central spatial area inside the peripheral region PE. When the indicated position (pen tip position) of the electronic pen 10A is in a spatial area other than the above, the selection unit 412 selects the information on the electronic pen 10A's indicated position from the spatial position detection unit 4101, which has undergone coordinate transformation processing by the coordinate transformation unit 413, and supplies it to the association unit 416.

[0095] Furthermore, the selection unit 418, based on the selection control signal SEA, selects the pen orientation information of the electronic pen 10A from the orientation detection unit 24 of the digitizer 20A and supplies it to the association unit 415 when the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and within the central spatial area inside the peripheral area PE. When the indicated position (pen tip position) of the electronic pen 10A is in a spatial area other than the above, it selects the pen orientation information converted from the pen orientation of the electronic pen 10A detected by the pen orientation detection unit 4102 from the orientation conversion unit 417 to the relative pen orientation and supplies it to the association unit 415.

[0096] Furthermore, the selection unit 419, based on the selection control signal SEA, selects the pen posture information converted from the pen posture of the electronic pen 10A detected by the attitude detection unit 24 of the digitizer 20A from the attitude conversion unit 417R to the absolute pen posture detected by the spatial position detection unit 30 when the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and within the spatial area of ​​the central area inside the peripheral area PE, and supplies it to the association unit 416. When the indicated position (pen tip position) of the electronic pen 10A is in a spatial area other than the above, it selects the pen posture information of the electronic pen 10A from the pen posture detection unit 4102 and supplies it to the association unit 416.

[0097] An example of the processing flow when the input information processing unit 41A of the spatial position indication system in this second embodiment is configured as a computer will be explained with reference to Figure 7 and its continuation, Figure 8. In this case, each block constituting the input information processing unit 41A shown in Figure 4 becomes a software function unit based on a computer program.

[0098] In other words, the input information processing unit 41A determines from the signal from the digitizer 20A whether or not the digitizer 20A is in a state to detect the indicated position of the electronic pen 10A (step S1). If in step S1 it is determined that the digitizer 20A is not in a state to detect the indicated position of the electronic pen 10A, the input information processing unit 41A determines whether or not the spatial position detection unit 30 can detect the electronic pen 10A (step S2). If in step S2 it is determined that the spatial position detection unit 30 cannot detect the electronic pen 10A, the input information processing unit 41A returns to step S1.

[0099] In step S1, when it is determined that the digitizer 20A has detected the position indicated by the electronic pen 10A, the input information processing unit 41A obtains the coordinates of the position indicated by the electronic pen 10A detected by the digitizer 20A (step S3), and determines whether the obtained coordinates of the position are within the surrounding area PE (step S4).

[0100] In step S4, if the acquired coordinates of the indicated position are determined to be within the inner region rather than the surrounding region PE, the input information processing unit 41A associates the information of the indicated position of the electronic pen 10A from the digitizer 20A with the pen orientation information and outputs it to the display image generation unit 42 for generating a drawn image (step S5).

[0101] Next, the input information processing unit 41A performs a coordinate transformation on the information of the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410, so that it becomes information in a coordinate system common to the digitizer 20A (step S6). Then, the input information processing unit 41A associates the coordinate-transformed information of the indicated position of the electronic pen 10A with the pen posture information detected by the pen posture detection unit 4102 of the spatial information processing unit 410, and outputs it to the display image generation unit 42 for VR image generation (step S7). After step S7, the input information processing unit 41A returns to step S1 and repeats the processing from step S1 onward.

[0102] Next, in step S4, when it is determined that the coordinates of the acquired instruction position are within the surrounding area PE, and in step S2, when it is determined that the spatial position detection unit 30 can detect the electronic pen 10A, the input information processing unit 41A performs a coordinate transformation to make the information of the instruction position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 into information of the same coordinate system as the digitizer 20A (step S11 in Figure 8). The input information processing unit 41A also acquires the attitude information of the digitizer 20A detected by the digitizer attitude detection unit 4103 of the spatial information processing unit 410 (step S12).

[0103] Next, the input information processing unit 41A converts the pen orientation information from the orientation detection unit 24 of the digitizer 20A into absolute orientation information in the spatial domain, rather than relative orientation with respect to the digitizer 20A, using the orientation information of the digitizer 20A acquired in step S12 (step S13). Then, the input information processing unit 41A associates the information of the indicated position of the electronic pen 10A obtained by coordinate transformation in step S11 with the absolute orientation information obtained in step S13 and outputs it to the display image generation unit 42 for drawing image generation (step S14).

[0104] Next, the input information processing unit 41A converts the pen posture information from the pen posture detection unit 4102 of the spatial information processing unit 410 into posture information relative to the digitizer 20A, using the posture information of the digitizer 20A acquired in step S12 (step S15). Then, the input information processing unit 41A associates the information of the indicated position of the electronic pen 10A obtained by coordinate transformation in step S11 with the absolute posture information obtained in step S13 and outputs it to the display image generation unit 42 for VR image generation (step S16). After step S16, the input information processing unit 41A returns to step S1 and repeats the processing from step S1 onward.

[0105] As described above, the input information processing unit 41A of the spatial position indication system in this second embodiment is configured such that, in 3D drawing processing and VR image processing, the information on the indicated position of the electronic pen 10A is used in the central region of the spatial region of the position detection area DT of the digitizer 20A, which is inside the peripheral region PE of the input surface 21S, with higher accuracy than when detected by the spatial position detection unit 30. Furthermore, in the spatial region of the spatial region of the position detection area DT of the digitizer 20A, which is inside the peripheral region PE of the input surface 21S (including the air above the input surface 21S) and the spatial region outside the spatial region of the position detection area DT, the information on the indicated position of the electronic pen 10A is used with higher accuracy than when detected by the spatial position detection unit 23 of the digitizer 20A.

[0106] Furthermore, as information on the pen posture of the electronic pen 10A in 3D drawing processing, in the central region of the spatial region of the position detection area DT of the digitizer 20A, which is inside the peripheral region PE of the input surface 21S, the pen posture information of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30. In the spatial region of the position detection area DT of the digitizer 20A, which is inside the peripheral region PE of the input surface 21S (including the air above the input surface 21S) and the spatial region outside the spatial region of the position detection area DT, the pen posture information of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0107] Therefore, in this second embodiment, as in the first embodiment, 3D rendering processing and VR image processing can always be performed using highly accurate information. In this case, as in the first embodiment, there is the advantage that the information detected by the digitizer 20A and the information detected by the spatial position detection unit 30 can be mutually utilized in both the 3D rendering processing and the VR image processing.

[0108] [Third Embodiment] The first and second embodiments described above are cases in which the input device according to this invention is applied to a spatial position indication system. However, the input device according to this invention is also applicable to a tablet device equipped with a digitizer when detecting position indication by an electronic pen within a position detection area DT including a hover area.

[0109] Figure 9 shows an overview of the system including the input device of this third embodiment. In Figure 9, the input device of the third embodiment is composed of an electronic pen 10B and a tablet device 60 equipped with a digitizer. As shown in Figure 9, the tablet device 60 of this input device is placed, for example, on a desk 90 and connected to a computer 80. In this example, the tablet device 60 is placed on the mounting surface of the desk 90 with, for example, an inclination angle θ.

[0110] In this third embodiment, a gyro sensor unit 70 for detecting the pen orientation of the electronic pen 10B in the three-dimensional space above the input surface 61S of the digitizer built into the tablet device 60 is detachably provided on the rear end of the electronic pen 10B opposite to the pen tip. The gyro sensor unit 70 may also be built into the rear end of the electronic pen 10B opposite to the pen tip.

[0111] Figure 10 is a block diagram illustrating an example of the functional configuration of the electronic pen 10B, tablet device 60, and gyro sensor unit 70 that constitute the input device of this third embodiment. In this case, the gyro sensor unit 70 is configured to include, for example, a 3-axis gyro sensor 71 and a wireless communication unit 72. The wireless communication unit 72 is configured to be, for example, a short-range wireless communication means conforming to the Bluetooth® standard. The wireless communication unit 72 is not limited to this and may be optical communication such as infrared communication.

[0112] In this example, the gyro sensor unit 70 is mounted on the housing of the electronic pen 10B, so the 3-axis gyro sensor 71 outputs an output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B in three-dimensional space. Here, the output signal of the electronic pen 10B detected by the 3-axis gyro sensor 71, corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B, is not corresponding to the relative orientation of the electronic pen 10B with respect to the input surface 61S of the tablet device 60, but rather to the absolute orientation with respect to the Earth's axis in three-dimensional space. The wireless communication unit 72 wirelessly transmits the output signal from the 3-axis gyro sensor 71 to the tablet device 60.

[0113] The electronic pen 10B is equipped with an interaction unit 100 for communication with the sensor unit 61 of the tablet device 60. The interaction unit 100 communicates with the sensor unit 61 of the tablet device 60 by either electromagnetic induction or electrostatic coupling, and exchanges signals. In this example, the interaction unit 100 of the electronic pen 10B communicates with the sensor unit 61 to detect the position of the electronic pen 10B's indicated position, but does not communicate signals to detect the pen's orientation, such as tilt angle, tilt direction, or rotation angle.

[0114] In the tablet device 60, the instruction position of the electronic pen 10B is detected by the instruction position detection unit 62 from the signal obtained as a result of interaction with the electronic pen 10B at the sensor unit 61. The information of the instruction position of the electronic pen 10B detected by the instruction position detection unit 62 is supplied to the association unit 63.

[0115] The tablet device 60 of this third embodiment includes a wireless communication unit 64 that communicates wirelessly with the wireless communication unit 72 of the gyro sensor unit 70. The detection output of the 3-axis gyro sensor 71 of the gyro sensor unit 70, received by the wireless communication unit 64, is supplied to the electronic pen attitude detection unit 65 to detect the pen attitude of the electronic pen 10B. The information on the pen attitude of the electronic pen 10B detected by the electronic pen attitude detection unit 65 is supplied to the relative attitude calculation unit 66.

[0116] The tablet device 60 in this embodiment is equipped with a gyro sensor 67 for detecting the tilt angle and tilt direction of the device. The sensor output of this gyro sensor 67 is supplied to the tablet attitude detection unit 68. Based on the sensor output of the gyro sensor 67, the tablet attitude detection unit 68 detects the tilt angle and tilt direction of the device and supplies the detected information on the tilt angle and tilt direction of the device (tablet attitude information) to the relative attitude calculation unit 66.

[0117] The relative posture calculation unit 66 calculates the relative pen posture of the electronic pen 10B with respect to the input surface 61S of the tablet device 60 by correcting the absolute pen posture information of the electronic pen 10B in three-dimensional space from the electronic pen posture detection unit 65 using the tablet posture information from the tablet posture detection unit 68. The relative posture calculation unit 66 then supplies the calculated relative pen posture information of the electronic pen 10B to the association unit 63.

[0118] The association unit 63, in the same manner as the association unit described above, associates information about the indicated position of the electronic pen 10B with information about the relative pen orientation of the electronic pen 10B and outputs this information to the computer 80.

[0119] As described above, in this third embodiment of the input device, when the indicated position by the tip of the electronic pen 10B is obtained, the gyro sensor unit 70 provided on the rear end of the electronic pen 10B can output information on the pen orientation of the electronic pen 10B detected in three-dimensional space as pen orientation information corresponding to the indicated position of the electronic pen 10B.

[0120] [Other embodiments or variations] In the first and second embodiments described above, the spatial position detection unit 30 is configured to include a light emission tracking device that emits infrared laser light and a tracker, but it goes without saying that the configuration is not limited to this. For example, a configuration using other non-visible light sensors, visible light sensors, or combinations thereof may also be used.

[0121] Furthermore, in the first and second embodiments described above, since the HMD 50 is worn, the operator of the electronic pen 10 cannot directly see the digitizer 20. Therefore, it is also possible to configure the system so that a virtual image of the digitizer 20 is drawn on the virtual space image displayed on the HMD 50, allowing the operator to recognize the position of the digitizer 20 on the display screen of the HMD 50.

[0122] Furthermore, in the first and second embodiments described above, the coordinate values ​​of the spatial position detection unit 30 were converted to the coordinate values ​​of the spatial coordinate system of the digitizer 20 in order to correct errors between the two spatial coordinate systems. However, conversely, the coordinate values ​​of the position detection area DT of the digitizer 20 may be converted to the coordinate values ​​of the spatial position detection unit 30. Note that this conversion between the two spatial coordinate systems is not mandatory and is optional.

[0123] In the first and second embodiments described above, the electronic pen and digitizer are of the electromagnetic induction type, but are not limited to this, and of course, electrostatic electronic pens and digitizers (including active electrostatic coupling and passive electrostatic coupling) can also be used.

[0124] Furthermore, the digitizers of the first and second embodiments and the tablet device of the third embodiment may be portable mobile phone terminals known as smartphones.

[0125] In the first and second embodiments, the case of 3D rendering using a spatial position indication system was described, but the rendering image may also be a 2D rendering image or a 2.5D rendering image. [Explanation of Symbols]

[0126] 10, 10A, 10B…Electronic pen, 20, 20A…Digitizer, 21S…Input surface, 22…Sensor unit, 23…Position detection unit, 24…Attitude detection unit, 30…Spatial position detection unit, 40…Spatial drawing information generation device, 41…Input information processing unit, 4101…Spatial position detection unit, 4102…Pen attitude detection unit, 50…HMD

Claims

1. A device having a digitizer that detects the position indicated by an electronic pen, A tracking device for detecting the position of the tracker of the electronic pen, and a tracking device for performing position input by the electronic pen in a VR (Virtual Reality) space set in a first coordinate system associated with the tracking device, When the device having the digitizer detects the position indicated by the electronic pen, a computer converts the coordinates in the first coordinate system corresponding to the position indicated by the electronic pen in the three-dimensional space where the electronic pen being tracked by the tracking device exists, to coordinates in a second coordinate system associated with the device having the digitizer, which corresponds to the position indicated by the electronic pen detected by the device having the digitizer, An instruction input processing system having the following features.

2. The computer calculates the relative orientation of the electronic pen with respect to the input surface of the device having the digitizer, based on the orientation of the electronic pen in the three-dimensional space where the electronic pen being tracked by the tracking device exists and the orientation of the device having the digitizer in the three-dimensional space being tracked by the tracking device. The instruction input processing system according to claim 1.

3. The second coordinate system associated with the device having the digitizer is a coordinate system relating to the position detection area of ​​the device having the digitizer, and the first coordinate system associated with the tracking device is a coordinate system relating to the position detection area of ​​the tracking device. The instruction input processing system according to claim 1.

4. The computer displays a VR image related to the electronic pen in the VR space on the display unit. The instruction input processing system according to claim 1.

5. The tracker of the aforementioned electronic pen has an LED (Light Emitting Diode) that emits light. The instruction input processing system according to claim 1.

6. The aforementioned VR space includes MR (Mixed Reality) space and AR (Augmented Reality) space. The instruction input processing system according to claim 1.

7. The computer calculates the coordinates in a second coordinate system associated with the device having the digitizer by correcting the coordinates in a first coordinate system associated with the tracking device. The instruction input processing system according to claim 1.

8. The data relating to the orientation of the electronic pen includes at least one of the tilt angle, tilt direction, or rotation angle of the electronic pen. The instruction input processing system according to claim 1.

9. When the position indicated by the electronic pen is detected by a device having a digitizer that detects the position indicated by the electronic pen, the coordinates in the first coordinate system corresponding to the position indicated by the electronic pen in the three-dimensional space where the electronic pen exists, which is tracked by a tracking device for performing position input by the electronic pen in a VR (Virtual Reality) space set to a first coordinate system associated with the tracking device that detects the position of the electronic pen's tracker, are converted to coordinates in a second coordinate system associated with the device having the digitizer that corresponds to the position indicated by the electronic pen detected by the device having the digitizer. Instruction input processing method.

10. Based on the orientation of the electronic pen in the three-dimensional space where the electronic pen is located and tracked by the tracking device, and the orientation of the device having the digitizer in the three-dimensional space tracked by the tracking device, the relative orientation of the electronic pen with respect to the input surface of the device having the digitizer is calculated. The instruction input processing method according to claim 9.

11. The second coordinate system associated with the device having the digitizer is a coordinate system relating to the position detection area of ​​the device having the digitizer, and the first coordinate system associated with the tracking device is a coordinate system relating to the position detection area of ​​the tracking device. The instruction input processing method according to claim 9.

12. The VR image relating to the electronic pen in the VR space is displayed on the display unit. The instruction input processing method according to claim 9.

13. As tracking for the tracker of the electronic pen, the position of the tracker is detected using light emitted from an LED (Light Emitting Diode). The instruction input processing method according to claim 9.

14. The aforementioned VR space includes MR (Mixed Reality) space and AR (Augmented Reality) space. The instruction input processing method according to claim 9.

15. The coordinates in the second coordinate system associated with the device having the digitizer are calculated by correcting the coordinates in the first coordinate system associated with the tracking device. The instruction input processing method according to claim 9.

16. The data relating to the orientation of the electronic pen includes at least one of the tilt angle, tilt direction, or rotation angle of the electronic pen. The instruction input processing method according to claim 9.

Images