VR image creation system and VR image creation method

Abstract

To provide a virtual reality (VR) image creation system and a VR image creation method for creating a VR image in a VR space by using an electronic pen.SOLUTION: A spatial position instruction system has: a digitizer 20 that detects a position indicated by an electronic pen 10; and a spatial drawing information generation device 40 including an input information processing unit 41 that detects the position of a tracker of the electronic pen to perform position input using the electronic pen in a virtual reality (VR) space, and a display image creation unit 42 that creates a VR image related to the electronic pen in the VR space, on the basis of the posture of the electronic pen and the position indicated by the electronic pen detected by the digitizer, in a three-dimensional space where the electronic pen is present which is detected on the basis of tracking of the tracker of the electronic pen performed by the input information processing unit.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This invention relates to a VR (Virtual Reality) image generation system and a VR image generation method.

Background Art

[0002] There is known a drawing system that creates an animation image or the like by continuously indicating a position with an electronic pen to a coordinate input device called a digitizer and drawing thereby.

[0003] Conventionally, various methods have been proposed as methods for detecting the inclination and rotation of an electronic pen. Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2016-126503) provides an electronic pen capable of detecting the inclination angle and rotation angle with respect to the input surface of the sensor unit of a position detection device. In the case of this Patent Document 1, between the electronic pen and the position detection device including the sensor unit, signals are transmitted and received by electromagnetic induction coupling, capacitance coupling, or the like, so that the position detection device detects the position indicated by the electronic pen and also detects the position coordinates of the indicated position of the electronic pen, as well as the inclination angle and rotation angle.

[0004] Also, Patent Document 2 (U.S. Patent No. 9,329,703 B2) describes that a sensor capable of measuring the movement and orientation of an electronic pen is built in or attached to the electronic pen, and by transmitting the detection output of the sensor to a position detection device, the movement and state (inclination and rotation) of the electronic pen can be detected by the position detection device.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the case of the above-mentioned Patent Document 1, the position coordinates and inclination of the pointing position of the electronic pen are detected by using a plurality of signal levels obtained from a plurality of loop coils around the pointing position by the electronic pen. However, in the peripheral area of the sensor unit, there has been a problem that the accuracy of the position coordinates of the pointing position of the electronic pen decreases and it becomes difficult to detect the inclination of the electronic pen.

[0007] According to the method of Patent Document 2, the problems in the case of Patent Document 1 do not occur. However, depending on the conditions, there are problems such as a slow reaction speed and a decrease in the accuracy of the position coordinates.

[0008] An object of the present invention is to provide a VR image generation system capable of solving the above problems.

Means for Solving the Problems

[0009] In order to solve the above problems, an apparatus having a digitizer for detecting a pointing position by an electronic pen, a tracking device for detecting the position of the tracker of the electronic pen, the tracking device for performing position input by the electronic pen in a VR (Virtual Reality) space, a computer that generates a VR image related to the electronic pen in the VR space based on the posture of the electronic pen in the three-dimensional space where the electronic pen exists detected based on the tracking of the tracker of the electronic pen by the tracking device and the pointing position by the electronic pen detected by the apparatus having the digitizer, A VR image generation system characterized by comprising the above is provided.

[0010] In the VR image generation system configured as described above, a VR image related to the electronic pen in the VR space is generated based on the posture of the electronic pen in the three-dimensional space detected by the tracking device and the indication position by the electronic pen detected by the device having the digitizer.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the input device according to the present invention will be described with reference to the drawings.

[0013] [First Embodiment] The embodiment of the input device described below relates to a spatial position indication system including a digitizer (tablet) having an electronic pen and a sensor unit that performs at least one of signal transmission and reception (hereinafter referred to as "signal exchange"), and a spatial position detection unit that detects indication position information by the electronic pen in space. Note that the position includes not only position coordinates but also posture (tilt and rotation), etc. In the following description, for the sake of clarity, the information on position coordinates and posture will be described separately.

[0014] The spatial position indication system according to this embodiment has a display unit configured by a head-mounted display, smart glasses, etc., and uses a 3D drawing space as a virtual reality (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc. Hereinafter abbreviated as VR) space. FIG. 1 is a diagram showing an overview of the overall configuration of a spatial position indication system including the input device of the first embodiment with the VR space as the 3D drawing space. FIG. 2 is a block diagram showing a detailed configuration example of the functions of each part of the spatial position indication system including the input device of the first embodiment.

[0015] That is, as shown in FIG. 1, the spatial position indication system of this example embodiment includes an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generation device 40, and a head-mounted display (hereinafter referred to as HMD) 50. As shown in FIG. 2, the spatial drawing information generation device 40 has functions of an input information processing unit 41 and a display image generation unit 42 in this example, and is configured by, for example, a computer.

[0016] Note that the input device according to this first embodiment includes a first detection unit that detects the indicated position by the electronic pen 10 in response to the transmission and reception of signals with the electronic pen 10, a second detection unit that detects the indicated position by the electronic pen 10 in the three-dimensional space where the electronic pen 10 exists, and a control unit that generates the position information of the electronic pen 10 based on these indicated positions. In this first embodiment, the first detection unit is constituted by the digitizer 20, the second detection unit is constituted by the spatial position detection unit 30 and the spatial information processing unit 410 of the input processing unit 41, and the control unit is constituted by the input information processing unit.

[0017] In this first embodiment, the electronic pen 10 uses an electromagnetic induction type electronic pen as an example, but it is not limited to the electromagnetic induction type, and an electrostatic coupling type or the like may be used. The digitizer 20 includes a thin rectangular parallelepiped housing 21, and its surface is used as the input surface 21S for position indication by the electronic pen 10. And the digitizer 20 includes a sensor unit 22 and a position detection unit 23 (see FIG. 2).

[0018] Although not shown in the figure, the sensor unit 22 is configured by arranging a plurality of loop coils in the lateral direction (X-axis direction) and the longitudinal direction (Y-axis direction) of the housing of the digitizer 20, respectively. In this example, the digitizer 20 is of the electromagnetic induction type according to the electronic pen 10, but it is not limited to the electromagnetic induction type.

[0019] Although not shown in the figure, the electronic pen 10 includes a resonance circuit (not shown) composed of a coil and a capacitor on the pen tip side, and electromagnetic induction coupling is performed between the loop coil of the sensor unit 22 of the digitizer 20 and the resonance circuit of the electronic pen 10, so that signals are transmitted and received between the electronic pen 10 and the sensor unit 22 of the digitizer 20.

[0020] The position detection unit 23 of the digitizer 20 supplies a signal to the electronic pen 10 through the loop coil of the sensor unit 22, receives the signal returned from the electronic pen 10 through the loop coil, and based on the received signal, detects the position indicated by the electronic pen 10 in the detection area of the sensor unit 22. In this embodiment, the digitizer 20 is configured to detect the position indicated by the pen tip of the electronic pen 10 as the indicated position of the electronic pen 10.

[0021] In the digitizer 20 of this example, a plurality of loop coils of the sensor unit 22 are arranged so as to cover substantially the entire area of the input surface 21S.

[0022] And in this embodiment, the position detection area where the digitizer 20 can detect the indicated position of the electronic pen 10 includes not only the planar area when the pen tip of the electronic pen 10 is in contact with the input surface 21S of the digitizer 20, but also the area where the pen tip of the electronic pen 10 is not in contact with the input surface 21S of the digitizer 20 and is separated from the input surface 21S in a direction orthogonal to the input surface 21S (the Z-axis direction orthogonal to the X-axis direction and the Y-axis direction), and is a spatial area (hover area in the hover state of the electronic pen 10) where the indicated position of the electronic pen 10 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0023] The spatial position detection unit 30, in this example, sets a three-dimensional space area where the digitizer 20 exists, and in the three-dimensional space area, detects the indicated position by the pen tip of the electronic pen 10, the posture of the electronic pen 10 (referred to as pen posture) such as the inclination angle and rotation angle of the electronic pen 10, and is configured to be able to detect the position of the digitizer 20 and the inclination angle and inclination direction from the horizontal plane.

[0024] The spatial position detection unit 30 is configured to include two light emission tracking devices 31A and 31B, and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. In this embodiment, as will be described later, in the HMD 50, a 3D drawing image drawn in the motion detection space area MD including the position detection area DT of the digitizer 20 is displayed as a virtual display image, and a virtual display image of the electronic pen 10 is also displayed.

[0025] The two light emission tracking devices 31A and 31B have the same configuration, and each includes a laser emission unit for infrared laser light, a search means for searching within the motion detection space area MD by the emitted infrared laser light, and a light position detection means for detecting the light emission of the light emission units of the trackers 32A, 32B, 32C, and 32D that have received the infrared laser light.

[0026] The objects (objects to be searched in the search space area of the spatial position detection unit 30) on which the trackers 32A, 32B, 32C, and 32D are mounted are, in this embodiment, as described above, the electronic pen 10 and the digitizer 20. That is, in this example, in order to be able to notify the position of the digitizer 20, the tilt angle with respect to the horizontal plane, and the tilt direction, trackers 32A and 32B are mounted at the upper left corner and the lower right corner of the housing of the digitizer 20, which is in the shape of a thin rectangular parallelepiped. Further, in order to notify the position of the electronic pen 10 and the pen posture (tilt angle and rotation angle), a tracker 32C is mounted on the tip side of the electronic pen 10, and a tracker 32D is mounted on the rear end side opposite to the tip side in the axial direction of the housing of the electronic pen 10.

[0027] The light emission tracking devices 31A and 31B control the laser emission unit by the search means, emit infrared laser light so as to search and scan within the motion detection space area MD to detect the tracker position, and perform a search. Each of the trackers 32A, 32B, 32C, and 32D monitors the reception of the infrared laser light with a sensor, and when the reception of the infrared laser light is detected by the sensor, the light emission unit composed of an LED is turned on.

[0028] The light emission tracking devices 31A and 31B detect the light emission of the light emitting units of the trackers 32A, 32B, 32C, and 32D, and thereby detect the positions within the motion detection space region MD of the object on which the trackers 32A, 32B, 32C, and 32D are mounted. The light emission tracking devices 31A and 31B are configured to be able to detect the elapsed time from the light emission time of the emitted infrared laser at the time of detection when detecting the light emission of the light emitting units of the trackers 32A, 32B, 32C, and 32D. In this case, each of the trackers 32A, 32B, 32C, and 32D emits different light according to its own identification information.

[0029] The two light emission tracking devices 31A and 31B are connected to the space drawing information generation device 40 by wire or wirelessly, and notify the space drawing information generation device 40 of the spatial position information in the motion detection space region MD of the detected trackers 32A, 32, 32C, and 32D.

[0030] In this example, the spatial position information in the motion detection space region MD of the trackers 32A, 32B, 32C, and 32D detected by the two light emission tracking devices 31A and 31B is supplied to the spatial information processing unit 410 of the input information processing unit 41 of the space drawing information generation device 40 as shown in FIG. 2.

[0031] In this example, the spatial information processing unit 410 includes a spatial position detection unit 4101, a pen posture detection unit 4102, and a digitizer posture detection unit 4103. The spatial position detection unit 4101 detects the position of the digitizer 20 in the motion detection space region MD from the trackers 32A and 32B, and detects the position indicated by the electronic pen 10 as the position of the tip of the electronic pen 10 and the position of the rear end of the electronic pen 10 from the trackers 32C and 32D. The pen posture detection unit 4102 detects, in this example, the pen posture including the tilt angle and rotation angle of the electronic pen 10 from the trackers 32C and 32D. Further, the digitizer posture detection unit 4103 detects the position indicated by the electronic pen 10 as the tip position.

[0032] And in this embodiment, the input information processing unit 41 generates information to be supplied to the display image generation unit 42 from the information on the indicated position by the electronic pen 10 in the position detection area DT detected by the digitizer 20, and the information on the position (indicated position and rear end position), pen attitude information, and digitizer attitude information of the electronic pen 10 in the motion detection space area MD detected by the space information processing unit 410 of the input information processing unit 41. Then, the input information processing unit 41 supplies the generated information to the display image generation unit 42.

[0033] And in this embodiment, as shown in FIG. 2, the display image generation unit 42 of the spatial drawing information generation device 40 includes a drawing image generation unit 421 for generating a 3D drawing image and a VR image generation unit 422 for generating a VR image to be displayed on the HMD 50.

[0034] The drawing image generation unit 421 generates a 3D drawing image based on the position indication by the electronic pen 10, and performs processes such as deformation, rotation, and movement of the 3D drawing image based on the gesture executed by the operator of the electronic pen 10. And in this embodiment, the drawing image generation unit 421 also reflects the pen attitude including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 with respect to the input surface 21S of the digitizer 20 in the generation of the 3D drawing image. For example, when the electronic pen 10 exchanges signals with the digitizer 20, the tilt angle and tilt direction with respect to the input surface 21S of the digitizer 20 are used. Hereinafter, the process related to the 3D drawing image is referred to as 3D drawing system processing.

[0035] In this embodiment, the pen attitude including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 can be detected by the spatial position detection unit 30 using the trackers 32C and 32D attached to the electronic pen 10. Therefore, in this embodiment, the input information processing unit 41 is configured to supply the information on the pen attitude of the electronic pen 10 detected by the spatial position detection unit 30 to the display image generation unit 42 as information for 3D drawing system processing.

[0036] That is, as will be described later, the input information processing unit 41 is configured to supply the display image generation unit 42 with information on the position indication by the electronic pen 10 (information on the position of the pen tip) and information on the pen posture of the electronic pen 10 so that information on the pen posture such as the inclination and rotation of the electronic pen 10 at the time of drawing input based on the position indication by the electronic pen 10 is reflected in the 3D drawing image.

[0037] Also, as shown in FIG. 2, the display image generation unit 42 is provided with a gesture detection processing unit 423 for detecting gestures executed by the operator of the electronic pen 10. In this embodiment, the input information processing unit 41 is configured to use the position information of the pen tip of the electronic pen 10 (corresponding to the indication position information) detected by the spatial position detection unit 30 and the information on the pen posture as information to be supplied to the gesture detection processing unit 423.

[0038] As described above, in this embodiment, not only the information on the indication position by the electronic pen 10 detected by the digitizer 20 but also the information on the indication position (position of the pen tip) of the electronic pen 10 and the information on the pen posture detected by the spatial position detection unit 30 are used as information for 3D drawing system processing.

[0039] The VR image generation unit 422 of the display image generation unit 42 generates a VR image to be displayed on the HMD 50. In this embodiment, the VR image includes the VR image of the electronic pen 10 and the VR image of the 3D drawing image generated by the drawing image generation unit 421. On the HMD 50, the VR image of the 3D drawing image generated by the drawing image generation unit 421 and the VR image of the electronic pen 10 are 3D displayed. Note that the VR image generation unit 422 may also generate a VR image of the digitizer 20. Hereinafter, the processing related to the generation of the VR image in the VR image generation unit 422 is referred to as VR image system processing.

[0040] In this embodiment, the VR image generation unit 422 is supplied with information for generating a VR image of the electronic pen 10 from the input information processing unit 41, and information on the generated 3D drawing image is supplied from the drawing image generation unit 421. Then, the VR image information generated by the VR image generation unit 422 is supplied to the HMD 50 through the display drive unit 424 and is displayed on a display screen of the HMD 50, which is, for example, an LCD (Liquid Crystal Display).

[0041] In this case, when the electronic pen 10 is present in the position detection area DT, information on the indicated position by the electronic pen from the digitizer 20, which can detect the position (pen tip position) of the electronic pen 10 with higher accuracy than the spatial position detection unit 30, and information on the pen posture such as the inclination and rotation detected by using the spatial position detection unit 30 are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0042] Also, when the electronic pen 10 is present in the motion detection space area MD, since the digitizer 20 cannot detect the indicated position of the electronic pen 10, information on the indicated position by the electronic pen detected by using the spatial position detection unit 30 and information on the pen posture are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0043] As described above, in this embodiment, as information for VR image system processing, not only information on the indicated position (pen tip position) of the electronic pen 10 and information on the pen posture detected by the spatial position detection unit 30 but also information on the indicated position by the electronic pen detected by the digitizer 20 are used.

[0044] The input information processing unit 41 generates information for drawing system processing and information for VR image system processing as described above from the information from the digitizer 20 and the information from the spatial position detection unit 30, supplies them to the display image generation unit 42, and generates a switching control signal SE for performing selection control according to whether the electronic pen 10 is present in either the position detection area DT or the motion detection space area MD.

[0045] As described above, in this embodiment, in each of the 3D drawing system processing and the VR image system processing, the information of the spatial coordinate system of the position detection region DT of the digitizer 20 and the information of the spatial coordinate system of the motion detection spatial region MD of the spatial position detection unit 30 can be used mutually. In this case, the two spatial coordinate systems can be set independently, but in this embodiment, by converting the information of one of the two spatial coordinate systems into the information of the other spatial coordinate system, it can be handled as information in a common coordinate space. In this embodiment, the information of the spatial coordinate system of the motion detection spatial region MD of the spatial position detection unit 30 is converted into the information of the spatial coordinate system of the position detection region DT of the digitizer 20.

[0046] In the spatial position indication system of this embodiment, the information of the pen posture of the electronic pen 10 detected in the motion detection spatial region MD of the spatial position detection unit 30 can be used in the 3D drawing system processing and also in the VR image system processing.

[0047] In this case, in the VR image system processing, the information of the pen posture of the electronic pen 10 is reflected in the posture of the VR image of the electronic pen 10. The information of the pen posture of this electronic pen 10 includes the information of the inclination angle of the electronic pen 10, but in the 3D drawing system processing, the inclination angle of this electronic pen 10 is the relative inclination angle with respect to the input surface of the sensor unit 22 of the digitizer 20. On the other hand, the information of the pen posture of the electronic pen 10 detected in the motion detection spatial region MD of the spatial position detection unit 30 is, for example, the inclination angle in the motion detection spatial region MD of the spatial position detection unit 30 based on the direction of the earth's gravity or the horizontal plane direction.

[0048] If the digitizer 20 can be installed such that the direction orthogonal to the input surface 21S of its sensor unit 22 exactly coincides with the Z-axis direction of the motion detection space region MD of the spatial position detection unit 30, then the tilt angle of the electronic pen 10 detected in the space of the motion detection space region MD of the spatial position detection unit 30 coincides with the relative tilt angle with respect to the input surface of the sensor unit 22 of the digitizer 20. However, in practice, the digitizer 20 may be installed at an angle with respect to the motion detection space region MD of the spatial position detection unit 30.

[0049] Therefore, in this embodiment, the absolute tilt angle of the electronic pen 10 detected in the space of the motion detection space region MD of the spatial position detection unit 30 is converted into the relative tilt angle with respect to the input surface of the sensor unit 22 of the digitizer 20 as described below.

[0050] FIG. 3 is a diagram used to explain the process of converting this tilt angle. This FIG. 3 shows a spherical coordinate system with the tip of the electronic pen 10 as the center point O of the sphere. The X-Y plane (the plane including the X-axis direction and the Y-axis direction orthogonal to each other) in this FIG. 3 is defined as the horizontal plane in the space region of the motion detection space region MD of the spatial position detection unit 30.

[0051] In FIG. 3, the point indicating the position of the edge of the rear end portion on the side opposite to the tip in the axial direction of the housing of the electronic pen 10 is in spherical coordinates. In this case, the radius r of the sphere is the length in the axial direction of the housing of the electronic pen 10. In the spherical coordinate system of FIG. 3, when the edge of the rear end portion in the axial direction of the housing of the electronic pen 10 is at point P, the tilt angle and tilt direction of the electronic pen 10 in the space of the motion detection space region MD can be detected as the tilt angle δ and the tilt direction α, respectively, by using the spatial position detection unit 30. Also, by using the spatial position detection unit 30, in the example of FIG. 3, the tilt angle of the housing of the digitizer 20 in the space of the motion detection space region MD can be detected as δ´, and the tilt direction can be detected as α´.

[0052] Then, as shown in FIG. 3, when the axial direction of the housing of the electronic pen 10 coincides with the vertical direction within the space of the motion detection space region MD, if the position of the edge on the rear end side of the electronic pen 10 is defined as point Q, when the axial direction of the housing of the electronic pen 10 is perpendicular to the input surface 21S of the sensor unit 22 of the digitizer 20, the position of the edge on the rear end side of the electronic pen 10 becomes point R.

[0053] Therefore, when the angle formed by the spatial vector OR from point O to point R and the spatial vector OP from point O to point P is obtained, by using this angle, it becomes possible to convert from the inclination angle and inclination direction of the electronic pen 10 detected using the spatial position detection unit 30 to the relative inclination angle and inclination direction of the electronic pen 10 with respect to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0054] Note that by performing the above reverse conversion, the relative inclination angle and inclination direction of the electronic pen 10 with respect to the input surface 21S of the sensor unit 22 of the digitizer 20 can be converted into the inclination angle and inclination direction of the electronic pen 10 detected using the spatial position detection unit 30.

[0055] Next, a configuration example of the input information processing unit 41 in FIG. 2 configured to realize the above will be described. That is, the position detection unit 23 constituting the digitizer 20 supplies the detection output of the indicated position of the electronic pen 10 as one of its input signals to the selection unit 411 and also supplies it as one of its input signals to the selection unit 412. Note that the information supplied from the position detection unit 23 to the selection units 411 and 412 includes, in addition to the detection output of the indicated position of the electronic pen 10, pen pressure information applied to the electronic pen 10.

[0056] Also, the spatial position detection unit 4101 of the spatial information processing unit 410 supplies the detection output of the spatial position of the pen tip of the electronic pen 10 (the indicated position of the electronic pen) to the coordinate conversion unit 413. This coordinate conversion unit 413 converts the information of the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 into the information of the spatial coordinate system of the position detection region DT of the digitizer 20. Then, the coordinate conversion unit 413 supplies the converted coordinate output to the selection units 411 and 412 as the other input signal.

[0057] And in this embodiment, the position detection unit 23 of the digitizer 20 supplies the information of the signal level of the received signal from the electronic pen 10 to the selection control signal generation unit 414. The selection control signal generation unit 414 detects the separation distance from the input surface 21S of the digitizer 20 at the pen tip of the electronic pen 10 from the signal level of the received signal from the electronic pen 10, and generates a selection control signal SE based on the detected separation distance from the input surface 21S.

[0058] In this case, when the separation distance from the input surface 21S of the digitizer 20 at the pen tip of the electronic pen 10 is equal to or less than the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen, the selection control signal SE controls to select the position detection output from the position detection unit 23 of the digitizer 20, and when it is greater than the critical height Lz, it is a signal that controls to select the detection output from the spatial position detection unit 4101.

[0059] The selection unit 411 switches one of the one input and the other input according to the selection control signal SE and supplies it to the association unit 415 for 3D drawing system processing. Also, the selection unit 412 switches one of the one input and the other input according to the selection control signal SE and supplies it to the association unit 416 for VR image system processing.

[0060] In the example of FIG. 2, two selection units 411 and 412 are provided for each type of processing, i.e., for 3D drawing-based processing and for VR image-based processing, to make it clear. However, in practice, only one common selection unit is needed, and the output of the common selection unit can be configured to be supplied to each of the association units 415 and 416.

[0061] The information on the pen posture detected by the pen posture detection unit 4102 of the spatial information processing unit 410 using the spatial position detection unit 30 is directly supplied to the association unit 416 for VR image-based processing.

[0062] The association unit 416 for VR image-based processing associates the information on the designated position of the electronic pen 10 from the selection unit 412 with the information on the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410, forms them into a pair, and supplies the pair to the VR image generation unit 422 of the display image generation unit 42. In this case, the association in the association unit 416 means associating the information on the detected pen posture at the designated position of the electronic pen 10 with the information on the designated position of the electronic pen 10 output from the selection unit 412 as a pair.

[0063] When both the information on the designated position of the electronic pen 10 and the information on the pen posture to be associated by the association unit 416 are information from the spatial information processing unit 410, the spatial information processing unit 410 adjusts the output timings of these pieces of information, so that the association unit 416 can simply associate the information at the same timing by pairing.

[0064] On the other hand, when the information on the designated position of the electronic pen 10 to be associated by the association unit 416 is information from the digitizer 20, the association unit 416 takes into account the deviation (processing delay) in the output timings of the detection results between the digitizer 20 and the spatial position detection unit 30 to associate the information on the designated position of the electronic pen 10 with the information on the pen posture.

[0065] The VR image generation unit 422 of the image generation unit 42 generates a VR image of the electronic pen 10 using the information on the indicated position of the electronic pen 10 and the information on the pen posture from this association unit 416. In this case, the VR image of the electronic pen 10 is generated so as to be displayed at a position based on the information on the indicated position of the electronic pen 10.

[0066] In this case, the information on the indicated position of the electronic pen 10 from the selection unit 412 is switched and selected by the selection control signal SE, while the information on the indicated position from the spatial position detection unit 4101 is coordinate-converted by the coordinate conversion unit 413 so as to be in a common coordinate space between the digitizer 20 and the spatial position detection unit 30. Therefore, even when the selection unit 412 is switched, there is no such display deviation that the display position of the VR image of the electronic pen 10 moves as if it jumps.

[0067] The information on the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410 is also supplied to the association unit 415 for 3D drawing system processing through the posture conversion unit 417. The posture conversion unit 417 is supplied with the information on the pen posture from the pen posture detection unit 4102 and the information on the posture of the digitizer 20 detected by the digitizer posture detection unit 4103. In the posture conversion unit 417, the pen posture of the electronic pen 10 in the spatial coordinate system detected by the spatial position detection unit 30, which was described with reference to FIG. 3, is converted into the relative pen posture with respect to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0068] The association unit 415 for 3D drawing system processing associates the information on the indicated position of the electronic pen 10 from the selection unit 411 and the information on the pen posture detected by the pen posture detection unit 4102 from the posture conversion unit 417, which has been converted into the information on the relative pen posture with respect to the input surface 21S of the digitizer 20, as a pair and supplies it to the drawn image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42. In this case, the association in the association unit 415 also means associating, as a pair, the information on the pen posture detected at the indicated position of the electronic pen 10 with respect to the indicated position of the electronic pen 10 output from the selection unit 412, similar to the association unit 416.

[0069] Based on the detection output of the indicated position of the electronic pen 10 from the digitizer 20 and the information on the pen posture detected by the spatial position detection unit 30 in this embodiment, the drawn image generation unit 421 has a pen drawing function for drawing a detailed line drawing or the like, and a gesture processing function for performing a drawing process based on the movement (gesture) detected by the gesture detection processing unit 423 based on the spatial position and pen posture of the electronic pen 10 detected by the spatial position detection unit 30.

[0070] And a selection control signal SE from the selection control signal generation unit 414 is supplied to the drawn image generation unit 421 and the gesture detection processing unit 423. The gesture detection processing unit 423 is controlled to operate when the indicated position of the electronic pen 10 is outside the spatial region of the position detection region DT of the digitizer 20. The drawn image generation unit 421 executes the pen drawing function when the indicated position of the electronic pen 10 is within the spatial region of the position detection region DT of the digitizer 20, and is switched and controlled to execute gesture processing when the indicated position of the electronic pen 10 is outside the spatial region of the position detection region DT of the digitizer 20.

[0071] In this case, in this embodiment, since coordinate conversion is performed using the coordinate conversion unit 413 so that the digitizer 20 and the spatial position detection unit 30 have a common coordinate space, even if the indicated position of the electronic pen 10 is switched by the selection unit 411, the indicated position of the electronic pen 10 will not shift in a way that causes a jump. Also, even if the pen posture is detected by the spatial position detection unit 30, it is converted into the relative pen posture with respect to the input surface 21S of the digitizer 20, so it is appropriately reflected in the 3D drawn image.

[0072] As described above, the 3D drawing image information generated by the drawing image generation unit 421 is supplied to the VR image generation unit 422 to be made into a VR image, and is then supplied to the HMD 50 through the display drive unit 424 for display. In this case, in this embodiment, the indicated position of the electronic pen 10 is made to have the same spatial coordinates in the digitizer 20 and the spatial position detection unit 30 by using the coordinate conversion unit 413. Therefore, the spatial coordinate system of the 3D drawing image and the spatial coordinate system of the VR image such as the electronic pen 10 are the same. Thus, since the indicated position of the electronic pen 10 with respect to the 3D drawing image does not differ between the 3D drawing system processing and the VR image system processing, the VR image generation unit 422 does not need to correct the spatial coordinates of both.

[0073] As can be understood from the above description, in this first embodiment, in the 3D drawing system processing, even if the electronic pen 10 and the digitizer 20 do not have a function of detecting the pen posture such as the inclination angle and rotation angle of the electronic pen 10, the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 can be used for generating a 3D drawing image. Further, in this first embodiment, there is also an effect that the information on the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 can be used for gesture detection for generating a 3D drawing image in a spatial region outside the position detection region DT of the digitizer 20. Furthermore, in this first embodiment, since the spatial coordinates of the detection region of the digitizer 20 and the spatial coordinates of the detection region of the spatial position detection unit 30 are treated as common coordinates, there is a feature that no coordinate shift occurs even when switching between the output from the digitizer 20 and the output of the spatial position detection unit 30 in the 3D drawing system processing.

[0074] Therefore, the operator can seamlessly perform operations by gesture from fine drawing simply by spatially moving the electronic pen 10 on the digitizer 20 without being conscious of switching between the digitizer 20 and the spatial position detection unit 30.

[0075] In the above-described first embodiment, in the VR image-based processing, as the position information (tip position information) of the electronic pen 10 for generating the VR image of the electronic pen 10, in the spatial region of the position detection area DT of the digitizer 20, the position detection output of the digitizer 20 with higher accuracy and faster reaction speed than the detection output of the spatial position detection unit 30 can be used. Therefore, a VR image that accurately reacts according to the operation of the actual user's electronic pen can be obtained.

[0076] Note that since the above-described spatial drawing information generation device 40 is configured by a computer, it goes without saying that each part of the input information processing unit 41 and the display image generation unit 42 can be configured as a software function unit executed by a software program.

[0077] [Second Embodiment] In the above-described first embodiment, the electronic pen 10 and the digitizer 20 did not have a pen posture detection function such as the tilt angle or rotation angle of the electronic pen. However, it can be configured to have a pen posture detection function such as the tilt angle or rotation angle of the electronic pen.

[0078] The input device of the second embodiment described below is a case where, in a system similar to the spatial position indication system described in the above-described first embodiment, the electronic pen and the digitizer have a pen posture detection function such as the tilt angle or rotation angle of the electronic pen.

[0079] FIG. 4 shows a configuration example of the main part of the spatial position indication system of this second embodiment. The spatial position indication system of this second embodiment has the same configuration as the first embodiment, but instead of the electronic pen 10 and the digitizer 20, an electronic pen 10A (not shown in FIG. 4) and a digitizer 20A having a configuration for detecting the pen posture are provided. Then, instead of the input information processing unit 41 of the spatial drawing information generation device 40 in the first embodiment, an input information processing unit 41A having a configuration as shown in FIG. 4 is provided.

[0080] In the spatial position indication system of this second embodiment, the spatial position detection unit 30, the display image generation unit 42, and the HMD 50 described in the first embodiment are also provided. However, since they are the same as those in the first embodiment, they are not shown in FIG. 4. Further, in the digitizer 20A and the input information processing unit 41A shown in FIG. 4, the same parts as the digitizer 20 and the input information processing unit 41 shown in FIG. 2 are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

[0081] As shown in FIG. 5(A), when the electronic pen 10A is perpendicular to the input surface of the sensor unit, the signal levels of a plurality of predetermined numbers, for example, seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, Xi+3 exhibit symmetry centered on the indicated position.

[0082] On the other hand, when the electronic pen 10A is tilted, the signal levels of these seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, Xi+3 change according to the direction and angle of tilt of the electronic pen 10A as shown in FIG. 5(B). Based on these plurality of signal levels of a predetermined number, the position coordinates, tilt angle, and tilt direction of the indicated position of the electronic pen 10A can be detected.

[0083] However, in the input surface (detection area) INS of the sensor unit as shown in FIG. 6, for example, in the peripheral area PE indicated by hatching outside the position of the dotted line, only a plurality of signal levels less than the above-mentioned predetermined number can be obtained. For this reason, in this second embodiment, in the peripheral area PE of the digitizer 20A, the detection accuracy of the indicated position (pen tip position) of the electronic pen 10A and the pen posture of the electronic pen 10A is low. In the case of this second embodiment, considering that the detection accuracy of the indicated position (pen tip position) of the electronic pen 10A and the pen posture of the electronic pen 10A detected by the spatial position detection unit 30 is higher.

[0084] That is, in this second embodiment, even when the pen tip position of the electronic pen 10A is in the peripheral region PE or the space region above it within the spatial region DT of the position detection region where the indication position and pen posture of the electronic pen 10A can be detected by the digitizer 20A, instead of the information on the indication position and pen posture of the electronic pen 10A detected by the digitizer 20A, the information on the indication position and pen posture of the electronic pen 10A detected by the spatial position detection unit 30 is used.

[0085] As shown in FIG. 4, the digitizer 20A of this second embodiment includes a posture detection unit 24. This posture detection unit 24 detects the pen posture such as the tilt angle, tilt direction, and rotation angle of the electronic pen 10A based on the signal exchange between the electronic pen 10A (not shown) and the sensor unit 22 of the digitizer 20A. Then, the information on the pen posture of the electronic pen 10A detected by this posture detection unit 24 is supplied to one input terminal of the selection unit 418 of the input information processing unit 41A. To the other input terminal of this selection unit 418, the information on the pen posture of the electronic pen 10A detected by using the spatial position detection unit 30 from the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied.

[0086] The information on the pen posture of the electronic pen 10A from the selection unit 418 is supplied to the association unit 415, and the same association process as described above is performed with the information on the indication position of the electronic pen 10A from the selection unit 411, and it is supplied to the drawing image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42.

[0087] Also, in this second embodiment, the information on the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied to one input terminal of the selection unit 419. Then, the information on the pen posture of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A is supplied to the posture conversion unit 417R, and the information on the digitizer posture including the tilt angle and tilt direction of the digitizer 20A detected by the digitizer posture detection unit 4103 of the spatial information processing unit 410 is supplied to this posture conversion unit 417R.

[0088] In the posture conversion unit 417R, inverse conversion is performed from the pen posture information of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and the posture information of the digitizer 20A from the digitizer posture detection unit 4103 of the spatial information processing unit 410. That is, the relative pen posture of the electronic pen 10A detected by the digitizer 20A with respect to the input surface 21S is converted into the absolute posture of the digitizer 20A in the spatial coordinate system of the motion detection spatial region MD detected by the spatial position detection unit 30.

[0089] Then, the pen posture information of the electronic pen 10A whose posture is converted by this posture conversion unit 417R is supplied to the other input terminal of the selection unit 419. The pen posture information of the electronic pen 10A selected and output as described later by the selection unit 419 is supplied to the association unit 416, and the same association process as that of the instruction position information of the electronic pen 10A from the selection unit 412 is performed, and it is supplied to the VR image generation unit 422 of the display image generation unit 42.

[0090] The selection control signal generation unit 414A of this second embodiment generates the selection control signals SEA of the selection unit 411, the selection unit 412, the selection unit 418, and the selection unit 419. Further, the selection control signal generation unit 414A generates the same selection control signal SE as the selection control signal generation unit 414 of the first embodiment described above, and performs switching control on the processes executed by the drawn image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42 in the same manner as in the first embodiment described above.

[0091] The selection control signal SEA from the selection control signal generation unit 414A of this second embodiment not only selectively controls the selection units 411, 412, 418, and 419 depending on whether the instruction position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT of the digitizer 20A or in other spatial regions, but also selectively controls the selection units 411, 412, 418, and 419 depending on whether it is in the peripheral region PE (see FIG. 6) of the input surface 21S of the digitizer 20A or in the central region inside the peripheral region PE of the input surface 21S.

[0092] Therefore, the selection control signal generation unit 414A of the second embodiment is supplied with, from the position detection unit 23 of the digitizer 20A, not only the information on the signal level of the received signal from the electronic pen 10A but also the information on the indicated position of the electronic pen 10A. Then, when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, and when it is in other spatial regions, the selection control signal generation unit 414A generates a selection control signal SEA for selectively controlling the selection units 411, 412, 418, and 419.

[0093] When the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 411 selects the information on the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A according to the selection control signal SEA and supplies it to the association unit 415. When the indicated position (the position of the pen tip) of the electronic pen 10A is in a spatial region other than the above, the selection unit 411 selects the information on the indicated position of the electronic pen 10A from the spatial position detection unit 4101 from the coordinate conversion unit 413 and supplies it to the association unit 415.

[0094] When the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 412 selects the information on the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A according to the selection control signal SEA and supplies it to the association unit 416. When the indicated position (the position of the pen tip) of the electronic pen 10A is in a spatial region other than the above, the selection unit 412 selects the information on the indicated position of the electronic pen 10A from the spatial position detection unit 4101 that has undergone coordinate conversion processing by the coordinate conversion unit 413 and supplies it to the association unit 416.

[0095] Further, when the instruction position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE according to the selection control signal SEA, the selection unit 418 selects the pen posture information of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and supplies it to the association unit 415. When the instruction position (pen tip position) of the electronic pen 10A is within a spatial region other than the above, the selection unit 418 selects the pen posture information converted into a relative pen posture from the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the posture conversion unit 417 and supplies it to the association unit 415.

[0096] Further, when the instruction position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE according to the selection control signal SEA, the selection unit 419 selects the pen posture information converted into an absolute pen posture detected by the spatial position detection unit 30 from the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A of the posture conversion unit 417R and supplies it to the association unit 416. When the instruction position (pen tip position) of the electronic pen 10A is within a spatial region other than the above, the selection unit 419 selects the pen posture information of the electronic pen 10A from the pen posture detection unit 4102 and supplies it to the association unit 416.

[0097] An example of the processing operation flow when the input information processing unit 41A of the spatial position instruction system of this second embodiment is configured by a computer will be described with reference to FIGS. 7 and 8 which is a continuation of FIG. 7. In this case, each block constituting the input information processing unit 41A shown in FIG. 4 becomes a software functional unit by a computer program.

[0098] That is, the input information processing unit 41A determines whether or not the state has changed to detecting the indicated position of the electronic pen 10A by the digitizer 20A from the signal from the digitizer 20A (step S1). When it is determined in this step S1 that the state has not changed to detecting the indicated position of the electronic pen 10A by the digitizer 20A, the input information processing unit 41A determines whether or not the electronic pen 10A can be detected by the spatial position detection unit 30 (step S2). When it is determined in this step S2 that the electronic pen 10A cannot be detected by the spatial position detection unit 30, the input information processing unit 41A returns the process to step S1.

[0099] When it is determined in step S1 that the state has changed to detecting the indicated position of the electronic pen 10A by the digitizer 20A, the input information processing unit 41A acquires the coordinates of the indicated position by the electronic pen 10A detected by the digitizer 20A (step S3), and determines whether or not the acquired coordinates of the indicated position are within the peripheral area PE (step S4).

[0100] When it is determined in step S4 that the acquired coordinates of the indicated position are not in the peripheral area PE but in its inner area, the input information processing unit 41A associates the information on the indicated position of the electronic pen 10A from the digitizer 20A with the information on the pen posture and outputs the result to the display image generation unit 42 for generating a drawing image (step S5).

[0101] Next, the input information processing unit 41A performs coordinate conversion to make the information on the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 have the information in the coordinate system common to the digitizer 20A (step S6). Then, the input information processing unit 41A associates the information on the indicated position of the electronically pen 10A after the coordinate conversion with the information on the pen posture detected by the pen posture detection unit 4102 of the spatial information processing unit 410 and outputs the result to the display image generation unit 42 for generating a VR image (step S7). After this step S7, the input information processing unit 41A returns the process to step S1 and repeats the processes after this step S1.

[0102] Next, in step S4, when it is determined that the coordinates of the acquired instruction position are within the peripheral area PE, and also in step S2, when it is determined that the electronic pen 10A can be detected by the spatial position detection unit 30, the input information processing unit 41A performs coordinate conversion to make the information on the instruction position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 into information in the same coordinate system as the digitizer 20A (step S11 in FIG. 8). Also, the input information processing unit 41A acquires the information on the posture of the digitizer 20A detected by the digitizer posture detection unit 4103 of the spatial information processing unit 410 (step S12).

[0103] Next, the input information processing unit 41A uses the information on the posture of the digitizer 20A acquired in step S12 to convert the information on the pen posture from the digitizer posture detection unit 24 of the digitizer 20A into information on the absolute posture in the spatial region, rather than the relative posture with respect to the digitizer 20A (step S13). Then, the input information processing unit 41A associates the information on the instruction position of the electronic pen 10A obtained by coordinate conversion in step S11 with the information on the absolute posture obtained in step S13 and outputs it to the display image generation unit 42 for generating a drawing image (step S14).

[0104] Next, the input information processing unit 41A uses the information on the posture of the digitizer 20A acquired in step S12 to convert the information on the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410 into information on the relative posture with respect to the digitizer 20A (step S15). Then, the input information processing unit 41A associates the information on the instruction position of the electronic pen 10A obtained by coordinate conversion in step S11 with the information on the absolute posture obtained in step S13 and outputs it to the display image generation unit 42 for generating a VR image (step S16). After this step S16, the input information processing unit 41A returns the process to step S1 and repeats the processes after this step S1.

[0105] Since the input information processing unit 41A of the spatial position indication system according to the second embodiment is configured as described above, in the central region inside the peripheral region PE of the input surface 21S within the spatial region of the position detection region DT of the digitizer 20A, the information on the indicated position of the electronic pen 10A in 3D drawing system processing and VR image system processing is more accurate than when detected by the spatial position detection unit 30, and the information on the indicated position of the electronic pen 10A detected by the position detection unit 23 of the digitizer 20A is used. And in the peripheral region PE (including the space above the input surface 21S) of the input surface 21S within the spatial region of the position detection region DT of the digitizer 20A and the spatial region outside the spatial region of the position detection region DT, the information on the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0106] Also, as the information on the pen posture of the electronic pen 10A in 3D drawing system processing, in the central region inside the peripheral region PE of the input surface 21S within the spatial region of the position detection region DT of the digitizer 20A, the information on the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30. And in the peripheral region PE (including the space above the input surface 21S) of the input surface 21S within the spatial region of the position detection region DT of the digitizer 20A and the spatial region outside the spatial region of the position detection region DT, the information on the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0107] Therefore, also in this second embodiment, as in the first embodiment, 3D drawing system processing and VR image system processing can be performed using always highly accurate information. In this case, as in the first embodiment, in each of the 3D drawing system processing and VR image system processing, there is an advantage that the information detected by the digitizer 20A and the information detected by the spatial position detection unit 30 can be used mutually, which is convenient.

[0108] [Third Embodiment] The first and second embodiments described above are cases where the input device according to the present invention is applied to a spatial position indication system. However, the input device according to the present invention is also applicable when detecting a position indication by an electronic pen within a position detection area DT including a hover area in a tablet device equipped with a digitizer.

[0109] FIG. 9 is a diagram showing an overview of a system including the input device of this third embodiment. In FIG. 9, the input device of the third embodiment is configured by an electronic pen 10B and a tablet device 60 equipped with a digitizer. As shown in FIG. 9, the tablet device 60 of this input device is placed on, for example, a desk 90 and is connected to a computer 80. Also, in this example, the tablet device 60 is placed with, for example, an inclination angle θ with respect to the placement surface of the desk 90.

[0110] And in this third embodiment, a gyro sensor unit 70 for detecting the pen posture of the electronic pen 10B in the three-dimensional space above the input surface 61S of the digitizer built in the tablet device 60 is detachably provided on the rear end side opposite to the pen tip side of the electronic pen 10B. Note that the gyro sensor unit 70 may be built in the rear end side opposite to the pen tip side of the electronic pen 10B.

[0111] FIG. 10 is a block diagram for explaining a functional configuration example of the electronic pen 10B, the tablet device 60, and the gyro sensor unit 70 that constitute the input device of this third embodiment. In this case, the gyro sensor unit 70 is configured to include, for example, a three-axis gyro sensor 71 and a wireless communication unit 72. The wireless communication unit 72 is configured by, for example, short-range wireless communication means conforming to the Bluetooth (registered trademark) standard. The wireless communication unit 72 is not limited to this and may be optical communication such as infrared communication.

[0112] In this example, since the gyro sensor unit 70 is attached to the housing of the electronic pen 10B, the three-axis gyro sensor 71 outputs an output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B in the three-dimensional space. Here, the output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B detected by the three-axis gyro sensor 71 is not corresponding to the relative posture with respect to the input surface 61S of the tablet device 60 of the electronic pen 10B, but is corresponding to the absolute posture based on the earth axis in the three-dimensional space. The wireless communication unit 72 wirelessly transmits the output signal from the three-axis gyro sensor 71 to the tablet device 60.

[0113] The electronic pen 10B includes an interaction unit 100 with the sensor unit 61 of the tablet device 60. The interaction unit 100 is coupled to the sensor unit 61 of the tablet device 60 by either an electromagnetic induction method or an electrostatic coupling method to exchange signals. The interaction unit 100 of the electronic pen 10B in this example exchanges position detection signals of the indicated position of the electronic pen 10B with the sensor unit 61, and does not exchange signals for detecting pen postures such as the tilt angle, tilt direction, and rotation angle.

[0114] In the tablet device 60, the indicated position of the electronic pen 10B is detected by the indicated position detection unit 62 from the signal obtained as a result of the interaction with the electronic pen 10B in the sensor unit 61. The information on the indicated position of the electronic pen 10B detected by the indicated position detection unit 62 is supplied to the association unit 63.

[0115] The tablet device 60 of this third embodiment includes a wireless communication unit 64 that performs wireless communication with the wireless communication unit 72 of the gyro sensor unit 70. The detection output of the three-axis gyro sensor 71 of the gyro sensor unit 70 received by this wireless communication unit 64 is supplied to the electronic pen posture detection unit 65, and the pen posture of the electronic pen 10B is detected. The information on the pen posture of the electronic pen 10B detected by this electronic pen posture detection unit 65 is supplied to the relative posture calculation unit 66.

[0116] The tablet device 60 of this embodiment includes a gyro sensor 67 for detecting the inclination angle and inclination direction of the own device. The sensor output of this gyro sensor 67 is supplied to the tablet attitude detection unit 68. The tablet attitude detection unit 68 detects the inclination angle and inclination direction of the own device based on the sensor output of the gyro sensor 67, and supplies information on the detected inclination angle and inclination direction of the own device (tablet attitude information) to the relative attitude calculation unit 66.

[0117] The relative attitude calculation unit 66 corrects the information on the absolute pen attitude of the electronic pen 10B in the three-dimensional space from the electronic pen attitude detection unit 65 using the tablet attitude information from the tablet attitude detection unit 68, and calculates the relative pen attitude of the electronic pen 10B with respect to the input surface 61S of the tablet device 60. Then, the relative attitude calculation unit 66 supplies the calculated information on the relative pen attitude of the electronic pen 10B to the association unit 63.

[0118] The association unit 63 associates the information on the instruction position of the electronic pen 10B and the information on the relative pen attitude of the electronic pen 10B in the same manner as the above-described association unit, and outputs them to the computer 80.

[0119] As described above, in the input device of this third embodiment, when the instruction position by the pen tip of the electronic pen 10B is obtained, the gyro sensor unit 70 provided on the rear end side of the electronic pen 10B corresponding to this can output the information on the pen attitude of the electronic pen 10B detected in the three-dimensional space as the information on the pen attitude corresponding to the instruction position of the electronic pen 10B.

[0120] [Other Embodiments or Modification Examples] In the above-described first and second embodiments, the spatial position detection unit 30 is configured to include a light-emitting tracking device that emits infrared laser light and a tracker, but it goes without saying that it is not limited to this configuration. For example, a configuration using other non-visible light sensors, visible light sensors, or combinations thereof may be used.

[0121] Also, in the above-described first and second embodiments, since the HMD 50 is worn, the operator of the electronic pen 10 cannot directly see the digitizer 20. Therefore, it is also possible to draw a virtual image of the digitizer 20 on the virtual space image displayed on the HMD 50 so that the operator can recognize the position of the digitizer 20 on the display screen of the HMD 50.

[0122] Also, in the above-described first and second embodiments, in order to correct the errors in the two spatial coordinate systems, the coordinate values of the spatial position detection unit 30 are converted into the coordinate values of the spatial coordinate system of the digitizer 20. Conversely, the coordinate values of the position detection area DT of the digitizer 20 may be converted into the coordinate values of the spatial position detection unit 30. Note that the conversion between these two spatial coordinate systems is not essential and is optional.

[0123] In the above-described first and second embodiments, the electronic pen and the digitizer use the electromagnetic induction method, but the present invention is not limited thereto, and it is of course possible to use an electrostatic method (including an active electrostatic coupling method and a passive electrostatic coupling method) for the electronic pen and the digitizer.

[0124] Also, the digitizer in the first to second embodiments and the tablet device in the third embodiment may be a portable mobile phone terminal called a so-called smartphone.

[0125] In the first and second embodiments, the case of performing 3D drawing by the spatial position indication system has been described, but the drawn image may be a 2D drawn image or a 2.5D drawn image.

Description of Reference Numerals

[0126] 10, 10A, 10B... electronic pen, 20, 20A... digitizer, 21S... input surface, 22... sensor unit, 23... position detection unit, 24... attitude detection unit, 30... spatial position detection unit, 40... spatial drawing information generation device, 41... input information processing unit, 4101... spatial position detection unit, 4102... pen attitude detection unit, 50... HMD

Claims

1. An apparatus having a digitizer for detecting an indicated position by an electronic pen, A tracking device for detecting the position of a tracker of the electronic pen, the tracking device for performing position input by the electronic pen in a VR (Virtual Reality) space, A computer that generates a VR image related to the electronic pen in the VR space based on the posture of the electronic pen in the three-dimensional space where the electronic pen exists detected based on the tracking of the tracker of the electronic pen by the tracking device and the indicated position by the electronic pen detected by the apparatus having the digitizer, A VR image generation system characterized by comprising the above.

2. The computer calculates coordinates corresponding to the indicated position by the electronic pen in the first coordinate system of the apparatus having the digitizer based on the indicated position by the electronic pen detected by the apparatus having the digitizer, and calculates data related to the posture of the electronic pen in the second coordinate system of the VR space different from the first coordinate system of the tablet based on the position of the tracker of the electronic pen detected by the tracking device. The VR image generation system according to claim 1, characterized by the above.

3. The first coordinate system of the apparatus having the digitizer is a coordinate system related to the position detection area of the apparatus having the digitizer, and the second coordinate system of the VR space is a coordinate system related to the position detection area of the tracking device. The VR image generation system according to claim 2, characterized by the above.

4. The computer causes a display unit to display a VR image related to the electronic pen in the VR space. The VR image generation system according to claim 1, characterized by the above.

5. When the indicated position by the electronic pen exists within a predetermined distance from the input surface of the apparatus having the digitizer, the computer calculates coordinates in the first coordinate system of the apparatus having the digitizer. The VR image generation system according to claim 2, characterized by the above.

6. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light. The VR image generation system according to claim 1, characterized by the above.

7. The VR space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space. The VR image generation system according to claim 1, characterized in that.

8. The computer calculates the coordinates in the second coordinate system of the VR space by correcting the coordinates in the first coordinate system of the device having the digitizer. The VR image generation system according to claim 2, characterized in that.

9. The computer generates a VR image related to the electronic pen in the VR space based on the coordinates in the second coordinate system of the VR space and the data related to the posture of the electronic pen in the second coordinate system of the VR space. The VR image generation system according to claim 8, characterized in that.

10. The data related to the posture of the electronic pen includes the tilt angle, tilt direction or rotation angle of the electronic pen. The VR image generation system according to claim 2, characterized in that.

11. A tracking device that detects the indicated position by the electronic pen detected by the device having the digitizer and the position of the tracker of the electronic pen, and generates a VR image related to the electronic pen in the VR space based on the posture of the electronic pen in the three-dimensional space where the electronic pen exists detected based on the tracking of the tracker of the electronic pen for performing position input by the electronic pen in the VR (Virtual Reality) space. The VR image generation method characterized by that.

12. Based on the indicated position by the electronic pen detected by the device having the digitizer, calculate the coordinates corresponding to the indicated position by the electronic pen in the first coordinate system of the device having the digitizer, and calculate the data related to the posture of the electronic pen in the second coordinate system of the VR space different from the first coordinate system of the tablet based on the position of the tracker of the electronic pen detected by the tracking device. The VR image generation method according to claim 11, characterized in that.

13. The first coordinate system of the device having the digitizer is a coordinate system related to the position detection area of the device having the digitizer, and the second coordinate system of the VR space is a coordinate system related to the position detection area of the tracking device. The VR image generation method according to claim 12, characterized in that.

14. Causing a display unit to display a VR image related to the electronic pen in the VR space The VR image generation method according to claim 11, characterized in that

15. When an instruction position by the electronic pen exists within a predetermined distance from an input surface of the device having the digitizer, calculating coordinates in a first coordinate system of the device having the digitizer The VR image generation method according to claim 12, characterized in that

16. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light The VR image generation method according to claim 11, characterized in that

17. The VR space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space The VR image generation method according to claim 11, characterized in that

18. Calculating coordinates in a second coordinate system of the VR space by correcting coordinates in a first coordinate system of the device having the digitizer The VR image generation method according to claim 12, characterized in that

19. Generating a VR image related to the electronic pen in the VR space based on coordinates in a second coordinate system of the VR space and data related to the posture of the electronic pen in the second coordinate system of the VR space The VR image generation method according to claim 18, characterized in that

20. The data related to the posture includes the tilt angle, tilt direction, or rotation angle of the electronic pen The VR image generation method according to claim 12, characterized in that

Images