Image generation method, computer, and image generation program

Abstract

To provide an image generating method, a computer and an image generating program that generates a Virtual Reality (VR) image in a VR space using an electronic pen.SOLUTION: In a spatial position designating system, a spatial drawing information generating device 40 includes: an input information processing unit that includes a spatial position detecting unit which calculates a relative attitude of an electronic pen with respect to a device that includes a digitizer 20 which detects a designated position of the electronic pen 10; and a display image generating unit that generates a VR image in a VR space using the calculated relative attitude of the electronic pen.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an input device suitable for use with an electronic pen in space. [Background technology]

[0002] 2. Description of the Related Art There is known a drawing system that creates animation images and the like by drawing by continuously specifying positions with an electronic pen on a coordinate input device called a digitizer.

[0003] Conventionally, various methods have been proposed as a method for detecting the tilt and rotation of an electronic pen. Patent Document 1 (JP 2016-126503 A) provides an electronic pen capable of detecting the tilt angle and rotation angle of a sensor unit of a position detection device relative to an input surface. In the case of Patent Document 1, a signal is transmitted between the electronic pen and a position detection device having a sensor unit by electromagnetic induction coupling or electrostatic capacitance coupling, so that the position detection device detects the position indicated by the electronic pen and detects the position coordinates of the indicated position of the electronic pen, as well as the tilt angle and rotation angle.

[0004] Furthermore, Patent Document 2 (U.S. Patent Publication 9,329,703 B2) describes how a sensor capable of measuring the movement and orientation of an electronic pen is built into or attached to the electronic pen, and the detection output of the sensor is transmitted to a position detection device, enabling the position detection device to detect the movement and state (tilt and rotation) of the electronic pen. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-126503 A [Patent Document 2] U.S. Patent Publication No. 9,329,703B2 Summary of the Invention [Problem to be solved by the invention]

[0006] In the case of the above-mentioned Patent Document 1, the position coordinates and tilt of the position pointed to by the electronic pen are detected using multiple signal levels obtained from multiple loop coils around the position pointed to by the electronic pen. However, in the peripheral area of ​​the sensor unit, there is a problem that the accuracy of the position coordinates of the position pointed to by the electronic pen decreases and it becomes difficult to detect the tilt of the electronic pen.

[0007] The method of Patent Document 2 does not have the problems of Patent Document 1. However, depending on the conditions, there are problems such as a slow reaction speed and a decrease in the accuracy of position coordinates.

[0008] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide an input device that can solve the above problems. [Means for solving the problem]

[0009] In order to solve the above problems, Calculating a relative orientation of the electronic pen with respect to a device having a digitizer that detects a position pointed to by the electronic pen; The region within the input surface of the device having a digitizer includes a peripheral region where the detection accuracy by the device having a digitizer is relatively low and a region inside the peripheral region where the detection accuracy by the device having a digitizer is relatively high. Central Territory and generating a virtual reality (VR) image of the electronic pen in a VR space using the pointing position detected by the digitizer and the relative orientation of the electronic pen associated with the pointing position when the pointing position by the electronic pen is present in the central region. When a pointing position by the electronic pen is present in a region different from the central region, a VR image of the electronic pen in the VR space is generated using the pointing position detected by the spatial position detection unit, which is located above the digitizer and tracks a tracker of the electronic pen, and the absolute attitude of the electronic pen associated with the pointing position. do, The present invention provides an image generating method comprising the steps of:

[0010] In the image forming method having the above-mentioned configuration, a relative attitude of the electronic pen with respect to a device having a digitizer that detects a position pointed to by the electronic pen is calculated, and the calculated relative attitude of the electronic pen is used to generate a virtual reality (VR) space. Electronic penGenerate VR images. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram for explaining an example of the configuration of a spatial position instruction system including a first embodiment of an input device according to the present invention; [Diagram 2] 2 is a block diagram for explaining an example of the configuration of each unit of the spatial position indication system of the example of FIG. 1. FIG. [Diagram 3] 1 is a diagram for explaining an embodiment of an input device according to the present invention; [Figure 4] FIG. 4 is a block diagram for explaining a configuration example of each part of an input device according to a second embodiment of the present invention. [Diagram 5] 11 is a diagram for explaining an operation of detecting the tilt of the electronic pen by transmitting and receiving signals between the electronic pen and a digitizer. FIG. [Figure 6] 11 is a diagram used to explain that the detection accuracy when the digitizer detects the pointing position and inclination of the electronic pen by sending and receiving signals with the electronic pen differs between the peripheral area and other areas of the input surface. [Figure 7] FIG. 11 is a diagram showing a part of a flowchart for explaining a main operation of the second embodiment of the input device according to the present invention. [Figure 8] FIG. 11 is a diagram showing a part of a flowchart for explaining a main operation of the second embodiment of the input device according to the present invention. [Figure 9] FIG. 13 is a diagram for explaining a configuration example of a third embodiment of an input device according to the present invention. [Figure 10] FIG. 13 is a block diagram for explaining a configuration example of each part of a third embodiment of an input device according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of an input device according to the present invention will be described with reference to the drawings.

[0013] [First embodiment] The embodiment of the input device described below relates to a spatial position indication system that includes a digitizer (pen tablet) that includes a sensor unit that transmits and receives signals from an electronic pen (hereinafter referred to as "signal transmission and reception"), and a spatial position detection unit that detects information on a position indicated by the electronic pen in space. Note that the position includes not only position coordinates but also attitude (tilt and rotation), but in the following description, position coordinates and attitude information are described separately in order to clarify the two.

[0014] In the spatial position instruction system according to this embodiment, the display unit is composed of a head-mounted display, smart glasses, etc., and a 3D rendering space is used as a virtual reality (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc., hereinafter abbreviated as VR) space. FIG. 1 is a diagram showing an outline of the overall configuration of a spatial position instruction system including an input device of the first embodiment in which a VR space is used as a 3D rendering space. FIG. 2 is a block diagram showing a detailed configuration example of the functions of each part of the spatial position instruction system including the input device of the first embodiment.

[0015] That is, as shown in Fig. 1, the spatial position indication system of the embodiment of this example is configured to include an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generating device 40, and a head mounted display (hereinafter referred to as HMD) 50. As shown in Fig. 2, in this example, the spatial drawing information generating device 40 has the functions of an input information processing unit 41 and a display image generating unit 42, and is configured, for example, by a computer.

[0016] The input device of the first embodiment includes a first detection unit that detects a position pointed to by the electronic pen 10 in response to transmission of a signal to and from the electronic pen 10, a second detection unit that detects a position pointed to by the electronic pen 10 in a three-dimensional space in which the electronic pen 10 exists, and a control unit that generates position information of the electronic pen 10 based on these pointed positions. In the first embodiment, the first detection unit is configured by the digitizer 20, the second detection unit is configured by the spatial position detection unit 30 and the spatial information processing unit 410 of the input processing unit 41, and the control unit is configured by the input information processing unit.

[0017] In the first embodiment, an electromagnetic induction type electronic pen is used as an example of electronic pen 10, but the electronic pen is not limited to the electromagnetic induction type, and electrostatic coupling type, etc. may also be used. Digitizer 20 has a thin rectangular parallelepiped housing 21, the surface of which serves as input surface 21S for position indication by electronic pen 10. Digitizer 20 also has a sensor unit 22 and a position detection unit 23 (see FIG. 2).

[0018] Although not shown in the figure, the sensor unit 22 is configured by arranging a plurality of loop coils in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) of the housing of the digitizer 20. In this example, the digitizer 20 is of an electromagnetic induction type in accordance with the electronic pen 10, but is not limited to the electromagnetic induction type.

[0019] Although not shown in the figure, the electronic pen 10 has a resonant circuit (not shown) on the pen tip side consisting of a coil and a capacitor, and signals are exchanged between the electronic pen 10 and the sensor section 22 of the digitizer 20 by electromagnetic induction coupling between the loop coil of the sensor section 22 of the digitizer 20 and the resonant circuit of the electronic pen 10.

[0020] The position detection unit 23 of the digitizer 20 supplies a signal to the electronic pen 10 through the loop coil of the sensor unit 22, and receives a signal returned from the electronic pen 10 through the loop coil, and detects a position indicated by the electronic pen 10 in the detection area of ​​the sensor unit 22 based on the received signal. Note that in this embodiment, the digitizer 20 is configured to detect a position indicated by the tip of the electronic pen 10 as the indicated position of the electronic pen 10.

[0021] In the digitizer 20 of this example, a plurality of loop coils of the sensor section 22 are arranged so as to cover almost the entire area of ​​the input surface 21S.

[0022] In this embodiment, the position detection area in which the digitizer 20 can detect the pointing position of the electronic pen 10 includes not only the planar area when the tip of the electronic pen 10 is in contact with the input surface 21S of the digitizer 20, but also a spatial area (hover area in the hover state of the electronic pen 10) in which the tip of the electronic pen 10 is not in contact with the input surface 21S of the digitizer 20 and is separated from the input surface 21S in a direction perpendicular to the input surface 21S (the Z-axis direction perpendicular to the X-axis and Y-axis directions) but in which the pointing position of the electronic pen 10 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0023] In this example, the spatial position detection unit 30 is configured to set a three-dimensional spatial region in which the digitizer 20 exists, and detect in that three-dimensional spatial region the position indicated by the tip of the electronic pen 10, the attitude of the electronic pen 10 (referred to as pen attitude), such as the tilt angle and rotation angle of the electronic pen 10, as well as the position of the digitizer 20 and the tilt angle and tilt direction from the horizontal plane.

[0024] The spatial position detection unit 30 includes two light-emitting tracking devices 31A and 31B and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. In this embodiment, as described later, the HMD 50 displays a 3D drawn image drawn in a motion detection space area MD including a position detection area DT of the digitizer 20 as a virtual display image, and also displays a virtual display image of the electronic pen 10.

[0025] The two light emitting tracking devices 31A, 31B have the same configuration and each includes a laser emitting unit of infrared laser light, a search means for searching the motion detection space region MD using the emitted infrared laser light, and a light position detection means for detecting the light emission of the light emitting units of the trackers 32A, 32B, 32C, 32D that receive the infrared laser light.

[0026] In this embodiment, the objects (search objects in the search space area of ​​the spatial position detection unit 30) to which the trackers 32A, 32B, 32C, and 32D are attached are the electronic pen 10 and the digitizer 20, as described above. That is, in this example, in order to be able to notify the position of the digitizer 20 and the tilt angle and tilt direction with respect to the horizontal plane, the trackers 32A and 32B are attached to the upper left corner and lower right corner of the housing of the digitizer 20, which is a thin rectangular parallelepiped. In addition, in order to notify the position and pen attitude (tilt angle and rotation angle) of the electronic pen 10, the tracker 32C is attached to the pen tip side of the electronic pen 10, and the tracker 32D is attached to the rear end side opposite to the pen tip side in the axial direction of the housing of the electronic pen 10.

[0027] The light emitting tracking devices 31A and 31B control the laser emitting unit by the search means, and emit infrared laser light to search and scan the motion detection space region MD to detect the tracker position. Each of the trackers 32A, 32B, 32C, and 32D monitors the reception of infrared laser light with a sensor, and lights up a light emitting unit consisting of an LED when the sensor detects the reception of infrared laser light.

[0028] The light-emitting tracking devices 31A and 31B detect the position of the object to which the tracker 32A, 32B, 32C, or 32D is attached in the motion detection space region MD by detecting the light emitted by the light-emitting part of the tracker 32A, 32B, 32C, or 32D. The light-emitting tracking devices 31A and 31B are configured to detect the time elapsed from the time of the infrared laser emission when the light-emitting part of the tracker 32A, 32B, 32C, or 32D is detected. In this case, each of the trackers 32A, 32B, 32C, or 32D emits light differently according to its own identification information.

[0029] The two light-emitting tracking devices 31A, 31B are connected to the spatial drawing information generating device 40 via a wired or wireless connection, and notify the spatial drawing information generating device 40 of spatial position information of the detected trackers 32A, 32, 32C, 32D in the motion detection spatial region MD.

[0030] In this example, spatial position information in the motion detection space region MD of the trackers 32A, 32B, 32C, and 32D detected by the two light-emitting tracking devices 31A and 31B is supplied to a spatial information processing unit 410 of the input information processing unit 41 of the spatial drawing information generating device 40, as shown in Figure 2.

[0031] In this example, the spatial information processing unit 410 includes a spatial position detection unit 4101, a pen attitude detection unit 4102, and a digitizer attitude detection unit 4103. The spatial position detection unit 4101 detects the position of the digitizer 20 in the motion detection space region MD from the trackers 32A and 32B, and detects the position indicated by the electronic pen 10 as the position of the pen tip of the electronic pen 10 and the position of the rear end of the electronic pen 10 from the trackers 32C and 32D. In this example, the pen attitude detection unit 4102 detects the pen attitude including the tilt angle and rotation angle of the electronic pen 10 from the trackers 32C and 32D. Moreover, the digitizer attitude detection unit 4103 detects the position indicated by the electronic pen 10 as the tip position.

[0032] In this embodiment, the input information processing unit 41 generates information to be supplied to the display image generating unit 42 from information on the pointing position by the electronic pen 10 in the position detection area DT detected by the digitizer 20, information on the position of the electronic pen 10 (pointing position and rear end position) in the motion detection space area MD detected by the spatial information processing unit 410 of the input information processing unit 41, pen attitude information, and digitizer attitude information. The input information processing unit 41 then supplies the generated information to the display image generating unit 42.

[0033] In this embodiment, the display image generation unit 42 of the spatial drawing information generating device 40 includes a drawing image generation unit 421 for generating a 3D drawing image, and a VR image generation unit 422 for generating a VR image to be displayed on the HMD 50, as shown in FIG. 2.

[0034] The drawn image generating unit 421 generates a 3D drawn image based on a position indication by the electronic pen 10, and performs processes such as deformation, rotation, and movement of the 3D drawn image based on a gesture performed by the operator of the electronic pen 10. In this embodiment, the drawn image generating unit 421 also reflects the pen attitude including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 with respect to the input surface 21S of the digitizer 20 in the generation of the 3D drawn image. For example, when the electronic pen 10 exchanges signals with the digitizer 20, the tilt angle and tilt direction of the digitizer 20 with respect to the input surface 21S are used. Hereinafter, processing related to a 3D drawn image is referred to as 3D drawing processing.

[0035] In this embodiment, the spatial position detection unit 30 can detect the pen attitude of the electronic pen 10, including the tilt angle, tilt direction, and rotation angle, using trackers 32C and 32D attached to the electronic pen 10. Therefore, in this embodiment, the input information processing unit 41 is configured to supply information on the pen attitude of the electronic pen 10 detected by the spatial position detection unit 30 to the display image generation unit 42 as information for 3D drawing processing.

[0036] That is, as described later, the input information processing unit 41 is configured to supply information on the position instruction by the electronic pen 10 (information on the position of the pen tip) and information on the pen posture of the electronic pen 10 to the display image generating unit 42 so that information on the pen posture, such as the tilt and rotation of the electronic pen 10 at the time of drawing input based on the position instruction by the electronic pen 10, is reflected in the 3D drawing image.

[0037] 2, the display image generating unit 42 is provided with a gesture detection processing unit 423 for detecting a gesture performed by the operator of the electronic pen 10. In this embodiment, the input information processing unit 41 is configured to supply the position information of the pen tip of the electronic pen 10 detected in the spatial position detecting unit 30 (corresponding to the information of the indicated position) and the pen attitude information to the gesture detection processing unit 423.

[0038] As described above, in this embodiment, not only information on the pointing position by the electronic pen 10 detected by the digitizer 20 is used as information for 3D drawing system processing, but also information on the pointing position (position of the pen tip) of the electronic pen 10 detected by the spatial position detection unit 30 and information on the pen posture.

[0039] The VR image generating unit 422 of the display image generating unit 42 generates a VR image to be displayed on the HMD 50. In this embodiment, the VR images include a VR image of the electronic pen 10 and a VR image of a 3D drawn image generated by the drawn image generating unit 421, and the VR image of the 3D drawn image generated by the drawn image generating unit 421 and the VR image of the electronic pen 10 are displayed in 3D on the HMD 50. Note that the VR image generating unit 422 may also generate a VR image of the digitizer 20. Hereinafter, processing related to the generation of a VR image in the VR image generating unit 422 is referred to as VR image-related processing.

[0040] In this embodiment, the VR image generation unit 422 is supplied with information for generating a VR image of the electronic pen 10 from the input information processing unit 41, and is also supplied with information of the generated 3D drawn image from the drawn image generation unit 421. The VR image information generated by the VR image generation unit 422 is then supplied to the HMD 50 via the display drive unit 424, and is displayed on a display screen of the HMD 50, for example, formed of an LCD (Liquid Crystal Display).

[0041] In this case, when the electronic pen 10 is present in the position detection area DT, information on the position indicated by the electronic pen from the digitizer 20, which can detect the position (pen tip position) of the electronic pen 10 with higher accuracy than the spatial position detection unit 30, and information on the pen posture such as tilt and rotation detected using the spatial position detection unit 30 are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0042] In addition, when the electronic pen 10 is present in the motion detection space region MD, the digitizer 20 cannot detect the pointing position of the electronic pen 10, so information on the pointing position by the electronic pen detected using the spatial position detection unit 30 and information on the pen posture are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0043] As described above, in this embodiment, information for VR image processing is used not only information on the pointing position (pen tip position) of the electronic pen 10 detected by the spatial position detection unit 30 and information on the pen posture, but also information on the pointing position by the electronic pen 10 detected by the digitizer 20.

[0044] The input information processing unit 41 generates information for drawing system processing and information for VR image system processing as described above from information from the digitizer 20 and information from the spatial position detection unit 30, and supplies them to the display image generation unit 42.It also generates a switching control signal SE for selection control depending on whether the electronic pen 10 is located in the position detection area DT or the motion detection space area MD.

[0045] As described above, in this embodiment, the information of the spatial coordinate system of the position detection region DT of the digitizer 20 and the information of the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 can be mutually used in each of the 3D drawing system processing and the VR image system processing. In this case, the two spatial coordinate systems can be set independently of each other, but in this embodiment, the information of one of the two spatial coordinate systems is converted into the information of the other spatial coordinate system, so that they can be handled as information in a common coordinate space. In this embodiment, the information of the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 is converted into the information of the spatial coordinate system of the position detection region DT of the digitizer 20.

[0046] In the spatial position indication system of this embodiment, information on the pen posture of the electronic pen 10 detected in the motion detection space domain MD of the spatial position detection unit 30 can be used in 3D drawing processing as well as in VR image processing.

[0047] In this case, in the VR image processing, the information on the pen attitude of the electronic pen 10 is reflected in the attitude of the VR image of the electronic pen 10. The information on the pen attitude of the electronic pen 10 includes information on the tilt angle of the electronic pen 10, but in the 3D drawing processing, the tilt angle of the electronic pen 10 is a relative tilt angle with respect to the input surface of the sensor section 22 of the digitizer 20. On the other hand, the information on the pen attitude of the electronic pen 10 detected in the motion detection space region MD of the spatial position detection unit 30 is, for example, a tilt angle in the motion detection space region MD of the spatial position detection unit 30 based on the direction of gravity of the earth or the direction of the horizontal plane.

[0048] If digitizer 20 can be installed so that the direction perpendicular to input surface 21S of sensor section 22 accurately coincides with the Z-axis direction of motion detection space area MD of spatial position detection unit 30, the tilt angle of electronic pen 10 detected in the space of motion detection space area MD of spatial position detection unit 30 coincides with the relative tilt angle with respect to the input surface of sensor section 22 of digitizer 20. However, in practice, digitizer 20 may be installed at an angle with respect to motion detection space area MD of spatial position detection unit 30.

[0049] Therefore, in this embodiment, the absolute tilt angle of the electronic pen 10 detected in the space of the motion detection space area MD of the spatial position detection unit 30 is converted into a relative tilt angle with respect to the input surface of the sensor section 22 of the digitizer 20 as described below.

[0050] Fig. 3 is a diagram used to explain the conversion process of the tilt angle. Fig. 3 shows a spherical coordinate system in which the pen tip of electronic pen 10 is set as point O, which is the center of the sphere. The XY plane (a plane including the mutually orthogonal X-axis and Y-axis directions) in Fig. 3 is set as a horizontal plane in the spatial domain of the motion detection spatial domain MD of spatial position detection unit 30.

[0051] In FIG. 3, the point indicating the position of the edge of the rear end of the electronic pen 10 on the opposite side to the pen tip in the axial direction of the housing of the electronic pen 10 is expressed by spherical coordinates. In this case, the radius r of the sphere is the length of the housing of the electronic pen 10 in the axial direction. In the spherical coordinate system of FIG. 3, when the edge of the rear end of the housing of the electronic pen 10 in the axial direction is at point P, the tilt angle and tilt direction of the electronic pen 10 in the space of the motion detection space region MD can be detected as δ and α by using the spatial position detection unit 30. In addition, by using the spatial position detection unit 30, in the example of FIG. 3, the tilt angle of the housing of the digitizer 20 in the space of the motion detection space region MD can be detected as δ' and the tilt direction as α'.

[0052] As shown in Figure 3, if the position of the rear edge of the electronic pen 10 when the axial direction of the housing of the electronic pen 10 is aligned with the vertical direction within the space of the motion detection spatial region MD is point Q, then the position of the rear edge of the electronic pen 10 when the axial direction of the housing of the electronic pen 10 is perpendicular to the input surface 21S of the sensor section 22 of the digitizer 20 is point R.

[0053] Therefore, by calculating the angle between the spatial vector OR from point O to point R and the spatial vector OP from point O to point P, it is possible to use this angle to convert the tilt angle and tilt direction of the electronic pen 10 detected using the spatial position detection unit 30 into the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor section 22 of the digitizer 20.

[0054] Furthermore, by performing the above-mentioned inverse conversion, the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor section 22 of the digitizer 20 can be converted into the tilt angle and tilt direction of the electronic pen 10 detected using the spatial position detection unit 30.

[0055] 2 configured to realize the above will be described below. That is, the position detection unit 23 constituting the digitizer 20 supplies a detection output of the pointing position of the electronic pen 10 to the selection unit 411 as one of its input signals, and also supplies the same to the selection unit 412 as one of its input signals. Note that the information supplied from the position detection unit 23 to the selection units 411 and 412 includes writing pressure information applied to the electronic pen 10 in addition to the detection output of the pointing position of the electronic pen 10.

[0056] Furthermore, the spatial position detection unit 4101 of the spatial information processing unit 410 supplies a detection output of the spatial position of the pen tip of the electronic pen 10 (position pointed to by the electronic pen) to the coordinate conversion unit 413. This coordinate conversion unit 413 converts the information of the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 into information of the spatial coordinate system of the position detection region DT of the digitizer 20. Then, the coordinate conversion unit 413 supplies the converted coordinate output to the selection units 411 and 412 as the other input signal.

[0057] In this embodiment, the position detection unit 23 of the digitizer 20 supplies information on the signal level of the signal received from the electronic pen 10 to the selection control signal generation unit 414. The selection control signal generation unit 414 detects the distance between the tip of the electronic pen 10 and the input surface 21S of the digitizer 20 from the signal level of the signal received from the electronic pen 10, and generates a selection control signal SE based on the detected distance from the input surface 21S.

[0058] In this case, the selection control signal SE is a signal that controls the selection of the position detection output from the position detection unit 23 of the digitizer 20 when the distance between the tip of the electronic pen 10 and the input surface 21S of the digitizer 20 is less than the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen, and controls the selection of the detection output from the spatial position detection unit 4101 when the distance is greater than the critical height Lz.

[0059] The selection unit 411 switches between one input and the other input in response to a selection control signal SE, and supplies the input to an associating unit 415 for 3D rendering processing. The selection unit 412 switches between one input and the other input in response to a selection control signal SE, and supplies the input to an associating unit 416 for VR image processing.

[0060] In the example of Figure 2, two selection units 411 and 412 are provided for each type of processing to clearly distinguish between 3D drawing processing and VR image processing. In reality, however, a single common selection unit is sufficient, and the output of the common selection unit can be configured to be supplied to each of the association units 415 and 416.

[0061] The pen attitude information detected by the pen attitude detection section 4102 of the spatial information processing section 410 using the spatial position detection unit 30 is supplied as is to an association section 416 for VR image processing.

[0062] The associating unit 416 for VR image processing associates information on the pointing position of the electronic pen 10 from the selection unit 412 with information on the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410, and supplies the information as a pair to the VR image generation unit 422 of the display image generation unit 42. In this case, the association in the associating unit 416 means associating information on the pen posture detected at the pointing position of the electronic pen 10 with the pointing position of the electronic pen 10 output from the selection unit 412, as a pair.

[0063] When the information on the pointing position of the electronic pen 10 and the information on the pen posture that are associated by the association unit 416 are both information from the spatial information processing unit 410, the output timing of the information is adjusted by the spatial information processing unit 410, and the association unit 416 can simply associate the information with the same timing by pairing them.

[0064] On the other hand, when the information on the pointing position of the electronic pen 10 associated by the association unit 416 is information from the digitizer 20, the association unit 416 associates the information on the pointing position of the electronic pen 10 with the information on the pen posture by taking into account the difference (processing delay) in the output timing of the detection results between the digitizer 20 and the spatial position detection unit 30.

[0065] The VR image generating unit 422 of the display image generating unit 42 generates a VR image of the electronic pen 10 using the information on the pointing position and the information on the pen attitude of the electronic pen 10 from the associating unit 416. In this case, the VR image of the electronic pen 10 is generated so as to be displayed at a position based on the information on the pointing position of the electronic pen 10.

[0066] In this case, the information on the pointing position of the electronic pen 10 from the selection unit 412 is switched and selected by the selection control signal SE, but the information on the pointing position from the spatial position detection unit 4101 is coordinate converted by the coordinate conversion unit 413 so that it becomes a coordinate space common to the digitizer 20 and the spatial position detection unit 30. Therefore, even when the selection unit 412 is switched, there is no display shift, such as the display position of the VR image of the electronic pen 10 jumping.

[0067] The pen attitude information from the pen attitude detection unit 4102 of the spatial information processing unit 410 is also supplied to an associating unit 415 for 3D drawing system processing through an attitude conversion unit 417. The attitude conversion unit 417 is supplied with the pen attitude information from the pen attitude detection unit 4102 and the attitude information of the digitizer 20 detected by the digitizer attitude detection unit 4103, and executes a process of converting the pen attitude of the electronic pen 10 in the spatial coordinate system detected by the spatial position detection unit 30 described with reference to Fig. 3 into a pen attitude relative to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0068] The associating unit 415 for 3D drawing processing associates and pairs information on the pointing position of the electronic pen 10 from the selection unit 411 with information from the attitude conversion unit 417, which is information on the pen attitude detected by the pen attitude detection unit 4102 and converted into information on the pen attitude relative to the input surface 21S of the digitizer 20, and supplies the paired information to the drawn image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42. In this case, the association in the associating unit 415, like the associating unit 416, means associating the pointing position of the electronic pen 10 output from the selection unit 412 with information on the pen attitude detected at the pointing position of the electronic pen 10 as a pair.

[0069] The drawn image generation unit 421 has a pen drawing function that draws fine line drawings, etc. based on the detection output of the pointing position of the electronic pen 10 from the digitizer 20 and, in this embodiment, information on the pen posture detected by the spatial position detection unit 30, and a gesture processing function that performs drawing processing based on the movement (gesture) detected by the gesture detection processing unit 423 based on the spatial position and pen posture of the electronic pen 10 detected by the spatial position detection unit 30.

[0070] A selection control signal SE is supplied to the drawn image generation unit 421 and the gesture detection processing unit 423 from the selection control signal generation unit 414, and the gesture detection processing unit 423 is controlled to operate when the pointing position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20, and the drawn image generation unit 421 is controlled to switch so as to execute a pen drawing function when the pointing position of the electronic pen 10 is within the spatial area of ​​the position detection area DT of the digitizer 20, and to execute gesture processing when the pointing position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20.

[0071] In this embodiment, the coordinate conversion unit 413 converts coordinates so that the digitizer 20 and the spatial position detection unit 30 have a common coordinate space, so that the pointing position of the electronic pen 10 does not jump or shift even if the pointing position of the electronic pen 10 is switched by the selection unit 411. Furthermore, even if the pen attitude is detected by the spatial position detection unit 30, it is converted into a pen attitude relative to the input surface 21S of the digitizer 20, and is appropriately reflected in the 3D drawn image.

[0072] As described above, the 3D drawn image information generated by the drawn image generating unit 421 is supplied to the VR image generating unit 422 and made into a VR image, and is then supplied to the HMD 50 via the display drive unit 424 and displayed. In this case, in this embodiment, the pointing position of the electronic pen 10 is set to a common spatial coordinate system between the digitizer 20 and the spatial position detection unit 30 by using the coordinate conversion unit 413, so that the spatial coordinate system of the 3D drawn image and the spatial coordinate system of the VR image such as the electronic pen 10 are the same. Therefore, the pointing position of the electronic pen 10 with respect to the 3D drawn image does not differ between the 3D drawing system processing and the VR image system processing, so the VR image generating unit 422 does not need to correct the spatial coordinates of both.

[0073] As can be seen from the above description, in the first embodiment, even if the electronic pen 10 and the digitizer 20 do not have a function for detecting the pen attitude such as the tilt angle or rotation angle of the electronic pen 10 in the 3D drawing system processing, the pen attitude of the electronic pen 10 detected by the spatial position detection unit 30 can be used for generating a 3D drawn image. In addition, in the first embodiment, there is also an effect that the information on the pen attitude of the electronic pen 10 detected by the spatial position detection unit 30 in a spatial area other than the position detection area DT of the digitizer 20 can be used for gesture detection for generating a 3D drawn image. Furthermore, in the first embodiment, the spatial coordinates of the detection area of ​​the digitizer 20 and the spatial coordinates of the detection area of ​​the spatial position detection unit 30 can be treated as common coordinates, so that there is a feature that no coordinate shift occurs even when the output from the digitizer 20 and the output from the spatial position detection unit 30 are switched in the 3D drawing system processing.

[0074] Therefore, the operator can seamlessly switch between fine drawing and gesture operations by simply moving the electronic pen 10 spatially over the digitizer 20 without being aware of switching between the digitizer 20 and the spatial position detection unit 30.

[0075] Furthermore, in the first embodiment described above, in the VR image processing, the position detection output of the digitizer 20, which has higher accuracy and faster response speed than the detection output of the spatial position detection unit 30, can be used in the spatial region of the position detection region DT of the digitizer 20 as position information of the electronic pen 10 (position information of the pen tip) for generating a VR image of the electronic pen 10, so that a VR image that responds appropriately to the actual operation of the electronic pen by the user can be obtained.

[0076] In addition, since the above-mentioned spatial drawing information generating device 40 is configured by a computer, it goes without saying that each of the input information processing unit 41 and the display image generating unit 42 can be configured as a software functional unit executed by a software program.

[0077] [Second embodiment] In the above-described first embodiment, the electronic pen 10 and the digitizer 20 are examples that do not have a function for detecting the pen attitude, such as the tilt angle and rotation angle, of the electronic pen. However, the electronic pen 10 and the digitizer 20 can be configured to have a function for detecting the pen attitude, such as the tilt angle and rotation angle, of the electronic pen.

[0078] The input device of the second embodiment described below is a system similar to the spatial position indication system described in the first embodiment above, in which the electronic pen and digitizer have a pen attitude detection function, such as the tilt angle and rotation angle of the electronic pen.

[0079] Fig. 4 shows an example of the configuration of the main parts of the spatial position indication system of the second embodiment. The spatial position indication system of the second embodiment has the same configuration as that of the first embodiment, but instead of the electronic pen 10 and the digitizer 20, an electronic pen 10A (not shown in Fig. 4) having a configuration for detecting the pen attitude and a digitizer 20A are provided. And instead of the input information processing unit 41 of the spatial drawing information generating device 40 of the first embodiment, an input information processing unit 41A having the configuration shown in Fig. 4 is provided.

[0080] The spatial position indication system of the second embodiment also includes the spatial position detection unit 30, the display image generation section 42, and the HMD 50 described in the first embodiment, but since these are similar to those in the first embodiment, they are not shown in Fig. 4. In addition, in the digitizer 20A and the input information processing section 41A shown in Fig. 4, the same parts as those in the digitizer 20 and the input information processing section 41 shown in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted.

[0081] As shown in Figure 5(A), when the electronic pen 10A is perpendicular to the input surface of the sensor unit, the signal levels of a predetermined number of loop coil positions, for example, seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3, are symmetrical around the indicated position.

[0082] In contrast, when the electronic pen 10A is tilted, the signal levels of these seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3 change according to the tilt direction and tilt angle of the electronic pen 10A, as shown in Fig. 5(B). Based on this predetermined number of multiple signal levels, the position coordinates of the pointing position of the electronic pen 10A, as well as the tilt angle and tilt direction can be detected.

[0083] However, in the input surface (detection area) INS of the sensor unit as shown in Fig. 6, for example, in the peripheral area PE indicated by diagonal lines outside the dotted line position, only a number of signal levels less than the above-mentioned predetermined number can be obtained. Therefore, in this second embodiment, in the peripheral area PE of the digitizer 20A, the detection accuracy of the pointing position (pen tip position) of the electronic pen 10A and the pen attitude of the electronic pen 10A is low, and in this second embodiment, the detection accuracy of the pointing position (pen tip position) of the electronic pen 10A and the pen attitude of the electronic pen 10A detected by the spatial position detection unit 30 is higher.

[0084] That is, in this second embodiment, even if the pen tip position of the electronic pen 10A is in the spatial area of ​​the position detection area DT in the digitizer 20A where the pointing position and pen attitude of the electronic pen 10A can be detected, when it is in the peripheral area PE and the spatial area above it, the information on the pointing position and pen attitude of the electronic pen 10A detected by the spatial position detection unit 30 is used instead of the information on the pointing position and pen attitude of the electronic pen 10A detected by the digitizer 20A.

[0085] As shown in Fig. 4, the digitizer 20A of the second embodiment includes a posture detection unit 24. The posture detection unit 24 detects the pen posture of the electronic pen 10A, such as the tilt angle, tilt direction, and rotation angle, based on the transmission and reception of signals between the electronic pen 10A (not shown) and the sensor unit 22 of the digitizer 20A. Information on the pen posture of the electronic pen 10A detected by the posture detection unit 24 is supplied to one input terminal of a selection unit 418 of the input information processing unit 41A. Information on the pen posture of the electronic pen 10A detected by the spatial position detection unit 30 from the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied to the other input terminal of the selection unit 418.

[0086] Information on the pen posture of the electronic pen 10A from the selection unit 418 is supplied to the association unit 415, which performs the same association process as described above with information on the pointing position of the electronic pen 10A from the selection unit 411, and supplies it to the drawing image generation unit 421 and gesture detection processing unit 423 of the display image generation unit 42.

[0087] In the second embodiment, information on the pen attitude of electronic pen 10A detected by pen attitude detection unit 4102 of spatial information processing unit 410 is supplied to one input terminal of selection unit 419. Information on the pen attitude of electronic pen 10A from attitude detection unit 24 of digitizer 20A is supplied to attitude conversion unit 417R, and information on the digitizer attitude including the tilt angle and tilt direction of digitizer 20A detected by digitizer attitude detection unit 4103 of spatial information processing unit 410 is supplied to this attitude conversion unit 417R.

[0088] In attitude conversion section 417R, the inverse conversion from attitude conversion section 417 is performed based on the pen attitude information of electronic pen 10A from attitude detection section 24 of digitizer 20A and the attitude information of digitizer 20A from digitizer attitude detection section 4103 of spatial information processing section 410. That is, the relative pen attitude of electronic pen 10A with respect to input surface 21S detected by digitizer 20A is converted into the absolute attitude of digitizer 20A in the spatial coordinate system of motion detection space region MD detected by spatial position detection unit 30.

[0089] Then, the pen attitude information of the electronic pen 10A whose attitude has been converted by this attitude conversion unit 417R is supplied to the other input terminal of the selection unit 419. The pen attitude information of the electronic pen 10A selected and output by the selection unit 419 as described later is supplied to the association unit 416, where the same association process as described above is performed with the information on the pointing position of the electronic pen 10A from the selection unit 412, and the information is supplied to the VR image generation unit 422 of the display image generation unit 42.

[0090] A selection control signal generating unit 414A in the second embodiment generates a selection control signal SEA for the selection units 411, 412, 418, and 419. The selection control signal generating unit 414A also generates a selection control signal SE similar to that of the selection control signal generating unit 414 in the above-described first embodiment, and controls switching of the processes executed by the drawing image generating unit 421 and the gesture detection processing unit 423 of the display image generating unit 42 in the same manner as in the above-described first embodiment.

[0091] The selection control signal SEA from the selection control signal generation unit 414A in this second embodiment not only selects and controls the selection units 411, 412, 418, 419 depending on whether the pointing position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT of the digitizer 20A or another spatial area, but also selects and controls the selection units 411, 412, 418, 419 depending on whether the pointing position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT of the digitizer 20A (see Figure 6) or a central area inside the peripheral area PE of the input surface 21S of the digitizer 20A.

[0092] Therefore, in the second embodiment, the selection control signal generating unit 414A is supplied with information on the pointing position of the electronic pen 10A from the position detecting unit 23 of the digitizer 20A in addition to information on the signal level of the received signal from the electronic pen 10A. The selection control signal generating unit 414A generates a selection control signal SEA that selectively controls the selection units 411, 412, 418, 419 when the pointing position (position of the pen tip) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE and when it is within any other spatial region.

[0093] When the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 411 selects information on the pointing position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 415, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 411 selects information on the pointing position of the electronic pen 10A from the spatial position detection unit 4101 of the coordinate conversion unit 413 and supplies it to the association unit 415, based on the selection control signal SEA.

[0094] In addition, when the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 412 selects information on the pointing position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 416, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 412 selects information on the pointing position of the electronic pen 10A from the spatial position detection unit 4101 that has been coordinate converted by the coordinate conversion unit 413 and supplies it to the association unit 416.

[0095] In addition, when the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 418 selects pen posture information of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and supplies it to the association unit 415, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 418 selects pen posture information converted from the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 from the posture conversion unit 417 into a relative pen posture and supplies it to the association unit 415.

[0096] In addition, when the selection control signal SEA indicates that the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 419 selects pen posture information converted from the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A from the posture conversion unit 417R into an absolute pen posture detected by the spatial position detection unit 30 and supplies it to the association unit 416, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 419 selects pen posture information of the electronic pen 10A from the pen posture detection unit 4102 and supplies it to the association unit 416.

[0097] An example of the flow of processing operations when the input information processing unit 41A of the spatial position indication system of the second embodiment is configured by a computer will be described with reference to Fig. 7 and its sequel Fig. 8. In this case, each block constituting the input information processing unit 41A shown in Fig. 4 becomes a software functional unit by a computer program.

[0098] That is, the input information processing unit 41A determines whether or not the digitizer 20A is in a state where it can detect the pointed position of the electronic pen 10A, based on a signal from the digitizer 20A (step S1). If it is determined in step S1 that the digitizer 20A is not in a state where it can detect the pointed position of the electronic pen 10A, the input information processing unit 41A determines whether or not the spatial position detection unit 30 can detect the electronic pen 10A (step S2). If it is determined in step S2 that the spatial position detection unit 30 cannot detect the electronic pen 10A, the input information processing unit 41A returns the process to step S1.

[0099] When it is determined in step S1 that the digitizer 20A is in a state where it can detect the pointed position of the electronic pen 10A, the input information processing unit 41A acquires the coordinates of the pointed position by the electronic pen 10A detected by the digitizer 20A (step S3), and determines whether the acquired coordinates of the pointed position are within the surrounding area PE (step S4).

[0100] If it is determined in step S4 that the coordinates of the acquired pointing position are not in the peripheral area PE but in its inner area, the input information processing unit 41A associates information on the pointing position of the electronic pen 10A from the digitizer 20A with information on the pen posture and outputs the associated information to the display image generation unit 42 for generating a drawn image (step S5).

[0101] Next, the input information processing unit 41A performs coordinate conversion to convert the information of the pointing position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 into information of a coordinate system common to that of the digitizer 20A (step S6). Then, the input information processing unit 41A associates the coordinate-converted information of the pointing position of the electronic pen 10A with the information of the pen attitude detected by the pen attitude detection unit 4102 of the spatial information processing unit 410, and outputs the information to the display image generating unit 42 for generating a VR image (step S7). After this step S7, the input information processing unit 41A returns the process to step S1 and repeats the processes from step S1 onwards.

[0102] Next, when it is determined in step S4 that the coordinates of the acquired indication position are within the surrounding area PE, and when it is determined in step S2 that electronic pen 10A can be detected by spatial position detection unit 30, input information processing unit 41A performs coordinate conversion to convert information of the indication position of electronic pen 10A detected by spatial position detection unit 4101 of spatial information processing unit 410 into information of a coordinate system common to digitizer 20A (step S11 in FIG. 8). Furthermore, input information processing unit 41A acquires information on the attitude of digitizer 20A detected by digitizer attitude detection unit 4103 of spatial information processing unit 410 (step S12).

[0103] Next, the input information processing unit 41A converts the pen attitude information from the attitude detection unit 24 of the digitizer 20A into absolute attitude information in the spatial domain, not a relative attitude with respect to the digitizer 20A, using the attitude information of the digitizer 20A acquired in step S12 (step S13).Then, the input information processing unit 41A associates the information on the pointing position of the electronic pen 10A obtained by the coordinate conversion in step S11 with the absolute attitude information obtained in step S13, and outputs them to the display image generating unit 42 for generating a drawn image (step S14).

[0104] Next, the input information processing unit 41A converts the pen attitude information from the pen attitude detection unit 4102 of the spatial information processing unit 410 into information on the attitude relative to the digitizer 20A, using the information on the attitude of the digitizer 20A acquired in step S12 (step S15). Then, the input information processing unit 41A associates the information on the pointing position of the electronic pen 10A obtained by the coordinate conversion in step S11 with the information on the absolute attitude obtained in step S13, and outputs the information to the display image generating unit 42 for generating a VR image (step S16). After this step S16, the input information processing unit 41A returns the process to step S1, and repeats the processes from step S1 onwards.

[0105] Since the input information processing unit 41A of the spatial position indication system of this second embodiment is configured as described above, in 3D drawing system processing and VR image system processing, information on the pointing position of the electronic pen 10A detected by the position detection unit 23 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30 in the central region inside the peripheral region PE of the input surface 21S within the spatial region of the position detection region DT of the digitizer 20A, and information on the pointing position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A in the peripheral region PE of the input surface 21S (including the space above the input surface 21S) and the spatial region outside the spatial region of the position detection region DT of the digitizer 20A.

[0106] In addition, as information on the pen posture of the electronic pen 10A in 3D drawing system processing, in the central region within the spatial region of the position detection region DT of the digitizer 20A that is inside the peripheral region PE of the input surface 21S, information on the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30, and in the spatial region within the spatial region of the position detection region DT of the digitizer 20A that is outside the peripheral region PE of the input surface 21S (including the space above the input surface 21S) and the spatial region of the position detection region DT of the digitizer 20A, information on the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0107] Therefore, in the second embodiment, as in the first embodiment, 3D drawing processing and VR image processing can be performed using highly accurate information at all times. In this case, as in the first embodiment, there is an advantage that the information detected by the digitizer 20A and the information detected by the spatial position detection unit 30 can be mutually used in the 3D drawing processing and the VR image processing, respectively, which is convenient.

[0108] [Third embodiment] The first and second embodiments described above are cases in which an input device according to the present invention is applied to a spatial position indication system, but the input device according to the present invention can also be applied to a tablet device equipped with a digitizer, where a position indication made by an electronic pen is detected within a position detection area DT including a hover area.

[0109] Fig. 9 is a diagram showing an outline of a system including an input device of the third embodiment. In Fig. 9, the input device of the third embodiment is configured by an electronic pen 10B and a tablet device 60 equipped with a digitizer. As shown in Fig. 9, the tablet device 60 of the input device is placed, for example, on a desk 90 and connected to a computer 80. In this example, the tablet device 60 is placed at an inclination angle θ with respect to the placement surface of the desk 90.

[0110] In the third embodiment, a gyro sensor unit 70 for detecting the pen attitude of the electronic pen 10B in a three-dimensional space above an input surface 61S of a digitizer built into the tablet device 60 is removably provided on the rear end side opposite to the pen tip side of the electronic pen 10B. The gyro sensor unit 70 may be built into the rear end side opposite to the pen tip side of the electronic pen 10B.

[0111] 10 is a block diagram for explaining an example of the functional configuration of the electronic pen 10B, tablet device 60, and gyro sensor unit 70 constituting the input device of this third embodiment. In this case, the gyro sensor unit 70 is configured to include, for example, a three-axis gyro sensor 71 and a wireless communication unit 72. The wireless communication unit 72 is configured, for example, by a short-range wireless communication means conforming to the Bluetooth (registered trademark) standard. The wireless communication unit 72 is not limited to this, and may be optical communication such as infrared communication.

[0112] In this example, the gyro sensor unit 70 is attached to the housing of the electronic pen 10B, so the three-axis gyro sensor 71 outputs an output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B in three-dimensional space. Here, the output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B detected by the three-axis gyro sensor 71 does not correspond to the relative attitude of the electronic pen 10B with respect to the input surface 61S of the tablet device 60, but corresponds to the absolute attitude based on the earth's axis in three-dimensional space. The wireless communication unit 72 wirelessly transmits the output signal from the three-axis gyro sensor 71 to the tablet device 60.

[0113] The electronic pen 10B includes an interaction unit 100 with the sensor unit 61 of the tablet device 60. The interaction unit 100 is coupled to the sensor unit 61 of the tablet device 60 by either an electromagnetic induction method or an electrostatic coupling method to exchange signals. The interaction unit 100 of the electronic pen 10B in this example exchanges with the sensor unit 61 a signal for detecting the position indicated by the electronic pen 10B, but does not exchange signals for detecting the pen attitude, such as the tilt angle, tilt direction, or rotation angle.

[0114] In the tablet device 60, a pointing position detection unit 62 detects the pointing position of the electronic pen 10B from a signal obtained as a result of interaction between the sensor unit 61 and the electronic pen 10B. Information on the pointing position of the electronic pen 10B detected by the pointing position detection unit 62 is supplied to an association unit 63.

[0115] The tablet device 60 of the third embodiment includes a wireless communication unit 64 that performs wireless communication with the wireless communication unit 72 of the gyro sensor unit 70. The detection output of the three-axis gyro sensor 71 of the gyro sensor unit 70 received by this wireless communication unit 64 is supplied to an electronic pen attitude detection unit 65, which detects the pen attitude of the electronic pen 10B. Information on the pen attitude of the electronic pen 10B detected by this electronic pen attitude detection unit 65 is supplied to a relative attitude calculation unit 66.

[0116] The tablet device 60 of this embodiment is equipped with a gyro sensor 67 for detecting the tilt angle and tilt direction of its own device. The sensor output of this gyro sensor 67 is supplied to a tablet attitude detection unit 68. The tablet attitude detection unit 68 detects the tilt angle and tilt direction of its own device based on the sensor output of the gyro sensor 67, and supplies information on the detected tilt angle and tilt direction of its own device (tablet attitude information) to a relative attitude calculation unit 66.

[0117] The relative attitude calculation unit 66 calculates the relative pen attitude of the electronic pen 10B with respect to the input surface 61S of the tablet device 60 by correcting the absolute pen attitude information in the three-dimensional space of the electronic pen 10B from the electronic pen attitude detection unit 65 using the tablet attitude information from the tablet attitude detection unit 68. Then, the relative attitude calculation unit 66 supplies the calculated relative pen attitude information of the electronic pen 10B to the association unit 63.

[0118] The associating unit 63 associates information on the pointing position of the electronic pen 10B with information on the relative pen attitude of the electronic pen 10B in the same manner as the above-mentioned associating unit, and outputs the information to the computer 80.

[0119] As described above, in the input device of the third embodiment, when the position indicated by the tip of electronic pen 10B is obtained, information on the pen attitude of electronic pen 10B detected in three-dimensional space by the gyro sensor unit 70 provided on the rear end side of electronic pen 10B corresponding to this can be output as pen attitude information corresponding to the indicated position of electronic pen 10B.

[0120] [Other embodiments or modifications] In the first and second embodiments described above, the spatial position detection unit 30 is configured to include a light emitting tracking device that emits infrared laser light and a tracker, but it goes without saying that the present invention is not limited to this configuration. For example, the spatial position detection unit 30 may be configured to use other non-visible light sensors, visible light sensors, or a combination of these.

[0121] In the above-described first and second embodiments, since the HMD 50 is worn, the operator of the electronic pen 10 cannot directly see the digitizer 20. Therefore, it is also possible to configure the system so that a virtual image of the digitizer 20 is drawn on the virtual space image displayed on the HMD 50, and the operator can recognize the position of the digitizer 20 on the display screen of the HMD 50.

[0122] In the above-described first and second embodiments, the coordinate values ​​of the spatial position detection unit 30 are converted into coordinate values ​​of the spatial coordinate system of the digitizer 20 in order to correct errors in the two spatial coordinate systems, but conversely, the coordinate values ​​of the position detection region DT of the digitizer 20 may be converted into coordinate values ​​of the spatial position detection unit 30. Note that conversion between these two spatial coordinate systems is not essential and is optional.

[0123] In the first and second embodiments described above, the electronic pen and digitizer are of the electromagnetic induction type, but this is not limited to this, and it is of course also possible to use an electronic pen and digitizer of the electrostatic type (including active electrostatic coupling type and passive electrostatic coupling type).

[0124] Moreover, the digitizers of the first and second embodiments and the tablet device of the third embodiment may be portable mobile phone terminals known as smartphones.

[0125] In the first and second embodiments, a case has been described in which 3D drawing is performed by the spatial position designation system, but the drawn image may be a 2D drawn image or a 2.5D drawn image. [Explanation of symbols]

[0126] 10, 10A, 10B... electronic pen, 20, 20A... digitizer, 21S... input surface, 22... sensor section, 23... position detection section, 24... attitude detection section, 30... spatial position detection unit, 40... spatial drawing information generating device, 41... input information processing section, 4101... spatial position detection section, 4102... pen attitude detection section, 50... HMD

Claims

1. Calculating a relative orientation of the electronic pen with respect to a device having a digitizer that detects a position pointed to by the electronic pen; The region within the input surface of the device having the digitizer includes a peripheral region where the detection accuracy by the device having the digitizer is relatively low and a central region that is inside the peripheral region and where the detection accuracy by the device having the digitizer is relatively high, and when a position pointed by the electronic pen exists in the central region, a VR image of the electronic pen in a VR (Virtual Reality) space is generated using the pointed position detected by the digitizer and the relative attitude of the electronic pen associated with the pointed position, and when a position pointed by the electronic pen exists in a region different from the central region, a VR image of the electronic pen in the VR space is generated using the pointed position detected using a spatial position detection unit located above the digitizer and tracking a tracker possessed by the electronic pen and the absolute attitude of the electronic pen associated with the pointed position.

13. An image generating method comprising:

2. calculating a relative orientation of the electronic pen based on an orientation of the electronic pen detected by a device having the digitizer; 2. The image generating method according to claim 1 .

3. The relative orientation of the electronic pen is an orientation calculated using the orientation of the electronic pen detected by a gyro sensor of the electronic pen and the orientation of the device having the digitizer detected by a gyro sensor of the device having the digitizer.

2. The image generating method according to claim 1 .

4. generating a VR image in the VR space using a pointing position of the electronic pen and a relative orientation of the electronic pen detected by a device having the digitizer; 2. The image generating method according to claim 1 .

5. The present invention has a processing unit that calculates a relative orientation of the electronic pen with respect to a device having a digitizer that detects a position pointed by the electronic pen, the region within an input surface of the device having a digitizer including a peripheral region where the detection accuracy by the device having a digitizer is relatively low and a central region that is an area inside the peripheral region and where the detection accuracy by the device having a digitizer is relatively high, and when a position pointed by the electronic pen exists in the central region, generates a VR image of the electronic pen in a virtual reality (VR) space using the pointed position detected by the digitizer and the relative orientation of the electronic pen associated with the pointed position, and when a position pointed by the electronic pen exists in a region different from the central region, generates a VR image of the electronic pen in the VR space using the pointed position detected by a spatial position detection unit that is located above the digitizer and tracks a tracker of the electronic pen.

2. A computer comprising:

6. The relative orientation of the electronic pen is an orientation calculated based on an orientation of the electronic pen detected by a device having the digitizer.

6. The computer of claim 5.

7. The relative orientation of the electronic pen is an orientation calculated using the orientation of the electronic pen detected by a gyro sensor of the electronic pen and the orientation of the device having the digitizer detected by a gyro sensor of the device having the digitizer.

6. The computer of claim 5.

8. the processing unit generates a VR image in the VR space using a position pointed to by the electronic pen and a relative attitude of the electronic pen detected by a device having the digitizer; 6. The computer of claim 5.

9. Computer, means for calculating a relative attitude of the electronic pen with respect to a device having a digitizer that detects a position pointed by the electronic pen, the region within an input surface of the device having a digitizer including a peripheral region where the detection accuracy by the device having a digitizer is relatively low and a central region that is an area inside the peripheral region and where the detection accuracy by the device having a digitizer is relatively high, generating a VR image of the electronic pen in a VR (Virtual Reality) space when the position pointed by the electronic pen exists in the central region using the pointed position detected by the digitizer and the relative attitude of the electronic pen associated with the pointed position, and generating a VR image of the electronic pen in a VR space when the position pointed by the electronic pen exists in a region different from the central region using the pointed position detected by a spatial position detection unit located above the digitizer and tracking a tracker of the electronic pen, the spatial position detection unit being located above the digitizer; An image generation program to function as a.

10. The relative orientation of the electronic pen is an orientation calculated based on an orientation of the electronic pen detected by a device having the digitizer.

10. The image generating program according to claim 9.

11. The relative orientation of the electronic pen is an orientation calculated using the orientation of the electronic pen detected by a gyro sensor of the electronic pen and the orientation of the device having the digitizer detected by a gyro sensor of the device having the digitizer.

10. The image generating program according to claim 9.

12. generating a VR image in the VR space using a pointing position of the electronic pen and a relative orientation of the electronic pen detected by a device having the digitizer; 10. The image generating program according to claim 9.

Images