VR image generation system and VR image generation method

Abstract

To provide a virtual reality (VR) image creation system and a VR image creation method for creating a VR image in a VR space by using an electronic pen.SOLUTION: A spatial position instruction system has: a digitizer 20 that detects a position indicated by an electronic pen 10; and a spatial drawing information generation device 40 including an input information processing unit 41 that detects the position of a tracker of the electronic pen to perform position input using the electronic pen in a virtual reality (VR) space, and a display image creation unit 42 that creates a VR image related to the electronic pen in the VR space, on the basis of the posture of the electronic pen and the position indicated by the electronic pen detected by the digitizer, in a three-dimensional space where the electronic pen is present which is detected on the basis of tracking of the tracker of the electronic pen performed by the input information processing unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a VR (Virtual Reality) image generation system and a VR image generation method. [Background technology]

[0002] 2. Description of the Related Art Drawing systems are known that create animation images and the like by drawing by continuously specifying positions with an electronic pen on a coordinate input device called a digitizer.

[0003] Various methods have been proposed for detecting the tilt and rotation of an electronic pen. Patent Document 1 (JP 2016-126503 A) provides an electronic pen that can detect the tilt angle and rotation angle of a sensor unit of a position detection device relative to an input surface. In the case of Patent Document 1, signals are exchanged between the electronic pen and a position detection device equipped with a sensor unit by electromagnetic induction coupling, electrostatic capacitance coupling, or the like, so that the position detection device detects the position indicated by the electronic pen and also detects the position coordinates of the position indicated by the electronic pen, as well as the tilt angle and rotation angle.

[0004] Furthermore, Patent Document 2 (U.S. Patent Publication No. 9,329,703 B2) describes that a sensor capable of measuring the movement and orientation of an electronic pen is built into or attached to the electronic pen, and the detection output of the sensor is transmitted to a position detection device, thereby enabling the position detection device to detect the movement and state (tilt and rotation) of the electronic pen. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-126503 [Patent Document 2] U.S. Patent No. 9,329,703 B2 Summary of the Invention [Problem to be solved by the invention]

[0006] In the case of the above-mentioned Patent Document 1, the position coordinates and tilt of the position pointed by the electronic pen are detected using multiple signal levels obtained from multiple loop coils around the position pointed by the electronic pen. However, there are problems in that the accuracy of the position coordinates of the position pointed by the electronic pen decreases in the area surrounding the sensor unit, and it becomes difficult to detect the tilt of the electronic pen.

[0007] The method of Patent Document 2 does not have the problems of Patent Document 1. However, depending on the conditions, there are problems such as a slow reaction speed and a decrease in the accuracy of position coordinates.

[0008] An object of the present invention is to provide a VR image generation system that can solve the above problems. [Means for solving the problem]

[0009] To solve the above problems, a device having a digitizer that detects a position pointed to by an electronic pen; a tracking device for detecting the position of the tracker of the electronic pen; Set in a first coordinate system associated with the tracking device a tracking device for inputting a position using the electronic pen in a VR (Virtual Reality) space; The tracking device It will be tracked The electronic Nga The orientation of the electronic pen in the existing three-dimensional space 、 and 、 The position pointed by the electronic pen is detected by the device having the digitizer. The coordinates in the second coordinate system associated with the device having the corresponding digitizer are converted into coordinates in the first coordinate system. a computer that generates a VR image related to the electronic pen in the VR space based on the The present invention provides a VR image generation system having the following features.

[0010] In the VR image generation system configured as described above, a VR image of the electronic pen in VR space is generated based on the orientation of the electronic pen in three-dimensional space detected by a tracking device and the position pointed to by the electronic pen detected by a device having a digitizer. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of the configuration of a spatial position indication system including a first embodiment of an input device according to the present invention. [Figure 2] 2 is a block diagram for explaining an example of the configuration of each part of the spatial position indication system of the example of FIG. 1. FIG. [Figure 3] 1A and 1B are diagrams for explaining an embodiment of an input device according to the present invention; [Figure 4] FIG. 4 is a block diagram for explaining an example of the configuration of each part of a second embodiment of an input device according to the present invention. [Figure 5] 10 is a diagram for explaining an operation of detecting the tilt of the electronic pen by transmitting and receiving signals between the electronic pen and the digitizer. FIG. [Figure 6] 10A and 10B are diagrams useful in explaining that the detection accuracy when the digitizer detects the pointing position and tilt of the electronic pen by sending and receiving signals to and from the electronic pen differs between the peripheral area and other areas of the input surface. [Figure 7] FIG. 10 is a diagram showing a part of a flowchart for explaining the main operation of the second embodiment of the input device according to the present invention. [Figure 8] FIG. 10 is a diagram showing a part of a flowchart for explaining the main operation of the second embodiment of the input device according to the present invention. [Figure 9] FIG. 10 is a diagram illustrating a configuration example of a third embodiment of an input device according to the present invention. [Figure 10] FIG. 10 is a block diagram for explaining an example of the configuration of each part of a third embodiment of an input device according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of an input device according to the present invention will be described with reference to the drawings.

[0013] [First embodiment] The embodiment of the input device described below relates to a spatial position indication system that includes a digitizer (pen tablet) having a sensor unit that transmits and / or receives signals from an electronic pen (hereinafter referred to as "signal transmission and reception"), and a spatial position detection unit that detects position information indicated by the electronic pen in space. Note that the position includes not only position coordinates but also attitude (tilt and rotation), but in the following description, position coordinates and attitude information will be explained separately to clarify the two.

[0014] In the spatial position indication system according to this embodiment, the display unit is configured with a head-mounted display, smart glasses, or the like, and a 3D rendering space is used as a virtual reality space (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc., hereinafter abbreviated as VR). FIG. 1 is a diagram showing an outline of the overall configuration of the spatial position indication system including an input device of the first embodiment, in which the VR space is used as the 3D rendering space. FIG. 2 is a block diagram showing a detailed configuration example of the functions of each part of the spatial position indication system including the input device of the first embodiment.

[0015] 1, the spatial position indication system of this embodiment is configured to include an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generation device 40, and a head-mounted display (hereinafter referred to as HMD) 50. As shown in FIG. 2, in this example, the spatial drawing information generation device 40 has the functions of an input information processing unit 41 and a display image generation unit 42, and is configured, for example, by a computer.

[0016] The input device of this first embodiment includes a first detection unit that detects a position pointed to by the electronic pen 10 in response to transmission of a signal to and from the electronic pen 10, a second detection unit that detects a position pointed to by the electronic pen 10 in the three-dimensional space in which the electronic pen 10 exists, and a control unit that generates position information of the electronic pen 10 based on these pointed positions. In this first embodiment, the first detection unit is made up of the digitizer 20, the second detection unit is made up of the spatial position detection unit 30 and the spatial information processing unit 410 of the input processing unit 41, and the control unit is made up of the input information processing unit.

[0017] In this first embodiment, the electronic pen 10 is an electromagnetic induction type electronic pen, but is not limited to the electromagnetic induction type and may be an electrostatic coupling type or the like. The digitizer 20 has a thin rectangular parallelepiped housing 21, the surface of which serves as an input surface 21S for position indication by the electronic pen 10. The digitizer 20 also has a sensor unit 22 and a position detection unit 23 (see FIG. 2).

[0018] Although not shown, the sensor unit 22 is configured by a plurality of loop coils arranged in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) of the housing of the digitizer 20. In this example, the digitizer 20 is of an electromagnetic induction type to match the electronic pen 10, but is not limited to the electromagnetic induction type.

[0019] Although not shown in the figure, the electronic pen 10 has a resonant circuit (not shown) on the pen tip side consisting of a coil and a capacitor, and signals are exchanged between the electronic pen 10 and the sensor section 22 of the digitizer 20 by electromagnetic induction coupling between the loop coil of the sensor section 22 of the digitizer 20 and the resonant circuit of the electronic pen 10.

[0020] The position detection unit 23 of the digitizer 20 supplies a signal to the electronic pen 10 through the loop coil of the sensor unit 22, and also receives a signal returned from the electronic pen 10 through the loop coil, and based on the received signal, detects the position indicated by the electronic pen 10 in the detection area of ​​the sensor unit 22. In this embodiment, the digitizer 20 is configured to detect the position indicated by the tip of the electronic pen 10 as the indicated position of the electronic pen 10.

[0021] In the digitizer 20 of this example, a plurality of loop coils of the sensor section 22 are arranged so as to cover almost the entire area of ​​the input surface 21S.

[0022] In this embodiment, the position detection area in which the digitizer 20 can detect the pointing position of the electronic pen 10 includes not only the planar area when the tip of the electronic pen 10 is in contact with the input surface 21S of the digitizer 20, but also a spatial area (hover area in the hover state of the electronic pen 10) in which the tip of the electronic pen 10 is not in contact with the input surface 21S of the digitizer 20 and is spaced from the input surface 21S in a direction perpendicular to the input surface 21S (the Z-axis direction perpendicular to the X-axis and Y-axis directions), but in which the pointing position of the electronic pen 10 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0023] In this example, the spatial position detection unit 30 is configured to set a three-dimensional spatial area in which the digitizer 20 exists, and detect the position indicated by the tip of the electronic pen 10 in that three-dimensional spatial area, as well as the attitude of the electronic pen 10 (referred to as pen attitude), such as the tilt angle and rotation angle of the electronic pen 10, and to detect the position of the digitizer 20 and the tilt angle and tilt direction from the horizontal plane.

[0024] The spatial position detection unit 30 includes two light-emitting tracking devices 31A and 31B and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. In this embodiment, as will be described later, the HMD 50 displays a 3D drawn image drawn in a motion detection space area MD including a position detection area DT of the digitizer 20 as a virtual display image, and also displays a virtual display image of the electronic pen 10.

[0025] The two light-emitting tracking devices 31A, 31B have the same configuration, and each includes a laser light emitting unit of infrared laser light, a search means for searching the motion detection space region MD with the emitted infrared laser light, and a light position detection means for detecting the light emission of the light-emitting units of the trackers 32A, 32B, 32C, 32D that receive the infrared laser light.

[0026] In this embodiment, the objects (search targets in the search space area of ​​the spatial position detection unit 30) to which the trackers 32A, 32B, 32C, and 32D are attached are the electronic pen 10 and the digitizer 20, as described above. That is, in this example, in order to be able to notify the position of the digitizer 20 and the tilt angle and tilt direction with respect to the horizontal plane, the trackers 32A and 32B are attached to the upper left and lower right corners of the housing of the digitizer 20, which has a thin rectangular parallelepiped shape. In addition, in order to notify the position and pen attitude (tilt angle and rotation angle) of the electronic pen 10, a tracker 32C is attached to the pen tip side of the electronic pen 10, and a tracker 32D is attached to the rear end side, opposite the pen tip side, in the axial direction of the housing of the electronic pen 10.

[0027] The light-emitting tracking devices 31A and 31B use search means to control the laser light-emitting units to emit and scan infrared laser light within the motion detection space region MD to detect the tracker position. Each of the trackers 32A, 32B, 32C, and 32D monitors the reception of infrared laser light with a sensor, and when the sensor detects the reception of infrared laser light, it turns on a light-emitting unit consisting of an LED.

[0028] The light-emitting tracking devices 31A and 31B detect the light emitted by the light-emitting units of the trackers 32A, 32B, 32C, and 32D to detect the position of the object to which the tracker 32A, 32B, 32C, and 32D is attached within the motion detection space region MD. When the light-emitting tracking devices 31A and 31B detect the light emitted by the light-emitting units of the trackers 32A, 32B, 32C, and 32D, they are also configured to detect the elapsed time from the time of the infrared laser emission at the time of detection. In this case, each of the trackers 32A, 32B, 32C, and 32D emits light differently according to its own identification information.

[0029] The two light-emitting tracking devices 31A, 31B are connected to the spatial drawing information generating device 40 by wire or wirelessly, and notify the spatial drawing information generating device 40 of the spatial position information of the detected trackers 32A, 32, 32C, 32D in the motion detection spatial region MD.

[0030] In this example, the spatial position information of the trackers 32A, 32B, 32C, and 32D detected by the two light-emitting tracking devices 31A and 31B in the motion detection space region MD is supplied to a spatial information processing unit 410 of the input information processing unit 41 of the spatial drawing information generating device 40, as shown in Figure 2.

[0031] In this example, the spatial information processing unit 410 includes a spatial position detection unit 4101, a pen attitude detection unit 4102, and a digitizer attitude detection unit 4103. The spatial position detection unit 4101 detects the position of the digitizer 20 in the motion detection space region MD from the trackers 32A and 32B, and detects the position pointed to by the electronic pen 10 as the position of the pen tip of the electronic pen 10 and the position of the rear end of the electronic pen 10 from the trackers 32C and 32D. In this example, the pen attitude detection unit 4102 detects the pen attitude, including the tilt angle and rotation angle of the electronic pen 10, from the trackers 32C and 32D. Furthermore, the digitizer attitude detection unit 4103 detects the position pointed to by the electronic pen 10 as the tip position.

[0032] In this embodiment, the input information processing unit 41 generates information to be supplied to the display image generation unit 42 from information on the pointing position by the electronic pen 10 in the position detection area DT detected by the digitizer 20, information on the position (pointing position and rear end position) of the electronic pen 10 in the motion detection space area MD detected by the space information processing unit 410 of the input information processing unit 41, pen attitude information, and digitizer attitude information. The input information processing unit 41 then supplies the generated information to the display image generation unit 42.

[0033] In this embodiment, the display image generation unit 42 of the spatial drawing information generation device 40 includes a drawing image generation unit 421 for generating a 3D drawing image, and a VR image generation unit 422 for generating a VR image to be displayed on the HMD 50, as shown in FIG. 2.

[0034] The drawn image generation unit 421 generates a 3D drawn image based on a position indication by the electronic pen 10, and also performs processes such as deformation, rotation, and movement of the 3D drawn image based on a gesture performed by the operator of the electronic pen 10. In this embodiment, the drawn image generation unit 421 also reflects the pen attitude, including the tilt angle, tilt direction, and rotation angle, of the electronic pen 10 with respect to the input surface 21S of the digitizer 20, in the generation of the 3D drawn image. For example, when the electronic pen 10 exchanges signals with the digitizer 20, the tilt angle and tilt direction of the digitizer 20 with respect to the input surface 21S are used. Hereinafter, processing related to a 3D drawn image will be referred to as 3D drawing-related processing.

[0035] In this embodiment, the spatial position detection unit 30 can detect the pen attitude of the electronic pen 10, including the tilt angle, tilt direction, and rotation angle, using trackers 32C and 32D attached to the electronic pen 10. Therefore, in this embodiment, the input information processing unit 41 is configured to supply information on the pen attitude of the electronic pen 10 detected by the spatial position detection unit 30 to the display image generation unit 42 as information for 3D drawing processing.

[0036] That is, as will be described later, the input information processing unit 41 is configured to supply the display image generating unit 42 with information on the position instruction by the electronic pen 10 (information on the position of the pen tip) and information on the pen attitude of the electronic pen 10 so that the 3D drawing image reflects information on the pen attitude, such as the tilt and rotation of the electronic pen 10 at the time of drawing input based on the position instruction by the electronic pen 10.

[0037] 2, the display image generation unit 42 is provided with a gesture detection processing unit 423 for detecting a gesture performed by the operator of the electronic pen 10. In this embodiment, the input information processing unit 41 is configured to supply the gesture detection processing unit 423 with position information of the pen tip of the electronic pen 10 (corresponding to information on the indicated position) and information on the pen attitude detected by the spatial position detection unit 30.

[0038] As described above, in this embodiment, not only information on the position pointed to by the electronic pen 10 detected by the digitizer 20 is used as information for 3D drawing system processing, but also information on the position pointed to by the electronic pen 10 (position of the pen tip) detected by the spatial position detection unit 30 and information on the pen posture.

[0039] The VR image generation unit 422 of the display image generation unit 42 generates a VR image to be displayed on the HMD 50. In this embodiment, the VR images include a VR image of the electronic pen 10 and a VR image of a 3D drawn image generated by the drawn image generation unit 421, and the VR image of the 3D drawn image generated by the drawn image generation unit 421 and the VR image of the electronic pen 10 are displayed in 3D on the HMD 50. Note that the VR image generation unit 422 may also generate a VR image of the digitizer 20. Hereinafter, processing related to the generation of a VR image in the VR image generation unit 422 will be referred to as VR image-related processing.

[0040] In this embodiment, the VR image generation unit 422 is supplied with information for generating a VR image of the electronic pen 10 from the input information processing unit 41, and is also supplied with information on the generated 3D drawn image from the drawn image generation unit 421. The VR image information generated by the VR image generation unit 422 is then supplied to the HMD 50 via the display drive unit 424, and displayed on a display screen of the HMD 50, for example, made up of an LCD (Liquid Crystal Display).

[0041] In this case, when the electronic pen 10 is present in the position detection area DT, information on the position indicated by the electronic pen from the digitizer 20, which can detect the position (pen tip position) of the electronic pen 10 with higher accuracy than the spatial position detection unit 30, and information on the pen attitude such as tilt and rotation detected using the spatial position detection unit 30 are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0042] Furthermore, when the electronic pen 10 is present in the motion detection space area MD, the digitizer 20 cannot detect the pointing position of the electronic pen 10, so information on the pointing position by the electronic pen detected using the spatial position detection unit 30 and information on the pen posture are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0043] As described above, in this embodiment, information used for VR image processing is not only information on the pointing position (pen tip position) of the electronic pen 10 detected by the spatial position detection unit 30 and information on the pen posture, but also information on the pointing position by the electronic pen 10 detected by the digitizer 20.

[0044] The input information processing unit 41 generates information for drawing system processing and information for VR image system processing as described above from information from the digitizer 20 and information from the spatial position detection unit 30, supplies this to the display image generation unit 42, and also generates a switching control signal SE for selection control depending on whether the electronic pen 10 is located in the position detection area DT or the motion detection space area MD.

[0045] As described above, in this embodiment, the information in the spatial coordinate system of the position detection region DT of the digitizer 20 and the information in the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 can be mutually used in each of the 3D rendering-based processing and the VR image-based processing. In this case, the two spatial coordinate systems can be set independently, but in this embodiment, the information in one of the two spatial coordinate systems is converted into the information in the other spatial coordinate system so that they can be treated as information in a common coordinate space. In this embodiment, the information in the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 is converted into the information in the spatial coordinate system of the position detection space region DT of the digitizer 20.

[0046] In the spatial position indication system of this embodiment, information on the pen posture of the electronic pen 10 detected in the motion detection space area MD of the spatial position detection unit 30 can be used in 3D drawing-based processing as well as in VR image-based processing.

[0047] In this case, in VR image processing, information about the pen attitude of the electronic pen 10 is reflected in the attitude of the VR image of the electronic pen 10. This information about the pen attitude of the electronic pen 10 includes information about the tilt angle of the electronic pen 10, but in 3D drawing processing, this tilt angle of the electronic pen 10 is the tilt angle relative to the input surface of the sensor section 22 of the digitizer 20. On the other hand, information about the pen attitude of the electronic pen 10 detected in the motion detection space region MD of the spatial position detection unit 30 is the tilt angle in the motion detection space region MD of the spatial position detection unit 30, for example, based on the direction of gravity of the Earth or the direction of the horizontal plane.

[0048] If digitizer 20 can be installed so that the direction perpendicular to input surface 21S of sensor section 22 thereof accurately coincides with the Z-axis direction of motion detection space region MD of spatial position detection unit 30, the tilt angle of electronic pen 10 detected in the space of motion detection space region MD of spatial position detection unit 30 will coincide with the tilt angle of digitizer 20 relative to the input surface of sensor section 22. In practice, however, digitizer 20 may be installed at an angle with respect to motion detection space region MD of spatial position detection unit 30.

[0049] Therefore, in this embodiment, the absolute tilt angle of the electronic pen 10 detected in the space of the motion detection space area MD of the spatial position detection unit 30 is converted into a relative tilt angle with respect to the input surface of the sensor unit 22 of the digitizer 20 as described below.

[0050] Fig. 3 is a diagram used to explain the conversion process of this tilt angle. Fig. 3 shows a spherical coordinate system in which the pen tip of electronic pen 10 is set as point O, which is the center of the sphere. The XY plane (a plane including the X-axis direction and the Y-axis direction which are orthogonal to each other) in Fig. 3 is the horizontal plane in the spatial domain of the motion detection spatial domain MD of spatial position detection unit 30.

[0051] In FIG. 3, the point indicating the position of the rear edge of the housing of the electronic pen 10 on the opposite side of the pen tip in the axial direction of the housing is expressed in spherical coordinates. In this case, the radius r of the sphere is the length of the housing of the electronic pen 10 in the axial direction. In the spherical coordinate system of FIG. 3, when the rear edge of the housing of the electronic pen 10 in the axial direction is at point P, the tilt angle and tilt direction of the electronic pen 10 within the space of the motion detection space region MD can be detected as δ and α, using the spatial position detection unit 30. Furthermore, in the example of FIG. 3, the tilt angle of the housing of the digitizer 20 within the space of the motion detection space region MD can be detected as δ' and α', using the spatial position detection unit 30.

[0052] As shown in Figure 3, if the position of the rear edge of the electronic pen 10 when the axial direction of the housing of the electronic pen 10 is aligned with the vertical direction within the space of the motion detection space region MD is point Q, then the position of the rear edge of the electronic pen 10 when the axial direction of the housing of the electronic pen 10 is perpendicular to the input surface 21S of the sensor unit 22 of the digitizer 20 is point R.

[0053] Therefore, by calculating the angle formed by the spatial vector OR from point O to point R and the spatial vector OP from point O to point P, it is possible to use this angle to convert the tilt angle and tilt direction of the electronic pen 10 detected using the spatial position detection unit 30 into the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor section 22 of the digitizer 20.

[0054] Furthermore, by performing the above-mentioned inverse conversion, the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor section 22 of the digitizer 20 can be converted into the tilt angle and tilt direction of the electronic pen 10 detected using the spatial position detection unit 30.

[0055] 2 configured to achieve the above will be described. That is, the position detection unit 23 constituting the digitizer 20 supplies a detection output of the pointing position of the electronic pen 10 to the selection unit 411 as one of its input signals, and also supplies the same to the selection unit 412 as one of its input signals. Note that the information supplied from the position detection unit 23 to the selection units 411 and 412 includes information on the writing pressure applied to the electronic pen 10 in addition to the detection output of the pointing position of the electronic pen 10.

[0056] Furthermore, the spatial position detection unit 4101 of the spatial information processing unit 410 supplies a detection output of the spatial position of the pen tip of the electronic pen 10 (position indicated by the electronic pen) to the coordinate conversion unit 413. This coordinate conversion unit 413 converts information in the spatial coordinate system of the motion detection space region MD of the spatial position detection unit 30 into information in the spatial coordinate system of the position detection region DT of the digitizer 20. Then, the coordinate conversion unit 413 supplies the converted coordinate output to the selection units 411 and 412 as the other input signal.

[0057] In this embodiment, the position detection unit 23 of the digitizer 20 supplies information on the signal level of the signal received from the electronic pen 10 to the selection control signal generation unit 414. The selection control signal generation unit 414 detects the distance between the pen tip of the electronic pen 10 and the input surface 21S of the digitizer 20 from the signal level of the signal received from the electronic pen 10, and generates a selection control signal SE based on the detected distance from the input surface 21S.

[0058] In this case, the selection control signal SE is a signal that controls the digitizer 20 to select the position detection output from the position detection unit 23 when the distance between the tip of the electronic pen 10 and the input surface 21S of the digitizer 20 is less than the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen, and controls the digitizer 20 to select the detection output from the spatial position detection unit 4101 when the distance is greater than the critical height Lz.

[0059] The selection unit 411 switches between one input and the other input in response to a selection control signal SE, and supplies the input to an associating unit 415 for 3D rendering processing. The selection unit 412 switches between one input and the other input in response to a selection control signal SE, and supplies the input to an associating unit 416 for VR image processing.

[0060] In the example of Figure 2, two selection units 411 and 412 are provided for each type of processing to clearly distinguish between 3D drawing processing and VR image processing, but in reality, a single common selection unit is sufficient, and the output of this common selection unit can be configured to be supplied to each of the association units 415 and 416.

[0061] Information about the pen attitude detected by the pen attitude detection section 4102 of the spatial information processing section 410 using the spatial position detection unit 30 is supplied as is to the associating section 416 for VR image processing.

[0062] The associating unit 416 for VR image processing associates information on the pointing position of the electronic pen 10 from the selecting unit 412 with information on the pen attitude from the pen attitude detecting unit 4102 of the spatial information processing unit 410, and supplies the paired information to the VR image generating unit 422 of the display image generating unit 42. In this case, the association in the associating unit 416 means associating information on the pen attitude detected at the pointing position of the electronic pen 10 with the pointing position of the electronic pen 10 output from the selecting unit 412, as a pair.

[0063] When the information on the pointing position of the electronic pen 10 and the information on the pen posture that are associated by the association unit 416 are both information from the spatial information processing unit 410, the spatial information processing unit 410 adjusts the output timing of the information, and the association unit 416 can simply associate the information by pairing information with the same timing.

[0064] On the other hand, when the information on the pointing position of the electronic pen 10 associated by the association unit 416 is information from the digitizer 20, the association unit 416 associates the information on the pointing position of the electronic pen 10 with the information on the pen posture by taking into account the difference (processing delay) in the output timing of the detection results between the digitizer 20 and the spatial position detection unit 30.

[0065] The VR image generation unit 422 of the display image generation unit 42 generates a VR image of the electronic pen 10 using the information on the pointing position of the electronic pen 10 and the information on the pen attitude from the association unit 416. In this case, the VR image of the electronic pen 10 is generated so as to be displayed at a position based on the information on the pointing position of the electronic pen 10.

[0066] In this case, the information on the pointing position of the electronic pen 10 from the selection unit 412 is switched and selected by the selection control signal SE, but the information on the pointing position from the spatial position detection unit 4101 is coordinate converted by the coordinate conversion unit 413 so that it becomes a coordinate space common to the digitizer 20 and the spatial position detection unit 30. Therefore, even when the selection unit 412 is switched, there is no display shift such as the display position of the VR image of the electronic pen 10 jumping.

[0067] The pen attitude information from the pen attitude detection unit 4102 of the spatial information processing unit 410 is also supplied to the associating unit 415 for 3D drawing system processing via the attitude conversion unit 417. The attitude conversion unit 417 is supplied with the pen attitude information from the pen attitude detection unit 4102 and the attitude information of the digitizer 20 detected by the digitizer attitude detection unit 4103, and performs processing to convert the pen attitude of the electronic pen 10 in the spatial coordinate system detected by the spatial position detection unit 30, as described with reference to Fig. 3, into a pen attitude relative to the input surface 21S of the sensor unit 22 of the digitizer 20.

[0068] The associating unit 415 for 3D drawing processing associates and pairs information on the pointing position of the electronic pen 10 from the selecting unit 411 with information from the attitude converting unit 417, which is information on the pen attitude detected by the pen attitude detecting unit 4102 and converted into information on the pen attitude relative to the input surface 21S of the digitizer 20, and supplies the paired information to the drawn image generating unit 421 and the gesture detection processing unit 423 of the display image generating unit 42. In this case, the association by the associating unit 415, like the associating unit 416, means associating the pointing position of the electronic pen 10 output from the selecting unit 412 with information on the pen attitude detected at the pointing position of the electronic pen 10 as a pair.

[0069] The drawing image generation unit 421 has a pen drawing function that draws fine line drawings, etc. based on the detection output of the pointing position of the electronic pen 10 from the digitizer 20 and, in this embodiment, information on the pen posture detected by the spatial position detection unit 30, and a gesture processing function that performs drawing processing based on the movement (gesture) detected by the gesture detection processing unit 423 based on the spatial position and pen posture of the electronic pen 10 detected by the spatial position detection unit 30.

[0070] The drawing image generation unit 421 and the gesture detection processing unit 423 are supplied with a selection control signal SE from the selection control signal generation unit 414, and the gesture detection processing unit 423 is controlled to operate when the pointing position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20, and the drawing image generation unit 421 is controlled to switch so as to execute a pen drawing function when the pointing position of the electronic pen 10 is within the spatial area of ​​the position detection area DT of the digitizer 20, and to execute gesture processing when the pointing position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20.

[0071] In this case, in this embodiment, coordinate conversion is performed using the coordinate conversion unit 413 so that the digitizer 20 and the spatial position detection unit 30 have a common coordinate space, so that even if the selection unit 411 switches the pointing position of the electronic pen 10, the pointing position of the electronic pen 10 does not jump or shift. Furthermore, even if the pen attitude is detected by the spatial position detection unit 30, it is converted into a pen attitude relative to the input surface 21S of the digitizer 20, and is therefore appropriately reflected in the 3D drawn image.

[0072] As described above, the 3D drawn image information generated by the drawn image generation unit 421 is supplied to the VR image generation unit 422 to be converted into a VR image, and then supplied to the HMD 50 via the display drive unit 424 to be displayed. In this case, in this embodiment, the coordinate conversion unit 413 is used to convert the pointing position of the electronic pen 10 into spatial coordinates that are common to the digitizer 20 and the spatial position detection unit 30, so the spatial coordinate system of the 3D drawn image and the spatial coordinate system of the VR image of the electronic pen 10 and the like are the same. Therefore, the pointing position of the electronic pen 10 with respect to the 3D drawn image does not differ between the 3D drawing-based processing and the VR image-based processing, and the VR image generation unit 422 does not need to correct the spatial coordinates of the two.

[0073] As can be seen from the above description, the first embodiment has the advantage that, in 3D drawing processing, the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 can be used to generate a 3D drawn image, even if the electronic pen 10 and the digitizer 20 do not have a function for detecting the pen posture, such as the tilt angle or rotation angle, of the electronic pen 10. The first embodiment also has the advantage that information on the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 in a spatial region other than the position detection region DT of the digitizer 20 can be used for gesture detection to generate a 3D drawn image. Furthermore, the first embodiment has the advantage that the spatial coordinates of the detection region of the digitizer 20 and the spatial coordinates of the detection region of the spatial position detection unit 30 can be treated as common coordinates, so that no coordinate shift occurs when switching between the output from the digitizer 20 and the output from the spatial position detection unit 30 in 3D drawing processing.

[0074] Therefore, the operator can seamlessly switch from fine drawing to gesture operation simply by moving the electronic pen 10 spatially on the digitizer 20 without being aware of switching between the digitizer 20 and the spatial position detection unit 30.

[0075] Furthermore, in the first embodiment described above, in the VR image processing, the position detection output of the digitizer 20, which has higher accuracy and faster response speed than the detection output of the spatial position detection unit 30, can be used as the position information of the electronic pen 10 (position information of the pen tip) for generating a VR image of the electronic pen 10 in the spatial region of the position detection region DT of the digitizer 20, so that a VR image that responds accurately to the operation of the electronic pen by the actual user can be obtained.

[0076] Since the above-mentioned spatial drawing information generating device 40 is configured by a computer, it goes without saying that each part of the input information processing unit 41 and the display image generating unit 42 can be configured as a software function part executed by a software program.

[0077] [Second embodiment] In the first embodiment described above, the electronic pen 10 and the digitizer 20 are examples that do not have a function for detecting the pen attitude, such as the tilt angle or rotation angle of the electronic pen, but they can be configured to have a function for detecting the pen attitude, such as the tilt angle or rotation angle of the electronic pen.

[0078] The input device of the second embodiment described below is a system similar to the spatial position indication system described in the first embodiment, in which the electronic pen and digitizer have a function for detecting the pen attitude, such as the tilt angle and rotation angle of the electronic pen.

[0079] Fig. 4 shows an example of the configuration of the main parts of the spatial position indication system of the second embodiment. The spatial position indication system of the second embodiment has the same configuration as the first embodiment, but instead of the electronic pen 10 and the digitizer 20, it has an electronic pen 10A (not shown in Fig. 4) and a digitizer 20A that are configured to detect the pen attitude. And instead of the input information processing unit 41 of the spatial drawing information generating device 40 of the first embodiment, it has an input information processing unit 41A configured as shown in Fig. 4.

[0080] The spatial position indication system of the second embodiment also includes the spatial position detection unit 30, display image generation section 42, and HMD 50 described in the first embodiment, but since these are the same as those in the first embodiment, they are not shown in Fig. 4. In addition, in the digitizer 20A and input information processing section 41A shown in Fig. 4, the same parts as those in the digitizer 20 and input information processing section 41 shown in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted.

[0081] As shown in Figure 5(A), when the electronic pen 10A is perpendicular to the input surface of the sensor unit, the signal levels of a predetermined number of loop coils, for example, seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3, are symmetrical around the pointing position.

[0082] In contrast, when the electronic pen 10A is tilted, the signal levels of these seven loop coils, Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3, change depending on the tilt direction and angle of the electronic pen 10A, as shown in Fig. 5(B). Based on this predetermined number of signal levels, the position coordinates of the pointing position of the electronic pen 10A, as well as the tilt angle and tilt direction, can be detected.

[0083] 6, for example, in the peripheral area PE indicated by diagonal lines outside the dotted line, only a plurality of signal levels less than the predetermined number can be obtained. Therefore, in the second embodiment, the detection accuracy of the pointing position (pen tip position) and the pen attitude of the electronic pen 10A is low in the peripheral area PE of the digitizer 20A, and in this second embodiment, the detection accuracy of the pointing position (pen tip position) and the pen attitude of the electronic pen 10A detected by the spatial position detection unit 30 is higher.

[0084] That is, in this second embodiment, even if the pen tip position of the electronic pen 10A is in the spatial region of the position detection region DT in which the pointing position and pen attitude of the electronic pen 10A can be detected in the digitizer 20A, when it is in the peripheral region PE and the spatial region above it, the information on the pointing position and pen attitude of the electronic pen 10A detected by the spatial position detection unit 30 is used instead of the information on the pointing position and pen attitude of the electronic pen 10A detected by the digitizer 20A.

[0085] 4, the digitizer 20A of the second embodiment includes an attitude detection unit 24. This attitude detection unit 24 detects the pen attitude of the electronic pen 10A, such as the tilt angle, tilt direction, and rotation angle, based on the exchange of signals between the electronic pen 10A (not shown) and the sensor unit 22 of the digitizer 20A. Information about the pen attitude of the electronic pen 10A detected by this attitude detection unit 24 is supplied to one input terminal of a selection unit 418 of the input information processing unit 41A. Information about the pen attitude of the electronic pen 10A detected using the spatial position detection unit 30 is supplied to the other input terminal of this selection unit 418 from the pen attitude detection unit 4102 of the spatial information processing unit 410.

[0086] The pen posture information of the electronic pen 10A from the selection unit 418 is supplied to the association unit 415, and the information on the pointing position of the electronic pen 10A from the selection unit 411 is subjected to the same association process as described above, and then supplied to the drawing image generation unit 421 and gesture detection processing unit 423 of the display image generation unit 42.

[0087] Furthermore, in the second embodiment, information on the pen attitude of electronic pen 10A detected by pen attitude detection unit 4102 of spatial information processing unit 410 is supplied to one input terminal of selection unit 419. Information on the pen attitude of electronic pen 10A from attitude detection unit 24 of digitizer 20A is supplied to attitude conversion unit 417R, and information on the digitizer attitude including the tilt angle and tilt direction of digitizer 20A detected by digitizer attitude detection unit 4103 of spatial information processing unit 410 is supplied to attitude conversion unit 417R.

[0088] Attitude conversion section 417R performs the inverse conversion from that of attitude conversion section 417, using information on the pen attitude of electronic pen 10A from attitude detection section 24 of digitizer 20A and information on the attitude of digitizer 20A from digitizer attitude detection section 4103 of spatial information processing section 410. In other words, the relative pen attitude of electronic pen 10A with respect to input surface 21S detected by digitizer 20A is converted into an absolute attitude of digitizer 20A in the spatial coordinate system of motion detection space region MD detected by spatial position detection unit 30.

[0089] Then, the pen attitude information of the electronic pen 10A whose attitude has been converted by this attitude conversion unit 417R is supplied to the other input terminal of the selection unit 419. The pen attitude information of the electronic pen 10A selected and output by the selection unit 419 as described below is supplied to the association unit 416, where it is associated with the information on the pointing position of the electronic pen 10A from the selection unit 412 in the same manner as described above, and is supplied to the VR image generation unit 422 of the display image generation unit 42.

[0090] A selection control signal generation unit 414A of the second embodiment generates a selection control signal SEA for the selection units 411, 412, 418, and 419. The selection control signal generation unit 414A also generates a selection control signal SE similar to the selection control signal generation unit 414 of the first embodiment described above, and controls switching of the processes executed by the drawing image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42 in the same manner as in the first embodiment described above.

[0091] The selection control signal SEA from the selection control signal generation unit 414A of this second embodiment not only selects and controls the selection units 411, 412, 418, 419 depending on whether the pointing position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT of the digitizer 20A or another spatial area, but also selects and controls the selection units 411, 412, 418, 419 depending on whether it is in the peripheral area PE (see Figure 6) of the input surface 21S of the digitizer 20A or a central area inside the peripheral area PE of the input surface 21S.

[0092] For this reason, the selection control signal generation unit 414A of this second embodiment is supplied with information on the pointing position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A in addition to information on the signal level of the received signal from the electronic pen 10A. The selection control signal generation unit 414A generates a selection control signal SEA that selectively controls the selection units 411, 412, 418, 419 when the pointing position (position of the pen tip) of the electronic pen 10A is within the spatial region of the position detection region DT and also within the spatial region of the central region inside the peripheral region PE, and when it is within any other spatial region.

[0093] When the selection control signal SEA indicates that the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 411 selects the information on the pointing position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 415; when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 411 selects the information on the pointing position of the electronic pen 10A from the spatial position detection unit 4101 of the coordinate conversion unit 413 and supplies it to the association unit 415.

[0094] In addition, when the selection control signal SEA indicates that the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 412 selects the information on the pointing position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association unit 416, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 412 selects the information on the pointing position of the electronic pen 10A from the spatial position detection unit 4101 that has been coordinate converted by the coordinate conversion unit 413 and supplies it to the association unit 416.

[0095] In addition, when the selection control signal SEA indicates that the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 418 selects the pen posture information of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and supplies it to the association unit 415, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 418 selects the pen posture information converted into a relative pen posture from the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 from the posture conversion unit 417 and supplies it to the association unit 415.

[0096] In addition, when the selection control signal SEA indicates that the pointing position (pen tip position) of the electronic pen 10A is within the spatial region of the position detection region DT and within the spatial region of the central region inside the peripheral region PE, the selection unit 419 selects pen attitude information converted from the pen attitude of the electronic pen 10A detected by the attitude detection unit 24 of the digitizer 20A from the attitude conversion unit 417R into an absolute pen attitude detected by the spatial position detection unit 30 and supplies it to the association unit 416, and when the pointing position (pen tip position) of the electronic pen 10A is in a spatial region other than the above, the selection unit 419 selects pen attitude information of the electronic pen 10A from the pen attitude detection unit 4102 and supplies it to the association unit 416.

[0097] An example of the flow of processing operations when the input information processing unit 41A of the spatial position indication system of the second embodiment is configured by a computer will be described with reference to Fig. 7 and its sequel Fig. 8. In this case, each block configuring the input information processing unit 41A shown in Fig. 4 becomes a software function unit implemented by a computer program.

[0098] That is, the input information processing unit 41A determines, from a signal from the digitizer 20A, whether or not the digitizer 20A is ready to detect the position pointed to by the electronic pen 10A (step S1). If it is determined in step S1 that the digitizer 20A is not ready to detect the position pointed to by the electronic pen 10A, the input information processing unit 41A determines whether or not the spatial position detecting unit 30 can detect the electronic pen 10A (step S2). If it is determined in step S2 that the spatial position detecting unit 30 cannot detect the electronic pen 10A, the input information processing unit 41A returns the process to step S1.

[0099] When it is determined in step S1 that the digitizer 20A is in a state where it can detect the position pointed to by the electronic pen 10A, the input information processing unit 41A acquires the coordinates of the position pointed to by the electronic pen 10A detected by the digitizer 20A (step S3), and determines whether the acquired coordinates of the pointed to position are within the surrounding area PE (step S4).

[0100] In step S4, if it is determined that the coordinates of the acquired indication position are not in the peripheral area PE but in its inner area, the input information processing unit 41A associates the information on the indication position of the electronic pen 10A from the digitizer 20A with the information on the pen posture and outputs them to the display image generation unit 42 for generating a drawing image (step S5).

[0101] Next, the input information processing unit 41A performs coordinate conversion on the information of the pointing position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 to convert the information into information in a coordinate system common to that of the digitizer 20A (step S6). Then, the input information processing unit 41A associates the coordinate-converted information of the pointing position of the electronic pen 10A with the information of the pen attitude detected by the pen attitude detection unit 4102 of the spatial information processing unit 410, and outputs the information to the display image generation unit 42 for use in generating a VR image (step S7). After step S7, the input information processing unit 41A returns the process to step S1 and repeats the processes from step S1 onwards.

[0102] Next, when it is determined in step S4 that the coordinates of the acquired pointing position are within the peripheral region PE, and when it is determined in step S2 that electronic pen 10A can be detected by spatial position detection unit 30, input information processing unit 41A performs coordinate conversion to convert the information of the pointing position of electronic pen 10A detected by spatial position detection unit 4101 of spatial information processing unit 410 into information in a coordinate system common to digitizer 20A (step S11 in FIG. 8). Furthermore, input information processing unit 41A acquires information on the attitude of digitizer 20A detected by digitizer attitude detection unit 4103 of spatial information processing unit 410 (step S12).

[0103] Next, the input information processing unit 41A converts the pen attitude information from the attitude detection unit 24 of the digitizer 20A into absolute attitude information in the spatial domain, rather than a relative attitude with respect to the digitizer 20A, using the attitude information of the digitizer 20A acquired in step S12 (step S13).Then, the input information processing unit 41A associates the information on the pointing position of the electronic pen 10A obtained by the coordinate conversion in step S11 with the absolute attitude information obtained in step S13, and outputs the information to the display image generation unit 42 for use in generating a drawn image (step S14).

[0104] Next, the input information processing unit 41A converts the pen attitude information from the pen attitude detection unit 4102 of the spatial information processing unit 410 into information about the attitude relative to the digitizer 20A, using the attitude information of the digitizer 20A acquired in step S12 (step S15). Then, the input information processing unit 41A associates the information about the pointing position of the electronic pen 10A obtained by the coordinate conversion in step S11 with the information about the absolute attitude obtained in step S13, and outputs the associated information to the display image generation unit 42 for use in generating a VR image (step S16). After step S16, the input information processing unit 41A returns the process to step S1 and repeats the processes from step S1 onwards.

[0105] Since the input information processing unit 41A of the spatial position indication system of this second embodiment is configured as described above, in 3D drawing system processing and VR image system processing, in the central region of the spatial region of the position detection region DT of the digitizer 20A that is inside the peripheral region PE of the input surface 21S, information on the indication position of the electronic pen 10A detected by the position detection unit 23 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30, and in the spatial region of the spatial region of the position detection region DT of the digitizer 20A that is outside the peripheral region PE of the input surface 21S (including the space above the input surface 21S) and the spatial region of the position detection region DT, information on the indication position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0106] Furthermore, as information on the pen posture of the electronic pen 10A in 3D drawing system processing, in the central region of the spatial region of the position detection region DT of the digitizer 20A that is inside the peripheral region PE of the input surface 21S, information on the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A is used with higher accuracy than when detected by the spatial position detection unit 30, and in the spatial region of the spatial region of the position detection region DT of the digitizer 20A that is outside the peripheral region PE of the input surface 21S (including the space above the input surface 21S) and the spatial region of the position detection region DT, information on the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial position detection unit 30 is used with higher accuracy than when detected by the position detection unit 23 of the digitizer 20A.

[0107] Therefore, in the second embodiment, as in the first embodiment, it is possible to always use highly accurate information to perform 3D drawing processing and VR image processing. In this case, as in the first embodiment, there is an advantage that the information detected by the digitizer 20A and the information detected by the spatial position detection unit 30 can be mutually used in the 3D drawing processing and the VR image processing, respectively, which is convenient.

[0108] [Third embodiment] The first and second embodiments described above are cases in which an input device according to the present invention is applied to a spatial position indication system, but the input device according to the present invention can also be applied to cases in which a position indication by an electronic pen is detected within a position detection area DT including a hover area in a tablet device equipped with a digitizer.

[0109] Fig. 9 is a diagram showing an outline of a system including an input device according to the third embodiment. In Fig. 9, the input device according to the third embodiment is made up of an electronic pen 10B and a tablet device 60 equipped with a digitizer. As shown in Fig. 9, the tablet device 60 of this input device is placed, for example, on a desk 90 and connected to a computer 80. In this example, the tablet device 60 is placed at an inclination angle θ with respect to the surface of the desk 90 on which it is placed.

[0110] In the third embodiment, a gyro sensor unit 70 for detecting the pen attitude of the electronic pen 10B in the three-dimensional space above the input surface 61S of the digitizer built into the tablet device 60 is removably provided on the rear end side opposite the pen tip side of the electronic pen 10B. The gyro sensor unit 70 may also be built into the rear end side opposite the pen tip side of the electronic pen 10B.

[0111] 10 is a block diagram illustrating an example of the functional configuration of the electronic pen 10B, tablet device 60, and gyro sensor unit 70 that constitute the input device of this third embodiment. In this case, the gyro sensor unit 70 is configured to include, for example, a three-axis gyro sensor 71 and a wireless communication unit 72. The wireless communication unit 72 is configured, for example, by short-range wireless communication means conforming to the Bluetooth (registered trademark) standard. The wireless communication unit 72 is not limited to this, and may also be optical communication such as infrared communication.

[0112] In this example, the gyro sensor unit 70 is attached to the housing of the electronic pen 10B, and the triaxial gyro sensor 71 outputs an output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B in three-dimensional space. Here, the output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B detected by the triaxial gyro sensor 71 does not correspond to the relative orientation of the electronic pen 10B with respect to the input surface 61S of the tablet device 60, but corresponds to the absolute orientation of the electronic pen 10B based on the earth's axis in three-dimensional space. The wireless communication unit 72 wirelessly transmits the output signal from the triaxial gyro sensor 71 to the tablet device 60.

[0113] The electronic pen 10B has an interaction unit 100 that interacts with the sensor unit 61 of the tablet device 60. The interaction unit 100 is coupled to the sensor unit 61 of the tablet device 60 by either an electromagnetic induction method or an electrostatic coupling method to exchange signals. The interaction unit 100 of the electronic pen 10B in this example exchanges signals for detecting the position pointed to by the electronic pen 10B with the sensor unit 61, but does not exchange signals for detecting the pen attitude, such as the tilt angle, tilt direction, or rotation angle.

[0114] In the tablet device 60, a pointing position detection unit 62 detects the pointing position of the electronic pen 10B from a signal obtained as a result of interaction between the sensor unit 61 and the electronic pen 10B. Information on the pointing position of the electronic pen 10B detected by the pointing position detection unit 62 is supplied to an association unit 63.

[0115] The tablet device 60 of the third embodiment includes a wireless communication unit 64 that communicates wirelessly with the wireless communication unit 72 of the gyro sensor unit 70. The detection output of the three-axis gyro sensor 71 of the gyro sensor unit 70 received by this wireless communication unit 64 is supplied to an electronic pen attitude detection unit 65, which detects the pen attitude of the electronic pen 10B. Information on the pen attitude of the electronic pen 10B detected by this electronic pen attitude detection unit 65 is supplied to a relative attitude calculation unit 66.

[0116] The tablet device 60 of this embodiment is equipped with a gyro sensor 67 for detecting the tilt angle and tilt direction of the device itself. The sensor output of this gyro sensor 67 is supplied to a tablet attitude detection unit 68. The tablet attitude detection unit 68 detects the tilt angle and tilt direction of the device itself based on the sensor output of the gyro sensor 67, and supplies information on the detected tilt angle and tilt direction of the device itself (tablet attitude information) to a relative attitude calculation unit 66.

[0117] The relative attitude calculation unit 66 calculates the relative pen attitude of the electronic pen 10B with respect to the input surface 61S of the tablet device 60 by correcting the information on the absolute pen attitude of the electronic pen 10B in three-dimensional space from the electronic pen attitude detection unit 65 using the information on the tablet attitude from the tablet attitude detection unit 68. Then, the relative attitude calculation unit 66 supplies the calculated information on the relative pen attitude of the electronic pen 10B to the association unit 63.

[0118] The associating unit 63 associates information on the pointing position of the electronic pen 10B with information on the relative pen attitude of the electronic pen 10B, in the same manner as the above-described associating unit, and outputs the information to the computer 80.

[0119] As described above, in the input device of this third embodiment, when the position indicated by the pen tip of electronic pen 10B is obtained, the gyro sensor unit 70 provided on the rear end side of electronic pen 10B corresponding to this can output information on the pen attitude of electronic pen 10B detected in three-dimensional space as information on the pen attitude corresponding to the position indicated by electronic pen 10B.

[0120] [Other embodiments or modifications] In the first and second embodiments described above, the spatial position detection unit 30 is configured to include a light-emitting tracking device that emits infrared laser light and a tracker, but it goes without saying that the configuration is not limited to this. For example, it may be configured to use other non-visible light sensors, visible light sensors, or a combination thereof.

[0121] Furthermore, in the first and second embodiments described above, since the operator of the electronic pen 10 is wearing the HMD 50, the operator cannot directly see the digitizer 20. Therefore, it is also possible to draw a virtual image of the digitizer 20 on the virtual space image displayed on the HMD 50, so that the operator can recognize the position of the digitizer 20 on the display screen of the HMD 50.

[0122] Furthermore, in the first and second embodiments described above, in order to correct errors between the two spatial coordinate systems, the coordinate values ​​of the spatial position detection unit 30 are converted into coordinate values ​​of the spatial coordinate system of the digitizer 20, but conversely, the coordinate values ​​of the position detection region DT of the digitizer 20 may be converted into coordinate values ​​of the spatial position detection unit 30. Note that conversion between these two spatial coordinate systems is not essential and is optional.

[0123] In the first and second embodiments described above, the electronic pen and digitizer are of the electromagnetic induction type, but this is not limited to this, and it is of course also possible to use an electronic pen and digitizer of the electrostatic type (including the active electrostatic coupling type and the passive electrostatic coupling type).

[0124] The digitizers of the first and second embodiments and the tablet device of the third embodiment may be portable mobile phone terminals known as smartphones.

[0125] In the first and second embodiments, the case where 3D drawing is performed by the spatial position instruction system has been described, but the drawn image may be a 2D drawn image or a 2.5D drawn image. [Explanation of symbols]

[0126] 10, 10A, 10B... electronic pen, 20, 20A... digitizer, 21S... input surface, 22... sensor section, 23... position detection section, 24... attitude detection section, 30... spatial position detection unit, 40... spatial drawing information generation device, 41... input information processing section, 4101... spatial position detection section, 4102... pen attitude detection section, 50... HMD

Claims

1. a device having a digitizer that detects a position pointed to by an electronic pen; a tracking device that detects the position of a tracker of the electronic pen and that is used to input a position using the electronic pen in a virtual reality (VR) space set in a first coordinate system associated with the tracking device; a computer that generates a VR image of the electronic pen in the VR space based on the orientation of the electronic pen in a three-dimensional space where the electronic pen is tracked by the tracking device and based on coordinates in the first coordinate system calculated by converting coordinates in a second coordinate system associated with the device having a digitizer that correspond to a position pointed by the electronic pen and detected by the device having a digitizer; and A VR image generation system comprising:

2. the second coordinate system associated with the device having the digitizer is a coordinate system relative to a position detection area of ​​the device having the digitizer, and the first coordinate system associated with the tracking device is a coordinate system relative to a position detection area of ​​the tracking device; 2. The VR image generation system according to claim 1.

3. the computer causes a display unit to display a VR image relating to the electronic pen in the VR space; 2. The VR image generation system according to claim 1.

4. the computer calculates coordinates in a second coordinate system associated with the device having a digitizer when a position indicated by the electronic pen is within a predetermined distance from an input surface of the device having a digitizer; 2. The VR image generation system according to claim 1.

5. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light.

2. The VR image generation system according to claim 1.

6. The VR space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space.

2. The VR image generation system according to claim 1.

7. the computer calculates coordinates in a first coordinate system associated with the tracking device by correcting coordinates in a second coordinate system associated with an apparatus having the digitizer; 2. The VR image generation system according to claim 1.

8. The data relating to the attitude of the electronic pen includes a tilt angle, a tilt direction, or a rotation angle of the electronic pen; 2. The VR image generation system according to claim 1.

9. A VR image of the electronic pen in the VR space is generated based on the posture of the electronic pen in a three-dimensional space in which the electronic pen exists and is tracked by a tracking device set in a first coordinate system associated with a tracking device for performing position input by the electronic pen in the VR space, and coordinates in the first coordinate system calculated by coordinate conversion of coordinates in a second coordinate system associated with a device having a digitizer that corresponds to the position indicated by the electronic pen detected by the device having a digitizer that detects the position indicated by the electronic pen, A VR image generating method comprising:

10. the second coordinate system associated with the device having the digitizer is a coordinate system relative to a position detection area of ​​the device having the digitizer, and the first coordinate system associated with the tracking device is a coordinate system relative to a position detection area of ​​the tracking device; The VR image generating method according to claim 9 .

11. displaying a VR image relating to the electronic pen in the VR space on a display unit; The VR image generating method according to claim 9 .

12. calculating coordinates in a second coordinate system associated with the device having the digitizer when the position indicated by the electronic pen is within a predetermined distance from an input surface of the device having the digitizer; The VR image generating method according to claim 9 .

13. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light. The VR image generating method according to claim 9 .

14. The VR space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space. The VR image generating method according to claim 9 .

15. calculating coordinates in a first coordinate system associated with the tracking device by correcting coordinates in a second coordinate system associated with a device having the digitizer; The VR image generating method according to claim 9 .

16. The data regarding the posture includes a tilt angle, a tilt direction, or a rotation angle of the electronic pen. The VR image generating method according to claim 9 .

Images