Virtual reality image generation system and method

Abstract

The invention relates to a virtual reality image generation system and method. Provided is an input device capable of satisfactorily detecting the pointing position and pen orientation of an electronic pen in a three-dimensional space in consideration of accuracy and reaction speed. An input device includes: a first detection unit that detects a first instruction position of an electronic pen on the basis of transmission and reception of a signal to the electronic pen; a second detection unit that detects a second instruction position of the electronic pen in a three-dimensional space in which the electronic pen is located; and a control unit that controls whether or not to output the second indicated position in place of the first indicated position.

Description

[0001] This application is a divisional application of an application which entered the Chinese national phase on August 26, 2020, with PCT application number PCT / JP2018 / 047700, international application date December 26, 2018, Chinese application number 201880090305.2, and invention name “Input device using an electronic pen”. Technical Field

[0002] The present invention relates to an input device suitable for using an electronic pen in space. Background Art

[0003] There is known a drawing system that creates a moving image or the like by drawing by continuously indicating positions with an electronic pen to a coordinate input device called a digitizer.

[0004] In the past, various methods have been proposed as methods for detecting the tilt and rotation of an electronic pen. Patent document 1 (Japanese Patent Publication No. 2016-126503) provides an electronic pen capable of detecting the tilt angle and rotation angle of an input surface relative to a sensor unit of a position detection device. In the case of Patent document 1, the position detection device detects the position indicated by the electronic pen by transmitting and receiving signals using electromagnetic induction coupling, electrostatic capacitance coupling, etc. between the electronic pen and the position detection device having the sensor unit, and detects the position coordinates and the tilt angle and rotation angle of the position indicated by the electronic pen.

[0005] In addition, Patent Document 2 (U.S. Patent No. 9,329,703B2) states that a sensor capable of measuring the movement and orientation of the electronic pen is built into or installed in the electronic pen, and the movement and state (tilt, rotation) of the electronic pen can be detected in the position detection device by sending the detection output of the sensor to the position detection device.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2016-126503

[0009] Patent Document 2: U.S. Patent No. 9,329,703B2 Summary of the invention

[0010] Problems to be solved by the invention

[0011] In the case of the above-mentioned Patent Document 1, the position coordinates and the tilt of the position pointed by the electronic pen are detected using a plurality of signal levels obtained from a plurality of loop coils around the position pointed by the electronic pen as the center. However, there is a problem that the accuracy of the position coordinates of the position pointed by the electronic pen decreases in the peripheral area of ​​the sensor unit and it is difficult to detect the tilt of the electronic pen.

[0012] According to the method of Patent Document 2, the problem of Patent Document 1 does not occur. However, there is a problem that the response speed becomes slow or the accuracy of the position coordinates decreases depending on the conditions.

[0013] An object of the present invention is to provide an input device that can solve the above problems.

[0014] Means for solving problems

[0015] In order to solve the above-mentioned problem, an input device is provided, comprising:

[0016] a first detection unit for detecting a first indicated position of the electronic pen on the sensor unit based on the exchange of signals between the electronic pen and the sensor unit;

[0017] A second detection unit detects a second indication position of the electronic pen in the three-dimensional space where the electronic pen is located; and

[0018] The control unit generates position information of the electronic pen based on the first indicated position and the second indicated position.

[0019] In the input device of the above structure, the indicated position includes not only the position coordinates but also the posture (tilt, rotation). Furthermore, according to the input device of the above structure, for example, it is possible to output the information of the posture included in the indicated position of the electronic pen detected by the second detection unit together with the position coordinates of the first indicated position. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a diagram for explaining a configuration example of a spatial position indicating system including a first embodiment of the input device of the present invention.

[0021] Figure 2 It is used to illustrate Figure 1 A block diagram of an example of the structure of each part of the spatial position indication system of an example.

[0022] Figure 3 It is a diagram for explaining an embodiment of the input device of the present invention.

[0023] Figure 4 This is a block diagram for explaining a configuration example of each part of the second embodiment of the input device of the present invention.

[0024] Figure 5 This is a diagram for explaining the operation of detecting the tilt of the electronic pen by exchanging signals between the electronic pen and the digitizer.

[0025] Figure 6 This is a diagram for explaining that the detection accuracy when the digitizer detects the pointed position and tilt of the electronic pen by exchanging signals with the electronic pen differs between the peripheral area of ​​the input surface and other areas.

[0026] Figure 7 This is a diagram showing a part of a flowchart for explaining the main operation of the second embodiment of the input device of the present invention.

[0027] Figure 8 This is a diagram showing a part of a flowchart for explaining the main operation of the second embodiment of the input device of the present invention.

[0028] Fig. 9 It is a diagram for explaining a configuration example of a third embodiment of the input device of the present invention.

[0029] Fig.10 This is a block diagram for explaining a configuration example of each part of the third embodiment of the input device of the present invention. DETAILED DESCRIPTION

[0030] Below, while referring to the attached Figure 1 Embodiments of the input device of the present invention will be described below.

[0031] [First embodiment]

[0032] The embodiment of the input device described below relates to a spatial position indicating system, which includes: a digitizer (handwriting tablet), a sensor unit that sends and receives signals to and from an electronic pen (hereinafter referred to as "signal transmission and reception"), and a spatial position detection unit that detects the indicated position information of the electronic pen in space. It should be noted that the position includes not only the position coordinates but also the posture (tilt, rotation) and the like, but in the following description, the position coordinates and the posture information are described separately to make both clear.

[0033] In the spatial position indication system of this embodiment, the display unit is composed of a head-mounted display, smart glasses, etc., and uses 3D to depict the space as a virtual space (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc., hereinafter referred to as VR). Figure 1 This is a diagram showing an overview of the overall structure of a spatial position indication system including an input device according to a first embodiment that uses a VR space as a 3D rendering space. Figure 2 This is a block diagram showing a detailed configuration example of functions of each part of the spatial position indicating system including the input device according to the first embodiment.

[0034] That is, Figure 1 As shown in FIG. 1 , the spatial position indicating system of the embodiment of this example is configured to include an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial description information generating device 40, and a head mounted display (hereinafter referred to as HMD) 50. Figure 2 As shown, in this example, the space rendering information generating device 40 has the functions of an input information processing unit 41 and a display image generating unit 42 and is constituted by, for example, a computer.

[0035] It should be noted that the input device of the first embodiment includes a first detection unit that detects the indicated position of the electronic pen 10 by exchanging signals with the electronic pen 10, a second detection unit that detects the indicated position of the electronic pen 10 in the three-dimensional space where the electronic pen 10 is located, and a control unit that generates position information of the electronic pen 10 based on the indicated positions. In the first embodiment, the first detection unit is composed of the digital converter 20, the second detection unit is composed of the spatial position detection unit 30 and the spatial information processing unit 410 of the input processing unit 41, and the control unit is composed of the input information processing unit.

[0036] In the first embodiment, the electronic pen 10 uses an electromagnetic induction electronic pen as an example, but it is not limited to the electromagnetic induction method, and an electrostatic coupling method or the like may be used. The digitizer 20 includes a thin rectangular parallelepiped housing 21, the surface of which is used as an input surface 21S for position indication of the electronic pen 10. The digitizer 20 also includes a sensor unit 22 and a position detection unit 23 (see Figure 2 ).

[0037] Although not shown in the figure, the sensor unit 22 is composed of a plurality of loop coils arranged in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) of the housing of the digitizer 20. In this example, the digitizer 20 is configured as an electromagnetic induction method in conjunction with the electronic pen 10, but is not limited to the electromagnetic induction method.

[0038] Although omitted in the figure, the electronic pen 10 has a resonant circuit (omitted in the figure) composed of a coil and a capacitor on the pen tip side, and electromagnetic induction coupling is performed between the ring coil of the sensor part 22 of the digitizer 20 and the resonant circuit of the electronic pen 10, thereby transmitting and receiving signals between the electronic pen 10 and the sensor part 22 of the digitizer 20.

[0039] The position detection unit 23 of the digitizer 20 supplies a signal to the electronic pen 10 via the loop coil of the sensor unit 22, receives a signal fed back from the electronic pen 10 via the loop coil, and detects the position indicated by the electronic pen 10 in the detection area of ​​the sensor unit 22 based on the received signal. It should be noted that in this embodiment, the digitizer 20 is configured to detect the position indicated by the pen tip of the electronic pen 10 as the indicated position of the electronic pen 10.

[0040] In the digitizer 20 of this example, the plurality of loop coils of the sensor unit 22 are arranged so as to cover substantially the entire area of ​​the input surface 21S.

[0041] Furthermore, in this embodiment, the position detection area in the digitizer 20 that can detect the indicated position of the electronic pen 10 includes not only a planar area when the pen tip of the electronic pen 10 is in contact with the input surface 21S of the digitizer 20, but also a spatial area (a hovering area in a hovering state of the electronic pen 10) where the pen tip of the electronic pen 10 is not in contact with the input surface 21S of the digitizer 20 but is separated from the input surface 21S in a direction orthogonal to the input surface 21S (a Z-axis direction orthogonal to the X-axis direction and the Y-axis direction) but the indicated position of the electronic pen 10 can be detected by transmitting and receiving signals based on electromagnetic coupling.

[0042] In this example, the spatial position detection unit 30 is configured to be able to set a three-dimensional spatial region where the digitizer 20 is located, and in this three-dimensional spatial region, detect the indicated position of the pen tip of the electronic pen 10, the inclination angle, rotation angle of the electronic pen 10 and other postures of the electronic pen 10 (referred to as pen posture), and detect the position of the digitizer 20 and the inclination angle and inclination direction relative to the horizontal plane.

[0043] The spatial position detection unit 30 is configured to include two light emitting tracking devices 31A and 31B and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. It should be noted that in this embodiment, as described later, in the HMD 50, a 3D depicted image depicted in the motion detection space area MD including the position detection area DT of the digitizer 20 is displayed as a virtual display image, and a virtual display image of the electronic pen 10 is displayed.

[0044] The two light emitting tracking devices 31A and 31B have the same structure, and are respectively equipped with a laser light emitting unit for infrared laser, a search unit for searching within the motion detection space area MD using the emitted infrared laser, and a light position detection unit for detecting the light emitted by the light emitting unit of the tracker 32A, 32B, 32C, 32D that receives the infrared laser.

[0045] In this embodiment, as described above, the objects (exploration objects in the exploration space area of ​​the spatial position detection unit 30) equipped with the trackers 32A, 32B, 32C, and 32D are the electronic pen 10 and the digitizer 20. That is, in this example, in order to be able to notify the position of the digitizer 20 and the tilt angle and tilt direction relative to the horizontal plane, the trackers 32A and 32B are installed at the upper left corner and the lower right corner of the housing of the thin rectangular parallelepiped digitizer 20. In addition, in order to notify the position of the electronic pen 10 and the pen posture (tilt angle, rotation angle), the electronic pen 10 is equipped with a tracker 32C on the pen tip side, and a tracker 32D is installed on the rear end side of the housing of the electronic pen 10 on the opposite side of the pen tip side in the axial direction of the housing of the electronic pen 10.

[0046] The light emitting tracking devices 31A and 31B use a search unit to control the laser light emitting unit, and emit infrared laser light in a scanning manner in the motion detection space area MD to detect the position of the tracker. The trackers 32A, 32B, 32C, and 32D each use a sensor to monitor the light received by the infrared laser, and when the infrared laser light is detected by the sensor, the light emitting unit composed of the LED is turned on.

[0047] The light emitting tracking devices 31A and 31B detect the position of the object equipped with the tracker 32A, 32B, 32C, and 32D in the motion detection space area MD by detecting the light emission of the light emitting unit of the tracker 32A, 32B, 32C, and 32D. The light emitting tracking devices 31A and 31B are configured so that when the light emission of the light emitting unit of the tracker 32A, 32B, 32C, and 32D is detected, the time elapsed from the time of the infrared laser emission at the time of the detection can also be detected. In this case, the tracker 32A, 32B, 32C, and 32D each performs different light emission corresponding to its own identification information.

[0048] The two light emitting tracking devices 31A and 31B are connected to the space description information generating device 40 by wire or wirelessly, and notify the space description information generating device 40 of the detected spatial position information of the trackers 32A, 32, 32C, and 32D in the motion detection space area MD.

[0049] In this example, if Figure 2 As shown, the spatial position information of the trackers 32A, 32B, 32C, and 32D in the motion detection space area MD detected by the two light emitting tracking devices 31A and 31B is supplied to the spatial information processing unit 410 of the input information processing unit 41 of the spatial depiction information generating device 40 .

[0050] In this example, the spatial information processing unit 410 is composed of a spatial position detection unit 4101, a pen posture detection unit 4102, and a digitizer posture detection unit 4103. The spatial position detection unit 4101 detects the position of the digitizer 20 in the motion detection space area MD from the trackers 32A and 32B, and detects the indicated position of the electronic pen 10 as the position of the pen tip of the electronic pen 10 and the position of the rear end of the electronic pen 10 from the trackers 32C and 32D. In this example, the pen posture detection unit 4102 detects the pen posture including the tilt angle and rotation angle of the electronic pen 10 from the trackers 32C and 32D. In addition, the digitizer posture detection unit 4103 detects the indicated position of the electronic pen 10 as the position of the tip.

[0051] Furthermore, in this embodiment, the input information processing unit 41 generates information to be supplied to the display image generating unit 42 based on the information of the indicated position of the electronic pen 10 in the position detection area DT detected by the digitizer 20, the information of the position (indicated position and rear end position) of the electronic pen 10 in the motion detection space area MD detected by the space information processing unit 410 of the input information processing unit 41, the pen posture information, and the digitizer posture information. Furthermore, the input information processing unit 41 supplies the generated information to the display image generating unit 42.

[0052] And, in this embodiment, if Figure 2 As shown, the display image generating unit 42 of the space rendering information generating device 40 includes a rendered image generating unit 421 for generating a 3D rendered image and a VR image generating unit 422 for generating a VR image to be displayed on the HMD 50 .

[0053] The image generation unit 421 generates a 3D image based on the position indication of the electronic pen 10, and performs processing such as deformation, rotation, and movement of the 3D image based on the gestures performed by the operator of the electronic pen 10. In addition, in this embodiment, the image generation unit 421 also reflects the pen posture including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 relative to the input surface 21S of the digitizer 20 in the generation of the 3D image. For example, when the electronic pen 10 and the digitizer 20 exchange signals, the tilt angle and tilt direction relative to the input surface 21S of the digitizer 20 are used. Hereinafter, the processing related to the 3D image is referred to as 3D rendering system processing.

[0054] In this embodiment, the trackers 32C and 32D mounted on the electronic pen 10 can be used to detect the pen posture including the tilt angle, tilt direction, and rotation angle of the electronic pen 10 by the spatial position detection unit 30. Therefore, in this embodiment, the input information processing unit 41 is configured to supply the information of the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 to the display image generation unit 42 as information for 3D rendering system processing.

[0055] That is, as described later, the input information processing unit 41 is configured to supply information on the position indication of the electronic pen 10 (information on the position of the pen tip) and information on the pen posture of the electronic pen 10 to the display image generating unit 42 in a manner that reflects information on the pen posture such as the tilt and rotation of the electronic pen 10 during drawing input based on the position indication of the electronic pen 10 in the 3D drawing image.

[0056] In addition, if Figure 2 As shown, the display image generation unit 42 is provided with a gesture detection processing unit 423 for detecting a gesture performed by the operator of the electronic pen 10. In addition, in this embodiment, the input information processing unit 41 is configured to set the position information (corresponding to the indicated position) of the pen tip of the electronic pen 10 detected by the spatial position detection unit 30 and the information of the pen posture as information supplied to the gesture detection processing unit 423.

[0057] As described above, in this embodiment, as information for 3D rendering processing, not only the information of the indicated position of the electronic pen 10 detected by the digitizer 20 but also the information of the indicated position (position of the pen tip) and the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 are used.

[0058] The VR image generation unit 422 of the display image generation unit 42 generates a VR image to be displayed in the HMD 50. In this embodiment, the VR image includes a VR image of the electronic pen 10 and a VR image of a 3D image generated by the image generation unit 421. The VR image of the 3D image generated by the image generation unit 421 and the VR image of the electronic pen 10 are displayed in 3D in the HMD 50. It should be noted that the VR image generation unit 422 may also generate a VR image of the digitizer 20. Hereinafter, the processing related to the generation of the VR image in the VR image generation unit 422 is referred to as VR image processing.

[0059] In this embodiment, the VR image generation unit 422 is supplied with information for generating a VR image of the electronic pen 10 from the input information processing unit 41, and is supplied with information of the generated 3D rendered image from the rendered image generation unit 421. The VR image information generated by the VR image generation unit 422 is supplied to the HMD 50 via the display driving unit 424, and is displayed on a display screen of the HMD 50, for example, composed of an LCD (Liquid Crystal Display).

[0060] In this case, when the electronic pen 10 is present in the position detection area DT, information on the indicated position of the electronic pen from the digitizer 20 that can detect the position of the electronic pen 10 (pen tip position) with higher accuracy than the spatial position detection unit 30 and information on the pen posture such as tilt and rotation detected using the spatial position detection unit 30 are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0061] In addition, when the electronic pen 10 is present in the motion detection space area MD, the indicated position of the electronic pen 10 cannot be detected in the digitizer 20. Therefore, the information on the indicated position of the electronic pen and the information on the pen posture detected using the spatial position detection unit 30 are supplied from the input information processing unit 41 to the VR image generation unit 422 of the display image generation unit 42.

[0062] As described above, in this embodiment, as information for VR image processing, not only the information on the indicated position (position of the pen tip) of the electronic pen 10 and the information on the pen posture detected by the spatial position detection unit 30 but also the information on the indicated position of the electronic pen 10 detected by the digitizer 20 is used.

[0063] The input information processing unit 41 generates the information for the drawing system processing and the information for the VR image system processing described above based on the information from the digital converter 20 and the information from the spatial position detection unit 30, and supplies them to the display image generation unit 42, and generates a switching control signal SE for performing selection control corresponding to which area of ​​the position detection area DT and the motion detection space area MD the electronic pen 10 is located.

[0064] As described above, in this embodiment, in each of the 3D rendering system processing and the VR image system processing, the information of the spatial coordinate system of the position detection area DT of the digitizer 20 and the information of the spatial coordinate system of the motion detection space area MD of the spatial position detection unit 30 can be mutually utilized. In this case, the two spatial coordinate systems can be set independently, but in this embodiment, by converting the information of the spatial coordinate system of one of the two spatial coordinate systems into the information of the spatial coordinate system of the other, it is possible to process them as information in a common coordinate space. In this embodiment, the information of the spatial coordinate system of the motion detection space area MD of the spatial position detection unit 30 is converted into the information of the spatial coordinate system of the position detection area DT of the digitizer 20.

[0065] In the spatial position indicating system of this embodiment, the information of the pen posture of the electronic pen 10 detected in the motion detection space area MD of the spatial position detecting unit 30 can be used in the 3D rendering system processing and also in the VR image system processing.

[0066] In this case, in the VR image processing, the information of the pen posture of the electronic pen 10 is reflected in the posture of the VR image of the electronic pen 10. The information of the pen posture of the electronic pen 10 includes information of the tilt angle of the electronic pen 10, but in the 3D rendering processing, the tilt angle of the electronic pen 10 is a relative tilt angle with respect to the input surface of the sensor unit 22 of the digitizer 20. On the other hand, the information of the pen posture of the electronic pen 10 detected in the motion detection space area MD of the spatial position detection unit 30 is, for example, a tilt angle in the motion detection space area MD of the spatial position detection unit 30 based on the gravity direction of the earth or the horizontal plane direction.

[0067] If the digitizer 20 can be installed so that the direction perpendicular to the input surface 21S of the sensor section 22 thereof accurately matches the Z-axis direction of the motion detection space area MD of the spatial position detection unit 30, the tilt angle of the electronic pen 10 detected in the space of the motion detection space area MD of the spatial position detection unit 30 matches the relative tilt angle with respect to the input surface of the sensor section 22 of the digitizer 20. However, in practice, the digitizer 20 is sometimes installed tilted with respect to the motion detection space area MD of the spatial position detection unit 30.

[0068] Therefore, in this embodiment, the absolute tilt angle of the electronic pen 10 detected in the space of the motion detection space area MD of the spatial position detection unit 30 is converted into a relative tilt angle with respect to the input surface of the sensor unit 22 of the digitizer 20 as described below.

[0069] Figure 31 is a diagram for explaining the process of converting the tilt angle. Figure 3 1 shows a spherical coordinate system with the tip of the electronic pen 10 as the point O at the center of the sphere. Figure 3 The XY plane in FIG. 1 (a plane including the X-axis direction and the Y-axis direction orthogonal to each other) is set as a horizontal plane in the spatial area of ​​the motion detection spatial area MD of the spatial position detection unit 30 .

[0070] exist Figure 3 In , the point indicating the position of the edge of the rear end portion of the electronic pen 10 on the opposite side of the pen tip in the axial direction of the housing is a spherical coordinate. In this case, the radius r of the sphere is the length of the housing of the electronic pen 10 in the axial direction. Figure 3 In the spherical coordinate system of , when the end edge of the rear end portion of the housing of the electronic pen 10 in the axial direction is at point P, the tilt angle and tilt direction of the electronic pen 10 in the space of the action detection space area MD can be detected as δ and the tilt direction as α by using the spatial position detection unit 30. Figure 3 In the example, the tilt angle of the housing of the digitizer 20 in the space of the motion detection space region MD can be detected as δ', and the tilt direction can be detected as α'.

[0071] And, if Figure 3 As shown, if the position of the end edge of the rear end side of the electronic pen 10 when the axial direction of the shell of the electronic pen 10 becomes a state consistent with the vertical direction in the space of the motion detection space area MD is set to point Q, then the position of the end edge of the rear end side of the electronic pen 10 when the axial direction of the shell of the electronic pen 10 becomes a direction perpendicular to the input surface 21S of the sensor part 22 of the digitizer 20 is set to point R.

[0072] Therefore, if the angle formed by the spatial vector OR of point R from point O and the spatial vector OP of point P from point O is calculated, then by using the angle, the tilt angle and tilt direction of the electronic pen 10 detected by the spatial position detection unit 30 can be converted into the relative tilt angle and tilt direction of the electronic pen 10 relative to the input surface 21S of the sensor part 22 of the digitizer 20.

[0073] It should be noted that, by performing the above-mentioned inverse transformation, the relative tilt angle and tilt direction of the electronic pen 10 with respect to the input surface 21S of the sensor unit 22 of the digitizer 20 can be transformed into the tilt angle and tilt direction of the electronic pen 10 detected by the spatial position detection unit 30 .

[0074] Next, the configuration for realizing the above functions Figure 24. In other words, the position detection unit 23 constituting the digitizer 20 supplies the detection output of the indicated position of the electronic pen 10 to the selection unit 411 as one of its input signals, and supplies it to the selection unit 412 as one of its input signals. It should be noted that the information supplied from the position detection unit 23 to the selection units 411 and 412 includes, in addition to the detection output of the indicated position of the electronic pen 10, the writing pressure information applied to the electronic pen 10.

[0075] In addition, the spatial position detection unit 4101 of the spatial information processing unit 410 supplies the detection output of the spatial position of the pen tip of the electronic pen 10 (the indicated position of the electronic pen) to the coordinate conversion unit 413. The coordinate conversion unit 413 converts the information of the spatial coordinate system of the motion detection space area MD of the spatial position detection unit 30 into the information of the spatial coordinate system of the position detection area DT of the digitizer 20. Then, the coordinate conversion unit 413 supplies the converted coordinate output to the selection units 411 and 412 as the input signal of the other one.

[0076] Furthermore, in this embodiment, the position detection unit 23 of the digitizer 20 supplies information on the signal level of the reception signal from the electronic pen 10 to the selection control signal generation unit 414. The selection control signal generation unit 414 detects the separation distance of the pen tip of the electronic pen 10 from the input surface 21S of the digitizer 20 based on the signal level of the reception signal from the electronic pen 10, and generates the selection control signal SE based on the detected separation distance from the input surface 21S.

[0077] In this case, the selection control signal SE is the following signal: when the separation distance between the pen tip of the electronic pen 10 and the input surface 21S of the digitizer 20 is below the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hovering state of the electronic pen, control is performed in a manner such that the position detection output from the position detection unit 23 of the digitizer 20 is selected; when it is greater than the above-mentioned critical height Lz, control is performed in a manner such that the detection output from the spatial position detection unit 4101 is selected.

[0078] The selection unit 411 switches one of the inputs of one side and the inputs of the other side according to the selection control signal SE, and supplies it to the association establishment unit 415 for 3D rendering system processing. In addition, the selection unit 412 switches one of the inputs of one side and the inputs of the other side according to the selection control signal SE, and supplies it to the association establishment unit 416 for VR image system processing.

[0079] It should be noted that in Figure 2In the example, two selection units 411 and 412 are provided for each processing in a manner that clearly divides the processing into 3D rendering system processing and VR image system processing. However, in fact, the selection unit can be a shared one, and the output of the shared selection unit can be supplied to each of the association establishment units 415 and 416.

[0080] The information of the pen posture detected by the pen posture detection unit 4102 of the spatial information processing unit 410 using the spatial position detection unit 30 is directly supplied to the association unit 416 for VR image processing.

[0081] The association unit 416 for VR image processing associates the information of the indicated position of the electronic pen 10 from the selection unit 412 with the information of the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410, and supplies them as a pair to the VR image generation unit 422 of the display image generation unit 42. In this case, association in the association unit 416 means associating the information of the pen posture detected at the indicated position of the electronic pen 10 with the indicated position of the electronic pen 10 output from the selection unit 412 as a pair.

[0082] When the information of the indicated position of the electronic pen 10 and the pen posture that are associated in the association establishing unit 416 are both from the spatial information processing unit 410, the output timing of these information is adjusted by the spatial information processing unit 410, and in the association establishing unit 416, the association can be established simply by pairing the information with the same timing.

[0083] In contrast, when the information on the indicated position of the electronic pen 10 associated in the association establishing unit 416 is information from the digitizer 20, the association establishing unit 416 associates the information on the indicated position of the electronic pen 10 with the information on the pen posture by taking into account the deviation (processing delay) in the output timing of the detection results in the digitizer 20 and the spatial position detection unit 30.

[0084] The VR image generating unit 422 of the display image generating unit 42 generates a VR image of the electronic pen 10 using the information of the pointed position of the electronic pen 10 and the information of the pen posture from the associating unit 416. In this case, the VR image of the electronic pen 10 is generated so as to be displayed at a position based on the information of the pointed position of the electronic pen 10.

[0085] In this case, the information on the indicated position of the electronic pen 10 from the selection unit 412 is switched and selected by the selection control signal SE, but the information on the indicated position from the spatial position detection unit 4101 is coordinate-transformed by the coordinate transformation unit 413 in such a manner as to become a common coordinate space in the digitizer 20 and the spatial position detection unit 30. Therefore, when the selection unit 412 is switched, there will be no display deviation such as the display position of the VR image of the electronic pen 10 moving in a jumpy manner.

[0086] In addition, the information of the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied to the association establishing unit 415 for 3D rendering system processing through the posture conversion unit 417. The information of the pen posture from the pen posture detection unit 4102 and the information of the posture of the digitizer 20 detected by the digitizer posture detection unit 4103 are supplied to the posture conversion unit 417, and the posture conversion unit 417 performs the operation of using the pen posture detection unit 4102 to detect the pen posture of the digitizer 20. Figure 3 The process described above is to convert the pen posture of the electronic pen 10 in the spatial coordinate system detected by the spatial position detection unit 30 into the relative pen posture with respect to the input surface 21S of the sensor unit 22 of the digitizer 20 .

[0087] The associating unit 415 for 3D drawing processing associates information about the indicated position of the electronic pen 10 from the selecting unit 411 and information about the pen posture detected by the pen posture detecting unit 4102 from the posture converting unit 417, which is converted into information about the relative pen posture with respect to the input surface 21S of the digitizer 20, and supplies the information as a pair to the drawn image generating unit 421 and the gesture detecting unit 423 of the display image generating unit 42. In this case, the association in the associating unit 415 also means associating the information about the pen posture detected at the indicated position of the electronic pen 10 with the indicated position of the electronic pen 10 output from the selecting unit 412 as a pair, similarly to the association unit 416.

[0088] The drawing image generating unit 421 has a pen drawing function for drawing fine line drawings etc. based on the detection output of the indicated position of the electronic pen 10 from the digitizer 20 and the information of the pen posture detected by the spatial position detecting unit 30 in this embodiment, and a gesture processing function for performing drawing processing based on the action (gesture) detected by the gesture detection processing unit 423 according to the spatial position and pen posture of the electronic pen 10 detected by the spatial position detecting unit 30.

[0089] In addition, the selection control signal SE from the selection control signal generating unit 414 is supplied to the drawing image generating unit 421 and the gesture detection processing unit 423, and the gesture detection processing unit 423 is controlled in such a manner as to perform an action when the indicated position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20, and the drawing image generating unit 421 is switch-controlled in such a manner as to execute the pen drawing function when the indicated position of the electronic pen 10 is within the spatial area of ​​the position detection area DT of the digitizer 20 and to execute gesture processing when the indicated position of the electronic pen 10 is outside the spatial area of ​​the position detection area DT of the digitizer 20.

[0090] In this case, in this embodiment, since the coordinate conversion is performed using the coordinate conversion unit 413 so as to form a common coordinate space in the digitizer 20 and the spatial position detection unit 30, the indicated position of the electronic pen 10 does not deviate in a jumpy manner even if the indicated position of the electronic pen 10 is switched by the selection unit 411. In addition, even if the pen posture is detected by the spatial position detection unit 30, it is converted into a pen posture relative to the input surface 21S of the digitizer 20, and is therefore appropriately reflected in the 3D rendered image.

[0091] As described above, the 3D rendered image information generated by the rendered image generation unit 421 is supplied to the VR image generation unit 422 to be set as a VR image, and is supplied to the HMD 50 through the display drive unit 424 and displayed on the HMD 50. In this case, in this embodiment, the indicated position of the electronic pen 10 becomes a common spatial coordinate in the digitizer 20 and the spatial position detection unit 30 by using the coordinate conversion unit 413, so the spatial coordinate system of the 3D rendered image is the same as the spatial coordinate system of the VR image of the electronic pen 10, etc. Therefore, the indicated position of the electronic pen 10 relative to the 3D rendered image does not differ between the 3D rendering system processing and the VR image system processing, and therefore, the VR image generation unit 422 does not need to correct the spatial coordinates of both.

[0092] As can be seen from the above description, in the first embodiment, the following effect is achieved: in the 3D rendering system process, even if the electronic pen 10 and the digitizer 20 do not have the function of detecting the pen posture of the electronic pen 10, such as the tilt angle and the rotation angle, the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 can be used in the generation of the 3D rendering image. In addition, in the first embodiment, the following effect is also achieved: in the spatial area other than the position detection area DT of the digitizer 20, the information of the pen posture of the electronic pen 10 detected by the spatial position detection unit 30 can be used as gesture detection for the generation of the 3D rendering image. In addition, in the first embodiment, there is a feature that since the spatial coordinates of the detection area of ​​the digitizer 20 and the spatial coordinates of the detection area of ​​the spatial position detection unit 30 can be processed as common coordinates, no coordinate offset occurs when the output from the digitizer 20 and the output from the spatial position detection unit 30 are switched in the 3D rendering system process.

[0093] Therefore, the operator can seamlessly perform gesture-based operations from fine drawing simply by spatially moving the electronic pen 10 on the digitizer 20 without being aware of the switching between the digitizer 20 and the spatial position detecting unit 30 .

[0094] In addition, in the above-mentioned first embodiment, in the VR image system processing, as the position information of the electronic pen 10 (position information of the pen tip) for generating a VR image of the electronic pen 10, the position detection output of the digitizer 20 with higher accuracy and faster response speed than the detection output in the spatial position detection unit 30 can be used in the spatial area of ​​the position detection area DT of the digitizer 20, thereby obtaining a VR image that accurately reflects the actual operation of the electronic pen by the user.

[0095] It should be noted that since the above-mentioned space rendering information generating device 40 is constituted by a computer, each unit of the input information processing unit 41 and the display image generating unit 42 can be constituted as a software function unit executed by a software program.

[0096] [Second embodiment]

[0097] The first embodiment described above is an example in which the electronic pen 10 and the digitizer 20 do not have a function of detecting the pen posture such as the tilt angle and the rotation angle of the electronic pen, but a configuration having a function of detecting the pen posture such as the tilt angle and the rotation angle of the electronic pen is possible.

[0098] The input device of the second embodiment described below is a system similar to the spatial position indicating system described in the first embodiment described above, in which the electronic pen and the digitizer have a function of detecting the pen posture such as the tilt angle and rotation angle of the electronic pen.

[0099] Figure 4 The spatial position indicating system of the second embodiment has the same structure as the first embodiment, but an electronic pen 10A having a structure for detecting a pen posture is provided instead of the electronic pen 10 and the digitizer 20 (in Figure 4 In addition, the input information processing unit 41 of the space description information generating device 40 in the first embodiment is replaced by the following Figure 4 The input information processing unit 41A has the structure shown.

[0100] The spatial position indicating system of the second embodiment also includes the spatial position detecting unit 30, the display image generating unit 42, and the HMD 50 described in the first embodiment, but they are the same as those of the first embodiment. Figure 4 In addition, Figure 4 In the digital converter 20A and the input information processing unit 41A shown in FIG. Figure 2 The same parts of the digitizer 20 and the input information processing unit 41 are denoted by the same reference numerals, and detailed description thereof is omitted.

[0101] like Figure 5 As shown in (A), when the electronic pen 10A is perpendicular to the input surface of the sensor unit, the signal levels of a predetermined number of (e.g., 7) loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3 are symmetrical around the indicated position.

[0102] On the other hand, when the electronic pen 10A is tilted, Figure 5 As shown in (B), the signal levels of the seven loop coil positions Xi-3, Xi-2, Xi-1, Xi, Xi+1, Xi+2, and Xi+3 vary according to the tilt direction and tilt angle of the electronic pen 10A. Based on the predetermined number of signal levels, the position coordinates of the pointed position of the electronic pen 10A and the tilt angle and tilt direction can be detected.

[0103] However, in Figure 6In the input surface (detection area) INS of the sensor unit shown, for example, in the peripheral area PE indicated by the diagonal lines on the outside of the dotted line, only a plurality of signal levels less than the above-mentioned predetermined number can be obtained. Therefore, in the peripheral area PE of the digitizer 20A, the detection accuracy of the indicated position (pen tip position) of the electronic pen 10A and the pen posture of the electronic pen 10A becomes low in the second embodiment. In the case of the second embodiment, the detection accuracy of the indicated position (pen tip position) of the electronic pen 10A and the pen posture of the electronic pen 10A detected by the spatial position detection unit 30 becomes high.

[0104] That is, in this second embodiment, even if the pen tip position of the electronic pen 10A is in the spatial area of ​​the position detection area DT that can detect the indicated position and pen posture of the electronic pen 10A in the digitizer 20A, when it is in the peripheral area PE and the spatial area above it, the information on the indicated position and pen posture of the electronic pen 10A detected by the spatial position detection unit 30 is used instead of the information on the indicated position and pen posture of the electronic pen 10A detected by the digitizer 20A.

[0105] like Figure 4 As shown in FIG. 1 , the digitizer 20A of the second embodiment includes a posture detection unit 24. The posture detection unit 24 detects the pen posture of the electronic pen 10A, such as the tilt angle, tilt direction, and rotation angle, based on the exchange of signals between the electronic pen 10A (not shown) and the sensor unit 22 of the digitizer 20A. The information of the pen posture of the electronic pen 10A detected by the posture detection unit 24 is supplied to one input terminal of the selection unit 418 of the input information processing unit 41A. The information of the pen posture of the electronic pen 10A detected by the unit 30 for detecting a spatial position from the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied to the other input terminal of the selection unit 418.

[0106] The information on the pen posture of the electronic pen 10A from the selection unit 418 is supplied to the association unit 415 , and is subjected to the same association processing as described above with the information on the indicated position of the electronic pen 10A from the selection unit 411 , and is supplied to the drawn image generation unit 421 and the gesture detection processing unit 423 of the display image generation unit 42 .

[0107] In the second embodiment, the information of the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial information processing unit 410 is supplied to one input terminal of the selection unit 419. Furthermore, the information of the pen posture of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A is supplied to the posture conversion unit 417R, and the information of the digitizer posture including the tilt angle and tilt direction of the digitizer 20A detected by the digitizer posture detection unit 4103 of the spatial information processing unit 410 is supplied to the posture conversion unit 417R.

[0108] In the posture conversion unit 417R, the inverse conversion is performed with respect to the posture conversion unit 417 based on the information of the pen posture of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and the information of the posture of the digitizer 20A from the digitizer posture detection unit 4103 of the spatial information processing unit 410. That is, the relative pen posture of the electronic pen 10A with respect to the input surface 21S detected by the digitizer 20A is converted into the absolute posture of the digitizer 20A in the spatial coordinate system of the motion detection space area MD detected by the spatial position detection unit 30.

[0109] Then, the information of the pen posture of the electronic pen 10A after the posture transformation by the posture transformation unit 417R is supplied to the other input terminal of the selection unit 419. The information of the pen posture of the electronic pen 10A selected and output by the selection unit 419 as described later is supplied to the association unit 416, and is subjected to the same association process as described above with the information of the indicated position of the electronic pen 10A from the selection unit 412, and is supplied to the VR image generation unit 422 of the display image generation unit 42.

[0110] The selection control signal generating unit 414A of the second embodiment generates the selection control signal SEA for the selecting units 411, 412, 418, and 419. The selection control signal generating unit 414A generates the same selection control signal SE as the selection control signal generating unit 414 of the first embodiment described above, and performs switching control on the processing performed by the depicted image generating unit 421 and the gesture detection processing unit 423 of the display image generating unit 42 in the same manner as in the first embodiment described above.

[0111] The selection control signal SEA from the selection control signal generating unit 414A of the second embodiment controls the selection of the selection units 411, 412, 418, and 419 not only according to whether the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT of the digitizer 20A or outside the spatial area thereof, but also according to whether the indicated position (pen tip position) of the electronic pen 10A is within the spatial area of ​​the position detection area DT of the digitizer 20A or outside the spatial area thereof. Figure 6) or the central area inside the peripheral area PE of the input surface 21S to select and control the selection units 411, 412, 418, and 419.

[0112] Therefore, in addition to the information on the signal level of the received signal from the electronic pen 10A, the position detection unit 23 of the digitizer 20A also supplies the information on the indicated position of the electronic pen 10A to the selection control signal generation unit 414A of the second embodiment. The selection control signal generation unit 414A generates the selection control signal SEA for performing selection control on the selection units 411, 412, 418, and 419 when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and the spatial area of ​​the central area inside the peripheral area PE, and when it is in the spatial area outside the spatial area.

[0113] The selection unit 411 selects the control signal SEA and, when the indicated position of the electronic pen 10A (the position of the pen tip) is within the spatial area of ​​the position detection area DT and within the spatial area of ​​the central area inside the peripheral area PE, selects the information of the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association establishing unit 415; and, when the indicated position of the electronic pen 10A (the position of the pen tip) is within a spatial area other than the above, selects the information of the indicated position of the electronic pen 10A from the spatial position detection unit 4101 from the coordinate transformation unit 413 and supplies it to the association establishing unit 415.

[0114] In addition, the selection unit 412 selects the information on the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association establishing unit 416 when the indicated position of the electronic pen 10A (the position of the pen tip) is within the spatial area of ​​the position detection area DT and within the spatial area of ​​the central area inside the peripheral area PE by selecting the control signal SEA, and selects the information on the indicated position of the electronic pen 10A from the position detection unit 23 of the digitizer 20A and supplies it to the association establishing unit 416, and selects the information on the indicated position of the electronic pen 10A from the spatial position detection unit 4101 after the coordinate transformation processing is performed in the coordinate transformation unit 413 and supplies it to the association establishing unit 416 when the indicated position of the electronic pen 10A (the position of the pen tip) is within the spatial area other than the above.

[0115] In addition, the selection unit 418 selects the information on the pen posture of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and supplies it to the association establishment unit 415 when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and within the spatial area of ​​the central area inside the peripheral area PE by selecting the control signal SEA, and selects the information on the pen posture of the electronic pen 10A from the posture detection unit 24 of the digitizer 20A and supplies it to the association establishment unit 415, and selects the information on the pen posture after the pen posture of the electronic pen 10A is transformed from the pen posture detected by the pen posture detection unit 4102 to the relative pen posture from the posture transformation unit 417 and supplies it to the association establishment unit 415 when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial area other than the above.

[0116] In addition, the selection unit 419 selects the control signal SEA and, when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial area of ​​the position detection area DT and within the spatial area of ​​the central area inside the peripheral area PE, selects the information on the pen posture from the posture conversion unit 417R after the pen posture of the electronic pen 10A is converted from the pen posture detected by the posture detection unit 24 of the digitizer 20A to the absolute pen posture detected by the spatial position detection unit 30, and supplies it to the association establishment unit 416; when the indicated position (the position of the pen tip) of the electronic pen 10A is within the spatial area other than the above, selects the information on the pen posture of the electronic pen 10A from the pen posture detection unit 4102, and supplies it to the association establishment unit 416.

[0117] Reference Figure 7 and as a follow-up Figure 8 An example of the flow of processing operations when the input information processing unit 41A of the spatial position indication system of the second embodiment is composed of a computer is described below. Figure 4 Each block of the input information processing unit 41A shown is a software function unit implemented by a computer program.

[0118] That is, the input information processing unit 41A determines whether the state of detecting the indicated position of the electronic pen 10A by the digitizer 20A is reached based on the signal from the digitizer 20A (step S1). If it is determined in step S1 that the state of detecting the indicated position of the electronic pen 10A by the digitizer 20A is not reached, the input information processing unit 41A determines whether the electronic pen 10A can be detected by the spatial position detection unit 30 (step S2). If it is determined in step S2 that the electronic pen 10A cannot be detected by the spatial position detection unit 30, the input information processing unit 41A returns the process to step S1.

[0119] When it is determined in step S1 that the indicated position of the electronic pen 10A is detected in the digitizer 20A, the input information processing unit 41A obtains the coordinates of the indicated position of the electronic pen 10A detected by the digitizer 20A (step S3), and determines whether the obtained coordinates of the indicated position are within the peripheral area PE (step S4).

[0120] When it is determined in step S4 that the coordinates of the acquired indicated position are not in the peripheral area PE but in the inner area thereof, the input information processing unit 41A associates the information of the indicated position of the electronic pen 10A from the digitizer 20A with the information of the pen posture and outputs it to the display image generating unit 42 for generating a drawn image (step S5 ).

[0121] Next, the input information processing unit 41A performs coordinate transformation for converting the information of the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 into information of a coordinate system common to the digitizer 20A (step S6). Then, the input information processing unit 41A associates the information of the indicated position of the electronic pen 10A after the coordinate transformation with the information of the pen posture detected by the pen posture detection unit 4102 of the spatial information processing unit 410, and outputs it to the display image generation unit 42 for VR image generation (step S7). After step S7, the input information processing unit 41A returns the process to step S1, and repeats the process after step S1.

[0122] Next, when it is determined in step S4 that the coordinates of the acquired indicated position are within the peripheral area PE, and when it is determined in step S2 that the electronic pen 10A can be detected by the spatial position detection unit 30, the input information processing unit 41A performs coordinate transformation (i.e., coordinate transformation) for converting the information of the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial information processing unit 410 into information of a coordinate system common to the digitizer 20A. Figure 8 In addition, the input information processing unit 41A obtains information on the posture of the digitizer 20A detected by the digitizer posture detection unit 4103 of the spatial information processing unit 410 (step S12).

[0123] Next, the input information processing unit 41A uses the information on the posture of the digitizer 20A obtained in step S12 to convert the information on the pen posture from the posture detection unit 24 of the digitizer 20A into information on the absolute posture in the spatial region rather than the relative posture with respect to the digitizer 20A (step S13). Then, the input information processing unit 41A associates the information on the indicated position of the electronic pen 10A obtained by performing the coordinate conversion in step S11 with the information on the absolute posture obtained in step S13, and outputs the information to the display image generation unit 42 for generating a drawn image (step S14).

[0124] Next, the input information processing unit 41A uses the information on the posture of the digitizer 20A obtained in step S12 to convert the information on the pen posture from the pen posture detection unit 4102 of the spatial information processing unit 410 into information on the relative posture with respect to the digitizer 20A (step S15). Then, the input information processing unit 41A associates the information on the indicated position of the electronic pen 10A obtained by the coordinate conversion in step S11 with the information on the absolute posture obtained in step S13, and outputs it to the display image generation unit 42 for VR image generation (step S16). After step S16, the input information processing unit 41A returns the process to step S1, and repeats the process after step S1.

[0125] Since the input information processing unit 41A of the spatial position indication system of the second embodiment is constructed as described above, the information on the indicated position of the electronic pen 10A in the 3D rendering system processing and the VR image system processing uses the information on the indicated position of the electronic pen 10A detected by the position detection unit 23 of the digitizer 20A with higher accuracy than that detected by the spatial position detection unit 30 in the central area inside the peripheral area PE of the input surface 21S in the spatial area of ​​the position detection area DT of the digitizer 20A, and uses the information on the indicated position of the electronic pen 10A detected by the spatial position detection unit 4101 of the spatial position detection unit 30 with higher accuracy than that detected by the position detection unit 23 of the digitizer 20A in the spatial area of ​​the position detection area DT of the digitizer 20A.

[0126] In addition, as information on the pen posture of the electronic pen 10A in the 3D drawing system processing, information on the pen posture of the electronic pen 10A detected by the posture detection unit 24 of the digitizer 20A with higher accuracy than that detected in the spatial position detection unit 30 is used in the central area on the inner side of the peripheral area PE of the input surface 21S in the spatial area of ​​the position detection area DT of the digitizer 20A, and information on the pen posture of the electronic pen 10A detected by the pen posture detection unit 4102 of the spatial position detection unit 30 with higher accuracy than that detected in the position detection unit 23 of the digitizer 20A is used in the spatial area of ​​the peripheral area PE of the input surface 21S (including the space above the input surface 21S) in the spatial area of ​​the position detection area DT of the digitizer 20A and in the spatial area outside the spatial area of ​​the position detection area DT.

[0127] Therefore, in the second embodiment, it is also possible to always use high-precision information to perform 3D rendering system processing and VR image system processing as in the first embodiment. In this case, as in the first embodiment, there is the following advantage: in each of the 3D rendering system processing and the VR image system processing, the information detected by the digitizer 20A and the information detected by the spatial position detection unit 30 can be mutually utilized, which is convenient.

[0128] [Third Embodiment]

[0129] The first and second embodiments described above are cases where the input device of the present invention is applied to a spatial position indication system, but the input device of the present invention can also be applied to a case where a position indication of an electronic pen is detected within a position detection area DT including a hovering area in a tablet device equipped with a digitizer.

[0130] Fig. 9 1 is a diagram showing an overview of a system including an input device according to the third embodiment. Fig. 9 In the third embodiment, the input device is composed of an electronic pen 10B and a tablet device 60 having a digitizer. Fig. 9 As shown, the tablet device 60 of the input device is placed on, for example, a table 90 and connected to a computer 80. In this example, the tablet device 60 is placed at an inclination angle θ with respect to the placement surface of the computer 90.

[0131] Furthermore, in the third embodiment, the gyro sensor unit 70 for detecting the pen posture of the electronic pen 10B in the three-dimensional space above the input surface 61S of the digitizer built into the tablet device 60 is provided in a pluggable manner at the rear end side of the electronic pen 10B on the side opposite to the pen tip side. It should be noted that the gyro sensor unit 70 may also be built into the rear end side of the electronic pen 10B on the side opposite to the pen tip side.

[0132] Fig.10 1 is a block diagram for explaining an example of the functional configuration of the electronic pen 10B, the tablet device 60, and the gyro sensor unit 70 constituting the input device of the third embodiment. In this case, the gyro sensor unit 70 is configured to include, for example, a three-axis gyro sensor 71 and a wireless communication unit 72. The wireless communication unit 72 is, for example, a short-range wireless communication unit of the Bluetooth (registered trademark) standard. The wireless communication unit 72 is not limited to this, and may also be optical communication such as infrared communication.

[0133] In this example, the gyro sensor unit 70 is mounted on the housing of the electronic pen 10B, so the three-axis gyro sensor 71 outputs an output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B in three-dimensional space. Here, the output signal corresponding to the tilt angle, tilt direction, and rotation angle of the electronic pen 10B detected by the three-axis gyro sensor 71 does not correspond to the relative posture of the electronic pen 10B with respect to the input surface 61S of the tablet device 60, but corresponds to the absolute posture based on the earth's axis in three-dimensional space. The wireless communication unit 72 wirelessly transmits the output signal from the three-axis gyro sensor 71 to the tablet device 60.

[0134] The electronic pen 10B includes an interaction unit 100 that interacts with the sensor unit 61 of the tablet device 60. The interaction unit 100 is coupled to the sensor unit 61 of the tablet device 60 by either electromagnetic induction or electrostatic coupling to exchange signals. The interaction unit 100 of the electronic pen 10B in this example exchanges signals for detecting the position of the electronic pen 10B with the sensor unit 61, but does not exchange signals for detecting the pen posture such as the tilt angle, tilt direction, and rotation angle.

[0135] In tablet device 60 , indicated position detection unit 62 detects the indicated position of electronic pen 10B based on a signal obtained as a result of interaction with electronic pen 10B in sensor unit 61 . Information on the indicated position of electronic pen 10B detected by indicated position detection unit 62 is supplied to association unit 63 .

[0136] The tablet device 60 of the third embodiment includes a wireless communication unit 64 for wirelessly communicating with the wireless communication unit 72 of the gyro sensor unit 70. The detection output of the three-axis gyro sensor 71 of the gyro sensor unit 70 received by the wireless communication unit 64 is supplied to the electronic pen posture detection unit 65, and the pen posture of the electronic pen 10B is detected. The information of the pen posture of the electronic pen 10B detected by the electronic pen posture detection unit 65 is supplied to the relative posture calculation unit 66.

[0137] The tablet device 60 of this embodiment includes a gyro sensor 67 for detecting the tilt angle and tilt direction of the device. The sensor output of the gyro sensor 67 is supplied to the tablet posture detection unit 68. The tablet posture detection unit 68 detects the tilt angle and tilt direction of the device based on the sensor output of the gyro sensor 67, and supplies the information of the detected tilt angle and tilt direction of the device (tablet posture information) to the relative posture calculation unit 66.

[0138] The relative posture calculation unit 66 calculates the relative pen posture of the electronic pen 10B with respect to the input surface 61S of the tablet device 60 by correcting the information of the absolute pen posture of the electronic pen 10B in the three-dimensional space from the electronic pen posture detection unit 65 using the information of the tablet posture from the tablet posture detection unit 68. Then, the relative posture calculation unit 66 supplies the calculated information of the relative pen posture of the electronic pen 10B to the association establishment unit 63.

[0139] Similar to the above-described associating unit, associating unit 63 associates the information on the designated position of electronic pen 10B with the information on the relative pen posture of electronic pen 10B, and outputs the association to computer 80 .

[0140] As described above, in the input device of the third embodiment, when the indicated position of the pen tip of the electronic pen 10B is obtained, the information of the pen posture of the electronic pen 10B detected in three-dimensional space by the gyro sensor unit 70 provided on the rear end side of the electronic pen 10B can be output as the information of the pen posture corresponding to the indicated position of the electronic pen 10B.

[0141] [Other embodiments or modifications]

[0142] In the first and second embodiments described above, the spatial position detection unit 30 is configured to include a light emitting tracking device and a tracker that emit infrared laser light, but is not limited to this configuration. For example, it may also be configured to use other non-visible light sensors, visible light sensors, or a combination thereof.

[0143] In the first and second embodiments described above, since the operator of the electronic pen 10 wears the HMD 50, it is impossible to directly observe the digitizer 20. Therefore, a virtual image of the digitizer 20 may be drawn on the virtual space image displayed on the HMD 50, and the operator may recognize the position of the digitizer 20 on the display screen of the HMD 50.

[0144] In addition, in the first and second embodiments described above, in order to correct the errors in the two spatial coordinate systems, the coordinate values ​​of the spatial position detection unit 30 are transformed into the coordinate values ​​of the spatial coordinate system of the digitizer 20, but conversely, the coordinate values ​​of the position detection area DT of the digitizer 20 may be transformed into the coordinate values ​​of the spatial position detection unit 30. It should be noted that the transformation between the two spatial coordinate systems is not essential but arbitrary.

[0145] In the first and second embodiments described above, the electronic pen and the digitizer use electromagnetic induction, but the present invention is not limited thereto and an electronic pen and a digitizer using electrostatic method (including active electrostatic coupling method and passive electrostatic coupling method) may also be used.

[0146] Furthermore, the digitizers of the first to second embodiments and the tablet device of the third embodiment may be portable mobile phone terminals called so-called smartphones.

[0147] It should be noted that, in the first embodiment and the second embodiment, the case where 3D rendering is performed using the spatial position indication system is described, but the rendered image may be a 2D rendered image or a 2.5D rendered image.

[0148] Description of symbols

[0149] 10, 10A, 10B…electronic pen, 20, 20A…digital converter, 21S…input surface, 22…sensor unit, 23…position detection unit, 24…posture detection unit, 30…space position detection unit, 40…space description information generating device, 41…input information processing unit, 4101…space position detection unit, 4102…pen posture detection unit, 50…HMD.

Claims

1. A virtual reality (VR) image generation system, comprising: sensor; a detection circuit, wherein the detection circuit detects a first indicated position indicated by the electronic pen in a first three-dimensional space according to the exchange of signals between the electronic pen and the sensor during operation; one or more processors; as well as one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to: detecting a first posture of the electronic pen in a second three-dimensional space, the first posture comprising at least one of: a first tilt angle, a first tilt direction, and a first rotation angle of the electronic pen, and When the electronic pen is in the first three-dimensional space, a first VR image drawn by the electronic pen is generated based on the first indication position and the first posture.

2. The VR image generation system according to claim 1, wherein: When the instructions are executed by the one or more processors, the one or more processors are caused to: detect a second indicated position indicated by the electronic pen in the second three-dimensional space, and when the electronic pen is in the second three-dimensional space, generate the first VR image drawn by the electronic pen based on the second indicated position and the first posture.

3. The VR image generation system according to claim 1, wherein: When the instructions are executed by the one or more processors, the one or more processors are caused to generate a second VR image of the electronic pen based on the first gesture.

4. The VR image generation system according to claim 1, wherein: When the instructions are executed by the one or more processors, the one or more processors are caused to generate a second VR image of the electronic pen based on the first indicated position indicated by the electronic pen in the first three-dimensional space.

5. The VR image generation system according to claim 1, wherein when the instructions are executed by the one or more processors, the one or more processors: detecting a second indicated position indicated by the electronic pen in the second three-dimensional space, and A second VR image of the electronic pen is generated based on the second indicated position indicated by the electronic pen in the second three-dimensional space.

6. The VR image generation system according to claim 3, wherein: The detection circuit is operable to detect a second posture of the electronic pen in the first three-dimensional space, the second posture comprising at least one of: a second tilt angle, a second tilt direction, and a second rotation angle of the electronic pen, and When the instructions are executed by the one or more processors, the one or more processors are caused to: generate the second VR image of the electronic pen based on the second posture when the electronic pen is in the first three-dimensional space.

7. The VR image generation system according to claim 6, wherein: The first inclination angle is an inclination angle relative to a specific direction in the second three-dimensional space, and The second inclination angle is an inclination angle relative to an input surface of the sensor.

8. The VR image generation system according to claim 3, wherein: When executed by the one or more processors, the instructions cause the one or more processors to: generate a third VR image of the device including the sensor.

9. The VR image generation system according to claim 1, wherein: The instructions, when executed by the one or more processors, cause the one or more processors to: detect a gesture of a device including the sensor.

10. The VR image generation system according to claim 1, wherein: The instructions, when executed by the one or more processors, cause the one or more processors to: detect the first posture of the electronic pen based on an output from a gyro sensor included in the electronic pen.

11. A method for generating a virtual reality (VR) image, comprising: Detecting a first indicated position indicated by the electronic pen in a first three-dimensional space by a detection circuit according to the exchange of signals between the electronic pen and the sensor; detecting a first posture of the electronic pen in a second three-dimensional space, the first posture comprising at least one of: a first tilt angle, a first tilt direction, and a first rotation angle of the electronic pen, and When the electronic pen is in the first three-dimensional space, a first VR image drawn by the electronic pen is generated based on the first indication position and the first posture.

12. The VR image generation method according to claim 11, further comprising: A second indicated position indicated by the electronic pen in the second three-dimensional space is detected, and when the electronic pen is in the second three-dimensional space, the first VR image drawn by the electronic pen is generated based on the second indicated position and the first posture.

13. The VR image generation method according to claim 11, further comprising: A second VR image of the electronic pen is generated based on the first gesture.

14. The VR image generation method according to claim 11, further comprising: A second VR image of the electronic pen is generated based on the first indicated position indicated by the electronic pen in the first three-dimensional space.

15. The VR image generation method according to claim 11, further comprising: detecting a second indicated position indicated by the electronic pen in the second three-dimensional space, and A second VR image of the electronic pen is generated based on the second indicated position indicated by the electronic pen in the second three-dimensional space.

16. The VR image generation method according to claim 13, further comprising: The detection circuit detects a second posture of the electronic pen in the first three-dimensional space, wherein the second posture includes at least one of the following: a second tilt angle, a second tilt direction, and a second rotation angle of the electronic pen, and The method further includes generating the second VR image of the electronic pen based on the second posture when the electronic pen is in the first three-dimensional space.

17. The VR image generation method according to claim 16, wherein: The first inclination angle is an inclination angle relative to a specific direction in the second three-dimensional space, and The second inclination angle is an inclination angle relative to an input surface of the sensor.

18. The VR image generation method according to claim 13, further comprising: A third VR image of the device including the sensor is generated.

19. The VR image generation method according to claim 11, further comprising: A gesture of a device including the sensor is detected.

20. The VR image generation method according to claim 11, further comprising: The first posture of the electronic pen is detected based on an output from a gyro sensor included in the electronic pen.

21. A virtual reality (VR) image generation system, comprising: an electronic device including a digitizer, the digitizer being operable to detect a position indicated by the electronic pen; and one or more processors; and one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to: calculating a relative posture of the electronic pen relative to the electronic device, and A VR image is generated based on the relative posture of the electronic pen with respect to the electronic device.

22. The VR image generation system according to claim 21, wherein: The relative posture of the electronic pen with respect to the electronic device is calculated based on the posture of the electronic pen detected by the electronic device.

23. The VR image generation system according to claim 21, wherein: The relative posture of the electronic pen with respect to the electronic device is calculated based on the posture of the electronic pen detected by the gyro sensor of the electronic pen and the posture of the electronic device detected by the gyro sensor of the electronic device.

24. A method for generating a virtual reality (VR) image, comprising: calculating a relative posture of the electronic pen relative to an electronic device including a digitizer, the digitizer detecting a position indicated by the electronic pen, and A VR image is generated based on the relative posture of the electronic pen with respect to the electronic device.

25. The VR image generation method according to claim 24, wherein: The calculating includes calculating the relative posture of the electronic pen with respect to the electronic device based on the posture of the electronic pen detected by the electronic device.

26. The VR image generation method according to claim 24, wherein: The calculating includes calculating the relative posture of the electronic pen with respect to the electronic device based on the posture of the electronic pen detected by a first gyro sensor of the electronic pen and the posture of the electronic device detected by a second gyro sensor of the electronic device.

Images