Position indicator and position detection system

Abstract

This invention provides a position indicator that enables accurate input of position in a VR (Virtual Reality) space. [Solution] The system comprises a rod-shaped housing, a position indicator provided at one end of the rod-shaped housing in the axial direction, and a tracker located at a position different from the position of the position indicator when the user grips the position indicator side of the rod-shaped housing with a first finger and a second finger different from the first finger, and the tracker having a specific positional relationship with the position of the position indicator. Based on the position of the tracker detected by an external device and the specific positional relationship, the position to be input into the VR space is calculated.

Description

Technical Field

[0001] The present invention relates to a position indicator and a position detection system suitable for use, for example, in three-dimensional drawing (3D drawing).

Background Art

[0002] There is known a drawing system for creating an animation image or the like by continuously indicating a position with an electronic pen with respect to a coordinate input device called a digitizer. In this case, the operator performs a position indication operation for generating a drawn image in a state where the tip of the electronic pen is in contact with the input surface of a tablet device incorporating the digitizer, or in a state where it is not in contact with the input surface of the tablet device but is placed in an upper region where the position can be detected (hover state). The digitizer detects the position indicated by the electronic pen, generates a drawn image as the detection result, and displays it on the display screen. The operator performs drawing while checking the drawn image displayed on the display screen.

[0003] Recently, drawing systems and applications have emerged that enable a drawn image displayed on a two-dimensional display screen to be visually drawn and expressed like a three-dimensional image. In that case, the drawing system or application uses a motion sensor to detect motion operations (gestures) such as scraping, denting, or bulging on the drawn image generated using the digitizer by the movement of the operator's hand or finger, and performs three-dimensional drawing expression processing based on the detected motion operations (gestures).

[0004] Incidentally, user interfaces are available that can perform both positional input and operation input such as gestures, as described above. For example, Patent Document 1 (U.S. Patent No. 9,367,169) discloses a touch controller that uses a device equipped with a touch sensor that detects the operator's finger touch and a motion sensor that detects movement, and is configured to switch the touch sensor from hover event detection mode to gesture event detection mode in response to a signal from the motion sensor. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] U.S. Patent No. 9367169 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, in Patent Document 1, the hover event detection mode and the gesture event detection mode are switched within a limited spatial range from the input surface of the touch sensor, which is the hover region. Therefore, the operator must maintain a hover state on the touch sensor while performing a gesture. As a result, the operator has to perform gestures in a limited spatial region, which makes it difficult to perform tasks for 3D rendering. Furthermore, Patent Document 1 assumes operation with the operator's finger, which makes it difficult to perform more precise instructions. In addition, in Patent Document 1, the hover event detection mode and the gesture event detection mode are switched according to the signal from the motion sensor, which means that the tilt of the device must be changed each time to switch modes.

[0007] The purpose of this invention is to provide a position indicator and a position detection system that can solve the above-mentioned problems. [Means for solving the problem]

[0008] To solve the above problems, A position indicator for inputting a location in a VR (Virtual Reality) space, A rod-shaped casing, A position indicator is provided at one end of the rod-shaped housing in the axial direction, A tracker located at a position different from the position of the position indicator located near the first and second fingers when the user grips the position indicator side of the rod-shaped housing with the first finger and a second finger different from the first finger, and the tracker has a specific positional relationship with the position of the position indicator, It has, Based on the position of the tracker detected by the external device and the specific positional relationship, the position to be input into the VR space is calculated. The present invention provides a position indicator characterized by the following features.

[0009] Furthermore, in order to solve the above problems, A position detection system including a position indicator, a spatial position detection unit, and a computer for inputting a position into a VR (Virtual Reality) space, The position indicator is, A rod-shaped casing, A position indicator is provided at one end of the rod-shaped housing in the axial direction, A tracker located at a position different from the position of the position indicator located near the first and second fingers when the user grips the position indicator side of the rod-shaped housing with the first finger and a second finger different from the first finger, and the tracker has a specific positional relationship with the position of the position indicator, It has, The aforementioned spatial position detection unit is It has a position detection unit that detects the position of the tracker, The aforementioned computer, The circuit has a function that calculates the position to be input into the VR space based on the position of the tracker detected by the spatial position detection unit and the specific positional relationship. Provided is a position detection system characterized by the following.

[0010] The position indicator with the above configuration is a tracker that is located at a position different from the position of the position indicator portion that is in the vicinity of the first finger and the second finger of the user when the user holds the position indicator portion side of the rod-shaped housing with the first finger and the second finger different from the first finger, and has a tracker that has a specific positional relationship with the position of the position indicator portion.

[0011] The position detection system is configured to include a position indicator, a spatial position detection unit, and a computer. The spatial position detection unit detects the position of the tracker of the position indicator, and the computer calculates the position indicated by the position indicator in the VR space based on the position of the tracker detected by the spatial position detection unit and the specific positional relationship between the tracker and the position indicator portion.

[0012] With this configuration, in the above position detection system, the operator can perform an input operation on the 3D drawing image in the VR space by the position indication by the position indicator.

Brief Description of the Drawings

[0013] [Figure 1] It is a diagram for explaining the outline of the first embodiment of the position detection system according to this invention. [Figure 2] It is a block diagram for explaining a configuration example of the first embodiment of the position detection system according to this invention. [Figure 3] It is a diagram for explaining the spatial coordinate system in the first embodiment of the position indication system according to this invention. [Figure 4] It is a diagram for explaining the switching of the coordinate system for position detection of the electronic pen in the first embodiment of the position indication system according to this invention. [Figure 5] It is a perspective view showing the appearance of the first embodiment of the position indicator according to this invention. [Figure 6]This is a diagram for explaining a configuration example of a first embodiment of a position indicator according to the present invention. [Figure 7] This is a diagram for explaining a configuration example of a first embodiment of a position indicator according to the present invention. [Figure 8] This is a diagram for explaining another configuration example of a first embodiment of a position indicator according to the present invention. [Figure 9] This is a diagram for explaining a configuration example of a second embodiment of a position indicator according to the present invention. [Figure 10] This is a block diagram for explaining a configuration example of a second embodiment of a position detection system according to the present invention. [Figure 11] This is a diagram for explaining a configuration example of another embodiment of a position indicator according to the present invention. [Embodiments for Carrying Out the Invention]

[0014] Hereinafter, several embodiments of a position detection system according to the present invention will be described with reference to the drawings.

[0015] [First Embodiment] The position detection system of the first embodiment according to the present invention has a display unit configured as a head-mounted display, and is an example of a configuration of a spatial position detection system with a 3D drawing space as a virtual reality (including VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc. Hereinafter, abbreviated as VR) space.

[0016] FIG. 1 is a diagram showing an overview of the overall configuration of the spatial position detection system of this first embodiment with a VR space as a 3D drawing space. FIG. 2 is a block diagram showing a detailed configuration example of the functions of each part of the spatial position detection system of this first embodiment.

[0017] As shown in Figure 1, the spatial position detection system of this first embodiment comprises a three-dimensional position indicator, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generation device 40, and a head-mounted display (hereinafter referred to as HMD) 50. As shown in Figure 2, in this example, the spatial drawing information generation device 40 has the functions of an input information processing unit 41 and a display image generation unit 42, and is composed of, for example, a personal computer. The input information processing unit 41 has the functions of a spatial position detection device. As shown in Figure 2, in this specification, for the sake of explanation, each processing function executed by a program of a personal computer is shown as a "circuit" block.

[0018] In this first embodiment of the spatial position detection system, a spatial drawing information generation device 40 comprising an input information processing unit 41 and a display image generation unit 42 is used. However, the spatial position detection system of this invention may be configured to consist of a 3D position indicator, a digitizer 20, a spatial position detection unit 30, and an input information processing unit 41, with the display image generation unit 42 being provided separately.

[0019] The 3D position indicator 10 has the function of setting the position of the tip of one end of the rod-shaped part in the axial direction as the indicated position on the input surface of the digitizer 20, and also setting it as the indicated position in 3D space detected by the spatial position detection function of the input information processing unit 41 using the spatial position detection unit 30.

[0020] As shown in Figure 1, in this example, the rod-shaped portion of the 3D position indicator 10 is composed of a removable electronic pen 11. In this example, the tip 11a of the core body (hereinafter referred to as the pen tip 11a) protruding from an opening at one end of the cylindrical housing of the electronic pen 11 in the axial direction becomes the tip of the rod-shaped portion at one end in the axial direction. In this first embodiment, an electromagnetic induction type electronic pen is used as an example of the electronic pen 11, but it is not limited to the electromagnetic induction type, and an electrostatic coupling type electronic pen may also be used.

[0021] The electronic pen 11 may, of course, be provided integrally with or fixed to the 3D position indicator 10, rather than being removable. In other words, the rod-shaped portion may be provided integrally with or fixed to the 3D position indicator 10. A detailed explanation of the configuration example of the 3D position indicator 10 will be given later.

[0022] In this example, the digitizer 20 comprises a thin, rectangular housing 21, the surface of which serves as the input surface 21S for position indication by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10. The digitizer 20 also includes a position detection sensor unit 22 and a position detection circuit 23 (see Figure 2).

[0023] The position detection sensor unit 22, although not shown in the diagram, is composed of multiple loop coils arranged in the lateral direction (X-axis direction) and the vertical direction (Y-axis direction) of the housing of the digitizer 20. The electronic pen 11 of the 3D position indicator 10, although not shown in the diagram, is equipped with a resonant circuit (not shown) consisting of a coil and a capacitor on the pen tip 11a side. By electromagnetic induction coupling between the loop coil of the position detection sensor unit 22 of the digitizer 20 and the resonant circuit of the electronic pen 11, the electronic pen 11 and the position detection sensor unit 22 of the digitizer 20 interact and exchange signals. In this example, the digitizer 20 is of the electromagnetic induction type to match the electronic pen 11, but in the case of an electrostatic coupling type, interaction with the electronic pen will occur via electrostatic coupling.

[0024] The position detection circuit 23 of the digitizer 20 supplies a signal to the electronic pen 11 through the loop coil of the position detection sensor unit 22, and also receives a signal returned from the electronic pen 11 through the loop coil. Based on the received signal, it detects the position indicated by the tip 11a of the electronic pen 11 within the detection area of ​​the position detection sensor unit 22. In this embodiment, the digitizer 20 is configured to detect the position indicated by the tip 11a of the electronic pen 11 as the indicated position of the electronic pen 11.

[0025] In this example of the digitizer 20, multiple loop coils of the sensor unit 22 are arranged to cover almost the entire area of ​​the input surface 21S.

[0026] In this embodiment, the position detection region in which the digitizer 20 can detect the indicated position of the electronic pen 11 includes not only the planar region when the tip 11a of the electronic pen 11 is in contact with the input surface 21S of the digitizer 20, but also a spatial region (hover region in the hover state of the electronic pen 11) in which the tip 11a of the electronic pen 11 is not in contact with the input surface 21S of the digitizer 20, and is spaced apart from the input surface 21S in directions perpendicular to the input surface 21S (the Z-axis direction perpendicular to the X-axis direction and the Y-axis direction), but in which the indicated position of the electronic pen 11 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0027] Figure 3 is a diagram illustrating the detection spatial area of ​​the electronic pen 11's indicated position on the 3D position indicator 10 in the digitizer 20, and also shows the detection spatial area (3D spatial area) of the object to be detected, including the 3D position indicator 10, in the spatial position detection unit 30 described later.

[0028] For example, in Figure 1, when the position P0 of the upper left corner of the input surface 21S of the digitizer 20 is set to the coordinates of the origin in the X-axis, Y-axis, and Z-axis directions ((X,Y,Z)=(0,0,0)), the position detection region DT in which the electronic pen 11 can be indicated on the digitizer 20 is the planar region of the input surface 21S and the spatial region of the rectangular parallelepiped above the input surface 21S, as shown by the shaded area in Figure 3.

[0029] In other words, as shown in Figure 3, if the length of the input surface 21S of the digitizer 20 in the X-axis direction is Lx, the length in the Y-axis direction is Ly, and the critical height position in the Z-axis direction where the hover state can be detected is Lz, then, as shown in Figure 3, the region enclosed by the coordinate positions of the eight points P0(0,0,0), P1(Lx,0,0), P2(Lx,Ly,0), P3(0,Ly,0), P4(0,0,Lz), P5(Lx,0,Lz), P6(Lx,Ly,Lz), and P7(0,Ly,Lz) becomes the position detection region DT of the digitizer 20. In this digitizer 20, the information of the indicated position of the electronic pen 11 detected in the position detection region DT is supplied to the input information processing unit 41 of the spatial drawing information generation device 40.

[0030] In this example, the spatial position detection unit 30 is configured to set a three-dimensional spatial region in which the digitizer 20 exists, and to detect the position of the three-dimensional position indicator 10 and the position of the digitizer 20 within that three-dimensional spatial region.

[0031] As shown in Figure 3, the spatial position detection unit 30 is configured to search for objects located in a spatial region that includes the position detection region DT of the digitizer 20, using that region as its search region. In this embodiment, the objects detected by the spatial position detection unit 30 are the 3D position indicator 10 and the digitizer 20. For convenience, in the following description, the 3D spatial region (search region) set by this spatial position detection unit 30 will be referred to as the motion detection spatial region MD.

[0032] In this embodiment, the spatial position detection unit 30 detects the position of an object through optical interaction with the object, and as shown in Figures 1 and 2, it consists of two light-emitting tracking devices 31A and 31B and an optical position notification unit (hereinafter referred to as a tracker) attached to the object.

[0033] Each tracker includes a light-receiving sensor that detects infrared laser light from light-emitting tracking devices 31A and 31B, and a light-emitting unit, such as an LED (Light Emitting Diode), that notifies the light-emitting tracking devices 31A and 31B of the detection of infrared laser light by the light-receiving sensor. Multiple light-receiving sensors are provided in each tracker so that they can receive laser light from any direction.

[0034] In this embodiment, the objects to which the trackers are attached are the 3D position indicator 10 and the digitizer 20. Specifically, in this example, in order to notify the spatial position and orientation (orientation of the input surface 21S) of the digitizer 20 in the motion detection spatial region MD, trackers 24A and 24B are attached to the upper left and lower right corners of the housing of the thin, rectangular digitizer 20. As mentioned above, since the trackers can detect not only the position but also the orientation, only one tracker may be attached to the digitizer 20.

[0035] Furthermore, in order to notify the spatial position and orientation of the 3D position indicator 10 in the motion detection spatial region MD, a tracker 12 is attached to the rear end of the 3D position indicator 10, in this example, on the side opposite to the pen tip 11a in the axial direction of the electronic pen 11. The tracker 12 constitutes the spatial position indicator part of the 3D position indicator 10.

[0036] The trackers are also equipped with, in this example, a 9-axis sensor for detecting movement and direction (orientation). From each tracker, the light-receiving output of the light-receiving sensor and the output of the 9-axis sensor are supplied to the spatial rendering information generation device 40 by wire, wirelessly, or via the attached object. In this case, the information from trackers 12, 24A, and 24B each includes its own identification information.

[0037] The spatial rendering information generation device 40 detects the orientation and movement of the object to which the tracker is attached in three-dimensional space from the light-receiving outputs of multiple light-receiving sensors from the tracker and / or the output of the 9-axis sensor. In order to detect the orientation and movement of the object to which the tracker is attached in three-dimensional space, the tracker only needs to be equipped with either multiple light-receiving sensors or a 9-axis sensor.

[0038] The two light-emitting tracking devices 31A and 31B have the same configuration and each includes a laser-emitting unit for infrared laser light, a search means for searching within the motion detection spatial region MD using the emitted infrared laser light, and an optical position detection means for detecting the emission of light from the light-emitting units of the trackers 12, 24A, and 24B that have received the infrared laser light.

[0039] In this case, for example, the light emission tracking device 31A scans the motion detection space MD horizontally using infrared laser light, and searches for the horizontal scanning position to move sequentially in the vertical direction at a predetermined pitch τX. The light emission tracking device 31B also scans the motion detection space MD vertically using infrared laser light, and searches for the vertical scanning position to move sequentially in the horizontal direction at a predetermined pitch τY.

[0040] Each of the trackers 12, 24A, and 24B monitors the reception of infrared laser light with a light receiving sensor, and lights up an LED light-emitting unit when it detects the reception of infrared laser light.

[0041] The light emission tracking devices 31A and 31B detect the position of the object to which the trackers 12, 24A, and 24B are attached within the motion detection spatial area MD by detecting the light emission of the light-emitting parts of the trackers 12, 24A, and 24B. The light emission tracking devices 31A and 31B are configured to also detect the elapsed time from the time the infrared laser emitted light was emitted to the point in time when the light emission of the light-emitting parts of the trackers 12, 24A, and 24B is detected. In this case, each of the trackers 12, 24A, and 24B emits a different light according to its own identification information.

[0042] As a result, the spatial position detection unit 30 can detect the positions of the trackers 12, 24A, and 24B attached to the object within the motion detection spatial region MD (i.e., the position of the object) with an accuracy corresponding to a predetermined pitch τX and a predetermined pitch τY.

[0043] The two light-emitting tracking devices 31A and 31B are connected to the spatial drawing information generation device 40 by wire or wirelessly, and notify the spatial drawing information generation device 40 of the spatial position information of the detected trackers 12, 24A, and 24B in the motion detection spatial region MD. In this case, the information from the two light-emitting tracking devices 31A and 31B each includes their own identification information.

[0044] The spatial position information in the motion detection spatial region MD of the trackers 12, 24A, and 24B, detected by the two light emission tracking devices 31A and 31B, is supplied to the spatial position detection circuit 410 of the input information processing unit 41 of the spatial drawing information generation device 40, as shown in Figure 2.

[0045] As mentioned above, the light receiving output from the light receiving sensors of trackers 12, 24A, and 24B, as well as the detection output from the 9-axis sensor, are also supplied to the spatial position detection circuit 410 of the input information processing unit 41 of the spatial drawing information generation device 40.

[0046] In this example, the spatial position detection circuit 410 includes a position indicator detection circuit 4101, an indicated position calculation circuit 4102, and a digitizer detection circuit 4103. In this example, as will be described later, it also includes an operation information detection circuit 4104 that detects operation information from an operation unit provided on the 3D position indicator 10. The operation information detected by the operation information detection circuit 4104 is supplied from the 3D position indicator 10 to the spatial position detection circuit 410 of the spatial drawing information generation device 40, along with the light receiving output of the light receiving sensor and the output of the 9-axis sensor.

[0047] The position indicator detection circuit 4101 detects the position of the 3D position indicator 10 within the motion detection spatial region MD based on the position information detected by the light emission tracking devices 31A and 31B through optical interaction between the 3D position indicator 10 and the tracker 12. It also detects the orientation of the 3D position indicator 10, including the direction in which it is facing within the motion detection spatial region MD, from the light received output of the light receiving sensor from the tracker 12 and the detection output of the 9-axis sensor. In this case, the orientation of the 3D position indicator 10 includes the axial direction of the electronic pen 11 as a rod-shaped part and the rotational position (rotation angle) around this axial direction.

[0048] The position information of the 3D position indicator 10, including the direction it is facing, and the orientation information of the 3D position indicator 10, detected by the position indicator detection circuit 4101, are supplied to the indicated position calculation circuit 4102. Here, the indicated position of the 3D position indicator 10 is the position of the pen tip 11a of the electronic pen 11, and is different from the position of the 3D position indicator 10 detected by the position indicator detection circuit 4101. As will be described later, in the 3D position indicator 10, the pen tip 11a of the electronic pen 11 is mounted and configured to have a specific positional and directional relationship with the position and direction of the 3D position indicator 10 detected by the position indicator detection circuit 4101 of the spatial position detection circuit 410.

[0049] The instruction position calculation circuit 4102 stores correction information in the correction memory 4102M that allows it to detect the instruction position made by the pen tip 11a of the electronic pen 11 from the position information and direction information of the 3D position indicator 10 detected by the position indicator detection circuit 4101, based on the above-mentioned specific positional and directional relationships.

[0050] The instruction position calculation circuit 4102 calculates and outputs the instruction position of the pen tip 11a of the electronic pen 11 from the position information and direction information of the 3D position indicator 10 detected by the position indicator detection circuit 4101 and the correction information stored in the correction memory 4102M. In addition, the instruction position calculation circuit 4102 calculates the orientation in the axial direction of the electronic pen 11 (the tilt of the electronic pen 11) based on the orientation information of the 3D position indicator 10 detected by the position indicator detection circuit 4101, and also calculates the rotation and movement of the electronic pen 11, outputting this as orientation information of the electronic pen 11.

[0051] The digitizer detection circuit 4103 of the spatial position detection circuit 410 detects the position of the digitizer 20 within the motion detection spatial region MD based on the position information detected by the light emission tracking devices 31A and 31B through optical interaction with the trackers 24A and 24B. The digitizer detection circuit 4103 also detects the orientation of the digitizer 20, including the direction in which the input surface 21S of the digitizer 20 is facing within the motion detection spatial region MD, from the light received output of the light receiving sensors from the trackers 24A and 24B and the detection output of the 9-axis sensor, and outputs the detected orientation information together with the position information.

[0052] Furthermore, the operation information detection circuit 4104 detects and outputs operation information from the operation section of the 3D position indicator 10.

[0053] The input information processing unit 41 generates information to be supplied to the display image generation unit 42 from the information on the position indicated by the tip 11a of the electronic pen 11 of the 3D position indicator 10 in the position detection region DT, which was detected by the digitizer 20 as described above, the position (indication position) of the tip 11a of the electronic pen 11 of the 3D position indicator 10 in the motion detection spatial region MD, which was detected by the spatial position detection circuit 410 as described above, the orientation information of the electronic pen 11, the orientation information of the 3D position indicator 10, and the operation information. The input information processing unit 41 then supplies the generated information to the display image generation unit 42.

[0054] In this embodiment, the display image generation unit 42 includes, as shown in Figure 2, a drawing image generation circuit 421 for generating a 3D drawing image and a VR image generation circuit 422 for generating a VR image to be displayed on the HMD 50. Hereinafter, the processing related to the generation of the 3D drawing image in the drawing image generation circuit 421 will be referred to as the 3D drawing system processing. Similarly, the processing related to the generation of the VR image in the VR image generation circuit 422 will be referred to as the VR image system processing. In this embodiment, the HMD 50 displays the 3D drawing image drawn in the motion detection space area MD, which includes the position detection area DT of the digitizer 20, as a virtual display image, along with virtual display images of the electronic pen 11 of the 3D position indicator 10 and the digitizer 20.

[0055] The image generation circuit 421 generates a 3D image based on the position indication by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 and the orientation information of the electronic pen 11, and also performs processing such as deformation, rotation, and movement of the 3D image based on gestures performed by the operator of the 3D position indicator 10. As shown in Figure 2, the display image generation unit 42 is provided with a gesture detection processing circuit 423 for detecting gestures performed by the operator of the 3D position indicator 10, and the gesture detection results from this gesture detection processing circuit 423 are supplied to the image generation circuit 421. The gesture detection processing circuit 423 detects gestures performed by the operator of the 3D position indicator 10 from changes in the position and orientation of the 3D position indicator 10.

[0056] The VR image generation circuit 422 of the display image generation unit 42 generates a VR image to be displayed on the HMD 50. In this embodiment, the VR image includes a VR image of the electronic pen 11, a VR image of the digitizer 20, and a VR image of the 3D drawing generated by the drawing image generation circuit 421. Alternatively, a VR image of the 3D position indicator 10, including the electronic pen 11, may be generated.

[0057] The input information processing unit 41 generates information for drawing processing and information for VR image processing as described above from the information from the digitizer 20 and the information from the spatial position detection unit 30, and supplies them to the display image generation unit 42.

[0058] In this case, when the position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is within the position detection area DT, the input information processing unit 41 supplies the information of the position indicated by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, detected by the digitizer 20 which can detect the position of the pen tip 11a of the electronic pen 11 with higher accuracy than the spatial position detection unit 30, to the VR image generation circuit 422 of the display image generation unit 42.

[0059] Furthermore, when the position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is in a space other than the position detection area DT, the input information processing unit 41 supplies the position information of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, which is detected using the spatial position detection unit 30, to the VR image generation circuit 422 of the display image generation unit 42. In other words, the input information processing unit 41 supplies the information of the indicated position of the 3D position indicator 10 to the display image generation unit 42, switching between the position detection area DT and the motion detection spatial area MD depending on whether the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is in the position detection area DT or the motion detection spatial area MD.

[0060] In this embodiment, the input information processing unit 41, as described above, uses the orientation information of the 3D position indicator 10 detected using the spatial position detection unit 30 to supply orientation information including the tilt and rotation of the electronic pen 11, calculated by the indicator position calculation circuit 4102, to the drawing image generation circuit 421 and the VR image generation circuit 422 of the display image generation unit 42, regardless of the spatial position where the pen tip 11a of the electronic pen 11 is located.

[0061] [Regarding coordinate transformations between spatial coordinate systems for detecting the indicated position of the 3D position indicator 10] Incidentally, in this embodiment, the spatial coordinate system of the position detection area DT of the digitizer 20 and the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30 can be set independently, as shown in Figure 3.

[0062] As described above, in this embodiment, the spatial coordinate system information of the position detection area DT of the digitizer 20 and the spatial coordinate system information of the motion detection spatial area MD of the spatial position detection unit 30 are mutually utilized in both the 3D rendering system processing and the VR image system processing. In this case, there would be no problem if the two spatial coordinate systems could be completely common, but as described above, they can be set independently, so the points described below need to be considered.

[0063] In Figure 3, the three axes of the spatial coordinate system of the motion detection spatial region MD are shown as the Xs axis, Ys axis, and Zs axis using the suffix s, distinguishing them from the X, Y, and Z axes of the spatial coordinate system of the detection region DT of the digitizer 20. Note that in Figure 3, for convenience, the directions of the Xs, Ys, and Zs axes are shown as being the same as the directions of the X, Y, and Z axes. However, due to factors such as the inclination of the input surface 21S when the digitizer 20 is installed within the motion detection spatial region MD, the directions of the Xs, Ys, and Zs axes may differ from the directions of the X, Y, and Z axes.

[0064] However, in this embodiment, the digitizer 20 is positioned in the motion detection spatial region MD of the spatial position detection unit 30, and the spatial coordinate system of the motion detection spatial region MD of the spatial position detection unit 30 is configured to include at least a portion of the spatial coordinate system of the position detection region DT of the digitizer 20 as a common region. In this embodiment, the information of the position indicated by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is configured to undergo coordinate transformation using spatial position correction information, which will be described later, by using a common region between the spatial coordinate system of the position detection region DT and the spatial coordinate system of the motion detection spatial region MD.

[0065] In this case, if the directions of the Xs, Ys, and Zs axes are the same as the directions of the X, Y, and Z axes, then the spatial coordinate system of the position detection area DT and the spatial coordinate system of the motion detection area MD can be treated as a single spatial coordinate system by considering the difference in the origin positions of both spatial coordinate systems. That is, for example, if the offset values ​​in the X, Y, and Z directions between the origin position of the spatial coordinate system of the position detection area DT and the origin position of the spatial coordinate system of the motion detection area MD are OFSx, OFSy, and OFSz, respectively, then by calculating Xs-OFSx (=X), Ys-OFSy (=Y), and Zs-OFSz (=Z), the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection area MD can be converted to the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection area DT.

[0066] However, in the spatial position indication system of this first embodiment, as described above, the spatial coordinate system of the position detection area DT of the independently provided digitizer 20 and the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30 may differ in the directions of the X, Y, and Z axes and the directions of the Xs, Ys, and Zs axes. Furthermore, even when the directions of the X, Y, and Z axes and the directions of the Xs, Ys, and Zs axes are the same, it is difficult to accurately define the offset value of the origin position, and there is a risk that it will differ from one spatial position indication system to another.

[0067] This would result in different coordinate positions in the two spatial coordinate systems. When switching between the spatial coordinate system of the position detection area DT of the digitizer 20 and the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30, there is a risk that the coordinates of the pen tip 11a of the electronic pen 11 on the HMD 50's display screen may jump. This would be inconvenient for the user attempting to input a drawing image, as they would have to re-indicate the position.

[0068] Therefore, in this embodiment, at least a portion of the position detection area DT of the digitizer 20 is a common spatial area with the motion detection spatial area MD of the spatial position detection unit 30. This is used to generate correction information for the discrepancy between the spatial coordinate system of the position detection area DT and the spatial coordinate system of the motion detection spatial area MD, and to perform coordinate transformation. In this example, the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection spatial area MD detected using the spatial position detection unit 30 are converted to the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection area DT of the digitizer 20, thereby obtaining coordinate values ​​that correct for the discrepancy between the two. The correction information for this transformation will be described next.

[0069]

number

[0070] (Equation 1) is the determinant for linearly transforming the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection spatial region MD detected by the spatial position detection unit 30 to the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection region DT of the digitizer 20. This determinant is 3 rows and 3 columns, and its components are a ij Let's represent it as (i,j=1,2,3).

[0071] In this first embodiment, at least a portion of the position detection area DT of the digitizer 20 is a common spatial area with the motion detection spatial area MD of the spatial position detection unit 30, and correction information for conversion between the spatial coordinate system of the position detection area DT and the spatial coordinate system of the motion detection spatial area MD is generated.

[0072] Specifically, as shown in Figure 3, at least three points Pa, Pb, and Pc are positioned within the common spatial area of ​​the position detection area DT of the digitizer 20 and the motion detection spatial area MD of the spatial position detection unit 30. The coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection area DT of the digitizer 20 and the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30 are then obtained from each device. Ideally, the coordinate values ​​obtained from these devices will be the same, but without calibration, these coordinate values ​​will not usually match. Generally, since the accuracy of position detection by the digitizer 20 is higher than the accuracy of position detection using the spatial position detection unit 30, it is preferable to match the coordinate values ​​of the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30 with the coordinate values ​​of the spatial coordinate system of the position detection area DT of the digitizer 20.

[0073] By substituting the coordinate values ​​(X,Y,Z) of the spatial coordinate system of the corresponding position detection area DT and the coordinate values ​​(Xs,Ys,Zs) of the spatial coordinate system of the motion detection area MD into Equation 1 for each point specified within the common area, three equations are obtained. By specifying the positions of at least three points within the common area, a 11 ~a 33 Since we can obtain at least nine equations about a 11 ~a 33 Each of these values ​​can be determined. Furthermore, the transformation between the spatial coordinate system of the position detection area DT and the spatial coordinate system of the motion detection area MD is not limited to the above method, and the coordinate values ​​of at least three points in the common area may be obtained by machine learning or by user calibration.

[0074] As described above, in the spatial position indication system of this embodiment, the discrepancy between the spatial coordinate system of the position detection area DT of the digitizer 20 and the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30 is corrected so that they can be treated as a single common coordinate system. This correction process is performed in the input information processing unit 41.

[0075] Therefore, even if the information regarding the indicated position of the electronic pen 10 in the HMD50's display image is switched between the spatial coordinate system of the position detection area DT of the digitizer 20 and the spatial coordinate system of the motion detection spatial area MD of the spatial position detection unit 30, both can be treated as a single common coordinate system. Thus, for example, a situation where the indicated position by the pen tip 11a of the electronic pen 11 changes will not occur.

[0076] [Switching the spatial coordinate system for detecting the indicated position of the 3D position indicator 10] In the spatial position indication system of the first embodiment, the spatial position detection unit 30 is configured to have at least a portion of the position detection area DT of the digitizer 20 as a common area within the motion detection spatial area MD of the spatial position detection unit 30.

[0077] In this embodiment of the spatial position indication system, the system switches between using the coordinates of the indicated position of the pen tip 11a of the electronic pen 11 of the 3D position indication system 10 detected by the digitizer 20, or using the coordinates of the indicated position of the pen tip 11a of the electronic pen 11 of the 3D position indication system 10 detected by the spatial position detection unit 30, depending on the distance from the input surface 21S of the sensor unit 22 of the digitizer 20 to the position of the pen tip 11a of the electronic pen 11 of the 3D position indication system 10 within the spatial motion detection area MD.

[0078] In this embodiment, the distance θth from the input surface 21S that serves as the switching point (distance in the Z-axis direction) is set to be less than or equal to the critical height distance Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen 11 of the 3D position indicator 10. In this example, the distance θth from the input surface 21S that serves as the switching point is set to be equal to the critical height distance Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen 11 of the 3D position indicator 10, i.e., the length Lz in the Z-axis direction of the position detection region DT, as shown in Figure 4.

[0079] In other words, in the spatial position indication system of this embodiment, as shown in Figure 4, when the distance from the input surface 21S of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is θth or less, the digitizer 20 uses the information of the position indicated by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, which is detected in its position detection region DT.

[0080] Furthermore, in the spatial position indication system of this embodiment, as shown in Figure 4, when the distance from the input surface 21S of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is greater than θth, the information of the indicated position by the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, detected using the spatial position detection unit 30, is used.

[0081] In this embodiment, the distance (Z-axis distance) between the tip 11a of the electronic pen 11 of the 3D position indicator 10 and the input surface 21S of the sensor unit 22 of the digitizer 20 is detected by the input information processing unit 41 based on the signal level (signal strength) of the signal received from the electronic pen 11 at the sensor unit 22 of the digitizer 20, as this value corresponds to the distance.

[0082] [Switching process in the input information processing unit 41] Next, an example of the configuration of the input information processing unit 41 shown in Figure 2, which is configured to achieve the above, will be described. Specifically, the position detection circuit 23, which constitutes the digitizer 20, supplies the detection output of the indicated position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 to the selection circuit 411 as one of its input signals, and also supplies it to the selection circuit 412 as one of its input signals. The information supplied from the position detection circuit 23 to the selection circuits 411 and 412 includes not only the detection output of the indicated position of the pen tip 11a of the electronic pen 11, but also pressure information applied to the pen tip 11a of the electronic pen 11.

[0083] Furthermore, the indicated position calculation circuit 4102 of the spatial position detection circuit 410 supplies the detected output of the spatial position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 (the indicated position of the electronic pen 11) to the coordinate transformation circuit 413. This coordinate transformation circuit 413 uses the component a of the 3x3 matrix in (Equation 1) described above. 11 ~a 33 The system stores this information and, by performing the calculation (Equation 1) described above, converts the spatial coordinate system information of the motion detection spatial region MD of the spatial position detection unit 30 into the spatial coordinate system information of the position detection region DT of the digitizer 20. The coordinate transformation circuit 413 then supplies the converted coordinate output to the selection circuits 411 and 412 as the other input signal.

[0084] In this embodiment, the position detection circuit 23 of the digitizer 20 supplies information about the signal level of the received signal from the electronic pen 10 to the selection control signal generation circuit 414. The selection control signal generation circuit 414 detects the distance between the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 and the input surface 21S of the digitizer 20 from the signal level of the received signal from the electronic pen 10, and generates a selection control signal SE based on the detected distance from the input surface 21S. The generated selection control signal SE is supplied to selection circuits 411 and 412.

[0085] Then, the selection circuits 411 and 412 are controlled by the selection control signal SE to select the position detection output from the position detection circuit 23 of the digitizer 20 when the distance between the pen tip 11a of the electronic pen 11 and the input surface 21S of the digitizer 20 is less than or equal to the critical height Lz in the Z-axis direction at which the digitizer 20 can detect the hover state of the electronic pen 11, and to select the output from the coordinate transformation circuit 413 when the distance is greater than the critical height Lz.

[0086] The information regarding the indicated position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, received from the selection circuit 411, is supplied to the image generation circuit 421 and the gesture detection processing circuit 423 for 3D drawing system processing. In addition, the information regarding the indicated position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10, received from the selection circuit 412, is supplied to the VR image generation circuit 422 for VR image system processing.

[0087] Then, the orientation information of the electronic pen 11 from the indicated position calculation circuit 4102 of the spatial position detection circuit 410 and the operation information of the operation part of the 3D position indicator 10 from the operation information detection circuit 4104 are supplied to the drawing image generation circuit 421, the gesture detection processing circuit 423, and the VR image generation circuit 422, respectively. In addition, the position information and orientation information of the digitizer 20 from the digitizer detection circuit 4103 are supplied to the drawing image generation circuit 421 and the VR image generation circuit 422, respectively. As will be described later, in the drawing image generation circuit 421, the orientation information of the digitizer 20 is used to convert the orientation information of the electronic pen 11 into orientation information of the digitizer 20 relative to the input surface.

[0088] [Processing operation of the display image generation unit 42] The image generation circuit 421 has a pen drawing function that draws detailed line drawings and the like based on the detection output of the indicated position of the pen tip 11a of the electronic pen 11 from the selection circuit 411 and the orientation information of the electronic pen 11 detected by the spatial position detection unit 30. The image generation circuit 421 also has a gesture processing function that performs drawing processing based on the movement (gesture) detected by the gesture detection processing circuit 423, based on the spatial position of the electronic pen 11 and the orientation information of the electronic pen detected by the spatial position detection unit 30.

[0089] In this example, the image generation circuit 421 and the gesture detection processing circuit 423 are supplied with a selection control signal SE from the selection control signal generation circuit 414. As a result, the image generation circuit 421 operates to execute the pen drawing function when the position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is within the spatial region of the position detection area DT of the digitizer 20. The gesture detection processing circuit 423 operates to execute gesture processing when the position of the pen tip 11a of the electronic pen 11 is outside the spatial region of the position detection area DT of the digitizer 20.

[0090] In this case, the drawing image generation circuit 421 reflects the tilt included in the orientation information of the electronic pen 11 in the thickness of the drawing line, etc. However, when the position of the pen tip 11a of the electronic pen 11 is within the position detection area DT, this tilt must be the tilt with respect to the input surface 21S of the digitizer 20.

[0091] However, since the orientation information of the electronic pen 11 from the instruction position calculation circuit 4102 is detected using the spatial position detection unit 30, if the plane direction of the input surface 21S of the digitizer 20 is offset from the plane direction of the XY plane in the motion detection spatial region MD of the spatial position detection unit 30, it does not indicate the inclination of the digitizer 20 with respect to the input surface 21S.

[0092] Therefore, the image generation circuit 421 of this embodiment is equipped with a processing function that uses the attitude information of the digitizer 20 from the digitizer position detection circuit 4103 to correct the tilt of the electronic pen 11 detected in the coordinate system of the motion detection spatial region MD of the spatial position detection circuit 410 to the tilt relative to the input surface of the digitizer 20.

[0093] Furthermore, the image generation circuit 421 and the gesture detection processing circuit 423 also perform processing based on the operation information of the operation section of the 3D position indicator 10 from the operation information detection circuit 4104.

[0094] The VR image generation circuit 422 of the display image generation unit 42 generates a VR image of the electronic pen 11 using the position information of the pen tip 11a of the electronic pen 11 and the orientation information of the electronic pen 11 from the 3D position indicator 10 of the selection circuit 412. This VR image of the electronic pen 11 is generated so that the position of the pen tip 11a of the electronic pen 11 is displayed at the detected indicator position.

[0095] In this case, the information on the indicated position of the pen tip 11a of the electronic pen 11 from the spatial position detection circuit 410 is transformed by the coordinate transformation circuit 413 so that it is the same indicated position information as that of the digitizer 20. Therefore, the position information of the pen tip 11a of the electronic pen 11 from the 3D position indicator 10 from the selection circuit 412 does not change when the selection circuit 412 switches due to the selection control signal SE, so there is no display misalignment such as the display position of the VR image of the electronic pen 11 jumping.

[0096] In this embodiment, the VR image generation circuit 422 generates a VR image of the digitizer 20 based on the position and orientation information of the digitizer 20 from the digitizer detection circuit 4103 and displays it in the motion detection space area MD. The VR image generation circuit 422 then receives operation information from the operation information detection circuit 4104 of the operation section of the 3D position indicator 10 and, for example, displays a menu for 3D drawing in VR, or displays a VR image of an operation button. The drawing image generation circuit 421 recognizes the position of these VR images of menus and operation buttons in the motion detection space area MD, and can detect whether the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 has indicated (similar to clicking) the position of these VR images and perform the corresponding processing.

[0097] Then, as described above, the 3D drawing image information generated by the drawing image generation circuit 421 is supplied to the VR image generation circuit 422 to be converted into a VR image, and together with the aforementioned VR image, is supplied to the HMD 50 via the display drive circuit 424 for display.

[0098] [Example configuration of 3D position indicator 10] Next, an example of the configuration of the 3D position indicator 10 of this embodiment will be described with reference to Figures 5 to 7. Figure 5 is a perspective view showing the external appearance of the 3D position indicator 10 of this embodiment. Figure 6 is a diagram illustrating the components of the 3D position indicator 10 of this embodiment and their assembly.

[0099] The three-dimensional position indicator 10 in this embodiment consists of an electronic pen 11 shown in Figure 6(A) and a holder 13 that constitutes a gripping part held by the user. The tracker 12 that constitutes the spatial position indicator part is mounted on the holder 13.

[0100] In this example, as shown in Figure 6(B), the tip of the core body of the electronic pen 11 protrudes as the pen tip 11a from one opening in the axial direction of the cylindrical rod-shaped housing 110. Although not shown in the figure, a coil wound around a ferrite core is arranged near the pen tip 11a in the hollow part of the cylindrical housing 110 of the electronic pen 11, as is well known, and a capacitor connected in parallel with the coil to form a resonant circuit is arranged on a printed circuit board disposed in the hollow part. In this example, the electronic pen 11 is also provided with a side switch 11s exposed from the housing 110 so that it can be operated by the user to control the on / off state of the capacitor for changing the frequency of the resonant circuit.

[0101] The holder 13 includes a grip portion 131 that is covered by the palm of the user when they grasp the 3D position indicator 10, an electronic pen mounting portion 132, and a tracker mounting portion 133.

[0102] The electronic pen mounting section 132 has a cylindrical shape and includes a cylindrical recessed hole 132a into which the electronic pen 11 is inserted in its axial direction from the rear end 11b opposite to the pen tip 11a side and locked into the holder 13. The cylindrical recessed hole 132a has an axial length such that when the electronic pen 11 is inserted and mounted, the electronic pen 11 is held in a state in which its pen tip 11a side is exposed and the side switch 11s is exposed to the outside and can be operated.

[0103] In this example, as shown in Figure 6(A), the electronic pen 11 is provided with a ring-shaped groove 11c on the axial rear end 11b side, and the cylindrical recessed portion 132a has a ring-shaped projection 132b formed therein that fits with the ring-shaped groove 11c of the electronic pen 11 to lock the electronic pen 11 inside the cylindrical recessed portion 132a.

[0104] In this example, a stepped portion 11d is formed on the rear end 11b side of the electronic pen 11. When this stepped portion 11d abuts against the end of the cylindrical recessed portion 132a, the ring-shaped groove 11c of the electronic pen 11 and the ring-shaped projection 132b inside the cylindrical recessed portion 132a engage, thereby locking the electronic pen 11 into the cylindrical recessed portion 132a of the electronic pen mounting portion 132. The configuration for locking the electronic pen 11 into the cylindrical recessed portion 132a of the electronic pen mounting portion 132 is not limited to the configuration with a ring-shaped groove 11c and a ring-shaped projection 132b as in this example. Any configuration is acceptable as long as the electronic pen 11 can be locked into the cylindrical recessed portion 132a of the electronic pen mounting portion 132 when the stepped portion 11d on the rear end 11b side of the electronic pen 11 abuts against the end of the cylindrical recessed portion 132a.

[0105] The grip portion 131 extends from the circumferential surface of the cylindrical electronic pen mounting portion 132 in a direction perpendicular to the axial direction and is shaped to be easily grasped by the user. The grip portion 131 is provided with operation buttons 131a and 131b in a location that can be operated with the index finger or the like when the user is holding it, and also has an operation portion 131c that can be rotated and pressed with the pad of the user's thumb. In the example of the 3D position indicator 10 shown in Figures 5 and 6, the 3D position indicator 10 is designed for left-handed users, and the circular operation portion 131c is attached so that it can be operated with the left thumb when the grip portion 131 is held in the left hand.

[0106] The tracker mounting portion 133 is attached to the circumferential surface of the cylindrical electronic pen mounting portion 132, extending in a direction perpendicular to the axial direction from an angular position 180 degrees opposite to the angular position where the grip portion 131 is formed. The aforementioned tracker 12 is provided on this tracker mounting portion 133. Therefore, when the user holds the grip portion 131 in their palm, the tracker 12 attached to the tracker mounting portion 133 is on the upper side of the back of the hand and is not covered by the hand.

[0107] In this example, the tracker 12 has a shape such that it has three protrusions on the upper surface (plane) 12a of the disc-shaped member, as shown in Figure 5. As shown in Figures 5 and 6(B) and (C), the tracker 12 is equipped with multiple light-receiving sensors 121 on the three protrusions and the disc-shaped member so that it can receive infrared laser light regardless of the direction from which the infrared laser light is coming. In addition, one of the three protrusions of the tracker 12 is equipped with an LED 122, which is a light-emitting element that notifies the system of the reception of infrared laser light when any of the light-receiving sensors 121 receive the infrared laser light. The 9-axis sensor is located inside the tracker 12.

[0108] Although not shown in the diagram, when the tracker 12 and the spatial drawing information generation device 40 are connected by a wire, the tracker 12 is provided with a connector to which a cable for connecting to the spatial drawing information generation device 40 is connected. When connecting wirelessly, communication means are provided inside the tracker 12.

[0109] In this example, as shown in Figures 6(B) and (C), the tracker 12 is mounted such that the plane direction of the upper surface 12a of the disc-shaped member (the direction in which the surface containing the upper surface 12a is facing) is perpendicular to the extension direction of the tracker mounting portion 133, and parallel to the axial direction of the cylindrical electronic pen mounting portion 132. The LED 122 is mounted on the extension direction of the tracker mounting portion 133.

[0110] As described above, by using the spatial position detection unit 30, the position indicator detection circuit 4101 of the spatial position detection circuit 410 detects the position Pt of the LED 122 provided on the tracker 12 as the position of the tracker 12 in the spatial coordinate system of the motion detection spatial region MD. In addition, as previously described, the position indicator detection circuit 4101 detects the orientation (direction) of the 3D position indicator 10, in this example, as the plane direction of the upper surface 12a of the tracker 12 (the direction in which the plane containing the upper surface is facing).

[0111] As is clear from Figure 5, the position Pt of the tracker 12 detected by the position indicator detection circuit 4101 of the spatial position detection circuit 410 is different from the position Pp of the pen tip 11a of the electronic pen 11 mounted on the holder 13. However, the position Pt of the tracker 12 (position of LED 122) detected by the position indicator detection circuit 4101 of the spatial position detection circuit 410 and the position Pp of the pen tip 11a of the electronic pen 11 mounted on the holder 13 have a specific positional relationship in the 3D position indicator 10.

[0112] Here, the three-dimensional coordinate system defining the mechanical configuration of the three-dimensional position indicator 10 is configured as shown in Figure 5, with the position Pt of the tracker 12 as the origin, the coordinate axis Zt being the direction parallel to the upper surface 12a of the tracker 12 and parallel to the axial direction of the cylindrical electronic pen mounting part 132, the coordinate axis Yt being the direction parallel to the upper surface 12a of the tracker 12 and perpendicular to the axial direction of the cylindrical electronic pen mounting part 132, and the coordinate axis Xt being the direction perpendicular to the upper surface 12a of the tracker 12 (the direction perpendicular to coordinate axes Zt and Yt). With this configuration, the position Pt of the tracker 12 detected by the position indicator detection circuit 4101 of the spatial position detection circuit 410 and the position Pp of the pen tip 11a of the electronic pen 11 mounted on the holder 13 have the positional relationship shown in Figure 7.

[0113] In other words, in the space of coordinate axes Xt, Yt, Zt, the position Pp of the pen tip 11a of the electronic pen 11 mounted on the holder 13 is shifted by -Δx in the direction of the coordinate axis Xt and by +Δz in the direction of the coordinate axis Zt relative to the position Pt of the tracker 12, and the shift Δy in the direction of the coordinate axis Yt is 0 (Δy=0). Therefore, by using these values ​​as a correction value COR (=(-Δx, Δy(=0), +Δz)), the position Pp of the pen tip 11a of the electronic pen 11 can be calculated from the position Pt of the tracker 12.

[0114] In this embodiment, since the three-dimensional position indicator 10 is mechanically configured as described above, the displacement amount Δx in the Xt axis direction and the displacement amount Δz in the Zt axis direction can be defined as mechanical dimensional values ​​of the three-dimensional position indicator 10. That is, Δx is determined as the distance from the position of the LED 122 to the central axis position of the electronic pen mounting part 132, and Δz is determined as the distance from the intersection point of the position of the LED 122 and the central axis position of the electronic pen mounting part 132 to the tip of the pen tip 11a of the electronic pen 11.

[0115] In this case, the position Pt of the tracker 12 detected by the spatial position detection circuit 410 using the spatial position detection unit 30 is in the spatial coordinate system (Xs, Ys, Zs) of the spatial position detection circuit 410, and it must be considered that the direction of the coordinate axes does not necessarily coincide with the three-dimensional coordinate system (Xt, Yt, Zt) that defines the mechanical configuration of the three-dimensional position indicator 10 described above. This is because the three-dimensional position indicator 10 is used by the user, orienting it in any direction.

[0116] However, in this embodiment, the orientation of the 3D position indicator 10 detected by the position indicator detection circuit 4101, that is, the direction in which the upper surface 12a of the tracker 12 is facing (the direction perpendicular to the upper surface 12a), is detected by the spatial position detection circuit 410 using the spatial position detection unit 30. Once the direction in which the upper surface 12a of the tracker 12 is facing can be detected, the directions of each coordinate axis of the 3D coordinate system (Xt, Yt, Zt) that defines the mechanical configuration of the 3D position indicator 10 can be determined in the spatial coordinate system (Xs, Ys, Zs) of the spatial position detection circuit 410.

[0117] In this way, if the direction of each coordinate axis of the three-dimensional coordinate system (Xt, Yt, Zt) that defines the mechanical configuration of the three-dimensional position indicator 10 can be determined in the spatial coordinate system (Xs, Ys, Zs) of the spatial position detection circuit 410, the indicated position of the pen tip 11a of the electronic pen 11 in the spatial coordinate system (Xs, Ys, Zs) of the spatial position detection circuit 410 can be calculated from the position Pt of the tracker 12 detected by the position indicator detection circuit 4101 of the spatial position detection circuit 410 using the correction value COR.

[0118] Based on the above, the correction value COR(=(-Δx,Δy(=0),+Δz)) is stored in the correction memory 4102M of the indicated position calculation circuit 4102 of the spatial position detection circuit 410. Then, the indicated position calculation circuit 4102 uses the orientation of the 3D position indicator 10 detected by the position indicator detection circuit 4101 and the correction value COR stored in the correction memory 4102M to calculate the indicated position of the pen tip 11a of the electronic pen 11 from the position Pt of the tracker 12 of the 3D position indicator 10, as described above.

[0119] In the 3D position indicator 10 shown in Figures 5 and 6, the positional relationship between the tracker 12, the electronic pen mounting section 132, and the electronic pen 11 is configured as described above, in order to allow the correction value COR to be easily set from the mechanical dimensional values ​​of the 3D position indicator 10. However, the tracker 12 may be attached to the holder 13 in any manner.

[0120] In that case, the correction value COR can be calculated as follows and stored in the correction memory 4102M.

[0121] In other words, the 3D position indicator 10 is positioned within the motion detection spatial region MD by adjusting the orientation of the tracker 12 so that the directions of the coordinate axes of the 3D coordinate system (Xt, Yt, Zt) that defines the mechanical configuration of the 3D position indicator 10 coincide with the directions of the coordinate axes of the spatial coordinate system (Xs, Ys, Zs) detected by the spatial position detection circuit 410 using the spatial position detection unit 30.

[0122] Then, the position Pp of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10 is detected using the digitizer 20, and the detected coordinate value is converted to a coordinate value in the spatial coordinate system (Xs, Ys, Zs) detected by the spatial position detection circuit 410 using the spatial position detection unit 30. This conversion is the inverse conversion of the conversion shown in Equation 1. In addition, the position Pt of the tracker 12 of the 3D position indicator 10 is detected as a coordinate value in the spatial coordinate system (Xs, Ys, Zs) detected by the spatial position detection circuit 410.

[0123] Then, the amount of difference (Δx, Δy, Δz) between the coordinate value of the detected position Pt of the tracker 12 and the coordinate value of the position Pp of the pen tip 11a of the electronic pen 11 is calculated, and this calculated amount of difference (Δx, Δy, Δz) is stored as a correction value COR in the correction memory 4102M.

[0124] The process of storing the correction value COR in this correction memory 4102M may be performed when the spatial position detection system is constructed, or it may be performed by the user afterward as appropriate.

[0125] As described above, in the spatial position detection system of the above embodiment, when drawing is performed on the digitizer 20 using the 3D position indicator 10 and gesture operations are performed in the space defined by the spatial position detection unit 30, the indicated position can always be the position of the pen tip 11a of the electronic pen 11 of the 3D position indicator 10. This has the effect of allowing the user to perform operations that maintain consistency from drawing to gestures.

[0126] Furthermore, in the first embodiment described above, the spatial coordinates of the detection area of ​​the digitizer 20 and the spatial coordinates of the detection area of ​​the spatial position detection unit 30 can be treated as common coordinates. Therefore, in 3D rendering processing, no coordinate shift occurs when switching between the output from the digitizer 20 and the output from the spatial position detection unit 30.

[0127] Therefore, the operator can seamlessly perform operations ranging from detailed drawing to gesture-based control simply by spatially moving the electronic pen 10 on the digitizer 20, without being aware of switching between the digitizer 20 and the spatial position detection unit 30.

[0128] Furthermore, in the first embodiment described above, in VR image processing, the position information of the electronic pen 11 (position information of the pen tip 11a) for generating the VR image of the electronic pen 11 of the 3D position indicator 10 can be used as the position information of the electronic pen 11 (position information of the pen tip 11a). In the spatial region of the position detection area DT of the digitizer 20, the position detection output of the digitizer 20 is more accurate and has a faster response speed than the detection output of the spatial position detection unit 30. As a result, a VR image that accurately responds to the operation of the electronic pen 11 by the actual user can be obtained.

[0129] Furthermore, since the spatial drawing information generation device 40 described above is composed of a computer, it goes without saying that the parts of the input information processing unit 41 and the display image generation unit 42 can be configured as software function units executed by a software program.

[0130] In the first embodiment described above, the correction memory 4102M of the indicated position calculation circuit 4102 is pre-stored with correction values ​​(Δx, Δy, Δz) at the time of factory shipment. However, this correction memory 4102M may also function as a buffer memory to receive correction value information from the 3D position indicator 10.

[0131] In this case, the 3D position indicator 10 is configured to transmit information such as the light output information from the light receiving sensor of the tracker 12, the output information from the 9-axis sensor, the operation information from the operation buttons 131a and 131b, and the correction values ​​(Δx, Δy, Δz) mentioned above to the spatial position detection circuit 410, as shown in Figure 8. The position calculation circuit 4102 of the spatial position detection circuit 410 then uses the correction values ​​stored in the correction memory 4102M, which acts as a buffer memory, to detect the position of the pen tip 11a of the electronic pen 11 as the position indicated by the 3D position indicator 10. In Figure 8, "ID" refers to the identification information of the 3D position indicator 10.

[0132] With this configuration, the 3D position indicator used in the spatial position detection system can utilize pens with different amounts of deviation between the position of the tracker 12 and the position of the pen tip 11a of the electronic pen 11. Furthermore, it becomes possible to attach multiple types of electronic pens with different axial lengths to the 3D position indicator 10. However, in this case, the 3D position indicator 10 will store correction values ​​corresponding to each electronic pen with different axial length dimensions, and the user will be able to select which correction value to use by operating the control device, depending on the attached electronic pen.

[0133] In the first embodiment described above, the three-dimensional position indicator 10 allows for the insertion and removal of the electronic pen 11, but the electronic pen 11 may be fixed to the holder 13. Furthermore, instead of using the electronic pen 11 as the rod-shaped portion, the rod-shaped portion may be integrally constructed with the holder 13 by arranging the functional part of the electronic pen 11—that is, a coil wound around a ferrite core and a capacitor connected in parallel to the coil to form a resonant circuit—within the rod-shaped portion.

[0134] [Second Embodiment] The second embodiment is a modification of the first embodiment described above, and the same reference numerals are used for parts identical to those in the first embodiment, and their descriptions are omitted.

[0135] In the three-dimensional position indicator 10 of the first embodiment described above, the position of the pen tip 11a of the electronic pen 11 is always set as the position indicated by the three-dimensional position indicator 10. However, when the electronic pen 11 is not attached to the holder 13, the position of the tracker 12 may be set as the indicated position, allowing it to be used for other purposes.

[0136] In other words, in the 3D position indicator 10A of this second embodiment, when the electronic pen 11 is attached to the holder 13A, the position of the pen tip 11a of the attached electronic pen 11 is used as the indicator position, as in the first embodiment. However, when the electronic pen 11 is not attached to the holder 13A, the position of the tracker 12 is used as the indicator position.

[0137] Figure 9 is a diagram illustrating an example of the configuration of the main parts of the 3D position indicator 10A of this second embodiment. Figure 9 shows a cross-sectional view of a portion of the grip portion 131, the electronic pen mounting portion 132, and the tracker mounting portion 133 of the holder 13A of the 3D position indicator 10A of this second embodiment.

[0138] In this second embodiment, the holder 13A of the 3D position indicator 10A is provided with mounting detection means for detecting whether or not the electronic pen 11 is mounted. In this example, the mounting detection output of the electronic pen 11 from this mounting detection means is notified to the spatial position detection circuit 410A (see Figure 10) of the spatial drawing information generation device 40A in this second embodiment, along with the light receiving output of the light receiving sensor, etc., via the tracker 12. Of course, the mounting detection output of the electronic pen 11 from the mounting detection means may be configured to be notified to the spatial position detection circuit 410A of the spatial drawing information generation device 40A independently, rather than via the tracker 12.

[0139] In this example, as shown in Figure 9, a switch member 134 is used as the mounting detection means, which includes a press 134a that can be elastically displaced in the direction of arrow AR. The switch member 134 is in an off state when the press 134a is not pressed and protruding outside the housing of the switch member 134, and is in an on state when the press 134a is pressed and pushed into the housing of the switch member 134.

[0140] Therefore, when the electronic pen 11 is inserted into the cylindrical recessed hole 132Aa of the electronic pen mounting section 132A, as shown in Figure 9, the housing of the electronic pen 11 presses down the presser 134a of the switch member 134, causing the switch state of the switch member 134 to change (in this example, from off to on). This switch state of the switch member 134 is then supplied to the spatial position detection circuit 410A of the spatial drawing information generation device 40A via the tracker 12 as the electronic pen 11 mounting detection output.

[0141] Figure 10 shows an example of the configuration of the spatial drawing information generation device 40A, particularly the spatial position detection circuit 410A, in this second embodiment. In this example of Figure 10, the same reference numerals are used for parts that are the same as those in the spatial drawing information generation device 40 and spatial position detection circuit 410 of the first embodiment described above, and their detailed descriptions are omitted.

[0142] In this second embodiment of the spatial position detection circuit 410A, in addition to the position indicator detection circuit 4101, the indicated position calculation circuit 4102, the digitizer detection circuit 4103, and the operation information detection circuit 4104 described above, an electronic pen attachment detection circuit 4105 and a switch circuit 4106 are provided.

[0143] The electronic pen mounting detection circuit 4105 receives and analyzes the mounting detection output of the electronic pen 11 from the 3D position indicator 10A, detects whether or not the electronic pen 11 is mounted on the electronic pen mounting section 132, and outputs a mounting detection output DP as a result of the detection.

[0144] One input terminal T of the switch circuit 4106 is supplied with position information TI of the tracker 12 of the 3D position indicator 10A detected by the position indicator detection circuit 4101, and the other input terminal P is supplied with position information PI of the pen tip 11a of the attached electronic pen 11, which is calculated by the indicator position calculation circuit 4102.

[0145] The switch circuit 4106 is then supplied with the attachment detection output DP of the electronic pen attachment detection circuit 4105 as a switching control signal. When the electronic pen 11 is not attached to the 3D position indicator 10A, the switch circuit 4106 is switched to one input terminal T and outputs the position information TI of the tracker 12 as output information. When the electronic pen 11 is attached to the 3D position indicator 10A, the switch circuit 4106 is switched to the other input terminal P and outputs the position information PI of the pen tip 11a of the electronic pen 11 as output information. The output information of the switch circuit 4106 is then supplied to the coordinate transformation circuit 413.

[0146] Furthermore, the attachment detection output DP of the electronic pen attachment detection circuit 4105 is supplied to the VR image generation circuit 422A. When the VR image generation circuit 422A determines from the attachment detection output DP that the electronic pen 11 is attached to the 3D position indicator 10A, it generates a VR image of the electronic pen 11 in the same manner as in the first embodiment described above and displays it on the HMD 50. In this second embodiment, when the VR image generation circuit 422A determines from the attachment detection output DP that the electronic pen 11 is not attached to the 3D position indicator 10A, it does not generate a VR image of the electronic pen 11, but instead generates, for example, a VR image of the tracker 12 or a predetermined mark at the detected position of the tracker 12 and displays it on the HMD 50.

[0147] As described above, in the second embodiment, the operation is the same as in the first embodiment only when the electronic pen 11 is attached to the 3D position indicator 10A. When the electronic pen 11 is not attached to the 3D position indicator 10A, it can be used for other purposes using the tracker position.

[0148] In the example described above, a switch member 134 is provided as a mounting detection means to automatically detect when the electronic pen 11 is mounted on the electronic pen mounting section 132A of the holder 13 of the 3D position indicator 10A. However, the mounting detection means is not limited to the switch member 134.

[0149] Furthermore, in the second embodiment described above, a mounting detection means is provided to automatically detect when the electronic pen 11 is mounted on the electronic pen mounting section 132A of the holder 13 of the 3D position indicator 10A. However, instead of a mounting detection means, the mounting of the electronic pen 11 may be notified to the spatial position detection circuit 410A of the spatial drawing information generation device 40A by the user's operation.

[0150] Furthermore, in the case where the user notifies the 3D position indicator 10 of the attachment of the electronic pen 11 to the 3D position indicator by operating the control unit of the 3D position indicator 10, even if the electronic pen 11 is not actually attached, the VR image generation circuit 422A can generate a VR image of the electronic pen 11 and display it on the HMD 50, assuming that the electronic pen 11 is virtually attached to the 3D position indicator 10.

[0151] [Modified versions of the above embodiments] In the three-dimensional position indicator 10 of the above-described embodiment, the holder 13 is configured to include a grip portion 131. However, as shown in Figures 11(A), (B), and (C), the three-dimensional position indicator 10B may be configured to include a holder 13B consisting of an electronic pen mounting portion 132 and a tracker mounting portion 133, without the grip portion 131. The three-dimensional position indicator 10B has the same configuration as the three-dimensional position indicator 10 described with reference to Figures 5 and 6, except for a change in the configuration of the holder, as indicated by the same reference numerals in Figures 11(A), (B), and (C).

[0152] In the case of this 3D position indicator 10B, the operation buttons 131a and 131b shown in Figures 5 and 6 may be provided on the electronic pen mounting section 132. Also, in the 3D position indicator 10B, the electronic pen 11 may be fixed to the holder 13B, or instead of using the electronic pen 11 as the rod-shaped part, the functional part of the electronic pen 11 may be arranged inside the rod-shaped part so that the rod-shaped part is integrally constructed with the holder 13B. Furthermore, it goes without saying that the configuration of the 3D position indicator 10B shown in Figure 11 is also applicable to the case of the second embodiment.

[0153] In the above-described embodiment, the spatial position detection unit 30 is configured to include a light emission tracking device that emits infrared laser light and a tracker, but it goes without saying that the configuration is not limited to this. For example, a configuration using other non-visible light sensors, visible light sensors, or a combination thereof may also be used.

[0154] Alternatively, the electronic pen may be equipped with a battery and a means for transmitting and receiving radio waves, and the spatial position of the electronic pen may be detected by transmitting radio waves to the electronic pen from an external source and receiving radio waves from the electronic pen. Alternatively, magnetic resonance or ultrasound may be used. Alternatively, the spatial position of the object to be detected (electronic pen or tablet) may be photographed with one or more cameras, and the spatial position of the object may be detected using the captured images.

[0155] Furthermore, in the above-described embodiment, a VR image of the digitizer 20 is generated and the HMD 50's display screen is configured to allow the operator to recognize the position of the digitizer 20. However, the HMD may also be configured to be AR (augmented reality) compatible, allowing the operator to directly view the digitizer 20.

[0156] Furthermore, in the above-described embodiment, gestures based on the movement of the electronic pen were detected in the spatial region outside the position detection area DT of the digitizer. However, even in the aforementioned outer spatial region, it is also possible to detect the position indicated by the tip of the electronic pen, rather than the movement of the electronic pen, and use that for drawing.

[0157] Furthermore, in the above-described embodiment, the distance between the tip 11a of the electronic pen 11 and the input surface of the digitizer 20 is detected based on the signal reception level between the sensor unit 22 of the digitizer 20 and the electronic pen 11. However, the method for detecting the distance between the tip 11a of the electronic pen 11 and the input surface of the digitizer 20 is not limited to this.

[0158] For example, the spatial position detection unit 30 can detect the position of the pen tip 11a of the electronic pen 10 and the position of the input surface 21S of the digitizer 20. From the detected positions of the pen tip 11a of the electronic pen 10 and the input surface 21S of the digitizer 20, the separation distance between the pen tip 11a of the electronic pen 11 and the input surface 21S of the digitizer 20 can be detected, and a selection control signal for the selection circuit 401 can be generated.

[0159] Furthermore, the means of displaying three-dimensional images are not limited to HMDs; 3D displays, AI (Aerial Imaging) plates, and even hologram technology may be used. Alternatively, a display such as an LCD may be provided on the digitizer 20, and 3D display may be shown on that display. These display means may be used in conjunction with the HMD to allow persons other than the HMD wearer to view the images displayed on the HMD.

[0160] Furthermore, while the coordinate transformation is performed by converting the coordinate values ​​of the pen tip of the electronic pen detected by the spatial position detection unit 30 into coordinate values ​​of the coordinate system in the digitizer 20, the reverse may also be performed by converting the coordinate values ​​of the pen tip of the electronic pen detected by the digitizer 20 into coordinate values ​​of the coordinate system in the spatial position detection unit 30.

[0161] Furthermore, while the above-described embodiment uses an electromagnetic induction type electronic pen and digitizer, it is not limited to this, and of course, it is also possible to use an electrostatic type (including active electrostatic coupling type and passive electrostatic coupling type) or other types of electronic pens and digitizers.

[0162] Furthermore, the digitizer in the above-described embodiment may be a portable mobile phone terminal known as a smartphone, or it may be a personal computer equipped with a digitizer.

[0163] In the above explanation, we described the case where 3D drawing is performed using a spatial position indication system, but the drawing image targeted by this invention may also be a 2D drawing image or a 2.5D drawing image. [Explanation of Symbols]

[0164] 10...3D position indicator, 11...Electronic pen, 11a...Pen tip of electronic pen 11, 12...Tracker, 13...Holder, 20...Digitizer, 22...Position detection sensor, 23...Position detection circuit, 24A,24B...Tracker, 30...Spatial position detection unit, 31A,31B...Light emission tracking device, 40...Spatial drawing information generation device, 41...Input information processing unit, 42...Display image generation unit, 50...HMD, 131...Grip unit, 132...Electronic pen mounting unit, 133...Tracker mounting unit, 410...Spatial position detection circuit, 413...Coordinate transformation circuit, 421...Drawing image generation circuit, 422...VR image generation circuit

Claims

1. A position indicator for inputting a position into a VR (Virtual Reality) space, A rod-shaped casing, A position indicator is provided at one end of the rod-shaped housing in the axial direction, A tracker located at a position different from the position of the position indicator located near the first and second fingers when the user grips the position indicator side of the rod-shaped housing with the first finger and a second finger different from the first finger, and the tracker has a specific positional relationship with the position of the position indicator, It has, Based on the position of the tracker detected by the external device and the specific positional relationship, the position to be input into the VR space is calculated. A position indicator characterized by the following features.

2. An operating part provided in a position that can be operated by the first finger or the second finger, further comprising an operating part provided on the position indicator side of the rod-shaped housing. The position indicator according to feature 1.

3. The system further includes a communication unit that communicates with the external device in response to the operation of the control unit by the user. The position indicator according to feature 2.

4. The position indicator is a position indicator that indicates the position on the input surface of the position detection sensor. The position indicator unit is coupled to the position detection sensor using an electromagnetic induction method. The position indicator according to feature 1.

5. The position indicator is a position indicator that indicates the position on the input surface of the position detection sensor. The position indicator unit is coupled to the position detection sensor using an electrostatic coupling method. The position indicator according to claim 1, characterized in that it is a position indicator.

6. The aforementioned VR space includes an MR (Mixed Reality) space or an AR (Augmented Reality) space. The position indicator according to feature 1.

7. The tracker is positioned so as not to be covered by the user's hand when the user grips the rod-shaped housing. The position indicator according to feature 1.

8. The aforementioned tracker includes an LED (Light Emission Diode). The position indicator according to feature 1.

9. A position detection system including a position indicator, a spatial position detection unit, and a computer for inputting a position into a VR (Virtual Reality) space, The position indicator is, A rod-shaped casing, A position indicator is provided at one end of the rod-shaped housing in the axial direction, A tracker located at a position different from the position of the position indicator located near the first and second fingers when the user grips the position indicator side of the rod-shaped housing with the first finger and a second finger different from the first finger, and the tracker has a specific positional relationship with the position of the position indicator, It has, The aforementioned spatial position detection unit is It has a position detection unit that detects the position of the tracker, The aforementioned computer, The circuit has a function that calculates the position to be input into the VR space based on the position of the tracker detected by the spatial position detection unit and the specific positional relationship. A position detection system characterized by the following features.

10. The position detection system is a system having a position detection sensor. The aforementioned computer, The circuit has a coordinate transformation that performs a coordinate transformation to make one of the coordinate systems of the VR space and the coordinate system on the input surface of the position detection sensor match the other. The position detection system according to claim 9.

11. The position detection system further includes a display that shows the trajectory of the position indicated by the position indicator. The image displayed on the aforementioned display, including the trajectory of the aforementioned position, is a VR (Virtual Reality) image. The position detection system according to claim 9.

Images