Coordinate calculation system and coordinate calculation method

Abstract

To satisfactorily calculate coordinates in space of a position of a pen tip of an electronic pen.SOLUTION: A coordinate calculation system includes: a tablet that detects a position of a pen tip of an electronic pen; a tracking device that detects a position of a tracker of the electronic pen, performs a position input by the electronic pen in VR (Virtual Reality) space, and is located above the tablet; and a computer that calculates first coordinates in a first coordinate system of the VR (Virtual Reality) space on the basis of the position of the tracker detected by the tracking device in a first mode, and calculates second coordinates in a second coordinate system of the tablet different from the first coordinate system on the basis of the position of the pen tip detected by the tablet in a second mode switched from the first mode according to predetermined operation.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a coordinate calculation system and a coordinate calculation method suitable for use with an electronic pen in space. [Background technology]

[0002] Drawing systems that create animation images and the like by drawing by continuously specifying positions with an electronic pen on a coordinate input device called a digitizer are known. In this case, an operator performs a position specifying operation to generate a drawing image while the electronic pen is in contact with the input surface of a tablet device incorporating a digitizer, or while not in contact with the input surface of the tablet device but placed in an upper area where the position can be detected (hover state). The digitizer detects the position specified by the electronic pen, generates a drawing image as a result of the detection, and displays it on a display screen. The operator performs drawing while checking the drawing image displayed on the display screen.

[0003] Recently, systems and applications have appeared that allow a drawn image displayed on a two-dimensional display screen to be visually expressed as a three-dimensional image (for example, by rotating or transforming it). In these cases, a motion sensor is used to detect the movements (gestures) of the operator's hands or fingers, and the drawn image processing is performed based on the detected movements (gestures).

[0004] Conventionally, the process of creating a two-dimensional image based on detecting the position indicated by an electronic pen on a digitizer and the process of drawing and handling the two-dimensional image as if it were a three-dimensional image were carried out as separate, independent processes.

[0005] Meanwhile, there is provided a user interface that allows both the above-mentioned position input and operation input such as gestures (see, for example, Patent Documents 1 and 2).

[0006] Patent Document 1 discloses a touch controller configured to switch from a hover event detection mode to a gesture event detection mode in response to a signal from a motion sensor.

[0007] Furthermore, Patent Document 2 discloses a controller means that alternately determines position information detected via a touch sensor display and position information detected via a non-contact detection means. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent No. 9,367,169 [Patent Document 2] U.S. Patent No. 9,323,379 Summary of the Invention [Problem to be solved by the invention]

[0009] However, the method of switching in response to a signal from a motion sensor, as in Patent Document 1, has the problem that the tilt of the device must be changed each time for switching.

[0010] Furthermore, in the case of Patent Document 2, the touch sensor display and the non-contact detection means are switched in a time-division manner, which causes a problem of reduced time resolution for each.

[0011] SUMMARY OF THE INVENTION An object of the present invention is to provide a coordinate calculation system that can solve the above problems. [Means for solving the problem]

[0012] To solve the above problems, a tablet that detects the position of the tip of the electronic pen; a tracking device that detects the position of a tracker of the electronic pen, the tracking device being used to input a position in a VR (Virtual Reality) space using the electronic pen, and that is located above the tablet; a computer that, in a first mode, calculates a first coordinate in a first coordinate system of the VR (Virtual Reality) space based on the position of the tracker detected by the tracking device, and, in a second mode switched from the first mode in response to a predetermined operation, calculates a second coordinate in a second coordinate system of the tablet, which is different from the first coordinate system, based on the position of the pen tip detected by the tablet; The coordinate calculation system is characterized by having the following. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram for explaining an overview of a first embodiment of a spatial position indication system according to the present invention. [Figure 2] 1 is a block diagram illustrating an example of the configuration of a spatial position indication system according to a first embodiment of the present invention. [Figure 3] FIG. 1 is a diagram for explaining a spatial coordinate system in a first embodiment of a spatial position indication system according to the present invention. [Figure 4] FIG. 2 is a diagram for explaining the operation of a part of the first embodiment of the spatial position indication system according to the present invention. [Figure 5] 3A and 3B are diagrams illustrating switching of coordinate systems for detecting the position of an electronic pen in the spatial position indication system according to the first embodiment of the present invention. [Figure 6] 3 is a diagram showing an example of a display image on a display screen of a display unit in the spatial position indication system according to the first embodiment of the present invention. FIG. [Figure 7] FIG. 10 is a diagram for explaining a modified example of the first embodiment of the spatial position indication system according to the present invention. [Figure 8]FIG. 10 is a diagram for explaining a modified example of the first embodiment of the spatial position indication system according to the present invention. [Figure 9] FIG. 10 is a diagram for explaining a modified example of the first embodiment of the spatial position indication system according to the present invention. [Figure 10] FIG. 10 is a diagram for explaining a modified example of the first embodiment of the spatial position indication system according to the present invention. [Figure 11] FIG. 2 is a diagram for explaining an overview of a second embodiment of a spatial position indication system according to the present invention. [Figure 12] FIG. 4 is a diagram for explaining the main operation of the second embodiment of the spatial position indication system according to the present invention. [Figure 13] FIG. 4 is a block diagram illustrating a configuration example of a second embodiment of a spatial position indication system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of a spatial position indication system according to the present invention will be described with reference to the drawings.

[0015] [First embodiment] FIG. 1 is a diagram showing an example of the overall configuration of a first embodiment of a spatial position indication system according to the present invention. As shown in FIG. 1, the spatial position indication system of the first embodiment includes an electronic pen 1 and a tablet device 2. In this embodiment, the tablet device 2 has an internal configuration as shown in FIG. 2. Generally, when the height position in the Z-axis direction from the input surface of the tablet device 2 to the tip of the electronic pen 1 is smaller than a critical height position Lz (e.g., height position B1 in FIG. 1), the pointing position of the electronic pen 1 detected by the pointing position detection device 203 is used. On the other hand, when the height position in the Z-axis direction from the input surface of the tablet device 2 to the tip of the electronic pen 1 is equal to or greater than the critical height position Lz (e.g., height position B2 in FIG. 1), the pointing position of the electronic pen 1 detected by the spatial position detection device 204 is used. In general, the accuracy of position detection by the pointing position detection device 203 is higher than the accuracy of position detection by the spatial position detection device 204. Therefore, when the electronic pen 1 moves away from the input surface for receiving instruction input of the tablet device 2 and is positioned in the hover area above it, it is preferable to continue using position detection by the instruction position detection device unit 203, and when the height position becomes equal to or higher than the critical height position Lz, to use position detection by the spatial position detection device unit 204.

[0016] The tablet device 2 of the spatial position indication system of the first embodiment includes an LCD (Liquid Crystal Display) 201 as an example of a display unit, and a display screen 201D of the LCD 201 is disposed on the surface of a housing 202.

[0017] In the tablet device 2, a sensor unit 2031 of an indication position detection device unit (hereinafter referred to as a digitizer) 203 that detects an indication position by the electronic pen 1 is disposed on the back side of the LCD 201 in a state where it is superimposed on the LCD 201. As shown in FIG. 2, the digitizer 203 includes the sensor unit 2031 and a position detection circuit 2032.

[0018] Although not shown, the sensor unit 2031 is configured by a plurality of loop coils arranged in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) of the housing 202 of the tablet device 2. In this example, the digitizer 203 is of an electromagnetic induction type, but the present embodiment is not limited to this.

[0019] On the other hand, the electronic pen 1 has a resonant circuit (not shown) on the pen tip side, which is made up of a coil and a capacitor, and signals are exchanged between the electronic pen 1 and the sensor section 2031 of the digitizer by electromagnetic induction coupling between the loop coil of the sensor section 2031 of the digitizer 203 and the resonant circuit of the electronic pen 1. The electronic pen 1 is also of the electromagnetic induction type, but this embodiment is not limited to this.

[0020] The position detection circuit 2032 of the digitizer 203 supplies a signal to the electronic pen 1 through the loop coil of the sensor unit 2031, and also receives a signal from the electronic pen 1 through the loop coil, and based on the received signal, detects a position indicated by the electronic pen 1 in the detection area of ​​the sensor unit 2031. In this embodiment, the digitizer 203 is configured to detect a position indicated by the pen tip of the electronic pen 1.

[0021] In this example, the loop coil of the sensor unit 2031 of the digitizer 203 is arranged to cover almost the entire area of ​​the display screen 201D of the LCD 201, and the area that is almost the same as the entire surface of the display screen 201D is the input surface 203S of the electronic pen 1 for the sensor unit 2031 (the area surrounded by the dotted line in Figure 1).

[0022] In this embodiment, the position detection area in which the digitizer 203 can detect the pointing position of the electronic pen 1 includes not only the planar area when the tip of the electronic pen 1 is in contact with the input surface 203S of the digitizer 203, but also a spatial area (hover area in the hover state of the electronic pen 1) in which the tip of the electronic pen 1 is not in contact with the input surface 203S of the digitizer 203 and is spaced from the input surface 203S in a direction perpendicular to the input surface 203S (the Z-axis direction perpendicular to the X-axis and Y-axis directions), but in which the pointing position of the electronic pen 1 can be detected through the transmission and reception of signals by electromagnetic coupling.

[0023] For example, in FIG. 1, when the position P0 of the upper left corner of the input surface 203S of the digitizer 203 is set to the coordinates of the origin in the X-axis direction, Y-axis direction, and Z-axis direction ((X, Y, Z) = (0, 0, 0)), the position detection area DT in which the pointing position of the electronic pen 1 on the digitizer 203 can be detected is the planar area of ​​the input surface 203S shown with diagonal lines in FIG. 3 and the spatial area of ​​a rectangular parallelepiped above the input surface 203S.

[0024] That is, as shown in Figure 1, if the length in the X-axis direction of the input surface 203S of the digitizer 203 is Lx, the length in the Y-axis direction is Ly, and the critical height position in the Z-axis direction at which the hover state can be detected is Lz, then, as shown in Figure 3, the area surrounded by the eight coordinate point positions of P0 (0,0,0), P1 (Lx,0,0), P2 (Lx,Ly,0), P3 (0,Ly,0), P4 (0,0,Lz), P5 (Lx,0,Lz), P6 (Lx,Ly,Lz), and P7 (0,Ly,Lz) becomes the position detection area DT of the digitizer 203.

[0025] The tablet device 2 of this first embodiment is provided with a spatial position detection device unit 204 as means for detecting the spatial position of the electronic pen 1, separate from the digitizer 203. As shown in Fig. 1, a spatial position detection member 2041 of the spatial position detection device unit 204 is provided on the surface of the housing 202. In this example, although not shown, the spatial position detection member 2041 is configured to include an invisible light sensor (for example, an infrared light emitting unit (infrared LED) and an infrared camera), but this embodiment is not limited to this, and other sensors such as a visible light sensor or a combination thereof may also be used.

[0026] The infrared light emitting unit of the spatial position detection member 2041 of the spatial position detection device unit 204 sets a spatial region including at least a part of the position detection region DT of the digitizer as a search region, and emits infrared light to search for an object present in the search region. That is, the infrared light emitting unit of the spatial position detection member 2041 emits infrared light to search a space including at least a part of the position detection region DT of the digitizer. The infrared camera of the spatial position detection member 2041 then receives the infrared light reflected from the object present in the spatial region, thereby detecting the presence and position of the object. As described above, in this embodiment, the object detected by the spatial position detection device unit 204 is the electronic pen 1, and the position of its pen tip is detected, for example.

[0027] In this second embodiment, the spatial position detection device unit 204 is configured to detect the spatial position of the pen tip of the electronic pen 1 in consideration of continuity with the pointing position of the electronic pen 1 detected by the digitizer 203, but this embodiment is not limited to this and may be configured to detect the position of one or more other parts of the electronic pen 1. The tablet device 2 of this embodiment may be configured to detect a gesture performed by the operator of the electronic pen 1 from the movement of the pen tip obtained from the spatial position of the pen tip of the electronic pen 1 detected by the spatial position detection device unit 204.

[0028] The spatial position detection circuit 2042 (see FIG. 2) of the spatial position detection device unit 204 detects the position of the object within the aforementioned large spatial region, in this example, the pen tip position of the electronic pen 1, from light reception information (such as the light reception direction and the difference between the light emission time and the light reception time) of the reflected light received by the infrared camera. Then, in this example, the spatial position detection device unit 204 detects the movement of the pen tip position of the electronic pen 1 within the spatial region. In this specification, for convenience, the spatial region searched by the spatial position detection device unit 204 will be referred to as the movement detection spatial region MD.

[0029] The spatial coordinate system of the motion detection space region MD of the spatial position detection device unit 204 can be set independently of the spatial coordinate system of the position detection region DT of the digitizer 203. In Fig. 3, the three axes of the spatial coordinate system of the motion detection space region MD are indicated using the suffix s as the Xs-axis, Ys-axis, and Zs-axis to distinguish them from the X-axis, Y-axis, and Z-axis of the spatial coordinate system of the detection region DT of the digitizer 203. In this example, as shown in Fig. 1, the motion detection space region MD of the spatial position detection device unit 204 is determined such that the center position of the portion of the housing 202 of the tablet device 2 where the spatial position detection member 2041 is installed is set as the origin position of the spatial coordinate system of the motion detection space region MD.

[0030] In FIG. 3, for the sake of convenience, the directions of the Xs-axis, Ys-axis, and Zs-axis are shown as being the same as the directions of the X-axis, Y-axis, and Z-axis, respectively. However, due to the relationship between the infrared light-emitting unit constituting the spatial position detection member 2041 and the optical axis direction of the infrared camera, the directions of the Xs-axis, Ys-axis, and Zs-axis may be different from the directions of the X-axis, Y-axis, and Z-axis, respectively.

[0031] However, in this embodiment, the position information of the pen tip position of the electronic pen 1 is configured to be able to perform coordinate conversion using spatial position correction information, which will be described later, by using a common area between the spatial coordinate system of the position detection area DT of the digitizer 203 and the spatial coordinate system of the motion detection space area MD of the spatial position detection device unit 204. In other words, the spatial coordinate system of the motion detection space area MD of the spatial position detection device unit 204 is arranged so as to include at least a part of the spatial coordinate system of the position detection area DT of the digitizer 203 as a common area.

[0032] Note that, if the directions of the Xs-axis, Ys-axis, and Zs-axis are the same as the directions of the X-axis, Y-axis, and Z-axis, the spatial coordinate system of the position detection region DT of the digitizer 203 and the spatial coordinate system of the motion detection space region MD can be commonly treated as a single spatial coordinate system by taking into consideration the difference in the origin positions of both spatial coordinate systems. That is, for example, if the offset values ​​in the X-axis direction, Y-axis direction, and Z-axis direction between the origin position of the spatial coordinate system of the position detection region DT of the digitizer 203 and the origin position of the spatial coordinate system of the motion detection space region MD are Δx, Δy, and Δz, respectively, then the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection space region MD can be converted to coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection region DT of the digitizer 203 by calculating Xs-Δx (=X), Ys-Δy (=Y), and Zs-Δz (=Z).

[0033] However, in the tablet device 2 of the first embodiment, as described above, the directions of the X-axis, Y-axis, and Z-axis and the directions of the Xs-axis, Ys-axis, and Zs-axis may differ between the spatial coordinate system of the position detection region DT and the spatial coordinate system of the motion detection region MD of the independently provided digitizer 203. Furthermore, even if the directions of the X-axis, Y-axis, and Z-axis are the same as the directions of the Xs-axis, Ys-axis, and Zs-axis, it is difficult to accurately define the offset value of the origin position, and there is a risk that the offset value will differ for each tablet device 2.

[0034] This results in different coordinate positions in the two spatial coordinate systems, and for example, when the pen tip position of electronic pen 1 is indicated on display screen 201D with a cursor such as an arrow, when switching between the two spatial coordinate systems of position detection region DT of digitizer 203 and the motion detection spatial region MD, there is a risk that the cursor CS may jump from the display position of solid line to the state of dashed line cursor CS', as shown in Fig. 4. This means that a user who is trying to input a drawing image must re-indicate the indicated position.

[0035] Therefore, in this first embodiment, by utilizing the fact that at least a portion of the position detection region DT of the digitizer 203 is a spatial region common to the motion detection spatial region MD, correction information for the deviation between the spatial coordinate system of the detection region DT of the digitizer 203 and the spatial coordinate system of the motion detection spatial region MD is generated. In this example, the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection region DT of the digitizer 203 are converted into coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection spatial region MD detected by the spatial position detection circuit 2042 of the spatial position detection device unit 204, thereby making it possible to obtain coordinate values ​​in which the deviation between the two has been corrected. The correction information for this conversion will be described next.

[0036] Equation 1 shows a determinant for linearly converting the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection space area MD detected by the spatial position detection circuit 2042 of the spatial position detection device unit 204 into the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the position detection area DT of the digitizer 203. This determinant has 3 rows and 3 columns, and its components are a ij It is expressed as (i,j=1, 2, 3).

[0037]

number

[0038] That is, as shown in FIG. 3, at least three points Pa, Pb, and Pc are specified within the spatial region common to the digitizer's position detection region DT and motion detection spatial region MD, and the coordinate values ​​(X, Y, Z) of each point in the spatial coordinate system of the digitizer's position detection region DT and the coordinate values ​​(Xs, Ys, Zs) of each point in the spatial coordinate system of the motion detection spatial region MD are acquired from the respective devices. Ideally, the coordinate values ​​acquired from these devices will be the same, but unless calibration is performed, these coordinate values ​​will usually not match. Generally, because the position detection accuracy of the digitizer 203 is higher than that of the spatial position detection device unit 204, it is preferable to align the coordinate values ​​of the spatial coordinate system of the motion detection spatial region MD of the spatial position detection device unit 204 with the coordinate values ​​of the spatial coordinate system of the position detection region DT of the digitizer 203.

[0039] For each point specified within the common area, three equations are obtained by substituting the coordinate values ​​(X, Y, Z) of the spatial coordinate system of the corresponding position detection area DT and the coordinate values ​​(Xs, Ys, Zs) of the spatial coordinate system of the motion detection space area MD into Equation 1. By specifying the positions of at least three points within the common area, 11 ~a 33 Since we have at least nine equations for a 11 ~a 33 The conversion between the spatial coordinate system of the detection region DT of the digitizer and the spatial coordinate system of the motion detection region MD is not limited to the above method, and may be performed by machine learning using the coordinate values ​​of at least three points in the common region, or by user calibration.

[0040] As described above, the tablet device 2 of the first embodiment is configured so that at least a part of the position detection area DT of the digitizer 203 is included in the motion detection space area MD of the spatial position detection device unit 204. In other words, the tablet device 2 is configured so that the motion detection space area MD of the spatial position detection device unit 204 has at least a part of the position detection area DT of the digitizer 203 as a common area.

[0041] In this embodiment, the tablet device 2 is configured to switch between using the indicated position (first position) of the electronic pen 1 detected by the digitizer 203 or the position in the spatial domain of the electronic pen 1 detected by the spatial position detection device unit 204 (second position) depending on the distance (distance in the Z-axis direction) between the pen tip position of the electronic pen 1 and the input surface 203S of the sensor unit 2031 of the digitizer 203.

[0042] 5, when the distance from the input surface 203S (distance in the Z-axis direction) that is the switching point is θth, when the distance from the input surface 203S of the electronic pen 1 is smaller than θth, the tablet device 2 operates as a device in which the digitizer 203 detects the position pointed to by the electronic pen 1 in its position detection region DT and performs processing according to the detected pointed position. On the other hand, when the distance from the input surface 203S of the electronic pen 1 is larger than θth, the tablet device 2 is switched to operate as a device in which the spatial position detection unit 204 detects the position of the electronic pen 1 within the spatial region and performs processing according to the position.

[0043] In this embodiment, the separation distance θth (separation distance in the Z-axis direction) from the input surface 203S that serves as the switching point is set to be equal to or less than the critical height distance Lz in the Z-axis direction at which the digitizer 203 can detect the hover state of the electronic pen 1. In this example, the separation distance θth from the input surface 203S that serves as the switching point is set to be equal to the critical height distance Lz in the Z-axis direction at which the digitizer 203 can detect the hover state of the electronic pen 1, that is, the length Lz in the Z-axis direction of the position detection area DT, as shown in FIG.

[0044] In other words, when the electronic pen 1 is within the position detection area DT of the digitizer 203, the tablet device 2 uses the results of detection of the position of the electronic pen 1 by the digitizer 203, and when the electronic pen 1 is outside the position detection area DT of the digitizer 203, it switches to use the results of detection of the movement of the electronic pen 1 by the spatial position detection device unit 204, and the drawing process is also switched according to this switching.

[0045] In this embodiment, the distance (distance in the Z-axis direction) between the tip position of the electronic pen 1 and the input surface 203S of the sensor unit 2031 of the digitizer 203 is detected by the tablet device 2 based on the signal level (signal strength) of the signal received from the electronic pen 1 at the sensor unit 2031 of the digitizer 203, since the signal level (signal strength) of the signal received from the electronic pen 1 is a value corresponding to the distance.

[0046] Next, an example of the components of the tablet device 2 that realize the above will be described. That is, the position detection circuit 2032 that constitutes the digitizer 203 supplies the detection output of the pointing position of the electronic pen 1 as one of the input signals of the selection circuit 205. Note that the information supplied from this position detection circuit 2032 to the selection circuit 205 includes not only the detection output of the pointing position of the electronic pen 1 but also information about the writing pressure applied to the electronic pen 1. This writing pressure information makes it possible to determine whether the electronic pen 1 is in contact with the input surface 203S, and also, when drawing a line, to draw a line of a thickness that corresponds to the magnitude of the writing pressure.

[0047] The spatial position detection circuit 2042 of the spatial position detection device unit 204 supplies the detection output of the spatial position of the electronic pen 1 to the spatial position coordinate correction circuit 206. The spatial position coordinate correction circuit 206 receives the component a of the determinant shown in the above (Equation 1) from the correction information memory 207. 11 ~a 33is supplied. The spatial position coordinate correction circuit 206 performs the calculation of the determinant of Equation 1 using the components supplied from the correction information memory 207, and converts the coordinate information in the spatial coordinate system of the spatial position detection device unit 204 into coordinate information in the spatial coordinate system of the digitizer 203. Then, the spatial position coordinate correction circuit 206 supplies the converted coordinate output to the gesture detection circuit 208.

[0048] The gesture detection circuit 208 detects the movement (gesture) of the pen tip of the electronic pen 1 based on the coordinate output from the spatial position coordinate correction circuit 206, and supplies the detection output to the selection circuit 205 as the other input signal.

[0049] In this embodiment, the position detection circuit 2032 of the digitizer 203 supplies information on the signal level of the signal received from the electronic pen 1 to the separation distance detection circuit 210. In this embodiment, the separation distance detection circuit 210 detects the separation distance between the pen tip of the electronic pen 1 and the input surface 203S of the digitizer 203 from the signal level of the signal received from the electronic pen 1. The separation distance detection circuit 210 then supplies information on the detected separation distance to the selection control signal generation circuit 211.

[0050] The selection control signal generation circuit 211 generates a selection control signal SE that controls the selection circuit 205 to select the detection output of the digitizer 203 when the distance between the tip of the electronic pen 1 and the input surface 203S of the digitizer 203 is equal to or less than the distance Lz, and controls the selection circuit 205 to select the detection output of the gesture detection circuit 208 when the distance is greater than the distance Lz, and selects and controls the selection circuit 205 using the generated selection control signal SE.

[0051] The selection circuit 205 switches between one input and the other input in response to a selection control signal SE, and supplies the selected input to the rendering processing circuit 212 .

[0052] The drawing processing circuit 212 includes a pen drawing processing unit 2121 that draws fine line drawings and the like based on the indication position detection output of the electronic pen 1 from the digitizer 203, and a gesture processing unit 2122 that performs drawing processing based on a movement (gesture) detected based on the spatial position of the electronic pen 1 from the spatial position detection device unit 204. The drawing processing circuit 212 receives a switching control signal SW from the switching signal generation circuit 211, and the drawing processing circuit 212 switches between the pen drawing processing unit 2121 and the gesture processing unit 2122 in conjunction with the switching of the switch circuit 205 to perform drawing processing of a 3D drawn image.

[0053] The 3D drawing image information generated by the drawing processing circuit 212 is supplied to the display unit 201, which in this example is configured as an LCD, via the display drive circuit 213, and displayed on the display screen 201D.

[0054] Fig. 6 shows an example of a 3D drawn image displayed on the display screen 201D of the display unit 201. In this embodiment, as shown in Fig. 6, the display screen 201D displays a current mode display 220 that notifies the user whether the switching state of the switch circuit 205 is on the digitizer 203 side (tablet mode) or on the gesture detection circuit 208 side (space mode). This current mode display 220 is generated by the drawing processing circuit 212 based on, for example, the switching control signal SW, and is displayed on the display image.

[0055] The current mode display 220 is intended to inform the user whether the electronic pen 1 is present within the position detection area DT of the digitizer 203 or within the motion detection space area MD of the spatial position detection device unit 204. In the example of Figure 6, it consists of a bar display 221 which indicates in which area the electronic pen 1 is present by the height of a bar 221B which is shaded, and a character display field 222 which indicates which area the bar display 221 is indicating.

[0056] Furthermore, if the drawing processing circuit 212 is supplied with the detection output of the separation distance detection circuit 210 and the coordinate output from the spatial position coordinate correction circuit 206, the drawing processing circuit 212 can know the separation distance of the electronic pen 1 from the input surface 203S, and therefore the height of the bar of the bar display 221 of the position of the electronic pen 1 in the current mode display 202 can be displayed according to the separation distance of the electronic pen 1 from the input surface 203S.

[0057] FIG. 6A shows the display screen when the electronic pen 1 is present within the position detection region DT of the digitizer 203 and a rectangular parallelepiped object OBJ is drawn with the electronic pen 1.

[0058] Figure 6(B) shows an example of a display image when the user moves the electronic pen 1 from the state shown in Figure 6(A) outside the position detection area DT of the digitizer 203 and performs a gesture operation to rotate the rectangular parallelepiped object OBJ, for example, as indicated by the arrow AR in Figure 6(B).

[0059] In Figure 6, the drawing in the position detection area DT of the digitizer 203 and the drawing in the motion detection space area MD of the spatial position detection device unit 204 are just examples, and drawing by gesture operation can also perform drawing processes such as moving an object, applying pressure, or pulling and stretching a part of it to deform the object.

[0060] The electronic pen 1 is provided with a pen pressure detection circuit that detects the pen pressure applied to the pen tip when in contact with the input surface 203S, and information on the detection results of this pen pressure detection circuit is configured to be transmitted to the tablet device 2. Fine drawing operations using the electronic pen 1 are often performed while in contact with the input surface 203S. Therefore, the tablet device 2 may distinguish between a state in which the electronic pen 1 is in contact with the input surface 203S and a hover state in which the electronic pen 1 is not in contact, based on the information on the pen pressure detection results received from the electronic pen 1, and may display text such as "contact state" or "hover state" on the display screen 201D to notify the operator of this.

[0061] The tablet device 2 includes a control circuit 200, which may be a computer, and the above-described circuits operate under the control of the control circuit 200. The processes of the spatial position coordinate correction circuit 206, gesture detection circuit 208, separation distance detection circuit 210, selection control signal generation circuit 211, and drawing processing circuit 212 can also be configured as software functional units that are executed by the control circuit 200 using a software program.

[0062] The correction information memory 207 stores the components a of the determinant 11 ~a 33 However, in this embodiment, the user can store the correction information (components a of the determinant) in the tablet device 2 before starting up the tablet device 2 by having the tablet device 2 execute a correction information generation and storage process so that the error of each tablet device 2 can be corrected. 11 ~a 33 ) can be stored in the correction information memory 207.

[0063] 2 is a processing unit for this purpose, and executes the correction information generation process under the control of the control circuit 200. The correction information generation circuit 209 can also be provided as an application for the correction information generation and storage process included in the control circuit 200.

[0064] When the tablet device 2 is first started up, the control circuit 200 controls the correction information generation circuit 209 to operate. Then, the control circuit 200 prompts the user to use the electronic pen 1 to indicate the positions of at least three points in the position detection area DT of the digitizer 203, which is common to the spatial position detection device unit 204 and the digitizer 203. When the user indicates the positions of three or more points in response, as described above, the detection coordinate information of the digitizer 203 and the detection coordinate information of the spatial position detection device unit 204 for the three or more points is taken into the correction information generation circuit 209.

[0065] Then, the correction information generating circuit 209 uses the coordinate information of the three or more points that have been taken in to calculate the component a of the determinant as described above. 11 ~a 33 is calculated and stored in the correction information memory 207.

[0066] The component a of the determinant generated by the correction information generating circuit 209 is 11 ~a 33 The calculation of and the process of storing in the correction information memory 207 may of course be set in advance before selling to the user, rather than being performed by the user.

[0067] As configured as described above, in the tablet device 2 of this embodiment, when the electronic pen 1 is in contact with or hovering over the input surface 203S of the digitizer 203, the digitizer 203 detects the position indicated by the electronic pen 1 and performs detailed line drawing, and when the electronic pen 1 is farther away from the input surface 230S than the separation distance Lz, the digitizer 203 switches to spatial position detection by the spatial position detection device unit 204, detects the movement (gesture) of the electronic pen 1, and performs an operation according to the detected movement (gesture).

[0068] Therefore, the operator can seamlessly switch from fine drawing to gesture operation simply by moving the electronic pen 1 spatially on the tablet device 2 without being aware of switching between the digitizer 203 and the spatial position detection device unit 204.

[0069] In the tablet device 2 of this embodiment, the coordinate output of the digitizer 203 is converted into values ​​in the spatial coordinate system of the spatial position detection device unit 204, so that even when the spatial coordinate system is switched, the problem of the cursor position displayed on the display screen jumping in response to the position detection of the electronic pen 1 can be prevented.

[0070] In addition, the display screen 201D of the display unit 201 of the tablet device 2 displays whether the pointing position of the electronic pen 1 is detected by the digitizer 203 or the spatial position of the electronic pen 1 is detected by the spatial position detection device unit 204, depending on the distance from the input surface 203S of the electronic pen 1, so that the operator of the electronic pen 1 can accurately understand what operation should be performed at the spatial position of the electronic pen 1 at that time.

[0071] [Modification of the first embodiment] <Variation 1> In the above description of the first embodiment, the digitizer 203 calculates the distance between the electronic pen 1 and the input surface 203S based on the signal level of the signal received from the electronic pen 1, and generates a switching signal for the switch circuit 205 from the calculated distance. However, the method for calculating the distance between the electronic pen 1 and the input surface 203S is not limited to the above example.

[0072] For example, when the electronic pen 1 receives a signal from the sensor unit 2031 of the digitizer 203 of the tablet device 2, it can detect the signal level of the received signal and transmit information about the signal level to the tablet device 2. Fig. 7 is a diagram showing an example of the configuration of the electronic pen 1A and the main parts of the tablet device 2A in such a configuration.

[0073] In this example, electronic pen 1A includes a received signal level detection circuit 101 that detects the signal level of a signal received from sensor unit 2031 of digitizer 203 of tablet device 2A, and also includes a wireless communication unit 102 that performs wireless communication according to, for example, the Bluetooth (registered trademark) standard. Electronic pen 1A then transmits information about the signal level of the signal received from sensor unit 2031, detected by received signal level detection circuit 101, to tablet device 2A via wireless communication unit 102.

[0074] The tablet device 2A includes a wireless communication unit 221 conforming to the Bluetooth (registered trademark) standard for wireless communication with the wireless communication unit 102, and also includes a separation distance detection circuit 222 and a selection control signal generation circuit 223 instead of the separation distance detection circuit 210 and the switching signal generation circuit 211 shown in Fig. 2. In this example, the separation distance detection circuit 222 detects the separation distance between the electronic pen 1A and the input surface 203S of the digitizer 203 based on signal level information from the electronic pen 1A received by the wireless communication unit 221, and supplies information about the detected separation distance to the selection control signal generation circuit 223.

[0075] The selection control signal generation circuit 223 generates a selection control signal SEa that controls the selection circuit 205 to select the detection output of the digitizer 203 when the distance between the tip of the electronic pen 1 and the input surface 203S of the digitizer 203 is equal to or less than the distance Lz, and to select the detection output of the gesture detection circuit 208 when the distance is greater than the distance Lz, and controls the selection circuit 205 to switch using the generated switching control signal SEa.

[0076] The other configurations of the tablet device 2A are the same as those shown in FIG.

[0077] 7, the radio wave intensity of the signal transmitted from the sensor unit 2031 of the digitizer 203 to the electronic pen 1A is greater than the radio wave intensity of the signal received by the tablet device 2 from the electronic pen 1. Therefore, the separation distance detection circuit 222 and the switching signal generation circuit 223 of the tablet device 2A can detect the position of the separation distance Lz of the switching point more accurately than in the case of the tablet device 2 with the configuration of FIG.

[0078] <Variation 2> In the above example, the distance between the input surface 203S of the digitizer 203 and the electronic pen 1 or 1A is detected based on the reception level of the signal between the sensor unit 2031 of the digitizer 203 and the electronic pen 1 or 1A. However, it is also possible to generate a selection control signal for the selection circuit 205 from the pen tip position of the electronic pen 1 detected by the spatial position detection device unit 204.

[0079] Fig. 8 is a diagram showing an example of the configuration of the main parts of a tablet device 2B configured in this manner. That is, in the tablet device 2B of this example, instead of the separation distance detection circuit 210 and the switching signal generation circuit 211 in the example of Fig. 2, a separation distance detection circuit 224 and a switching signal generation circuit 225 are provided, as shown in Fig. 8. In this example, the position coordinate output of the spatial position coordinate correction circuit 206 is supplied to the separation distance detection circuit 224. The separation distance detection circuit 224 detects the spatial position of the electronic pen 1 from the position coordinate output of the spatial position coordinate correction circuit 206, detects the separation distance of the electronic pen 1A from the input surface 203S of the digitizer 203 from the Z coordinate, and supplies the detection output to the selection control signal generation circuit 225.

[0080] The selection control signal generation circuit 225 generates a selection control signal SEb that controls the selection circuit 205 to select the detection output of the digitizer 203 when the distance between the tip of the electronic pen 1 and the input surface 203S of the digitizer 203 is equal to or less than the distance Lz, and to select the detection output of the gesture detection circuit 208 when the distance is greater than the distance Lz, and controls the selection circuit 205 by the generated switching control signal SEb.

[0081] The other configurations of the tablet device 2A are the same as those shown in FIG.

[0082] <Variation 3> In the above description of the embodiment, gestures are detected only from changes in the position of the electronic pen 1 detected by the spatial position detection device unit 204. However, if the electronic pen is provided with a motion detection sensor that detects the motion of the electronic pen, and the detection output of the motion detection sensor is also sent to the tablet device, and the tablet device also refers to the detection output of the motion detection sensor, it becomes possible to reflect more precise gesture operations in the drawing.

[0083] Fig. 9 is a diagram showing an example of the configuration of the main parts of the electronic pen 1C and tablet device 2C in such a configuration. In the example of Fig. 9, the electronic pen 1C includes a wireless communication unit 102 similar to that of the modified example of Fig. 7, and also includes a 9-axis sensor 103 as its own movement detection sensor. As is well known, the 9-axis sensor 103 is a combination of a 3-axis gyro sensor, a 3-axis acceleration sensor, and a 3-axis geomagnetic sensor, and outputs 9-axis sensor output in response to the movement of the electronic pen 1C.

[0084] The electronic pen 1C transmits the nine-axis sensor output regarding its own movement detected by the nine-axis sensor 103 to the tablet device 2C via the wireless communication unit .

[0085] 7, and also includes a pen movement detection circuit 226. In the tablet device 2C, the wireless communication unit 221 receives nine-axis sensor output from the electronic pen 1C and supplies it to the pen movement detection circuit 226. The pen movement detection circuit 226 analyzes the nine-axis sensor output to detect the movement of the electronic pen 1C and supplies the detection output to the gesture detection circuit 208'. The gesture detection circuit 208' refers to the movement detection output of the electronic pen 1C from the pen movement detection circuit 226 in addition to the change in spatial position coordinates from the spatial position coordinate correction circuit 206, and detects a gesture made by the operator of the electronic pen 1C.

[0086] The other configurations of the tablet device 2C are the same as those shown in FIG.

[0087] According to the example of FIG. 9, tablet device 2C can precisely detect gestures made by the operator of electronic pen 1C, and has the effect of being able to reflect more precise gesture movements in the drawing than before.

[0088] In the above example, when detecting a gesture, tablet device 2C uses the motion detection output of electronic pen 1C detected by 9-axis sensor 103 and received from electronic pen 1C. However, it may also be used when detecting a position indicated by electronic pen 1C with digitizer 203. For example, since the tilt of electronic pen 1C can be detected from the pen motion detection output, the position indicated by electronic pen 1C can be detected by referring to the tilt detection result. Furthermore, the tilt of electronic pen 1C can be reflected in the thickness of a line drawing.

[0089] Based on a similar idea, when the position indicated by the electronic pen 1C is detected by the digitizer 203 in the tablet device 2 or tablet devices 2A to 2C, the tilts of the electronic pens 1, 1A, 1B and 1C can be detected from the position detection results of the spatial position detection device unit 204, and the position indicated by the electronic pen 1C can be detected by referring to the tilts.

[0090] The motion detection output of the pen motion detection circuit 226 of the tablet device 2C is not only used to detect gestures and the position indicated by the electronic pen, but can also be independently reflected in the drawing. Needless to say, the tablet device may be configured to detect not only drawing operations but also other input operations.

[0091] <Variation 4> In the above-described embodiment, taking into consideration the possibility of an error occurring in the position detection results of the electronic pen between the spatial coordinate system of the digitizer 203 and the spatial coordinate system of the spatial position detection device unit 204, the coordinate output of the spatial coordinate system of the spatial position detection device unit 204 is converted into the coordinate output of the spatial coordinate system of the digitizer 203 so that, for example, the display position of the cursor on the display screen 201D does not change, such as jumping, when switching between the two coordinate systems.

[0092] However, if the cursor position differs when the two coordinate systems are switched without such correction, the change in cursor position when switching can be made less noticeable by displaying the cursor so that its position gradually changes from the cursor position in the spatial coordinate system before the switch to the cursor position in the spatial coordinate system after the switch.

[0093] 10, if the cursor position on the display screen 201D before switching of the spatial coordinate system is displayed as a solid-line cursor CS, and after switching of the spatial coordinate system the cursor jumps as a dashed-line cursor CS', the display position is gradually changed from the position of the solid-line cursor CS to the position of the dashed-line cursor CS'. To do this, the gradually changing cursor position coordinate Pc is calculated from the coordinate position information Pa of the solid-line cursor CS and the coordinate position information Pb of the dashed-line cursor CS'.

[0094] That is, it is calculated using the following formula 2.

[0095]

number

[0096] In (Equation 2), k1 and k2 are weighting coefficients. That is, from the point of time when the cursor position coordinate Pc is switched to the coordinate position information Pa of the solid-line cursor CS, the value of the weighting coefficient k1 gradually decreases from "1", while the value of the weighting coefficient k2 gradually increases from "0".

[0097] Then, the value of the weighting coefficient k2 gradually decreases to "0" until the position of the cursor CS reaches the position of the dashed cursor CS', and the value of the weighting coefficient k2 gradually increases to "1".

[0098] As a result, even if the display position changes from cursor CS to cursor CS', the change is not noticeable.

[0099] Note that k1 and k2 may be variables k1(t) and k2(t) that change with time t. In this case, the value of weighting coefficient k1(t) can be changed so as to gradually decrease from "1" from the point in time when the cursor position coordinate Pc is switched to the coordinate position information Pa of the solid-line cursor CS, while the value of weighting coefficient k2(t) can be changed so as to gradually increase from "0".

[0100] <Variation 5> In the above explanation, when the position of the electronic pen becomes a state where the distance from the input surface 203S of the digitizer 203 exceeds Lz, the detection of the position indicated by the electronic pen by the digitizer 203 is immediately switched to the detection of the spatial position of the electronic pen by the spatial position detection device unit 204. However, in some cases, even if the operator moves the electronic pen to a position at a distance of Lz or more from the input surface 203S of the digitizer 203 of the tablet device, that state may not be due to a gesture input for drawing.

[0101] Therefore, spatial position detection of the electronic pen by spatial position detection device unit 204 may be started not immediately when the separation distance from input surface 203S of digitizer 203 exceeds Lz, but also when a predetermined start trigger is generated. In this case, an example of the predetermined start trigger is when the operator operates a side switch provided on the electronic pen so that the operator can operate it, and operation information of the side switch is sent to the tablet device via wireless communication unit 102.

[0102] In addition, not only when the position of the electronic pen (position of the pen tip) is moved to a position away from the input surface of the digitizer, but also when it is moved from a spatial position away from the input surface of the digitizer so that the distance to the input surface of the digitizer is within Lz, the spatial coordinate system may be switched from the spatial coordinate system of the spatial position detection device unit to the spatial coordinate system of the digitizer in accordance with user operation information, such as a side switch.

[0103] Furthermore, if the electronic pen is equipped with a 9-axis sensor, the 9-axis sensor may detect a predetermined movement of the electronic pen as a predetermined start trigger movement, and the detection result may be transmitted to the tablet device via the wireless communication unit 102. Furthermore, the spatial position detection unit 204 of the tablet device may be configured to detect a gesture based on the movement of the electronic pen, and when a gesture based on a predetermined movement of the electronic pen is detected as a predetermined start trigger movement, a processing operation for detecting a gesture to be the target of drawing processing may be started.

[0104] Furthermore, the spatial coordinate system of the spatial position detection device unit and the spatial coordinate system of the digitizer may be switched based on the movement of the electronic pen detected by the 9-axis sensor.

[0105] <Other variations> In the above-described embodiment, the tablet device is configured to include a display unit, but the display unit may be configured to be separate from the tablet device and connected to the tablet device wirelessly or via a cable. In that case, it goes without saying that the input surface 203S of the digitizer 203 of the tablet device is not the surface of the display screen of the display unit, but the upper surface of the sensor unit 2031 of the digitizer 203. Furthermore, the component on which the spatial position detection device unit 204 is disposed is not limited to the digitizer 203, and it may be provided on a display unit or other component separate from the digitizer 203.

[0106] Furthermore, the spatial position indication system of the above-described embodiment is configured as a system consisting of a tablet device and an electronic pen, with the tablet device including all of the digitizer, spatial position detection device unit, and drawing information generation unit. However, the components other than the digitizer 203 and spatial position detection device unit 204 may be configured as a personal computer, for example, and the tablet device having only the digitizer function and the spatial position detection device unit may be connected to the personal computer to form a spatial position indication system.

[0107] In the above-described embodiment, in order to correct errors between the two spatial coordinate systems, the coordinate values ​​of the spatial coordinate system of the digitizer 203 are converted into coordinate values ​​of the spatial position detection device unit 204, but conversely, the coordinate values ​​of the spatial position detection device unit 204 may be converted into coordinate values ​​of the detection area DT of the digitizer 203.

[0108] In the above example, the current mode display 220 is displayed on the display screen 201D to notify the operator whether the tablet device 2 is in tablet mode or space mode. However, it goes without saying that the current mode display 220 is not limited to the bar display shown in Fig. 6. The current mode may also be indicated by sound from a speaker provided in the tablet device 2.

[0109] [Second embodiment] The spatial position indication system according to the present invention may have a display unit configured as a head-mounted display, and the 3D rendering space may be a virtual reality (VR (Virtual Reality), MR (Mixed Reality), AR (Augmented Reality), etc., hereinafter abbreviated as VR) space. Fig. 11 is a diagram showing an outline of the overall configuration of a spatial position indication system according to a second embodiment in which the 3D rendering space is a VR space.

[0110] That is, as shown in FIG. 11, the spatial position indication system of this second embodiment is configured to include an electronic pen 10, a digitizer 20, a spatial position detection unit 30, a spatial drawing information generation device 40, and a head-mounted display (hereinafter referred to as HMD) 50.

[0111] Like the electronic pen 1 of the first embodiment, the electronic pen 10 of the second embodiment is an electromagnetic induction type electronic pen, but this embodiment is not limited thereto. The digitizer 20 has a thin rectangular parallelepiped housing 21 similar to the housing of the tablet device 2 of the first embodiment, and its surface serves as an input surface 21S. The digitizer 20 has a configuration similar to that of the digitizer 203 of the first embodiment, including a sensor unit 22 and a position detection circuit 23. The digitizer 20 has a position detection area DT' that includes not only the input surface 21S but also a hover area, which is a spatial area above the input surface 21S, similar to the position detection area DT of the digitizer 203 of the first embodiment (see the position detection area DT in FIG. 3 ). In this digitizer 20, position information of the electronic pen 10 detected in the position detection area DT' is supplied to the spatial drawing information generating device 40.

[0112] The spatial position detection unit 30 sets a motion detection target spatial area for detecting the motion of the operator of the electronic pen 10, and is configured to include two light-emitting tracking devices 31A and 31B and a plurality of light position notification units (hereinafter referred to as trackers) 32A, 32B, 32C, and 32D. In this second embodiment, as will be described later, a 3D drawing image drawn in the motion detection target spatial area is displayed as a virtual display image on the HMD 50. Hereinafter, the motion detection target spatial area in this second embodiment will be referred to as the motion detection target spatial area MDv.

[0113] The two light-emitting tracking devices 31A, 31B have the same configuration, and each includes a laser light emitting unit that emits infrared laser light, a search means that searches the motion detection target spatial region MDv using the emitted infrared laser light, and a light position detection means that detects the light emitted by the light-emitting units of the trackers 32A, 32B, 32C, 32D that receive the infrared laser light.

[0114] The trackers 32A, 32B, 32C, and 32D are equipped with sensors that detect infrared laser light from the light-emitting tracking devices 31A and 31B, and light-emitting units, such as LEDs (Light Emitting Diodes), that notify the light-emitting tracking devices 31A and 31B when the sensors detect the infrared laser light. The trackers 32A, 32B, 32C, and 32D are attached to objects whose positions are to be detected within the motion detection target spatial region MDv.

[0115] The light-emitting tracking devices 31A and 31B use a search means to control the laser emitter to emit an infrared laser beam to search and scan the motion detection target space region MDv to detect the tracker position. Each of the trackers 32A, 32B, 32C, and 32D monitors the reception of the infrared laser beam with a sensor and turns on an LED light-emitting unit when the sensor detects the reception of the infrared laser beam. The light-emitting tracking devices 31A and 31B detect the position of the object to which the tracker 32A, 32B, 32C, or 32D is attached within the motion detection target space region MDv by detecting the emission of the light-emitting unit of the tracker 32A, 32B, 32C, or 32D. The light-emitting tracking devices 31A and 31B are configured to detect the elapsed time from the time the infrared laser was emitted at the time of detection when they detect the emission of the light-emitting unit of the tracker 32A, 32B, 32C, or 32D.

[0116] The two light-emitting tracking devices 31A, 31B are connected to the spatial drawing information generating device 40 by wire or wirelessly, and notify the spatial drawing information generating device 40 of the spatial position information of the detected tracker in the motion detection target spatial region MDv.

[0117] In this embodiment, the objects to which trackers 32A, 32B, 32C, and 32D are attached are the electronic pen 10, the digitizer 20, and, in this example, as shown in FIG. 11 , a dedicated glove unit (glove unit) 60 worn by the operator, each of which is equipped with a tracker. That is, in this example, in order to be able to notify the position of the digitizer 20, trackers 32A and 32B are attached to the upper left and lower right corners of the housing of the digitizer 20, which has a thin rectangular parallelepiped shape. Furthermore, a tracker 32C is attached to the electronic pen 10 in order to notify the position and movement of the electronic pen 10. Furthermore, a tracker 32D is attached to the glove unit 60 in order to notify the position and movement of the operator's hand.

[0118] The spatial drawing information generating device 40 is configured, for example, by a personal computer, and executes processing according to a program for generating spatial drawing information. This spatial drawing information generating device 40 performs a switching process so that the coordinate values ​​of the pointing position of the electronic pen 10 can be handled seamlessly, even when the pointing position of the electronic pen 10 moves from the position detection region DT' of the digitizer 20 to the motion detection target spatial region MDv of the spatial position detection unit 30, or when it moves in the opposite direction, and its main processing function is to appropriately convert the coordinate values ​​of the pointing position of the electronic pen 10 in accordance with the switching. An overview of the main processing functions of this spatial drawing information generating device 40 will be described with reference to FIG. 12.

[0119] For example, as shown in Figure 12(A), when the position of the pen tip of the electronic pen 10 moves from a spatial area subject to motion detection outside the position detection area DT' of the digitizer 20 to within the position detection area DT' of the digitizer 20, the spatial position detection unit 30 detects the positions P1, P2, P3, etc. of the pen tip of the electronic pen 10 as position coordinates Av1, Av2, Av3, etc. in the three-dimensional coordinate space (Xv, Yv, Zv) of the spatial position detection unit 30, as shown in Figure 12(A), and supplies them to the spatial drawing information generation device 40.

[0120] Then, for positions P2 and P3 where the pen tip position of the electronic pen 10 is within the position detection area DT' of the digitizer 20, the digitizer 20 also detects the pen tip position of the electronic pen 10 as position coordinates At2, At3, ... in the three-dimensional coordinate space (X, Y, Z) of the digitizer 20, as shown in Figure 12 (B), and supplies this to the spatial drawing information generating device 40.

[0121] In the spatial drawing information generating device 40, as shown in Figure 12 (C), when the pen tip of the electronic pen 10 enters the position detection area DT' of the digitizer 20, the position information Aout is switched from the position information detected by the spatial position detection unit 30 to the position information detected by the digitizer 20 and output.

[0122] In this example, in the spatial drawing information generating device 40, the position coordinates Av1, AV2, Av3, ... of the pen tip of the electronic pen 10 detected by the spatial position detection unit 30 are converted to coordinates conv(Av1), conv(AV2), conv(Av3), ... so that they can be handled in common with the digitizer 20, as in the first embodiment described above.

[0123] Here, as in the case described above, if the directions of the Xs-axis, Ys-axis, and Zs-axis are the same as the directions of the X-axis, Y-axis, and Z-axis, the coordinate conversion process can be performed by using an offset value that takes into account the difference in the origin positions of the spatial coordinate system of the position detection region DT' of the digitizer 20 and the spatial coordinate system of the motion detection spatial region MD. Also, if the directions of the X-axis, Y-axis, and Z-axis are different from the directions of the Xs-axis, Ys-axis, and Zs-axis, the coordinate conversion can be performed by using the determinant shown in Equation 1, as in the first embodiment described above.

[0124] As a result, when the pen tip of the electronic pen 10 moves along a locus as shown in FIG. 12, the position information output Aout of the pen tip of the electronic pen 10 in the spatial drawing information generating device 40 is as follows: at position P1, the coordinate value conv(Av1) detected and converted by the spatial position detection unit 30 is used as the detection output; at positions P2 and P3, positions At2 and At3 detected by the digitizer 20 are used as the detection output; Aout=P1,P2,P3,... =conv(Av1),At2,At3,... This becomes:

[0125] Therefore, in the spatial drawing information generating device 40, the pen tip position of the electronic pen 10 can be handled seamlessly even when the movement trajectory of the pen tip of the electronic pen 10 straddles the position detection area DT' where the pen tip position of the electronic pen 10 can be detected by the digitizer 20 and the area outside the position detection area DT' where the pen tip position can be detected only by the spatial position detection unit 30.

[0126] FIG. 13 shows an example of the configuration in which the processing functions executed by the space drawing information generating device 40 in the second embodiment are shown as a functional block configuration.

[0127] In the spatial drawing information generating device 40, the detection output of the position pointed by the electronic pen 10 from the position detection circuit 23 of the digitizer 20 is supplied to a selection circuit 401 as one of its inputs.

[0128] The spatial position information of each of the trackers 32A to 32D from the two light-emitting tracking devices 31A and 31B of the spatial position detection unit 30 is supplied to a spatial position detection circuit 402 of the spatial drawing information generation device 40. The spatial position detection circuit 402 sets a spatial region of three coordinate axes (also in this example, for convenience, designated as Xs, Ys, and Zs) as a motion detection target spatial region MDv, as a spatial region in which the positions of the trackers 32A to 32D can be detected by the spatial position detection unit 30, and detects the spatial position coordinates of the three axes Xs, Ys, and Zs of the digitizer 20, the electronic pen 10, and the glove unit 60 from the spatial position information of each of the trackers 32A to 32D from the two light-emitting tracking devices 31A and 31B.

[0129] Therefore, in the spatial position indication system of the second embodiment, the entire position detection region DT' of the digitizer 20 is included within the motion detection target spatial region MDv of the spatial position detection unit 30. In this embodiment, the digitizer 20 is installed at a fixed position within the motion detection target spatial region MDv. In other words, the position detection region DT' of the digitizer 20 is included as a fixed region within the motion detection target spatial region MDv in the spatial position detection circuit 402. In other words, the motion detection target spatial region MDv of the spatial position detection circuit 402 includes the position detection region DT' of the digitizer 20 as a common region.

[0130] If the mounting positions of the trackers 32A and 32B attached to the digitizer 20 are accurate, the offset of the origin positions of the coordinate system (X, Y, Z) of the position detection area DT of the digitizer 20 and the coordinate system (Xs, Ys, Zs) of the motion detection space area MD' of the spatial position detection circuit 402 can be adjusted.

[0131] However, in this embodiment, taking into consideration the possibility of an error in accuracy occurring between the coordinate system (X, Y, Z) of the position detection area DT' of the digitizer 20 and the coordinate system (Xs, Ys, Zs) of the motion detection target spatial area MDv of the spatial position detection circuit 402, the spatial position detection circuit 402 has the function of the spatial position coordinate correction circuit 206 described in the first embodiment. Also in this second embodiment, the component a of the determinant shown in FIG. 5, which is calculated in the same manner as the correction information memory 207 in the first embodiment, 11 ~a 33 A correction information memory 403 is provided to store the above.

[0132] Then, the spatial position detection circuit 402 calculates the component a of the determinant from the correction information memory 403. 11 ~a 33 The determinant of (Equation 1) is calculated using the above equation, and the spatial position coordinate outputs of the trackers 32A to 32D are supplied to the gesture detection circuit 404.

[0133] The gesture detection circuit 404 detects the movement (gesture) of the electronic pen 10 and the movement of the glove unit 60 based on the coordinate output from the spatial position detection circuit 402, and supplies the detection output to the selection circuit 401 as the other input signal.

[0134] In the second embodiment, similarly to the first embodiment, the position detection circuit 23 of the digitizer 20 supplies information on the signal level of the signal received from the electronic pen 10 to the separation distance detection circuit 405 of the spatial drawing information generation device 40. The separation distance detection circuit 405 detects the separation distance between the pen tip of the electronic pen 10 and the input surface 21S of the digitizer 20 from the signal level of the signal received from the electronic pen 10. The separation distance detection circuit 405 then supplies information on the detected separation distance to the selection control signal generation circuit 406.

[0135] The selection control signal generation circuit 406 generates a selection control signal SE' that controls the selection circuit 401 to select the detection output of the position detection circuit 23 of the digitizer 20 when the separation distance of the pen tip of the electronic pen 10 from the input surface 21S of the digitizer 20 is equal to or less than the distance Lz at which the above-mentioned hover state can be detected, and controls the selection circuit 401 to select the detection output of the gesture detection circuit 404 when the separation distance is greater than the distance Lz, and controls the selection circuit 401 with the generated switching control signal SE'. The selection control signal generation circuit 406 also supplies the switching control signal SE' to the drawing information generation circuit 407.

[0136] The selection circuit 401 selects one of the inputs in response to a selection control signal SW′, and supplies the selected input to the drawing information generation circuit 407 .

[0137] The drawing information generation circuit 407 generates drawing information in accordance with operations on the electronic pen 10 by the operator of the electronic pen 10 for drawing within the motion detection target spatial region MDv. In this second embodiment, operations on the electronic pen 10 by the operator of the electronic pen 10 for drawing include an operation of positioning the electronic pen 10 within the position detection region DT' of the digitizer 20 and inputting a designated position with the electronic pen 10, and an operation of positioning the electronic pen 10 outside the position detection region DT' of the digitizer 20 within the motion detection target spatial region MDv and moving the electronic pen 1 to make a gesture for drawing.

[0138] When the electronic pen 10 is located within the position detection region DT' of the digitizer 20, this is detected by the separation distance detection circuit 405, and based on the detection output, the selection control signal generation circuit 406 generates a selection control signal SE' that controls the selection circuit 401 to select the detection output of the position detection circuit 23 of the digitizer 20. Therefore, the detection output of the position indicated by the electronic pen 10 from the position detection circuit 23 of the digitizer 20 is supplied to the drawing information generation circuit 407 via the selection circuit 401.

[0139] The drawing information generating circuit 407 performs drawing processing on the received output of the detection of the position pointed to by the electronic pen 10 as a line drawing input, thereby generating a fine 3D drawn image.

[0140] Furthermore, when the electronic pen 1 is located outside the position detection region DT' of the digitizer 20 in the motion detection target spatial region MDv, this is detected by the separation distance detection circuit 405, and based on the detection output, the switching signal generation circuit 406 generates a selection control signal SE' that controls the selection circuit 401 to select the detection output of the gesture detection circuit 404. Therefore, gesture information based on the motion detection output of the electronic pen 10 and the glove unit 60 from the gesture detection circuit 404 is supplied to the drawing information generation circuit 407 via the selection circuit 401.

[0141] The drawing information generation circuit 407 performs 3D drawing processing based on the received gesture information, that is, the drawing processing is performed in accordance with the movement of the electronic pen 10 detected by the gesture detection circuit 404 and the movement of the glove unit 60 worn by the operator.

[0142] In this case, the drawing information generation circuit 407 determines whether to perform drawing processing based on the detection output of the pointing position by the electronic pen 10 as line drawing input or based on gesture information, depending on the selection control signal SE' of the selection circuit 401.

[0143] The drawing image information generated by the drawing information generating circuit 407 in the above manner is supplied to the HMD 50 via the display drive circuit 408 and displayed as a virtual image on a display screen made of, for example, an LCD.

[0144] Therefore, the operator can seamlessly switch from fine drawing to gesture drawing simply by moving the electronic pen 10 spatially, without being aware of switching between position detection by the digitizer 20 and position detection (motion detection) by the spatial position detection unit 30, which is a remarkable effect.

[0145] In this case, in the second embodiment, the drawing information generation circuit 407 includes a unit for generating a virtual space image to be displayed on the HMD 50, and can draw an image drawn by the operator of the electronic pen 10 on this virtual space image. The position of a virtual object included in this virtual space image within the motion detection target space region MDv can be determined in advance. The drawing information generation circuit 407 can detect, based on gesture information from the gesture detection circuit 404, when the operator touches a virtual object included in the virtual space image with the glove unit 60. When the operator touches a virtual object included in the virtual space image with the glove unit 60, the drawing information generation circuit 407 outputs a detection output to the tactile sensation occurrence information generation unit 409.

[0146] When the tactile sensation generation information generating unit 409 receives from the drawing information generating circuit 407 a detection output of the operator's touching operation on a virtual object included in the virtual space image, it supplies tactile sensation generation information that generates a tactile sensation to the glove unit 60. The glove unit 60 is provided with a tactile sensation generation device that vibrates based on this tactile sensation generation information, and when the tactile sensation generation device is driven, the operator is notified that he or she has virtually touched the virtual object.

[0147] The drawing information generation circuit 407 generates drawing image information to be output so that the display screen of the HMD 50 displays, depending on the distance between the electronic pen 10 and the input surface 21S of the digitizer 20, whether the indicated position of the electronic pen 10 is detected by the digitizer 20 or the spatial position of the electronic pen 10 is detected by the spatial position detection unit 30. Therefore, the operator of the electronic pen 1 can accurately grasp what drawing operation should be performed at the current spatial position of the electronic pen 10.

[0148] Note that the above-described modified examples of the first embodiment are also applicable to this second embodiment. In the modified example 2, it goes without saying that the spatial position detection unit 30 and the spatial position detection circuit 402 are used instead of the spatial position detection device section 204 in the second embodiment.

[0149] [Modification of the second embodiment] In the second embodiment described above, the spatial position detection unit 30 is configured to include a light-emitting tracking device that emits infrared laser light and a tracker, but it goes without saying that the configuration is not limited to this. For example, it may be configured to use other non-visible light sensors, visible light sensors, or a combination of these.

[0150] Alternatively, the electronic pen may be provided with a battery and a radio wave transmitting / receiving means, and the spatial position of the electronic pen may be determined by transmitting radio waves to the electronic pen from an external device and receiving the radio waves from the electronic pen. Magnetic resonance or ultrasonic waves may also be used. Furthermore, the object (electronic pen or tablet) whose spatial position is to be detected may be photographed with one or more cameras, and the spatial position of the object may be detected using the photographed image.

[0151] In the second embodiment described above, since the HMD 50 is worn, the operator of the electronic pen 10 cannot directly see the digitizer 20. Therefore, a virtual image of the digitizer 20 can be drawn on a virtual space image displayed on the HMD 50, and the operator can recognize the position of the digitizer 20 on the display screen of the HMD 50. Furthermore, the HMD may of course be configured to be AR (augmented reality) compatible, allowing the operator to directly see the digitizer 20. Furthermore, although the glove unit 60 and the electronic pen 10 are configured separately, the glove unit 60 and the electronic pen 10 may be configured as a combined unit.

[0152] Furthermore, the means for displaying a three-dimensional image is not limited to an HMD, but may also use a 3D display, an AI (Aerial Imaging) plate, or even hologram technology. Furthermore, a display such as an LCD may be provided on the digitizer 20, and 3D display may be performed on that display. These display means may be used together with an HMD to allow a person other than the HMD wearer to view the rendered image displayed on the HMD.

[0153] In the second embodiment, the coordinate conversion is performed by converting the coordinate values ​​of the electronic pen tip detected by the spatial position detection unit 30 into coordinate values ​​in the coordinate system of the digitizer 20, but conversely, the coordinate values ​​of the electronic pen tip detected by the digitizer 20 may also be converted into coordinate values ​​in the coordinate system of the spatial position detection unit 30.

[0154] Furthermore, in the above example, the distance between the pen tip position of the electronic pen 10 and the input surface of the digitizer 20 is detected based on the reception level of the signal between the sensor unit 22 of the digitizer 20 and the electronic pen 10, but the method of detecting the distance between the pen tip position of the electronic pen 10 and the input surface of the digitizer 20 is not limited to this.

[0155] For example, in this second embodiment, the spatial position detection unit 30 can detect the pen tip position of the electronic pen and the position of the input surface of the digitizer 20, so that the distance between the pen tip position of the electronic pen 10 and the input surface of the digitizer 20 can be detected from the detected pen tip position of the electronic pen 10 and the position of the input surface of the digitizer 20, and a selection control signal for the selection circuit 401 can be generated.

[0156] [Other embodiments or modifications] In the above-described embodiment, gestures based on the movement of the electronic pen are detected in the spatial area outside the position detection area DT or DT' of the digitizer as an example of a pointing position detection device unit. However, it is also possible to detect the pointing position by the electronic pen based on the movement of the tip of the electronic pen, rather than the movement of the electronic pen, in the outer spatial area.

[0157] In the above-described embodiment, when the position of the electronic pen moves from within the position detection area DT or DT' of the digitizer, which is an example of an indication position detection device unit, to an outer spatial area, the drawing processing circuit 212 and the drawing information generation circuit 407 immediately execute drawing processing in accordance with a gesture based on motion detection in the motion detection spatial area MD or the motion detection target spatial area MDv. However, the operator does not necessarily move the electronic pen from within the position detection area DT or DT' of the digitizer to an outer spatial area in order to perform a gesture for drawing, and may move it in this way simply to pause input of a drawing operation.

[0158] Taking this into consideration, when the position of the electronic pen moves from within the position detection area DT or DT' of the digitizer, which is an example of an indication position detection device unit, to an outer spatial area, when a further specified trigger event is detected, the drawing processing circuit 212 and the drawing information generation circuit 407 can be configured to perform drawing processing according to a gesture based on motion detection in the motion detection spatial area MD or the motion detection target spatial area MDv.

[0159] In this case, the selection circuit 205 or the selection circuit 401 may perform selection control when the position of the electronic pen moves from within the position detection region DT or DT' of the digitizer, which is an example of an indication position detection device unit, to a spatial region outside of that region, or may perform selection control when a further trigger event is detected. Of course, when the position of the electronic pen moves from a spatial region outside of that region into the position detection region DT or DT' of the digitizer, which is an example of an indication position detection device unit, the position of the electronic pen 1 is immediately detected by the digitizer 203 or the digitizer 20.

[0160] That is, for example, only when the operator moves the position of the electronic pen from within the position detection area DT or DT' of the digitizer, which is an example of an indication position detection device unit, to an outer spatial area, and also operates the side switch mounted on the electronic pen as a predetermined trigger event, the drawing processing circuit 212 and the drawing information generation circuit 407 are configured to execute drawing processing according to a gesture based on motion detection in the motion detection spatial area MD or the motion detection target spatial area MDv.

[0161] The predetermined trigger event is not limited to the operation of a side switch. For example, it may be a specific gesture by the operator of the electronic pen. Alternatively, it may be a voice instruction by the operator. When a voice instruction is used as the predetermined trigger event, a microphone is provided in the tablet device 2 in the first embodiment, and a microphone is provided in the spatial drawing information generation device in the second embodiment, and the voice picked up by the microphone is subjected to voice recognition to determine whether or not it is a specific trigger event by the operator. Alternatively, a microphone may be provided in the electronic pen, and the picked-up voice information may be transmitted via a wireless communication unit installed in the electronic pen, and received by a wireless communication unit installed in the tablet device 2 or the spatial drawing information generation device for voice recognition.

[0162] In the first and second embodiments described above, the electronic pen and digitizer are of the electromagnetic induction type, but this is not limited to this, and it is of course also possible to use an electronic pen and digitizer of the electrostatic type (including the active electrostatic coupling type and the passive electrostatic coupling type).

[0163] Furthermore, the tablet device 2 of the first embodiment and the digitizer of the second embodiment may be a portable mobile phone terminal known as a smartphone, or may be a personal computer with a digitizer.

[0164] In the above description, a case where 3D drawing is performed using a spatial position instruction system has been described, but the drawn image that is the subject of the present invention may be a 2D drawn image or a 2.5D drawn image. [Explanation of symbols]

[0165] 1, 1A, 1C, 10...electronic pen, 2...tablet device, 20...digitizer, 30...spatial position detection unit, 40...spatial drawing information generation device, 50...head-mounted display, 201...LCD, 202...housing of tablet device 2, 203...digitizer, 204...spatial position detection device section, 205, 401...selection circuit, 206...spatial position coordinate correction circuit, 207...correction information memory, 208, 404...gesture detection circuit, 210, 405...separation distance detection circuit, 211, 406...selection control signal generation circuit, 213...drawing processing circuit, 407...drawing information generation circuit

Claims

1. a tablet that detects the position of the tip of the electronic pen; a tracking device that detects the position of a tracker of the electronic pen, the tracking device being used to input a position in a virtual reality (VR) space using the electronic pen, and that is positioned above the tablet; a computer that, in a first mode, calculates a first coordinate in a first coordinate system of the VR (Virtual Reality) space based on the position of the tracker detected by the tracking device, and, in a second mode switched from the first mode in response to a predetermined operation, calculates a second coordinate in a second coordinate system of the tablet, which is different from the first coordinate system, based on the position of the pen tip detected by the tablet.

2. the second coordinate system of the tablet is a coordinate system related to a position detection area of ​​the tablet, and the first coordinate system of the VR (Virtual Reality) space is a coordinate system related to a position detection area of ​​the tracking device; 2. The coordinate calculation system according to claim 1.

3. The computer displays a virtual image in the VR (Virtual Reality) space on a display unit.

2. The coordinate calculation system according to claim 1.

4. the computer calculates the second coordinates in a second coordinate system of the tablet when the pen tip of the electronic pen is located within a predetermined distance from the input surface of the tablet; 2. The coordinate calculation system according to claim 1.

5. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light.

2. The coordinate calculation system according to claim 1.

6. The VR (Virtual Reality) space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space.

2. The coordinate calculation system according to claim 1.

7. the computer calculates third coordinates in a first coordinate system of the VR (Virtual Reality) space based on the second coordinates in a second coordinate system of the tablet; 2. The coordinate calculation system according to claim 1.

8. the computer corrects the second coordinates in the second coordinate system of the tablet and calculates the third coordinates in the first coordinate system of the VR (Virtual Reality) space; 8. The coordinate calculation system according to claim 7.

9. the computer causes a display unit to display a virtual image in the VR (Virtual Reality) space based on the first coordinates and the third coordinates in a first coordinate system of the VR (Virtual Reality) space; 8. The coordinate calculation system according to claim 7.

10. In a first mode, a first coordinate in a first coordinate system of a VR (Virtual Reality) space is calculated based on the position of a tracker of the electronic pen detected by a tracking device, and in a second mode switched from the first mode in response to a predetermined operation, a second coordinate in a second coordinate system of the tablet, which is different from the first coordinate system, is calculated based on the position of the pen tip of the electronic pen detected by a tablet. A coordinate calculation method characterized by:

11. the second coordinate system of the tablet is a coordinate system related to a position detection area of ​​the tablet, and the first coordinate system of the VR (Virtual Reality) space is a coordinate system related to a position detection area of ​​the tracking device; The coordinate calculation method according to claim 10 .

12. displaying a virtual image corresponding to the tablet in the VR (Virtual Reality) space on a display unit; The coordinate calculation method according to claim 10 .

13. calculating second coordinates of the tablet in the second coordinate system when the pen tip of the electronic pen is located within a predetermined distance from the input surface of the tablet; The coordinate calculation method according to claim 10 .

14. The tracker of the electronic pen has an LED (Light Emitting Diode) that emits light. The coordinate calculation method according to claim 10 .

15. The VR (Virtual Reality) space includes an MR (Mixed Reality) space and an AR (Augmented Reality) space. The coordinate calculation method according to claim 10 .

16. calculating third coordinates in a first coordinate system of the VR (Virtual Reality) space based on the second coordinates in a second coordinate system of the tablet; The coordinate calculation method according to claim 10 .

17. correcting the second coordinates in the second coordinate system of the tablet to calculate the third coordinates in a first coordinate system of the VR (Virtual Reality) space; 17. The coordinate calculation method according to claim 16.

18. displaying a virtual image in the VR (Virtual Reality) space on a display unit based on the first coordinates and the third coordinates in a first coordinate system of the VR (Virtual Reality) space; 17. The coordinate calculation method according to claim 16.

Images