Controller and Tracking System

Abstract

[Problem] To provide a controller which can highly accurately detect position and direction. [Solution] A controller comprising: a pen unit formed in a pen-shape; a grip unit that intersects an axial direction of the pen unit; and a first light-emitting unit disposed on a surface of an end portion of the grip unit, the end portion being closer to the axial direction of the pen unit.

Description

[Technical Field]

[0001] The present invention relates to a controller and a tracking system, and in particular to a controller used in a space (hereinafter referred to as "XR space") created using XR technologies such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality), and a tracking system for tracking the movement of such a controller. [Background technology]

[0002] In XR technology, a handheld controller is used by a user to indicate a position within an XR space. The tracking of the controller is performed by a tracking system including a camera and a computer connected to the camera. When the user moves the controller within the camera's range, the computer detects the position and orientation of the controller based on the image captured by the camera, and tracks the movement of the controller based on the detection results.

[0003] Patent Document 1 discloses an example of a pen-type controller, which is a type of handheld controller. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 225170 Summary of the Invention [Problem to be solved by the invention]

[0005] The controller disclosed in Patent Document 1 has a plurality of light emitting diodes (hereinafter referred to as "LEDs") on its surface. A computer that tracks the movement of the controller is configured to detect the position and orientation of the controller by detecting these LEDs in the image captured by the camera.

[0006] However, if the LED is obscured by the user's hand or the like, it becomes difficult to detect the position and orientation of the controller with high accuracy.

[0007] Therefore, one object of the present invention is to provide a pen-type controller that can detect the position and orientation with high accuracy.

[0008] Furthermore, in the above-mentioned conventional tracking system, since distortion in the image captured by the camera reduces the accuracy of detecting the position and orientation of the controller, a camera capable of capturing images with minimal distortion was used. Specifically, a camera with a global shutter that can scan each row of the image sensor at once was used.

[0009] However, cameras with global shutters are generally expensive, which increases the cost of the entire tracking system. Therefore, there is a need for a technology that can detect the position and orientation of a controller with high accuracy while using a camera with a rolling shutter, which is cheaper than a global shutter.

[0010] Therefore, one object of the present invention is to provide a tracking system that is capable of detecting the position and orientation of a controller with high accuracy even when using a camera with a rolling shutter. [Means for solving the problem]

[0011] The controller according to the present invention has a pen-shaped pen portion, a grip portion that intersects with the axial direction of the pen portion, and a first light-emitting portion that is disposed on the surface of the end of the grip portion that is closest to the axial direction of the pen portion.

[0012] The tracking system according to the present invention is a tracking system for tracking the movement of the controller, and includes one or more cameras, each having a rolling shutter and arranged so that the sub-scanning direction of the rolling shutter coincides with the vertical direction, and a computer that tracks the movement of the controller based on images captured by the one or more cameras. [Effects of the Invention]

[0013] The controller according to the present invention makes it possible to detect the position and orientation of the controller with high accuracy.

[0014] The tracking system according to the present invention makes it possible to detect the position and orientation of the controller with high accuracy using a rolling shutter. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram showing a state in which a tracking system 1 according to an embodiment of the present invention is used; [Figure 2] 1A and 1B are diagrams illustrating a state in which a pen-type controller according to an embodiment of the present invention is used; [Figure 3] 1A to 1C are perspective views of a controller 6 viewed from various angles. [Figure 4] 1A to 1C are perspective views of a controller 6 viewed from various angles. [Figure 5] 10A and 10B are diagrams illustrating the rotation of the controller 6 around the pen axis. [Figure 6] 10A and 10B are diagrams illustrating the rotation of the controller 6 around the pen axis. [Figure 7]10A and 10B are diagrams showing images taken by cameras 4b and 4c capturing a controller 6 according to an embodiment of the present invention. [Figure 8] FIG. 1 is a diagram illustrating the arrangement of cameras 4a to 4c. [Figure 9] 1(a) is a diagram showing the image sensor 40 built into each of the cameras 4a to 4c, and FIG. 1(b) is a diagram illustrating the operation of the rolling shutter. [Figure 10] FIG. 10 is a diagram showing the structure of cameras 4a to 4c employed to arrange cameras 4a to 4c so that the sub-scanning direction of the rolling shutter coincides with the vertical direction. [Figure 11] FIG. 10 is a diagram showing a state in which a tracking system 1 according to a modified example of the embodiment of the present invention is used. [Figure 12] FIG. 10 is a diagram illustrating a controller 6 according to a modified example of the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0017] 1 is a diagram showing a state in which a tracking system 1 according to this embodiment is in use. As shown in the figure, the tracking system 1 is configured to have a computer 2, a position detection device 3, three cameras 4a to 4c, a head-mounted display 5, and a pen-shaped controller 6. The computer 2, the position detection device 3, the cameras 4a to 4c, the head-mounted display 5, and the controller 6 are each configured to be able to communicate with each other via wired or wireless communication.

[0018] As shown in FIG. 1, a user uses the tracking system 1 while sitting in a desk chair 101, wearing a head-mounted display 5 on his / her head, and holding a controller 6 in his / her right hand. An XR space rendered by a computer 2 is displayed on the display surface of the head-mounted display 5, and the user operates the controller 6 above the desk 100 while viewing this XR space. The controller 6 is a pen-shaped device with a grip attached, and controls 3D objects displayed in the XR space (specifically, drawing 3D objects, moving 3D objects, etc.). Furthermore, the controller 6 is used to perform 2D input using the position detection device 3.

[0019] 1, the computer 2 is configured as a notebook personal computer placed in the center of the desk 100. However, the computer 2 does not necessarily have to be placed in the center of the desk 100, and may be placed in a position where it can communicate with the position detection device 3, the cameras 4a to 4c, the head-mounted display 5, and the controller 6. Furthermore, the computer 2 may be configured as various types of computers other than a notebook personal computer, such as a desktop personal computer, a tablet personal computer, a smartphone, or a server computer.

[0020] The computer 2 serves to track the movements of the head-mounted display 5, controller 6, and position detection device 3 by periodically detecting the positions and tilts of these devices based on images captured by the cameras 4a to 4c. The computer 2 generates an XR space and 3D objects to be displayed therein based on the movements of each device being tracked and the operation states of each operation button and dial button (described below) provided on the controller 6, and performs processing to render the generated XR space and 3D objects and send them to the head-mounted display 5. The head-mounted display 5 serves to display the XR space including one or more 3D objects by displaying the rendered images sent from the computer 2.

[0021] 1, the position detection device 3 is configured as a tablet placed on the top surface of the desk 100 in a position corresponding to the front side of the computer 2 as seen from the user. However, the position detection device 3 does not necessarily have to be placed in this position, and may be placed within reach of the user sitting in the desk chair 101. The position detection device 3 and the computer 2 may also be configured as an integrated device, such as a tablet terminal.

[0022] The position detection device 3 has the function of periodically detecting the position of the pen tip of the controller 6 on the touch surface and sequentially transmitting the detected positions to the computer 2. Based on the transmitted positions, the computer 2 generates and renders stroke data that constitutes a 2D or 3D object. There are no particular limitations on the specific method of position detection by the position detection device 3, but it is preferable to use, for example, an active electrostatic method or an electrostatic induction method.

[0023] Each of the cameras 4a to 4c is an imaging device for capturing still or moving images, and is configured to sequentially supply the captured images to the computer 2. Camera 4a is positioned opposite the user across the desk 100, camera 4b is positioned above the user's left side, and camera 4c is positioned above the user's right side, each oriented so as to capture an image of the top surface of the desk 100. Each of the cameras 4a to 4c is a camera with a rolling shutter, and is positioned so that the sub-scanning direction of the rolling shutter coincides with the vertical direction in order to minimize distortion of the controller 6 in the image. This point will be described in detail later.

[0024] FIG. 2 is a diagram showing a state in which a user is holding the controller 6 in his / her right hand. Also, FIGS. 3(a), 3(b), 4(a), and 4(c) are perspective views of the controller 6 viewed from various angles. As shown in these figures, the controller 6 is configured to include a pen-shaped pen section 6p and a grip section 6g fixed to the pen section 6p so that its longitudinal direction intersects with the axial direction of the pen section 6p. Hereinafter, the axial direction of the pen section 6p will be referred to as the x-direction, a direction in a plane formed by the x-direction and the longitudinal direction of the grip section 6g and perpendicular to the x-direction will be referred to as the z-direction, and a direction perpendicular to both the x-direction and the z-direction will be referred to as the y-direction.

[0025] As shown in FIG. 3(a), the surface of the pen unit 6p is provided with pressure pads 6pa and 6pb and shift buttons 6pc and 6pd. The pressure pads 6pa and 6pb are components including a pressure sensor and a touch sensor, and are located near the pen tip on the side of the pen unit 6p, symmetrically with respect to the xz plane. The pressure detected by the pressure sensor is used for selection or drawing in an application. Meanwhile, information indicating the presence or absence of a touch detected by the touch sensor is used to determine whether the pressure sensor output is on or off and to realize a light double tap. The shift buttons 6pc and 6pd are switches assigned to application menus, and are located between the pressure pads 6pa and 6pb and the grip unit 6g, symmetrically with respect to the xz plane. As can be seen from FIG. 2, a user holding the controller 6 in their right hand operates the pressure pad 6pa and shift button 6pc with their thumb and the pressure pad 6pb and shift button 6pd with their index finger.

[0026] As shown in FIGS. 3(a)(b) and 4(a)(b), the surface of the grip portion 6g is provided with a tactile top button 6ga, a grab button 6gb, tactile buttons 6gc and 6gd, a dial button 6ge, and a recess 6gf. The tactile top button 6ga is a switch that functions as a power button when pressed and held. It is located on the surface of one of the longitudinal ends of the grip portion 6g, closer to the pen unit 6p. Hereinafter, this end will be referred to as the "upper end," and the other of the longitudinal ends of the grip portion 6g, farther from the pen unit 6p, will be referred to as the "lower end." The dial button 6ge is a rotatable ring-shaped member that outputs a rotation amount. This rotation amount is used, for example, to rotate a selected object. The dial button 6ge is also located on the upper end of the grip portion 6g, surrounding the tactile top button 6ga.

[0027] The grab button 6ga is a switch used to grab and move an object, and is located near the lower end of the pen tip side of the grip portion 6g. The tactile buttons 6gc and 6gd are switches used as auxiliary buttons like the right button on a mouse, and are located near the pen portion 6p when viewed in the z direction on the pen tip side of the grip portion 6g. The tactile button 6gc is located on the thumb side when holding the controller 6 in the right hand, and the tactile button 6gd is located on the index finger side when holding the controller 6 in the right hand.

[0028] As can be seen from FIG. 2, a user holding the controller 6 in their right hand will press the grab button 6ga with their middle finger. They will also press the tact button 6gc with their thumb and the tact button 6gd with their index finger. The rotation of the dial button 6ge and the pressing of the tact top button 6ga are performed with the user's thumb. However, the tact top button 6ga and the dial button 6ge are located in positions that cannot be operated unless the user intentionally lifts their thumb to the upper end of the grip portion 6g, and therefore are exposed and not hidden by the user's hand in the normal state.

[0029] 2, the recess 6gf is configured to fit snugly between the base of the index finger and the base of the thumb when the user holds the controller 6. Providing this recess 6gf in the controller 6 reduces fatigue experienced by the user when using the controller 6.

[0030] One or more LEDs are arranged on the surfaces of the pen portion 6p and the grip portion 6g. In this embodiment, each LED is configured as a so-called point light source LED. The computer 2 is configured to detect the position and orientation of the controller 6 by detecting these LEDs from the images captured by the cameras 4a to 4c.

[0031] Specifically, the arrangement of the LEDs is described as follows: one or more LEDs are arranged in each of three sections PT1 to PT3 of the controller 6 shown in Fig. 3(b). Section PT1 is the section of the pen unit 6p located on the pen tip side as viewed from the grip section 6g, section PT2 is the section of the pen unit 6p located on the pen rear side as viewed from the grip section 6g, and section PT3 is the grip section 6g. In the illustrated example, two LEDs 10a-1 and 10a-2 are arranged in section PT1, four LEDs 10b-1 to 10b-4 are arranged in section PT2, and one LED 10c is arranged in section PT3.

[0032] The two LEDs 10a-1 and 10a-2 corresponding to portion PT1 are arranged side by side in the same position as viewed in the x direction, slightly closer to the grip portion 6g than the pen tip. Of the four LEDs 10b-1 to 10b-4 corresponding to portion PT2, LED 10b-4 is arranged at the end of the pen. Meanwhile, the other three LEDs 10b-1 to 10b-3 are arranged in a zigzag pattern from the grip portion 6g to the end of the pen. That is, LED 10b-1 and LED 10b-3 are located closer to the right side of portion PT2, and LED 10b-2 is located closer to the left side of portion PT2.

[0033] The LED 10c corresponding to portion PT3 is disposed on the surface of the upper end of the grip portion 6g (more specifically, on the surface of the tactile top button 6ga). As described above, when the user is not operating the tactile top button 6ga, the tactile top button 6ga is exposed and not hidden by the user's hand. Therefore, by providing the LED 10c on the surface of this tactile top button 6ga, the computer 2 can constantly detect the controller 6 with a high probability, and therefore, it becomes possible to detect the position and orientation of the controller 6 with high accuracy. Furthermore, by not disposing the LED below the grip portion 6g, the LED pattern in the image is simplified, making it easier for the computer 2 to recognize the shape.

[0034] Furthermore, by providing the LED 10c on the surface of the upper end of the grip portion 6g, it is possible to obtain the effect of accurately detecting the rotation of the controller 6 around the pen axis. This point will be described in detail below with reference to FIGS. 5 and 6.

[0035] Figures 5 and 6 are diagrams showing rotation of the controller 6 around the pen axis. Figures 5(a) and (c) and 6(a) and (c) are perspective views of the controller 6 seen from the pen tip side, respectively, and show the state in which the controller 6 is rotated around the pen axis (x direction) in the order of Figures 5(a), 5(c), 6(a), and 6(c). Figures 5(b) and (d) and 6(b) and (d) are perspective views of the controller 6 seen from the left side, and correspond to Figures 5(a) and (c) and 6(a) and (c), respectively.

[0036] When the controller 6 is in the state shown in Figures 5(a) and 5(b), the image captured by the camera 4b on the left side of the controller 6 captures the LED 10b-2 in the PT2 portion of the pen unit 6p, but not the LED 10c on the surface of the tactile top button 6ga. However, as the controller 6 rotates around the pen axis, the LED 10c also appears in the image. The distance Lz in the z direction between the LED 10c and the LED 10b-2 captured in the image decreases as the controller 6 rotates, as shown in Figures 5(c) and 5(d), 6(a) and 6(b), and 6(c) and 6(d). Meanwhile, the distance Lx in the x direction between the LED 10c and the LED 10b-2 remains constant. Therefore, the computer 2 can derive the rotation angle of the controller 6 around the pen axis based on information such as Lz, Lx, and the distances and angles to other LEDs.

[0037] Returning to FIG. 3, the controller 6 according to this embodiment has LEDs 10a-1 and 10a-2 in the portion PT1 on the pen tip side. This allows the center of gravity of the coordinates derived by the computer 2 to be closer to the pen tip side than when no LEDs are provided in the portion PT1. Therefore, also from this point of view, it can be said that the use of the controller 6 according to this embodiment makes it possible to detect the position and orientation of the controller 6 with high accuracy.

[0038] Furthermore, LEDs 10a-1 and 10a-2 provided in portion PT1 are arranged asymmetrically with respect to the xz plane including the axial direction of the pen unit 6p and the longitudinal direction of the grip unit 6g. Similarly, LEDs 10b-1, 10b-2, and 10b-3 provided in portion PT2 are also arranged asymmetrically with respect to the xz plane including the axial direction of the pen unit 6p and the longitudinal direction of the grip unit 6g. That is, as described above, the three LEDs 10b1, 10b-2, and 10b-3 are arranged in a zigzag pattern from the grip unit 6g to the end of the pen. This allows the computer 2 to distinguish between the left and right sides of the controller 6.

[0039] 7 shows images captured by cameras 4b and 4c of controller 6 according to this embodiment. Each bright dot in the image corresponds to an LED on the surface of controller 6. As shown in the figure, when controller 6 according to this embodiment is used, there is a clear difference in the arrangement of the LEDs between the image captured by camera 4b, which captures controller 6 from the left side, and the image captured by camera 4c, which captures controller 6 from the right side. Therefore, computer 2 can distinguish between the left and right sides of controller 6 from the images captured by cameras 4b and 4c.

[0040] 3(a), the LEDs are concentrated on the right side of the controller 6. The controller 6 is often used while being held in the right hand, in which case the left side of the controller 6 tends to be in the blind spot of the cameras 4a to 4c, so by doing so, the LEDs can be more easily captured in the images captured by the cameras 4a to 4c when the controller 6 is used while being held in the right hand.

[0041] Next, a preferred arrangement of the cameras 4a to 4c will be described. FIGS. 8(a) to 8(c) are diagrams for explaining the arrangement of the cameras 4a to 4c. The desk 100 and desk chair 101 shown in FIGS. 8(a) and 8(c) are the same as those shown in FIG. 1, and a user uses the tracking system 1 while sitting in the desk chair 101 facing the computer 2 on the desk 100. FIG. 8(b) is a cross-sectional view of the cameras 4b and 4c corresponding to line AA in FIG. 8(a). Hereinafter, the direction from camera 4b toward camera 4c will be referred to as the X direction, the direction from the user toward camera 4a will be referred to as the Y direction, and the vertical direction will be referred to as the Z direction.

[0042] The illustrated position P1 is the position of the head-mounted display 5 shown in Fig. 1, and the two positions P2 are the positions of the user's shoulders. The cameras 4a to 4c are positioned so that they can capture the entire portion of the substantially sector-shaped area E that extends from these positions toward the computer 2 and is located on the desk 100. Specifically, the arrangement of cameras 4a to 4c is determined by calculating the distance X1 in the X direction from area E to each of cameras 4b and 4c, the distance Y1 in the Y direction from the rear end of area E to each of cameras 4b and 4c, the distance Y2 in the Y direction from the rear end of area E to camera 4a, the distance Z1 from the floor surface to cameras 4b and 4c, the distance Z2 from the floor surface to camera 4a, the angle φ1 that the shooting direction of each of cameras 4c and 4b makes with the X direction in the XY plane, and the angle φ2 that the shooting direction of each of cameras 4c and 4b makes with the X direction in the XZ plane, based on the size, shape, and position of area E defined by lengths D1 to D4 and angle θ1 shown in FIG. 8(a), and lengths D6 to D11 and angles θ2 and θ3 shown in FIG. 8(c), the distance from the rear end of area E to the front end of desk 100 defined by length D5 shown in FIG. 8(a), and the height of desk 100 defined by length D12 shown in FIG. By determining the placement of the cameras 4a to 4c in this manner, the computer 2 can preferably detect the position and inclination of each of the head-mounted display 5, the controller 6, and the position detection device 3 based on the images captured by the cameras 4a to 4c.

[0043] Next, the structure of the cameras 4a to 4c employed to arrange the cameras 4a to 4c so that the sub-scanning direction of the rolling shutter coincides with the vertical direction will be described. Below, the rolling shutter will be described first with reference to Fig. 9, and then the structure of the cameras 4a to 4c according to this embodiment will be specifically described with reference to Figs. 10 and 11.

[0044] FIG. 9(a) is a diagram showing the image sensor 40 built into each of the cameras 4a to 4c. Each square shown in the figure represents a pixel, and as shown in the figure, the image sensor 40 is composed of a pixel matrix in which a plurality of pixels are arranged in a matrix. In the following, the number of rows in this pixel matrix is ​​referred to as N. The row direction of the pixel matrix is ​​also referred to as the "main scanning direction," and the column direction is also referred to as the "sub-scanning direction."

[0045] Figure 9(b) is a diagram explaining the operation of the rolling shutter. The horizontal axis of the figure represents time, and the vertical axis represents the main scanning direction of the pixel matrix. The horizontally long rectangles shown in the figure represent the time it takes to scan multiple pixels in one row in the sub-scanning direction.

[0046] As shown in Figure 9(b), cameras 4a-4c with rolling shutters are configured to scan (expose and read) multiple pixels in each row along the sub-scanning direction while moving from row to row in the main scanning direction. As a result of the operation of cameras 4a-4c, the start of scanning the nth (n = 2 to N) row is delayed by a time tRow from the start of scanning the n-1th row. Furthermore, the start of scanning the Nth row is delayed by a time tTotal = tRow × (N-1) from the start of scanning the 1st row.

[0047] Because of this delay, cameras 4a-4c with rolling shutters capture images at different times in the main scanning direction of the object, resulting in distortion in the image if the object moves quickly in the sub-scanning direction. Therefore, in this embodiment, the structure of cameras 4a-4c is devised so that cameras 4a-4c can be installed so that the sub-scanning direction of the rolling shutters coincides with the vertical direction. Since users typically move controller 6 frequently in the horizontal direction but rarely in the vertical direction, this arrangement minimizes distortion of controller 6 in the image.

[0048] FIG. 10 shows the structure of the cameras 4a to 4c employed to position the cameras 4a to 4c so that the sub-scanning direction of the rolling shutter coincides with the vertical direction. As shown in the figure, the cameras 4a to 4c each have a shutter 41 and a tripod-mounting screw hole 42. The screw hole 42 is used to mount the cameras to a tripod 50 with a camera mounting screw facing upward. The screw hole 42 is provided in the cameras 4a to 4c so that the axial direction of the screw hole 42 is parallel to the sub-scanning direction of the shutter 41. This makes it possible to position the cameras 4a to 4c so that the sub-scanning direction of the rolling shutter coincides with the vertical direction when the cameras 4a to 4c are mounted on the tripod 50. This minimizes distortion of the controller 6 in the image.

[0049] As described above, according to the pen-shaped controller 6 of this embodiment, the LED 10c is provided at the upper end of the grip portion 6g, a part that is unlikely to be hidden by the user's hand, so that the computer 2 can detect the position and orientation of the pen-shaped controller 6 with high accuracy.

[0050] Furthermore, according to the tracking system 1 of this embodiment, even though the cameras 4a to 4c having a rolling shutter as the shutter 41 are used, distortion of the controller 6 in the image can be minimized, making it possible to detect the position and orientation of the controller 6 with high accuracy.

[0051] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and it goes without saying that the present invention can be embodied in various forms without departing from the spirit of the present invention.

[0052] For example, in the above embodiment, an example has been described in which seven LEDs (two LEDs 10a-1 and 10a-2, four LEDs 10b-1 to 10b-4, and one LED 10c) are provided on the controller 6, but this number of seven is just an example, and it goes without saying that other numbers of LEDs may be arranged. However, if a large number of LEDs are arranged on the surface of the controller 6, the LEDs will overlap in the images captured by the cameras 4a to 4c, making it difficult to distinguish between the individual LEDs, so having too many LEDs is also inappropriate. The number of LEDs, seven, adopted in this embodiment is an optimized number taking this into consideration.

[0053] Furthermore, in the above embodiment, an example in which three cameras 4a to 4c are used has been described, but four or more cameras may also be used.

[0054] FIG. 11 is a diagram showing a state in which a tracking system 1 according to a modification of this embodiment is in use. The tracking system 1 according to this modification is similar to the tracking system 1 according to this embodiment, except that it has four cameras 4a to 4d. The cameras 4a to 4d are positioned so that they can capture images of the desk 100 from above the four corners of the desk 100. This enables the computer 2 to appropriately detect the positions and inclinations of the head-mounted display 5, controller 6, and position detection device 3 based on the images captured by the cameras 4a to 4c. As in this embodiment, the specific positions of the cameras 4a to 4d can be determined so that they can capture the entire portion of the area E shown in FIGS. 8(a) and 8(c) that is located on the desk 100.

[0055] In addition, in the above embodiment, the LEDs arranged on the surface of the controller 6 are configured as so-called point light sources, but at least some of the LEDs may be LEDs with a larger light-emitting area than so-called point light sources.

[0056] 12(a) and 12(b) are diagrams illustrating a controller 6 according to a modification of the present embodiment. In this modification, the LED 10c provided on the surface of the upper end of the grip portion 6g and the LED 10b-4 provided at the end of the pen are configured as LEDs with a larger light-emitting area than so-called point light sources. Furthermore, in this modification, the LEDs 10c and 10b-4 are configured as hemispheres by being arranged along the shape of the installation location. This allows the computer 2 to determine the center and radius of the hemisphere from the circle appearing in the images of the cameras 4a to 4c, and to obtain coordinates from only the images of the LEDs 10c and 10b-4. This allows the position and orientation of the controller 6 to be detected with even higher accuracy. [Explanation of symbols]

[0057] 1. Tracking System 2. Computer 3 Position detection device 4a~4d Camera 5. Head-mounted display 6 Controller 6g Grip 6ga tact top button 6gb grab button 6gc, 6gd tact button 6ge dial button 6gf recess 6p pen section 6pa, 6pb pressure pad 6pc,6pd shift button 10a-1,10a-2,10b-1~10b-4,10c LED 40 image sensors 41 Shutter 42 screw holes 50 tripod 100 desks 101 Desk Chair PT1~PT3 Controller 6 part

Claims

1. a pen portion formed in a pen shape; a grip portion intersecting the axial direction of the pen portion and extending between a first end and a second end; a first light emitting unit disposed at the first end of the grip unit, the first end being closer to the axial direction of the pen unit than the second end of the grip unit; one or more second light-emitting units disposed in the pen unit, the second light-emitting units including one or more fourth light-emitting units disposed in the pen unit on the rear side of the pen; the first light-emitting unit and the one or more second light-emitting units are used by a camera to capture light emitted from the first light-emitting unit and the one or more second light-emitting units; controller.

2. a first operation portion provided at an end of the grip portion that is close to the axial direction of the pen portion; the first light-emitting unit is disposed on the first operation unit; The controller of claim 1 .

3. a second operating portion provided so as to surround the first operating portion; The controller of claim 2 .

4. The one or more second light-emitting units include one or more third light-emitting units arranged on the pen unit on the pen tip side facing the pen rear side, A controller according to any one of claims 1 to 3.

5. The one or more third light emitting units are provided asymmetrically with respect to a plane including an axial direction of the pen unit and a longitudinal direction of the grip unit. The controller of claim 4 .

6. The one or more fourth light emitting units are provided asymmetrically with respect to a plane including an axial direction of the pen unit and a longitudinal direction of the grip unit. The controller of claim 1 .

7. the pen unit has a first side and a second side, the one or more third light-emitting units are provided on the first side surface and the second side surface, and the number of the third light-emitting units provided on the first side surface is the same as the number of the third light-emitting units provided on the second side surface; The controller of claim 5 .

8. the pen unit has a first side and a second side, the one or more fourth light-emitting units are provided on the first side surface and the second side surface, and the number of light-emitting units provided on the first side surface is different from the number of light-emitting units provided on the second side surface; The controller of claim 1 .

9. 9. A tracking system for tracking the movement of a controller according to any one of claims 1 to 8, comprising: one or more cameras each having a rolling shutter and arranged such that the sub-scanning direction of the rolling shutter coincides with the vertical direction; a computer that tracks the movement of the controller based on the arrangement of light emitted by the first light-emitting unit and the one or more second light-emitting units in the images captured by the one or more cameras; Tracking system including.

10. The one or more cameras It is used in a state where it is fixed to a tripod with a camera mounting screw facing upward, a screw hole for fixing the tripod, the screw hole being provided so that the axial direction is parallel to the sub-scanning direction of the rolling shutter; The tracking system of claim 9.

Images