XR controller

Abstract

 Provided is an XR controller configured to use a plurality of markers arranged on a surface of a housing to detect the position and posture thereof with high accuracy. The XR controller according to the present invention includes: a housing; and first markers arranged in a phyllotaxy pattern on a surface of the housing.

Description

XR Controller

[0001] The present invention relates to an XR-compatible controller (hereinafter referred to as an XR controller) used in a space (hereinafter referred to as an "XR space") created using XR technologies such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality).

[0002] A system is known in which a plurality of markers are arranged on the surface of an XR controller as trackers, and the markers are photographed and tracked using one or more cameras, thereby detecting the position and orientation of the XR controller. Patent Literature 1 discloses an example of this type of system. The document also discloses an example of a method for arranging markers on the surface of the housing of an XR controller that has a shape similar to a pen with a grip attached.

[0003] International Publication No. 2022 / 201693

[0004] In order to detect the position and orientation of the XR controller with high accuracy, it is necessary that at least three markers are placed sufficiently far apart in the image captured by the camera, and that the placement patterns of these three markers are sufficiently different (dissimilar) depending on the shooting direction.

[0005] Therefore, one object of the present invention is to provide an XR controller configured so that the position and orientation can be detected with high accuracy.

[0006] The XR controller according to the present invention is an XR controller including a housing and first markers arranged phyllotactically on a surface of the housing.

[0007] According to the present invention, it is possible to detect the position and orientation of an XR controller with high accuracy.

[0008] 1A and 1B are diagrams illustrating a state in which a tracking system 1 according to a first embodiment of the present invention is in use; FIG. 1C is a diagram illustrating a state in which a user is holding a controller 6 with his / her right hand; FIG. 1D is a perspective view of the controller 6 viewed from various angles; FIG. 1E is a perspective view of the controller 6 viewed from various angles; FIG. 1F is a perspective view of the controller 6 according to the first embodiment of the present invention; FIG. 1G is a diagram illustrating a method of arranging multiple markers in the background art (arranged at approximately uniform density); FIG. 1H is a diagram illustrating a method of arranging multiple markers in the background art (arranged at vertices of a regular polyhedron or a geodesic dome); FIG. 1H is a diagram illustrating a method of arranging multiple markers in the background art (arranged at vertices of a regular polyhedron or a geodesic dome); FIG. 1I is a diagram illustrating a method of arranging multiple markers in the background art (arranged in a grid-like pattern); FIG. 1I is a diagram illustrating a type of spiral phyllotaxis, and FIG. 1I is a diagram illustrating an example of 3 / 8 phyllotaxis; FIG. 1I is a diagram illustrating a specific example of the arrangement of multiple markers in a spiral arrangement unit 10; FIG. 1I is a diagram illustrating a specific example of the arrangement of multiple markers in a spiral arrangement unit 10; FIG. 1I is a perspective view of a controller 6 according to a modified example of the first embodiment of the present invention; FIG. 1I is a perspective view of a controller 6 according to a second embodiment of the present invention; 1A is a diagram showing an example of arranging multiple markers on the surface of an ellipsoid, and FIG. 1B is a diagram showing an example of arranging multiple markers on the surface of a cylinder. A diagram showing another example (double spiral) of arranging multiple markers in the spiral arrangement section 12. A diagram showing another example (combination of spirals that are inverted relative to each other in the phyllotactic axis direction) of arranging multiple markers in the spiral arrangement section 12. A diagram showing another example (spirals with varying spacing in the phyllotactic axis direction) of arranging multiple markers in the spiral arrangement section 12. A perspective view of a controller 6 according to a first modified example of the second embodiment of the present invention. A perspective view of a controller 6 according to a second modified example of the second embodiment of the present invention. A perspective view of a controller 6 according to a third modified example of the second embodiment of the present invention. A perspective view of a controller 6 according to a fourth modified example of the second embodiment of the present invention. (a) and (b) are diagrams showing examples of marker arrangement in the non-helical arrangement section 16. A perspective view of a controller 6 according to a fifth modified example of the second embodiment of the present invention.FIG. 10 is a diagram showing an example of a specific arrangement of a plurality of markers on a controller 6 according to a fifth modified example of the second embodiment of the present invention. FIG. 11 is a diagram showing an example of a specific arrangement of a plurality of markers on a controller 6 according to a fifth modified example of the second embodiment of the present invention. FIG. 12 is a diagram showing an example of a specific arrangement of a plurality of markers on a controller 6 according to a fifth modified example of the second embodiment of the present invention.

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0010] 1 is a diagram showing a state in which a tracking system 1 according to a first embodiment of the present invention is in use. As shown in the figure, the tracking system 1 is configured to have a computer 2, a position detection device 3, three cameras 4a to 4c, a head-mounted display 5, and a pen-shaped controller 6. The computer 2, the position detection device 3, the cameras 4a to 4c, the head-mounted display 5, and the controller 6 are each configured to be able to communicate with each other via wired or wireless communication.

[0011] 1, a user uses the tracking system 1 while sitting in a desk chair 101, wearing a head-mounted display 5 on his / her head, and holding a controller 6 in his / her right hand. An XR space rendered by a computer 2 is displayed on the display surface of the head-mounted display 5, and the user operates the controller 6 above the desk 100 while viewing this XR space.

[0012] The controller 6 is an XR controller used in the XR space, and is used to control 3D objects displayed in the XR space (specifically, to draw the 3D objects, move the 3D objects, etc.). The controller 6 is formed in a pen-like shape, and is also used to perform input using the position detection device 3.

[0013] 1, the computer 2 is configured as a notebook personal computer placed in the center of the desk 100. However, the computer 2 does not necessarily have to be placed in the center of the desk 100, and may be placed in a position where it can communicate with the position detection device 3, the cameras 4a to 4c, the head-mounted display 5, and the controller 6. Furthermore, the computer 2 may be configured as various types of computers other than a notebook personal computer, such as a desktop personal computer, a tablet personal computer, a smartphone, or a server computer.

[0014] The computer 2 serves to track the movements of the head-mounted display 5, controller 6, and position detection device 3 by periodically detecting the positions and orientations of these devices based on images captured by the cameras 4 a to 4 c. The computer 2 generates an XR space and 3D objects to be displayed therein based on the movements of each device being tracked and the operation states of each operation button and dial button (described below) provided on the controller 6, and performs processing to render the generated XR space and 3D objects and transmit them to the head-mounted display 5. The head-mounted display 5 serves to display the XR space including one or more 3D objects by displaying the rendered image transmitted from the computer 2.

[0015] 1, the position detection device 3 is configured as a tablet placed on the top surface of the desk 100 in a position corresponding to the front side of the computer 2 as seen from the user. However, the position detection device 3 does not necessarily have to be placed in this position, and may be placed within reach of the user sitting in the desk chair 101. The position detection device 3 and the computer 2 may also be configured as an integrated device, such as a tablet terminal.

[0016] The position detection device 3 has a function of periodically detecting the position of the pen tip of the controller 6 on the touch surface and sequentially transmitting the detected positions to the computer 2. Based on the transmitted positions, the computer 2 generates and renders stroke data that constitutes a 2D or 3D object. There are no particular limitations on the specific method of position detection by the position detection device 3, but it is preferable to use, for example, an active electrostatic method or an electrostatic induction method.

[0017] Cameras 4a to 4c are each an imaging device for capturing still or moving images, and are configured to sequentially supply the captured images to computer 2. Camera 4a is positioned opposite the user across desk 100, camera 4b is positioned above the user's left side, and camera 4c is positioned above the user's right side, each oriented so as to capture an image of the top surface of desk 100. Cameras 4a to 4c each have a rolling shutter, and are positioned so that the sub-scanning direction of the rolling shutter coincides with the vertical direction in order to minimize distortion of controller 6 in the image.

[0018] Fig. 2 is a diagram showing a state in which a user is holding the controller 6 in their right hand. Figs. 3(a), 3(b), 4(a), and 4(b) are perspective views of the controller 6 viewed from various angles. Note that the actual controller 6 has a spiral arrangement section 10 shown in Fig. 5, which will be described later, but which is not shown in these figures. Details of the spiral arrangement section 10 will be described in detail later with reference to Figs. 5 to 12.

[0019] 2, 3(a), 3(b), 4(a), and 4(b), the controller 6 is configured to have a pen-shaped pen section 6p and a grip section 6g fixed to the pen section 6p so that its longitudinal direction intersects with the axial direction of the pen section 6p. Hereinafter, the axial direction of the pen section 6p will be referred to as the x-direction, a direction in a plane formed by the x-direction and the longitudinal direction of the grip section 6g and perpendicular to the x-direction will be referred to as the z-direction, and a direction perpendicular to both the x-direction and the z-direction will be referred to as the y-direction.

[0020] As shown in FIG. 3A , the surface of the pen unit 6 p is provided with pressure pads 6 pa and 6 pb and shift buttons 6 pc and 6 pd. The pressure pads 6 pa and 6 pb are components including a pressure sensor and a touch sensor, and are located near the pen tip on the side of the pen unit 6 p, symmetrically with respect to the xz plane. The pressure detected by the pressure sensor is used for selection or drawing in an application. Meanwhile, information indicating the presence or absence of a touch detected by the touch sensor is used to determine whether the pressure sensor output is on or off and to realize a light double tap. The shift buttons 6 pc and 6 pd are switches assigned to application menus, and are located between the pressure pads 6 pa and 6 pb and the grip unit 6 g, symmetrically with respect to the xz plane. As can be seen from FIG. 2 , a user holding the controller 6 in their right hand operates the pressure pad 6 pa and shift button 6 pc with their thumb and the pressure pad 6 pb and shift button 6 pd with their index finger.

[0021] As shown in FIGS. 3( a ), 3 ( b ), 4 ( a ), and 4 ( b ), the surface of the grip portion 6g is provided with a tactile top button 6ga, a grab button 6gb, tactile buttons 6gc and 6gd, a dial button 6ge, and a recess 6gf. The tactile top button 6ga is a switch that functions as a power button when pressed and held. It is located on the surface of one of the longitudinal ends of the grip portion 6g, closer to the pen unit 6p. Hereinafter, this end will be referred to as the “upper end,” and the other of the longitudinal ends of the grip portion 6g, farther from the pen unit 6p, will be referred to as the “lower end.” The dial button 6ge is a rotatable ring-shaped member configured to output a rotation amount. This rotation amount is used, for example, to rotate a selected object. The dial button 6ge is also located on the upper end of the grip portion 6g, surrounding the tactile top button 6ga.

[0022] The grab button 6gb is a switch used to grab and move an object, and is located near the lower end of the pen tip side surface of the grip portion 6g. The tact buttons 6gc and 6gd are switches used as auxiliary buttons like the right button on a mouse, and are located near the pen portion 6p when viewed in the z direction on the pen tip side surface of the grip portion 6g. The tact button 6gc is located on the thumb side when holding the controller 6 in the right hand, and the tact button 6gd is located on the index finger side when holding the controller 6 in the right hand.

[0023] As can be seen from Figure 2, a user holding the controller 6 in their right hand will press the grab button 6gb with their middle finger. They will also press the tact button 6gc with their thumb and the tact button 6gd with their index finger. The rotation of the dial button 6ge and the pressing of the tact top button 6ga are performed with the user's thumb. However, because the tact top button 6ga and the dial button 6ge are located in positions that cannot be operated unless the user intentionally lifts their thumb to the upper end of the grip portion 6g, they are exposed and not hidden by the user's hand in the normal state.

[0024] 2, the recess 6gf is configured to fit snugly between the base of the index finger and the base of the thumb when the user holds the controller 6. Providing this recess 6gf in the controller 6 reduces fatigue experienced by the user when using the controller 6.

[0025] Next, the spiral arrangement unit 10 provided in the controller 6 so that the computer 2 can detect the position and orientation of the controller 6 with high accuracy will be described in detail.

[0026] 5 is a perspective view of the controller 6 according to this embodiment. As shown in the figure, the controller 6 includes a spherical portion 7 attached to the end of a pen portion 6p. The spherical portion 7, together with the pen portion 6p and the grip portion 6g, constitutes the housing of the controller 6. A plurality of markers are arranged in a phyllotactic arrangement on the surface of the spherical portion 7, and these markers constitute a spiral arrangement portion 10. The phyllotactic arrangement will be described in detail later with reference to FIGS. 10 to 12.

[0027] Although there are no particular limitations on the specific type of markers that make up spiral arrangement unit 10, infrared LEDs (Light Emitting Diodes) are preferable. In this case, cameras 4a to 4c are each configured as infrared cameras that can visualize infrared light, and computer 2 is configured to detect the position and orientation of controller 6 based on the arrangement of markers that appear in the images captured by cameras 4a to 4c. The arrangement of multiple markers in spiral arrangement unit 10 will be specifically described below with reference to FIGS. 6 to 12.

[0028] First, Figures 6 to 9 are diagrams illustrating methods for arranging multiple markers. Below, various arrangement methods are described with reference to Figures 6 to 9, and then the detailed configuration of the spiral arrangement unit 10 according to this embodiment and the advantages of employing the spiral arrangement unit 10 according to this embodiment are described in detail with reference to Figures 10 to 12.

[0029] FIG. 6 is a diagram showing an example in which multiple markers are arranged at approximately uniform density. In FIG. 6, the spheres represent the spherical portions 7, and the black circles represent individual markers. Hereinafter, this arrangement will be referred to as an "approximately uniform density arrangement." FIGS. 6(a) to 6(c) respectively show an example in which multiple markers are arranged at approximately uniform density at high density, an example in which multiple markers are arranged at approximately uniform density at medium density, and an example in which multiple markers are arranged at approximately uniform density at low density. Methods for determining the specific positions of each marker in an approximately uniform density arrangement include generating multiple arrangement patterns randomly using a sequential random method, a Poisson disk sampling method, or the like, and then determining the most suitable arrangement from among these using a method such as the Monte Carlo method.

[0030] In order for the computer 2 to detect the position and orientation of the controller 6 with high accuracy, it is necessary to arrange the markers on the surface of the housing so that at least three markers are captured in the images captured by each of the cameras 4 a to 4 c with sufficient distance between them (i.e., without uneven placement), and so that the placement patterns (geometric features) of the markers are sufficiently different (i.e., dissimilar) depending on the capture direction. Hereinafter, this arrangement will be referred to as the "preferred arrangement." When using the approximately uniform density arrangement shown in FIG. 6 , it is possible to achieve the preferred arrangement if there are a sufficient number of markers. However, considering the size of each infrared LED and the size of the spherical portion 7, there is a limit to the number of markers that can be placed on the surface of the spherical portion 7. Therefore, achieving the preferred arrangement using an approximately uniform density arrangement is not realistic.

[0031] 7 and 8 are diagrams showing examples of arranging markers at the vertices of a regular polyhedron (regular dodecahedron, regular icosahedron, etc.) or a geodesic dome (a solid formed by increasing the number of vertices by subdividing the faces of a regular polyhedron or semi-regular polyhedron). Hereinafter, an arrangement based on this example will be referred to as a "polyhedron-derived arrangement."

[0032] FIG. 7( a) shows an example of a regular dodecahedron, and FIG. 7( b) shows an example in which markers are placed at positions on the surface of a sphere corresponding to each of the 20 vertices of the regular dodecahedron. In FIG. 7( b), the sphere represents the spherical portion 7, the black circles represent individual markers, and the arrows extending from the black circles represent the normal direction of the spherical surface (curved surface) at the position of the black circles. This also applies to FIGS. 9( b), 11, 15( a), 15( b), 16( a), 17( a), and 18( a), which will be described later. However, in FIGS. 16( a) and 17( a), white circles and x marks are used instead of black circles for ease of illustration.

[0033] FIG. 8( a) shows projections of a spherical portion 7 having 20 markers arranged at 20 vertices of a regular dodecahedron from the x direction (upper left), z direction (lower left), and y direction (lower right), and FIG. 8( b) shows an image obtained when the spherical portion 7 of FIG. 8( a) is photographed by camera 4a. Each view in FIG. 8( a) is a perspective view, and not only the markers located on the front side as viewed from the line of sight, but also the markers located on the back side are plotted. This also applies to FIGS. 12( a), 16( b), 17( b), and 18( b) shown below. However, in FIG. 8( a), the arrangement of the markers on the front side and the arrangement of the markers on the back side are the same, so the resulting image is identical to that obtained when only the markers located on the front side are drawn.

[0034] As can be seen from Figures 7 and 8, when using a polyhedron-derived layout, it is possible to realize that at least three markers are captured in the images captured by each of cameras 4a to 4c while being sufficiently spaced apart (i.e., without uneven placement). On the other hand, a problem with the polyhedron-derived layout is that it is not possible to realize a layout in which the placement patterns (geometric features) of the markers are sufficiently different (i.e., dissimilar) depending on the shooting direction. Hereinafter, this problem will be referred to as the "aperture problem."

[0035] The opening problem occurs in polyhedron-derived layouts because the layout is too symmetrical. To explain this in a specific example, among the three projections shown in FIG. 8( a), the projection from the z direction (lower left) and the projection from the y direction (lower right) are identical. Furthermore, the projection from the x direction (upper left) is also identical to the projection from the z direction (lower left) and the projection from the y direction (lower right), except that it is rotated 90 degrees. In other words, the marker layout pattern (geometric features) does not differ sufficiently depending on the shooting direction. As a result, even when looking at the image shown in FIG. 8( b), it is impossible to determine from which direction the image was taken. Therefore, it can be said that it is difficult to achieve a suitable layout using polyhedron-derived layouts.

[0036] 9A and 9B are diagrams showing examples of arranging multiple markers in a grid pattern (such as a square grid, a hexagonal grid, a rectangular grid, or an oblique grid). Hereinafter, this type of arrangement will be referred to as a "planar grid-derived arrangement." FIG. 9A shows an example of a square grid drawn on a spherical surface, and FIG. 9B shows an example of markers arranged at some of the intersections of the square grid shown in FIG. 9A (i.e., the intersections of line segments that divide the latitude and longitude equally).

[0037] Using a planar grid-derived layout, it is possible to ensure that at least three markers are captured in the images captured by each of the cameras 4a to 4c with sufficient distance between them (i.e., without uneven placement). However, the planar grid-derived layout also suffers from the same aperture problem as the polyhedron-derived layout. Therefore, it can be said that it is difficult to achieve an optimal layout even with a planar grid-derived layout.

[0038] The spiral arrangement unit 10 according to this embodiment overcomes the drawbacks of the above-described substantially uniform density arrangement, polyhedron-derived arrangement, and planar grid-derived arrangement, and it is possible to arrange multiple markers on the surface of the housing so that at least three markers appear sufficiently separated (i.e., without unevenness in arrangement) in the images captured by each of the cameras 4a to 4c, and the arrangement patterns (geometric features) are sufficiently different (i.e., dissimilar) depending on the shooting direction. This will be described in detail below with reference to FIGS. 10 to 12.

[0039] First, to give a general explanation of phyllotaxis, it refers to the arrangement of leaves on stems in plants in nature, and includes "alternate phyllotaxis," in which one leaf grows on each node of the stem, "opposite phyllotaxis," in which two leaves grow on each node of the stem, and "whorled phyllotaxis," in which three or more leaves grow on each node of the stem. Among alternate phyllotaxis, the one in which leaves grow in a spiral along the extension direction of the stem is particularly called "spiral phyllotaxis."

[0040] Figure 10(a) is a diagram showing types of spiral phyllotaxis. As shown in the figure, there are various types of spiral phyllotaxis classified by the way the leaves are attached, each of which is called r / n phyllotaxis. In r / n phyllotaxis, the leaf that emerges from the stem at the same angle (as viewed from directly above) as the first leaf is the n+1st leaf, and the spiral rotates r times around the stem between the first and n+1st leaves. As shown in Figure 10(a), various combinations of r and n are possible, such as 1 / 2 phyllotaxis, 1 / 3 phyllotaxis, 2 / 5 phyllotaxis, 3 / 8 phyllotaxis, 5 / 13 phyllotaxis, and 8 / 21 phyllotaxis.

[0041] The r and n of r / n phyllotaxy are r / n = F ｋ / (mF ｋ +F ｋ－１ ) is known to satisfy the relationship (Simper-Brown's law). ｋ is the k-th term of the Fibonacci sequence, and m is a natural number. In many cases, m=2, in which case r and n are composed of the Fibonacci number (1, 1, 2, 3, 5, 8...) and the Fibonacci number two after it (2, 3, 5, 8, 13, 21...).

[0042] Figure 10(b) shows an example of a 3 / 8 phyllotaxy (r = 3, n = 8). The "135 degrees" shown in the figure is the projection angle between the nth and n+1th plates (the angle when viewed from the axial direction of the spiral) and is called the "opening degree." As shown in Figure 10(a), the opening degree varies depending on the type of phyllotaxy. Specifically, for a 1 / 2 phyllotaxy, it is 180 degrees, for a 1 / 3 phyllotaxy, it is 120 degrees, for a 2 / 5 phyllotaxy, it is 144 degrees, for a 3 / 8 phyllotaxy, it is 135 degrees, for a 5 / 13 phyllotaxy, it is 1800 / 13 ≒ 138.5 degrees, and for an 8 / 21 phyllotaxy, it is 2880 / 21 ≒ 137.1 degrees.

[0043] 11 and 12 are diagrams showing specific examples of the arrangement of multiple markers (multiple markers arranged in a phyllotactic arrangement) in the spiral arrangement section 10. FIG. 11 shows an example in which 22 markers are arranged on the surface of a sphere in an 8 / 21 phyllotactic arrangement. As shown in the figure, the multiple markers constituting the spiral arrangement section 10 are arranged so that the axis of the phyllotactic arrangement (a line corresponding to the stem, hereinafter referred to as the "phyllotactic axis") passes through the center of the sphere constituting the spherical section 7. The spacing between the markers in the phyllotactic axis direction may be equal. Note that while FIG. 11 shows an example in which the pen axis (= x-axis) and the phyllotactic axis coincide with each other, they do not necessarily have to coincide. FIG. 12(a) shows projections of the spherical section 7 having multiple markers according to the example of FIG. 11 from the x direction (upper left), z direction (lower left), and y direction (lower right), and FIG. 12(b) shows an image obtained when the spherical section 7 of FIG. 12(a) is photographed with a camera 4a.

[0044] As can be seen from Figures 11 and 12, when using a phyllotactic arrangement, it is possible to ensure that at least three markers are sufficiently spaced apart (i.e., evenly spaced) in the images captured by each of cameras 4a-4c. Additionally, by using a phyllotactic arrangement, the markers can be positioned so that their placement patterns (geometric features) are sufficiently different (i.e., dissimilar) depending on the shooting direction, thereby avoiding the occurrence of the opening problem. Looking at the image in Figure 12(b), it can be seen that this image corresponds to the upper left image (projection from the x direction) of Figure 12(a), not the lower left image (projection from the z direction) or the lower right image (projection from the y direction). This demonstrates that the opening problem does not occur when using a phyllotactic arrangement. Therefore, it can be said that a suitable placement can be achieved using a phyllotactic arrangement.

[0045] As described above, the configuration of controller 6 according to this embodiment makes it possible to arrange multiple markers on the surface of the housing so that at least three markers are captured in the images captured by each of cameras 4 a to 4 c with sufficient distance between them (i.e., with even placement), and so that the placement patterns (geometric features) are sufficiently different (i.e., dissimilar) depending on the shooting direction. Therefore, the position and orientation of controller 6 can be detected with high accuracy using the multiple markers arranged on the surface of the housing.

[0046] FIG. 13 is a perspective view of a controller 6 according to a modification of this embodiment. The controller 6 according to this modification further includes a spherical portion 8 attached to the upper end of the grip portion 6g of the controller 6 according to this embodiment. The spherical portion 8, together with the pen portion 6p, the grip portion 6g, and the spherical portion 7, constitutes the housing of the controller 6. A plurality of markers (infrared LEDs) are arranged phyllotactically on the surface of the spherical portion 8, similar to the surface of the spherical portion 7, and these markers form a spiral arrangement portion 11. The computer 2 is configured to detect the position and orientation of the controller 6 based on the arrangement of the markers on each of the spiral arrangement portions 10 and 11 that appear in the images captured by the cameras 4a to 4c. This allows the computer 2 to detect the orientation of the controller 6 from the relative positional relationship between the spiral arrangement portions 10 and 11, thereby enabling the position and orientation of the controller 6 to be detected with even greater accuracy.

[0047] It should be noted that the spiral arrangement portion 10 and the spiral arrangement portion 11 do not have to have exactly the same shape. For example, the spherical portion 7 and the spherical portion 8 may be different in size, or different types of phyllotaxis may be adopted for the spiral arrangement portion 10 and the spiral arrangement portion 11. This prevents the computer 2 from mistaking the spiral arrangement portion 10 for the spiral arrangement portion 11 and detecting them as different ones.

[0048] Next, a tracking system 1 according to a second embodiment of the present invention will be described. The tracking system 1 according to this embodiment differs from the tracking system 1 according to the first embodiment in the configuration of the controller 6. Therefore, in the following, the same components as those in the tracking system 1 according to the first embodiment will be assigned the same reference numerals as those in the first embodiment and their description will be omitted, and the description will continue focusing on the differences from the first embodiment.

[0049] 14 is a perspective view of the controller 6 according to the present embodiment. As shown in the figure, the controller 6 according to the present embodiment is configured to have a spiral arrangement portion 12 on the side surface of the pen portion 6p, rather than the spherical portion 7 (see FIG. 5). The controller 6 according to the present embodiment does not have a spherical portion 7.

[0050] To explain the configuration of the spiral arrangement portion 12 in detail, the spiral arrangement portion 12 is arranged on the side of the pen portion 6p closer to the distal end than the grip portion 6g. Although not shown in Figure 14, the spiral arrangement portion 12 is configured with a plurality of markers arranged phyllotactically so that the phyllotactic axis coincides with the pen axis. The markers are preferably infrared LEDs, as in the first embodiment.

[0051] Figure 15(a) shows an example of multiple markers arranged on the surface of an ellipsoid, and Figure 15(b) shows an example of multiple markers arranged on the surface of a cylinder. Both examples show 22 markers arranged according to an 8 / 21 phyllotaxis. In the former, the phyllotaxis axis coincides with the major axis of the ellipsoid, while in the latter, the phyllotaxis axis coincides with the axis of the cylinder. The spacing between markers in the phyllotaxis axis direction can be equal, as in the case of arrangement on the surface of a sphere. As shown in these examples, the curved surface on which multiple markers are phyllotactically arranged can be not only the surface of a sphere described in the first embodiment, but also surfaces of various shapes. The spiral arrangement unit 12 according to this embodiment utilizes this property of phyllotaxis to arrange multiple markers on the side of the pen unit 6p.

[0052] The configuration of controller 6 according to this embodiment also makes it possible to arrange multiple markers on the surface of the housing so that at least three markers are captured in the images captured by each of cameras 4 a to 4 c with sufficient distance between them (i.e., even placement), and so that the placement patterns (geometric features) are sufficiently different (i.e., dissimilar) depending on the shooting direction. Therefore, the position and orientation of controller 6 can be detected with high accuracy using the multiple markers arranged on the surface of the housing.

[0053] Here, the arrangement of the multiple markers in the spiral arrangement section 12 is not limited to the simple phyllotactic arrangement as shown in Figures 15(a) and 15(b). This point will be explained in detail below with reference to Figures 16 to 18.

[0054] 16 to 18 are diagrams showing other examples of the arrangement of multiple markers in the spiral arrangement section 12. Each diagram (a) is a transparent perspective view of the spiral arrangement section 12, and each diagram (b) is a diagram showing projections of the spiral arrangement section 12 shown in (a) from the x direction (upper left), z direction (lower left), and y direction (lower right).

[0055] FIG. 16 shows an example of a spiral arrangement section 12 constructed with two spirals each consisting of a plurality of markers arranged according to a 3 / 8 phyllotaxis. In the figure, a white circle indicates one spiral, and a cross indicates the other spiral. In this example, the other spiral is the first spiral rotated 180° around the phyllotaxis axis (around the x-axis), forming a double spiral. The spiral arrangement section 12 according to this embodiment can also be constructed with such a double spiral, which further reduces the risk of the opening problem. While FIG. 16 shows an example of a double spiral, the spiral arrangement section 12 can also be constructed with a multiple spiral (three or more).

[0056] Figure 17 also shows an example in which the spiral arrangement section 12 is configured with two spirals each consisting of a plurality of markers arranged in a 3 / 8 phyllotaxis. Again, the open circle indicates one spiral, and the cross indicates the other. In this example, the other spiral is the inverse of the first spiral in the phyllotaxis direction (x-direction). The spiral arrangement section 12 according to this embodiment can also be configured by combining spirals that are inverted in the phyllotaxis direction, which further reduces the risk of the opening problem.

[0057] In the example of Figure 17, the number of markers per spiral is seven, but when spirals inverted in the phyllotactic axis direction (x direction) are combined, the number of markers becomes 13 (the number does not become 7 x 2 = 14 because two of the markers are in the same position). Therefore, according to the example of Figure 17, it can be said that it is possible to realize a similar number of markers in a narrow range in the phyllotactic axis direction compared to when multiple markers are placed in a 5 / 13 phyllotactic arrangement at the same pitch in the phyllotactic axis direction as each spiral.

[0058] FIG. 18 shows an example in which the spiral arrangement section 12 is formed by a single spiral consisting of multiple markers arranged in a 3 / 8 phyllotaxis. However, in this example, the distance between two adjacent markers in the phyllotaxis direction (x direction) increases in an arithmetic progression. The spiral arrangement section 12 according to this embodiment can also be formed by a modified phyllotaxis arrangement in which the distance between the adjacent markers in the phyllotaxis direction is varied, thereby further reducing the risk of the opening problem. Note that while FIG. 18 shows an example in which the distance between the adjacent markers in the phyllotaxis direction increases in an arithmetic progression, the distance between the adjacent markers in the phyllotaxis direction may be changed in other ways. For example, the distance may be increased in a geometric progression.

[0059] Next, the controller 6 according to first to fourth modifications of this embodiment will be described with reference to FIGS.

[0060] FIG. 19 is a perspective view of a controller 6 according to a first modification of this embodiment. The controller 6 according to this modification further includes a spiral arrangement unit 13 provided on a side surface of the pen unit 6p closer to the pen tip than the grip unit 6g. Similar to the spiral arrangement unit 12, the spiral arrangement unit 13 includes a plurality of markers phyllotactically arranged so that the phyllotactic axis coincides with the pen axis. The specific arrangement of the plurality of markers in the spiral arrangement unit 13 may be the same as or different from that of the spiral arrangement unit 12. For example, the spiral arrangement unit 12 may include a plurality of markers arranged in a simple phyllotactic arrangement, while the spiral arrangement unit 13 may include a plurality of markers arranged in a phyllotactic arrangement with an irregular spiral structure as described with reference to FIGS. 16 to 18 . By adopting different arrangements for the spiral arrangement portion 12 and the spiral arrangement portion 13, the computer 2 can easily distinguish and detect the spiral arrangement portion 12 and the spiral arrangement portion 13 based on the arrangement of the markers of the spiral arrangement portions 12 and 13 that appear in the images captured by the cameras 4 a to 4 c. Furthermore, it is possible to reduce the risk of the opening problem occurring compared to when the spiral arrangement portion 12 is used alone.

[0061] 20 is a perspective view of a controller 6 according to a second modification of this embodiment. The controller 6 according to this modification further includes a non-helical arrangement unit 14 arranged at the end of the pen unit 6p in addition to the components of the controller 6 according to this embodiment. The non-helical arrangement unit 14 includes one or more markers arranged without following a phyllotactic arrangement. By using such a non-helical arrangement unit 14 in combination with the helical arrangement unit 12, the risk of the opening problem can be reduced, allowing the computer 2 to detect the position and orientation of the controller 6 with higher accuracy than when the helical arrangement unit 12 is used alone.

[0062] 21 is a perspective view of a controller 6 according to a third modified example of this embodiment. The controller 6 according to this modified example is configured with the components of the controller 6 according to the third modified example of this embodiment further including a non-helical arrangement section 15 arranged at the pen tip. Like the non-helical arrangement section 14, the non-helical arrangement section 15 is configured with one or more markers arranged without following a phyllotactic arrangement. By using not only the non-helical arrangement section 14 but also the non-helical arrangement section 15 in combination with the spiral arrangement section 12, the risk of the opening problem occurring can be further reduced, allowing the computer 2 to detect the position and orientation of the controller 6 with higher accuracy than when the non-helical arrangement section 14 and the spiral arrangement section 12 are used in combination.

[0063] FIG. 22 is a perspective view of a controller 6 according to a fourth modification of this embodiment. The controller 6 according to this modification further includes a non-helical arrangement portion 16 provided on a side surface of the pen unit 6p closer to the pen tip than the grip portion 6g. In the first modification, the spiral arrangement portion 13 is provided on a side surface of the pen unit 6p closer to the pen tip than the grip portion 6g. However, since providing the spiral arrangement portion 13 requires a large space, as shown in FIG. 19 , in the first modification, the installation space for the pressure pads 6pa and 6pb is encroached upon by the spiral arrangement portion 13. The non-helical arrangement portion 16 can be installed without requiring a large space like the spiral arrangement portion 13, thereby avoiding such encroachment and enabling the installation of the pressure pads 6pa and 6pb.

[0064] 23( a) and 23(b) are diagrams illustrating examples of marker placement in the non-helical placement section 16. In the example of FIG. 23(a), markers (black circles) are placed at each of the three vertices of an equilateral triangle placed in the yz cross section (cross section perpendicular to the pen axis) of the circular pen unit 6p. This placement can reduce the likelihood of the opening problem occurring compared to using only the spiral placement section 12. In the example of FIG. 23(b), markers (black circles) are placed at three of the five vertices (vertices A, C, and D, when the vertices are labeled A to E clockwise) of a regular pentagon placed in the yz cross section (cross section perpendicular to the pen axis) of the circular pen unit 6p. This placement can reduce the likelihood of the opening problem occurring even when the spiral placement section 12 is not photographed and only the non-helical placement section 16 is photographed.

[0065] 24 is a perspective view of a controller 6 according to a fifth modified example of this embodiment. In addition to the helical arrangement section 12 and the non-helical arrangement section 16 in controller 6 according to the fourth modified example of this embodiment, controller 6 according to this modified example further uses non-helical arrangement section 14, thereby further reducing the risk of the opening problem occurring. This enables computer 2 to detect the position and orientation of controller 6 with higher accuracy than when only helical arrangement section 12 and non-helical arrangement section 16 are used.

[0066] 25 to 27 are diagrams showing examples of specific arrangements of multiple markers on a controller 6 according to a fifth modified example of this embodiment. FIGS. 25(a) to 25(d) and 26(a) to 26(d) each show a side view of the controller 6 obtained when the viewpoint is rotated around the x direction (the axial direction of the pen portion 6p) by the angle shown. Also, FIG. 27(a) shows the side view of the controller 6 as seen from the pen tip, and FIG. 27(b) shows the side view of the controller 6 as seen from the end of the pen. FIGS. 25 to 27 show an example in which a spiral arrangement section 12 is formed by the arrangement of multiple markers shown in FIG. 17, and a non-spiral arrangement section 16 is formed by the arrangement of multiple markers shown in FIG. 23(b).

[0067] 25 to 27, dashed circles are drawn at the locations of the markers to make the position of each marker easier to understand. Furthermore, for the spiral arrangement section 12, the serial numbers 1 to 8 of the eight markers constituting one spiral are shown in dashed circles, and the serial numbers A to H of the eight markers constituting the other spiral are shown in dashed circles. As can be seen from these serial numbers, the example shown in FIGS. 25 to 27 realizes the arrangement of multiple markers shown in FIG. 17 (one spiral consisting of eight markers arranged according to a 3 / 8 phyllotaxis, and the other spiral formed by flipping the first spiral in the phyllotactic axis direction (x direction)).

[0068] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and it goes without saying that the present invention can be embodied in various forms without departing from the spirit of the present invention.

[0069] For example, the phyllotactic arrangement of the irregular spiral structure (double spiral or multiple spiral, a combination of spirals inverted in the phyllotactic axis direction, or a spiral with varying spacing in the phyllotactic axis direction) described with reference to Figures 16 to 18 may be applied to the spiral arrangement sections 10 and 11 described in the first embodiment.

[0070] Furthermore, a single controller 6 may be configured by appropriately combining the spiral placement units 10 and 11 described in the first embodiment with the spiral placement units 12 and 13 and the non-helical placement units 14 to 16 described in the second embodiment. For example, the controller 6 shown in FIG. 13 may be combined with the spiral placement units 12 and 13 and the non-helical placement unit 15, or with the spiral placement unit 12 and the non-helical placement units 15 and 16. A single controller 6 may also be configured by appropriately selecting some of the spiral placement units 10 and 11 described in the first embodiment and the spiral placement units 12 and 13 and the non-helical placement units 14 to 16 described in the second embodiment. For example, a controller 6 may be configured without the spiral placement unit 10 but with the spiral placement unit 11, and without the spiral placement unit 12 but with the spiral placement unit 13 and the non-helical placement units 14 to 16.

[0071] REFERENCE SIGNS LIST 1 Tracking system 2 Computer 3 Position detection device 4a to 4c Camera 5 Head-mounted display 6 Controller 6g Grip portion 6ga Tact top button 6gb Grab button 6gc, 6gd Tact button 6ge Dial button 6gf Recessed portion 6p Pen portion 6pa, 6pb Pressure pad 6pc, 6pd Shift button 7, 8 Spherical portion 10 to 13 Spiral arrangement portion 14 to 16 Non-spiral arrangement portion 100 Desk 101 Desk chair

Claims

1. The housing and First markers arranged phyllotactically on the surface of the housing; An XR controller including:

2. the housing includes a pen portion formed in a pen shape, The first marker is disposed on the surface of the pen unit. The XR controller of claim 1 .

3. further comprising a second marker different from the first marker; the second markers are arranged phyllotactically at positions closer to the pen tip than the first markers on the surface of the housing; The XR controller of claim 2 .

4. The first marker is phyllotactically arranged according to a double helix structure. The XR controller according to any one of claims 1 to 3.

5. The first markers are arranged phyllotactically by a combination of spirals inverted relative to each other in the direction of the phyllotactic axis. The XR controller according to any one of claims 1 to 3.

6. the first markers are phyllotactically arranged in a spiral with varying spacing along the phyllotactic axis; The XR controller according to any one of claims 1 to 3.

7. the housing includes a spherical portion formed in the shape of a sphere, The first marker is arranged so that a phyllotactic axis passes through the center of a sphere constituting the spherical portion. The XR controller of claim 1 .

8. The housing further includes a pen portion formed in a pen shape, the spherical portion is attached to the distal end of the pen portion; The XR controller of claim 7.

9. further comprising a second marker arranged phyllotactically on the surface of the housing; the housing includes a first spherical portion and a second spherical portion each formed in a spherical shape; the first marker is arranged so that a phyllotactic axis passes through the center of a sphere constituting the first spherical portion; The second marker is arranged so that a phyllotactic axis passes through the center of a sphere constituting the second spherical portion. The XR controller of claim 1 .

10. The housing includes: a pen portion formed in a pen shape; a grip portion fixed to the pen portion such that the longitudinal direction thereof intersects with the axial direction of the pen portion; the first spherical portion is disposed at a distal end of the pen portion; the second spherical portion is disposed at one of both longitudinal ends of the grip portion that is closer to the pen portion; The XR controller of claim 9.