Controller device, control method for the controller device, and program

Abstract

This controller device comprises a controller body and a control unit. The control unit receives a user operation on the controller body, infers the attitude of the controller body, and generates and outputs information representing the content of the received operation by using the result of inference.

Description

Technical Field

[0004] , ,

[0006] , , , , , , , , , , , ,

[0003] ,

[0005] , ,

[0001] The present invention relates to a controller device, a control method for the controller device, and a program.

Background Art

[0002] In a VR (Virtual Reality) game or the like where an HMD (Head Mounted Display) is worn, in order to improve the immersion feeling, it may be arranged not to show the user an image of the real space. In such a case, since the user cannot see their hands, they cannot confirm the direction of the game controller in their hand.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Therefore, currently, the user gropes for the shape of the controller device (which is generally asymmetric) and adjusts the direction of the controller device so as to hold it in the assumed manner, resulting in low convenience.

[0004] In view of the above actual situation, one object of the present invention is to provide a controller device, a control method for the controller device, and a program that can improve convenience.

Means for Solving the Problems

[0005] One aspect of the present invention for solving the problems of the above conventional example is a controller device including a controller main body and a control unit, wherein the control unit includes an operation reception means for receiving a user's operation on the controller main body, an estimation means for estimating the posture of the controller main body, and an operation information generation means for generating and outputting information representing the content of the received operation using the estimation result of the estimation means.

Effects of the Invention

[0006] <00According to the present invention, the posture is estimated, and information representing the content of the accepted operation is generated using the estimation result. Therefore, information representing the content of the operation can be generated regardless of how the user holds the device, thereby improving convenience. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic block diagram showing an example configuration of a controller device and an information processing device connected thereto according to an embodiment of the present invention. [Figure 2] This is a block diagram of a configuration of an example of a controller device according to an embodiment of the present invention. [Figure 3] This is a functional block diagram of an example of a controller device according to an embodiment of the present invention. [Figure 4] This is an explanatory diagram illustrating an example of detecting a user's hand or fingers using a controller device according to an embodiment of the present invention. [Figure 5] This flowchart illustrates an example of the operation of a controller device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described with reference to the drawings. In the following description and drawings, the sizes of each part and the ratios of length, width, and height are illustrative examples, and actual sizes and ratios may differ.

[0009] As illustrated in Figure 1, the controller device 10 according to an embodiment of the present invention is configured to include a controller body 11 and a circuit unit 12 built into the controller body 11, and is connected to the information processing device 30 via wired or wireless communication.

[0010] This information processing device 30 is, for example, a home game console and comprises a control unit 31, a storage unit 32, a communication unit 33, and a display unit 34. The control unit 31 is a program control device such as a CPU and operates according to a program stored in the storage unit 32. In this embodiment, the control unit 31 uses information representing the content of user operations received from the controller device 10 via the communication unit 33 to execute processing such as game applications. The control unit 31 also performs processing such as displaying guidance to the user in response to instructions received from the controller device 10.

[0011] The storage unit 32 includes disk devices, memory devices, etc., and holds programs executed by the control unit 31. This storage unit 32 may also function as the work memory for the control unit 31.

[0012] The communication unit 33 is connected to the controller device 10 via wireless or wired communication, receives information representing the user's operation from the controller device 10, and outputs it to the control unit 31.

[0013] The display unit 34 is, for example, a display controller, and outputs and displays an image to a display (not shown) according to instructions input from the control unit 31.

[0014] The controller body 11 preferably includes a housing 110 that is rotationally symmetrical about at least one axis, and for example is a housing that is substantially spherical. However, a sphere is just one example, and the housing 110 may be cylindrical or other shapes, or it may be a regular hexahedron or other shape even if it is not rotationally symmetrical. Furthermore, the surface of the housing 110 may be given an uneven surface by blasting or the like.

[0015] In one example of this embodiment, the housing 110 may be elastically deformable, in which case it is constructed using an elastically deformable material such as a polymer gel material such as a silicone polymer gel material or a urethane gel material, or various elastomer materials such as a polystyrene elastomer, an olefin elastomer, a polyvinyl chloride elastomer, a polyurethane elastomer, a polyester elastomer, or a polyamide elastomer.

[0016] As illustrated in Figure 2, the circuit unit 12 includes a control unit 121, a storage unit 122, a sensor unit 123, and a communication unit 124. Each of these components of the circuit unit 12 is powered by a battery (not shown). This battery may be a rechargeable secondary battery, for example, that can be charged by wireless or wired power supply.

[0017] Furthermore, if the controller body 11 has a rotationally symmetrical shape, it is preferable that the circuit section 12 be positioned on the axis of rotation. For example, if the controller body 11 is spherical, it is preferable that the center of gravity of the circuit section 12 be located at the center of the spherical housing 110 of the controller body 11. This is to ensure smooth rotation around the axis of rotation.

[0018] The control unit 121 of the circuit unit 12 includes a program control device such as a CPU and operates according to a program stored in the storage unit 122. In this embodiment, the control unit 121 accepts user input to the controller body 11. The control unit 121 also estimates the orientation of the controller body 11, and using this estimation result, generates information representing the content of the accepted operation and outputs it to the information processing device 30. The detailed operation of the control unit 121 will be described later.

[0019] The storage unit 122 is, for example, a memory device that holds programs executed by the control unit 121. This storage unit 122 also functions as the work memory for the control unit 121.

[0020] The sensor unit 123 includes a plurality of surface sensors 123a such as an electrostatic sensor, a pressure sensor, a strain sensor, a temperature sensor, and a sensor formed by a combination thereof, which can detect a force applied by a user in contact with or in proximity to the surface of the controller body 11, or a force applied by the user to the surface of the controller body 11.

[0021] Also, this sensor unit 123 is provided with an attitude sensor 123b including at least one of a position information sensor such as GPS, an acceleration sensor, a geomagnetic sensor, and a gyro sensor, which detects the attitude and movement of the controller device 10. This attitude sensor 123b may be, for example, a so-called 9-axis IMU sensor including all of an acceleration sensor, a geomagnetic sensor, and a gyro sensor. In the following example, it is assumed that the attitude sensor 123b is a 9-axis IMU sensor.

[0022] This attitude sensor 123b detects the acceleration in three axial directions (hereinafter referred to as ξ, η, and ζ axes) orthogonal to each other with the position of the attitude sensor 123b as the origin, the direction of geomagnetism (the direction of the north pole or the south pole) in the coordinate system (ξηζ orthogonal coordinate system, hereinafter referred to as the sensor coordinate system) defined by the three axes, and the rotational angular velocity around each of the ξ, η, and ζ axes, and outputs a signal representing the result of the detection to the control unit 121.

[0023] In an example of this embodiment, in the controller device 10, an analog multiplexer (MPX) may be used to sequentially select and output the output of any one of the acceleration sensor, the geomagnetic sensor, and the gyro sensor among the attitude sensors 123b, perform A / D conversion, and after performing predetermined signal processing such as filter processing by a DSP (Digital Signal Processer) or the like, output the result to the control unit 121.

[0024] The surface sensor 123a detects user contact with the controller body 11 or deformation operations performed by the user on the controller body 11, and outputs an electrical signal based on the detected contact or deformation operation to the control unit 121. The output of this surface sensor 123a may also be A / D converted, and after predetermined signal processing such as filtering by a DSP or the like, it may be output to the control unit 121. It is widely known that the system should include an oscillator circuit that generates a signal to be supplied to the capacitance sensor, so a detailed explanation is omitted here.

[0025] In one example of this embodiment, multiple surface sensors 123a are arranged on or near the surface of the controller body 11 (in a range at a predetermined distance from the surface), preferably evenly distributed across the entire surface. For example, these surface sensors 123a are positioned at the vertices of a regular octahedron or dodecahedron inscribed within the spherical controller body 11.

[0026] In the following, it is assumed that the position of each of the multiple surface sensors 123a in their respective sensor coordinate systems is known.

[0027] By using this surface sensor 123a, the control unit 121 can determine the user's (1) Operation by touching the controller body 11 (2) Pressing a part of the controller body 11 with your finger (3) The operation of bringing your hand closer to the controller body 11 This makes it possible to detect things like these. In addition, the position where the user touched, pressed, or brought their hand close to the sensor coordinate system can be detected from the position of the surface sensor 123a that detected them.

[0028] Furthermore, if the controller body 11 is made of an elastically deformable material, by using a surface sensor 123a that detects the direction and magnitude of the force applied by the user to the controller body 11, the control unit 121 can perform the operations (1) to (3) above, (4) The operation of rubbing the surface of the controller body 11 with the palm or fingers. (5) Twisting the controller body 11 (6) An operation to pinch a part of the controller body 11 (an operation to pinch and pull) (7) Pressing or squeezing the entire controller body 11 These can then be detected.

[0029] For example, operations (1) touching and (3) bringing a hand close can be detected by using a capacitive sensor as the surface sensor 123a.

[0030] Furthermore, (4) operations such as rubbing the surface, (5) twisting operations, (6) operations such as pinching a part of it (pinching and pulling operations), specifically operations such as pinching and pulling a part of it while holding the controller body 11 with one hand, and (7) operations such as crushing and squeezing, can be detected by using pressure sensors, strain sensors, acceleration sensors, etc. as surface sensors 123a. The position and direction in the sensor coordinate system of these operations, such as the position of rubbing, the direction of twisting, the position of pinching, and the direction of crushing, are detected by the position of the surface sensor 123a that detected them.

[0031] The communication unit 124 is an interface for wired connections such as USB, or wireless connections such as Bluetooth (registered trademark), and is connected to the information processing device 30 via wired or wireless connection. The communication unit 124 sends information to the information processing device 30 according to instructions input from the control unit 121. The communication unit 124 also outputs information received from the information processing device 30 to the control unit 121.

[0032] Next, the operation of the control unit 121 of the circuit unit 12 will be described. This control unit 121 executes a program stored in the memory unit 122, thereby realizing a configuration that functionally includes an operation reception unit 21, a posture estimation unit 22, an operation information generation unit 23, and a transmission unit 24, as illustrated in Figure 3.

[0033] The operation reception unit 21 receives user operations on the controller body 11. Specifically, based on the detection results output by each surface sensor 123a, the operation reception unit 21 detects user operations as exemplified in (1) to (7) above, and outputs information on the direction in which the controller body 11 is pressed as direction information in the sensor coordinate system.

[0034] The posture estimation unit 22 estimates the posture of the controller body 11 at a predetermined timing. This timing may be, for example, the timing when the operation reception unit 21 detects a predetermined initialization operation, or each time the operation reception unit 21 detects an operation. In one example of this embodiment, this initialization operation may be, for example, an operation in which a part of the controller body 11 is pressed with a force greater than a predetermined threshold for a predetermined time or longer, or an operation in which the controller is grasped and shaken a predetermined number of times (for example, twice) within an angle range greater than a predetermined angle range (back and forth within that angle range).

[0035] As an example, when the attitude estimation unit 22 detects an initialization operation, it uses the output of the attitude sensor 123b, which is output by the sensor unit 123, to detect the direction of gravity in the sensor coordinate system. Here, the method of detecting the direction of gravity using the 9-axis IMU sensor, which is the attitude sensor 123b, is widely known, so a detailed explanation will be omitted.

[0036] Furthermore, after detecting the direction of gravity, the attitude estimation unit 22 estimates the attitude of the controller body 11 using the output of the sensor unit 123. Specifically, after the initialization operation is performed and the direction of gravity is detected, the attitude estimation unit 22 instructs the information processing device 30 to output a guidance screen that guides the user to perform a predetermined operation. The predetermined operation guided here is, for example, an operation to apply acceleration in a predetermined direction to the controller device 10. As an example, the attitude estimation unit 22 instructs the information processing device 30 to display a message such as, "Hold the controller device 10 with your right hand and shake it from side to side." At this time, the user shall follow this instruction, grasp the controller device 10 with their right hand, and perform an action of shaking it from side to side from their perspective.

[0037] The posture estimation unit 22 uses the detection results output by the sensor unit 123 in response to the operations performed by the user, as guided by the sensor, to estimate the posture of the controller body 11. As an example, the posture estimation unit 22 uses the output of the surface sensor 123a to estimate the user's left-right direction.

[0038] This estimation is performed, for example, as follows: When the user picks up the controller body 11, the positions where the fingers are positioned and the surface sensors 123a located near or in contact with the palm of the hand detect the user's fingers and hand, as illustrated in Figure 4(a). In Figure 4, the left-right direction from the user's perspective is defined as the X-axis (right is the positive direction from the user's perspective), the normal direction to the coronal plane of the user's body is defined as the Y-axis (away from the user is the positive direction), and the direction of gravity is defined as the Z-axis (up is the positive direction). Hereafter, this XYZ Cartesian coordinate system will be referred to as the user coordinate system.

[0039] In Figure 4(a), for illustrative purposes, the surface sensors 123a are assumed to be located at the vertices of a regular octahedron inscribed within the spherical controller body 11. However, in reality, a larger number of surface sensors 123a may be arranged.

[0040] There are various ways in which the user can grasp the spherical controller body 11, but in order to grasp it with the right hand without unnaturally bending the wrist, (a) Grasp the controller body 11 from above (palm of hand is above the controller body 11) (b) Grasp the controller body 11 from below (palm of hand is below the controller body 11) (c) Grasp the controller body 11 from the right side (palm of hand is on the right side of the controller body 11) It will be one of the following.

[0041] For example, when a user grasps the controller body 11 from above, as illustrated in Figures 4(a) and (b), at least the surface sensors 123a-B located in the direction of gravity (negative Z-axis direction) from the center of the controller body 11 and the surface sensor 123a-P located on the user's body side will not detect contact with the user's fingers or hand H.

[0042] Meanwhile, the surface sensor 123a-L on the left side of the user and the surface sensor 123a-F on the opposite side of the body detect contact with the user's fingers, and the surface sensor 123a-T located above the controller body 11 detects contact with the palm of the hand.

[0043] When the user moves the controller body 11 to the left or right, the posture estimation unit 22 assumes that the surface sensors 123a-L and R in that direction are in the left or right direction of the user (i.e., that direction is parallel to the X-axis), and refers to the detection results of surface sensors 123a-P on the user's body side and surface sensor 123a-F on the opposite side of the body, which are different from surface sensors 123a-L and R and are in a direction perpendicular to the direction of gravity (i.e., in the horizontal plane).

[0044] Then, assuming the user's body is on the side where contact has not been detected (in this example, surface sensor 123a-P), the direction of the Y-axis is determined. The direction of the X-axis (in this example, the right side of the user is considered the positive direction) is then determined using the Y-axis direction determined here and the Z-axis direction determined by the direction of gravity.

[0045] Similarly, when the user grasps the controller body 11 from below (Figure 4(c)) or when the user grasps the controller body 11 from the right side (Figure 4(d)), the attitude estimation unit 22 determines the direction of the X, Y, and Z axes through the same process.

[0046] Note that this example is just one example; any method that can detect the user's body orientation, that is, determine the direction of the X, Y, and Z axes, may be used.

[0047] The attitude estimation unit 22 obtains a transformation formula (hereinafter referred to as the user coordinate system transformation formula; this transformation formula is expressed as a quaternion) that converts between the sensor coordinate system, which consists of each axis of the direction of acceleration detected by the attitude sensor 123b, and the user coordinate system (XYZ coordinate system) using the method described above. This user coordinate system transformation formula corresponds to attitude information that represents the attitude of the controller device 10 (each direction of the controller body as seen from the user).

[0048] Furthermore, the attitude estimation unit 22 obtains a conversion formula that converts between the direction detected by the attitude sensor 123b (direction of the North or South Pole) and the user coordinate system (this is called the direction conversion formula; this conversion formula is also expressed using quaternions).

[0049] The operation information generation unit 23 acquires at least the user coordinate system transformation formula obtained by the posture estimation unit 22, and each time the user performs an operation, it converts information representing the user's operation (all expressed in the sensor coordinate system), such as the position where the user's hand or fingers made contact detected by the operation reception unit 21, or the direction of the force the user applied to the controller body 11, into information representing the user's operation in the user coordinate system using the above transformation formula.

[0050] This allows us to obtain information that represents the user's operation, such as whether the pressure is being applied from the right or left side from the user's perspective.

[0051] The transmission unit 24 transmits information representing the user's operations in the user coordinate system, generated by the operation information generation unit 23, to the information processing device 30.

[0052] The information processing device 30 receives information from the controller device 10 that represents the user's operations in the user coordinate system, and uses it for processing such as controlling a game application.

[0053] [Example of operation] The controller device 10 of this embodiment basically has the above configuration and operates as follows. A user of this controller device 10 initially grasps the controller device 10 and performs a predetermined initialization operation.

[0054] The controller device 10 then performs an initialization process as illustrated in Figure 5 (S11), first detecting the direction of gravity in the sensor coordinate system using the output of the attitude sensor 123b.

[0055] This initialization operation involves moving the controller device 10 back and forth in the left-right direction relative to the user, then moving the controller device 10 back and forth in the front-back direction (sequentially swinging it in the left-right and front-back directions), or holding the controller device 10 in a stationary position with a predetermined surface facing the user's stomach (for example, pressing the predetermined surface against the user's stomach), thereby grasping the controller device 10 in a predetermined posture. Specifically, the controller device 10 communicates with the information processing device 30 and instructs the information processing device 30 to display a message such as, "Hold the controller device 10 with your right hand and swing it from side to side." The information processing device 30 responds to this instruction and displays a message such as, "Hold the controller device 10 with your right hand and swing it from side to side."

[0056] When the user follows this instruction and grasps the controller device 10 with their right hand and shakes it from side to side from their perspective, the controller device 10 detects the orientation of the X-axis based on the detection result of the user's hand and fingers contacting the surface sensor 123a and the detection result of the direction of movement by the posture sensor 123b. Here, the direction in which the controller device 10 first moves is defined as the negative X-axis direction.

[0057] Subsequently, the controller device 10 communicates with the information processing device 30 again and instructs the information processing device 30 to display a message such as, "Hold the controller device 10 in your right hand, first press it against your stomach, and then shake it back and forth." The information processing device 30 responds to this instruction and displays a message such as, "Hold the controller device 10 in your right hand, first press it against your stomach, and then shake it back and forth."

[0058] When the user follows these instructions, maintaining a grip on the controller device 10 with their right hand, presses the controller device 10 against their abdomen, and then swings it back and forth relative to themselves, the controller device 10 detects the direction of the user's body (negative Y-axis direction) based on the detection results of the user's hand and finger contact by the surface sensor 123a and the detection results of the direction of movement by the posture sensor 123b. In this example, the initial direction of movement is away from the user's body (positive Y-axis direction).

[0059] Specifically, as already mentioned, the controller device 10 detects the positive orientations of the X and Y axes from the direction of movement detected by the attitude sensor 123b.

[0060] The controller device 10 then defines the Z-axis direction as the direction orthogonal to both the detected X and Y axes, and determines the positive Z-axis direction, for example, assuming a left-handed system. This allows the controller device 10 to obtain vectors in the sensor coordinate system representing the positive directions of the X, Y, and Z axes in the user coordinate system. Finally, the controller device 10 obtains a user coordinate system transformation formula to convert between the sensor coordinate system and the user coordinate system (XYZ coordinate system), and completes the initialization process. In this example, the direction of gravity is the negative Z-axis direction in the user coordinate system when the user operates the controller device 10 while standing. When the user operates the controller device 10 while lying down, the direction of gravity is the negative Y-axis direction in the user coordinate system.

[0061] The controller device 10 repeats the following steps S12 to S15 until the user's operation is completed (for example, until a predetermined period of time has passed without the user's hand or fingers being detected).

[0062] The controller device 10 updates the user coordinate system transformation formula in accordance with the change in attitude of the controller device 10 after the initialization process (S12: Update of attitude information). This update method can be any widely known method (for example, the method disclosed at https: / / qiita.com / Tanba28 / items / 5092c3e5e2c631b804f3, etc.), such as a method using the output of the gyro sensor or a method using the azimuth transformation formula in addition to the output of the gyro sensor.

[0063] The controller device 10 acquires information representing user operations, such as the position where the user's hand or fingers are in contact with the controller body 11 as detected by the surface sensor 123a (all expressed in the sensor coordinate system) (S13), and converts the acquired information into information representing user operations in the user coordinate system using the user coordinate system transformation formula updated in step S12 (S14).

[0064] The controller device 10 then outputs the information representing the user's operations in the user coordinate system, obtained in step S14, to the information processing device 30 (S15).

[0065] The information processing device 30 receives information from the controller device 10 that represents the user's operations in the user coordinate system, and uses it for processing the control of the game application, etc.

[0066] [Reset via initial operation] The controller device 10 of this embodiment may also proceed to the initialization process (step S11) and continue processing whenever a predetermined initialization operation is performed. This allows the user to reset the operation of the controller device 10 at any time.

[0067] [Other examples of how to operate the controller device] The explanation so far has described examples of operations such as grasping the controller device 10 with one hand and pressing, gripping and deforming it with one of the fingers, or pinching a part of it. However, the methods of operating the controller device 10 are not limited to these examples.

[0068] As described above, the controller device 10 of this embodiment may be gripped not only with five fingers, but also with three fingers: the thumb, index finger, and middle finger. In either case, if the device is gripped naturally without twisting the wrist, it is assumed that, as illustrated in Figure 4, the palm of the right hand will be either facing upwards (Figure 4(b)), downwards (Figure 4(c)), or to the right (Figure 4(d)). Therefore, the controller device 10 can detect the direction of the palm depending on which direction the surface sensor 123a is detecting contact with the user's hand and fingers, and can also estimate the position of the thumb and the positions of each finger, such as the index finger and middle finger.

[0069] Therefore, when the controller body 11 is pressed at any of the estimated finger positions, the controller device 10 may send information to the information processing device 30 indicating which finger pressed the controller.

[0070] In this example, the information processing device 30 can process different operations depending on the settings of a game application, for example, when the thumb is pressed and when the index finger is pressed. In other words, in this example, no matter how the user holds the controller device 10, the user's finger positions on the controller body 11 will function in the same way as buttons on a conventional controller. Furthermore, by combining this with the press information, it can be used, for example, to determine whether the user is pressing with the optimal pressure using a specified finger in massage training software, or whether they are kneading bread with the optimal pressure or making rice balls with the correct hand shape in a cooking game.

[0071] Furthermore, since the controller body 11 of the controller device 10 in this embodiment preferably has a shape that is symmetrical about at least one axis, it may be rotated around this axis.

[0072] When a user rolls the controller device 10, for example, on their palm, the controller device 10 obtains information on the direction, amount, and speed of rotation by sequentially detecting the position of contact with the user's hand, or by updating the user coordinate system transformation formula as described earlier, such as the output of a gyro sensor. The controller device 10 transmits this information to the information processing device 30, which uses the information on the direction, amount, and speed of rotation of the controller device 10 for processing the control of the game application, etc. For example, it can be applied to games where a ball is rolled along a predetermined course, or games where the surface of an object is polished.

[0073] Furthermore, if the user is not wearing an HMD, or is wearing a transparent HMD, and the controller device 10 can be left out of the user's hands, the user may roll the controller device 10 on a desk or the floor. In this case, the controller device 10 obtains information on the direction of rotation, amount of rotation, and rotation speed by updating the user coordinate system transformation formula, such as the output of a gyro sensor, as described above.

[0074] Furthermore, the controller device 10 may be thrown for use. In this example, the controller device 10 is determined to be in the air from the point when the user's hand or fingers are in contact with it, until the point when the user's hand or fingers are no longer detected to the controller device 10, until an impact exceeding a predetermined acceleration is applied to the controller device 10 (when it lands or collides with something) (or until the controller device 10 comes to a stop, that is, until the user coordinate system transformation formula is no longer updated for a predetermined time due to the process of updating the user coordinate system transformation formula). While the controller device 10 is determined to be in the air, it detects its own movement trajectory and rotation using the output of the gyro sensor or the process of updating the user coordinate system transformation formula. The controller device 10 then transmits this detected movement trajectory and rotation information to the information processing device 30, which uses the movement trajectory and rotation information of the controller device 10 while it is in the air for processing the control of the game application, etc. This example can be applied to games that involve throwing a ball, such as baseball games or ball-throwing games. For example, the information processing device 30 may use information about the position of the user's finger in contact with the surface of the controller body 11 just before it is determined to be in the air, or use the finger position information along with information about the trajectory and rotation, to perform game processing as if a pitch of a curveball or forkball had been thrown.

[0075] Furthermore, while the above description has focused on examples where the user holds the device in their hand, the controller device 10 of this embodiment may also be used while placed on a desk or other surface. For example, it could be operated by crushing the controller device 10 placed on a desk, by manipulating an object by applying shearing force with the index finger, or by using it as a substitute for a computer pointing device (a so-called mouse) to control the movement of a cursor on a graphical user interface or the selection of radio buttons.

[0076] [If multiple controller devices exist] Furthermore, there may be two controller devices 10 connected to the information processing device 30, one for each user. In this case, when each controller device 10 communicates with the information processing device 30, it includes identification information uniquely assigned to itself in the information it sends to the information processing device 30.

[0077] In this case, the information processing device 30 may also guide the user by saying, "First, pick up the controller device in your right hand," and sequentially select the controller devices 10 to hold in each hand, and identify the identification information of the controller device 10 held in the user's right hand and the controller device 10 held in their left hand based on the order in which they were picked up.

[0078] In another example, the information processing device 30 may instruct the user to perform some operation on the controller devices 10 held in each hand, such as "Please tap the controller device in your right hand against the desk," and by detecting that the corresponding operation has been performed, it may identify the identification information of the controller device 10 held in the user's right hand and the identification information of the controller device 10 held in their left hand.

[0079] In examples where a user utilizes multiple controller devices 10, the user can perform an initialization operation on each controller device 10 at any time (for example, when switching the controller devices 10 held in the left and right hands). For this reason, the controller device 10 may be configured to accept the initialization operation and perform initialization processing when a predetermined operation is performed.

[0080] Furthermore, the number of controller devices 10 connected to the information processing device 30 does not have to be two per user; it may be three or more. In this example, the user performs an initialization operation on each controller device 10 in advance, places each controller device 10 within their reach, and picks up and operates the appropriate controller device 10. For example, in a game such as ball toss, it is conceivable that the user would sequentially throw multiple controller devices 10. [Explanation of Symbols]

[0081] 10 Controller device, 11 Controller main body, 12 Circuit unit, 21 Operation reception unit, 22 Pose estimation unit, 23 Operation information generation unit, 24 Transmission unit, 30 Information processing device, 31 Control unit, 32 Storage unit, 33 Communication unit, 34 Display unit, 110 Housing, 121 Control unit, 122 Storage unit, 123 Sensor unit, 124 Communication unit.

Claims

1. The controller itself, A sensor unit comprising one or more sensors connected to the controller body, It includes a control unit, The control unit includes an operation receiving means for receiving user operations on the controller body, Estimation means for estimating the orientation of the controller body, An operation information generation means generates and outputs information representing the content of the accepted operation using the estimation result of the estimation means, A controller device including guidance means for providing guidance to the user on how to apply acceleration or movement in a predetermined direction to the controller body, The estimation means estimates the orientation of the controller body using the detection output of the sensor unit in response to the operation guided by the guidance means. The guiding means further provides the user with additional guidance regarding the operation of applying acceleration in the direction normal to the coronal plane of the user's body, which is perpendicular to the predetermined direction. The estimation means estimates the orientation of the controller body using the detection output of the sensor unit in response to an operation that applies acceleration in the predetermined direction and the normal direction guided by the guidance means. Controller device.

2. A controller device according to claim 1, The one or more sensors in the sensor unit are Accelerometer, Geomagnetic sensor, and Gyroscope sensor, It is equipped with at least one of the following sensors: The estimation means is a controller device that estimates the orientation of the controller body using the detection output of the sensor unit at the time the user's operation is accepted.

3. A controller device according to any one of claims 1 to 2, The controller body is a controller device formed from an elastically deformable material.

4. A controller device according to any one of claims 1 to 3, The controller body is a controller device having a shape that is rotationally symmetrical about at least one axis.

5. A controller device according to claim 4, The controller body is a spherical controller device.

6. The controller itself, A sensor unit comprising one or more sensors connected to the controller body, Means of guidance, A control method for a controller device comprising a control unit, wherein the control method is The control unit accepts user input to the controller body, The orientation of the controller body is estimated, Using the estimated posture, information representing the content of the accepted operation is generated and output. The guidance means provides the user with instructions on how to apply acceleration or movement in a predetermined direction to the controller device. This includes, Estimating the orientation of the controller body includes using the detection output of the sensor unit in accordance with the operation guided by the guidance means, The guiding means further provides the user with additional guidance regarding the operation of applying acceleration in the direction normal to the coronal plane of the user's body that is perpendicular to the predetermined direction, The control unit includes estimating the orientation of the controller body using the detection output of the sensor unit in response to an operation in which acceleration is applied in the predetermined direction and the normal direction guided by the guide means. A control method for a controller device.

7. The controller itself, A sensor unit comprising one or more sensors connected to the controller body, The control unit of the controller device, which includes the following, An operation receiving means for receiving user operations on the controller body, Estimation means for estimating the orientation of the controller body, An operation information generation means generates and outputs information representing the content of the accepted operation using the estimation result of the estimation means, Guidance means for providing the user with instructions on how to apply acceleration or movement in a predetermined direction to the controller device. It is a program that makes it function as such. The estimation means estimates the orientation of the controller body using the detection output of the sensor unit in response to the operation guided by the guidance means. The guiding means further provides the user with additional guidance regarding the operation of applying acceleration in the direction normal to the coronal plane of the user's body, which is perpendicular to the predetermined direction. The estimation means estimates the orientation of the controller body using the detection output of the sensor unit in response to an operation that applies acceleration in the predetermined direction and the normal direction guided by the guidance means. program.

Images