Controller device

Abstract

This controller device presents a force to sense to a user by means of: a support body 10 including a grip part 11 gripped by the user; and a force sense presentation part 30 which is coupled to the support body 10 via a joint and is a movable weight that is movable relative to the position where the joint is coupled.

Description

Controller Device

[0001] The present invention relates to a controller device that is operated by a user.

[0002] In recent years, controller devices capable of presenting a force sense to a user have become widely used.

[0003] However, providing a variety of force feedback to provide a rich user experience increases the cost of the device.

[0004] The present invention has been made in view of the above circumstances, and one of its objects is to provide a controller device that can realize the presentation of a variety of force sensations at a relatively low cost.

[0005] One aspect of the present invention for solving the problems of the above-mentioned conventional examples is a controller device that includes a support body including a gripping portion that is held by a user, and a force sense presentation unit that is connected to the support body via a joint and has a force sense presentation means that presents a force sense to the user by means of a movable weight that is movable relative to the position to which the joint is connected.

[0006] Another aspect of the present invention is a controller device that includes a support body including a grip portion that is held by a user, a center of gravity adjustment body that is connected to the support body via a joint that is rotatable around multiple axes and has a weight that can be positioned at a point a predetermined distance from the rotation axis of the joint, and a control circuit that controls the rotation of the first and second joints.

[0007] According to the present invention, it is possible to realize the presentation of a variety of force sensations at a relatively low cost.

[0008] FIG. 1 is a configuration block diagram showing an example of a system including a controller device according to an embodiment of the present invention. FIG. 2 is a schematic configuration diagram showing an example of a controller device according to an embodiment of the present invention. FIG. 3 is a schematic explanatory diagram showing an example of the configuration of a force feedback unit of a controller device according to an embodiment of the present invention. FIG. 4 is a block diagram showing an example of the configuration of a circuit unit provided in a controller device according to an embodiment of the present invention. FIG. 5 is a schematic configuration diagram showing another example of a controller device according to an embodiment of the present invention. FIG. 6 is an explanatory diagram showing an example of the operation of a controller device according to an embodiment of the present invention. FIG. 7 is an explanatory diagram showing example contents of instructions to a controller device according to an embodiment of the present invention. FIG. 8 is an explanatory diagram showing another example of the operation of a controller device according to an embodiment of the present invention.

[0009] An embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, a controller device 1 according to an embodiment of the present invention is communicably connected to an information processing device 2, such as a home game console or a personal computer, via wired or wireless communication. Note that the following description and drawings are merely examples, and the size and proportions of each part may be changed as appropriate.

[0010] 2, one aspect of the controller device 1 includes a support 10, a joint unit 20, and a force feedback unit 30. The support 10 also includes a grip unit 11.

[0011] In one example of this embodiment, the joint unit 20 includes a base 21, an extension 22, and at least one joint element 23. The joint unit 20 may also include an actuator 25 that corresponds to at least one of the joint elements 23 and drives the joint elements 23 to change their rotation angle or movement amount. However, if each joint element 23 of the joint unit 20 is adjusted manually, the actuator 25 is not necessarily required.

[0012] In the following example, the joint elements 23 include a rotary joint element 23a and a linear joint element 23bb. However, this is just an example, and other joint elements 23 may be provided instead of or in addition to these.

[0013] The extension 22 of the joint unit 20 is, for example, a columnar body, with a base 21 disposed at one end and a rotary joint element 23a disposed at the other end. The rotary joint element 23a rotates around an axis perpendicular to the longitudinal axis of the extension 22 within at least a predetermined angular range (α in FIG. 2). A linear joint element 23b is disposed on the side of the rotary joint element 23a facing the extension 22, allowing the relative position of the rotary joint element 23a and the haptic unit 30 to be moved within a predetermined range on a rail (corresponding to a connecting element) disposed along the longitudinal axis (z-axis in FIG. 2) of the main body 31 of the haptic unit 30 (described later) (indicated by an arrow in FIG. 2). The linear joint element 23b can employ a widely known mechanism, such as a rack-type rail engaged with a pinion.

[0014] The gripping portion 11 of the support body 10 has a cylindrical shape. One end 11e of this gripping portion 11 is fixed to the base portion 21 of the joint portion 20. The force sense presentation unit 30 also has a cylindrical shape, and if the linear joint element 23b of the joint portion 20 is realized by a mechanism using a rack and pinion, a rack can be arranged along the outer periphery of the force sense presentation unit 30 in the longitudinal direction, and the pinion can be engaged with this rack. This connects the linear joint element 23b of the joint portion 20 to the force sense presentation unit 30 so as to be able to drive the force sense presentation unit 30 in the axial direction. In one example of this embodiment, the force sense presentation unit 30 and the support body 10 are connected to each other via the joint portion 20 in this manner.

[0015] In this embodiment, the rotary joint element 23a and the linear joint element 23b make it possible to change the longitudinal position of the force sense presentation unit 30 and the position of the rotation axis, and also to change the rotation angle.

[0016] 3 , the force sense presentation unit 30 includes a cylindrical main body 31 (corresponding to the force sense presentation unit main body) and a force sense presentation element 32 built into the main body 31. The force sense presentation element 32 also includes electromagnets 321 and 322 disposed at both axial ends of the main body 31, and a weight 323 disposed movably within the cylindrical main body 31. The weight 323 includes a columnar main body 323a and a sliding part 323b wound around the outer periphery of the main body. The outer diameter of the sliding part 323b is slightly smaller than the inner circumference of the main body 31, thereby allowing the weight 323 to slide axially within the main body 31. The main body 323a is magnetized with a north pole on one side and a south pole on the other side in the longitudinal direction. The weight 323 corresponds to the movable weight of the present invention that can move relative to the position where the joint (corresponding to the joint unit 20 in this embodiment) is connected.

[0017] In one example of this embodiment, the weight of this weight 323 is set to be approximately the same as or greater than the weight of the support body 10, and is also set to be greater than the weight of the joint unit 20. In this way, the position of the center of gravity of the entire controller device 1 can be changed by moving the position of the force sense presentation unit 30 by the linear joint element 23b, or by driving the weight 323 of the force sense presentation unit 30. The force sense presentation unit 30 in this example corresponds to an example of a center of gravity adjusting body of the present invention.

[0018] Furthermore, in this embodiment, a circuit section 40 is disposed inside the controller device 1, such as in the grip section 11 of the support body 10. As illustrated in Fig. 4, this circuit section 40 includes a control section 41, a storage section 42, an interface section 43, a sensor section 44, and a communication section 45.

[0019] Here, the control unit 41 is a program-controlled device such as a CPU, and operates according to a program stored in the storage unit 42. In this embodiment, the control unit 41 receives information (referred to as operation information) that indicates at least one of the attitude of the grip unit 11, the grip pressure (grip pressure) applied by the user, the movement direction and movement speed of the grip unit 11 (which can be obtained by a gyro sensor or the like) detected by a sensor unit 44 (described later), and the rotation angle and position of each joint element 23, and sends the information detected by the sensor unit 44 to the information processing device 2 via the communication unit 45.

[0020] The control unit 41 also receives instructions input from the information processing device 2 via the communication unit 45, and controls the force-feedback supply unit 30 in accordance with the received instructions. The operation of the control unit 41 will be described later.

[0021] The storage unit 42 is a memory device or the like including a non-volatile memory, and holds a program executed by the control unit 41. This program may be provided by being stored in a computer-readable, non-transitory recording medium, and may be stored in the storage unit 42. The storage unit 42 also operates as a work memory for the control unit 41.

[0022] The interface unit 43 is interposed between the control unit 41 and peripheral devices such as the sensor unit 44, and outputs information output from the peripheral devices such as the sensor unit 44 to the control unit 41. The interface unit 43 also sends instructions output by the control unit 41 to peripheral devices designated by the control unit 41.

[0023] In one example of the present embodiment, the sensor unit 44 includes an acceleration sensor that detects the attitude of the grip unit 11 and a strain sensor that detects the grip pressure of the grip unit 11 by the user, and outputs operation information detected by these sensors to the control unit 41. In one example of the present embodiment, the acceleration sensor detects the direction of gravity, and detects the degree to which the longitudinal axis of the grip unit 11 is tilted with respect to the direction of gravity (an example of the attitude of the grip unit 11), and outputs the detected information to the control unit 41.

[0024] The sensor unit 44 also detects information on the displacement (rotation angle, movement amount, etc.) for each degree of freedom of each joint element 23 and outputs the detected information to the control unit 41 .

[0025] The communication unit 45 is a wired interface such as a USB (Universal Serial Bus) interface or a wireless communication interface such as Bluetooth (registered trademark), and transmits and receives information to and from the information processing device 2 .

[0026] The controller device 1 of this embodiment further includes a rechargeable power supply unit, which supplies power to each part to operate it; however, since a widely known power supply unit can be used for such a power supply unit, detailed explanation will be omitted here.

[0027] Next, an example of the control operation of the control unit 41 on the force sense presentation unit 30 will be described. In one example of this embodiment, the information processing device 2 outputs an instruction for the force sense to be presented by the controller device 1. As in the example already described, if the force sense presentation unit 30 presents the force sense by controlling the vertical movement of the weight 323 with the voltage applied to the electromagnets 321 and 322, this instruction includes information specifying the change over time in the voltage applied to the electromagnets 321 and 322 (such as the magnitude (peak value) of the voltage, waveform shape, frequency, and duty ratio).

[0028] The control unit 41 controls the voltages applied to the electromagnets 321 and 322 of the force sense presentation unit 30 in accordance with this instruction. The control unit 41 of this embodiment uses the instructed peak value A and frequency ω to set the voltage V applied to the electromagnets 321 and 322 as follows: V = A sin(2πωt). At this time, the control unit 41 controls the electromagnets 321 and 322 so that the poles facing the weight 323 always have the same polarity. This makes it possible to present force senses such as, for example, periodically moving the weight 323 up and down within the main body 31 of the force sense presentation unit 30 (thereby making it possible to change the center of gravity), rapidly moving the weight 323 to present vibrations, or moving it in one direction to cause the weight 323 to collide with one of the electromagnets 321 and 322 to present an impact.

[0029] Furthermore, the rotary joint element 23a and the linear joint element 23b may each be provided with a locking mechanism that fixes the joint at a fixed rotation angle and at a fixed position (such locking mechanisms are widely known, and therefore will not be described in detail here).

[0030] In this example, the user fixes the rotary joint element 23 a and the linear joint element 23 b at a desired rotation angle and at a desired position. As an example, by adjusting the rotation angle of the rotary joint element 23 a and setting it so that the longitudinal direction of the grip unit 11 and the longitudinal direction of the force sense presentation unit 30 are parallel (so that the grip unit 11 and the force sense presentation unit 30 are arranged on a straight line), the user will feel a sense of center of gravity as if they were holding a sword, while on the other hand, by setting it so that the longitudinal direction of the force sense presentation unit 30 is perpendicular to the longitudinal direction of the grip unit 11, when the weight 323 moves within the main body 31 of the force sense presentation unit 30, the user will receive a force sensation similar to that of firing a gun.

[0031] Furthermore, if the user adjusts the rotation angle of the rotary joint element 23a so that the longitudinal direction of the gripping unit 11 and the longitudinal direction of the force feedback unit 30 are parallel, and then adjusts the position of the linear joint element 23b so that the range of movement of the weight 323 is farther away from the gripping unit 11, the force feedback will be presented while feeling the center of gravity at a greater distance, as if swinging a long sword, due to the movement of the weight 323. On the other hand, if the position of the linear joint element 23b is adjusted so that the range of movement of the weight 323 is closer to the gripping unit 11, the movement of the weight 323 will cause the center of gravity to fluctuate relatively closer to the user, and the force feedback will be presented as if using a dagger.

[0032] As described above, according to this embodiment, it is possible to realize the presentation of a variety of force sensations at a relatively low cost.

[0033] [Another example of a joint] In the explanation so far, the joint unit 20 has been described as using the rotary joint element 23a and the linear joint element 23b to change the angle between the gripping unit 11 and the force feedback unit 30 (the intersection angle of each axis when each longitudinal axis is virtually extended) and the distance between the gripping unit 11 and the force feedback unit 30, but the configuration of the joint unit 20 is not limited to this example.

[0034] For example, the joint unit 20 may include a gimbal, and the angle between the grip unit 11 and the force feedback unit 30 may be changed about two axes (a first rotation axis in the longitudinal direction of the extension unit 22 and a second rotation axis perpendicular to the first rotation axis). Alternatively, instead of a gimbal, a spherical omnidirectional gear mechanism may be used to change the angle between the grip unit 11 and the force feedback unit 30 about two or more axes, including the first and second rotation axes (spherical omnidirectional gear mechanisms are described in detail in Tadakuma, Kenjiro et al., Study on a Spherical Omnidirectional Gear Mechanism with Two Degrees of Freedom, Journal of the Robotics Society of Japan, Vol. 36, No. 9, pp. 49-, 2018, and so a detailed description thereof will be omitted here).

[0035] [Example of Controlling a Joint] As already described, in this embodiment, the rotation angle and movement amount of the rotary joint element 23 a and the linear joint element 23 b of the joint unit 20, or a gimbal or spherical omnidirectional drive gear mechanism that replaces them, may be controlled by the actuator 25.

[0036] The actuator 25 of the joint unit 20 in this example is controlled by, for example, the control unit 41. Also in this example, the information processing device 2 instructs the rotation angle of the rotary joint element 23a and the movement position of the linear joint element 23b.

[0037] The control unit 41 drives the actuator 25 to rotate the rotary joint element 23a around the rotation axis to the rotation angle instructed by the information processing device 2, and moves the position of the linear joint element 23b to the instructed position. Thereafter, the control unit 41 controls the actuator 25 to maintain the rotation angle and position.

[0038] Furthermore, even when the joint part 20 is a gimbal or the like instead of being configured to include a rotary joint element 23a or a linear joint element 23b, instructions for the respective drive amounts (amount of rotation or amount of movement) are received from the information processing device 2 according to the degree of freedom of the joint part 20, the actuator 25 of the joint part 20 is driven in accordance with the instructions to control the rotation angle or amount of movement, and then the actuator 25 is driven to maintain the rotation angle or amount of movement.

[0039] 5, the joint unit 20 may include a plurality of extension units 22 connected in a row via joint elements 23. In this example, the joint elements 23 may be provided between the grip unit 11 and the extension units 22, and between the extension units 22 and the force feedback unit 30, in addition to between the adjacent extension units 22.

[0040] In this example, the joint elements 23 between adjacent extension units 22 or between the grip unit 11 and the extension unit 22 may be, for example, spherical omnidirectional gear mechanisms, and may be joints that are rotatable about a first rotation axis in the longitudinal direction of the extension unit 22 and a second rotation axis perpendicular to the first rotation axis. Similarly, the joint element 23 disposed between the extension unit 22 and the force sense presentation unit 30 may be a spherical omnidirectional gear mechanism, and may be a joint that is rotatable about a first rotation axis in the longitudinal direction of the extension unit 22 and a second rotation axis perpendicular to the first rotation axis, or may include a rotary joint element 23 a and a linear joint element 23 b, as in the example already described.

[0041] Furthermore, each joint element 23 in this example may be driven by an actuator 25 provided corresponding to the degree of freedom of each joint element 23, making it possible to control the rotation angle, movement position, etc. In this case, the information processing device 2 instructs the rotation angle and movement amount of each joint element 23, and the control unit 41 controls the drive amount and drive direction of the corresponding actuator 25 based on the instruction received from the information processing device 2. This control can be performed by well-known feedback control or the like, using information detected by the sensor unit 44.

[0042] [Operation Example] The controller device 1 of this embodiment basically has the above-described configuration and operates as follows: In the following example, as illustrated in Fig. 5 , the joint unit 20 of the controller device 1 has two columnar extension units 22, a first extension unit 22a and a second extension unit 22b, and the grip unit 11 of the support body 10, the two extension units 22, and the main body unit 31 of the force feedback unit 30 are connected in a row via joint elements 23.

[0043] That is, in this example, one longitudinal end of the gripping portion 11 and one longitudinal end of the first extension portion 22a are connected via joint element 23a, the other longitudinal end of the first extension portion 22a and one longitudinal end of the second extension portion 22b are connected via joint element 23b, and the other longitudinal end of the second extension portion 22b and the main body portion 31 of the force feedback unit 30 are connected via joint elements 23c and 23d.

[0044] In the following example, both joint elements 23a and 23b are assumed to be joints that can rotate around a first rotation axis in the longitudinal direction of extension portion 22 and a second rotation axis perpendicular to the first rotation axis. The longitudinal axis of the corresponding extension portion 22 is taken as the rotation axis, the rotation angle around this rotation axis is taken as θ, and the angle (tilt angle) between the axial directions of the two connected members (for example, the longitudinal directions of grip portion 11 and extension portion 22) is taken as φ.

[0045] Furthermore, joint element 23c is a rotary joint element, and joint element 23d is a linear joint element. Joint element 23c, which is a rotary joint element, connects the other longitudinal end of second extension portion 22b to joint element 23d so that the other longitudinal end of second extension portion 22b can rotate around a predetermined axis (the longitudinal axis of second extension portion 22b). Joint element 23d connects joint element 23c so that the position of the center of rotation is movable on the surface of main body portion 31 of force sense presentation unit 30 within a predetermined range along the longitudinal axis of main body portion 31 (including the longitudinal center position of main body portion 31, with this position being the origin ζ = 0). Such a connection can be realized by a mechanism using a rack and pinion, as in the example already described.

[0046] Initially, the control unit 41 of the controller device 1 sets θ=0 and φ=0 for both of the joint elements 23: the joint element 23a interposed between the gripping portion 11 and the first extension portion 22a, and the joint element 23b interposed between the first extension portion 22a and the second extension portion 22b.

[0047] Furthermore, the rotation angle α of the joint element 23c interposed between the second extension portion 22b and the main body portion 31 of the force feedback unit 30 is set to 0 (at this time, the longitudinal direction of the main body portion 31 is parallel to the direction of θ=0), and ζ=0.

[0048] At this time, the longitudinal directions of the gripping portion 11 and the two extension portions 22 of the support body 10 are all parallel, the main body portion 31 of the force feedback unit 30 is in a direction perpendicular to the longitudinal direction of the gripping portion 11, and the center of the main body portion 31 is located on an extension line of the longitudinal direction of the gripping portion 11.

[0049] The control unit 41 of the controller device 1 acquires operation information regarding the attitude of the grip unit 11 and information regarding the rotation angle and movement position of each joint element 23 from the sensor unit 44 , and sends the acquired information to the information processing device 2 .

[0050] The information processing device 2 executes an application program such as a game, acquires operation information and the like from the controller device 1, and uses the acquired information for processing the application program.

[0051] For example, the information processing device 2 determines the direction of movement (within the virtual space of the game) of a virtual object appearing in the game, such as a fighter jet in a game in which a player controls a fighter jet, based on the attitude, movement direction, and movement speed of the grip unit 11. Furthermore, the information processing device 2 determines the type and direction of the force sensation to be presented to the user through application program processing, and sends instructions to the controller device 1 to present the force sensation of the determined type and direction.

[0052] [Presentation of inertia] Specifically, in a game in which the direction of travel of a virtual object is controlled, the user is instructed to hold the gripping portion 11 so that its longitudinal direction faces forward and is parallel to the floor, and to control the direction of travel of the virtual object by turning the controller device 1 left or right.

[0053] 6, the information processing device 2 instructs that the inclination angle φ1 for the joint element 23a be 60 degrees, the inclination angle φ for the joint element 23b be 120 degrees, and also instructs that the rotation angle of the joint element 23c, which is a rotational joint element, be α = 60 degrees, and the position of the joint element 23d be ζ ​​= 0. In this way, the center of the main body 31 of the force sense presentation unit 30 is positioned on the longitudinal axis of the gripping unit 11, and the shortest distance from the end of the gripping unit 11 on the force sense presentation unit 30 side to the weight 323 inside the force sense presentation unit 30 is shorter than when the inclination angles φ1 = φ2 = 0 for the joint elements 23a, b are set as shown in FIG.

[0054] The information processing device 2 executing the game application in the above example instructs the user to hold the controller device 1 so that the longitudinal direction of the main body 31 of the force feedback unit 30 is parallel to the floor surface and parallel to the left-right direction of the user.

[0055] Thereafter, when the user moves the grip unit 11 in the left-right direction, the sensor unit 44 detects the movement, and the control unit 41 sends information representing the movement to the information processing device 2. The information processing device 2 receives this information and controls the direction of travel of the virtual object, and also instructs the controller device 1 to move the weight 323 of the force feedback unit 30 in the direction opposite to the movement represented by the received information.

[0056] The control unit 41 of the controller device 1 drives the weight 323 of the force feedback unit 30 in accordance with an instruction received from the information processing device 2. In this example, the information processing device 2 moves the weight 323 in the opposite direction to the user's movement, so that the user receives a force feedback that feels as if inertia is acting on the operation (as if a force is being applied in a direction opposite to the operation direction).

[0057] Furthermore, even when a similar inertia is presented, as illustrated in Figure 5, if the inclination angles of each joint element 23a, b are φ1 = φ2 = 0, the center of the main body 31 of the force feedback unit 30 is located on the longitudinal axis of the gripping unit 11, as in the example of Figure 6, but the shortest distance from the end of the gripping unit 11 on the force feedback unit 30 side to the weight 323 inside the force feedback unit 30 is longer than in the example of Figure 6.

[0058] At this time, when the user grips the grip unit 11 of the controller device 1 and moves the grip unit 11 left and right in the same manner as in the above example, the sensor unit 44 detects the movement, and the control unit 41 sends information representing the movement to the information processing device 2. The information processing device 2 receives this information and controls the direction of travel of the virtual object, and also instructs the controller device 1 to move the spindle 323 of the force sense presentation unit 30 in the direction opposite to the movement represented by the received information. The control unit 41 of the controller device 1 then drives the spindle 323 of the force sense presentation unit 30 in accordance with the instruction received from the information processing device 2.

[0059] In this case, the user feels a force sensation as if the inertia is acting while holding a rod that is longer than the example in Fig. 6. This example can be applied to games such as balance games in which a long rod must be kept in a stable position.

[0060] [Another example of driving the force feedback unit] In the examples of Figures 5 and 6, the information processing device 2 uses information on the attitude of the grip unit 11, the direction of movement, and the speed of movement sent by the control unit 41 of the controller device 1 to give instructions to present a force feedback that is opposite to the direction of movement (M in Figure 7) from the start of movement until time t0, as illustrated in Figure 7, and then gives instructions to present a force feedback by moving the weight 323 in the direction of movement at a predetermined speed or a speed determined based on the speed of movement (represented by slope a).This means that the user holding the controller device 1 will be presented with a force feedback that feels as if the controller device 1 is bending and moving (as if the movement is following with a delay).

[0061] FIG. 7 is an explanatory diagram that schematically shows an example of time-series control of the movement amount of the weight 323 relative to the movement direction of the controller device 1.

[0062] By appropriately changing force parameters such as the time t0 until tracking begins, the amount of movement r of the weight 323 in the opposite direction (to the direction of movement of the controller device 1) before tracking begins, and the speed of tracking (slope a of the amount of movement), the information processing device 2 can present a force sensation to the user as if they were holding a virtual object such as a whip, fishing rod, morning star, handbag, lasso, or chain.

[0063] For example, when a user selects a weapon in a game application, the controller device 1 sets the above-mentioned force parameters in accordance with the selected weapon and instructs the controller device 1 to control the amount of movement of the weight 323.

[0064] [Time-series control of joint elements] Furthermore, in one example of this embodiment, in the controller device 1 illustrated in FIGS. 5 and 6 , by appropriately setting the inclination angles φ1 and φ2 of the joint elements 23 a and 23 b and controlling the rotation angle α of the joint element 23 c, the controller device 1 is made L-shaped with the grip portion 11 as the short side, and the longitudinal direction of the main body 31 of the force feedback unit 30 is made substantially parallel to the long side of the L shape (a direction substantially perpendicular to the longitudinal direction of the grip portion 11), a user holding this grip portion 11 will receive a force feedback as if they were holding a gun with a relatively long barrel.

[0065] In this state, when the user performs an operation to shoot, the information processing device 2 receives the operation, executes processing related to the game, and instructs the controller device 1 to change the tilt angle φ1 of the joint element 23a beyond 0 degrees, as illustrated in Fig. 8, and to control the tilt angle φ2 of the joint element 23b and the rotation angle α of the joint element 23c to maintain the longitudinal axis of the main body 31 of the force sense presentation unit 30 without fluctuating from the beginning (S1). Furthermore, in addition to these, the information processing device 2 instructs the controller device 1 to collide the weight 323 inside the main body 31 of the force sense presentation unit 30 with one end of the main body 31 to generate an impact, at the timing when these fluctuations end.

[0066] When the controller device 1 controls each part in accordance with the instructions in this example, the user holding the controller device 1 will experience a relatively strong force sensation as if they were knocked back by a gunshot. Furthermore, if the information processing device 2 instructs the controller device 1 to simply cause the weight 323 inside the main body 31 of the force sense presentation unit 30 to collide with one end of the main body 31 during a gunshot operation, without changing the tilt angle or rotation angle of each joint element 23, it is possible to present a force sensation to the user that they are experiencing recoil from the gunshot but are not being knocked back.

[0067] Furthermore, if the information processing device 2 is instructed to vary the amount of change in the tilt angle and rotation angle of each joint element 23, and to move the force feedback unit 30 as far away as possible from the gripping unit 11 to generate an impact with the weight 323 (S2), the user holding the controller device 1 that operates in accordance with this instruction will feel a stronger knockback.

[0068] During these controls, the information processing device 2 may instruct the speed of change (amount of change per unit time) of the tilt angle, rotation angle, movement amount, etc. of the joint element 23. By increasing the amount of change per unit time (increasing the speed of change), it becomes possible to present a force sensation such that the user holding the controller device 1 following this instruction feels a stronger impact.

[0069] 5 and 6, the center of gravity of the controller device 1 is located on an extension of the grip portion 11 in the longitudinal direction, and is relatively far away in Fig. 5 and relatively close in Fig. 6. In Fig. 8, the approximate center of gravity of each device is indicated by point G. In this way, in the example of this embodiment, the center of gravity can be changed to match the shape of a virtual object, and the position of the center of gravity can also be changed over time.

[0070] [Recognition of shape, position, and posture using a camera] In the examples described above, the information processing device 2 recognizes the posture of the grip portion 11 of the controller device 1, the displacement of each joint element 23, and even the shape of the controller device 1 based on the information received from the controller device 1, but the present embodiment is not limited to this.

[0071] For example, the information processing device 2 may be equipped with a camera C, which captures an image of the user and the controller device 1 held by the user, and from the image capture results, recognize the attitude of the grip unit 11 of the controller device 1, the displacement of each joint element 23, the shape of the controller device 1, and the relative position between the user and each part of the controller device 1. This recognition process may be performed by arranging machine-recognizable markers on each part of the controller device 1 and recognizing the position of the markers, etc. Furthermore, this recognition process may be performed using a machine learning model obtained by machine learning the relationship between the image and the attitude, shape, etc. of the grip unit 11 of the controller device 1.

[0072] Furthermore, camera C may be capable of recognizing depth (distance from camera C), in which case the depth information may be further used to recognize the posture of the grip portion 11 of the controller device 1, the displacement of each joint element 23, and even the shape of the controller device 1, as well as the relative position between the user and each part of the controller device 1.

[0073] [Expression that does not make the user feel the weight] Furthermore, the information processing device 2 may control the displacement of the joint elements 23 of the controller device 1 (the amount of displacement according to each degree of freedom, such as the tilt angle, rotation angle, movement position, etc.) and the state of the force feedback in the force feedback unit 30 (such as the movement and position of the weight 323) so that the user does not feel the weight of the controller device 1 as much as possible.

[0074] Specifically, when the information processing device 2 attempts to control the controller device 1 so that the user does not feel its weight as much as possible, it moves the center of gravity of the controller device 1 as close as possible to the position of the user's hand (the grip portion 11), as illustrated in Figure 6.

[0075] Alternatively, if the information processing device 2 can recognize the position of the user's shoulder and the positions of each part of the controller device 1 from an image captured by the camera C or the like, it may sequentially instruct the displacement of the joint element 23 of the controller device 1, and control the controller device 1 so that the center of gravity of the controller device 1 approaches as close as possible to an axis in the vertical direction (a direction parallel to the direction of gravity) from the user's hand or shoulder. For example, if the center of gravity can be set within a predetermined distance range from the axis in the vertical direction of the user's shoulder, the information processing device 2 controls the controller device 1 by sequentially instructing the displacement of the joint element 23 of the controller device 1 so that the center of gravity of the controller device 1 approaches the axis in the vertical direction of the shoulder, and if the center of gravity cannot be set within the predetermined distance range from the axis in the vertical direction of the shoulder, the information processing device 2 sequentially instructs the displacement of the joint element 23 of the controller device 1 so that the center of gravity of the controller device 1 approaches the axis in the vertical direction relative to the user's hand.

[0076] A user holding a controller device 1 following this instruction will feel a weight in a direction close to the vertically downward position of the hand or shoulder, for example. However, since this direction is the direction in which the weight of the arm or hand is perceived in the first place, it is possible to express the weight of the controller device 1 itself in a way that minimizes the sense of it.

[0077] [Priority of center of gravity and force feedback] The controller device 1 of this embodiment is capable of controlling the center of gravity position and the direction of force feedback (direction of movement of the weight 323) as described above, but due to limitations in the displacement of each joint element 23, there are cases where, when the center of gravity position is moved to a desired position, the direction of force feedback cannot be set to the desired direction, or when the direction of force feedback is set to the desired direction, the center of gravity position cannot be set to the desired position.

[0078] In these cases, the information processing device 2 may (1) in a force sense direction priority mode, first align the axial direction of the main body part 31 of the force sense presentation unit 30 with the desired direction, and then control the displacement of each joint element 23 so that the center of gravity position is as close as possible to the desired position, or (2) in a gravity center priority mode, first control the displacement of the joint element 23 so that the center of gravity position is the desired position, and then control the displacement of the joint element 23 between the joint part 20 and the force sense presentation unit 30 so that the axial direction of the main body part 31 of the force sense presentation unit 30 is as close as possible to the desired direction.

[0079] Furthermore, (3) as an average mode, the displacement of each joint element 23 may be controlled to minimize the average (which may be a weighted average according to the importance of each, or may be a normal arithmetic average, or other statistical value) of the difference between the axial direction of the main body 31 of the force feedback unit 30 and its desired direction, and the difference between the center of gravity position and its desired position.

[0080] The displacement of each of these joint elements 23 may be obtained by analyzing, using an inverse kinematic model, the direction of the force to be virtually presented by the force sense presentation unit 30, and the relative relationship between the center of gravity position changed by the displacement of the joint elements 23 and the attitude (longitudinal direction) of the grip unit 11. A widely known method can be used to calculate the displacement of the joint elements 23 using such an inverse kinematic model.

[0081] [Grip pressure of gripping portion] Furthermore, if the controller device 1 is equipped with a sensor unit 44 that detects the grip pressure of the gripping portion 11 (the magnitude of the user's grip force) and the control unit 41 sends information on the detected grip pressure, the information processing device 2 may use this grip pressure information to correct the force sensation to be presented or the instruction to displace the joint element 23.

[0082] [Correction Based on Distance] The information processing device 2 may also correct the displacement instruction for each joint element 23 according to the distance between the joint element 23 and the grip unit 11 at the time of the instruction. For example, the information processing device 2 may correct the displacement speed of the joint element 23 (for example, the magnitude of change in tilt angle per unit time) so that the greater the distance from the grip unit 11. Conversely, the information processing device 2 may control the displacement speed of the joint element 23 (for example, the magnitude of change in tilt angle per unit time) so that the greater the distance from the grip unit 11. The former is suitable for presenting a force sensation as if operating a whip, and the latter is suitable for presenting a force sensation as if hanging a carrier bag.

[0083] [Interaction with user's body, etc.] Furthermore, in one example of this embodiment, the information processing device 2 may perform an operation of displacing the joint element 23 of the controller device 1 to bring the controller device 1 into contact with a predetermined part of the body of a user who is in the vicinity of the controller device 1, such as a user holding the controller device 1.

[0084] In this example, the information processing device 2 detects, by means of a camera C or the like, each part of the controller device 1 and the position of a part (e.g., shoulder) of the body of the user holding the controller device 1 that the controller device 1 is intended to come into contact with. The information processing device 2 then generates an instruction to control the displacement (tilt angle, rotation angle, movement position, etc.) of each joint element 23 so that, for example, the tip of the controller device 1 (e.g., the end of the main body 31 of the force feedback unit 30) comes into contact with a predetermined part of the user's body, and sends the instruction to the controller device 1.

[0085] Furthermore, the controller device 1 may be provided with a distance sensor using infrared rays, ultrasonic waves, or the like at the end (tip) of the main body 31 of the force feedback unit 30. In this example, the control unit 41 of the controller device 1 may perform multi-stage control in response to an instruction from the information processing device 2, such as moving, for example, the tip of the controller device 1 close to the user's body, and then detecting the distance to the user's body while slowing down the displacement speed to bring the tip even closer to the user's body and bringing the tip into contact with the user's body. This is for the safety of the user.

[0086] Conversely to this example, the information processing device 2 may use a camera C or the like to detect the positions of each part of the controller device 1 and objects (the user or other obstacles) in the vicinity of the controller device 1, and when the controller device 1 is deformed by displacing the joint elements 23 of the controller device 1, check whether each part of the controller device 1 will collide with any object (or will come within a predetermined distance), and if it determines that a collision will occur, may instruct the displacement of the joint elements 23 to be stopped.

[0087] Similarly, if the controller device 1 is equipped with a distance sensor using infrared rays, ultrasound, or the like at the end (tip) of the main body 31 of the force feedback unit 30, the controller device 1 itself may displace each joint element 23 while checking whether the joint element 23 will collide with any object (or come within a predetermined distance) during the displacement of the joint element 23, and may stop the displacement of the joint element 23 when it determines that a collision will occur.

[0088] [Haptic sense presentation device in gripping unit] The controller device 1 may further include a haptic sense presentation device such as a vibrator in the gripping unit 11. In this example, the information processing device 2 outputs an instruction to control the haptic sense presentation device in the gripping unit 11 at an appropriate timing, and the control unit 41 of the controller device 1 drives the haptic sense presentation device in the gripping unit 11 in accordance with the instruction.

[0089] According to this example, the force sensation of cutting with a chainsaw can be more clearly presented.

[0090] [Other application examples of force sense presentation] In addition to the above examples, for example, umbrella: the center of gravity is rapidly shifted, and then an impact is expressed by the force sense presentation unit 30; shovel: the center of gravity is shifted as far outward as possible from the grip unit 11, to express a state in which a heavy object is attached to the tip; slingshot that is played with a rubber band: the joint elements 23 are controlled so that the center of gravity shifts slowly when the rubber band is being pulled, and when an operation to release the rubber band is performed, the joint elements 23 are controlled so that the center of gravity shifts relatively quickly in a direction away from the user's body; when drinking a drink, for example, or to express a liquid: the center of gravity position is controlled according to the attitude of the grip unit 11 to express a state in which liquid is being held, or when the attitude of the grip unit 11 (particularly the angle towards the tip) exceeds a threshold, the displacement of the joint elements is controlled so that the center of gravity position gradually approaches an axis in the vertical direction from the hand or shoulder of the user holding the controller device 1, thereby expressing a reduction in the sense of weight. - Turning a lever, opening a door...Using the detection result of the posture of the gripping unit 11, when the detected posture satisfies a predetermined condition, the force feedback unit 30 is driven to express the impact caused by operating the lever; - Cutting...Depending on the instruction, the movement direction of the weight 323 is periodically reversed (the period of this reversal control depends on a parameter for the magnitude of the impact), and this control is stopped when it is determined that the cutting has been completed.Examples of such applications are possible.

[0091] [Processing on the Controller Device Side] In the explanation up to this point, the control unit 41 of the controller device 1 controls each unit in accordance with instructions from the information processing device 2, but the present embodiment is not limited to this example. For example, the control unit 41 may control each unit in accordance with user operations (such as tilting the grip unit 11 or changes in grip strength of the grip unit 11) without instructions from the information processing device 2.

[0092] In another example, the controller device 1 may receive settings such as weight parameters and length parameters (these settings may be made directly by the user to the controller device 1 or may be instructed by the information processing device 2), and correct the amount of displacement, displacement speed, etc. of the joint element 23 in accordance with instructions received from the information processing device 2. In this example, the information processing device 2 and the controller device 1 cooperate to present a force sense and control changes in the center of gravity.

[0093] For example, the controller device 1 may perform correction so that the larger the weight parameter, the larger the tilt angle or rotation angle (amplitude as seen from the user) of the joint element 23 that is farther from the grip portion 11 becomes.

[0094] [Moveable Direction of Weight] In the description up to this point, the weight 323 is movable along the longitudinal axis of the main body 31 of the force sense presentation unit 30, but this embodiment is not limited to this. For example, the weight 323 may be movable in the circumferential direction of the cylindrical main body 31 with the longitudinal axis of the main body 31 of the force sense presentation unit 30 as the axis of rotation. Furthermore, if the main body 31 has another shape, such as a sphere, the weight 323 may be movable in any direction within the main body 31.

[0095] Effect of the embodiment According to the controller device 1 of the present embodiment, a predetermined attitude of the grip unit 11 is used as a reference attitude, and the change over time in the center of gravity position (such as the amount of movement) can be controlled in accordance with the difference between the reference attitude and the actual attitude of the grip unit 11. Furthermore, the attitude of a virtual object managed by the information processing device 2 is controlled based on information about the attitude of the controller device 1 (such as the attitude of the grip unit), and the center of gravity position, amount of vibration, amount of impact, etc. of the controller device 1 are controlled based on the attitude of the virtual object, making it possible to present a variety of force sensations to the user.

[0096] REFERENCE SIGNS LIST 1 Controller device, 2 Information processing device, 10 Support, 11 Grip portion, 20 Joint portion, 21 Base portion, 22 Extension portion, 23 Joint element, 25 Actuator, 30 Force sense presentation portion, 31 Main body portion, 32 Force sense presentation element, 40 Circuit portion, 41 Control portion, 42 Memory portion, 43 Interface portion, 44 Sensor portion, 45 Communication portion, 321, 322 Electromagnet, 323 Weight.

Claims

1. A controller device comprising: a support including a gripping portion to be gripped by a user; and a force sense presentation unit connected to the support via a joint, the force sense presentation unit having a force sense presentation means for presenting a force sense to the user by a movable weight that is movable relative to a position to which the joint is connected.

2. A controller device according to claim 1, wherein the force feedback unit has a hollow body and a movable weight movable within the hollow body.

3. A controller device as described in claim 1, wherein the force sense presentation unit comprises a connecting portion connected to the joint, and a force sense presentation unit main body arranged to be movable relative to the connecting portion.

4. A controller device comprising: a support including a gripping portion to be gripped by a user; a center of gravity adjustment body connected to the support via a joint rotatable around multiple axes and equipped with a weight that can be positioned at a point a predetermined distance from the rotation axis of the joint; and a control circuit that controls the rotation of the joint.

5. A controller device as described in claim 4, wherein the support has at least one element body interposed between the grip portion and the center of gravity adjustment body, and this element body is connected to an adjacent element body, the grip portion, or the center of gravity adjustment body via a joint.

6. A controller device according to any one of claims 1 to 5, wherein the joint is a joint that is rotatable around at least one axis.

Images