Information processing method, information processing program, and information processing system

Abstract

To provide a program, information processing method, and system that enable the control of virtual objects in accordance with the user's intentions. [Solution] The program causes the computer to function as a means for translating a first virtual object based on first data, a means for setting a second virtual object to a movable state in which it is translated based on first data, based on third data corresponding to a first operation performed on the mouse when the first virtual object and the second virtual object are in a predetermined positional relationship, and a means for correcting the direction of translation of the second virtual object based on first data based on the rotation of the mouse indicated by second data when it is in the movable state. When it is not in the movable state, the first virtual object is translated based on first data without correction of the direction of translation based on the rotation of the mouse indicated by second data.

Description

Technical Field

[0001] The present disclosure relates to information processing for controlling a virtual object based on data output from a mouse.

Background Art

[0002] There is a technology for controlling a pointer on a display based on a movement operation and a rotation operation of a mouse (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above technology, there is room for improvement so that the virtual object can be controlled in accordance with the user's intention.

Means for Solving the Problems

[0005] In view of the above, for example, the following configuration examples can be cited.

[0006] (Configuration 1) Configuration 1 is an information processing method to be executed by a processor of an information processing apparatus, and causes the processor to perform the following. Obtain first data based on the output of an optical sensor provided in the mouse, obtain second data based on the output of an attitude sensor provided in the mouse, translate the first virtual object in parallel based on the first data, perform first control on the first virtual object based on the second data, and when the first data and the second data satisfy a first condition, while performing first control on the object based on the second data, make the amount of parallel translation of the first virtual object based on the first data smaller than the amount of parallel translation when the first condition is not satisfied. <00,00038>

[0007] According to the above configuration example, if the first condition is met, the influence of the first data on performing the first control is reduced. This makes it easier to perform the first control in accordance with the user's intentions.

[0008] (Configuration 2) Configuration 2 may, in Configuration 1 described above, prevent the first virtual object from being translated when the first condition is met.

[0009] (Composition 3) Configuration 3 allows the processor to determine, in Configuration 1 above, the degree to which the translation amount of the first virtual object is reduced according to the first data and / or the second data.

[0010] According to the above configuration example, for example, the operability when translating a first virtual object while rotating it can be improved.

[0011] (Composition 4) Configuration 4 is one of the above configurations 1 to 3, in which the first control is a control that rotates the first virtual object.

[0012] (Composition 5) Configuration 5 may, in Configuration 4 described above, correspond the direction of movement of the first virtual object based on the first data to the orientation of the first virtual object after rotation.

[0013] According to the above configuration example, intuitive operation becomes possible.

[0014] (Composition 6) Configuration 6 may, in Configuration 5 above, rotate the direction of movement of the first virtual object based on the first data by the first angle when the first virtual object rotates by the first angle.

[0015] According to the above configuration example, for instance, the direction of the user's hand movement and the direction of movement of the first virtual object can be made to match, enabling operation without any sense of unnaturalness.

[0016] Configuration 7 In Configuration 7, in the above Configuration 5 or 6, the processor may acquire third data based on a first operation on the mouse, and set the posture of the first virtual object to a predetermined posture based on the third data.

[0017] According to the above configuration example, as an example, regarding the correspondence relationship between the posture of the mouse and the posture of the first virtual object, when the correspondence relationship becomes one that the user does not intend, the correspondence relationship can be reset. Thereby, the operability can be improved.

[0018] Configuration 8 In Configuration 8, in any of the above Configurations 5 to 7, the processor may execute a predetermined game process, and display a screen for guiding the initial posture of the mouse at the start of the game by the predetermined game process at the start of the play of the game.

[0019] According to the above configuration example, by allowing the user to take the initial posture of the mouse at the start of the game, it is possible to prevent a sense of incongruity from occurring in the correspondence relationship between the mouse and the posture of the first virtual object thereafter.

[0020] Configuration 9 In Configuration 9, in the above Configuration 8, the processor may acquire fourth data based on a second operation on the mouse, and start the play of the game based game based on the fourth data.

[0021] According to the above configuration example, by causing the user to trigger the start of the game, it is possible to suppress the occurrence of a sense of discomfort in operation. Whether the mouse is in a predetermined posture cannot always be grasped on the information processing device side. Even if it can be grasped, the predetermined posture may be a posture that causes a sense of discomfort to the user. For example, even if the mouse is guided to face forward and the user intends to adopt the posture as such, actually, depending on the angle of the arm and hand, it may be slightly turned to the right or left from the predetermined posture. In view of this point, by causing the user to adopt a posture that does not cause a sense of discomfort to themselves and then causing the trigger for starting the game, it is possible to suppress the occurrence of a sense of discomfort in subsequent operations.

[0022] (Configuration 10) In Configuration 10, in any of the above Configurations 1 to 9, the processor may be caused to acquire fifth data based on a third operation on the mouse and acquire sixth data based on a fourth operation on the mouse. Then, when the first virtual object is in a predetermined positional relationship with the second virtual object, the second virtual object may be made movable based on the fifth data and made immovable based on the sixth data, and the movable second virtual object may be moved and rotated in accordance with the movement and rotation of the first virtual object.

[0023] According to the above configuration example, the operability with respect to the second virtual object can be improved. When moving or rotating the second virtual object, there is a limit to operating it without changing the holding of the above mouse. For example, with just one rotation operation without changing the holding, the second virtual object may not be able to be rotated significantly. Therefore, by making it possible to switch between a movable state and an immovable state and interposing the first virtual object, the operability with respect to the second virtual object can be improved. As an example, after rotating the second virtual object to a certain extent during the movable state, while making it immovable, the rotational posture of the user's hand is returned to its original state and then made movable again, making it possible to further rotate the second virtual object.

[0024] (Composition 11) Configuration 11 may be any of the above configurations 1 to 10, where the first condition is a condition indicating that the center of the mouse's rotation operation is within a predetermined range.

[0025] (Composition 12) In configuration 12, the predetermined range in configuration 11 may be the range included in the bottom surface of the mouse.

[0026] According to the above configuration example, the accuracy of detecting movements such as rotating the mouse in place without moving it can be improved.

[0027] (Composition 13) Configuration 13 may be any of the above configurations 1 to 10, where the first condition is a condition indicating that the amount of horizontal movement of the mouse is small relative to the amount of rotation of the mouse.

[0028] (Composition 14) Configuration 14 may be configured in any of the above configurations 1 to 13 to cause the processor to obtain seventh data stored in the mouse and to set the first condition based on the seventh data.

[0029] According to the above configuration example, it becomes possible to set a first condition that corresponds to the characteristics of the mouse's rotation trajectory, based on the size and shape of the mouse's bottom surface, or the overall shape of the mouse. [Brief explanation of the drawing]

[0030] [Figure 1] Block diagram showing an example of the hardware configuration of the information processing device 2 and the mouse 40. [Figure 2] An example of the appearance of Mouse 40 [Figure 3] An example of the appearance of Mouse 40 [Figure 4] An example of a game screen for the game processing assumed in this embodiment. [Figure 5] An example of a game screen for the game processing assumed in this embodiment. [Figure 6]An example of a game screen for the game processing assumed in this embodiment. [Figure 7] An example of a game screen for the game processing assumed in this embodiment. [Figure 8] An example of a game screen for the game processing assumed in this embodiment. [Figure 9] An example of a game screen for the game processing assumed in this embodiment. [Figure 10] A diagram illustrating an example of the "rotate in place" operation. [Figure 11] A diagram illustrating an example of the "rotate in place" operation. [Figure 12] A diagram illustrating an example of the "rotate in place" operation. [Figure 13] A diagram illustrating an example of the "rotate in place" operation. [Figure 14] A diagram illustrating an example of the "rotate in place" operation. [Figure 15] A diagram illustrating an example of the "rotate in place" operation. [Figure 16] A diagram illustrating an example of the "rotate in place" operation. [Figure 17] A diagram illustrating an example of the "rotate in place" operation. [Figure 18] A diagram illustrating an example of the "rotate in place" operation. [Figure 19] A diagram illustrating an example of the "rotate in place" operation. [Figure 20] A diagram illustrating an example of a center of rotation. [Figure 21] A diagram illustrating an example of a center of rotation. [Figure 22] A diagram illustrating an example of a center of rotation. [Figure 23] Diagram to explain the symbols [Figure 24] A diagram to explain the correction of the direction of movement. [Figure 25] A diagram to explain the correction of the direction of movement. [Figure 26] A diagram to explain the correction of the direction of movement. [Figure 27] A diagram to explain the correction of the direction of movement. [Figure 28] A diagram to explain the correction of the direction of movement. [Figure 29] A memory map showing an example of various data stored in the storage unit 22 of the information processing device 2. [Figure 30] A flowchart illustrating the details of the game processing. [Figure 31] A flowchart illustrating the details of the game processing. [Figure 32] A flowchart illustrating the details of the game processing. [Figure 33] Diagram illustrating a modified example. [Figure 34] Diagram illustrating a modified example. [Modes for carrying out the invention]

[0031] One embodiment will be described below.

[0032] [Hardware configuration of information processing device 2] Figure 1 is a block diagram showing an example of the hardware configuration of an information processing device 2 and a mouse, which is an example of an input device, according to this embodiment. In Figure 1, the information processing device 2 includes a processor 21. The processor 21 is an information processing unit that performs various information processing tasks performed by the information processing device 2. In this embodiment, the processor 21 is composed of a System-on-a-chip (SoC) that includes at least a CPU (Central Processing Unit) function and a GPU (Graphics Processing Unit) function. In other embodiments, the CPU and GPU may be configured separately. The processor 21 performs various information processing tasks by executing an information processing program (for example, a game program) stored in the memory unit 22. The memory unit 22 may be an internal storage medium such as flash memory or DRAM (Dynamic Random Access Memory), or it may be configured to utilize an external storage medium mounted in a slot (not shown).

[0033] Furthermore, the information processing device 2 includes a communication unit 23 for communicating with other information processing devices and predetermined servers.

[0034] Furthermore, the information processing device 2 includes an input device communication unit 24 for the information processing device 2 to communicate with various input devices via wired or wireless communication. In this embodiment, a mouse 40, as described later, will be used as an example of an input device.

[0035] Furthermore, a display unit 30 (for example, a monitor) is connected to the information processing device 2 via an image and sound output unit 25. The processor 21 outputs images and sounds generated by the execution of the above-mentioned information processing to the display unit 30 via the image and sound output unit 25.

[0036] Next, the mouse 40 described above will be explained. The mouse 40 of this embodiment is an input device that can be connected to the information processing device 2 by wireless communication. Figures 2 and 3 show schematic diagrams of the appearance of the mouse 40 assumed in this embodiment. Figure 2 is a perspective view showing an example of the mouse 40, and Figure 3 is a six-view drawing showing an example of the mouse 40. Roughly speaking, the mouse 40 assumed in this embodiment has a form that is like a general PC mouse with its width reduced and a roughly plate-like shape. Specifically, the length of the mouse 40 in the left-right direction is shorter than the length in the up-down direction and the length in the front-back direction. The length of the mouse 40 in the up-down direction is shorter than the length in the front-back direction. Here, as shown in Figure 2, in this embodiment, the left-right direction of the mouse 40 is the x-axis direction, the front-back direction is the y-axis direction, and the up-down direction is the z-axis direction. The z-axis direction is the direction perpendicular to the bottom surface of the mouse 40 (or the direction perpendicular to the work surface when the bottom surface of the mouse 40 is placed on the work surface). Furthermore, two buttons 42A and 42B are provided on the top surface of the mouse 40. Button 42A may be used for left-click operations, and button 42B may be used for right-click operations. In addition, a mouse sensor 43 is exposed on the bottom surface of the mouse 40. In this embodiment, the mouse sensor 43 is exposed through an opening 45 provided slightly in front of the center of the bottom surface. However, the position in which the mouse sensor 43 is exposed is not limited to this position. In this embodiment, as an example of the mouse sensor 43, the case in which an optical sensor is used will be described. Note that the mouse sensor 43 does not have to be directly exposed through the opening 45 provided on the bottom surface of the mouse 40. For example, a light guide path may be formed from the opening 45 provided on the bottom surface of the mouse 40 to the mouse sensor 43 provided inside the mouse 40. In addition, buttons 42C and 42D are also provided on the left side of the mouse 40. For example, when operating the mouse 40 with the right hand, buttons 42C and 42D are provided in a position that can be operated with the right thumb. Note that the operating means provided on the left side is not limited to buttons. For example, instead of buttons, or in addition to them, a stick or a touch panel may be provided.

[0037] Returning to Figure 1, let's explain the hardware configuration of the mouse 40 again. The mouse 40 is equipped with the buttons 42A to 42D described above. Hereafter, these will sometimes be collectively referred to simply as buttons 42.

[0038] Furthermore, the mouse 40 is equipped with an optical mouse sensor 43 as described above. The mouse sensor 43 detects the movement of the mouse 40 and outputs the direction and amount of movement, etc.

[0039] Furthermore, the mouse 40 is equipped with an attitude sensor 44. Specifically, the mouse 40 is equipped with an angular velocity sensor as the attitude sensor 44. In this embodiment, the angular velocity sensor detects angular velocity around three predetermined axes. Note that the attitude sensor 44 is not limited to an angular velocity sensor; in other embodiments, various sensors capable of detecting attitude, such as a geomagnetic sensor, may be used. These sensors may also be used in combination, or other sensors such as an acceleration sensor may be used in combination. Alternatively, multiple optical sensors may be combined and used as the attitude sensor 44. For example, it may be possible to detect that the mouse 40 has been rotated based on the difference in the detected values ​​of each optical sensor (difference in direction of movement, etc.).

[0040] The mouse 40 also includes a wireless communication unit 41 for wireless communication with the input device communication unit 24. Information indicating the pressed state of the button unit 42, various detection results from the mouse sensor 43, and various detection results from the posture sensor 44 are repeatedly output to the wireless communication unit 41 at appropriate intervals and transmitted to the information processing device 2.

[0041] [Regarding the processing assumed in this embodiment] Next, an overview of the information processing assumed in this embodiment will be described. In this embodiment, game processing is assumed as an example of information processing. Figures 4 to 9 show an example of a game screen for the game processing assumed in this embodiment. The game in this example is assumed to be a 2D game, and a game image that looks like an overhead view of a virtual space is displayed. In Figure 4, obstacle objects 101 arranged to form a maze, a hand object 102 which is also used as a mouse pointer, and a stick object 103 which is the object that the user controls are shown. In this embodiment, the initial posture of the stick object 103 (posture at the start of the game) is assumed to be a vertically elongated posture as shown in Figure 4. This game is a game in which the player moves the stick object 103 from the start point to the goal point while avoiding contact with the obstacle objects 101. An example of how to operate the stick object 103 will be described below.

[0042] This game provides an interactive experience where the user grabs and moves the stick object 103 with the hand object 102. Therefore, starting from the state shown in Figure 4, the user first moves the mouse 40 downwards and to the left to move the hand object 102 to a position where it overlaps with the stick object 103. This results in the state shown in Figure 5.

[0043] Next, by pressing button 42A (hereinafter referred to as left-click on), the display of the hand object 102 changes from an image showing an open hand to an image showing it grasping the stick object 103, as shown in Figure 6. This indicates that the hand object 102 is grasping the stick object 103. In this game, when the hand object 102 is grasping the stick object 103 in this state, it becomes possible to move the stick object 103 (specifically, move and rotate it). Hereafter, this state will be referred to as the "movable state". Also, when the button 42A is released while in the movable state (referred to as left-click off), the movable state is released, and the display of the hand object 102 returns to its original state.

[0044] While maintaining the movable state shown in Figure 6, the user can move the mouse 40 forward in a parallel motion, thereby moving the rod object 103 and the hand object 102 upwards on the screen, as shown in Figure 7. More precisely, in this embodiment, the rod object 103 is controlled to move in accordance with the movement of the hand object 102.

[0045] Next, after moving the rod object 103 to the corner of the passage as shown in Figure 7 above, the user attempts to move the rod object 103 to the right along the passage. During this movement, the user can rotate the rod object 103 so that it is in a horizontal position, preventing it from coming into contact with the obstacle object 101. The operation to rotate the rod object 103 at this time is as follows: While the user keeps the mouse 40 left-clicked (moving), the user rotates the mouse 40 in place, for example, clockwise. In other words, the user rotates the mouse 40 around the z-axis in Figure 2, or in other words, in the yaw direction, while minimizing parallel movement of the mouse 40 itself. In the following explanation, when we simply say "rotation of the mouse 40", we mean rotation around the z-axis in Figure 2. Also, in the following, this operation of rotating the mouse 40 in place will be called "rotation in place". This "in-place rotation" allows the hand object 102 and the rod object 103 to be rotated around an axis perpendicular to the display surface, i.e., the depth axis, as shown in Figure 8. More precisely, in this embodiment, the rotation of the rod object 103 is controlled in accordance with the rotation of the hand object 102. Here, as an example, let's assume that the hand object 102 and the rod object 103 are ultimately rotated 90° clockwise from the state shown in Figure 7.

[0046] If the rod object 103 can be rotated to a horizontal orientation, the user can move the rod object 103 to the right without it coming into contact with the obstacle object 101, as shown in Figure 9, by moving the mouse 40 to the right while maintaining its movable state.

[0047] The user must move the rod object 103 to the goal using the "movement" and "rotation" operations, which is the condition for clearing the game. If the rod object 103 comes into contact with an obstacle object 101, the user loses a life.

[0048] Here, I will add some details regarding the above-mentioned movable state and the "rotate in place" operation. As described above, if you want to rotate the mouse 40 without moving it, you could, for example, hold the mouse 40 with your right thumb, middle finger, ring finger, and little finger and rotate it (using your index finger for left-clicking). In such an operation, you could rotate it using the base of your fingers or your wrist as a pivot point, but the range of motion of your fingers and wrist is limited, so for example, if you want to rotate the rod object 103 90° clockwise, it is not guaranteed that you can rotate it 90° in one rotation. In this game, by making it possible to move the rod object 103 only when it is in the above-mentioned movable state, the following operation is made possible. That is, in order to rotate the rod object 103 90°, you can rotate it partway (for example, about 45 degrees), release the movable state, return the mouse 40 to its position before rotation, and then put it back into the movable state and add the remaining rotation. To illustrate an example of this operation flow, let's assume the initial state is as shown in Figure 10. In Figure 10, the upper half of the rectangular frame shows the display state of the rod object 103 and the hand object 102, while the lower half shows a schematic diagram of the mouse 40 in real space from an overhead perspective.

[0049] When the user left-clicks in the state shown in Figure 10, the mouse becomes movable as shown in Figure 11. In Figure 11, button 42A is blacked out to indicate that the left click is engaged. Next, suppose the mouse 40 is rotated to the angle shown in Figure 12. The hand object 102 and the rod object 103 rotate accordingly. When the mouse 40 has been rotated to this point, it is no longer possible to rotate it further. At this point, the user releases their finger from button 42A to disengage the left click. As a result, the state shown in Figure 13 is reached. In Figure 13, the image of the hand object 102 has returned to the image of when it is not movable (hereinafter referred to as the "non-movable state"). Then, as shown in Figure 14, the user rotates the mouse 40 counterclockwise (without left-clicking) to return the mouse 40 to its original position. At this time, the hand object 102 returns to its original position by rotating counterclockwise in accordance with the rotation of the mouse 40, while the position of the rod object 103 remains unchanged.

[0050] Then, as shown in Figure 15, the user switches to the movable state by left-clicking again, and rotates the mouse 40 clockwise, as shown in Figure 16. This rotates the rod object 103, giving it a horizontal orientation.

[0051] Subsequently, as shown in Figure 17, the movable state is released by left-clicking off, and the mouse 40 returns to its position before rotation, as shown in Figure 18. Then, as shown in Figure 19, the movable state is switched back on, and the rod object 103 can be moved to the right by translating the mouse 40, for example, to the right. Of course, the rod object 103 can also be moved to the right by translating the mouse 40 to the right while it is in the state shown in Figure 16.

[0052] In this way, by repeatedly switching between the movable and immobile states and performing the rotation operation in place, the rod object 103 can be rotated to the desired angle. Note that this switching of the movable state is used not only for rotation but also when moving the rod object 103. For example, the rod object 103 can be moved parallel to the right from the left edge of the screen to near the center, the movable state can be temporarily deactivated, the mouse 40 can be returned to its original position, and then the rod object can be made movable again to move it.

[0053] As mentioned above, in this game, the hand object 102 and the stick object 103 can be manipulated by the user. However, in the following explanation, the hand object 102 and the stick object 103 may be collectively referred to as "manipulable objects."

[0054] By the way, as mentioned above, in this game, for example, if you want to change the orientation of the rod object 103 at the corner of a passage, you are required to rotate the mouse 40 as described above. On the other hand, if the rod object 103 hits the obstacle object 101, it results in a miss, so the user is required to perform careful and delicate rotation operations. However, even if you perform the above-mentioned "rotation in place" operation, the rotation center of the mouse 40 may not coincide with the opening 45 due to the range of motion of the user's hand and fingers, and the positional relationship between the opening 45 provided on the bottom surface to guide light to the mouse sensor 43 and the rotation center. In particular, when the user rotates the mouse 40 by a certain angle, it is difficult to always make the rotation center of the mouse 40 coincide with the opening 45. In other words, even if you change the orientation of the mouse 40 with the "rotation in place" movement, the mouse 40 actually moves horizontally relative to the work surface, so the output of the mouse sensor 43 changes according to the horizontal movement of the mouse 40. For example, Figures 20 to 22 show examples of when the mouse 40 is "rotated in place". All of the figures show examples where the center of rotation is within the bottom surface of the mouse 40, but the position within this bottom surface is different in each case. Thus, even if the user performs "rotation in place," as long as the position of the opening 45 and the position of the center of rotation do not coincide, the mouse sensor 43 can output a slight parallel movement corresponding to the mouse 40. Also, in the examples of Figures 20 and 21, the mouse sensor 43 moves downward to the right on the work surface, and in the example of Figure 22, it moves upward to the right. Thus, the amount and direction of movement detected by the mouse sensor 43 differ depending on where the rotation is centered. If such horizontal movement is directly reflected in the movement of the rod object 103, even if the user intends to rotate the rod object 103 only, the rod object 103 will rotate and move horizontally at the same time, increasing the likelihood that the rod object 103 will come into contact with the obstacle object 101.

[0055] Based on the points described above, in this embodiment, when the mouse sensor 43 detects a translation of the mouse 40, it distinguishes between the movement associated with the "in-place rotation" operation described above and the movement due to a normal translation that is not an "in-place rotation". Specifically, in this embodiment, it is determined whether the movement detected by the mouse sensor 43 is associated with an "in-place rotation". If it is not an "in-place rotation" (in the case of a normal translation), the target object is rotated based on the output of the posture sensor 44, and the target object is moved using the output data of the mouse sensor 43. On the other hand, in the case of an "in-place rotation", the rotation control of the target object based on the output of the posture sensor 44 is performed in the same way as above, but the control performed based on the output data from the mouse sensor 43 is different. Specifically, even if the content of the output data of the mouse sensor 43 is such that, under normal circumstances (i.e., if the conditions for being considered an "in-place rotation" are not met), the control to move the target object by a first distance is not performed, and a different control is performed. More specifically, when moving the target object using the output data from the mouse sensor 43, the amount of movement is reduced when it is a "rotation in place" compared to when it is not a "rotation in place". Reducing the amount of movement includes setting the amount of movement to zero.

[0056] [Regarding the determination principle for "in-place rotation"] Next, the principle for determining whether or not the rotation is "in-place" as described above in this embodiment will be explained. When a user intentionally performs an "in-place rotation" action, the rotation center may be located somewhere within the bottom surface (contact surface) of the mouse 40. On the other hand, rotation may occur unintentionally when a user moves the mouse 40 in a parallel direction, in which case the rotation center may be located somewhere outside the bottom surface of the mouse 40, such as the user's elbow. Therefore, in this embodiment, whether or not the rotation is considered "in-place" is determined by determining whether or not the rotation center of the mouse 40 is located within the bottom surface. Note that, as shown in Figures 2 and 3 above, the bottom surface of the mouse 40 in this embodiment is long in the y-axis direction and short in the x-axis direction, so it is assumed that no movement (shift) of the rotation center in the x-axis direction occurs during an "in-place rotation" action.

[0057] Strictly speaking, in a series of "in-situ rotation" movements, the center of rotation itself is considered to change while rotating, within the scope of the above premise. However, when observing instantaneous minute changes, the center of rotation can be considered to be fixed. From this, as shown in Figure 23, if the minute rotation angle is Δθ, the amount of displacement of the opening 45 is dx and dy, and the distance between the center of rotation and the position of the opening 45 is d, then the following relationship is considered to hold. <When the center of rotation is below the position of the opening 45> dx = -d×sin(Δθ) ≒ -d×Δθ dy= -d×(1-cos(Δθ)) ≒ -d×(Δθ) 2 / 2 <When the center of rotation is above the position of the opening 45> dx = d × sin(Δθ) ≈ d × Δθ dy = d×(1-cos(Δθ)) ≒ d×(Δθ) 2 / 2 The above relationship utilizes the fact that when Δθ is sufficiently small, the following holds true. sin(Δθ) ≈ Δθ cos(Δθ) ≈ 1-(Δθ) 2 / 2

[0058] The distance d can be calculated using the following formula, based on the above formula. JPEG2026123101000002.jpg1592 The above formula focuses on dx, but the distance d can also be calculated using the following formula, which focuses on dy. JPEG2026123101000003.jpg1992

[0059] Next, in order to determine whether the distance d(d1, d2) obtained above indicates that the center of rotation of the mouse 40 is located within the bottom surface, we determine whether the distance d falls within a threshold. Here, if the position of the opening 45 is offset from the center in the anterior-posterior direction on the bottom surface of the mouse, the distance from the opening 45 to the front end of the bottom surface of the mouse and the distance to the rear end will be different, so the threshold to be compared with the distance d will be different. Therefore, we first determine whether the center of rotation is on the front side or the rear side of the opening 45 based on the following conditions. Note that in Figure 23, the sign of Δθ is assumed to be positive for counterclockwise rotation and negative for clockwise rotation. (Condition 1: The center of rotation is behind the opening 45) dy < 0 and dx and Δθ have opposite signs. (Condition 2: The center of rotation is in front of the opening 45) dy≧0 and dx and Δθ have the same sign. If condition 1 is met, D1 (D1A, D1B) is set as the threshold. If condition 2 is met, D2 (D2A, D2B) is set as the threshold. In this embodiment, since the opening 45 is located in front of the center of the bottom surface in the front-to-back direction, D1 is a larger value than D2. Note that if condition 1 is not met, it may be determined that condition 2 is met. Also, conditions 1 and 2 are just examples; for example, condition 1 may only check if dy < 0, or other conditions may be checked.

[0060] In the above, if condition 1 is met, the following determination is made in this embodiment. (Condition 3) d1 <D1A (Condition 4) d2 <D1B In this embodiment, if both conditions 3 and 4 are met, it is assumed that the user is intentionally "rotating the mouse 40 in place". In this case, dx and dy obtained from the output of the mouse sensor 43 are not treated as the amount of translation in normal movement. Note that in the above, D1B is a larger value than D1A. This is because, as shown in Figure 2, the mouse 40 in this embodiment has a long shape in the y-axis direction, and when it rotates in place, dy is smaller than dx. Therefore, d2 may vary more than d1 due to the influence of errors in the mouse sensor 43, but even if there is some variation, it will fall within D1B.

[0061] In the above, "in-situ rotation" was determined when both conditions 3 and 4 were met, but it is also acceptable to determine "in-situ rotation" when either one of the conditions is met. Alternatively, only one of conditions 3 or 4 may be used for the determination, or other conditions may be used. Furthermore, in the above, the threshold value compared to distance d was set from either D1 or D2 depending on whether the rotation center is in front of or behind the opening 45, but a common threshold value may be used. Alternatively, a coefficient corresponding to whether the rotation center is in front of or behind the opening 45 may be multiplied by d1 or d2 before being compared with the threshold value.

[0062] Furthermore, as described above, in this embodiment, when "in-place rotation" is determined, the amount of movement of the target object based on the output data of the mouse sensor 43 is reduced compared to when "in-place rotation" is not determined. Here, even in the case of "in-place rotation," the amount of movement may be reduced, but the target object may still be moved. For example, when moving the mouse 40 horizontally while rotating it, the above determination may result in partial determination of "in-place rotation." In this case, especially when moving the mouse 40 slowly, the rod object 103 may stop at various points during movement, making smooth movement impossible. Therefore, even if "in-place rotation" is determined, moving the target object can suppress the feeling of discomfort to the user. The degree of reduction in the amount of movement of the target object may be a fixed value or a percentage, or it may be a variable value or a percentage. For example, the smaller the value of distance d, the greater the degree of reduction. Alternatively, the degree of reduction may be determined based on data from either the mouse sensor 43 or the posture sensor 44.

[0063] The above determination principle focuses on the assumption that when a user is attempting to perform "in-place rotation," the center of rotation of the mouse is within the range of the bottom surface of the mouse 40. However, "in-place rotation" may be determined by other principles. For example, when a user intends to perform "in-place rotation," it is assumed that the amount of translation detected by the mouse sensor 43 is (very) small compared to the amount of rotation of the mouse 40. Focusing on this assumption, when the amount of translation is small compared to the amount of change in posture, control based on translation may be suppressed. The resulting conditions may be the same as or different from the various conditions described above.

[0064] [Movement direction correction based on posture] In the above explanation, regarding the relationship between the movement direction of the mouse 40 and the movement direction of virtual objects on the screen, it is assumed that the orientation shown in Figure 24 is considered as the reference orientation, and that the up, down, left, and right directions on a predetermined work surface to which the bottom surface of the mouse 40 can make contact and move coincide with the up, down, left, and right directions on the screen. Specifically, the positive y-axis direction of the mouse 40 (up on the work surface) corresponds to the up direction on the screen, and the positive x-axis direction of the mouse 40 (right on the work surface) corresponds to the right direction on the screen. Also, in Figure 24, it is assumed that the monitor is on the positive y-axis side of the mouse 40's local coordinate system. In other words, it is assumed that the plane containing the display surface and the y-axis of the mouse 40's local coordinate system are approximately orthogonal. Note that when using a mouse with a personal computer, for example, it is common to maintain such a reference orientation and positional relationship while operating.

[0065] However, this game requires the user to rotate the mouse 40. For example, the user rotates the mouse 40 around the z-axis from the reference position shown above, causing the mouse 40 to "rotate in place". In this embodiment, the hand object 102's posture is controlled to be linked to the mouse 40's posture, so the hand object 102's posture changes (rotates) in conjunction with the mouse 40's "rotation in place". For example, if the mouse 40 is rotated approximately 30° counterclockwise (+30°) from the reference posture in Figure 24, the mouse 40 will be in the posture shown in Figure 25. Consequently, the hand object 102 will also rotate approximately 30°. Now, consider the case where, while maintaining this posture, the user moves the mouse 40 in real space to the right as shown in Figure 26. In other words, we assume that the user intends to move the hand object 102 (and the rod object 103 if it is movable) to the right within the game screen. In this case, considering that the mouse 40 is tilted 30° from the reference position described above, the direction in which the mouse 40 moves is detected as 30° diagonally downward to the right when viewed from an overhead perspective of the work surface. In other words, the direction of movement as seen from the mouse 40, i.e., the direction of movement relative to the mouse's local coordinate system, is determined to be diagonally backward to the right. As a result, on the game screen, for example, as shown in Figure 27, the hand object 102 will move to the lower right, which may result in the user's intention to move it directly to the right not being reflected.

[0066] In view of the above, in this embodiment, the movement direction of the hand object 102 based on the output data of the mouse sensor 43 is controlled to correspond to the posture of the hand object 102 after rotation. Specifically, in this embodiment, the following control is performed. First, the posture of the mouse 40 at the start of gameplay is stored as the reference posture. In this game, the user is requested to perform the operation to start gameplay, and the posture of the mouse 40 when this operation is performed is stored as the reference posture. Here, the reference posture of the mouse 40 is the posture shown in Figure 24 above, and is assumed to be a vertically elongated posture which is the same as the initial posture of the rod object 103. Then, during gameplay, the difference data between the reference posture of the mouse 40 and the current posture of the mouse 40 is recorded. This difference data is, for example, the difference in angle around the z axis between the reference posture and the current posture. To give a specific example, if the mouse 40 is rotated 90° clockwise from the reference position, the difference data will be recorded as "-90°", if it is rotated 90° counterclockwise, as "+90°", and if it is rotated 180°, as "+180° (-180° is also acceptable)". Then, when determining the direction of movement of the hand object 102, the direction of movement of the mouse 40 (the direction of movement as seen from the mouse 40) obtained from the output data of the mouse sensor 43 is corrected using this difference data, and then the movement of the hand object 102 is controlled. For example, let's assume that the difference from the reference position is 30° counterclockwise (+30°) as shown in Figure 25 above. In this case, when determining the direction of movement of the hand object 102, as shown in Figure 28, a correction is made to rotate the direction of movement of the mouse 40 obtained from the output data of the mouse sensor 43 by approximately 30° counterclockwise. By performing this correction of the direction of movement, the direction in which the hand (grabbing the mouse 40) is moved in real space can be made to match the direction of movement of the hand object 102 on the screen, enabling intuitive movement of the hand object 102. Furthermore, by controlling the movement of the rod object 103 in accordance with the movement of the hand object 102, the same result can be obtained for the direction of movement of the rod object 103.

[0067] Note that the above control for correcting the direction of movement is just one example. As another example, instead of the posture of the mouse 40, the difference between the initial posture of the hand object 102 and the current posture of the hand object 102 may be used as the difference data. Alternatively, instead of using a reference posture, the instantaneous amount of rotation of the mouse 40 may be sequentially reflected in the correction amount for the direction of movement of the hand object 102.

[0068] [Details of the process in this embodiment] Next, the game processing in this embodiment will be described in more detail with reference to Figures 29 to 32.

[0069] [About the data used] First, the various data used in the processing of this embodiment will be described. Figure 29 is a memory map showing an example of the various data stored in the storage unit 22 of the information processing device 2. The storage unit 22 stores the game program 601, object data 602, operation data 603, mouse reference posture data 607, differential posture data 608, movable state flag 609, operation buffer 610, etc. The game program 601 is a program for executing the game processing according to this embodiment.

[0070] Object data 602 contains data for various objects that appear in the game, such as the rod object 103 and the hand object 102. Object data 602 also includes data indicating the reference pose (initial pose) of the rod object 103 and the hand object 102.

[0071] Operation data 603 is data that indicates the operation performed on the mouse 40. Operation data 603 includes mouse sensor data 604, attitude sensor data 605, and button data 606. Mouse sensor data 604 is data output from mouse sensor 43. Mouse sensor data 604 includes data indicating the amount and direction of movement of the mouse 40. Based on this data, the current position of the mouse 40 in a predetermined two-dimensional coordinate system (mouse coordinate system) can be determined, for example, in the form of xy coordinates in that coordinate system. Attitude sensor data 605 is data output from attitude sensor 44. In this example, attitude sensor data 605 includes angular velocity around predetermined three axes (xyz axes in Figure 2). Button data 606 is data indicating the pressed state of buttons 42A to 42D.

[0072] Mouse reference posture data 607 stores data indicating the posture used as the reference posture for mouse 40.

[0073] The differential pose data 608 is data that shows the difference in rotation angle around the z axis between the current pose of mouse 40 and the reference pose.

[0074] The operational status flag 609 is a flag that indicates whether or not the device is in the operational state described above. When it is on, it indicates that the device is in the operational state.

[0075] The operation buffer 610 is a memory area for temporarily storing operation data, for example, up to several frames prior. The operation buffer 610 is used, for example, to calculate the difference between the position and orientation of the mouse 40 one frame ago and the current position and orientation of the mouse 40.

[0076] [About flowcharts] Next, the details of the processing in this embodiment will be described. Here, we will mainly describe the processing related to the manipulation of the rod object 103 as described above, and the details of other game processing will be omitted. In this embodiment, the flowchart shown below is realized by one or more processors reading and executing programs stored in one or more memories. Furthermore, this flowchart is merely one example of the processing process. Therefore, the processing order of each step may be changed if similar results can be obtained. Also, the values ​​of the variables and the thresholds used in the judgment step are merely examples, and other values ​​may be adopted as needed.

[0077] Figures 30 to 32 are flowcharts detailing the game processing according to this embodiment. This processing is an example of game processing for one stage in a stage-clear type game. The processing loop from steps S3 to S23 is repeated multiple times per second depending on the frame rate.

[0078] First, in step S1, the processor 21 displays a guide screen to the user to explain the game rules and how to operate the mouse 40, prior to the start of actual gameplay. This screen displays an image instructing the user to place the mouse 40 on a predetermined work surface in a predetermined position. This predetermined position is, for example, the same as the initial position of the hand object 102. In this embodiment, it is assumed to be the position shown in Figure 24. The screen also displays that the game can be started by pressing a predetermined button (game start operation).

[0079] Next, in step S2, the processor 21 detects that the user has initiated the game and performs the game start process. This process records data indicating the mouse's orientation at this point as mouse reference orientation data 607. It also defines the correspondence (initial correspondence) between the up, down, left, and right directions in the mouse's local coordinate system and the up, down, left, and right directions on the screen at the start of gameplay. For example, initial correspondence data (not shown) indicating this correspondence may be generated and stored in the memory unit 22. In this game, the guide screen is displayed to use the orientation shown in Figure 24 as the reference orientation, and it is assumed that the game starts with the orientation shown in Figure 24. If the operation is performed as assumed, the initial correspondence is defined such that the positive y-axis direction in the mouse's local coordinate system in Figure 2 corresponds to the top of the screen and the positive x-axis direction corresponds to the right of the screen. In addition, the processor 21 constructs a maze-like virtual space as shown in Figure 4 and generates and displays a game image with a rod object 103 placed at the starting point.

[0080] The game start process (recording of mouse reference posture data 607) may be performed each time, for example, when a player makes a mistake in the middle of a stage and restarts.

[0081] Next, in step S3, the processor 21 acquires operation data 603. The processor 21 also calculates the current position and orientation of the mouse 40 based on the operation data 603.

[0082] Next, in step S4, the processor 21 determines, based on the operation data 603, whether the user has performed an operation to switch between the movable state and the immovable state (hereinafter referred to as the switching operation). In this embodiment, the switching from the immovable state to the movable state is the pressing of button 42A when the hand object 102 and the rod object 103 are in a predetermined positional relationship. More specifically, this positional relationship is one in which a part of the hand object 102 and the rod object 103 overlap. In other examples, they do not need to overlap and may, for example, be adjacent. The switching from the movable state to the immovable state is achieved by releasing the pressed button 42A.

[0083] If, as a result of the above determination, a switching operation is performed (YES in step S4), in step S5, the processor 21 sets the movable state flag 609 to on or off so that it switches to a state different from the current state between the movable state and the non-movable state. For example, if a switching operation (pressing button 42A) is performed when the movable state flag 609 is off, the processor 21 sets the movable state flag 609 to on. After that, the process proceeds to step S6, which will be described later.

[0084] On the other hand, if the result of the determination in step S4 is that no switching operation has been performed (NO in step S4), then in step S6, the processor 21 determines whether or not a posture reset operation has been performed. A posture reset operation is an operation to return the posture of the hand object 102 to its initial posture. This operation is used, for example, to correct the correspondence between the posture of the mouse 40 and the posture of the hand object 102 when the user does not intend it. For example, if the game is started with the posture shown in Figure 24 as the reference posture, it is desirable that the hand object 102 be displayed in a posture rotated 45° to the right when the mouse 40 is rotated 45° to the right. However, due to the accumulation of detection errors of the posture sensor 44, etc., it is possible that the posture of the hand object 102 may be such that it is rotated only 30°, resulting in a discrepancy between the actual posture of the mouse 40 and the posture of the hand object 102. In such cases, for example, by setting the mouse 40 to the above-mentioned reference posture and then performing a posture reset operation, the posture of the hand object 102 can be returned to its initial posture, thereby establishing a correspondence between the reference posture of the mouse 40 and the posture of the hand object 102. In this embodiment, the posture reset operation is assumed to be the pressing of button 42D.

[0085] In other embodiments, the posture reset operation described above may be configured to detect when the mouse 40 is lifted, and the posture reset operation may be treated as having been performed when the mouse is lifted. The method for detecting the lift is not limited. For example, mouse sensor data 604 or posture sensor data 605 may be used. Alternatively, the mouse 40 may be equipped with other sensors for detecting the lift. The mouse 40 may determine that the mouse has been lifted, or the information processing device 2 may determine that the mouse has been lifted based on the output from the mouse 40. When a user performs a lift-up action, it is assumed that they also intend to return the hand holding the mouse 40 to its original position. Therefore, by resetting the posture of the hand object 102 in accordance with such an action, when the lifted mouse 40 is returned to the work surface, the posture of the mouse 40 and the posture of the hand object 102 correspond, allowing the user to continue the operation without any discomfort. In this configuration, the finger movement required to press the button 42D is also unnecessary, and the posture reset of the hand object 102 can be performed by a natural and intuitive action during the process of operating the mouse 40. Furthermore, in addition to lifting the mouse, pressing the aforementioned button 42D or similar may be required as part of the posture reset operation. Alternatively, the posture reset may be performed when the lifted mouse 40 is returned to the work surface.

[0086] In another embodiment, if a posture reset operation is performed while the rod object 103 is in a movable state, the posture of the rod object 103 may also be returned to its initial posture.

[0087] If, as a result of the above determination, a posture reset operation is performed (YES in step S6), then in step S9, the processor 21 resets the posture of the hand object 102. In addition, along with this reset, the processor 21 may reset the current posture of the mouse 40 at this point as the mouse reference posture data 607 and initialize the differential posture data 608. After that, the process proceeds to step S21, which will be described later.

[0088] On the other hand, if the result of the determination in step S6 is that no posture reset operation has been performed (NO in step S6), then in step S10 of Figure 31, the processor 21 determines whether an action considered to be "rotation in place" has been performed on the mouse 40, based on the mouse sensor data 604 and the posture sensor data 605. This determination is made according to the principle described above. If the determination is made to be "rotation in place" (YES in step S10), then in step S11, the processor 21 determines the degree to which the mouse sensor data 604 is reflected in the amount of movement of the hand object 102, according to a parameter relating to the distance between the opening 45 and the center of rotation (for example, the distance d above). As an example, the degree of reflection may be smaller the smaller the parameter relating to the distance between the opening 45 and the center of rotation. That is, the closer the distance between the opening 45 and the center of rotation, the less likely it is that the user intends to move in parallel, so the amount of movement of the hand object 102 based on the mouse sensor data 604 may be made smaller. Next, in step S12, the processor 21 determines the amount of movement of the hand object 102 based on the reflection degree. Then, the process proceeds to step S15.

[0089] On the other hand, if the result of the determination in step S10 is not "rotation in place" (NO in step S10), the mouse 40 is considered to be in one of the following states: stopped, rotating and translating significantly, or translating without rotating. Therefore, in step S13, the processor 21 determines whether or not translation (not due to "rotation in place") has occurred in the mouse 40 based on the mouse sensor data 604 and the operation buffer 610. In other words, it is determined whether or not there has been no change in the position of the mouse 40 and the mouse 40 is not moving at all. If the result of this determination is that no translation has occurred (NO in step S13), the process proceeds to step S21 described below. On the other hand, if translation has occurred (YES in step S13), in step S14, the processor 21 determines the amount of movement of the hand object 102 based on the mouse sensor data 604. For example, the operation buffer 610 is referenced and the amount of movement is determined based on the magnitude of the change from the position of the mouse 40 in the previous frame to the current position.

[0090] Next, in step S15, the processor 21 calculates the difference in angle around the z-axis between the current posture of the mouse 40 and the reference posture described above, and records (or updates if already recorded) the difference posture data 608.

[0091] Next, in step S16, the processor 21 corrects the direction of movement derived from the mouse sensor data 604 based on the mouse sensor data 604 and the differential posture data 608, as shown in Figure 28 above, and determines the direction of movement of the hand object 102 based on this. First, the processor 21 calculates the direction of movement as seen from the mouse 40, as described above, based on the mouse sensor data 604. Next, the processor 21 converts the calculated direction of movement of the mouse 40 into a direction of movement within the screen, based on the initial correspondence above. Next, the processor 21 rotates the direction of movement within the screen by the angle indicated by the differential posture data 608. Then, the processor 21 determines the direction of movement after the rotation as the direction of movement of the hand object 102.

[0092] Next, in step S17, the processor 21 moves the hand object 102 based on the amount and direction of movement described above.

[0093] Next, in step S18, the processor 21 rotates the hand object 102 based on the amount of rotation calculated from the attitude sensor data 605.

[0094] Next, in step S19 of Figure 32, the processor 21 determines whether the hand object 102 is currently in a movable state based on the movable state flag 609. If the determination is that the hand object 102 is in a movable state (YES in step S19), in step S20, the processor 21 moves and rotates the rod object 103 in accordance with the movement and rotation of the hand object 102. For example, the processor 21 applies the amount of movement, direction of movement, and amount of rotation of the hand object 102, determined based on the mouse sensor data 604 and posture sensor data 605, to the rod object 103 to control its operation. Alternatively, the rod object 103 may be moved and rotated to follow the movement and rotation of the hand object 102.

[0095] On the other hand, if the result of the determination in step S19 is that the object is not in a movable state (NO in step S19), the process in step S20 is skipped. In other words, only the hand object 102 is moved and rotated.

[0096] Next, in step S21, the processor 21 performs collision detection between the rod object 103 and the obstacle object 101, etc. The processor 21 also executes various game processes, including processing based on the collision detection.

[0097] Next, in step S22, the processor 21 generates a game image that reflects the above processing and outputs it to the display unit 30.

[0098] Next, in step S23, the processor 21 determines whether a predetermined game termination condition has been met. For example, it determines whether the rod object 103 has reached the goal or whether the conditions for a game over have been met. If the game termination condition has not yet been met (NO in step S23), the process returns to step S3 and is repeated. If the condition is met (YES in step S23), the processor 21 terminates the game process.

[0099] This concludes the detailed explanation of the game processing in this embodiment.

[0100] Thus, in this embodiment, the output data from the mouse sensor 43 is handled differently depending on whether it is a "rotation in place" operation or not. As described above, even when a "rotation in place" operation is performed, unless the positions of the opening 45 and the rotation center coincide, the output of the mouse sensor 43 will also change in accordance with the translation of the mouse 40, and object movement control may be executed even if the user did not intend it. Therefore, when the conditions for being considered a "rotation in place" are met as described above, rotation control of the target object based on the posture sensor data 605 is executed, while the amount of translation of the target object is made smaller than in the case of a non-"rotation in place" operation. This makes it easier for the user's intentions to be reflected in the "rotation in place" operation.

[0101] [Differentiation] In the above embodiment, the mouse sensor 43 detected the movement of the mouse 40 and output the direction and amount of movement. In other embodiments, the mouse sensor 43 may output only data related to reflected light from the work surface, and the information processing device 2 may output whether the mouse has moved, the direction and amount of movement, etc., based on this data. Also, in the above embodiment, the information processing device 2 calculated the current position of the mouse 40 in the mouse coordinate system. However, the mouse sensor 43 may calculate the current position of the mouse 40 and transmit data related to this to the information processing device 2. Furthermore, neither the information processing device 2 nor the mouse sensor 43 is required to calculate the current position of the mouse 40. The same applies to the posture sensor 44; the actual posture may be calculated by either the information processing device 2 or the mouse 40.

[0102] The shape of the mouse 40 in the above embodiment is an example. For example, it may have a grip that is easy for the user to grasp and lift. For example, the mouse 40 may be used like a general game controller. That is, a game controller having a mouse sensor 43 is included in the scope of the mouse in this disclosure. The mouse 40 may also be detachable from other devices. For example, a pair of two mice may be used. In this case, one mouse may have an operating means on its left side, as in the mouse 40 in the above embodiment, and the other mouse may have an operating means on its right side.

[0103] In the above embodiment, the virtual object was movable in two dimensions on the screen, but it may be made movable only in one dimension. Also, the range of rotation of the virtual object may be limited. Furthermore, the rotation angle of the mouse 40 and the rotation angle of the virtual object do not have to match.

[0104] In the above embodiment, an image of the virtual object viewed from above was displayed on the screen, and the virtual object moved on a plane. In other embodiments, an image of the virtual object viewed from behind may be displayed on the screen. Furthermore, the virtual object may move on a curved surface instead of a plane. Such cases are also included in translation as described in this disclosure. The plane on which the virtual object moves may be an invisible plane, or it may not be defined as a plane, as long as the virtual object is translated as a result.

[0105] In the above embodiment, an example was shown in which the rotation of the mouse 40 is also reflected in the rotation of the hand object 102. In other embodiments, the rotation of the mouse 40 may be reflected only in the rod object 103, and the hand object 102 may not be rotated.

[0106] Furthermore, in the above embodiment, an example was shown in which a hand object 102 is displayed as a mouse pointer, and an operation experience is provided in which the user grasps and moves the rod object 103 with the hand object 102. In other embodiments, the game may be similar to the above but without the hand object 102. In this case, the translation and rotation operations of the mouse 40 are directly reflected in the movement and rotation of the rod object 103. When using such an operation system, for example, the movable state and the immovable state may be controlled to switch each time a left click operation (on / off operation of button 42A) is performed.

[0107] Furthermore, in the above embodiment, the target object was "moved" based on the output data from the mouse sensor 43, and the target object was "rotated" based on the output data from the posture sensor 44. In other embodiments, the output data from the posture sensor 44 may be used for object control other than "rotation". For example, if the mouse 40 is rotated clockwise (relative to the reference posture shown in Figure 24 above), the target object is in an "attack state" where it is performing an attack action. On the other hand, if the mouse 40 is rotated counterclockwise, the target object is in a "defense state" where it is performing a defensive action.

[0108] Furthermore, in the above embodiment, control was performed to reduce the amount of translation in the case of "rotation in place," and control was performed to correct the direction of movement based on the output from the mouse sensor 43. In other words, an example of using two types of control in combination was shown. In other embodiments, only one of the controls may be performed depending on the game content, etc. For example, in the case of "rotation in place," the control to reduce the amount of translation may not be performed, and only the correction control for the direction of movement as described above may be performed. Alternatively, the determination of whether or not it is "rotation in place" may not be performed, and only the correction control for the direction of movement may be performed at all times or at appropriate times. Conversely, control may be performed for "rotation in place," but no correction for the direction of movement may be performed.

[0109] Furthermore, in the above embodiment, control was performed to reduce the amount of translation when it was determined that "rotation in place" was occurring. However, in other embodiments, control may be performed to reduce the amount of translation (including to zero) without performing a binary determination of whether or not it is "rotation in place". For example, control may be performed to reduce the amount of translation according to the distance d. As an example, the amount of translation may be obtained by multiplying dx or dy by a value between 0 and 1, which approaches 0 as d becomes smaller. In this case, for example, if d is less than or equal to a predetermined value, the above value may be fixed to 0, or if d is greater than or equal to another predetermined value, the above value may be fixed to 1.

[0110] Furthermore, in the above embodiment, in the initial state, the forward / backward and left / right movement of the mouse 40 corresponded to the up / down and left / right directions on the screen, respectively. In other embodiments, a different correspondence may be set in the initial state. For example, as shown in Figures 33 and 34, the forward direction from the perspective of the mouse 40 corresponds to the left direction on the screen, and the right direction corresponds to the up direction on the screen. In other words, the correspondence between the forward / backward and left / right directions in the local coordinate system of the mouse 40 and the movement direction of the object 112 on the screen may be changed from the general correspondence beforehand. For example, as shown in Figures 33 and 34, if the virtual object operated by the mouse 40 is a horizontally elongated object 112, the user can intuitively operate it by rotating the mouse 40 90 degrees to operate it in a horizontally elongated state, due to the similarity in shape. Note that control to correct the direction of movement based on the output from the mouse sensor 43 as described above may or may not be performed. Also, "in-place rotation" determination and various controls based on the "in-place rotation" determination may or may not be performed.

[0111] Furthermore, the above example shows how to determine whether it is "rotation in place" by calculating the distance d and determining whether it is within a predetermined threshold. In other embodiments, for example, if the amount of translation detected by the mouse sensor 43 within a predetermined period is less than or equal to a threshold, it may be considered "rotation in place" and control may be performed so as not to translate the object being manipulated. In other words, "rotation in place" may be determined without using the output of the attitude sensor as described above.

[0112] Furthermore, although the threshold value compared with distance d in the above embodiment was a predetermined value, it may be variable. For example, the threshold value may be changed depending on the mouse used. Depending on the mouse, the length of the bottom surface in the front-to-back direction may differ, or the bottom surface may not be vertically elongated in the front-to-back direction but also have a certain width, and the tendency of the position of the rotation center when "rotating in place" is intended may also differ depending on how the user holds the mouse. In this case, multiple threshold values ​​corresponding to multiple mice may be prepared, and the threshold value corresponding to the mouse being used may be selected. For example, a model identification ID may be stored in the mouse itself for each mouse model. The information processing device 2 may then obtain the model identification ID from the mouse connected to it, select a threshold value corresponding to it, and use it for the judgment process described above. In addition, the information processing device 2 (including a server, etc.) may receive data from the mouse, such as the model identification ID or, as an example, data indicating the shape of the bottom surface of the mouse, and calculate or correct the threshold value based on this information. Note that instead of making the threshold value variable depending on the mouse, the distance d may be made variable, or the comparison method may be changed.

[0113] Furthermore, the above embodiment gave an example of applying the above-described process to a game in which a rod object 103 is moved to the goal point in a maze-like passage. In addition, the above process can also be applied to the following game processes. For example, it can be applied to a brick-breaking game. Specifically, the bar in the brick-breaking game is moved left and right by moving the mouse 40. Also, by rotating the mouse 40, the bar is rotated, and the direction in which the ball is hit can be controlled to some extent. Note that in this case, the bar on the screen may move left and right by moving the mouse 40 in the forward and backward directions. That is, the user may operate the mouse by placing their hand on the mouse so that the long side of the mouse 40 is facing sideways. Also, the bar may only be moved left and right. In such a game, a guide screen may be displayed so that the position obtained by rotating the mouse 40 by 90° from the position shown in Figure 24 is used as the reference position. When the game start operation is performed, an initial correspondence relationship may be defined in which the positive y-axis direction of the mouse 40's local coordinate system corresponds to the left or right direction on the screen. In other words, an initial correspondence may be defined such that the relationship between the x and y axes of the mouse 40's local coordinate system, as shown in Figure 2, and the x and y axes of the screen do not match. In this case, the process of correcting the direction of movement described above will further correct the direction of movement derived based on the initial correspondence defined here. Furthermore, this can also be applied to a top-down view 2D shooting game in which a "tank" is controlled as the target object. In this case, for example, the tank's body is moved by translating the mouse 40. Also, the tank's gun barrel is rotated by rotating the mouse 40 to change the firing direction. The above processing can be applied when using such a control system.

[0114] Furthermore, the above processing can be applied to processes other than games. For example, when performing an operation such as pressing a stamp image in a paint tool, it becomes possible to rotate the stamp image by rotating the mouse 40 before pressing it.

[0115] In other embodiments, for example, the detection result of the acceleration sensor may be used in conjunction with the determination of the distance d and the threshold. In the case of "rotation in place," the acceleration is also expected to be a small value. Therefore, by further considering whether the acceleration is small, the accuracy of determining whether or not the movement is "rotation in place" can be improved.

[0116] Furthermore, the above embodiment described a case in which the game processing described above is performed on a single information processing device 2. The information processing device 2 may include multiple storage devices and processors. The game processing may be performed by dividing the processing among these devices. The information processing device may also include a server. The above processing may be performed in a distributed system consisting of multiple information processing devices, including a server. [Industrial applicability]

[0117] The information processing method, information processing program, and information processing system described herein can improve operability when manipulating virtual objects using a mouse and are useful for various information processing applications using a mouse. [Explanation of Symbols]

[0118] 2. Information Processing Device 21 processors 22 Memory section 30 Display section 40 mice

Claims

1. Computers, A means for acquiring first data based on the output of an optical sensor installed in a mouse, Means for acquiring second data based on the output of a posture sensor provided on the mouse, Means for obtaining third data based on operations performed on the mouse, Means for translating the first virtual object based on the first data, Means for setting the second virtual object to a movable state in which it is translated based on the first data, based on the third data corresponding to a first operation performed on the mouse when the first virtual object and the second virtual object are in a predetermined positional relationship, In the aforementioned movable state, the second data is used as a means to correct the translation direction of the second virtual object based on the first data, based on the rotation of the mouse indicated by the second data. When not in the aforementioned movable state, the first virtual object is translated based on the first data without any correction of the translation direction based on the mouse rotation indicated by the second data. program.

2. The aforementioned computer, This system functions as a means for calculating differential posture data that shows the difference between the reference posture of the mouse recorded at a predetermined timing and the current posture of the mouse. The correction of the translation direction of the second virtual object is performed by rotating the translation direction based on the first data based on the angle indicated by the difference pose data. The program according to claim 1.

3. The aforementioned computer, In the aforementioned movable state, This is configured to function as a means for rotating the first virtual object based on the second data, The correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the first virtual object. The program according to claim 1.

4. The aforementioned computer, In the aforementioned movable state, This is configured to function as a means for rotating the second virtual object based on the second data, The correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the second virtual object. The program according to claim 1.

5. The aforementioned computer, The program according to any one of claims 1 to 4, wherein in the movable state, the program functions as a means for rotating the second virtual object by a different angle with respect to the rotation angle of the mouse indicated by the second data.

6. The program according to claim 5, wherein the different angle is a larger angle than the rotation angle of the mouse indicated by the second data.

7. An information processing method to be executed by the processor of an information processing device, The aforementioned processor, First data is acquired based on the output of an optical sensor installed in the mouse. Second data is acquired based on the output of the posture sensor provided in the mouse. A third data is obtained based on the operation performed on the mouse. Based on the first data, the first virtual object is translated, Based on the third data corresponding to a first operation performed on the mouse when the first virtual object and the second virtual object are in a predetermined positional relationship, the second virtual object is set to a movable state in which it is translated based on the first data. In the aforementioned movable state, the translation direction of the second virtual object based on the first data is corrected based on the rotation of the mouse indicated by the second data. An information processing method wherein, when the object is not in the aforementioned movable state, the first virtual object is translated based on the first data without any correction of the translation direction based on the rotation of the mouse indicated by the second data.

8. The aforementioned processor, The system calculates differential posture data that shows the difference between the reference posture of the mouse recorded at a predetermined timing and the current posture of the mouse. The correction of the translation direction of the second virtual object is performed by rotating the translation direction based on the first data based on the angle indicated by the difference pose data. The information processing method according to claim 7.

9. The aforementioned processor, In the aforementioned movable state, The first virtual object is rotated based on the second data. The correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the first virtual object. The information processing method according to claim 7.

10. The aforementioned processor, In the aforementioned movable state, The second virtual object is rotated based on the second data. The information processing method according to claim 7, wherein the correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the second virtual object.

11. The aforementioned processor, The information processing method according to any one of claims 7 to 10, wherein, in the movable state, the second virtual object is rotated by a different angle with respect to the rotation angle of the mouse indicated by the second data.

12. The information processing method according to claim 11, wherein the different angle is a larger angle than the rotation angle of the mouse indicated by the second data.

13. A system comprising a mouse having an optical sensor and a posture sensor, and an information processing device having a storage unit in which a program is stored and a processor, The aforementioned mouse, The first data based on the output of the optical sensor is transmitted. A second data based on the output of the attitude sensor is transmitted. A third data is transmitted based on the operation performed on the mouse. The program, when executed, causes the processor to The first data, the second data, and the third data are acquired. Based on the first data, the first virtual object is translated, Based on the third data corresponding to a first operation performed on the mouse when the first virtual object and the second virtual object are in a predetermined positional relationship, the second virtual object is set to a movable state in which it is translated based on the first data. In the aforementioned movable state, the translation direction of the second virtual object based on the first data is corrected based on the rotation of the mouse indicated by the second data. When not in the aforementioned movable state, the first virtual object is translated based on the first data without any correction of the translation direction based on the mouse rotation indicated by the second data. system.

14. The aforementioned processor, The system calculates differential posture data that shows the difference between the reference posture of the mouse recorded at a predetermined timing and the current posture of the mouse. The correction of the translation direction of the second virtual object is performed by rotating the translation direction based on the first data based on the angle indicated by the difference pose data. The system according to claim 13.

15. The aforementioned processor, In the aforementioned movable state, The first virtual object is rotated based on the second data. The system according to claim 13, wherein the correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the first virtual object.

16. The aforementioned processor, In the aforementioned movable state, The second virtual object is rotated based on the second data. The correction of the translation direction of the second virtual object is performed by rotating the translation direction of the second virtual object in accordance with the rotation angle of the second virtual object. The system according to claim 13.

17. The aforementioned processor, The system according to any one of claims 13 to 16, wherein, in the movable state, the second virtual object is rotated by a different angle with respect to the rotation angle of the mouse indicated by the second data.

18. The system according to claim 17, wherein the different angle is a larger angle than the rotation angle of the mouse indicated by the second data.

Images