Information processing program, information processing system, and information processing method

Abstract

To improve the usability of gyro correction according to an input device.SOLUTION: On the basis of satisfaction of a first condition, an information processing program causes an operation mode to shift from a first operation mode in which a first object is operated on the basis of mouse sensor data of an input device to a second operation mode in which a second object is operated on the basis of gyro sensor data of the input device, and executes gyro correction in a direction perpendicular to the direction of gravity.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to information processing. [Background technology]

[0002] Conventionally, input devices have sometimes been equipped with gyro sensors. The gyro sensor can be used to calculate the current orientation of the input device by integrating changes in the orientation relative to a reference orientation. However, it is known that errors in the gyro sensor's measurement values ​​accumulate, resulting in an increase in the orientation error of the input device. For example, when a user is playing a game using a game controller equipped with a gyro sensor, the above-described errors can cause the user to feel uncomfortable with the correspondence between the actual orientation of the game controller and processing corresponding to the orientation of the game controller. In such cases, the user may perform gyro correction to eliminate the above-described errors, for example, by pressing a specific button (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-217139 Summary of the Invention [Problem to be solved by the invention]

[0004] There was room for improvement in the usability of gyro correction depending on the input device. [Means for solving the problem]

[0005] For example, the following configuration example can be given.

[0006] (Configuration example 1) Configuration example 1 is an information processing program executed by a computer including one or more processors, which causes the processor to operate a first object in a first operation mode based on mouse sensor data output from a mouse sensor of an input device, and to operate a second object in a second operation mode based on gyro sensor data of the input device, and transition from the first operation mode to the second operation mode based on satisfaction of a first condition, and perform gyro correction of the orientation in a direction perpendicular to the direction of gravity.

[0007] (Configuration example 2) Configuration example 2 is configured such that in configuration example 1, the processor executes gyro correction such that the direction in which the component of the longitudinal direction of the input device that is perpendicular to the direction of gravity faces is the front.

[0008] (Configuration example 3) Configuration example 3 is the same as configuration example 1 or 2, in which the processor is not caused to operate the first object based on gyro sensor data of the input device in the first operation mode.

[0009] (Configuration Example 4) Configuration example 4 is any one of the above configuration examples 1 to 3, in which the first object and the second object are different objects.

[0010] (Configuration Example 5) Configuration example 5 is any one of the above configuration examples 1 to 4, wherein a light aperture for the mouse sensor is provided on the bottom surface of the input device, and the first condition includes a condition that is met when the light aperture is not blocked.

[0011] (Configuration Example 6) Configuration example 6 is any of configuration examples 1 to 5 above, in which a light intake port for the mouse sensor is provided on the bottom surface of the input device, and the first condition includes that the longitudinal axis of the input device, which is parallel to the bottom surface and extends from the front to the back when the input device is held by a user, is inclined by a predetermined angle or more with respect to a direction perpendicular to the direction of gravity.

[0012] (Configuration Example 7) In configuration example 7, in configuration example 6, the first condition does not include a condition regarding the inclination of the short axis of the input device, which is parallel to the bottom surface and perpendicular to the long axis, with respect to a direction perpendicular to the direction of gravity.

[0013] (Configuration Example 8) Configuration example 8 is any one of configuration examples 1 to 7, and causes the processor to perform gyro correction based on whether the first condition is satisfied.

[0014] (Configuration Example 9) Configuration example 9 is any one of configuration examples 1 to 8, in which the processor controls the second object based on a movement in a rotational direction around a longitudinal axis facing the longitudinal direction of the input device.

[0015] The configuration examples described above may be interpreted as information processing systems or information processing methods. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the internal configuration of a game device 10. [Figure 2] Schematic diagram showing an example of the appearance of the right controller 15 and the left controller 16. [Figure 3] FIG. 10 is a diagram illustrating an example of how to hold the right controller 15. [Figure 4] FIG. 10 is a diagram illustrating an example of how to hold the right controller 15. [Figure 5] FIG. 1 is a diagram for explaining a first example of the present embodiment. [Figure 6] FIG. 1 is a diagram for explaining a first example of the present embodiment. [Figure 7] FIG. 10 is a diagram for explaining an example in which gyro correction is not performed in the first example of the present embodiment. [Figure 8] FIG. 10 is a diagram for explaining a second example of the present embodiment. [Figure 9] FIG. 10 is a diagram for explaining a second example of the present embodiment. [Figure 10] FIG. 10 is a diagram for explaining an example in which gyro correction is not performed in a second example of the present embodiment. [Figure 11] FIG. 1 is a diagram showing an example of various data stored in a storage unit (memory) 12. [Figure 12] FIG. 1 is a diagram showing an example of a flowchart of information processing; DETAILED DESCRIPTION OF THE INVENTION

[0017] An embodiment will be described below.

[0018] [Hardware configuration of information processing device] An information processing system for executing information processing according to this embodiment will be described. This information processing system is, for example, an information processing device such as a game device, a personal computer, a tablet terminal, a smartphone, a wearable terminal, or a server. Note that the information processing system according to this embodiment may be composed of multiple information processing devices, and may be composed of, for example, the above-mentioned game device and a server. In this embodiment, a game device will be described as an example of an information processing system and an information processing device.

[0019] FIG. 1 is a block diagram showing an example of the internal configuration of a game device 10 according to this embodiment. The game device 10 includes a processor 11. The processor 11 is an information processing unit that executes various types of information processing executed in the game device 10. The processor 11 may be composed of, for example, multiple processors or cores, typically multiple central processing units (CPUs) or cores, or may be composed of a system-on-a-chip (SoC) that includes multiple functions such as a CPU function and a graphics processing unit (GPU) function. The processor 11 executes various types of information processing by executing an information processing program (e.g., a game program) stored in a storage unit 12. The storage unit 12 may be, for example, an internal storage medium such as a flash memory or a dynamic random access memory (DRAM), or may be configured to use an external storage medium inserted into a slot (not shown). In this embodiment, the term "processor" may include at least a CPU, a GPU, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. In the present embodiment, the computer includes, for example, at least one processor and may further include a storage unit such as a memory. When the information processing system includes multiple information processing devices, each of the information processing devices may include at least one processor and may also include a storage unit.

[0020] The game device 10 includes a controller communication unit 13 for performing at least wireless communication with the right controller 15 and the left controller 16. The controller communication unit 13 may control wired communication between the game device 10 and the right controller 15 and the left controller 16, which are input devices. The controller communication unit 13 may be included in the processor 11.

[0021] A display unit 17 (e.g., a display) is connected to the game device 10 by wire or wirelessly via an image and sound output unit 14. The processor 11 outputs images and sounds generated by executing the above-described information processing, for example, via the image and sound output unit 14 to the display unit 17, which is also capable of sound output.

[0022] The right controller 15 is equipped with an inertial sensor. Specifically, the right controller 15 is equipped with an acceleration sensor 15c and a gyro sensor (which may also be called an angular velocity sensor) 15d. The acceleration sensor 15c detects the magnitude of acceleration along three predetermined axes (x, y, and z axes of the right controller coordinate system shown in FIG. 2(2)) that are orthogonal to each other. The acceleration sensor 15c may be configured to detect acceleration along one or two axes as needed. The gyro sensor 15d detects angular velocity around the above-mentioned three predetermined axes. The gyro sensor 15d may be configured to detect angular velocity around one or two axes as needed. The detection results by the acceleration sensor 15c and the gyro sensor 15d are repeatedly transmitted to the controller communication unit 13 at appropriate timing.

[0023] The right controller 15 is equipped with a mouse sensor 15e. As will be described later, the mouse sensor 15e acquires data that enables calculation of the movement of the right controller 15 due to mouse operation. This data is repeatedly transmitted to the controller communication unit 13 at appropriate timing.

[0024] The right controller 15 includes buttons 15f and an analog stick 15g. The analog stick (sometimes simply referred to as a "stick") 15g can be used as a direction control unit that can input directions. By tilting the stick 15g in any direction, the user can input a direction according to the tilt direction, and the input size can be determined according to the tilt angle. Data indicating the operation status of the buttons 15f and the stick 15g is repeatedly transmitted to the controller communication unit 13 at appropriate timings. The direction control unit may be a sliding stick, a directional key, or a set of four buttons instead of or in addition to the stick 15g.

[0025] The right controller 15 includes a processor 15a and a storage unit 15b. The processor 15a can acquire output data from, for example, an acceleration sensor 15c, a gyro sensor 15d, a mouse sensor 15e, a button 15f, and a stick 15g, and can perform various processes using the acquired data. For example, the processor 15a can use the acquired data to determine various operations performed on the right controller 15.

[0026] The left controller 16 includes a processor 15a, a memory unit 15b, an acceleration sensor 15c, a gyro sensor 15d, a mouse sensor 15e, buttons 15f, and a stick 15g that are included in the right controller 15, as well as a memory unit 16b, an acceleration sensor 16c, a gyro sensor 16d, a mouse sensor 16e, buttons 16f, and a stick 16g that have similar functions.

[0027] The right controller 15, the left controller 16 and the display unit 17 may or may not be considered to be included in the game device 10.

[0028] FIG. 2(1) is a schematic diagram showing an example of the appearance of the right controller 15. As shown in FIG. 2(1), the right controller 15 has, as an example, a plate shape with the y-axis direction as the longitudinal direction (a rectangular parallelepiped or a shape similar thereto in which the thickness in the x-axis direction is smaller than the thickness in the y-axis direction and the thickness in the z-axis direction, and the thickness in the z-axis direction is smaller than the thickness in the y-axis direction) (see the mutually orthogonal xyz coordinate system shown in FIG. 2(1)). The right controller 15 may have other shapes. The y-axis may be called the longitudinal axis, and the x-axis and z-axis may be called the short axes.

[0029] As shown in FIG. 2(1), the right controller 15 has a mouse sensor opening (which may also be called a "lighting port") 20 on its bottom. The mouse sensor opening 20 is an opening in a light guide path that guides light to the mouse sensor 15e located inside the opening 20. The mouse sensor 15e may be, for example, an optical mouse sensor and may include a light-emitting unit and a light-receiving unit. The light detected by the light-receiving unit may be visible light or light of an invisible wavelength. The mouse sensor 15e acquires data that enables calculation of the movement of the controller 15 on a work surface when the controller 15 is placed with its bottom facing the work surface. This allows the right controller 15 to be used as a mouse. This operation using the right controller 15 as a mouse is sometimes referred to as "mouse operation." The mouse sensor may be, for example, a sensor that detects the movement of a trackball. The work surface is not limited to a flat surface, but may be a curved surface, such as the surface of the user's thigh.

[0030] 2(1), the right controller 15 also includes a button 15f. The button 15f is provided, for example, at the tip of the front part at the top opposite the bottom. The stick 15g is provided, for example, on the left part, in a position that is easy to operate with the thumb when the user holds the right controller 15 in their right hand and operates the mouse (see FIG. 3).

[0031] As shown in Figure 2(2), the left controller 16 differs from the right controller 15 in that, for example, a stick (16g) is provided on the right side. Also, as shown in Figure 2(2), the mutually orthogonal xyz coordinate system defined on the left controller 16 has the z axis in the opposite direction to the xyz coordinate system defined on the right controller 15 (see Figure 2(1)).

[0032] There are no limitations on the number, shape, position, or type of directional control units and buttons included in the right controller 15 and the left controller 16. The right controller 15 and the left controller 16 may also include other control units operated by the user. Only one of the right controller 15 or the left controller 16 may include an inertial sensor and / or a mouse sensor.

[0033] [Examples of how to hold the controller] FIG. 3 is a schematic diagram showing an example of a state in which a user holds the right controller 15 in his / her right hand 25, places it on a work surface, and uses it as a mouse, that is, operates the mouse. FIG. 3 is a diagram seen from above in real space, as an example. As shown in FIG. 3, the user can perform mouse operations by moving the right controller 15 on the work surface, press the button 15f with his / her index finger or middle finger, and operate the stick 15g with his / her thumb. The right controller 15 is a controller for the right hand, with the stick 15g located in a position that is easy to operate with the thumb of the right hand. The following mainly describes a case in which a user operates the right controller 15 with his / her right hand. Note that the same applies to a case in which a user holds the left controller 16 in his / her left hand, places it on a work surface, and uses it as a mouse, so a description thereof will be omitted.

[0034] Fig. 4 is a schematic diagram showing an example of a state in which a user holds the right controller 15 in their right hand and operates it in the air. Fig. 4 is a diagram seen from above in real space, as an example. As shown in Fig. 4, the right controller 15 can be held and used so that the longitudinal direction of the right controller 15 is the up-down direction or the front-back direction for the user. Note that the left controller 16 can be used in the same manner when the user uses it with their left hand.

[0035] 4, the user can perform an operation (sometimes called a "gyro operation") to change the attitude of the right controller 15. Note that when the user holds and operates the left controller 16 in the left hand, the gyro operation can be performed in the same manner.

[0036] [Outline of information processing in this embodiment] Next, an overview of the information processing executed by the game device 10 according to this embodiment will be described. Hereinafter, this embodiment will be described with reference to a first example and a second example.

[0037] [First Example] The first embodiment is a game in which it is possible to switch between a gyro operation mode in which the movement of a reticle displayed on the screen of the display unit 17 (sometimes referred to as "screen 17") is controlled in response to gyro operation, and a mouse operation mode in which the movement of a mouse cursor displayed on screen 17 is controlled in response to mouse operation. Note that, for example, in the gyro operation mode, a bullet is fired at an enemy character using the reticle, and in the mouse operation mode, a weapon or item is selected using the mouse cursor, and the game progresses. The same reticle or cursor may be used in the gyro operation mode and the mouse operation mode, and the same in-game functions may be performed.

[0038] 5 and 6 are diagrams for explaining gyro correction in Example 1. Fig. 5(1) shows a scene in which, in gyro operation mode, the user holds right controller 15 with right hand 25 and performs gyro operation to manipulate the position of aim 100 displayed on screen 17.

[0039] In FIG. 5(1), the screen 17 is set facing the user, and the right controller 15 is held in the air by the user with its front facing the screen 17 (in other words, with the positive y-axis direction (see FIG. 2(1)) of the right controller 15 facing the screen 17). As an example, in the game of this embodiment, when performing gyro operation, the user is expected to operate the right controller 15 with the front facing the screen 17. In FIG. 5(1), the aim 100 is located in the center of the screen 17 depending on the orientation of the right controller 15. Note that in FIG. 5(1), (b) is a view seen from above, and (a) is a view seen horizontally, for convenience of illustration. The same is true for (a) and (b) of FIG. 5(2) and (b) of FIG. 6.

[0040] As shown in Figure 5(1)(b), consider mutually orthogonal x, y and z axes (sometimes called "fixed real space axes" or "A axis") that define directions in real space. The A axis is an arbitrary fixed axis in real space, and in this description, the positive y axis direction is the direction opposite to the direction of gravitational acceleration (sometimes called the "gravity direction"), and the positive x axis direction is the direction toward the screen 17.

[0041] Then, the processor 11 sets mutually orthogonal x, y and z axes (sometimes referred to as "reference real space axes" or "B axes") that serve as a reference for the attitude of the controller (see FIG. 5(1)(b)). For the controller or the processor 11, the B axis is considered to be fixed in real space, and rotation of the controller is recognized based on this B axis. For convenience of explanation, in the state shown in FIG. 5(1), the B axis is set to the same direction as the A axis. The y axis direction of the B axis is set using the direction of gravity that can be measured by the acceleration sensor 15c. The x axis and z axis directions of the B axis are set, for example, by performing gyro correction, which will be described later.

[0042] The processor 11 also calculates the direction in which the mutually orthogonal x, y and z axes (sometimes referred to as the "controller attitude axes" or "C axes") fixed to the right controller 15 are oriented relative to the B axis by integrating the rotation angle of the right controller 15 using the gyro sensor 15d. If the orientations of the A axis and the B axis match, the orientation of the C axis relative to the B axis indicates the orientation of the right controller 15 in real space.

[0043] The orientation of the C axis relative to the B axis is calculated by integrating the rotation angle of the right controller 15 as described above, and thus deviations may occur. That is, if the B axis is considered to be fixed relative to the A axis, deviations may occur between the orientation of the C axis relative to the A or B axis and the orientation of the real right controller 15 relative to the A or B axis. Alternatively, if the C axis is considered to be fixed to the real right controller 15, deviations may occur between the orientation of the A axis, i.e., the fixed axis in real space, and the orientation of the B axis, i.e., the axis in virtual real space for the right controller 15. In this specification, "gyro deviation" may sometimes refer to the latter. Note that the orientation relative to the direction of gravity can be corrected by using an acceleration sensor. Therefore, deviations around the B axis around the y axis (the direction in which the x and z axes point) may sometimes be referred to as "gyro deviation." In this specification, "gyro correction" may sometimes refer to correction to suppress gyro deviation. As an example, "gyro correction" may mean correcting the orientation of the B axis around the y axis.

[0044] 5(2) shows a scene in which, in mouse operation mode transitioned from the gyro operation mode shown in FIG. 5(1), the user is holding the right controller 15 in the right hand 25 and operating the mouse on the work surface to control the position of the mouse cursor (sometimes simply referred to as "cursor") 250 displayed on the screen 17. The position of the cursor 250 is controlled based on data output from the mouse sensor 15e that enables calculation of the movement distance and movement direction of the right controller 15 within the yz plane. In this embodiment, in mouse operation mode, object control based on changes in the attitude of the right controller 15, which is performed in gyro operation mode, is not performed.

[0045] In the gyro operation mode, if it is determined that the right controller 15 is in a state where it can be operated as a mouse, the mode is switched to the mouse operation mode. For example, if it is determined that the bottom of the right controller 15 is covered by a work surface or the like and that the longitudinal direction of the right controller 15 (i.e., the y-axis direction; see FIG. 2(1)) is approximately horizontal (for example, within a range of ±3°, ±5°, or ±8° from horizontal), it is determined that it is in a state where it can be operated as a mouse, and the mode is switched to the mouse operation mode. By setting the above two conditions for switching to the mouse operation mode as described above, it is possible to prevent a transition to the mouse operation mode against the user's will, for example, if the user unintentionally blocks the opening of the mouse sensor with their finger or if the controller is turned sideways. Note that it may be determined that it is in a state where it can be operated as a mouse only if it is determined that the bottom is covered or only if it is determined that the longitudinal direction is approximately horizontal. The condition for determining that it is in a state where it can be operated as a mouse may also be other conditions.

[0046] In FIG. 5(2), gyro deviation accumulated during operation in mouse operation mode occurs on the B axis.

[0047] Figure 6 shows a scene in which the right controller 15 is lifted from the work surface during the mouse operation mode shown in Figure 5(2). When the right controller 15 is in a state in which it can be gyro-operated, the processor 15a changes the operation mode to the gyro operation mode. As an example, when the mouse sensor 15e detects that the bottom of the right controller 15 is not blocked by a work surface or the like, the operation mode is changed to the gyro operation mode. There are no particular restrictions on the conditions for changing the operation mode to the gyro operation mode, and the attitude of the controller may be used, for example.

[0048] When the operation mode is set to the gyro operation mode, the processor 15a executes "gyro correction" to correct the orientation of the B axis. Specifically, in this embodiment, as shown in Fig. 6(b), the positive x-axis direction of the B axis is corrected to be the same as the positive y-axis direction of the C axis, more specifically, the direction of the component of the positive y-axis direction of the C axis that is perpendicular to the direction of gravity.

[0049] In this embodiment, in gyro operation mode, it is assumed that the user operates the right controller 15 so that the front of the controller faces the screen 17. Therefore, as described above, the longitudinal direction of the controller faces the front in real space, i.e., the positive x-axis direction of the A-axis, and the positive x-axis direction of the B-axis is corrected to match this direction, thereby enabling gyro correction to be performed with less discomfort to the user. In this embodiment, when gyro correction is performed, that is, when the direction of the directional component perpendicular to the direction of gravity in the positive y-axis direction of the C-axis matches the positive x-axis direction of the B-axis, the crosshair 100 is set to be centered in the left-right direction. Therefore, by performing gyro correction, the user can move the crosshair 100 left and right immediately after switching from mouse operation to gyro operation.

[0050] Fig. 7 is a diagram for explaining a case where gyro correction is not performed when transitioning from the mouse operation mode shown in Fig. 5(2) to the gyro operation mode. As shown in Fig. 7, gyro correction is not performed and the direction of the B axis around the y axis is not corrected, so the processor 11 recognizes that the front part of the right controller 15 is facing rightward, and as a result, the crosshair 100 is displayed shifted to the right side of the screen 17.

[0051] [Second Example] The second embodiment is a game in which the player operates the right controller 15 and the left controller 16 to move or change the direction of a character in a wheelchair, or to wave the character's hands.

[0052] 8 and 9 are diagrams for explaining gyro correction in the second embodiment. FIG. 8(1) shows a scene in which, in mouse operation mode, an operation is being performed on a field in a virtual space to move a wheelchair carrying a character. In this game, the user can move or change the direction of the wheelchair by operating the right controller 15 with the mouse to rotate the right wheel 202 of the wheelchair, and by operating the left controller 16 with the mouse to rotate the left wheel 201 of the wheelchair. In FIG. 8(1), the wheelchair and the character are objects to be operated. As will be described later, the character's right hand 204 and left hand 203 may also be objects to be operated.

[0053] By moving the right controller 15 on the work surface in the positive direction of the y axis in Figure 2(1), the user can move the right hand 204 to rotate the wheel 202 in a direction that causes the right side of the wheelchair to move forward, and by moving the right controller 15 on the work surface in the negative direction of the y axis in Figure 2(1), the user can move the right hand 204 to rotate the wheel 202 in a direction that causes the right side of the wheelchair to move backward. The left wheel 201 is operated in the same way. Below, the operation of the right controller 15 will be explained, and an explanation of the operation of the left controller 16 will be omitted as it is similar.

[0054] In FIG. 8(1), the screen 17 is set facing the user, and the right controller 15 is placed on the work surface with its front facing the screen 17 (in other words, with the positive y-axis direction of the right controller 15 (see FIG. 2(1)) facing the screen 17). In this embodiment, when the user operates the mouse, it is assumed that the user will operate the right controller 15 with the front facing the screen 17. Note that in FIG. 8(1), for convenience of illustration, (b) is a view from above, and (a) is a view from the horizontal direction (i.e., toward the user). Also, in FIG. 8(2), (b) is a view from above.

[0055] The definitions of the A-axis, B-axis, and C-axis shown in FIGS. 8 and 9 are the same as the definitions of the A-axis, B-axis, and C-axis shown in FIG. 5 and the like.

[0056] In Figure 8(1), gyro misalignment accumulated during operation in mouse operation mode, etc., occurs on the B axis. Specifically, a gyro misalignment occurs in which the positive x-axis direction of the B axis is misaligned by 90° with the positive y-axis direction of the C axis (which can also be said to be the positive x-axis direction of the A axis).

[0057] FIG. 8(2) shows a situation in which, during the mouse operation mode shown in FIG. 8(1), the right controller 15 is lifted from the work surface, for example, and the y-axis direction of the C-axis forms an angle of 30° with the horizontal plane. Note that, because the deviation of the B-axis from the direction of gravity can be corrected by an acceleration sensor, the horizontal plane may be considered to be based on either the A-axis or the B-axis. In FIG. 8(2), the work surface is a horizontal plane. When the processor 11 determines that the y-axis direction of the C-axis forms an angle of 30° with the horizontal plane, it performs gyro correction. This determination can be made, for example, using the direction of the horizontal plane detected using the acceleration sensor 15c. The processor 11 may also perform gyro correction by determining that the bottom of the right controller 15 is not blocked by a work surface or the like. The gyro correction is performed by aligning the orientation of the B-axis around the y-axis with the orientation of the C-axis around the x-axis, as shown in FIG. 8(2)(b). More specifically, the gyro correction is performed so that the positive x-axis direction of the B axis is in the same direction as the component of the positive y-axis direction of the C axis that is perpendicular to the direction of gravity.

[0058] Fig. 9(1) shows a scene in which, after gyro correction has been performed and switching has been made to gyro operation mode, the user is holding the right controller 15 in the right hand 25 and performing gyro operation. Note that Fig. 9(1) and Fig. 9(2)(a) and (b) are both views viewed horizontally.

[0059] In the second example, the position and orientation of the character's right hand 204 are controlled in accordance with changes in the posture of the right controller 15 in the gyro operation mode. For example, as shown in (b) of Figure 9(1), when the right controller 15 is raised so that the front of the right controller 15 faces directly upward, the character's right hand 204 is lifted directly upward.

[0060] The right controller 15 is also controlled so that the direction in which the bottom of the right controller 15 faces corresponds to the direction in which the palm of the character's right hand 204 faces. For example, when the bottom faces downward, the palm of the right hand 204 faces downward, and when the bottom faces forward, the palm of the right hand 204 faces forward. For example, in response to an operation of rotating the bottom around the y-axis of the C-axis (which can also be considered an operation of twisting the user's right hand), the right hand 204 also moves in a twisting motion. In FIG. 9(1)(b), the direction around the y-axis of the B-axis has been corrected to the correct direction by gyro correction, so that the palm of the character's right hand 204 faces in the appropriate direction in response to the orientation of the bottom of the right controller 15. In other words, when the bottom of the right controller 15 faces forward, the palm of the character's right hand 204 also faces forward.

[0061] 9(2), for example, when a gyro operation is performed to tilt the y-axis of the right controller 15 to the right, the character's right hand 204 is tilted to the right accordingly. In this way, by tilting the right controller 15, the character's right hand 204 can be moved, so that the character can be made to intuitively perform actions such as waving its hand.

[0062] 10 is a diagram illustrating a case where gyro correction is not performed and the direction of the B axis around the y axis is not corrected when transitioning from mouse operation mode to gyro operation mode as shown in FIG. 8 etc. In this case, the processor 11 recognizes that the bottom of the right controller 15 is facing right. Therefore, as shown in FIG. 10(1), the palm of the character's right hand 204 faces right. Furthermore, as shown in FIG. 10(2), when a gyro operation is performed to tilt the y axis of the right controller 15 rightward, as in FIG. 9(2), the character's right hand 204 is tilted forward.

[0063] [Information processing in this embodiment] The information processing of this embodiment will be described with reference to FIGS.

[0064] [About data usage] The following describes various types of data stored in the storage unit 12. Fig. 11 shows an example of data stored in the storage unit 12 of the game device 10. As shown in Fig. 11, the storage unit 12 is provided with at least a program storage area 300 and a data storage area 400.

[0065] The program memory area 300 stores at least a program 301. The data memory area 400 stores at least mouse sensor data 401, gyro sensor data 404, acceleration sensor data 405, button / stick data 406, axis data 407, operation mode data 408, object data 409, image data 410, and virtual camera control data 411.

[0066] The program 301 is a game program for executing game processing.

[0067] The mouse sensor data 401 is data relating to the outputs of the mouse sensor 15e and the mouse sensor 16e, and includes image clarity data 402 and dy / dz data 403.

[0068] The image clarity data 402 is calculated by the mouse sensor 15e and the mouse sensor 16e, respectively, and indicates the clarity of the mouse sensor image. The image clarity data 402 may be calculated based on, for example, the brightness of the mouse sensor image or the number of feature points in the mouse sensor image. The image clarity data 402 may be calculated by a processor in the controller, the processor 11, or the like, based on the output data of the mouse sensor. The brightness of the mouse sensor image or the number of feature points in the mouse sensor image may directly serve as the image clarity data. The image clarity data 402 may also be calculated based on other factors. If the clarity indicated by the image clarity data 402 is equal to or greater than a predetermined value, it can be estimated that the opening of the mouse sensor is blocked by a mounting surface or the like. Other data that can be used to estimate that the opening of the mouse sensor is blocked may be used instead of or in addition to the data indicating the clarity of the mouse sensor image.

[0069] The dy / dz data 403 is output data from the mouse sensors 15e and 16e, and indicates the movement distance per frame time (sometimes referred to as "dy / dz") in the y-axis direction and z-axis direction of the controller coordinate system (i.e., the yz plane; see FIGS. 2(1) and 2(2)) relative to the work surface, etc., when the opening of the mouse sensor is blocked by the work surface, etc. Note that dy / dz may be calculated from the output data of the mouse sensors by a processor provided in the controller, the processor 11, etc.

[0070] The gyro sensor data 404 is data output from the gyro sensors of the mouse sensor 15e and the mouse sensor 16e, and is data that enables calculation of, for example, angular velocity around the x, y, and z axes of the controller coordinate system (see FIGS. 2(1) and 2(2)). Using the gyro sensor data, for example, the attitude of the controller can be calculated.

[0071] The acceleration sensor data 405 is data output from the acceleration sensors of the mouse sensor 15e and the mouse sensor 16e, and is data that enables calculation of acceleration in the x, y, and z-axis directions of the controller coordinate system (see FIGS. 2(1) and 2(2)). Using the acceleration sensor data, for example, the movement of the controller and the direction of gravity can be calculated.

[0072] It should be noted that at least one of the gyro sensor data 404 and the acceleration sensor data 405 may be used to calculate the attitude and movement of the controller.

[0073] The button / stick data 407 is data that indicates the operating state of the buttons and sticks of the controller.

[0074] The axis data 407 is data indicating the directions of the B axis and C axis described with reference to FIG. 5 and the like.

[0075] The operation mode data 408 is data that indicates whether the mouse operation mode or the gyro operation mode is set for each of the right controller 15 and the left controller 16.

[0076] The object data 409 is data on virtual objects to be placed in a virtual space, such as the crosshair 100, the cursor 250, the object 200, the ground, buildings, etc. The object data 409 includes information on the position and posture of each virtual object.

[0077] The image data 410 is image data such as an animation image, a background, a virtual effect, etc. The crosshair 100 and the cursor 250 may be images instead of objects.

[0078] The virtual camera control data 411 is data for controlling a virtual camera that is placed in a virtual space and captures images of the virtual space.

[0079] In addition, the storage unit 12 stores various types of data used in drawing processing and the like as needed.

[0080] [Example of detailed information processing] Next, the processing according to this embodiment will be described with reference to flowcharts and the like. FIG. 12 is an example of a flowchart showing the processing according to this embodiment. Below, the processing of the first example and the second example will be described using FIG. 12. In the second example, the right controller 15 and the left controller 16 are used, so the processing shown in FIG. 12 is executed for each controller. Below, the processing characteristic of this embodiment will be mainly described, and other explanations such as drawing processing will basically be omitted. The processing may include other processing, or may not include some processing. The following processing is executed repeatedly at a predetermined interval (for example, a processing frame interval executed every 1 / 60 seconds).

[0081] When this game processing starts, in step S101, processor 11 determines whether or not the controller is in the gyro operation mode based on operation mode data 408. If the determination in step S101 is YES, the processing proceeds to step S102, and if the determination is NO, the processing proceeds to step S105.

[0082] In step S102, processor 11 executes processing in the gyro operation mode based on gyro sensor data 404, etc. For example, in the first embodiment, the position of aim 100 is controlled in response to gyro operation (see FIG. 5(1)), and a bullet is fired in the direction of aim 100 in response to a button press operation. For example, in the second embodiment, the movement of the character's hand is controlled in response to gyro operation (see FIG. 9). Thereafter, the processing proceeds to step S103.

[0083] In step S103, processor 11 determines whether the condition for transitioning to the mouse operation mode has been satisfied. For example, in the first and second embodiments, it may be determined that the condition for transitioning to the mouse operation mode has been satisfied when it is determined, based on mouse sensor data 401, gyro sensor data 404, etc., that the bottom of the controller is covered by a work surface or the like and that the longitudinal direction of the controller is approximately horizontal, as described in the description of FIG. 5. If the determination in step S103 is YES, the process proceeds to step S104, and if the determination is NO, the process proceeds to step S109.

[0084] In step S104, the processor 11 shifts the operation mode of the controller to the mouse operation mode, and then the process proceeds to step S109.

[0085] In step S105, the processor 11 executes processing in the mouse operation mode based on the mouse sensor data 401, etc. For example, in the first embodiment, the position of the cursor 250 is controlled in response to the mouse operation (see FIG. 5(2)). For example, in the second embodiment, the movement of the object 200 is controlled in response to the mouse operation (see FIG. 8(1)). Here, in the mouse operation mode, object control based on the gyro sensor data 404 is not executed, and the object is not operated based on the gyro sensor data 404. Thereafter, the processing proceeds to step S106.

[0086] In step S106, the processor 11 determines whether the transition condition to the gyro operation mode has been satisfied. For example, in the first embodiment, the transition condition to the mouse operation mode may be determined to be satisfied when it is determined based on the mouse sensor data 401 that the controller has been lifted from the work surface, as described with reference to FIG. 6 . For example, the transition condition to the mouse operation mode may be determined to be satisfied when it is determined that the clarity of the mouse sensor image indicated by the image clarity data 402 has fallen below a predetermined level. Alternatively, for example, if a distance sensor is provided on the bottom of the controller, and it is determined that the distance to the work surface or the like has reached a predetermined level, it may be determined that the controller has been lifted from the work surface, and it may be determined that the transition condition to the mouse operation mode has been satisfied. Alternatively, for example, if a button is provided on the bottom of the controller, and it is determined that the controller has been lifted from the work surface based on the pressed state of the button, it may be determined that the transition condition to the mouse operation mode has been satisfied. For example, in the second embodiment, it may be determined that the condition for transitioning to the mouse operation mode is satisfied when it is determined that the y-axis direction of the C-axis forms an angle of 30° with the horizontal plane, as described with reference to FIG. 8(2), based on the acceleration sensor data 405 and the axis data 407, etc. Furthermore, two or more of the above determination conditions may be combined, or other determination conditions may be used. If the determination in step S106 is YES, the process proceeds to step S107, and if the determination is NO, the process proceeds to step S109.

[0087] In step S107, the processor 11 switches the operation mode of the controller to the gyro operation mode, and then the process proceeds to step S108.

[0088] In step S108, the processor 11 performs gyro correction as described with reference to Fig. 6(1) and Fig. 8(2) based on the axis data 407. After that, the process proceeds to step S109.

[0089] In step S109, the processor 11 corrects the y-axis positive direction of the reference real space axis (B-axis) to be opposite to the direction of gravity, based on the direction of gravity calculated from the acceleration sensor data 405. After that, the process returns to step S101.

[0090] According to the above-described embodiment, when switching from mouse operation mode to gyro operation mode in accordance with the state of the controller, gyro correction is performed without, for example, requiring a button operation by the user (see FIGS. 6 and 8), resulting in excellent usability. As an example, by performing gyro correction based on the attitude of the controller and switching from mouse operation mode to gyro operation mode (see FIG. 8), for example, if the user is operating the mouse and lifts the controller off the work surface to adjust the position of the controller, it is possible to prevent the mode from being switched to gyro operation mode against the user's will.

[0091] For example, in the second embodiment, mouse operation is assumed to be a movement operation in which the user moves the controller back and forth on the work surface. Therefore, it is assumed that the front of the controller, i.e., the positive direction of the y-axis, is roughly facing the screen. When the user lifts the controller to perform gyro operation while performing such mouse operation, it is assumed that the controller's y-axis will be positioned so that it faces the opposite direction to the direction of gravity while still roughly facing the screen. Therefore, by performing gyro correction in the orientation described in the second embodiment, subsequent gyro operation will also be less awkward.

[0092] According to the above-described embodiment, in the mouse operation mode, object control based on changes in the controller's attitude is not performed, allowing the user to perform operations in the mouse operation mode without worrying about gyro misalignment.

[0093] Furthermore, according to the above-described embodiment, the object to be operated in the mouse operation mode and the object to be operated in the gyro operation mode are different objects (see FIGS. 5, 8(1), and 9(1)). This makes it possible to start operating the object to be operated in the gyro operation mode while suppressing any discomfort felt by the user due to the execution of gyro correction (for example, discomfort caused by an instantaneous change in the position or attitude of the object).

[0094] Furthermore, according to the above-described embodiment, the conditions for executing gyro correction (see FIG. 6(1) and FIG. 8(2)) do not include a condition related to the tilt of the controller in the z-axis direction. This makes it possible to avoid switching to gyro operation mode against the user's will, for example, when mouse operation is performed using a cylindrical curved surface such as the user's thigh as the work surface.

[0095] [Variations] In the mouse operation mode, the gyro sensor data (see FIG. 11) may also be used to control objects (for example, the cursor 250, the object 200, other objects, etc.).

[0096] The object to be operated in the mouse operation mode and the object to be operated in the gyro operation mode may be the same object.

[0097] Gyro correction and transition to gyro operation mode may be performed based on the tilt of the controller's z-axis relative to the horizontal plane.

[0098] Gyro correction may be performed before switching to the gyro operation mode. Gyro correction may be performed not only once when switching from the mouse operation mode to the gyro operation mode, but also, for example, continuously or periodically during the mouse operation mode. This ensures that when switching from the mouse operation mode to the gyro operation mode, the gyro deviation is appropriately corrected by the gyro correction that was last performed. Even in such a case, since the gyro correction is confirmed upon switching to the gyro operation mode, it can be said that gyro correction is performed when the conditions for switching to the gyro operation mode are met.

[0099] In addition, apart from the operation for executing the gyro correction described in the above embodiment (see Figures 6 and 8(2)), gyro correction may be executed in response to a predetermined operation (for example, pressing button 15f).

[0100] Furthermore, the object to be operated in the gyro operation mode may be switched in response to a predetermined operation (for example, pressing button 15f, a predetermined change in the controller's attitude, etc.). For example, in the second embodiment, the control (see FIG. 9(2)) that allows the character's hand to be moved by gyro operation may be changed to a control that displays a menu image in response to a predetermined operation and moves a cursor displayed on the menu image by gyro operation. For example, in the second embodiment, the control that allows the character's attitude, etc. to be changed by gyro operation may be changed to a control that displays a reticle for the character to shoot in response to a predetermined operation and moves the reticle by gyro operation. In this case, gyro correction may not be performed.

[0101] When performing gyro correction, there is no limitation on which axis of the reference real space axis is corrected to which direction. The correction method may vary depending on the game, or even within the same game, depending on the game situation. For example, when controlling an object operated by gyro operation, the appropriate way to hold the controller differs between the first mode and the second mode. As an example, in the first mode, the controller is assumed to be held in the manner shown in FIG. 9(1), and in the second mode, the controller is assumed to be held with both hands so that the longitudinal direction of the controller is oriented sideways and the bottom of the controller faces upward or forward. In such a case, the correction method when performing gyro correction may be determined depending on whether the object operated in the gyro operation mode after switching from the mouse operation mode is in the first mode or the second mode. Note that the conditions for performing gyro correction and / or the conditions for switching the operation mode may also be determined depending on whether the object after switching the operation mode is in the first mode or the second mode.

[0102] In addition, in the above-described embodiment, at least a part of the processing executed by the processor 11 may be executed by the processors (15a, 16a) of the controller.

[0103] The game device 10 is an example of an information processing device, and the information processing device may be a device that does not execute a game. Similarly, the information processing system may be a system that does not execute a game. The type of program is not limited, and a game program is one example. The program may be, for example, a drawing program, a content viewing program, or a menu program.

[0104] The shape of the controller is an example, and other shapes may be used. The game device 10 may be capable of using only one controller or three or more controllers. Also, the controller may not have some operation units, or may have other operation units.

[0105] The various data in the above-described embodiment are merely examples. Even if the data is converted into other data in each process, it may be considered to be substantially the same data.

[0106] At least a part of the series of processes described above may be executed by a server-side device in an information processing system including a terminal-side device and a server-side device capable of communicating with the terminal-side device via a network. Note that the server may be configured by a plurality of information processing devices, and the processes may be shared and executed by the plurality of information processing devices.

[0107] Although the present embodiment and its modifications have been described above, these descriptions are merely illustrative in all respects and are not intended to limit the scope of the present invention. It goes without saying that various improvements and modifications can be made to the above-described embodiment and its modifications. [Explanation of symbols]

[0108] 10 Information processing system (game device) 11, 15a, 16a processors 12, 15b, 16b Memory 15, 16 Controller 100 Aiming 200 objects 250 cursors 203, 204 Character hands

Claims

1. An information processing program that causes a computer including one or more processors to execute processing, The process comprises: In a first operation mode, operating a first object based on mouse sensor data output from a mouse sensor of an input device; In a second operation mode, operating a second object based on gyro sensor data of the input device; an information processing program including: transitioning from the first operation mode to the second operation mode based on a first condition being satisfied, and performing gyro correction of an orientation in a direction perpendicular to a direction of gravity.

2. The information processing program described in Claim 1, wherein the gyro correction is a correction that considers the direction in which the component perpendicular to the direction of gravity, among the components contained in the direction in which the longitudinal direction of the input device, faces, to be the front.

3. The process comprises: 2 . The information processing program according to claim 1 , further comprising operating the first object based on the mouse sensor data of the acquired gyro sensor data and the acquired mouse sensor data in the first operation mode.

4. The information processing program according to claim 1 , wherein the first object and the second object are different objects.

5. a light intake port for the mouse sensor is provided on the bottom surface of the input device; The information processing program according to claim 1 , wherein the first condition includes a condition that is met when the daylight opening is not blocked.

6. a light intake port for the mouse sensor is provided on the bottom surface of the input device; 2. The information processing program according to claim 1, wherein the first condition includes a longitudinal axis of the input device that is parallel to the bottom surface and extends from front to back when the input device is held by a user being inclined by a predetermined angle or more with respect to a direction perpendicular to the direction of gravity.

7. The information processing program according to claim 1 , wherein the first condition does not include a condition regarding an inclination of a short axis of the input device, which is parallel to the bottom surface and perpendicular to the long axis, with respect to a direction perpendicular to the direction of gravity.

8. A light intake port for the mouse sensor is provided on the bottom surface of the input device, The information processing program according to claim 1 , wherein the first condition includes a condition indicating that a bottom surface of the input device has been separated from a work surface.

9. The process comprises: The information processing program according to claim 1 , further comprising controlling the second object based on a rotational movement of the input device about a longitudinal axis facing a longitudinal direction of the input device.

10. An information processing system comprising one or more processors, The processor: In a first operation mode, a first object is operated based on mouse sensor data output from a mouse sensor of an input device; In a second operation mode, a second object is operated based on gyro sensor data of the input device; an information processing system that transitions from the first operation mode to the second operation mode based on a first condition being satisfied, and performs gyro correction of an orientation in a direction perpendicular to a direction of gravity.

11. 1. A computer-implemented information processing method including one or more processors, comprising: In a first operation mode, a first object is operated based on mouse sensor data output from a mouse sensor of an input device; In a second operation mode, a second object is operated based on gyro sensor data of the input device; an information processing method, wherein, based on a first condition being satisfied, the operation mode is transitioned from the first operation mode to the second operation mode, and gyro correction is performed for an orientation perpendicular to a direction of gravity.

Images