Information processing method, information processing system, and information processing program

Abstract

To provide a novel operation relating to a virtual object.SOLUTION: In an information processing method, a virtual object is controlled by output of a mouse sensor in a first mode, is controlled by output of an inertia sensor in a second mode, and is controlled by output of a direction operation unit in a third mode. In the first mode, it is transitioned to the second mode when at least a first condition is satisfied, and it is transitioned to the third mode when at least a second condition is satisfied.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to information processing for games and the like. [Background technology]

[0002] It has been known for some time that the aiming position can be switched between being operated based on the operation of an operation unit and being operated based on acquired coordinates. It is also known that the coordinates used to operate the aiming position may be acquired using, for example, an acceleration sensor. It is also known that this operation switching can take into account the acquired coordinates, the movement of the controller, and the operating state of the operation unit. For example, see paragraphs

[0146] ,

[0151] , and

[0155] of Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-90941 Summary of the Invention [Problem to be solved by the invention]

[0004] The above-described techniques do not take into consideration the case where there is an additional input means other than the above-described input means that is used to operate the aiming position. [Means for solving the problem]

[0005] For example, the following configuration example can be given.

[0006] (Configuration 1) Configuration 1 is an information processing method realized by a computer including one or more processors, and includes a mode setting step of setting one of a plurality of modes including a first mode, a second mode, and a third mode, and a virtual object control step of controlling a virtual object based on the output of a mouse sensor of at least one controller among one or more controllers in the first mode, based on the output of an inertial sensor of at least one controller among the one or more controllers in the second mode, and based on the output of a directional operation unit operated by a user of at least one controller among the one or more controllers in the third mode. In the mode setting step, when the mode is the first mode, the mode is transitioned to the second mode based on the satisfaction of at least a first condition, and based on the satisfaction of at least a second condition.

[0007] (Configuration 2) In configuration 2, in the above configuration 1, in the mode setting step, when the mode is the second mode, the mode may be transitioned to the third mode based on the second condition being satisfied, and the mode may be transitioned to the first mode based on at least the third condition being satisfied.

[0008] (Configuration 3) Configuration 3 may be configured in the above configuration 1 or 2, such that in the mode setting step, when the mode is the third mode, the mode is transitioned to the second mode based on at least the first condition being satisfied, and the mode is transitioned to the first mode based on at least the fourth condition being satisfied.

[0009] (Configuration 4) In a fourth configuration, in the third configuration, the third condition and the fourth condition each include a condition related to the output of the mouse sensor, and the time required for determining whether the third condition is met may be longer than the time required for determining whether the fourth condition is met.

[0010] (Configuration 5) Configuration 5 is any of configurations 1 to 4 above, in which the first condition includes a condition regarding the output of the inertial sensor, and in the first mode, if the output of the mouse sensor does not indicate movement of at least one controller having the mouse sensor, a transition to the second mode is made when at least the first condition is satisfied, and if the output of the mouse sensor indicates movement of the controller, a transition to the second mode may not be made even if the first condition is satisfied.

[0011] (Configuration 6) Configuration 6 may be such that, in any of configurations 1 to 5 above, the second mode is set when the output of the mouse sensor does not indicate movement of at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the directional operation unit is not operated.

[0012] (Configuration 7) Configuration 7 is any of configurations 1 to 6 above, wherein the mode setting step may transition between the third mode and one of the first mode or the second mode, but not between the third mode and the other mode, based on the output of a controller that has a directional operation unit and either a mouse sensor or an inertial sensor, but not the other.

[0013] (Configuration 8) Configuration 8 is any one of configurations 1 to 7, in which the computer executes game processing, and the virtual object may be a pointer used in the game processing. [Effects of the Invention]

[0014] According to this embodiment, it is possible to provide a new operation for a virtual object. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 shows an example of a state in which the right controller 3 and the left controller 4 are attached to the main unit 2. [Figure 2] Six-sided diagram showing an example of the right controller 3 [Figure 3]Six-sided diagram showing an example of the left controller 4 [Figure 4] A block diagram showing an example of the internal configuration of the main unit 2. [Figure 5] A block diagram showing an example of the internal configuration of the main unit 2, the right controller 3, and the left controller 4. [Figure 6] FIG. 10 shows an example of a state in which the right controller 3 is held and operated in the right hand. [Figure 7] FIG. 10 shows an example of a state in which the right controller 3 is held and operated in the right hand. [Figure 8] A diagram explaining the transition of controller operation modes [Figure 9] FIG. 10 shows an example of various data stored in the DRAM 69. [Figure 10] An example of a flowchart for information processing [Figure 11] An example of a flowchart for information processing [Figure 12] An example of a flowchart for information processing [Figure 13] An example of a flowchart for information processing [Figure 14] A diagram showing an example of the targeting screen display [Figure 15] A diagram showing an example of the targeting screen display [Figure 16] A diagram showing an example of the targeting screen display DETAILED DESCRIPTION OF THE INVENTION

[0016] An embodiment will be described below.

[0017] [Example of hardware configuration of information processing system]

[0018] A game system, which is an example of an information processing system according to this embodiment, will be described below. The example of the game system 1 according to this embodiment includes an information processing device (sometimes referred to as a "main unit") 2, a right controller 3, and a left controller 4. The right controller 3 and the left controller 4 are each detachable from the main unit 2 according to this embodiment.

[0019] FIG. 1 is a diagram showing an example of a state in which a right controller 3 and a left controller 4 are attached to a main unit 2. As shown in FIG. 1, the right controller 3 and the left controller 4 are each attached to the main unit 2 and integrated together. The main unit 2 is a device that executes various processes (e.g., game processes) in the game system 1. The main unit 2 is equipped with a display 72. The right controller 3 and the left controller 4 are input devices that include an operation unit and the like for the user to input. Note that, below, the right controller 3 and the left controller 4 may be collectively referred to as the "controllers."

[0020] The display 72 displays an image generated by the main device 2. The display 72 is, for example, a liquid crystal display (LCD). A touch panel is provided on the screen of the display 72. The touch panel is, for example, of a type that allows multi-touch input (for example, a capacitance type).

[0021] FIG. 2 is a six-sided schematic diagram showing an example of the right controller 3. As shown in FIG. 2, the right controller 3 has a vertically elongated plate shape, includes a housing 11, and has a front, rear, top, bottom, right, and left sections. In the right controller 3, the rear is located opposite the front section, the bottom is located opposite the top section, and the left section is located opposite the right section. The distance between the front and rear sections is greater than the distance between the top and bottom sections. The distance between the top and bottom sections is greater than the distance between the right and left sections. Note that in other embodiments, the magnitude relationship between these distances may be other magnitude relationships. Furthermore, in this embodiment, the direction connecting the bottom and top sections may be referred to as the up-down direction, the direction connecting the front and rear sections perpendicular to the up-down direction may be referred to as the front-rear direction, and the direction connecting the right and left sections perpendicular to the up-down direction and the front-rear direction may be referred to as the left-right direction. In FIG. 2, the coordinate system of the right controller 3 (sometimes referred to as the "right controller coordinate system") is shown with the x, y, and z axes illustrated in a front view in which the left side faces forward. In this coordinate system, the direction from the left side to the right side is the positive z-axis. The direction perpendicular to the z-axis, from the bottom to the top, is the positive x-axis, and the direction perpendicular to the z-axis and x-axis, from the rear to the front, is the positive y-axis. When the bottom faces the direction of gravity, the negative x-axis and the direction of gravity coincide. In addition, the x-axis, y-axis, and z-axis in the description related to the right controller 3 refer to the x-axis, y-axis, and z-axis in the right controller coordinate system unless otherwise specified. In this embodiment, each part, such as the front part and the bottom part, does not need to be completely flat and may have irregularities or slopes. For example, the bottom part includes a convex part 25, which will be described later. The direction in which each part faces and the direction connecting each part indicate approximate directions.

[0022] The right controller 3 has a protrusion 25 that fits into a recess (not shown) of the main unit 2 when attached to the main unit 2. As shown in Fig. 2, the protrusion 25 has a convex shape that protrudes in the negative x-axis direction, with a width that is shorter than the left-right direction of the right controller 3 and a front-to-back width that is shorter than the front-to-back direction of the right controller 3. In this embodiment, the protrusion 25 is part of the bottom.

[0023] As will be described later, the right controller 3 can also be held in a portrait orientation when detached from the main unit 2. The right controller 3 has a shape and size that allows it to be held in one hand, particularly the right hand, when held in a portrait orientation. The right controller 3 can also be held in a landscape orientation, and when held in a landscape orientation, it may be held with both hands (not shown).

[0024] The right controller 3 has, on its left side, an analog stick (sometimes simply referred to as a "stick") 22, which is an example of a direction input unit. The stick 22 can be used as a direction input unit that can be operated to input a direction. The stick may also be called a direction operation unit. The user can tilt the stick 22 in any direction to input a direction according to the tilt direction, and can input a magnitude according to the tilt angle. The user can also input buttons by pressing the stick 22. The direction input unit may be, for example, a cross key or a slide pad. The direction input unit may also be called a direction operation unit.

[0025] The right controller 3 has a set of four buttons on its left side: an A button 12, a B button 13, an X button 14, a Y button 15, a + (plus) button 16, and a home button 17. The right controller 3 has an R button 20 and a ZR button 21 on its front and top. The R button 20 and ZR button 21 may be provided only on the front or only on the top of the right controller 3. The right controller 3 has buttons 18 and 19 on the top surface 25a of the protrusion 25.

[0026] The right controller 3 has a mouse sensor opening 23 on the top surface 25a of the convex portion 25. The mouse sensor opening 23 is an opening in a light guide path that guides light to the mouse sensor 24 located inside the right controller 3. The mouse sensor 24 is an optical mouse sensor and may include a light-emitting unit and a light-receiving unit. The light detected by the light-receiving unit may be visible light or light of an invisible wavelength. The mouse sensor 24 may include at least a light-receiving unit, but may not include a light-emitting unit. The mouse sensor 24 acquires data that enables calculation of the movement of the right controller 3 on a mounting surface when the right controller 3 is placed with the top surface 25a of the convex portion 25 of the bottom facing the mounting surface. This allows the right controller 3 to be used as a mouse. This operation using the right controller 3 as a mouse is sometimes referred to as a "mouse operation." The mounting surface is not limited to a flat surface, but may be a curved surface, such as the surface of the user's thigh.

[0027] In this embodiment, the right controller 3 is provided with a terminal 26 on the protrusion 25, which enables the right controller 3 to perform wired communication with the main unit 2. As an example, the terminal 26 is provided on the inner circumferential surface of a recess provided in the top surface 25a of the protrusion 25.

[0028] FIG. 3 is a six-sided schematic diagram showing an example of the left controller 4. Description of components similar to those of the right controller 3 will be omitted. The left controller 4 includes, on its right side, a stick 42, a set of four buttons: a right button 32, a down button 33, an up button 34, and a left button 35; a capture button 37; and a - (minus) button 36. The buttons 32 to 35 may be a single cross button. Note that, on the right controller 3, the stick 22 is located behind the buttons 12 to 15, whereas on the left controller 4, the stick 42 is located in front of the buttons 32 to 35. In FIG. 3, the coordinate system of the left controller 4 (sometimes referred to as the "left controller coordinate system") is shown with the x, y, and z axes illustrated in a front view in which the right side faces forward. In this coordinate system, the direction from the right side to the left side is the positive z-axis direction. Additionally, the direction perpendicular to the z-axis, from the bottom to the top, is the positive x-axis direction, and the direction perpendicular to the z-axis and x-axis, from the rear to the front, is the positive y-axis direction. When the bottom faces the direction of gravity, the negative x-axis and the direction of gravity coincide. Note that the x-axis, y-axis, and z-axis in the explanations related to the left controller 4 refer to the x-axis, y-axis, and z-axis in the left controller coordinate system, unless otherwise specified.

[0029] The left controller 4 has a protrusion 45 that fits into a recess (not shown) of the main unit 2 when attached to the main unit 2. Similar to the right controller 3, the protrusion 45 has buttons 38 and 39, an opening 43 for the mouse sensor, a mouse sensor 44, and a terminal 46.

[0030] When the left controller 4 is detached from the main unit 2, it can be held in a portrait or landscape orientation, similar to the right controller 3.

[0031] FIG. 4 is a block diagram showing an example of the internal configuration of the main unit 2. The main unit 2 includes a processor 63. The processor 63 is an information processing unit that executes various types of information processing executed in the main unit 2. The processor 63 may be composed of, for example, multiple processors or cores, typically multiple central processing units (CPUs) or cores, or may be composed of a system-on-a-chip (SoC) that includes multiple functions such as a CPU function and a graphics processing unit (GPU) function. The processor 63 executes various types of information processing by executing an information processing program (e.g., a game program) stored in a storage unit (specifically, an internal storage medium such as a flash memory 68, or an external storage medium inserted into a slot 51, etc.). In this embodiment, the term "processor" may include at least a CPU, a GPU, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. In this embodiment, the computer may include, for example, at least one processor and may further include a storage unit such as a memory.

[0032] The main device 2 includes, as examples of internal storage media, a flash memory 68 and a DRAM (Dynamic Random Access Memory) 69. The flash memory 68 is a memory used primarily to store various types of data saved in the main device 2. The DRAM 69 is a memory used primarily to temporarily store various types of data used in information processing. The processor 63 reads and writes data from and to storage media such as the flash memory 68 and the DRAM 69 as appropriate to perform various types of information processing.

[0033] Furthermore, the main device 2 has various components as shown in Fig. 4. A brief explanation is given below. A recording medium slot interface (sometimes called a "slot I / F") 52 reads and writes data from a storage medium (e.g., a dedicated memory card) inserted into the recording medium slot 51 in response to instructions from the processor 63. A second slot I / F 54 reads and writes data from a storage medium inserted into the second slot 53 in response to instructions from the processor 63.

[0034] The network communication unit 66 communicates with external devices via a network (for example, internet communication using wireless communication). The controller communication unit 67 communicates with the right controller 3 and / or left controller 4 wirelessly (for example, communication in accordance with the Bluetooth (registered trademark) standard).

[0035] The left-side terminal 50 is a terminal for wired communication between the processor 63 and the left controller 4. The right-side terminal 65 is a terminal for wired communication between the processor 63 and the right controller 3. The lower terminal 64 is a terminal for communicating with other devices (such as a stationary monitor) via the cradle when the lower terminal 64 is attached to the cradle, for example.

[0036] The touch panel controller 70 generates data indicating, for example, the position where a touch input has been made based on a signal from the touch panel 71 arranged on the display surface of the display 72, and outputs the data to the processor 63. The display 72 displays an image generated by the processor 63 and / or an image acquired from an external source.

[0037] The codec circuit 74 controls the input and output of audio data to the speaker 73 and the audio input / output terminal 75 .

[0038] The power control unit 61 controls the power supply from the battery 62 to each part of the main device 2 (i.e., each part that receives power from the battery 62) based on instructions from the processor 63, and also starts or stops the power supply in response to pressing the power button 60.

[0039] The volume button 59 is a button for controlling the volume output from the speaker 73 etc. The cooling fan 58 is a fan for cooling the inside of the main body device 2.

[0040] The main body device 2 includes various sensors such as a magnetic sensor 55, an ambient light sensor 56, a temperature sensor 57, an acceleration sensor 76, and an angular velocity sensor 77. The processor 63 can execute various processes based on information from these sensors.

[0041] 5 is a block diagram showing an example of the internal configuration of the main unit 2, right controller 3, and left controller 4. Note that details of the internal configuration of the main unit 2 are omitted in FIG. 5 because they are shown in FIG. 4.

[0042] The left controller 4 is equipped with a communication control unit 80 that communicates with the main unit 2. As shown in FIG. 5, the communication control unit 80 is connected to various components, including a terminal 88. When the left controller 4 is attached to the main unit 2, the communication control unit 80 performs wired communication with the main unit 2 via the terminal 88, and when the left controller 4 is detached from the main unit 2, the communication control unit 80 performs wireless communication with the main unit 2 (specifically, communication in accordance with the Bluetooth (registered trademark) standard).

[0043] The left controller 4 includes a memory 81, such as a flash memory. The communication control unit 80 is configured with a processor, such as a microcomputer (also called a microprocessor), and executes firmware stored in the memory 81 to perform various processes.

[0044] The left controller 4 includes buttons 82 (specifically, buttons 32 to 34, etc.) and a stick 42. Each button 82 and stick 42 outputs to the communication control unit 80 information relating to an operation performed on that button 82 and stick 42.

[0045] The left controller 4 is equipped with an inertial sensor. Specifically, the left controller 4 is equipped with an acceleration sensor 83 and an angular velocity sensor 84 as inertial sensors. The acceleration sensor 83 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in FIG. 3 ). The acceleration sensor 83 may detect acceleration in one or two axial directions. The angular velocity sensor 84 detects angular velocity around the three predetermined axes. The angular velocity sensor 84 may detect angular velocity around one or two axes. The angular velocity sensor may also be called a "gyro sensor." The acceleration sensor 83 and the angular velocity sensor 84 are each connected to the communication control unit 80. The detection results of the acceleration sensor 83 and the angular velocity sensor 84 are repeatedly output to the communication control unit 80 at appropriate timing. The right controller 3 and the left controller 4 may each be equipped with either an acceleration sensor or an angular velocity sensor as inertial sensors, or may be equipped with other sensors.

[0046] The left controller 4 is equipped with a mouse sensor 44. The mouse sensor 44 acquires data for calculating the movement of the left controller 4 placed on the placement surface. The data acquired by the mouse sensor 44 is repeatedly output to the communication control unit 80 at appropriate timing.

[0047] The communication control unit 80 acquires information about inputs (specifically, information about button and stick operation, and detection results by the sensors) from each input unit (specifically, each button 82, stick 42, and each sensor 83, 84, and 44). The communication control unit 80 transmits operation data including the acquired information or information obtained by performing predetermined processing on the acquired information to the main unit 2. The operation data is repeatedly transmitted once every predetermined time.

[0048] By transmitting the operation data to the main unit 2, the main unit 2 can obtain the input performed on the left controller 4. That is, the main unit 2 can determine the operation of each button 82 and stick 42 based on the operation data. The main unit 2 can also calculate information regarding the movement and / or posture of the left controller 4 based on the operation data (specifically, the detection results of the acceleration sensor 83 and / or angular velocity sensor 84). The main unit 2 can also calculate information regarding mouse operations performed on the left controller 4 based on the operation data (specifically, the detection results of the mouse sensor 44).

[0049] The left controller 4 includes an amplifier 85 and a vibrator 86. The amplifier 85 amplifies the control signal received from the communication control unit 80 and generates a drive signal. The vibrator 86 vibrates in response to the drive signal generated by the amplifier 85, causing the left controller 4 to vibrate.

[0050] The left controller 4 is equipped with a power supply unit 87. The power supply unit 87 has a battery and a power control circuit. The power control circuit is connected to the battery and supplies power to each part of the left controller 4 (specifically, each part that receives power from the battery).

[0051] As shown in FIG. 5, the right controller 3 is equipped with a communication control unit 91, which is composed of a processor and the like and which communicates with the main unit 2. The right controller 3 is equipped with a memory 94 connected to the communication control unit 91. The communication control unit 91 is connected to each component, including a terminal 92. The communication control unit 91 and memory 94 have the same functions as the communication control unit 80 and memory 81 of the left controller 4. Therefore, the communication control unit 91 can communicate with the main unit 2 both via wired communication via the terminal 92 and via wireless communication not via the terminal 92, and controls the communication between the right controller 3 and the main unit 2.

[0052] The right controller 3 has the same input units as the left controller 4. Specifically, it has buttons 95 (A button 12, B button 13, X button 14, Y button 15, etc.), a stick 22, inertial sensors (acceleration sensor 96 and angular velocity sensor 97), and a mouse sensor 24. Each of these input units has the same function as the input units of the left controller 4, and operates in the same way.

[0053] The right controller 3 includes an amplifier 98, a vibrator 99, and a power supply unit 100. The amplifier 98, the vibrator 99, and the power supply unit 100 have the same functions as the amplifier 85, the vibrator 86, and the power supply unit 87 of the left controller 4, respectively, and operate in the same manner.

[0054] [How to hold the controller] FIG. 6 is a schematic diagram showing an example of a state in which a user holds the right controller 3 in their right hand, places it on a surface, and uses it as a mouse, that is, operates the mouse. As shown in FIG. 6, when viewed from the user's perspective, the front of the right controller 3 faces forward and the left side faces left. The palm of the user's right hand covers the upper side of the right controller 3. The user's right thumb is placed on the left side of the right controller 3. The user's right thumb is placed on, for example, the A button 12. The user's right index finger is placed on, for example, the R button 20, and the user's right middle finger is placed on, for example, the ZR button 21. The user can operate the R button 20 and the ZR button 21 with their right index finger or middle finger. The user can operate each input unit located on the left side with their right thumb. Note that when using the left controller 4 as a mouse with their left hand, the left controller 4 can be held in their left hand and used on a surface in the same manner. At this time, the right part of the left controller 4 faces right.

[0055] FIG. 7 is a schematic diagram showing an example of a state in which a user holds the right controller 3 in their right hand and operates it in the air. As shown in FIG. 7, when the right controller 3 is detached from the main unit 2, it can be held so that the longitudinal direction of the right controller 3 is the up-down direction or the front-back direction for the user, and can be used in the air. The user can operate the stick 22 (sometimes referred to as "stick operation") with, for example, the thumb of their right hand. The user can also perform a swing operation (sometimes referred to as "swing operation") or an operation to change the position of the right controller 3 they are holding (sometimes referred to as "position change operation"). The same procedure can be used when the user uses the left controller 4 in their left hand and in the state in which it is detached from the main unit 2.

[0056] [Controller operation mode] FIG. 8 is a diagram illustrating the operation modes of the controller and transitions between the operation modes. As shown in FIG. 8, the operation modes (sometimes simply referred to as "modes") of the controller include "mouse mode," "gyro mode," and "stick mode." Mouse mode is an operation mode in which a virtual object is controlled based on the output of a mouse sensor. Gyro mode is an operation mode in which a virtual object is controlled based on the output of a gyro sensor. Stick mode is an operation mode in which a virtual object is controlled based on the output of a stick operation. As will be described later using FIG. 14 and other figures, a crosshair 250, which is a pointer, is displayed on the display 72 as an example of a virtual object. Note that the crosshair is an example of a pointer, and its shape is not limited thereto, and may be, for example, an arrow-shaped cursor, a hand icon, or a frame. Furthermore, the pointer is an example of a virtual object to be controlled, and the virtual object may be, for example, a character object or another object. The "initial state" is a state that can be set when no operation mode has been determined or when the operation mode transitions between mouse mode, gyro mode, and stick mode. Note that in other embodiments, there may be no initial state.

[0057] In addition, in this embodiment, it is possible to transition from each mode to another mode, as shown in Fig. 8. Specific aspects of the transition will be described later.

[0058] [Details of information processing in this embodiment] Next, information processing according to this embodiment will be described in detail with reference to Figures 9 to 16. In this embodiment, it is possible to change the operation mode as described above, and the position of the aim 250 is controlled based on the operation corresponding to the operation mode. The following description will be given taking as an example a case where the right controller 3 is used. Note that the same can be applied to a case where the left controller 4 is used, and therefore a description thereof will be omitted.

[0059] [About data usage] Next, a description will be given of various data stored in the DRAM 69. Fig. 9 shows an example of data stored in the DRAM 69 of the main unit 2. As shown in Fig. 9, the DRAM 69 is provided with at least a program storage area 301 and a data storage area 302.

[0060] The program memory area 301 stores at least a program 401. The data memory area 302 stores at least operation mode data 402, mouse sensor data 403, stick / button input data 406, inertial sensor data 407, aiming coordinate data 408, object data 409, image data 410, virtual camera control data 411, and controller device data 412.

[0061] The program 401 is a game program for executing game processing.

[0062] The operation mode data 402 is data indicating whether the operation mode is mouse mode, gyro mode, or stick mode.

[0063] The mouse sensor data 403 is data relating to the output of the mouse sensor 24 and includes image clarity data 404 and dy / dz data 405 .

[0064] The image clarity data 404 is calculated by the mouse sensor 24 and indicates the clarity of the mouse sensor image. The image clarity data 404 is calculated, for example, based on the brightness of the mouse sensor image and / or the number of feature points in the mouse sensor image. The image clarity data 404 may be calculated by the communication control unit 91, the processor 63, or the like based on the output data of the mouse sensor 24. The brightness of the mouse sensor image or the number of feature points in the mouse sensor image may be used as the image clarity data. The image clarity data 404 may also be calculated based on other factors. If the clarity indicated by the image clarity data 404 is equal to or greater than a predetermined value, it can be estimated that the opening 23 of the mouse sensor 24 is blocked by a mounting surface or the like. Other data may be used instead of the data indicating the clarity of the mouse sensor image, as long as it can be used to estimate that the opening 23 of the mouse sensor 24 is blocked.

[0065] The dy / dz data 405 is output data of the mouse sensor 24, and indicates the movement distance per frame time (sometimes referred to as "dy / dz") in the y-axis direction and z-axis direction of the right controller coordinate system (i.e., the yz plane; see FIG. 2) relative to the placement surface or the like when the opening 23 of the mouse sensor 24 is blocked by the placement surface or the like. Note that dy / dz may be calculated from the output data of the mouse sensor 24 by the communication control unit 91, the processor 63, or the like.

[0066] The stick / button input data 406 is data indicating operations performed on the stick 22 and each button 95 of the right controller 3.

[0067] The inertial sensor data 407 is data output from the inertial sensor of the right controller 3, and is data that enables calculation of, for example, the acceleration in the x, y, and z-axis directions of the right controller coordinate system (see FIG. 2) and the angular velocity around the x, y, and z-axis. Using the inertial sensor data, for example, the attitude and movement of the right controller 3 can be calculated.

[0068] The aim coordinate data 408 is data that indicates the coordinates of the aim 250 in a screen coordinate system obtained by converting the virtual space photographed by the virtual camera into a planar coordinate system (sometimes simply referred to as "aim coordinates"). The aim coordinates move based on the dy / dz data in mouse mode, move based on the operation of the stick 22 in stick mode, and move based on changes in the attitude of the right controller 3 in gyro mode.

[0069] The object data 409 is data of virtual objects to be placed in the virtual space, such as a bullet fired in the direction of the crosshair 250, a player character, an opponent character, the ground, and the like.

[0070] The image data 410 is image data such as an image of the crosshair 250, which is a virtual object, an animation image, a background, and virtual effects. The image of the crosshair 250 is arranged in the above-described screen coordinate system and displayed on the display 72, etc. Note that instead of arranging the crosshair 250 in the screen coordinate system and displaying it on the display 72, etc., it is also possible to arrange the crosshair 250 in a virtual space and photograph it with a virtual camera, and then display it on the display 72, etc. In other words, the crosshair 250 may be object data 409 instead of image data 410.

[0071] The virtual camera control data 411 is data for controlling a virtual camera that is placed in a virtual space and captures images of the virtual space.

[0072] The controller device data 412 is data indicating the devices (e.g., mouse sensor, acceleration sensor, angular velocity sensor, stick, etc.) of the controller connected to the main unit 2. When the controller is connected to the main unit 2, data indicating the devices of the controller is transmitted from the controller to the main unit 2.

[0073] In addition, the DRAM 69 stores various types of data used in drawing processes and the like as needed.

[0074] [Example of detailed information processing] Next, the processing according to this embodiment will be described with reference to flowcharts and the like. FIGS. 10 to 13 are examples of flowcharts showing the processing according to this embodiment. FIG. 14 is a diagram for explaining the control of the aim 250 in mouse mode and stick mode. FIGS. 15 and 16 are diagrams for explaining the control of the aim 250 in gyro mode. Note that, below, processing characteristic of this embodiment will be mainly described, and other descriptions such as drawing processing will basically be omitted. Furthermore, the following processing is executed, for example, at predetermined intervals (for example, a processing frame interval executed every 1 / 60 seconds).

[0075] When this game processing starts, in step S101 of FIG. 10, the processor 63 determines whether a first mouse mode transition condition is satisfied based on the mouse sensor data 403. In this embodiment, the first mouse mode transition condition is a condition under which it can be inferred that the right controller 3 is being operated as a mouse. As an example, this condition is a condition under which it can be inferred that the opening 23 of the mouse sensor 24 is blocked because the clarity indicated by the image clarity data 404 is equal to or greater than a predetermined level, and a first predetermined number of consecutive frames (e.g., five frames) or more in which dy / dz, the movement distance per frame time of the right controller 3, is equal to or greater than a predetermined level (e.g., 0.1 mm or greater) have occurred. If the determination in step S101 is YES, processing proceeds to mouse mode processing in step S200. If the determination is NO, processing proceeds to step S102.

[0076] In step S102, the processor 63 determines, based on the controller device data 412, whether or not the right controller 3 connected to the main unit 2 has a gyro device (i.e., an angular velocity sensor). This determination may be made by other methods. For example, the main unit 2 may determine that the right controller 3 has a gyro device when it receives information related to the detection results of the angular velocity sensor that is repeatedly output at appropriate timing from the right controller 3. If the determination in step S102 is YES, the process proceeds to gyro mode processing in step S300. If the determination is NO, the process proceeds to stick mode processing in step S400. Note that the mouse mode is in effect when mouse mode processing is being executed, the stick mode is in effect when stick mode processing is being executed, and the gyro mode is in effect when gyro mode processing is being executed.

[0077] In step S201 of Fig. 11, processor 63 determines whether dy / dz indicated by the latest dy / dz data 405 is greater than a predetermined value (for example, 0.1 mm). This determination makes it possible to estimate whether the controller is being moved on the placement surface by mouse operation. Note that this determination only needs to be able to estimate whether the controller is being moved on the placement surface by mouse operation, and may be, for example, a determination as to whether dy / dz indicated by the latest dy / dz data is greater than 0 (zero). If the determination in step S201 is YES, the process proceeds to step S205, and if NO, the process proceeds to step S202.

[0078] In step S205, processor 63 controls the aim position based on dy / dz data 405 and aim coordinate data 408. Specifically, processor 63 moves the aim position indicated by aim coordinate data 408 in the direction indicated by the dy / dz indicated by the latest dy / dz data 405 by a distance according to the magnitude of the dy / dz.

[0079] Here, Figure 14 is a diagram for explaining the control of the aim position in mouse mode and stick mode. For example, when a mouse operation is performed by moving the right controller 3 placed on the placement surface in the positive direction of the z axis as shown in Figure 14(1)(b) through the processing of step S205, the aim 250 displayed on the display 72 moves to the right a distance corresponding to the mouse operation as shown in Figure 14(1)(a). Furthermore, although not shown, when the right controller 3 is operated with a mouse in another direction on the placement surface, the aim 250 displayed on the display 72 similarly moves in a direction and a distance corresponding to the mouse operation.

[0080] However, in the aim position control of step S205, the processor 63 may restrict the center of the aim 250 so that it does not move outside the display area (sometimes simply referred to as the "display area") of the display 72. For example, when the center of the aim 250 is located at the right edge of the display area as shown in FIG. 14(2)(a), even if a mouse operation is performed to move the right controller 3 placed on the placement surface in the positive direction of the z axis (see FIG. 2) as shown in FIG. 14(2)(b), the aim 250 is controlled so that it does not move any further to the right. As a result, a portion of the aim 250 is always displayed on the display 72. Note that the restriction on the movement of the aim 250 is not limited to this; for example, the movement may be restricted so that the entire aim 250 is located within the display area. Thereafter, the processing returns to step S201.

[0081] Regardless of the operation mode, and regardless of whether the aim is moving or not, processor 63 fires a virtual object, such as a bullet, in the direction indicated by the aim in response to a button operation (e.g., operation of R button 20) based on stick / button input data 406.

[0082] Returning to FIG. 11 , in step S202, processor 63 determines whether or not a stick mode transition condition is satisfied based on stick / button input data 406. In this embodiment, the stick mode transition condition is a condition that the stick 22 is operated. In this manner, in mouse mode, a transition to stick mode processing occurs in response to at least the stick mode transition condition being satisfied. Note that the mouse mode processing in FIG. 11 is one example, and in other embodiments, the condition for transitioning from mouse mode processing to stick mode processing may be simply that the stick mode transition condition described above is satisfied. If the determination in step S202 is YES, processing proceeds to stick mode processing; if the determination is NO, processing proceeds to step S203.

[0083] In step S203, the processor 63 determines whether a gyro mode transition condition is satisfied based on the inertial sensor data 407. In this embodiment, the gyro mode transition condition is used to estimate whether the controller has been swung by the user. As an example, the gyro mode transition condition is a condition that the right controller 3 is swung with an acceleration equal to or greater than a predetermined value (e.g., 0.2 G or greater). In this way, in mouse mode, a transition to gyro mode processing occurs when at least the gyro mode transition condition is satisfied. Note that the mouse mode processing in FIG. 11 is an example, and in other embodiments, the condition for transitioning from mouse mode processing to gyro mode processing may be simply that the above-described gyro mode transition condition is satisfied. If the determination in step S203 is YES, the process proceeds to gyro mode processing; if the determination is NO, the process proceeds to step S204.

[0084] In step S204, the processor 63 determines conditions for estimating whether the user intends to continue mouse operation. In the present embodiment, as an example, the processor 63 determines whether the time during which the mouse sensor opening is not blocked or the time during which the center of the aim is located in the edge region of the display range has reached a predetermined time based on the image clarity data 404 and the aim coordinate data 408. Specifically, the processor 63 measures the time during which the opening 23 of the mouse sensor 24 cannot be estimated to be blocked by a mounting surface or the like based on the image clarity data 404, and measures the time during which the center of the aim 250 is located in the edge region of the display range based on the aim coordinate data 408, and determines whether either of these times has reached a predetermined time (e.g., 3 seconds). As shown in FIG. 14, a border region of a predetermined width is preset in the display region of the display 72. Note that, for convenience of explanation, the border region is depicted so as to be visible in FIGS. 14 to 16, but in reality, the border region need not be displayed. If the determination in step S204 is YES, the process returns to step S101 in FIG. 10, and if the determination is NO, the process returns to step S201.

[0085] In step S204, processor 63 may determine whether the unobstructed time of the mouse sensor opening has reached a first time (e.g., 3 seconds) or whether the time during which the center of the aim is located in the edge region of the display range has reached a second time (e.g., 2 seconds or 4 seconds) different from the first time. Also, in step S204, if the unobstructed time of the mouse sensor opening and the time during which the center of the aim is located in the edge region of the display range at least partially overlap, processor 63 may regard these two times as a block of time having an overlapping portion and determine whether this block of time has reached a predetermined time (e.g., 3 seconds). Also, instead of measuring the time during which the center of the aim is located in the edge region of the display range, processor 63 may measure, for example, the time during which at least a portion of the aim is located within the display range.

[0086] 12, processor 63 determines whether or not a stick input has been made based on stick / button input data 406. If the determination in step S301 is YES, the process proceeds to step S305, and if the determination is NO, the process proceeds to step S302.

[0087] In step S305, processor 63 performs aim position control based on the stick input. Specifically, processor 63 moves the aim position indicated by aim coordinate data 408 in the direction indicated by the stick input by a distance corresponding to the magnitude of the stick input. For example, when the operation of tilting stick 22 in the positive x-axis direction as shown in FIG. 14(1)(c) is performed as a result of the processing in step S305, aim 250 displayed on display 72 moves to the right by a distance corresponding to the stick operation as shown in FIG. 14(1)(a). Furthermore, similar to the aim position control in step S205 in the mouse mode processing, the movement of the center of aim 250 may be limited so as not to move outside the display area (see FIG. 14(2)). Thereafter, the processing returns to step S301.

[0088] In step S302, processor 63 determines whether the first mouse mode transition condition is satisfied, similarly to step S101 in FIG. 10. Thus, in stick mode, transition to mouse mode processing occurs in response to at least the first mouse mode transition condition being satisfied. Note that the stick mode processing in FIG. 12 is an example, and in other embodiments, the condition for transitioning from stick mode processing to mouse mode processing may be simply that the first mouse mode transition condition is satisfied. If the determination in step S302 is YES, processing proceeds to mouse mode processing; if the determination is NO, processing proceeds to step S303.

[0089] In step S303, processor 63 determines whether the gyro mode transition condition is satisfied, similarly to step S203 in FIG. 11. Thus, in stick mode, a transition to gyro mode processing occurs in response to at least the gyro mode transition condition being satisfied. Note that the stick mode processing in FIG. 12 is an example, and in other embodiments, the condition for transitioning from stick mode processing to gyro mode processing may be simply that the above-described gyro mode transition condition is determined to be satisfied. If the determination in step S303 is YES, processing proceeds to gyro mode processing, and if the determination is NO, processing proceeds to step S304.

[0090] In step S304, processor 63 determines conditions for estimating whether the user intends to continue operating the stick. In the present embodiment, as an example, processor 63 determines whether a predetermined time has elapsed since the time when there is no stick input or the time during which the center of the aim is located in the edge region of the display range, based on stick / button input data 406 and aim coordinate data 408. If the determination in step S304 is YES, the process returns to step S101 in FIG. 10; if the determination is NO, the process returns to step S301.

[0091] In step S304, processor 63 may determine whether the time during which there is no stick input has reached a first time (e.g., 3 seconds), or whether the time during which the center of the aim is located in an edge region of the display range has reached a second time (e.g., 2 seconds or 4 seconds) different from the first time. Also, in step S304, if the time during which there is no stick input and the time during which the center of the aim is located in an edge region of the display range at least partially overlap, processor 63 may regard these two times as a block of time having an overlapping portion, and determine whether this block of time has reached a predetermined time (e.g., 3 seconds). Also, instead of measuring the time during which the center of the aim is located in an edge region of the display range, processor 63 may measure, for example, the time during which at least a portion of the aim is located within the display range.

[0092] 13, the processor 63 performs aim position control based on the inertial sensor data 407. Specifically, the processor 63 performs control to display the aim 250 at a position corresponding to the attitude of the right controller 3, based on the correspondence relationship (sometimes referred to as the "attitude-aiming relationship") that associates the attitude of the right controller 3 indicated by the inertial sensor data 407 with the aim position indicated by the aim coordinate data 408.

[0093] An example of the above control will be described using Figures 15 and 16. For example, when the orientation of the right controller 3 is as shown in Figure 15(1)(b), the sight 250 is displayed, for example, at the center of the screen as shown in Figure 15(1)(a). Then, for example, when the orientation of the right controller 3 is as shown in Figure 15(2)(b) (i.e., an orientation rotated right in the positive direction of the z-axis from the orientation shown in Figure 15(1)(b), the sight 250 is displayed at the position shown in Figure 15(2)(a) (i.e., a position to the right of the position shown in Figure 15(1)(a)). Then, for example, when the orientation of the right controller 3 is as shown in Figure 16(b) (i.e., an orientation rotated further right in the positive direction of the z-axis from the orientation shown in Figure 15(2)(b), the sight 250 is displayed at the position shown in Figure 16(a) (i.e., a position further right of the position shown in Figure 15(2)(a)). 16(b), in response to an operation of rotating the right controller 3 (i.e., an operation of changing the position), the aim 250 may not be displayed on the display 72 but may be located outside the display area. At this time, processing may be performed to actually move the aim 250 outside the display area, or processing may be performed to simply calculate the coordinates corresponding to the aim 250. The same applies when the right controller 3 assumes a different position, so a detailed description thereof will be omitted. After that, the processing proceeds to step S402.

[0094] In step S402, processor 63 determines whether or not the stick mode transition condition is satisfied, similarly to step S202 in Fig. 11. If the determination in step S402 is YES, the process proceeds to stick mode processing, and if the determination is NO, the process proceeds to step S403.

[0095] In step S403, the processor 63 determines whether a second mouse mode transition condition is satisfied based on the mouse sensor data 403. For example, the second mouse mode transition condition is a condition that requires more time to determine than the first mouse mode transition condition. As an example, the second mouse mode transition condition is a condition that indicates that the image clarity data 404 indicates a clarity level equal to or higher than a predetermined level, thereby inferring that the opening 23 of the mouse sensor 24 is blocked, and that the movement distance dy / dz of the right controller 3 per frame time is equal to or higher than a predetermined level (e.g., 0.1 mm or more) for a second predetermined number of consecutive frames (e.g., 180 frames). Note that the "second predetermined number" is greater than the "first predetermined number" used in step S101 of FIG. 10 and step S302 of FIG. 12. In this way, in gyro mode, a transition to mouse mode processing occurs when at least the second mouse mode transition condition is satisfied. Note that the gyro mode processing in Figure 13 is an example, and in other embodiments, the condition for transitioning from gyro mode processing to mouse mode processing may be simply that the second mouse mode transition condition described above is determined to be satisfied.

[0096] When operating the sight 250 by changing the orientation of the right controller 3 in gyro mode, the user often grips the right controller 3 and changes its orientation as shown in FIG. 7 , which can easily cause the mouse sensor opening 23 to be blocked by the middle finger of the right hand or the like. In this case, a mode transition from gyro mode to mouse mode may occur unintentionally. Therefore, in this embodiment, the determination of a transition from gyro mode to mouse mode is made more difficult than the determination of a transition from stick mode to mouse mode. As an example of a method for making this determination more difficult, the determination of a transition from gyro mode to mouse mode takes a relatively long time, as described above. This prevents an unintentional transition from gyro mode to mouse mode. Note that when operating the sight 250 by operating the stick in stick mode, the user often performs the operation without gripping the right controller 3 tightly, which makes it less likely that the mouse sensor opening 23 will be blocked by the middle finger of the right hand or the like. If the determination in step S403 is YES, the process proceeds to mouse mode processing, and if NO, the process proceeds to step S404.

[0097] In step S404, processor 63 determines whether the time during which the entirety of sight 250 is located outside the display range has reached a predetermined time (e.g., 3 seconds) based on aim coordinate data 408. If the determination in step S404 is YES, the process returns to step S101 in Fig. 10, and if NO, the process returns to step S401.

[0098] According to the present embodiment described above, when the operation modes include at least a mouse mode in which the aim is controlled based on the output of a mouse sensor, a gyro mode in which the aim is controlled based on the output of an inertial sensor, and a stick mode in which the aim is controlled based on stick input, it is possible to transition from the mouse mode to either the gyro mode or the stick mode. Therefore, the operation mode can be quickly transitioned from the mouse mode to the gyro mode or the stick mode in accordance with the user's intention.

[0099] Furthermore, according to this embodiment, it is possible to transition from gyro mode to either mouse mode or stick mode. Therefore, it is possible to quickly transition the operation mode from gyro mode to either mouse mode or stick mode in accordance with the user's intention. Furthermore, according to this embodiment, it is possible to quickly transition from stick mode to either mouse mode or gyro mode. Therefore, it is possible to quickly transition between the three operation modes in accordance with the user's intention.

[0100] Furthermore, according to this embodiment, as described with reference to FIG. 11 , if the determinations in steps S201 and S202 are NO and the determination in step S203 is YES, the mode transitions from mouse mode to gyro mode. Furthermore, if the determination in step S201 is YES, the mode transitions from mouse mode to gyro mode even if the output of the inertial sensor is YES in step S203. Thus, according to this embodiment, if a mouse is being operated in mouse mode, the mode remains in mouse mode regardless of the output of the inertial sensor. This prevents the mode from being unintentionally switched to gyro mode when the condition for transitioning to gyro mode is satisfied when no mouse operation is being performed, for example, when a swinging motion of the controller placed on the placement surface is detected.

[0101] Furthermore, according to this embodiment, when it is determined that no mouse operation, stick operation, or attitude change operation has been performed, and predetermined conditions (see S204, S304, and S404) are satisfied, the gyro mode is set (see FIGS. 10 to 13). Here, since the controller can be swung even in mouse mode and stick mode, detecting the user's intention to transition to gyro mode requires, for example, detecting a relatively large swing of the controller. On the other hand, if control is performed to transition to gyro mode by detecting a relatively large swing of the controller, it may be difficult to transition to gyro mode contrary to the user's intention. Therefore, in this embodiment, as described above, the gyro mode is set when it is estimated that the user has not performed any operation, for example, when the outputs of the mouse sensor and inertial sensor do not indicate movement of the controller and no stick operation has been performed. This achieves operability that is in line with the user's intention. In addition, in Figures 10 to 13, for example, the judgment processes of steps S204, S304, and S404 may not be performed, and the control may return to step S101 if a NO judgment is made in S203, if a NO judgment is made in S303, or if a NO judgment is made in S403.

[0102] Furthermore, according to this embodiment, a controller without a mouse sensor or an inertial sensor can also be connected to the main unit 2 and used. When a controller without a mouse sensor is connected to the main unit 2 and used, the operation mode transitions between stick mode and gyro mode according to the processes shown in FIGS. 10 to 13. When a controller without an inertial sensor is connected to the main unit 2 and used, the operation mode transitions between mouse mode and stick mode according to the processes shown in FIGS. 10 to 13. This allows the use of a controller without a mouse sensor or an inertial sensor. Furthermore, by connecting both a controller without a mouse sensor or an inertial sensor and a controller with a mouse sensor, a stick, and an inertial sensor to the main unit 2, two users can operate their respective controllers to control aiming, for example. Furthermore, three or more controllers can be connected to the main unit 2 and each controller can be operated to control aiming. Note that in this embodiment, an operation mode is set for each controller.

[0103] [Variations] In the above-described embodiment, the color, shape, etc. of the sight 250 may change depending on the operation mode. In such a case, the sights can be considered to be substantially the same.

[0104] In the above-described embodiment, the display position of the crosshair is not limited immediately after the transition of the operation mode. For example, the crosshair may be displayed in the same position as before the transition of the operation mode, or in the center of the screen, or may be different depending on the transition of the operation mode. Note that in mouse mode and stick mode, the crosshair may move outside the display screen, and in gyro mode, the crosshair may not be able to move outside the display screen.

[0105] In the above-described embodiment, the operation content assigned to each button may be switched depending on the operation mode. For example, in mouse mode, a bullet may be fired in response to operation of the A button 12, in stick mode, a bullet may be fired in response to operation of the ZR button 21, and in gyro mode, a bullet may be fired in response to operation of the R button 20.

[0106] In the above-described embodiment, the content of the game played in the virtual space may differ depending on the operation mode. For example, the type of bullets fired may differ depending on the operation mode, or the range in which a player object controlled by a user can move may differ.

[0107] In the above-described embodiment, an example was given in which a transition to a destination operation mode is made in accordance with the device output used to control the aim position in the destination operation mode. However, for example, a transition to a destination operation mode may be made in accordance with a device output different from the device output used to control the aim position in the destination operation mode. For example, instead of making a transition to mouse mode in accordance with mouse sensor output, a transition from gyro mode or stick mode to mouse mode may be made in accordance with inertial sensor output indicating that the negative x-axis direction of the right controller coordinate system (see FIG. 2) is oriented in the direction of gravity for a predetermined period of time (e.g., 0.5 seconds). Furthermore, a transition to an operation mode may be made in accordance with multiple types of device output. For example, a transition to mouse mode may be made based on mouse sensor output and inertial sensor output. For example, a transition to stick mode may be made based on stick output and mouse sensor output. For example, a transition to gyro mode may be made based on mouse sensor output and inertial sensor output.

[0108] Furthermore, in the above-described embodiment, when multiple controllers are connected to the main unit 2, an operation mode may be set for each controller, and aiming operations and the like may be performed according to operations on each controller.

[0109] Furthermore, the above-described game process of operating the aim to fire a bullet is one example, and may be, for example, game process of selecting an object pointed by a pointer such as the aim. The type of game is not limited, and examples include a fighting game, a puzzle game, a simulation game, and a music game. The game process of this embodiment is not limited to a game played by a single user, and may be a game in which multiple users compete or cooperate. For example, a first user may operate the movement of a character object, and a second user may operate the aim. For example, the movement of the character object by the first user may be controlled only by the stick of the controller used by the first user.

[0110] Furthermore, the above-described processing is not limited to being applied to game processing. For example, it may be applied to a drafting application, a video editing application, or an operating system. As an example, it may be applied to menu operations in an operating system. When it is applied to game processing, it may also be applied to menu operations within the game.

[0111] Furthermore, in the above-described embodiment, an example was given in which one controller has one stick (see FIG. 2, etc.), but one controller may have two or more sticks. In this case, for example, the aim may be operated or a transition to stick mode may occur in response to operation of one stick (see this embodiment), while the aim may not be operated or a transition to stick mode may not occur in response to operation of the other stick. In this case, the other stick may be used to move the player object or move the aim.

[0112] Alternatively, the user may hold and operate both the right and left controllers in one hand. In this case, for example, each controller may be operable in three modes, or only one controller may be operable in three modes, or the two controllers may share the operation modes. For example, the right controller may perform mouse operations, and the left controller may perform gyro and stick operations. Alternatively, the right controller may perform mouse and gyro operations, and the left controller may perform mouse and stick operations. In other words, the mode sharing includes cases where both controllers are compatible with some modes. In such a case, a controller that cannot perform aiming operations in a certain mode may not transition to that certain mode even if there is a device input corresponding to that certain mode. For example, if the right controller can perform mouse and gyro operations for aiming but cannot perform stick operations, the right controller may not transition to stick mode even if there is a stick operation. Note that devices not used for aiming operations may be used for other game processes. For example, in the above example, the stick on the right controller may be used to move the virtual camera or the player object.

[0113] In addition, in the above-described embodiment, an example has been given in which, in gyro mode, the attitude and movement of the controller are detected based on the output of a gyro sensor and processing is executed (see S401, S404, etc. in FIG. 13). However, in gyro mode, for example, the attitude and movement of the controller may be detected based on the output of an acceleration sensor, an optical sensor, etc., and similar processing may be executed. In this case, the process may not be called gyro mode or gyro mode processing, but may be called something else. In addition, in the above-described embodiment, an example has been given in which processing is executed based on stick output (see S202 in FIG. 11, S301, S304, S305 in FIG. 12, S402 in FIG. 13, etc.). However, for example, similar processing may be executed based on button output. In this case, the process may not be called stick mode or stick mode processing, but may be called something else.

[0114] In the above-described embodiment, an example was given in which the game program executed on the main unit determines and sets the mode. However, another program (e.g., an operation program) executed on the main unit may determine and set the mode, and the game program executed on the main unit may execute game processing based on the determination and setting. Furthermore, the processing for determining and setting the mode and the various processes constituting the game processing may be shared between the operation program and the game program. Furthermore, the controller may determine and set the mode.

[0115] Furthermore, in the above-described embodiment, an example has been given in which mode transitions are possible between mouse mode, gyro mode, and stick mode (see FIG. 8). However, possible transitions between modes are not limited to this. For example, mode transitions between gyro mode and stick mode may not be possible. For example, mode transitions from gyro mode and stick mode to mouse mode may not be possible. In this way, mode transitions from one mode to one or more other modes may not be possible.

[0116] Furthermore, the various data in the above-described embodiment are merely examples, and in each process, data converted into other data may be used as appropriate.

[0117] Furthermore, a game system is an example of an information processing system, and the information processing system may be a system in which a game is not executed. The main unit may be a general-purpose personal computer. The controller is also an example, and for example, its shape is not limited. The controller does not have to be detachable from the main unit. The controller is also an example, and for example, its shape is not limited. The controller does not have to be detachable from the main unit. Only one of the two controllers may have a mouse sensor. Only one of the two controllers may have a stick. The controllers do not have to be a pair.

[0118] At least a part of the series of processes described above may be executed by a server-side device in an information processing system including a terminal-side device and a server-side device capable of communicating with the terminal-side device via a network. Note that the server may be configured by a plurality of information processing devices, and the processes may be shared and executed by the plurality of information processing devices.

[0119] Although the present embodiment and its modifications have been described above, these descriptions are merely illustrative in every respect and are not intended to limit the scope of the present embodiment and its modifications. It goes without saying that various improvements and modifications can be made to the present embodiment and its modifications. [Explanation of symbols]

[0120] 1. Information Processing Systems 2 Main unit 3, 4 Controller 22, 42 sticks 24, 44 mouse sensor 63 processors 68, 69, 81, 94 Memory 72 Display 76, 77, 83, 84, 96, 97 Inertial sensors 82, 95 buttons 250 Aiming

Claims

1. 1. A computer-implemented information processing method including one or more processors, comprising: a mode setting step of setting one of a plurality of modes including a first mode, a second mode, and a third mode; Virtual objects, In the first mode, control is performed based on an output of a mouse sensor of at least one controller among the one or more controllers; In the second mode, control is performed based on an output of an inertial sensor included in at least one of the one or more controllers; a virtual object control step of controlling, in the third mode, a virtual object based on an output of a direction operation unit operated by a user, which is included in at least one controller of the one or more controllers; In the mode setting step, when the mode is the first mode, the mode is transitioned to the second mode based on at least a first condition being satisfied, and the mode is transitioned to the third mode based on at least a second condition being satisfied.

2. 2. The information processing method according to claim 1, wherein in the mode setting step, when the mode is the second mode, the mode is transitioned to the third mode based on the second condition being satisfied, and the mode is transitioned to the first mode based on at least the third condition being satisfied.

3. 3. The information processing method according to claim 2, wherein in the mode setting step, when the mode is the third mode, the mode is transitioned to the second mode based on at least the first condition being satisfied, and the mode is transitioned to the first mode based on at least a fourth condition being satisfied.

4. the third condition and the fourth condition each include a condition related to an output of a mouse sensor; The information processing method according to claim 3 , wherein the time required to determine whether the third condition is met is longer than the time required to determine whether the fourth condition is met.

5. the first condition includes a condition regarding an output of the inertial sensor, In the first mode, If the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, transition to the second mode when at least the first condition is satisfied; 2. The information processing method according to claim 1, wherein if the output of the mouse sensor indicates movement of the controller, the transition to the second mode is not made even if the first condition is satisfied.

6. 2. The information processing method according to claim 1, wherein the second mode is set when the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the directional control unit is not being operated.

7. 2. The information processing method of claim 1, wherein the mode setting step transitions between the third mode and one of the first mode or the second mode, but does not transition between the third mode and the other mode, based on the output of a controller that has the directional operation unit and either the mouse sensor or the inertial sensor, but not the other.

8. causing the computer to execute a game process; The information processing method according to claim 1 , wherein the virtual object is a pointer used in the game processing.

9. An information processing system comprising one or more controllers each having a mouse sensor, an inertial sensor, and a direction operation unit operated by a user, and an information processing unit each having one or more processors, a mode setting means for setting one of a plurality of modes including a first mode, a second mode, and a third mode; Virtual objects, In the first mode, control is performed based on the output of the mouse sensor; In the second mode, control is performed based on an output of the inertial sensor; a virtual object control unit that controls the virtual object based on an output of the direction operation unit when in the third mode; The mode setting means, when the mode is the first mode, transitions to the second mode based on at least a first condition being satisfied, and transitions to the third mode based on at least a second condition being satisfied.

10. 10. The information processing system according to claim 9, wherein the mode setting means, when the mode is the second mode, transitions to the third mode based on the second condition being satisfied, and transitions to the first mode based on at least a third condition being satisfied.

11. 11. The information processing system according to claim 10, wherein the mode setting means, when the mode is the third mode, transitions to the second mode based on at least the first condition being satisfied, and transitions to the first mode based on at least a fourth condition being satisfied.

12. the third condition and the fourth condition each include a condition related to an output of a mouse sensor; The information processing system according to claim 11 , wherein a time required for determining whether the third condition is satisfied is longer than a time required for determining whether the fourth condition is satisfied.

13. one or more processors, a mode setting step of setting one of a plurality of modes including a first mode, a second mode, and a third mode; Virtual objects, In the first mode, control is performed based on an output of a mouse sensor of at least one controller among the one or more controllers; In the second mode, control is performed based on an output of an inertial sensor included in at least one of the one or more controllers; a virtual object control step of controlling the virtual object based on an output of a direction operation unit of at least one controller among the one or more controllers in the third mode; In the mode setting step, when the mode is the first mode, the mode is transitioned to the second mode based on at least a first condition being satisfied, and the mode is transitioned to the third mode based on at least a second condition being satisfied.

14. 14. The information processing program according to claim 13, wherein in the mode setting step, when the mode is the second mode, the mode is transitioned to the third mode based on the second condition being satisfied, and the mode is transitioned to the first mode based on at least a third condition being satisfied.

15. 15. The information processing program according to claim 14, wherein in the mode setting step, when the mode is the third mode, the mode is transitioned to the second mode based on at least the first condition being satisfied, and the mode is transitioned to the first mode based on at least a fourth condition being satisfied.

16. the third condition and the fourth condition each include a condition related to an output of a mouse sensor; The information processing program according to claim 15 , wherein a time required for determining whether the third condition is met is longer than a time required for determining whether the fourth condition is met.

Images