Information processing method, information processing system, and information processing program

Abstract

To provide a novel operation relating to a virtual object.SOLUTION: In an information processing method, a virtual object is controlled by output of a mouse sensor in a first mode, is controlled by output of an inertia sensor in a second mode, and is controlled by output of a direction operation unit in a third mode. In the first mode, it is transitioned to the second mode when at least a first condition is satisfied, and it is transitioned to the third mode when at least a second condition is satisfied.SELECTED DRAWING: Figure 11

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present disclosure relates to information processing such as games.

Background Art

[0002] Conventionally, it is known to switch whether to operate the aiming position based on the operation of the operation unit or based on the acquired coordinates. Also, it is known that the acquisition of coordinates used for operating the aiming position may use, for example, an acceleration sensor. Further, it is known that the acquired coordinates, the movement of the controller, and the operation state of the operation unit can be considered in this operation switching. For example, refer to paragraphs

[0146] ,

[0151] ,

[0155] of Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-described technology, the case where there is an input means different from the above-described input means used for operating the aiming position is not considered.

Means for Solving the Problems

[0005] <, For example, the following configuration examples can be cited.

[0006] (Configuration 1) Configuration 1 is an information processing method implemented by a computer including one or more processors, comprising: a mode setting step of setting one of a plurality of modes including a first mode, a second mode, and a third mode; and a virtual object control step of controlling a virtual object based on the output of a mouse sensor of at least one of the one or more controllers when in the first mode, based on the output of an inertial sensor of at least one of the one or more controllers when in the second mode, and based on the output of a user-operated directional control unit of at least one of the one or more controllers when in the third mode, wherein in the mode setting step, when the mode is the first mode, the system transitions to the second mode based on the satisfaction of at least a first condition, and transitions to the third mode based on the satisfaction of at least a second condition.

[0007] (Configuration 2) Configuration 2, in the above Configuration 1, may, in the mode setting step, transition to the third mode based on the second condition being met when the mode is the second mode, and transition to the first mode based on at least the third condition being met.

[0008] (Composition 3) Configuration 3, in Configuration 1 or 2 described above, may, in the mode setting step, transition to the second mode based on the fulfillment of at least the first condition when the mode is the third mode, and transition to the first mode based on the fulfillment of at least the fourth condition.

[0009] (Composition 4) Configuration 4 is the same as in Configuration 3 above, in which the third and fourth conditions each include conditions related to the output of the mouse sensor, and the time required to determine the third condition may be longer than the time required to determine the fourth condition.

[0010] (Composition 5) Configuration 5 is one of the above configurations 1 to 4, wherein the first condition includes a condition relating to the output of the inertial sensor, and in the first mode, if the output of the mouse sensor does not indicate the movement of at least one controller having the mouse sensor, the system transitions to the second mode when at least the first condition is met, and if the output of the mouse sensor indicates the movement of the controller, the system does not need to transition to the second mode even if the first condition is met.

[0011] (Composition 6) Configuration 6 may set a second mode in any of the above configurations 1 to 5 when the output of the mouse sensor does not indicate movement of at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the directional control unit is not being operated.

[0012] (Composition 7) Configuration 7 is one of the above configurations 1 to 6, in which the mode setting step transitions between the third mode and one of the first or second modes based on the output of a controller that has a direction control unit and either a mouse sensor or an inertial sensor, but not the other, and does not require a transition to the other mode.

[0013] (Composition 8) Configuration 8 is one of the above configurations 1 to 7, in which the computer is made to execute game processing, and the virtual object may be a pointer used in the game processing. [Effects of the Invention]

[0014] According to this embodiment, it is possible to provide novel operations on virtual objects. [Brief explanation of the drawing]

[0015] [Figure 1] This diagram shows an example of the main unit 2 with the right controller 3 and left controller 4 attached. [Figure 2] A six-view drawing showing an example of the right controller 3. [Figure 3]Six-sided view showing an example of the left controller 4 [Figure 4] Block diagram showing an example of the internal configuration of the main body device 2 [Figure 5] Block diagram showing an example of the internal configurations of the main body device 2, the right controller 3, and the left controller 4 [Figure 6] Figure showing an example of a state where the right controller 3 is held and operated with the right hand [Figure 7] Figure showing an example of a state where the right controller 3 is held and operated with the right hand [Figure 8] Figure for explaining the transition of the operation mode of the controller [Figure 9] Figure showing examples of various data stored in the DRAM 69 [Figure 10] An example of a flowchart of information processing [Figure 11] An example of a flowchart of information processing [Figure 12] An example of a flowchart of information processing [Figure 13] An example of a flowchart of information processing [Figure 14] Figure showing an example of a sight screen display [Figure 15] Figure showing an example of a sight screen display [Figure 16] Figure showing an example of a sight screen display [Embodiment for Carrying Out the Invention]

[0016] Hereinafter, one embodiment will be described.

[0017] [Example of the Hardware Configuration of the Information Processing System]

[0018] Hereinafter, a game system, which is an example of the information processing system of the present embodiment, will be described. An example of the game system 1 in the present embodiment includes an information processing device (which may be referred to as the "main body device") 2, a right controller 3, and a left controller 4. In the present embodiment, the right controller 3 and the left controller 4 are each detachable from the main body device 2.

[0019] Figure 1 shows an example of the main unit 2 with the right controller 3 and left controller 4 attached. As shown in Figure 1, the right controller 3 and left controller 4 are attached to the main unit 2 and integrated together. The main unit 2 is a device that performs various processes (e.g., game processing) in the game system 1. The main unit 2 is equipped with a display 72. The right controller 3 and left controller 4 are input devices equipped with operation parts for user input. In the following, the right controller 3 and left controller 4 may be collectively referred to as "controllers".

[0020] The display 72 displays images generated by the main unit 2. The display 72 is, for example, a liquid crystal display (LCD). The screen of the display 72 is equipped with a touch panel. The touch panel is, for example, a multi-touch input type (e.g., capacitive type).

[0021] Figure 2 is a hexagonal schematic diagram showing an example of a right controller 3. As shown in Figure 2, the right controller 3 is a vertically elongated plate shape, includes a housing 11, and has a front, rear, top, bottom, right, and left section. In the right controller 3, the rear is located opposite the front, the bottom is located opposite the top, and the left is located opposite the right section. The distance between the front and rear is greater than the distance between the top and bottom. The distance between the top and bottom is greater than the distance between the right and left sections. In other embodiments, the relative magnitudes of these distances may be different. In this embodiment, the direction connecting the bottom and top may be called the up-down direction, the direction perpendicular to the up-down direction and connecting the front and rear may be called the front-back direction, and the direction perpendicular to the up-down direction and the front-back direction and connecting the right and left sections may be called the left-right direction. In Figure 2, the x, y, and z axes are shown in a front view with the left side facing forward, indicating the coordinate system of the right controller 3 (sometimes referred to as the "right controller coordinate system"). In this coordinate system, the direction from the left side to the right side is the positive z-axis direction. The direction perpendicular to the z-axis and from the bottom to the top is the positive x-axis direction, and the direction perpendicular to the z and x axes and from the rear to the front is the positive y-axis direction. When the bottom side faces the direction of gravity, the negative x-axis direction and the direction of gravity coincide. In the description related to the right controller 3, the x, y, and z axes refer to the x, y, and z axes in the right controller coordinate system unless otherwise specified. In this embodiment, the front and bottom parts do not need to be perfectly flat and may have irregularities or slopes. For example, the bottom part includes the convex part 25 described later. The directions of each part and the directions connecting each part are approximate directions.

[0022] The right controller 3 has a protrusion 25 that fits into a recess (not shown) of the main unit 2 when it is mounted on the main unit 2. As shown in Figure 2, the protrusion 25 is a convex shape that protrudes in the negative x-axis direction, with a width shorter in the left-right direction than the left-right direction of the right controller 3 and a width shorter in the front-back direction than the front-back direction. In this embodiment, the protrusion 25 is part of the bottom.

[0023] As will be described later, the right controller 3 can also be held in a vertical orientation when detached from the main unit 2. When held in a vertical orientation, the right controller 3 is shaped and sized to be held with one hand, especially the right hand. The right controller 3 can also be held in a horizontal orientation, and when held in a horizontal orientation, it may be held with both hands (not shown).

[0024] The right controller 3 is equipped with an analog stick (sometimes simply called a "stick") 22 on its left side, which is an example of a directional input unit. The stick 22 can be used as a directional input unit that allows for directional input. The stick may also be called a directional control unit. The user can input direction by tilting the stick 22 in any direction, and the magnitude of the input can be determined according to the angle of tilt. The user can also input buttons by pressing the stick 22. The directional input unit may also be, for example, a directional pad or a slide pad. The directional input unit may also be called a directional control unit.

[0025] The right controller 3 has a set of four buttons on its left side: A button 12, B button 13, X button 14, Y button 15, a + (plus) button 16, and a home button 17. The right controller 3 has an R button 20 and a ZR button 21 extending across its front and top. The R button 20 and ZR button 21 may be located only on the front of the right controller 3, or only on the top. The right controller 3 has buttons 18 and 19 on the top surface 25a of the protrusion 25.

[0026] The right controller 3 is provided with an opening 23 for a mouse sensor on the top surface 25a of the protrusion 25. The opening 23 for the mouse sensor is an opening in the light guide path that guides light to the mouse sensor 24 located inside the right controller 3. The mouse sensor 24 is an optical mouse sensor and may include a light-emitting part and a light-receiving part. The light detected by the light-receiving part may be visible light or light of an invisible wavelength. The mouse sensor 24 may have at least a light-receiving part and not a light-emitting part. The mouse sensor 24 acquires data that allows for the calculation of the movement of the right controller 3 on the mounting surface, with the top surface 25a of the protrusion 25 of the bottom facing the mounting surface. In this way, the right controller 3 can also be used as a mouse. The operation of using it as a mouse is sometimes called "mouse operation". The mounting surface is not limited to a flat surface, but may be a curved surface, for example, the surface of the user's thigh.

[0027] Furthermore, in this embodiment, the right controller 3 is provided with a terminal 26 on the protrusion 25 for the right controller 3 to communicate with the main unit 2 via wired connection. As an example, the terminal 26 is provided on the inner circumferential surface of a recess provided on the top surface 25a of the protrusion 25.

[0028] Figure 3 is a hexagonal schematic diagram showing an example of the left controller 4. The same configuration as the right controller 3 will not be explained. The left controller 4 has a stick 42, a set of four buttons (right direction button 32, down direction button 33, up direction button 34, left direction button 35), a capture button 37, and a minus button 36 on its right side. Buttons 32-35 may be a single directional pad. Note that in the right controller 3, the stick 22 is located behind buttons 12-15, whereas in the left controller 4, the stick 42 is located in front of buttons 32-35. In Figure 3, the x, y, and z axes are shown relative to a front view with the right side facing forward, indicating the coordinate system of the left controller 4 (sometimes referred to as the "left controller coordinate system"). In this coordinate system, the direction from the right side to the left side is the positive z-axis direction. Furthermore, the direction perpendicular to the z-axis and moving from the bottom to the top is the positive x-axis direction, and the direction perpendicular to the z-axis and x-axis and moving from the rear to the front is the positive y-axis direction. When the bottom faces the direction of gravity, the negative x-axis direction and the direction of gravity coincide. Note that the x-axis, y-axis, and z-axis in the explanation related to the left controller 4 refer to the x-axis, y-axis, and z-axis in the left controller coordinate system unless otherwise specified.

[0029] The left controller 4 has a protrusion 45 that fits into a recess (not shown) of the main unit 2 when it is attached to the main unit 2. The protrusion 45, like the right controller 3, has buttons 38 and 39, an opening 43 for the mouse sensor, a mouse sensor 44, and a terminal 46.

[0030] When the left controller 4 is detached from the main unit 2, it can be held in either a vertical or horizontal orientation, similar to the right controller 3.

[0031] Figure 4 is a block diagram showing an example of the internal configuration of the main unit 2. The main unit 2 includes a processor 63. The processor 63 is an information processing unit that performs various information processing tasks performed in the main unit 2. The processor 63 may consist of, for example, multiple processors or cores, typically multiple CPUs (Central Processing Units) or cores, or it may consist of an SoC (System-on-a-chip) that includes multiple functions such as CPU function and GPU (Graphics Processing Unit) function. The processor 63 performs various information processing tasks by executing information processing programs (for example, game programs) stored in a storage unit (specifically, an internal storage medium such as flash memory 68, or an external storage medium installed in a slot 51, etc.). In this embodiment, "processor" may include at least a CPU, GPU, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc. In this embodiment, the computer, as an example, includes at least one processor and may further include a storage unit such as memory.

[0032] The main unit 2 includes a flash memory 68 and a DRAM (Dynamic Random Access Memory) 69 as examples of internal storage media. The flash memory 68 is a memory mainly used to store various types of data stored in the main unit 2. The DRAM 69 is a memory mainly used to temporarily store various types of data used in information processing. The processor 63 reads and writes data to and from storage media such as the flash memory 68 and the DRAM 69 as appropriate and performs various information processing.

[0033] Furthermore, the main unit 2 has various configurations as shown in Figure 4. These are briefly described below. The recording medium slot interface (sometimes referred to as "slot I / F") 52 reads and writes data to and from the storage medium (for example, a dedicated memory card) installed in the recording medium slot 51, according to instructions from the processor 63. The second slot I / F 54 reads and writes data to and from the storage medium installed in the second slot 53, according to instructions from the processor 63.

[0034] The network communication unit 66 communicates with external devices via the network (for example, internet communication using wireless communication). The controller communication unit 67 communicates wirelessly with the right controller 3 and / or the left controller 4 (for example, communication in accordance with the Bluetooth® standard).

[0035] The left terminal 50 is a terminal for wired communication between the processor 63 and the left controller 4. The right terminal 65 is a terminal for wired communication between the processor 63 and the right controller 3. The lower terminal 64 is a terminal for communication with other devices (e.g., a stationary monitor, etc.) via the cradle when the lower terminal 64 is mounted on the cradle.

[0036] The touch panel controller 70 generates data indicating, for example, the location where a touch input occurred, based on signals from the touch panel 71 located on the display surface of the display 72, and outputs this data to the processor 63. The display 72 displays images generated by the processor 63 and / or images acquired from an external source.

[0037] The codec circuit 74 controls the input and output of audio data to the speaker 73 and the audio input / output terminal 75.

[0038] The power control unit 61 controls the power supply from the battery 62 to each part of the main unit 2 (i.e., each part that receives power from the battery 62) based on commands from the processor 63, and also starts or stops the power supply in response to the pressing of the power button 60.

[0039] The volume button 59 is used to control the volume output from the speaker 73, etc. The cooling fan 58 is a fan that cools the inside of the main unit 2.

[0040] The main unit 2 is equipped with various sensors, including a magnetic force sensor 55, an ambient light sensor 56, a temperature sensor 57, an acceleration sensor 76, and an angular velocity sensor 77. The processor 63 can perform various processes based on information from these sensors.

[0041] Figure 5 is a block diagram showing an example of the internal configuration of the main unit 2, the right controller 3, and the left controller 4. Note that the details of the internal configuration of the main unit 2 are shown in Figure 4 and are therefore omitted in Figure 5.

[0042] The left controller 4 includes a communication control unit 80 that communicates with the main unit 2. As shown in Figure 5, the communication control unit 80 is connected to each component, including terminals 88. When the left controller 4 is attached to the main unit 2, the communication control unit 80 communicates with the main unit 2 via wired communication through terminals 88. When the left controller 4 is detached from the main unit 2, the communication control unit 80 communicates with the main unit 2 wirelessly (specifically, communication in accordance with the Bluetooth® standard).

[0043] The left controller 4 includes a memory 81, such as flash memory. The communication control unit 80 is composed of a processor, such as a microcontroller (also called a microcomputer), and performs various processes by executing firmware stored in the memory 81.

[0044] The left controller 4 is equipped with buttons 82 (specifically, buttons 32-34, etc.) and a stick 42. Each button 82 and stick 42 outputs information about the operation performed on it to the communication control unit 80.

[0045] The left controller 4 is equipped with inertial sensors. Specifically, the left controller 4 is equipped with an acceleration sensor 83 and an angular velocity sensor 84 as inertial sensors. The acceleration sensor 83 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in Figure 3). Note that the acceleration sensor 83 may also detect acceleration in one axis direction or two axis directions. The angular velocity sensor 84 detects angular velocity around the three predetermined axes. Note that the angular velocity sensor 84 may also detect angular velocity around one axis or two axes. The angular velocity sensor may also be called a "gyro sensor". The acceleration sensor 83 and the angular velocity sensor 84 are each connected to the communication control unit 80. The detection results of the acceleration sensor 83 and the angular velocity sensor 84 are repeatedly output to the communication control unit 80 at appropriate timings. Note that the right controller 3 and the left controller 4 may be equipped with either an acceleration sensor or an angular velocity sensor as inertial sensors, or they may be equipped with other sensors.

[0046] The left controller 4 is equipped with a mouse sensor 44. The mouse sensor 44 acquires data for calculating the movement of the left controller 4 placed on the mounting surface. The data acquired by the mouse sensor 44 is repeatedly output to the communication control unit 80 at appropriate intervals.

[0047] The communication control unit 80 acquires information about inputs (specifically, information about the operation of buttons and sticks, and detection results from sensors) from each input unit (specifically, each button 82, stick 42, and each sensor 83, 84, and 44). The communication control unit 80 transmits operation data, which includes the acquired information or information that has been processed in a predetermined manner, to the main unit 2. The operation data is transmitted repeatedly at a rate of once every predetermined time.

[0048] When the above operation data is transmitted to the main unit 2, the main unit 2 can obtain the input made to the left controller 4. That is, the main unit 2 can determine the operation of each button 82 and stick 42 based on the operation data. In addition, the main unit 2 can calculate information about the movement and / or posture of the left controller 4 based on the operation data (specifically, the detection results of the acceleration sensor 83 and / or angular velocity sensor 84). Furthermore, the main unit 2 can calculate information about mouse operations made to the left controller 4 based on the operation data (specifically, the detection results of the mouse sensor 44).

[0049] The left controller 4 includes an amplifier 85 and a vibrator 86. The amplifier 85 amplifies the control signal received from the communication control unit 80 and generates a drive signal. The vibrator 86 vibrates in response to the drive signal generated by the amplifier 85, causing the left controller 4 to vibrate.

[0050] The left controller 4 includes a power supply unit 87. The power supply unit 87 has a battery and a power control circuit. The power control circuit is connected to the battery and supplies power to each part of the left controller 4 (specifically, each part that receives power from the battery).

[0051] As shown in Figure 5, the right controller 3 is composed of a processor and other components and includes a communication control unit 91 that communicates with the main unit 2. The right controller 3 also includes a memory 94 connected to the communication control unit 91. The communication control unit 91 is connected to each component, including the terminal 92. The communication control unit 91 and the memory 94 have the same functions as the communication control unit 80 and memory 81 of the left controller 4. Therefore, the communication control unit 91 can communicate with the main unit 2 both by wired communication via the terminal 92 and by wireless communication without using the terminal 92, and controls the communication that the right controller 3 makes to the main unit 2.

[0052] The right controller 3 is equipped with input ports in the same way as the left controller 4. Specifically, it is equipped with buttons 95 (A button 12, B button 13, X button 14, Y button 15, etc.), a stick 22, inertial sensors (accelerometer 96 and angular velocity sensor 97), and a mouse sensor 24. Each of these input ports has the same function and operates in the same way as the input ports of the left controller 4.

[0053] The right controller 3 comprises an amplifier 98, a vibrator 99, and a power supply unit 100. The amplifier 98, vibrator 99, and power supply unit 100 have the same functions and operate in the same manner as the amplifier 85, vibrator 86, and power supply unit 87 of the left controller 4, respectively.

[0054] [Regarding the gripping configuration of the controller] Figure 6 is a schematic diagram showing an example of a state in which a user grasps the right controller 3 with their right hand, places it on a mounting surface, and uses it as a mouse, i.e., a state of mouse operation. As shown in Figure 6, from the user's perspective, the front of the right controller 3 faces forward and the left side faces left. The user's right palm covers the upper side of the right controller 3. The user's right thumb is positioned on the left side of the right controller 3. The user's right thumb is placed on, for example, the A button 12. The user's right index finger is placed on, for example, the R button 20, and the user's right middle finger is placed on, for example, the ZR button 21. The user can operate the R button 20 and ZR button 21 with their right index finger or middle finger. The user can operate each input unit located on the left side with their right thumb. Note that when the user uses the left controller 4 as a mouse with their left hand, they can grasp the left controller 4 with their left hand and use it on the mounting surface in the same manner. At this time, the right side of the left controller 4 faces to the right.

[0055] Figure 7 is a schematic diagram showing an example of a user holding the right controller 3 in their right hand and operating it in the air. As shown in Figure 7, when the right controller 3 is detached from the main unit 2, it can be held in the air so that its longitudinal direction is in the vertical or horizontal direction for the user. The user can operate the stick 22 (sometimes referred to as "stick operation") with, for example, their right thumb. The user can also perform operations such as shaking the held right controller 3 (sometimes referred to as "shaking operation") or changing its posture (sometimes referred to as "posture change operation"). The same procedure can be used when the user uses the left controller 4, which is detached from the main unit 2, with their left hand.

[0056] [About the controller's operating modes] Figure 8 is a diagram illustrating the controller's operation modes and transitions between operation modes. As shown in Figure 8, the controller's operation modes (sometimes simply referred to as "modes") include "mouse mode," "gyro mode," and "stick mode." Mouse mode is an operation mode that controls virtual objects based on the output of the mouse sensor. Gyro mode is an operation mode that controls virtual objects based on the output of the gyro sensor. Stick mode is an operation mode that controls virtual objects based on the output of stick operation. As will be described later using Figure 14, etc., a reticle 250, which is a pointer, is displayed on the display 72 as an example of a virtual object. Note that the reticle is just one example of a pointer, and its shape is not limited to, for example, an arrow-shaped cursor, a hand icon, or a frame. Also, the pointer is just one example of a virtual object that is controlled, and the virtual object may be, for example, a character object or other object. The "initial state" is a state that can be set when the operation mode has not been determined, or when the operation mode transitions between mouse mode, gyro mode, and stick mode. Note that in other embodiments, there may be no initial state.

[0057] Furthermore, as shown in Figure 8, in this embodiment, it is possible to transition from one mode to another. The specific details of the transitions will be described later.

[0058] [Details of the information processing in this embodiment] Next, the information processing of this embodiment will be described in detail with reference to Figures 9 to 16. In this embodiment, it is possible to transition between operation modes as described above, and the position of the target 250 is controlled based on the operation corresponding to the operation mode. In the following, the case in which the right controller 3 is used will be used as an example. The case in which the left controller 4 is used can be considered similarly, so its explanation will be omitted.

[0059] [About the data used] Next, we will explain the various types of data stored in the DRAM 69. Figure 9 shows an example of data stored in the DRAM 69 of the main unit 2. As shown in Figure 9, the DRAM 69 is provided with at least a program storage area 301 and a data storage area 302.

[0060] The program storage area 301 stores at least the program 401. The data storage area 302 stores at least the operation mode data 402, mouse sensor data 403, stick / button input data 406, inertial sensor data 407, aiming coordinate data 408, object data 409, image data 410, virtual camera control data 411, and controller device data 412.

[0061] Program 401 is a game program for executing game processing.

[0062] Operation mode data 402 indicates whether the operation mode is mouse mode, gyro mode, or stick mode.

[0063] Mouse sensor data 403 is data relating to the output of the mouse sensor 24, and includes image clarity data 404 and dy / dz data 405.

[0064] Image clarity data 404 is data calculated by the mouse sensor 24 and indicates the clarity of the mouse sensor image. Image clarity data 404 is calculated, for example, based on the degree of brightness of the mouse sensor image and / or the degree of the number of feature points in the mouse sensor image. Alternatively, image clarity data 404 may be calculated by the communication control unit 91 or processor 63, etc., based on the output data of the mouse sensor 24. Furthermore, the degree of brightness of the mouse sensor image or the degree of the number of feature points in the mouse sensor image may be used directly as image clarity data. Also, image clarity data 404 may be calculated based on other factors. If the clarity indicated by image clarity data 404 is above a predetermined level, it can be estimated that the opening 23 of the mouse sensor 24 is blocked by the mounting surface, etc. Note that other data may be used instead of the data indicating the clarity of the mouse sensor image, as long as the data allows for the estimation that the opening 23 of the mouse sensor 24 is blocked.

[0065] The dy / dz data 405 is output data from the mouse sensor 24, and when the opening 23 of the mouse sensor 24 is blocked by the mounting surface, it is data that indicates the distance traveled per frame time in the y-axis and z-axis directions of the right controller coordinate system (i.e., the yz plane; see Figure 2) relative to the mounting surface, etc. (sometimes referred to as "dy / dz"). Note that dy / dz may be calculated from the output data of the mouse sensor 24 by the communication control unit 91 or the processor 63, etc.

[0066] The stick / button input data 406 is data indicating the operations performed on the stick 22 and each button 95 of the right controller 3.

[0067] The inertial sensor data 407 is data output from the inertial sensor of the right controller 3, and for example, it is data that allows calculation of acceleration in the xyz axis direction (see Figure 2) and angular velocity around the xyz axis in the right controller coordinate system. Using the inertial sensor data, for example, the attitude and movement of the right controller 3 can be calculated.

[0068] The aiming coordinate data 408 is data that shows the coordinates of the aiming reticle 250 (sometimes simply called "aiming coordinates") in the screen coordinate system obtained by converting the virtual space captured by the virtual camera into a planar coordinate system. The aiming coordinates move based on dy / dz data in mouse mode, based on the operation of the stick 22 in stick mode, and based on the attitude change of the right controller 3 in gyro mode.

[0069] Object data 409 is data for virtual objects placed in the virtual space, such as bullets fired in the direction of the aiming reticle 250, player characters, opponent characters, and the ground.

[0070] Image data 410 includes images of the virtual object, the crosshair 250, as well as animation images, backgrounds, virtual effects, and other image data. The image of the crosshair 250 is placed in the screen coordinate system described above and displayed on the display 72, etc. Alternatively, instead of displaying the crosshair 250 on the display 72, etc. by placing it in the screen coordinate system, it may be placed in a virtual space and captured by a virtual camera to be displayed on the display 72, etc. In other words, the crosshair 250 may be represented as object data 409 instead of image data 410.

[0071] The virtual camera control data 411 is data for controlling a virtual camera that is placed in a virtual space and takes pictures of that virtual space.

[0072] Controller device data 412 is data indicating the devices (e.g., mouse sensor, accelerometer, angular velocity sensor, stick, etc.) of the controller connected to the main unit 2. When the controller is connected to the main unit 2, data indicating the devices of the controller is transmitted from the controller to the main unit 2.

[0073] In addition, various types of data used for drawing and other processes are stored in DRAM69 as needed.

[0074] [Example of detailed information processing] Next, the processing according to this embodiment will be described with reference to flowcharts, etc. Figures 10 to 13 are examples of flowcharts showing the processing according to this embodiment. Figure 14 is a diagram for explaining the control of the aiming reticle 250 in mouse mode and stick mode. Figures 15 and 16 are diagrams for explaining the control of the aiming reticle 250 in gyro mode. In the following, we will mainly describe the processing that is characteristic of this embodiment, and other explanations such as drawing processing will be omitted in principle. Also, the following processing is performed at predetermined intervals, for example (for example, at processing frame intervals executed every 1 / 60th of a second).

[0075] When the game processing begins, in step S101 of Figure 10, the processor 63 determines whether the first mouse mode transition condition has been met based on the mouse sensor data 403. The first mouse mode transition condition in this embodiment is a condition in which it can be estimated that the right controller 3 is being operated by the mouse. For example, the condition is that the clarity indicated by the image clarity data 404 is above a predetermined level, which is estimated to be above a predetermined level, and that there have been a first predetermined number of frames (for example, 5 frames) or more in which the dy / dz, which is the distance moved per frame time of the right controller 3, is above a predetermined level (for example, 0.1 mm or more). If the determination in step S101 is YES, the process moves to the mouse mode processing in step S200; if it is NO, the process moves to step S102.

[0076] In step S102, the processor 63 determines, based on the controller device data 412, whether or not the right controller 3 connected to the main unit 2 has a gyro device (i.e., an angular velocity sensor). This determination may be made by other means. For example, if the main unit 2 receives information regarding the detection results of the angular velocity sensor repeatedly output from the right controller 3 at appropriate intervals, it may be determined that the right controller 3 has a gyro device. If the determination in step S102 is YES, the process moves to the gyro mode processing in step S300; if NO, the process moves to the stick mode processing in step S400. Note that the system is in mouse mode during mouse mode processing, stick mode during stick mode processing, and gyro mode during gyro mode processing.

[0077] In step S201 of Figure 11, the processor 63 determines whether the dy / dz value shown in the latest dy / dz data 405 is greater than a predetermined value (for example, 0.1 mm). This determination allows the processor to estimate whether the controller is moving on the mounting surface due to mouse operation. Note that this determination only needs to estimate whether the controller is moving on the mounting surface due to mouse operation; for example, it could be a determination of whether the dy / dz value shown in the latest dy / dz data is greater than 0 (zero). If the determination in step S201 is YES, the process moves to step S205; if NO, the process moves to step S202.

[0078] In step S205, the processor 63 performs aiming position control based on the dy / dz data 405 and aiming coordinate data 408. Specifically, the processor 63 moves the aiming position indicated by the aiming coordinate data 408 by a distance corresponding to the magnitude of the dy / dz in the direction indicated by the dy / dz shown in the latest dy / dz data 405.

[0079] Here, Figure 14 is a diagram illustrating the control of the aiming position in mouse mode and stick mode. When, for example, as shown in Figure 14(1)(b), a mouse operation is performed by step S205, causing the right controller 3 placed on the mounting surface to move in the z-axis positive direction, the aiming reticle 250 displayed on the display 72 moves to the right by a distance corresponding to the mouse operation, as shown in Figure 14(1)(a). Similarly, although not shown, if the right controller 3 is operated in another direction on the mounting surface, the aiming reticle 250 displayed on the display 72 also moves in a direction and by a distance corresponding to the mouse operation.

[0080] However, in the aiming position control of step S205, the processor 63 may restrict the movement of the center of the aiming reticle 250 so as not to move outside the display area of ​​the display 72 (sometimes simply referred to as the "display area"). For example, as shown in Figure 14(2)(a), when the center of the aiming reticle 250 is located at the right edge of the display area, even if a mouse operation is performed in the z-axis positive direction (see Figure 2) as shown in Figure 14(2)(b), the aiming reticle 250 is controlled so as not to move further to the right. As a result, a portion of the aiming reticle 250 is always displayed on the display 72. Note that the movement restriction of the aiming reticle 250 is not limited to this; for example, the movement restriction may be limited so that the entire aiming reticle 250 remains within the display area. After that, the process returns to step S201.

[0081] Regardless of the operating mode, and regardless of whether the reticle is moving or not, the processor 63 fires, for example, a virtual object such as a bullet in the direction indicated by the reticle, in response to a button operation (for example, the operation of the R button 20) based on the stick / button input data 406.

[0082] Returning to Figure 11, in step S202, the processor 63 determines whether the stick mode transition condition has been met based on the stick / button input data 406. In this embodiment, the stick mode transition condition is that the stick 22 has been operated. Thus, in mouse mode, the process transitions to stick mode processing when at least the stick mode transition condition is met. Note that the mouse mode processing in Figure 11 is just one example, and in other embodiments, the condition for transitioning from mouse mode processing to stick mode processing may be determined to be only that the above-described stick mode transition condition has been met. If the determination in step S202 is YES, the process moves to stick mode processing; if NO, the process moves to step S203.

[0083] In step S203, the processor 63 determines whether the gyro mode transition condition has been met based on the inertial sensor data 407. In this embodiment, the gyro mode transition condition is an estimation of whether the controller has been shaken by the user. As an example, the gyro mode transition condition is that the right controller 3 has been shaken with an acceleration of a predetermined value or higher (for example, 0.2G or higher). In this way, in mouse mode, the system transitions to gyro mode processing when at least the gyro mode transition condition is met. Note that the mouse mode processing in Figure 11 is just one example, and in other embodiments, the condition for transitioning from mouse mode processing to gyro mode processing may be determined to be only when the above-described gyro mode transition condition has been met. If the determination in step S203 is YES, the process moves to gyro mode processing; if NO, the process moves to step S204.

[0084] In step S204, the processor 63 determines conditions for estimating whether the user intends to continue mouse operation. In this embodiment, as an example, the processor 63 determines, based on image clarity data 404 and aiming coordinate data 408, whether a predetermined time has been reached for the time the mouse sensor aperture is not blocked, or for the time the center of the aiming reticle is located in the edge region of the display area. Specifically, the processor 63 measures the time when the aperture 23 of the mouse sensor 24 cannot be estimated to be blocked by the mounting surface, etc., based on the image clarity data 404, and measures the time when the center of the aiming reticle 250 is located in the edge region of the display area, based on the aiming coordinate data 408, and determines whether either of these times has reached a predetermined time (for example, 3 seconds). As shown in Figure 14, the display area of ​​the display 72 has a predetermined width of edge region set in advance. Note that in Figures 14 to 16, the edge region is depicted as visible for the sake of explanation, but in reality, the edge region does not need to be displayed. If the determination in step S204 is YES, the process returns to step S101 in Figure 10; if it is NO, the process returns to step S201.

[0085] In step S204, the processor 63 may determine whether the unoccluded time of the mouse sensor aperture has reached a first time (e.g., 3 seconds), or whether the time at which the center of the aiming reticle is located in the edge region of the display area has reached a second time (e.g., 2 seconds, 4 seconds) different from the first time. Also in step S204, if at least a portion of the unoccluded time of the mouse sensor aperture and the time at which the center of the aiming reticle is located in the edge region of the display area overlap, the processor 63 may consider these two times as a single unit of time with an overlapping portion, and determine whether this single unit of time has reached a predetermined time (e.g., 3 seconds). Alternatively, instead of measuring the time at which the center of the aiming reticle is located in the edge region of the display area, the processor 63 may measure, for example, the time at which at least a portion of the aiming reticle is located within the display area.

[0086] In step S301 of Figure 12, the processor 63 determines whether or not there was stick input based on the stick / button input data 406. If the determination in step S301 is YES, the process moves to step S305; if it is NO, the process moves to step S302.

[0087] In step S305, the processor 63 controls the aiming position based on the stick input. Specifically, the processor 63 moves the aiming position indicated by the aiming coordinate data 408 by a distance corresponding to the magnitude of the stick input in the direction indicated by the stick input. If, for example, the stick 22 is tilted in the positive x-axis direction as shown in Figure 14(1)(c) as a result of the processing in step S305, the aiming reticle 250 displayed on the display 72 moves to the right by a distance corresponding to the stick operation, as shown in Figure 14(1)(a). Also, similar to the aiming position control in step S205 in mouse mode processing, the movement of the aiming reticle 250 may be restricted so that the center does not move outside the display area (see Figure 14(2)). After that, the process returns to step S301.

[0088] In step S302, the processor 63 determines whether the first mouse mode transition condition has been met, similar to step S101 in Figure 10. In this way, in stick mode, the process transitions to mouse mode processing when at least the first mouse mode transition condition is met. Note that the stick mode processing in Figure 12 is just one example, and in other embodiments, the condition for transitioning from stick mode processing to mouse mode processing may be determined to be only when the first mouse mode transition condition described above has been met. If the determination in step S302 is YES, the process moves to mouse mode processing; if it is NO, the process moves to step S303.

[0089] In step S303, the processor 63 determines whether the gyro mode transition condition has been met, similar to step S203 in Figure 11. In this way, in stick mode, the process transitions to gyro mode processing when at least the gyro mode transition condition is met. Note that the stick mode processing in Figure 12 is just one example, and in other embodiments, the condition for transitioning from stick mode processing to gyro mode processing may be determined to be only when the above-mentioned gyro mode transition condition has been met. If the determination in step S303 is YES, the process moves to gyro mode processing; if NO, the process moves to step S304.

[0090] In step S304, the processor 63 determines a condition for estimating whether the user intends to continue operating the stick. In this embodiment, as an example, the processor 63 determines, based on the stick / button input data 406 and the aiming coordinate data 408, whether a predetermined time has been reached during which there has been no stick input or the center of the aiming point is located in the edge region of the display area. If the determination in step S304 is YES, the process returns to step S101 in Figure 10; if it is NO, the process returns to step S301.

[0091] In step S304, the processor 63 may determine whether the time without stick input has reached a first time (e.g., 3 seconds), or whether the time during which the center of the reticle was located at the edge of the display range has reached a second time (e.g., 2 seconds, 4 seconds) that is different from the first time. Also in step S304, if at least a portion of the time without stick input and the time during which the center of the reticle was located at the edge of the display range overlap, the processor 63 may consider these two times as a single unit of time with an overlapping portion, and determine whether this unit of time has reached a predetermined time (e.g., 3 seconds). Alternatively, instead of measuring the time during which the center of the reticle was located at the edge of the display range, the processor 63 may measure, for example, the time during which at least a portion of the reticle was located within the display range.

[0092] In step S401 of Figure 13, the processor 63 performs aiming position control based on the inertial sensor data 407. Specifically, the processor 63 performs control to display the aiming reticle 250 at a position corresponding to the attitude of the right controller 3, based on the correspondence between the attitude of the right controller 3 indicated by the inertial sensor data 407 and the aiming position indicated by the aiming coordinate data 408 (sometimes called the "attitude aiming relationship").

[0093] An example of the above control will be explained using Figures 15 and 16. For example, when the orientation of the right controller 3 is as shown in Figure 15(1)(b), the aiming reticle 250 is displayed, for example, at the center of the screen, as shown in Figure 15(1)(a). Then, for example, when the orientation of the right controller 3 becomes as shown in Figure 15(2)(b) (that is, an orientation rotated to the right in the positive z-axis direction from the orientation shown in Figure 15(1)(b)), the aiming reticle 250 is displayed at the position shown in Figure 15(2)(a) (that is, a position to the right of the position shown in Figure 15(1)(a)). Then, for example, when the orientation of the right controller 3 becomes as shown in Figure 16(b) (that is, an orientation rotated further to the right in the positive z-axis direction from the orientation shown in Figure 15(2)(b)), the aiming reticle 250 becomes at the position shown in Figure 16(a) (that is, a position further to the right of the position shown in Figure 15(2)(a)). Therefore, as shown in Figure 16(b), in response to the rotation operation of the right controller 3 (i.e., the attitude change operation), the targeting reticle 250 may be located outside the display area, although it will not be displayed on the display 72. At this time, the process may be to actually move the targeting reticle 250 outside the display area, or it may simply be to calculate the coordinates corresponding to the targeting reticle 250. The same applies when the right controller 3 is in a different attitude, so a detailed explanation of that will be omitted. The process then proceeds to step S402.

[0094] In step S402, the processor 63 determines whether the conditions for transitioning to stick mode have been met, similar to step S202 in Figure 11. If the determination in step S402 is YES, the process moves to stick mode processing; if NO, the process moves to step S403.

[0095] In step S403, the processor 63 determines whether a second mouse mode transition condition has been met based on the mouse sensor data 403. For example, the second mouse mode transition condition is a condition that takes more time to determine than the first mouse mode transition condition. As an example, the second mouse mode transition condition is that the clarity indicated by the image clarity data 404 is above a predetermined level, which is estimated to be above a predetermined level, and that a second predetermined number of frames (e.g., 180 frames) or more have been consecutive in which the dy / dz, which is the distance traveled per frame time of the right controller 3, is above a predetermined level (e.g., 0.1 mm or more). Note that the "second predetermined number" mentioned above is greater than the "first predetermined number" in the processing of step S101 in Figure 10 and step S302 in Figure 12. In this way, in gyro mode, the system transitions to mouse mode processing when at least the second mouse mode transition condition is met. Note that the gyro mode processing shown in Figure 13 is just one example, and in other embodiments, the condition for transitioning from gyro mode processing to mouse mode processing may be determined solely by whether the second mouse mode transition condition described above is met.

[0096] In gyro mode, when operating the aiming reticle 250 by changing the orientation of the right controller 3, the user grips the right controller 3 and changes its orientation as shown in Figure 7, making it easy for the user to grip the right controller 3 tightly and block the opening 23 of the mouse sensor with their right middle finger or the like. In this case, the mode may transition from gyro mode to mouse mode against the user's intention. Therefore, in this embodiment, the determination of transitioning from gyro mode to mouse mode is made more difficult to detect than the determination of transitioning from stick mode to mouse mode. As an example of how to make this determination more difficult, the determination of transitioning from gyro mode to mouse mode is made to take a relatively long time, as described above. This suppresses the transition of the operation mode from gyro mode to mouse mode against the user's intention. Note that in stick mode, when operating the aiming reticle 250 by stick operation, the user often operates without gripping the right controller 3 tightly, making it unlikely for the user to block the opening 23 of the mouse sensor with their right middle finger or the like. If the result in step S403 is YES, the process moves to mouse mode processing; if it is NO, the process moves to step S404.

[0097] In step S404, the processor 63 determines, based on the aiming coordinate data 408, whether the time during which the entire aiming reticle 250 is outside the display range has reached a predetermined time (for example, 3 seconds). If the determination in step S404 is YES, the process returns to step S101 in Figure 10; otherwise, the process returns to step S401.

[0098] According to the embodiment described above, when there are at least three operating modes—a mouse mode in which aiming is controlled based on the output of a mouse sensor, a gyro mode in which aiming is controlled based on the output of an inertial sensor, and a stick mode in which aiming is controlled based on stick input—it is possible to transition from mouse mode to either gyro mode or stick mode. Therefore, the operating mode can be quickly transitioned from mouse mode to either gyro mode or stick mode according to the user's intention.

[0099] Furthermore, according to this embodiment, it is possible to transition from gyro mode to either mouse mode or stick mode. Therefore, the operation mode can be quickly transitioned from gyro mode to either mouse mode or stick mode according to the user's intention. Furthermore, according to this embodiment, it is possible to transition from stick mode to either mouse mode or gyro mode. Therefore, it is possible to quickly transition between the three operation modes according to the user's intention.

[0100] Furthermore, according to this embodiment, as explained with reference to Figure 11, if NO is determined in steps S201 and S202 and YES is determined in step S203, the system transitions from mouse mode to gyro mode. Also, in the state where YES is determined in step S201, even if the output of the inertial sensor is determined to be YES in step S203, the system does not transition from mouse mode to gyro mode. Thus, according to this embodiment, when the mouse is being operated in mouse mode, the mouse mode continues regardless of the output of the inertial sensor. be By detecting when the conditions for transitioning to gyro mode are met, for example, when a movement of the controller placed on the mounting surface is detected, it is possible to prevent unintentional activation of gyro mode during mouse operation.

[0101] Furthermore, according to this embodiment, if it is determined that no mouse operation, stick operation, or posture change operation has been performed, and predetermined conditions (see S204, S304, S404) are met, the gyro mode is set (see Figures 10 to 13). Here, since the controller can be shaken even in mouse mode and stick mode, in order to detect the user's intention to transition to gyro mode, it is necessary to detect, for example, a sufficiently large shake of the controller. On the other hand, if the control to transition to gyro mode is based on detecting a sufficiently large shake of the controller, there is a risk that it will be difficult to transition to gyro mode against the user's intention. Therefore, in this embodiment, as described above, when it is presumed that the user is not performing any operation, for example, when the output of the mouse sensor and inertial sensor does not indicate movement of the controller and no stick operation is performed, the gyro mode is set. This realizes operability that is in line with the user's intention. In addition, in Figures 10 to 13, for example, the control may be such that the determination processes in steps S204, S304, and S404 are not performed, and the system returns to step S101 if the determination in S203 is NO, if the determination in S303 is NO, and if the determination in S403 is NO.

[0102] Furthermore, according to this embodiment, a controller without a mouse sensor or inertial sensor can also be connected to the main unit 2 and used. When a controller without a mouse sensor is connected to the main unit 2 and used, the operation mode transitions between stick mode and gyro mode according to the process shown in Figures 10 to 13. Similarly, when a controller without an inertial sensor is connected to the main unit 2 and used, the operation mode transitions between mouse mode and stick mode according to the process shown in Figures 10 to 13. Thus, according to this embodiment, a controller without a mouse sensor or inertial sensor can be used. In addition, both a controller without a mouse sensor or inertial sensor and a controller with a mouse sensor, stick, and inertial sensor can be connected to the main unit 2, and for example, two users can operate their respective controllers to control aiming. Furthermore, three or more controllers can be connected to the main unit 2, and each controller can be operated to control aiming. In this embodiment, an operation mode is set for each controller.

[0103] [Differentiation] In this embodiment described above, the color and shape of the aiming reticle 250 may change depending on the operating mode. Even in such cases, it can be considered to be substantially the same aiming reticle.

[0104] Furthermore, in the embodiment described above, the display position of the aiming reticle is not limited immediately after a transition in the operation mode. For example, the aiming reticle may be displayed in the same position as before the transition in the operation mode, in the center of the screen, or different depending on the operation mode after the transition. Note that in mouse mode and stick mode, the aiming reticle may move outside the display screen, and in gyro mode, the aiming reticle may not be able to move outside the display screen.

[0105] Furthermore, in the embodiment described above, the operations assigned to each button may be switched depending on the operation mode. For example, in mouse mode, a bullet may be fired in response to the operation of button A 12; in stick mode, a bullet may be fired in response to the operation of button ZR 21; and in gyro mode, a bullet may be fired in response to the operation of button R 20.

[0106] Furthermore, in the above-described embodiment, the game content played in the virtual space may differ depending on the operation mode. For example, the type of bullet fired may differ depending on the operation mode, or the range in which the player object controlled by the user can move may differ.

[0107] Furthermore, in the embodiment described above, an example was given in which the system transitions to the target operating mode in response to the device output used for aiming position control in the target operating mode. However, for example, the system may transition to the target operating mode in response to a device output different from the device output used for aiming position control in the target operating mode. For example, instead of transitioning to mouse mode in response to the mouse sensor output, the system may transition from gyro mode or stick mode to mouse mode in response to the inertial sensor output indicating for a predetermined time (e.g., 0.5 seconds) that the negative x-axis direction of the right controller coordinate system (see Figure 2) is facing the direction of gravity. In addition, the system may transition to multiple types of device outputs. For example, the system may transition to mouse mode based on the mouse sensor output and the inertial sensor output. For example, the system may transition to stick mode based on the stick output and the mouse sensor output. For example, the system may transition to gyro mode based on the mouse sensor output and the inertial sensor output.

[0108] Furthermore, in the embodiment described above, if multiple controllers are connected to the main unit 2, an operation mode may be set for each controller, and aiming operations and the like may be performed according to the operation on each controller.

[0109] Furthermore, the game processing described above, in which a bullet is fired by manipulating a reticle, is merely an example; for example, the game processing could involve selecting an object pointed to by a pointer such as a reticle. The type of game is also not limited; for example, it could be a competitive game, puzzle game, simulation game, music game, etc. Moreover, the game processing of this embodiment is not limited to a game played by a single user, but could be a game in which multiple users compete or cooperate. For example, the first user could move a character object, while the second user manipulates the reticle. For example, the movement of the character object by the first user could only be controlled using the stick of the controller used by the first user.

[0110] Furthermore, the above-described processes are not limited to game processing. For example, they may be applied to drafting applications, video editing applications, or operating systems. As one example, they may be applied to menu operations in an operating system. In the case of applying them to game processing, they may also be applied to menu operations within the game.

[0111] Furthermore, although the embodiment described above shows an example where one controller has one stick (see Figure 2, etc.), one controller may have two or more sticks. In this case, for example, the aiming reticle may be operated and the game may transition to stick mode in response to the operation of one stick (see this embodiment), while the aiming reticle may not be operated and the game may not transition to stick mode in response to the operation of the other stick. In this case, the other stick may be used to move the player object or move the aiming reticle.

[0112] Furthermore, the user may hold the right controller and the left controller in one hand and operate them accordingly. In this case, for example, either controller may be able to operate in three modes, or only one controller may be able to operate in three modes, or the modes may be divided between the two controllers. For example, the right controller may be used for mouse operation, and the left controller for gyro and stick operation. Alternatively, for example, the right controller may be used for mouse and gyro operation, and the left controller for mouse and stick operation. In other words, the division of modes includes cases where each controller supports some modes. When modes are divided in this way, a controller that cannot perform aiming operations in a given mode does not need to transition to that given mode even if there is a device input corresponding to that given mode. For example, if the right controller can perform mouse and gyro operations for aiming but cannot perform stick operations, the right controller does not need to transition to stick mode even if there is stick operation. Devices not used for aiming operations may be used for other game processing. For example, in the above example, the stick of the right controller may be used to move a virtual camera or player object.

[0113] Furthermore, in the embodiment described above, an example was given in which the attitude and movement of the controller are detected and processed based on the output of the gyro sensor in gyro mode (see S401, S404, etc. in Figure 13). However, in gyro mode, the attitude and movement of the controller may be detected and similar processing may be performed based on the output of, for example, an acceleration sensor or an optical sensor. In this case, it may be called by a different name than gyro mode or gyro mode processing. Furthermore, in the embodiment described above, an example was given in which processing is performed based on stick output (see S202 in Figure 11, S301, S304, S305 in Figure 12, S402 in Figure 13, etc.). However, similar processing may be performed based on button output, for example. In this case, it may be called by a different name than stick mode or stick mode processing.

[0114] Furthermore, in the embodiment described above, an example was given in which the game program executed on the main unit determines and sets the mode. However, another program executed on the main unit (for example, an operation program) may determine and set the mode, and the game program executed on the main unit may execute the game processing based on that determination and setting. Also, the operation program and the game program may share the processing of mode determination and setting, as well as the various processes that constitute the game processing. In addition, the controller may determine and set the mode.

[0115] Furthermore, in the embodiment described above, examples were given of mode transitions between mouse mode, gyro mode, and stick mode (see Figure 8). However, the possible transitions between modes are not limited to these. For example, mode transitions between gyro mode and stick mode do not have to be possible. For example, mode transitions from gyro mode and stick mode to mouse mode do not have to be possible. In this way, mode transitions from one mode to one or more other modes do not have to be possible.

[0116] Furthermore, the various data in this embodiment described above are merely examples, and other data converted to other forms may be used as appropriate in each process.

[0117] Furthermore, the game system is an example of an information processing system, and the information processing system may not be a system on which a game is executed. Also, the main unit may be a general-purpose personal computer. Similarly, the controller is just one example, and its shape is not limited, for example. The controller does not need to be detachable from the main unit. 。2 Only one of the controllers may have a mouse sensor. Only one of the two controllers may have a joystick. The controllers do not necessarily have to be in pairs.

[0118] At least a portion of the series of processes described above may be executed by the server-side device in an information processing system that includes a terminal-side device and a server-side device that can communicate via a network. The server may consist of multiple information processing devices, and the processing may be divided and executed by these multiple devices.

[0119] Although this embodiment and its variations have been described above, these descriptions are merely illustrative in every respect and are not intended to limit its scope. Furthermore, it goes without saying that various improvements and modifications can be made to this embodiment and its variations. [Explanation of Symbols]

[0120] 1. Information Processing System 2. Main unit 3, 4 Controllers 22, 42 sticks 24, 44 Mouse Sensors 63 processors 68, 69, 81, 94 Memory 72 displays 76, 77, 83, 84, 96, 97 Inertial Sensors Buttons 82 and 95 250 Aim

Claims

1. An information processing method implemented by a computer including one or more processors, A mode setting step in which one of several modes, including a first mode, a second mode, and a third mode, is set. Virtual objects, In the first mode, control is performed based on the output of a mouse sensor from at least one of the one or more controllers. In the second mode, control is performed based on the output of an inertial sensor of at least one of the one or more controllers. In the third mode, the system includes a virtual object control step which controls based on the output of a user-operated directional control unit, which is located in at least one of the one or more controllers. In the aforementioned mode setting step, When the mode is the first mode, the system transitions to the second mode based on the satisfaction of at least the first condition, and transitions to the third mode based on the satisfaction of at least the second condition. When the mode is the second mode, the system transitions to the first mode based on the fact that at least the third condition is met. When the mode is the third mode, the system transitions to the first mode based on the fact that at least the fourth condition is met. The third and fourth conditions each include conditions relating to the output of the mouse sensor, An information processing method wherein the time required to determine the third condition is longer than the time required to determine the fourth condition.

2. The information processing method according to claim 1, wherein in the mode setting step, when the mode is the second mode, the system transitions to the third mode based on the fact that the second condition is met.

3. The information processing method according to claim 2, wherein in the mode setting step, when the mode is the third mode, the transition to the second mode is made based on the fact that at least the first condition is met.

4. The first condition includes a condition relating to the output of the inertial sensor, In the first mode described above, If the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, then the system transitions to the second mode, at least the first condition is met. The information processing method according to claim 1, wherein if the output of the mouse sensor indicates movement of the controller, the system does not transition to the second mode even if the first condition is met.

5. The information processing method according to claim 1, wherein the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the direction control unit is not being operated, and the second mode is set.

6. The information processing method according to claim 1, wherein the mode setting step transitions between the third mode and one of the first mode or the second mode, and does not transition to the other mode, based on the output of a controller having the direction control unit and either the mouse sensor or the inertial sensor, but not the other.

7. The computer is made to execute the game process, The information processing method according to claim 1, wherein the virtual object is a pointer used in the game processing.

8. An information processing system comprising one or more controllers equipped with a mouse sensor, an inertial sensor, and a directional control unit operated by a user, and an information processing unit equipped with one or more processors, A mode setting means for setting one of a plurality of modes, including a first mode, a second mode, and a third mode, Virtual objects, In the first mode, control is performed based on the output of the mouse sensor. In the second mode, control is performed based on the output of the inertial sensor. In the third mode, the system includes virtual object control means that controls based on the output of the direction control unit, The mode setting means is When the mode is the first mode, the system transitions to the second mode based on the satisfaction of at least the first condition, and transitions to the third mode based on the satisfaction of at least the second condition. When the mode is the second mode, the system transitions to the first mode based on the fact that at least the third condition is met. When the mode is the third mode, the system transitions to the first mode based on the fact that at least the fourth condition is met. The third and fourth conditions each include conditions relating to the output of the mouse sensor, An information processing system in which the time required to determine the third condition is longer than the time required to determine the fourth condition.

9. The information processing system according to claim 8, wherein the mode setting means transitions to the third mode when the mode is the second mode, based on the fact that the second condition is met.

10. The information processing system according to claim 9, wherein the mode setting means transitions to the second mode when the mode is the third mode, based on the fact that at least the first condition is met.

11. The information processing system according to claim 8, wherein the second mode is set when the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the direction control unit is not being operated.

12. One or more processors, A mode setting step that allows setting one of several modes, including a first mode, a second mode, and a third mode, Virtual objects, In the first mode, control is performed based on the output of a mouse sensor possessed by at least one of the one or more controllers. In the second mode, control is performed based on the output of an inertial sensor of at least one of the one or more controllers. In the third mode, the system performs a virtual object control step in which it controls based on the output of a directional control unit of at least one of the one or more controllers, In the aforementioned mode setting step, When the mode is the first mode, the system transitions to the second mode based on the satisfaction of at least the first condition, and transitions to the third mode based on the satisfaction of at least the second condition. When the mode is the second mode, the system transitions to the first mode based on the fact that at least the third condition is met. When the mode is the third mode, the system transitions to the first mode based on the fact that at least the fourth condition is met. The third and fourth conditions each include conditions relating to the output of the mouse sensor, An information processing program in which the time required to determine the third condition is longer than the time required to determine the fourth condition.

13. The information processing program according to claim 12, wherein in the mode setting step, when the mode is the second mode, the program transitions to the third mode based on the fact that the second condition is met.

14. The information processing program according to claim 13, wherein in the mode setting step, when the mode is the third mode, the program transitions to the second mode based on the fact that at least the first condition is met.

15. The information processing program according to claim 12, which sets the second mode when the output of the mouse sensor does not indicate movement of the at least one controller having the mouse sensor, the output of the inertial sensor does not indicate movement of the controller, and the direction control unit is not being operated.

Images