Information processing method, information processing system, and information processing program

Abstract

To improve control of a collimation position.SOLUTION: An information processing method is configured to determine a position of a virtual object according to a posture of a controller, on the basis of a correspondence in which, when the controller is in a reference posture, the virtual object is positioned in a prescribed position in a display range, in a second mode for determining the position of the virtual object on the basis of output of an inertial sensor. When a first posture of the controller when the mode is switched from a first mode for determining the position of the virtual object on the basis of output of a mouse sensor or the like, to the second mode, satisfies a first condition, the reference posture is updated to a posture in which the position of the virtual object according to the first posture is within the display range.SELECTED DRAWING: Figure 11

Description

Technical Field

[0006] , ,

[0005] , , , ,

[0001] The present disclosure relates to information processing such as games.

Background Art

[0002] Conventionally, it is known to switch whether to operate the aiming position based on the operation of the operation unit or based on the acquired coordinates of the display screen. Also, it is known that the acquisition of coordinates used for the operation of the aiming position may use an acceleration sensor. Further, it is known that the acquired coordinates, the movement of the controller, and the operation state of the operation unit can be considered in this operation switching. For example, refer to paragraphs

[0146] ,

[0151] , and

[0155] of Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0007] (Configuration 2) Configuration 2, in Configuration 1 described above, may include the first condition that the position of the virtual object corresponding to the first pose is outside the display range.

[0008] (Composition 3) Configuration 3 may update the reference posture to the first posture if the first posture satisfies the first condition in configuration 1 or 2 described above.

[0009] (Composition 4) In configuration 4, the predetermined position in any of the above configurations 1 to 3 may be the center position of the display range.

[0010] (Composition 5) Configuration 5 may, in any of the above configurations 1 to 4, move the virtual object from the position of the virtual object in the first mode to the position of the virtual object in the second mode at a slower speed than when the first posture satisfies the first condition.

[0011] (Composition 6) Configuration 6 may, in any of the above configurations 1 to 5, position the virtual object within the display range when switching from the second mode to the first mode, if the virtual object's position is outside the display range.

[0012] (Composition 7) Configuration 7 may determine the position of the virtual object within the display range according to the position of the virtual object outside the display range when switching from the second mode to the first mode in Configuration 6.

[0013] (Composition 8) Configuration 8, in any of the above configurations 1 to 7, may update the position of the virtual object to a predetermined position in response to a button operation in the first mode and the second mode, and if the position of the virtual object is updated to a predetermined position, the reference posture may be updated to the posture of the controller at the time of the update.

[0014] (Composition 9) Configuration 9 may switch from the first mode to the second mode when, in any of the above configurations 1 to 8, the output of the inertial sensor satisfies the second condition.

[0015] (Composition 10) Configuration 10 may determine the position of the virtual object based on the output of the mouse sensor in any of the above configurations 1 to 9 when in the first mode.

[0016] (Composition 11) Configuration 11, in the above configuration 10, sets one of a plurality of modes, including a first mode, a second mode, and a third mode, in the mode setting step, determines the position of the virtual object based on the output of the direction control unit when the third mode is selected, and updates the reference posture to a posture in which the position of the virtual object corresponding to the first posture is within the display range when the first posture is selected when switching from the first or third mode to the second mode, if the first posture satisfies at least the first condition. [Effects of the Invention]

[0017] According to this embodiment, more appropriate control can be provided for virtual objects such as aiming.

Brief Description of Drawings

[0018] [Figure 1] Figure showing an example of a state where the right controller 3 and the left controller 4 are attached to the main body device 2 [Figure 2] Six-sided view showing an example of the right controller 3 [Figure 3] Six-sided view showing an example of the left controller 4 [Figure 4] Block diagram showing an example of the internal configuration of the main body device 2 [Figure 5] Block diagram showing an example of the internal configurations of the main body device 2, the right controller 3, and the left controller 4 [Figure 6] Figure showing an example of a state where the right controller 3 is held and operated by the right hand [Figure 7] Figure showing an example of a state where the right controller 3 is held and operated by the right hand [Figure 8] Figure for explaining the operation mode of the controller [Figure 9] Figure showing an example of the movement display of aiming [Figure 10] Figure showing an example of the movement display of aiming [Figure 11] Figure showing an example of the movement display of aiming [Figure 12] Figure for explaining the control of aiming when the operation mode is switched [Figure 13] Figure for explaining the control of aiming when the operation mode is switched [Figure 14] Figure showing an example of various data stored in the DRAM 69 [Figure 15] An example of a flowchart of information processing [Figure 16] An example of a flowchart of information processing [Figure 17] An example of a flowchart of information processing [Figure 18] An example of a flowchart of information processing [Modes for carrying out the invention]

[0019] One embodiment will be described below.

[0020] [Example of hardware configuration for an information processing system]

[0021] The following describes a game system, which is an example of the information processing system of this embodiment. An example of the game system 1 in this embodiment includes an information processing device (sometimes referred to as the "main unit") 2, a right controller 3, and a left controller 4. In this embodiment, the right controller 3 and the left controller 4 are detachable from the main unit 2.

[0022] Figure 1 shows an example of the main unit 2 with the right controller 3 and left controller 4 attached. As shown in Figure 1, the right controller 3 and left controller 4 are attached to the main unit 2 and integrated together. The main unit 2 is a device that performs various processes (e.g., game processing) in the game system 1. The main unit 2 is equipped with a display 72. The right controller 3 and left controller 4 are input devices equipped with operation parts for user input. In the following, the right controller 3 and left controller 4 may be collectively referred to as "controllers".

[0023] The display 72 displays images generated by the main unit 2. The display 72 is, for example, a liquid crystal display (LCD). The screen of the display 72 is equipped with a touch panel. The touch panel is, for example, a multi-touch input type (e.g., capacitive type).

[0024] Figure 2 is a hexagonal schematic diagram showing an example of a right controller 3. As shown in Figure 2, the right controller 3 is a vertically elongated plate shape, includes a housing 11, and has a front, rear, top, bottom, right, and left section. In the right controller 3, the rear is located opposite the front, the bottom is located opposite the top, and the left is located opposite the right section. The distance between the front and rear is greater than the distance between the top and bottom. The distance between the top and bottom is greater than the distance between the right and left sections. In other embodiments, the relative magnitudes of these distances may be different. In this embodiment, the direction connecting the bottom and top may be called the up-down direction, the direction perpendicular to the up-down direction and connecting the front and rear may be called the front-back direction, and the direction perpendicular to the up-down direction and the front-back direction and connecting the right and left sections may be called the left-right direction. In Figure 2, the x, y, and z axes are shown in a front view with the left side facing forward, indicating the coordinate system of the right controller 3 (sometimes referred to as the "right controller coordinate system"). In this coordinate system, the direction from the left side to the right side is the positive z-axis direction. The direction perpendicular to the z-axis and from the bottom to the top is the positive x-axis direction, and the direction perpendicular to the z and x axes and from the rear to the front is the positive y-axis direction. When the bottom side faces the direction of gravity, the negative x-axis direction and the direction of gravity coincide. In the description related to the right controller 3, the x, y, and z axes refer to the x, y, and z axes in the right controller coordinate system unless otherwise specified. In this embodiment, the front and bottom parts do not need to be perfectly flat and may have irregularities or slopes. For example, the bottom part includes the convex part 25 described later. The directions of each part and the directions connecting each part are approximate directions.

[0025] The right controller 3 has a protrusion 25 that fits into a recess (not shown) of the main unit 2 when it is mounted on the main unit 2. As shown in Figure 2, the protrusion 25 is a convex shape that protrudes in the negative x-axis direction, with a width shorter in the left-right direction than the left-right direction of the right controller 3 and a width shorter in the front-back direction than the front-back direction. In this embodiment, the protrusion 25 is part of the bottom.

[0026] As will be described later, the right controller 3 can also be held in a vertical orientation when detached from the main unit 2. When held in a vertical orientation, the right controller 3 is shaped and sized to be held with one hand, especially the right hand. The right controller 3 can also be held in a horizontal orientation, and when held in a horizontal orientation, it may be held with both hands (not shown).

[0027] The right controller 3 is equipped with an analog stick (sometimes simply called a "stick") 22 on its left side, which is an example of a directional input unit. The stick 22 can be used as a directional input unit that allows for directional input. The stick may also be called a directional control unit. The user can input direction by tilting the stick 22 in any direction, and the magnitude of the input can be determined according to the angle of tilt. The user can also input buttons by pressing the stick 22. The directional input unit may also be, for example, a directional pad or a slide pad. The directional input unit may also be called a directional control unit.

[0028] The right controller 3 has a set of four buttons on its left side: A button 12, B button 13, X button 14, Y button 15, a + (plus) button 16, and a home button 17. The right controller 3 has an R button 20 and a ZR button 21 extending across its front and top. The R button 20 and ZR button 21 may be located only on the front of the right controller 3, or only on the top. The right controller 3 has buttons 18 and 19 on the top surface 25a of the protrusion 25.

[0029] The right controller 3 is provided with an opening 23 for a mouse sensor on the top surface 25a of the protrusion 25. The opening 23 for the mouse sensor is an opening in the light guide path that guides light to the mouse sensor 24 located inside the right controller 3. The mouse sensor 24 is an optical mouse sensor and may include a light-emitting part and a light-receiving part. The light detected by the light-receiving part may be visible light or light of an invisible wavelength. The mouse sensor 24 may have at least a light-receiving part and not a light-emitting part. The mouse sensor 24 acquires data that allows for the calculation of the movement of the right controller 3 on the mounting surface, with the top surface 25a of the protrusion 25 of the bottom facing the mounting surface. In this way, the right controller 3 can also be used as a mouse. The operation of using it as a mouse is sometimes called "mouse operation". The mounting surface is not limited to a flat surface, but may be a curved surface, for example, the surface of the user's thigh.

[0030] Furthermore, in this embodiment, the right controller 3 is provided with a terminal 26 on the protrusion 25 for the right controller 3 to communicate with the main unit 2 via wired connection. As an example, the terminal 26 is provided on the inner circumferential surface of a recess provided on the top surface 25a of the protrusion 25.

[0031] Figure 3 is a hexagonal schematic diagram showing an example of the left controller 4. The same configuration as the right controller 3 will not be explained. The left controller 4 has a stick 42, a set of four buttons (right direction button 32, down direction button 33, up direction button 34, left direction button 35), a capture button 37, and a minus button 36 on its right side. Buttons 32-35 may be a single directional pad. Note that in the right controller 3, the stick 22 is located behind buttons 12-15, whereas in the left controller 4, the stick 42 is located in front of buttons 32-35. In Figure 3, the x, y, and z axes are shown relative to a front view with the right side facing forward, indicating the coordinate system of the left controller 4 (sometimes referred to as the "left controller coordinate system"). In this coordinate system, the direction from the right side to the left side is the positive z-axis direction. Furthermore, the direction perpendicular to the z-axis and moving from the bottom to the top is the positive x-axis direction, and the direction perpendicular to the z-axis and x-axis and moving from the rear to the front is the positive y-axis direction. When the bottom faces the direction of gravity, the negative x-axis direction and the direction of gravity coincide. Note that the x-axis, y-axis, and z-axis in the explanation related to the left controller 4 refer to the x-axis, y-axis, and z-axis in the left controller coordinate system unless otherwise specified.

[0032] The left controller 4 has a protrusion 45 that fits into a recess (not shown) of the main unit 2 when it is attached to the main unit 2. The protrusion 45, like the right controller 3, has buttons 38 and 39, an opening 43 for the mouse sensor, a mouse sensor 44, and a terminal 46.

[0033] When the left controller 4 is detached from the main unit 2, it can be held in either a vertical or horizontal orientation, similar to the right controller 3.

[0034] Figure 4 is a block diagram showing an example of the internal configuration of the main unit 2. The main unit 2 includes a processor 63. The processor 63 is an information processing unit that performs various information processing tasks performed in the main unit 2. The processor 63 may consist of, for example, multiple processors or cores, typically multiple CPUs (Central Processing Units) or cores, or it may consist of an SoC (System-on-a-chip) that includes multiple functions such as CPU function and GPU (Graphics Processing Unit) function. The processor 63 performs various information processing tasks by executing information processing programs (for example, game programs) stored in a storage unit (specifically, an internal storage medium such as flash memory 68, or an external storage medium installed in a slot 51, etc.). In this embodiment, "processor" may include at least a CPU, GPU, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc. In this embodiment, the computer, as an example, includes at least one processor and may further include a storage unit such as memory.

[0035] The main unit 2 includes a flash memory 68 and a DRAM (Dynamic Random Access Memory) 69 as examples of internal storage media. The flash memory 68 is a memory mainly used to store various types of data stored in the main unit 2. The DRAM 69 is a memory mainly used to temporarily store various types of data used in information processing. The processor 63 reads and writes data to and from storage media such as the flash memory 68 and the DRAM 69 as appropriate and performs various information processing.

[0036] Furthermore, the main unit 2 has various configurations as shown in Figure 4. These are briefly described below. The recording medium slot interface (sometimes referred to as "slot I / F") 52 reads and writes data to and from the storage medium (for example, a dedicated memory card) installed in the recording medium slot 51, according to instructions from the processor 63. The second slot I / F 54 reads and writes data to and from the storage medium installed in the second slot 53, according to instructions from the processor 63.

[0037] The network communication unit 66 communicates with external devices via the network (for example, internet communication using wireless communication). The controller communication unit 67 communicates wirelessly with the right controller 3 and / or the left controller 4 (for example, communication in accordance with the Bluetooth® standard).

[0038] The left terminal 50 is a terminal for wired communication between the processor 63 and the left controller 4. The right terminal 65 is a terminal for wired communication between the processor 63 and the right controller 3. The lower terminal 64 is a terminal for communication with other devices (e.g., a stationary monitor, etc.) via the cradle when the lower terminal 64 is mounted on the cradle.

[0039] The touch panel controller 70 generates data indicating, for example, the location where a touch input occurred, based on signals from the touch panel 71 located on the display surface of the display 72, and outputs this data to the processor 63. The display 72 displays images generated by the processor 63 and / or images acquired from an external source.

[0040] The codec circuit 74 controls the input and output of audio data to the speaker 73 and the audio input / output terminal 75.

[0041] The power control unit 61 controls the power supply from the battery 62 to each part of the main unit 2 (i.e., each part that receives power from the battery 62) based on commands from the processor 63, and also starts or stops the power supply in response to the pressing of the power button 60.

[0042] The volume button 59 is used to control the volume output from the speaker 73, etc. The cooling fan 58 is a fan that cools the inside of the main unit 2.

[0043] The main unit 2 is equipped with various sensors, including a magnetic force sensor 55, an ambient light sensor 56, a temperature sensor 57, an acceleration sensor 76, and an angular velocity sensor 77. The processor 63 can perform various processes based on information from these sensors.

[0044] Figure 5 is a block diagram showing an example of the internal configuration of the main unit 2, the right controller 3, and the left controller 4. Note that the details of the internal configuration of the main unit 2 are shown in Figure 4 and are therefore omitted in Figure 5.

[0045] The left controller 4 includes a communication control unit 80 that communicates with the main unit 2. As shown in Figure 5, the communication control unit 80 is connected to each component, including terminals 88. When the left controller 4 is attached to the main unit 2, the communication control unit 80 communicates with the main unit 2 via wired communication through terminals 88. When the left controller 4 is detached from the main unit 2, the communication control unit 80 communicates with the main unit 2 wirelessly (specifically, communication in accordance with the Bluetooth® standard).

[0046] The left controller 4 includes a memory 81, such as flash memory. The communication control unit 80 is composed of a processor, such as a microcontroller (also called a microcomputer), and performs various processes by executing firmware stored in the memory 81.

[0047] The left controller 4 is equipped with buttons 82 (specifically, buttons 32-34, etc.) and a stick 42. Each button 82 and stick 42 outputs information about the operation performed on it to the communication control unit 80.

[0048] The left controller 4 is equipped with inertial sensors. Specifically, the left controller 4 is equipped with an acceleration sensor 83 and an angular velocity sensor 84 as inertial sensors. The acceleration sensor 83 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in Figure 3). Note that the acceleration sensor 83 may also detect acceleration in one axis direction or two axis directions. The angular velocity sensor 84 detects angular velocity around the three predetermined axes. Note that the angular velocity sensor 84 may also detect angular velocity around one axis or two axes. The angular velocity sensor may also be called a "gyro sensor". The acceleration sensor 83 and the angular velocity sensor 84 are each connected to the communication control unit 80. The detection results of the acceleration sensor 83 and the angular velocity sensor 84 are repeatedly output to the communication control unit 80 at appropriate timings. Note that the right controller 3 and the left controller 4 may be equipped with either an acceleration sensor or an angular velocity sensor as inertial sensors, or they may be equipped with other sensors.

[0049] The left controller 4 is equipped with a mouse sensor 44. The mouse sensor 44 acquires data for calculating the movement of the left controller 4 placed on the mounting surface. The data acquired by the mouse sensor 44 is repeatedly output to the communication control unit 80 at appropriate intervals.

[0050] The communication control unit 80 acquires information about inputs (specifically, information about the operation of buttons and sticks, and detection results from sensors) from each input unit (specifically, each button 82, stick 42, and each sensor 83, 84, and 44). The communication control unit 80 transmits operation data, which includes the acquired information or information that has been processed in a predetermined manner, to the main unit 2. The operation data is transmitted repeatedly at a rate of once every predetermined time.

[0051] When the above operation data is transmitted to the main unit 2, the main unit 2 can obtain the input made to the left controller 4. That is, the main unit 2 can determine the operation of each button 82 and stick 42 based on the operation data. In addition, the main unit 2 can calculate information about the movement and / or posture of the left controller 4 based on the operation data (specifically, the detection results of the acceleration sensor 83 and / or angular velocity sensor 84). Furthermore, the main unit 2 can calculate information about mouse operations made to the left controller 4 based on the operation data (specifically, the detection results of the mouse sensor 44).

[0052] The left controller 4 includes an amplifier 85 and a vibrator 86. The amplifier 85 amplifies the control signal received from the communication control unit 80 and generates a drive signal. The vibrator 86 vibrates in response to the drive signal generated by the amplifier 85, causing the left controller 4 to vibrate.

[0053] The left controller 4 includes a power supply unit 87. The power supply unit 87 has a battery and a power control circuit. The power control circuit is connected to the battery and supplies power to each part of the left controller 4 (specifically, each part that receives power from the battery).

[0054] As shown in Figure 5, the right controller 3 is composed of a processor and other components and includes a communication control unit 91 that communicates with the main unit 2. The right controller 3 also includes a memory 94 connected to the communication control unit 91. The communication control unit 91 is connected to each component, including the terminal 92. The communication control unit 91 and the memory 94 have the same functions as the communication control unit 80 and memory 81 of the left controller 4. Therefore, the communication control unit 91 can communicate with the main unit 2 both by wired communication via the terminal 92 and by wireless communication without using the terminal 92, and controls the communication that the right controller 3 makes to the main unit 2.

[0055] The right controller 3 is equipped with input ports in the same way as the left controller 4. Specifically, it is equipped with buttons 95 (A button 12, B button 13, X button 14, Y button 15, etc.), a stick 22, inertial sensors (accelerometer 96 and angular velocity sensor 97), and a mouse sensor 24. Each of these input ports has the same function and operates in the same way as the input ports of the left controller 4.

[0056] The right controller 3 comprises an amplifier 98, a vibrator 99, and a power supply unit 100. The amplifier 98, vibrator 99, and power supply unit 100 have the same functions and operate in the same manner as the amplifier 85, vibrator 86, and power supply unit 87 of the left controller 4, respectively.

[0057] The right controller 3 includes a processing unit 90 and an NFC antenna 93. The processing unit 90 controls the NFC antenna 93 in response to commands from the main unit 2 via the communication control unit 91. The NFC antenna 93 performs short-range wireless communication based on the NFC (Near Field Communication) standard.

[0058] [Regarding the gripping configuration of the controller] Figure 6 is a schematic diagram showing an example of a state in which a user grasps the right controller 3 with their right hand, places it on a mounting surface, and uses it as a mouse, i.e., a state of mouse operation. As shown in Figure 6, from the user's perspective, the front of the right controller 3 faces forward and the left side faces left. The user's right palm covers the upper side of the right controller 3. The user's right thumb is positioned on the left side of the right controller 3. The user's right thumb is placed on, for example, the A button 12. The user's right index finger is placed on, for example, the R button 20, and the user's right middle finger is placed on, for example, the ZR button 21. The user can operate the R button 20 and ZR button 21 with their right index finger or middle finger. The user can operate each input unit located on the left side with their right thumb. Note that when the user uses the left controller 4 as a mouse with their left hand, they can grasp the left controller 4 with their left hand and use it on the mounting surface in the same manner. At this time, the right side of the left controller 4 faces to the right.

[0059] Figure 7 is a schematic diagram showing an example of a user holding the right controller 3 in their right hand and operating it in the air. As shown in Figure 7, when the right controller 3 is detached from the main unit 2, it can be held in the air so that its longitudinal direction is in the vertical or horizontal direction for the user. The user can operate the stick 22 (sometimes referred to as "stick operation") with, for example, their right thumb. The user can also perform operations such as shaking the held right controller 3 (sometimes referred to as "shaking operation") or changing its posture (sometimes referred to as "posture change operation"). The same procedure can be used when the user uses the left controller 4, which is detached from the main unit 2, with their left hand.

[0060] [Overview of the process in this embodiment] The processing outline of this embodiment will be described in detail below with reference to Figures 8 to 13. In the following description, the case in which the right controller 3 is used will be used as an example. In the game of this embodiment, for example, a virtual object called a crosshair is displayed, the crosshair moves in response to the operation, and a bullet can be fired in the direction of the crosshair according to a predetermined operation (for example, a predetermined button operation). The same considerations can be applied when the left controller 4 is used, so the explanation for that case will be omitted.

[0061] [About the controller's operating modes] Figure 8 is a diagram illustrating the controller's operating modes and the transitions between them. As shown in Figure 8, the controller's operating modes (sometimes simply referred to as "modes") include "mouse mode," "gyro mode," and "stick mode."

[0062] In this embodiment, in mouse mode, if a swing operation of the right controller 3 with a predetermined strength or greater (for example, a swing operation of 0.2G or more; hereinafter sometimes simply referred to as "swing operation") is detected by the inertial sensor, the system transitions to gyro mode. In mouse mode, if a stick operation of the stick 22 is detected, the system transitions to stick mode. In gyro mode, if a mouse operation of the right controller 3 is detected, the system transitions to mouse mode. In gyro mode, if a stick operation of the stick 22 is detected, the system transitions to stick mode. In stick mode, if a swing operation of the right controller 3 with a predetermined strength or greater (for example, a swing operation of 0.2G or more) is detected by the inertial sensor, the system transitions to gyro mode. In stick mode, if a mouse operation of the right controller 3 is detected, the system transitions to mouse mode. Note that the conditions for transitioning between operation modes are not limited to these, and other conditions may be used, or other conditions may be added.

[0063] Figure 9 is a diagram illustrating the control of the aiming reticle 250 in mouse mode and stick mode. First, with reference to Figure 9, the control of the aiming reticle 250 in mouse mode will be explained. Mouse mode is an operating mode in which the aiming reticle 250 is controlled based on the output of the mouse sensor.

[0064] [Regarding aiming control in each operating mode] As shown in Figure 9(1)(b), when a mouse operation is performed to move the right controller 3, which is placed on the mounting surface, in the positive z-axis direction of the right controller coordinate system, the aiming reticle 250 displayed on the display 72 (see position A) moves to the right by a distance corresponding to the mouse operation (see position B), as shown in Figure 9(1)(a). Similarly, although not shown, if the right controller 3 is operated on the mounting surface in another direction using the mouse, the aiming reticle 250 displayed on the display 72 moves in a direction and by a distance corresponding to that mouse operation.

[0065] However, in mouse mode, the aiming reticle 250 may be restricted so that its center does not move outside the display area of ​​the display 72 (sometimes simply referred to as the "display area"). For example, as shown in Figure 9(2)(a), with the center of the aiming reticle 250 located at the right edge of the display area, even if a mouse operation is performed where the right controller 3 mounted on the mounting surface moves in the positive z-axis direction of the right controller coordinate system, as shown in Figure 9(2)(b), the aiming reticle 250 is controlled so that it does not move any further to the right. As a result, a portion of the aiming reticle 250 is always displayed on the display 72.

[0066] Next, with reference to Figure 9, the control of the aiming reticle 250 in stick mode will be explained. Stick mode is an operating mode in which the aiming reticle 250 is controlled based on the output from the stick operation.

[0067] For example, as shown in Figure 9(1)(c), when the stick 22 is tilted in the positive x-axis direction of the right controller coordinate system, the reticle 250 displayed on the display 72 (see position A) moves to the right by a distance corresponding to the stick operation (see position B). Similarly, when the stick is operated in other directions (not shown), the reticle 250 displayed on the display 72 moves in a direction and by a distance corresponding to the stick operation. In addition, in stick mode, as in mouse mode, the movement of the reticle 250 may be restricted so that the center of the reticle 250 does not move outside the display area (see Figure 9(2)(a)). Note that the movement restriction of the reticle 250 in mouse mode and stick mode is not limited to these examples; for example, the movement may be restricted so that the entire reticle 250 is located within the display area, or a predetermined percentage (e.g., 1 / 4, 3 / 4) of the size of the reticle 250 in the vertical or horizontal direction is located within the display area.

[0068] Figures 10 and 11 illustrate the control of the targeting reticle 250 in gyro mode. Gyro mode is an operating mode in which the targeting reticle 250 is controlled based on the output of the inertial sensor. In gyro mode, the attitude of the right controller 3, indicated by the output of the inertial sensor, is associated with the display position of the targeting reticle 250, and the display position of the targeting reticle 250 is controlled according to the attitude of the right controller 3. This will be explained in detail below.

[0069] In this embodiment, the orientation of the right controller 3 when the aiming reticle 250 is displayed in the center of the display 72 is called the "reference orientation." The correspondence between the right controller 3 being in the reference orientation when the aiming reticle 250 is displayed in the center of the display 72 is called the "aiming orientation correspondence." Based on the aiming orientation correspondence, the display position of the aiming reticle 250 is controlled to move in accordance with the change in the orientation of the right controller 3. As will be described later using Figure 12, etc., the reference orientation may be reset.

[0070] For example, consider the case where the orientation of the right controller 3 (i.e., the reference orientation) when the aiming reticle 250 is displayed at the center position A of the display 72, as shown in Figure 10(1)(a), is the orientation shown in Figure 10(1)(b). In this case, for example, when the orientation of the right controller 3 becomes the orientation shown in Figure 10(2)(b) (i.e., the orientation rotated clockwise in the positive z-axis direction from the orientation shown in Figure 10(1)(b)), the aiming reticle 250 is displayed at the position shown in Figure 10(2)(a) (i.e., position B, which is to the right of position A shown in Figure 10(1)(a)). Then, for example, when the orientation of the right controller 3 becomes the orientation shown in Figure 11(b) (i.e., the orientation rotated further clockwise in the positive z-axis direction from the orientation shown in Figure 10(2)(b)), the aiming reticle 250 becomes the position shown in Figure 11(a) (i.e., position C, which is further to the right of position B shown in Figure 10(2)(a)). Therefore, as shown in Figure 11(b), in response to the rotation of the right controller 3, the target reticle 250 may be located outside the display area, although it will not be displayed on the display 72. In this case, the system may either actually move the target reticle 250 outside the display area, or it may simply calculate the coordinates corresponding to the target reticle 250. The same applies when the right controller 3 is in a different position, so a detailed explanation of that is omitted.

[0071] As described above, in gyro mode, the aiming reticle 250 is not restricted to being located outside the display area. In gyro mode, where the aiming reticle 250 moves according to the orientation of the right controller 3, restricting the aiming reticle 250 to being located outside the display area, as in mouse mode, could cause a significant change in the relationship between the orientation of the right controller 3 and the display position of the aiming reticle 250, potentially making it difficult for the user to operate. For this reason, in gyro mode, the control is not restricted to being located outside the display area.

[0072] [Regarding aiming control when switching operating modes] Figure 12 is a diagram illustrating the control of the aiming reticle 250 when transitioning from mouse mode or stick mode to gyro mode. Figure 13 is a diagram illustrating the control of the aiming reticle 250 when transitioning from gyro mode to mouse mode or stick mode. Note that the operation mode that is mouse mode or stick mode is sometimes referred to as "mouse / stick mode". In Figures 12 and 13, the calculated aiming position for displaying the aiming reticle 250 is indicated by the symbol 260. The center position of the aiming reticle 250 displayed on the display 72 is sometimes referred to as the "display aiming position", and the above calculated aiming position is sometimes referred to as the "target aiming position".

[0073] First, referring to Figure 12, we will explain the control of the aiming reticle 250 when transitioning from mouse / stick mode to gyro mode. Consider the case where a shaking operation is performed in mouse / stick mode (see Figure 12(1)) and the system transitions to gyro mode. In this case, if the target aiming position 260 corresponding to the attitude of the right controller 3 at the time of transition to gyro mode is outside the display range (see Figure 12(2-1)), the reference attitude of the right controller 3 is updated to the attitude at the time of transition to gyro mode, and the target aiming position 260 and the displayed aiming position are updated. PlaceThe reticle is reset to the center of the display range (see Figure 12(3-1)). In this way, when transitioning to gyro mode, if the aiming position corresponding to the attitude of controller 3 satisfies predetermined conditions, for example, if it is outside the display range, the aiming reticle 250 is displayed within the display range, thereby preventing user confusion due to losing sight of the aiming reticle 250. As another example, when the target aiming position at the time of transitioning to gyro mode is in a predetermined area within the display range, or when it is more than a predetermined distance away from the current displayed aiming position, the aiming reticle 250 may be displayed at a position within the display range, for example, a predetermined position. In this embodiment, since the aiming reticle 250 is displayed in the center of the display range (see Figure 12(3-1)), the user is prevented from losing sight of the aiming reticle 250. In addition, in this embodiment, the aiming reticle 250 moves instantaneously (including not only when moving at high speed, but also when moving instantaneously without intermediate steps) and is displayed, so the user can start operation comfortably. The position and speed of movement of the aiming reticle 250 after movement are not limited to the above.

[0074] In this embodiment, when transitioning to gyro mode, for example, if the target aiming position is outside the display range and the aiming reticle 250 is displayed in the center of the display range, the reference attitude is updated to the attitude of the controller 3 at the time of transitioning to gyro mode. This makes it easier for the user to operate the aiming reticle 250 in gyro mode thereafter. Note that such updating of the reference attitude is not required, and an attitude other than the attitude of the controller 3 at the time of transitioning to gyro mode, such as a predetermined attitude, may be updated as the reference attitude.

[0075] This section describes the case where, in mouse / stick mode, a shaking operation is performed to transition to gyro mode, and the target aiming position 260 corresponding to the orientation of the right controller 3 at the time of transition to gyro mode is within the display range (see Figure 12(2-2)). In this case, the aiming reticle 250 is moved and displayed toward the target aiming position 260 (see Figure 12(3-2)). The movement speed of the aiming reticle 250 in this case may be, for example, a speed that is visible to the user, and may be slower than the case where the aiming reticle 250 is instantly displayed within the display range when it is outside the display range (see Figure 12(3-1)). The aiming reticle 250 may also move while interpolating its position between its position before movement (see Figure 12(2-2)) and its position after movement (see Figure 12(3-2)). By moving the aiming reticle 250 in this way, it is possible to prevent the user from losing sight of the aiming reticle 250. Note that the aiming reticle 250 in Figures 12 (2-1) and (2-2) is the same as the aiming reticle 250 in mouse / stick mode. Also, the target aiming position 260 in Figures 12 (3-1) and (2-2) can be considered the effective initial aiming position in gyro mode.

[0076] Next, with reference to Figure 13, the control of the aiming reticle 250 when transitioning from gyro mode to mouse / stick mode will be explained. As shown in Figure 13(1), consider the case where, as an example, mouse operation or stick operation is performed in gyro mode, and the system transitions to mouse / stick mode. In this case, if the target aiming position 260 corresponding to the attitude of the right controller 3 at the time of transitioning to mouse / stick mode is outside the display range (see Figure 13(2-1)), the target aiming position 260 and the displayed aiming position are set to positions within the display range. In this embodiment, the displayed aiming position is, as an example, the position within the display range that is closest to the target aiming position 260 (see Figure 13(3-1)).

[0077] For example, consider the case where the display screen of display 72 is composed of 450 horizontal x 250 vertical dots, the lower left corner of the display range is the origin o(0,0), and the xy coordinates of the display range are in the range from (0,0) to (450,250). In this embodiment, for example, if the target aiming position 260 is at a coordinate outside the display range (600,100), the target aiming position 260 and the displayed aiming position are set to the nearest coordinate within the display range (450,100). For example, if the target aiming position 260 is at a coordinate outside the display range (100,350), the target aiming position 260 and the displayed aiming position are set to the nearest coordinate within the display range (100,250). For example, if the target aiming position 260 is at a coordinate outside the display range (600,350), the target aiming position 260 and the displayed aiming position are set to the nearest coordinate within the display range (450,250). In other words, the x and y coordinates of the target aiming position 260 and the displayed aiming position are converted to the nearest x and y coordinates within the display range, respectively.

[0078] In this way, when switching to mouse / stick mode, the Aim250 reticle is displayed within the display range, preventing user confusion when the Aim250 reticle disappears. Furthermore, since the Aim250 reticle is displayed at the nearest position within the display range, users who were previously operating in gyro mode can continue operating without any discomfort.

[0079] In this embodiment, when mouse or stick operation is performed in gyro mode and the system transitions to mouse / stick mode, if the target aiming position 260 corresponding to the orientation of the right controller 3 at the time of transition to mouse / stick mode is within the display range (see Figure 13(2-2)), the target aiming position 260 is set to the displayed aiming position (see Figure 13(3-2)).

[0080] [Details of the information processing in this embodiment] Next, the information processing of this embodiment will be described in detail with reference to Figures 14 to 18. The following explanation will use the case where the right controller 3 is used as an example. Note that the same considerations apply when the left controller 4 is used, so that explanation will be omitted.

[0081] [About the data used] Next, we will explain the various types of data stored in the DRAM 69. Figure 14 shows an example of data stored in the DRAM 69 of the main unit 2. As shown in Figure 14, the DRAM 69 is provided with at least a program storage area 301 and a data storage area 302.

[0082] The program storage area 301 stores at least the program 401. The data storage area 302 stores at least the operation mode data 402, mouse sensor data 403, stick / button input data 404, inertial sensor data 405, target aiming position data 406, display aiming position data 407, reference attitude data 408, aiming attitude correspondence data 409, supplementary flag data 410, object data 411, image data 412, and virtual camera control data 413.

[0083] Program 401 is a game program for executing game processing.

[0084] The operation mode data 402 is data indicating whether the operation mode is mouse mode, gyro mode, or stick mode, and includes the history of the operation mode up to a predetermined frame prior to that.

[0085] The mouse sensor data 403 is data relating to the output of the mouse sensor 24 and includes dy / dz data. The dy / dz data is output data of the mouse sensor 24 and, when the opening 23 of the mouse sensor 24 is blocked by the mounting surface, etc., it is data indicating the distance traveled per frame time in the y-axis and z-axis directions of the right controller coordinate system (i.e., the yz plane; see Figure 2) with respect to the mounting surface, etc. (sometimes referred to as "dy / dz"). Note that dy / dz may be calculated from the output data of the mouse sensor 24 by the communication control unit 91 or the processor 63, etc.

[0086] The stick / button input data 404 is data indicating the operations performed on the stick 22 and each button 95 of the right controller 3.

[0087] The inertial sensor data 405 is data output from the inertial sensor of the right controller 3, and for example, it is data that allows calculation of acceleration in the xyz axis direction (see Figure 2) and angular velocity around the xyz axis in the coordinate system of the right controller. Using the inertial sensor data, for example, the attitude and movement of the right controller 3 can be calculated.

[0088] The target aiming position data 406 is data that shows the target aiming position in a screen coordinate system obtained by converting the virtual space captured by the virtual camera into a planar coordinate system (see target aiming position 260 in Figures 12 and 13). The target aiming position moves based on dy / dz data in mouse mode, based on the operation of the stick 22 in stick mode, and based on the attitude change of the right controller 3 in gyro mode.

[0089] The display aiming position data 407 is data indicating the display position (i.e., the display aiming position) of the aiming reticle 250 in the screen coordinate system described above.

[0090] Reference attitude data 408 is data indicating the reference attitude, which is the attitude of the right controller 3 when the aiming reticle 250 is displayed in the center of the display 72.

[0091] The aiming attitude correspondence data 409 is data that shows the aiming attitude correspondence relationship, which is the correspondence relationship that becomes the reference attitude indicated by the reference attitude data 408 when the aiming reticle 250 is displayed in the center of the display 72.

[0092] The interpolation flag data 410 is flag data that indicates whether or not to perform an interpolation process (see Figure 12(3-2)) to move the aiming reticle 250 toward the target aiming position 260.

[0093] Object data 411 is data for virtual objects placed in the virtual space, such as bullets fired in the direction of the aiming reticle 250, player characters, opponent characters, and the ground.

[0094] Image data 412 includes images of the virtual object, the crosshair 250, as well as animation images, backgrounds, virtual effects, and other image data. The image of the crosshair 250 is placed in the screen coordinate system described above and displayed on the display 72, etc. Alternatively, instead of displaying the crosshair 250 on the display 72, etc. by placing it in the screen coordinate system, it may be placed in a virtual space and captured by a virtual camera to be displayed on the display 72, etc. In other words, the crosshair 250 may be represented as object data 411 instead of image data 412.

[0095] The virtual camera control data 413 is data for controlling a virtual camera that is placed in a virtual space and takes pictures of that virtual space.

[0096] In addition, various types of data used for drawing and other processes are stored in DRAM69 as needed.

[0097] [Example of detailed information processing] Next, the processing according to this embodiment will be described with reference to flowcharts, etc. Figures 15 to 18 are examples of flowcharts showing the processing according to this embodiment. In the following, mainly the processing characteristic of this embodiment will be described, and other explanations such as drawing processing will be omitted in principle. Furthermore, the following processing may be executed at predetermined intervals (for example, a processing frame interval where processing is performed every 1 / 60 second).

[0098] When the game processing begins, in step S101 of Figure 15, the processor 63 determines, based on the operation mode data 402, whether the current processing is in gyro mode. If the determination in step S101 is YES, the processing moves to step S102; if NO, the processing moves to step S104.

[0099] In step S102, the processor 63 determines, based on the operation mode data 402, whether the previous process was a process in gyro mode. If the determination in step S102 is YES, the process moves to step S103; otherwise, the process moves to the gyro mode transition process in step S200.

[0100] In step S103, the processor 63 controls the target aiming position based on the current attitude in gyro mode. Specifically, the processor 63 calculates the target aiming position corresponding to the current attitude of the right controller 3, which is calculated based on the inertial sensor data 405, based on the targeting attitude correspondence relationship shown in the targeting attitude correspondence relationship data 409 (see Figure 10, etc.). After that, the process moves to step S111 in Figure 16.

[0101] In step S200, the processor 63 executes gyro mode transition processing. Figure 17 is an example of a flowchart of gyro mode transition processing.

[0102] In step S201 of Figure 17, the processor 63 determines, based on the target aiming position data 406, whether the target aiming position based on the current attitude is within the display range. If the determination in step S201 is YES, the process moves to step S202; if NO, the process moves to step S203.

[0103] Step In step S202, the processor 63 sets the completion flag of the completion flag data 410 to ON. Then, the process moves to step S111 in Figure 16. stomach.

[0104] In step S203, the processor 63 sets the target aiming position indicated by the target aiming position data 406 and the display aiming position indicated by the display aiming position data 407 to the center of the display range (see (3-1) in Figure 12). The process then proceeds to step S204.

[0105] In step S204, the processor 63 sets the current attitude of the right controller 3, calculated based on the inertial sensor data 405, to the reference attitude indicated by the reference attitude data 408 (see (3-1) in Figure 12). In other words, the processor 63 updates the reference attitude to the current attitude. The process then proceeds to step S111 in Figure 16.

[0106] In step S104 of Figure 15, the processor 63 determines, based on the operation mode data 402, whether the previous process was in gyro mode. If the determination in step S104 is YES, the process moves to the mouse / stick mode transition process in step S300; otherwise, the process moves to step S105.

[0107] In step S300, the processor 63 executes mouse / stick mode transition processing. Figure 18 is an example of a flowchart of mouse / stick mode transition processing.

[0108] Figure 18 Step In S301, the processor 63 sets the completion flag of the completion flag data 410 to OFF. Then, the process moves to step S302.

[0109] In step S302, the processor 63 determines, based on the target aiming position data 406, whether the target aiming position is within the display range. If the determination in step S302 is YES, the process moves to step S303; if NO, the process moves to step S304.

[0110] In step S303, the processor 63 sets the display aiming position indicated by the display aiming position data 407 to the target aiming position indicated by the target aiming position data 406 (see (3-2) in Figure 13). The process then proceeds to step S111 in Figure 16.

[0111] In step S304, the processor 63 sets the display aiming position indicated by the display aiming position data 407 and the target aiming position indicated by the target aiming position data 406 to the nearest position within the display range (see (3-1) in Figure 12). The process then proceeds to step S111 in Figure 16.

[0112] In step S105 of Figure 15, the processor 63 controls the target aiming position based on the mouse sensor output or stick output (i.e., stick / button input data 404) in mouse mode or stick mode (see Figure 9). The process then proceeds to step S111 of Figure 16.

[0113] In step S111 of Figure 16, the processor 63 determines whether or not the reset button was pressed. Specifically, the processor 63 determines, based on the stick / button input data 404, whether or not, for example, the ZR button 21 of the right controller 3 was pressed. Note that the reset button may differ depending on the operation mode. If the determination in step S111 is YES, the process moves to step S112; if NO, the process moves to step S114.

[0114] In step S112, the processor 63 sets the target aiming position and the display aiming position to the center of the display range, similar to step S203 in Figure 17. The process then proceeds to step S113.

[0115] In step S113, the processor 63 sets the current posture to the reference posture, similar to step S204 in Figure 17. Then, the process moves to step S114.

[0116] In step S114, the processor 63 determines whether the completion flag indicated by the completion flag data 410 is ON or not. If the determination in step S114 is YES, the process moves to step S115; if NO, the process moves to step S119.

[0117] In step S115, the processor 63 determines, based on the target aiming position data 406 and the display aiming position data 407, whether the difference between the target aiming position and the display aiming position is greater than a predetermined value (e.g., 10 dots). If the determination in step S115 is YES, the process moves to step S116; if NO, the process moves to step S118. The process may also move to step S118 if the target aiming position is outside the display range.

[0118] In step S116, the processor 63 interpolates and updates the display aiming position indicated by the display aiming position data 407 to move closer to the target aiming position indicated by the target aiming position data 406 by a predetermined distance (for example, 10 dots) (see (3-2) in Figure 12). The process then proceeds to step S117.

[0119] In step S118, the processor 63 sets the completion flag of the completion flag data 410 to OFF. Then, the process moves to step S119.

[0120] In step S119, the processor 63 updates the display aiming position indicated by the display aiming position data 407 to the target aiming position indicated by the target aiming position data 406. The process then proceeds to step S117.

[0121] In step S117, the processor 63 displays the target 250 at the display target position indicated by the display target position data 407. After that, the process returns to step S101 in Figure 15.

[0122] According to this embodiment, in mouse / stick mode and gyro mode, the aiming position is reset in response to button operation, and the reference posture is reset in either mode (see S111-S113 in Figure 16). When the user resets by pressing a button in mouse / stick mode, the user's posture and / or grip on the controller may be in a comfortable position, and it is possible that the transition to gyro mode will not change significantly. Therefore, by resetting both the aiming position and the reference posture, it is possible that the user can smoothly begin posture change operations when transitioning to gyro mode.

[0123] Furthermore, in this embodiment, the controller's orientation when transitioning to gyro mode may deviate from the reference orientation because it satisfies the conditions for transitioning to gyro mode, potentially causing the aiming position to move significantly from the center of the display range. For example, such a situation may occur if the conditions for transitioning to gyro mode are at least one of the following: a swinging operation, a predetermined angle condition, an angular velocity condition, or an acceleration condition. According to this embodiment, even in such cases, the aiming reticle 250 is displayed within the display range, thus preventing the user from losing sight of the reticle.

[0124] Furthermore, in this embodiment, for example, if a posture change operation is performed in mid-air in gyro mode, then mouse operation is performed on a work surface such as a desk or the user's thigh, and then another posture change operation is performed in mid-air in gyro mode, the controller's posture in the first and second gyro modes may differ significantly. According to this embodiment, control is performed to reset the aiming position and reference posture (see (2-1)(3-1) in Figure 12, etc.), so it is possible to suppress the user from losing sight of the aiming reticle when repeatedly switching between operation in gyro mode and operation in mouse mode.

[0125] [Differentiation] Depending on the operating mode, the display characteristics of the virtual object, the crosshair 250, such as its color and shape, may change. Even in such cases, it can be considered to be substantially the same crosshair.

[0126] The virtual objects that can be controlled are not limited. For example, cursors such as arrow shapes, pointers, player objects operated by the user, and other virtual objects may be controlled.

[0127] In mouse / stick mode, similar to gyro mode (see Figure 11), the crosshair may be allowed to move outside the display range.

[0128] In the embodiment described above, an example having a mouse mode, a gyro mode, and a stick mode was given, but in other examples, the mouse mode may not be included, or the stick mode may not be included.

[0129] Furthermore, in the embodiment described above, an example was given in which the reference attitude is updated to the controller's current attitude when the condition is met that the target aiming position corresponding to the controller's attitude during gyro mode transition is outside the display range (see Figures 12 and 17). However, for example, the range of controller attitudes in which the target aiming position is outside the display range may be calculated in advance based on the reference attitude. Then, when the condition is met that the controller's attitude during gyro mode transition falls within the above-calculated range, the reference attitude may be updated to the controller's current attitude.

[0130] Furthermore, in the embodiment described above, an example was given in which, when the target aiming position corresponding to the controller's attitude during gyro mode transition is outside the display range, the aiming reticle is displayed in the center of the display range and the reference attitude is updated (see Figures 12 and 17). Since the target aiming position corresponds to the center of the aiming reticle, in this example, even if a part of the aiming reticle is displayed within the display range, if the target aiming position (i.e., the position corresponding to the center of the aiming reticle) is outside the display range, the aiming reticle will be displayed in the center of the display range and the reference attitude will be updated.Therefore, when the entire aiming reticle is outside the display range during gyro mode transition and the aiming reticle is not displayed at all, the control may be set to display the aiming reticle in the center of the display range and update the reference attitude.Also, when the entire aiming reticle is located within a predetermined edge area of ​​the display range (for example, an edge area with a width that can accommodate the entire or a part of the aiming reticle) and an area outside the display range during gyro mode transition, the control may be set to display the aiming reticle in the center of the display range and update the reference attitude. "A virtual object (e.g., a crosshair) is located within a certain range" can include both cases where the entire virtual object is located within a certain range and cases where only a part of the virtual object is located within a certain range. When transitioning to gyro mode, if the virtual object ends up located in the edge region of the display range, the control may be configured to display the virtual object in the center of the display range and update the reference orientation. When transitioning to gyro mode, if the virtual object does not end up located within a predetermined range that includes, for example, the center of the display range, the control may be configured to display the virtual object in the center of the display range and update the reference orientation.

[0131] If the target aiming position corresponding to the controller's attitude during a gyro mode transition is outside the display range, the control may be configured to display the aiming reticle so that its center is the closest position within the display range.

[0132] Furthermore, when transitioning to gyro mode, if the target aiming position is within the display range, the aiming reticle may be instantly moved to the target aiming position without performing interpolation updates (see (2-2) and (3-2) in Figure 12).

[0133] Furthermore, the various data in this embodiment described above are merely examples, and other data converted to other forms may be used as appropriate in each process.

[0134] Furthermore, the game system is an example of an information processing system, and the information processing system may be a system that does not run games. Also, the main unit may be a general-purpose personal computer.

[0135] The user may hold and operate the right and left controllers in one hand. In this case, for example, either controller may be able to operate in three modes, or only one controller may be able to operate in three modes, or the modes may be divided between the two controllers. For example, the right controller may be used for mouse operation and the left controller for gyro and stick operation. Alternatively, for example, the right controller may be used for mouse and gyro operation, and the left controller for mouse and stick operation. In other words, the division of modes includes cases where each controller supports some modes. When modes are divided in this way, a controller that cannot perform aiming operations in a given mode does not have to transition to that given mode even if there is a device input corresponding to that given mode. For example, if the right controller can perform mouse and gyro operations for aiming but cannot perform stick operations, the right controller does not have to transition to stick mode even if there is stick operation. Devices not used for aiming operations may be used for other game processing. For example, in the above example, the stick of the right controller may be used to move a virtual camera or player object.

[0136] The controller in this embodiment is an example, and its shape is not limited. The controller does not need to be detachable from the main unit. Neither of the two controllers has to have a mouse sensor. Only one of the two controllers has a mouse sensor. Neither of the two controllers has to have a stick. Only one of the two controllers has a stick. The controllers do not have to come in pairs. In this case, one controller does not need to have a mouse sensor or a stick. One controller may have two or more sticks. In this case, for example, the aiming reticle may be operated and the system may transition to stick mode in response to the operation of one stick, while the aiming reticle may not be operated and the system may not transition to stick mode in response to the operation of the other stick. In this case, the other stick may be used to move a virtual camera or player object.

[0137] The processing described above is not limited to game processing. For example, it may be applied to drafting applications, video editing applications, or operating systems. As one example, it may be applied to menu operations in an operating system. Furthermore, when applied to game processing, it may also be applied to menu operations within the game.

[0138] At least a portion of the series of processes described above may be executed by the server-side device in an information processing system that includes a terminal-side device and a server-side device that can communicate via a network. The server may consist of multiple information processing devices, and the processing may be divided and executed by these multiple devices.

[0139] Although this embodiment and its variations have been described above, these descriptions are merely illustrative in every respect and are not intended to limit its scope. Furthermore, it goes without saying that various improvements and modifications can be made to this embodiment and its variations. [Explanation of Symbols]

[0140] 1. Information Processing System 2. Main unit 3, 4 Controllers 22, 42 sticks 24, 44 Mouse Sensors 63 processors 68, 69, 81, 94 Memory 72 displays 76, 77, 83, 84, 96, 97 Inertial Sensors Buttons 82 and 95 250 Aiming (Displayed Aiming Position) 260 Target aiming position

Claims

1. An information processing method using a controller that includes at least one of a mouse sensor and a directional control unit operated by a user, and an inertial sensor, A mode setting step in which one of several modes, including a first mode and a second mode, is set. In the first mode, the position of the virtual object is determined based on the output of the direction control unit or the mouse sensor. In the second mode, the system includes a virtual object control step of determining the position of the virtual object based on the output of the inertial sensor, In the virtual object control step, In the second mode, the position of the virtual object is determined according to the orientation of the controller, based on the correspondence between the virtual object being located at a predetermined position within the display range when the controller is in a reference orientation. An information processing method that, when the controller's orientation when switching from the first mode to the second mode satisfies at least one first condition, updates the reference orientation to an orientation in which the position of the virtual object corresponding to the first orientation is within the display range.

2. The information processing method according to claim 1, wherein the first condition includes the position of the virtual object corresponding to the first posture being outside the display range.

3. The information processing method according to claim 2, wherein when the first posture satisfies the first condition, the reference posture is updated to the first posture.

4. The information processing method according to claim 3, wherein the predetermined position is the central position of the display range.

5. The information processing method according to claim 1, wherein when the first posture does not satisfy the first condition, the virtual object is moved at a slower speed from the position of the virtual object in the first mode to the position of the virtual object in the second mode than when the first posture satisfies the first condition.

6. The information processing method according to claim 1, wherein when switching from the second mode to the first mode, if the position of the virtual object is outside the display range, the virtual object is positioned within the display range.

7. The information processing method according to claim 6, wherein the position of the virtual object within the display range is determined according to the position of the virtual object outside the display range when switching from the second mode to the first mode.

8. In the first mode and the second mode, the position of the virtual object is updated to the predetermined position in response to the button operation. The information processing method according to claim 1, wherein when the position of the virtual object is updated to the predetermined position, the reference orientation is updated to the orientation of the controller at the time of the update.

9. The information processing method according to claim 1, wherein the system switches from the first mode to the second mode when at least the output of the inertial sensor satisfies the second condition.

10. The information processing method according to claim 1, wherein in the first mode, the position of the virtual object is determined based on the output of the mouse sensor.

11. In the mode setting step, one of the multiple modes, including the first mode, the second mode, and the third mode, is set. In the virtual object control step, In the third mode, the position of the virtual object is determined based on the output of the direction control unit. The information processing method according to claim 10, wherein when the first posture is switched from the first mode or the third mode to the second mode, the reference posture is updated to a posture in which the position of the virtual object corresponding to the first posture is within the display range, provided that the first posture satisfies at least the first condition.

12. An information processing system comprising a controller equipped with at least one of a mouse sensor and a directional control unit operated by a user, an inertial sensor, and an information processing unit, A mode setting means for setting one of several modes, including a first mode and a second mode, In the first mode, the position of the virtual object is determined based on the output of the direction control unit or the mouse sensor. In the second mode, the system includes virtual object control means that determines the position of the virtual object based on the output of the inertial sensor, The virtual object control means is In the second mode, the position of the virtual object is determined according to the orientation of the controller, based on the correspondence between the virtual object being located at a predetermined position within the display range when the controller is in a reference orientation. An information processing system that, when the controller's orientation when switching from the first mode to the second mode satisfies at least one first condition, updates the reference orientation to an orientation in which the position of the virtual object corresponding to the first orientation is within the display range.

13. The information processing system according to claim 12, wherein the first condition includes the position of the virtual object corresponding to the first posture being outside the display range.

14. The information processing system according to claim 13, wherein when the first posture satisfies the first condition, the reference posture is updated to the first posture.

15. The information processing system according to claim 14, wherein the predetermined position is the central position of the display range.

16. A processor in an information processing system comprising a controller equipped with a mouse sensor and at least one of a directional control unit operated by the user, an inertial sensor, and an information processing unit, A mode setting step that allows setting one of several modes, including a first mode and a second mode, In the first mode, the position of the virtual object is determined based on the output of the direction control unit or the mouse sensor. In the second mode, a virtual object control step is performed in which the position of the virtual object is determined based on the output of the inertial sensor. In the virtual object control step, In the second mode, the position of the virtual object is determined according to the orientation of the controller, based on the correspondence between the virtual object being located at a predetermined position within the display range when the controller is in a reference orientation. An information processing program that, when the controller's orientation when switching from the first mode to the second mode satisfies at least one first condition, updates the reference orientation to an orientation in which the position of the virtual object corresponding to the first orientation is within the display range.

17. The information processing program according to claim 16, wherein the first condition includes the position of the virtual object corresponding to the first posture being outside the display range.

18. The information processing program according to claim 17, which updates the reference posture to the first posture when the first posture satisfies the first condition.

19. The information processing program according to claim 18, wherein the predetermined position is the central position of the display range.

20. The information processing program according to claim 16, wherein the information processing program is a program that causes the processor to execute game processing.

Images