Information processing methods, information processing systems, computer-readable non-transitory recording media containing information processing programs, and computer program products containing information processing programs.

Abstract

This invention provides an information processing method, an information processing system, a computer-readable non-transitory recording medium containing an information processing program, and a computer program product containing the information processing program. It improves the control of the alignment position. In a second mode where the position of a virtual object is determined based on the output of an inertial sensor, the position of the virtual object is determined according to the controller's posture, based on a correspondence between the virtual object's position within the display range when the controller is in a reference posture. When switching from a first mode where the position of the virtual object is determined based on the output of a mouse sensor, etc., to the second mode, if the controller's first posture satisfies a first condition, the reference posture is updated to a posture that ensures the position of the virtual object corresponding to the first posture is within the display range.

Description

Technical Field

[0001] This disclosure relates to an information processing method. Background Technology

[0002] Previously known techniques involved switching between operating the aiming position based on the operation of the operating unit or operating the aiming position based on the coordinates obtained from the display screen. Additionally, it is known that an accelerometer can be used to obtain the coordinates used in the operation of the aiming position. Furthermore, it is known that techniques can take into account the acquisition of coordinates, the action of the controller, and the operating state of the operating unit during the switching of this operation. For example, see paragraphs

[0146] ,

[0151] , and

[0155] of Patent Document 1.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-90941 Summary of the Invention

[0006] The problem the invention aims to solve

[0007] In techniques like those described above, there is room for improvement in the control of the crosshair position.

[0008] Solution for solving the problem

[0009] For example, the following structural examples can be given.

[0010] (Structure 1)

[0011] Structure 1 is an information processing method that uses a controller, the controller having at least one of a mouse sensor and a directional control unit operated by a user, and an inertial sensor, wherein any one of a plurality of modes including a first mode and a second mode is set; in the first mode, the position of a virtual object is determined based on the output of the directional control unit or the mouse sensor; in the second mode, the position of the virtual object is determined based on the output of the inertial sensor; when the position of the virtual object is determined in the second mode, the position of the virtual object is determined according to the posture of the controller based on a correspondence between the virtual object's position within the display range when the controller is in a reference posture; when the controller's posture, i.e., the first posture, satisfies at least a first condition when switching from the first mode to the second mode, the reference posture is updated to a posture that makes the position of the virtual object corresponding to the first posture within the display range.

[0012] (Structure 2)

[0013] Structure 2 can be, in the above structure 1, the first condition includes the position of the virtual object corresponding to the first posture being outside the display range.

[0014] (Structure 3)

[0015] Structure 3 can be, in the above structure 1 or 2, in the information processing method, when the first pose satisfies the first condition, the reference pose is updated to the first pose.

[0016] (Structure 4)

[0017] Structure 4 can be any of the structures 1 to 3 above, where the specified position is the center of the display range.

[0018] (Structure 5)

[0019] Structure 5 can be any of the structures 1 to 4 above. In the information processing method, when the first posture does not meet the first condition, compared with when the first posture meets the first condition, the virtual object is moved slowly from the position of the virtual object in the first mode toward the position of the virtual object in the second mode.

[0020] (Structure 6)

[0021] Structure 6 can be any of the structures 1 to 5 above. In the information processing method, when switching from the second mode to the first mode, if the virtual object is outside the display range, the virtual object is placed within the display range.

[0022] (Structure 7)

[0023] Structure 7 can be, in the above structure 6, in the information processing method, determining the position of the virtual object within the display range based on the position of the virtual object outside the display range when switching from the second mode to the first mode.

[0024] (Structure 8)

[0025] Structure 8 can be any of the structures 1 to 7 above. In the information processing method, in the first mode and the second mode, the position of the virtual object is updated to a specified position according to the button operation. When the position of the virtual object is updated to the specified position, the reference pose is updated to the pose of the controller when the update is performed.

[0026] (Structure 9)

[0027] Structure 9 can be any of the structures 1 to 8 above, in the information processing method, when at least the output of the inertial sensor satisfies the second condition, switching from the first mode to the second mode.

[0028] (Structure 10)

[0029] Structure 10 can be any of the structures 1 to 9 above, in the information processing method, in the first mode, determining the position of the virtual object based on the output of the mouse sensor.

[0030] (Structure 11)

[0031] Structure 11 can be, in the above structure 10, in the information processing method, any mode among multiple modes including a first mode, a second mode and a third mode is set, when the position of the virtual object is determined in the third mode, the position of the virtual object is determined based on the output of the direction operation unit, and when the first posture when switching from the first mode or the third mode to the second mode at least satisfies the first condition, the reference posture is updated to a posture that makes the position of the virtual object corresponding to the first posture within the display range.

[0032] The effects of the invention

[0033] According to this embodiment, more appropriate control can be provided for virtual objects such as the crosshair. Attached Figure Description

[0034] Figure 1 This is a diagram showing an example of a state where the right controller 3 and the left controller 4 are installed in the main unit 2.

[0035] Figure 2 This is a six-sided view showing an example of the right controller 3.

[0036] Figure 3 This is a six-sided view showing an example of the left controller 4.

[0037] Figure 4 This is a block diagram showing an example of the internal structure of the main body device 2.

[0038] Figure 5 This is a block diagram showing an example of the internal structure of the main unit 2, the right controller 3, and the left controller 4.

[0039] Figure 6 This is a diagram illustrating an example of holding and operating the right controller 3 with the right hand.

[0040] Figure 7 This is a diagram illustrating an example of holding and operating the right controller 3 with the right hand.

[0041] Figure 8 This is a diagram used to illustrate the operating modes of the controller.

[0042] Figure 9 This is an example diagram showing the movement of the crosshair.

[0043] Figure 10 This is an example diagram showing the movement of the crosshair.

[0044] Figure 11 This is an example diagram showing the movement of the crosshair.

[0045] Figure 12 This diagram illustrates the control of the crosshair when switching operating modes.

[0046] Figure 13 This diagram illustrates the control of the crosshair when switching operating modes.

[0047] Figure 14 This is a diagram showing examples of various types of data stored in DRAM 69.

[0048] Figure 15 This is an example of an information processing flowchart.

[0049] Figure 16 This is an example of an information processing flowchart.

[0050] Figure 17 This is an example of an information processing flowchart.

[0051] Figure 18 This is an example of an information processing flowchart. Detailed Implementation

[0052] The following describes one implementation method.

[0053] [Examples of hardware architecture for information processing systems]

[0054] The following describes a game system as an example of the information processing system in this embodiment. An example of the game system 1 in this embodiment includes an information processing device (sometimes referred to as the "main device") 2, a right controller 3, and a left controller 4. The main device 2 in this embodiment can have the right controller 3 and the left controller 4 installed and removed respectively.

[0055] Figure 1 This diagram illustrates an example of a configuration where the main unit 2 is equipped with a right controller 3 and a left controller 4. (See diagram for example.) Figure 1 As shown, the right controller 3 and the left controller 4 are integrated and mounted on the main unit 2. The main unit 2 is a device that performs various processes (e.g., game processing) in the game system 1. The main unit 2 includes a display 72. The right controller 3 and the left controller 4 are input devices with operation units for user input. Furthermore, the right controller 3 and the left controller 4 are sometimes collectively referred to as "controllers" below.

[0056] The display 72 displays an image generated by the main device 2. For example, the display 72 is a liquid crystal display (LCD). A touch panel is provided on the screen of the display 72. For example, the touch panel is capable of multi-touch input (e.g., capacitive touch).

[0057] Figure 2 This is a six-sided schematic diagram showing an example of the right controller 3. For example... Figure 2 As shown, the right controller 3 is a long, plate-shaped component with a housing 11, comprising a front, rear, upper, bottom, right, and left portion. In the right controller 3, the rear portion is located opposite the front portion, the bottom portion is located opposite the upper portion, and the left portion is located opposite the right portion. The distance between the front and rear portions is greater than the distance between the upper and bottom portions. The distance between the upper and bottom portions is greater than the distance between the right and left portions. Furthermore, in other embodiments, these distances may be in other relative terms. Additionally, in this embodiment, the direction connecting the bottom and upper portions may be called the vertical direction, the direction perpendicular to the vertical direction and connecting the front and rear portions may be called the front-back direction, and the direction perpendicular to both the vertical and front-back directions and connecting the right and left portions may be called the left-right direction. Furthermore, in… Figure 2 The front view of the left side facing forward illustrates the x, y, and z axes, showing the coordinate system of the right controller 3 (sometimes referred to as the "right controller coordinate system"). In this coordinate system, the direction from left to right is the positive z-axis. Additionally, the direction perpendicular to the z-axis and from bottom to top is the positive x-axis, and the direction perpendicular to both the z-axis and x-axis and from rear to front is the positive y-axis. When the bottom faces the direction of gravity, the negative x-axis aligns with the direction of gravity. Furthermore, unless otherwise specified, the x-axis, y-axis, and z-axis in the description relating to the right controller 3 refer to the x-axis, y-axis, and z-axis in the right controller coordinate system. In this embodiment, the front, bottom, and other parts do not need to be completely flat; they may have irregularities or be inclined. For example, the bottom includes a protrusion 25, described later. The directions in which each part faces and the directions connecting the parts to each other represent general directions.

[0058] The right controller 3 has a protrusion 25 that, when mounted on the main unit 2, is inserted into a recess (not shown) of the main unit 2. For example... Figure 2 As shown, the protrusion 25 is a convex shape that protrudes in the negative x-axis direction, with a width shorter in the left-right direction than the width of the right controller 3 in the left-right direction and a width shorter in the front-back direction than the width of the right controller 3 in the front-back direction. In this embodiment, the protrusion 25 is part of the bottom.

[0059] The right controller 3 can also be held vertically while detached from the main unit 2, as will be described later. The right controller 3 is shaped and sized to be held with one hand, especially the right hand, when held vertically. In addition, the right controller 3 can also be held horizontally, and when held horizontally, it can also be held with both hands (not shown).

[0060] The right controller 3 has an analog joystick (sometimes simply called a "joystick") 22 on its left side, which serves as an example of a direction input unit. The joystick 22 can be used as a direction input unit for inputting direction. Alternatively, it can be referred to as a direction operation unit. The user can input direction corresponding to the tilting direction by tilting the joystick 22 in any direction, and can input the magnitude corresponding to the tilting angle. Additionally, the user can input buttons by pressing the joystick 22. The direction input unit can also be, for example, a directional pad or a slider. Alternatively, the direction input unit can be referred to as a direction operation unit.

[0061] The right controller 3 has four buttons on its left side as a group: A button 12, B button 13, X button 14, Y button 15, + (positive) button 16, and home button 17. The right controller 3 also has an R button 20 and a ZR button 21 distributed across its front and top. Alternatively, the R button 20 and ZR button 21 may be located only on the front of the right controller 3, or only on its top. The right controller 3 has buttons 18 and 19 on the top surface 25a of the protrusion 25.

[0062] The right controller 3 has an opening 23 for a mouse sensor on the top surface 25a of the protrusion 25. The opening 23 is a light-guiding path for guiding light to the mouse sensor 24 disposed inside the right controller 3. The mouse sensor 24 is an optical mouse sensor and may have a light-emitting part and a light-receiving part. The light detected by the light-receiving part can be visible light or invisible wavelength light. Furthermore, the mouse sensor 24 may have at least a light-receiving part, but may not have a light-emitting part. The mouse sensor 24 acquires data that can be used to calculate the movement of the right controller 3 on the mounting surface, etc., and the right controller 3 is mounted on the mounting surface with the top surface 25a of the protrusion 25 in the bottom facing the mounting surface. Thus, the right controller 3 can be used as a mouse. The operation of using this mouse is sometimes referred to as "mouse operation." Furthermore, the mounting surface is not limited to a plane, but may also be a curved surface, such as the surface of the user's thigh.

[0063] In this embodiment, the right controller 3 has a terminal 26 on the protrusion 25 for wired communication between the right controller 3 and the main body device 2. As an example, the terminal 26 is provided on a recessed inner peripheral surface provided on the top surface 25a of the protrusion 25.

[0064] Figure 3 This is a six-sided schematic diagram showing an example of the left controller 4. Descriptions regarding its similar structure to the right controller 3 are omitted. The left controller 4 has a joystick 42 on its right side, four buttons as a group: a right direction button 32, a down direction button 33, an up direction button 34, and a left direction button 35, as well as a screenshot button 37 and a negative (-) button 36. Buttons 32-35 can also be a single directional pad. Furthermore, in the right controller 3, the joystick 22 is located behind buttons 12-15; conversely, in the left controller 4, the joystick 42 is located in front of buttons 32-35. Figure 3 The front view, facing forward from the right, illustrates the x, y, and z axes, showing the coordinate system of the left controller 4 (sometimes referred to as the "left controller coordinate system"). In this coordinate system, the direction from right to left is the positive z-axis. Additionally, the direction perpendicular to the z-axis and from bottom to top is the positive x-axis, and the direction perpendicular to both the z-axis and x-axis and from rear to front is the positive y-axis. When the bottom faces the direction of gravity, the negative x-axis aligns with the direction of gravity. Furthermore, unless otherwise specified, the x, y, and z axes used in the description relating to the left controller 4 refer to the x, y, and z axes in the left controller coordinate system.

[0065] The left controller 4 has a protrusion 45 that is inserted into the recess (not shown) of the main body 2 when it is installed on the main body 2. Similar to the right controller 3, the protrusion 45 has buttons 38 and 39, an opening 43 for a mouse sensor, a mouse sensor 44, and a terminal 46.

[0066] Similar to the right controller 3, the left controller 4 can also be held longitudinally or laterally while detached from the main unit 2.

[0067] Figure 4This is a block diagram illustrating an example of the internal structure of the main device 2. The main device 2 includes a processor 63. The processor 63 is an information processing unit that performs various information processing tasks executed within the main device 2. The processor 63 may be composed of multiple processors or cores, typically multiple CPUs (Central Processing Units) or cores, or it may be composed of a System-on-a-Chip (SoC) that includes multiple functions such as CPU functions and GPU (Graphics Processing Unit) functions. The processor 63 performs various information processing tasks by executing information processing programs (e.g., game programs) stored in a storage unit (specifically, internal storage media such as flash memory 68, or external storage media such as slot 51). Furthermore, in this embodiment, the "processor" may also include at least a CPU, GPU, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc. In addition, in this embodiment, as an example, the computer includes at least one processor, and may also include a storage unit such as memory.

[0068] As an example of internal storage media, the main unit 2 includes flash memory 68 and DRAM (Dynamic Random Access Memory) 69. Flash memory 68 is primarily used to store various types of data stored in the main unit 2. DRAM 69 is primarily used to temporarily store various types of data used in information processing. The processor 63 performs various information processing tasks by appropriately reading or writing data between itself and storage media such as flash memory 68 and DRAM 69.

[0069] In addition, such as Figure 4 As shown, the main unit 2 has various structures. A brief description follows. The recording medium slot interface (sometimes called "slot I / F") 52 reads and writes data to the storage medium (e.g., a dedicated memory card) installed in the recording medium slot 51, according to instructions from the processor 63. The second slot I / F 54 reads and writes data to the storage medium installed in the second slot 53, according to instructions from the processor 63.

[0070] The network communication unit 66 communicates with external devices via a network (e.g., Internet communication using wireless communication). The controller communication unit 67 communicates wirelessly with the right controller 3 and / or the left controller 4 (e.g., communication conforming to the Bluetooth standard).

[0071] The left terminal 50 is for wired communication between the processor 63 and the left controller 4. The right terminal 65 is for wired communication between the processor 63 and the right controller 3. The lower terminal 64 is, for example, a terminal for communication with other devices (e.g., a fixed monitor) via a bracket when the lower terminal 64 is mounted on a bracket.

[0072] The touch panel controller 70 generates data, for example, indicating the location of a touch input, based on signals from the touch panel 71 configured on the display surface of the display 72, and outputs this data to the processor 63. The display 72 displays images generated by the processor 63 and / or images acquired from external sources.

[0073] The encoding / decoding circuit 74 controls the input and output of audio data for the speaker 73 and the audio input / output terminal 75.

[0074] The power control unit 61 controls the power supply from the battery 62 to each part of the main unit 2 (that is, each part that receives power from the battery 62) based on instructions from the processor 63. In addition, the power supply is started or stopped by pressing the power button 60.

[0075] Volume button 59 is used to operate the volume output from speaker 73, etc. Additionally, cooling fan 58 is a fan that cools the interior of the main unit 2.

[0076] The main unit 2 is equipped with various sensors, including a magnetic sensor 55, an ambient light sensor 56, a temperature sensor 57, an acceleration sensor 76, and an angular velocity sensor 77. The processor 63 is capable of performing various processes based on information from these sensors.

[0077] Figure 5 This is a block diagram illustrating an example of the internal structure of the main unit 2, the right controller 3, and the left controller 4. Furthermore, details of the internal structure of the main unit 2 are provided in [the diagram / illustration]. Figure 4 As shown in, therefore in Figure 5 The text is omitted.

[0078] The left controller 4 includes a communication control unit 80 for communicating with the main unit 2. For example... Figure 5As shown, the communication control unit 80 is connected to various components, including the terminal 88. When the left controller 4 is installed on the main unit 2, the communication control unit 80 communicates with the main unit 2 via the terminal 88 via a wired connection. When the left controller 4 is removed from the main unit 2, the communication control unit 80 communicates wirelessly with the main unit 2 (specifically, communication conforming to the Bluetooth standard).

[0079] The left controller 4 may include a memory 81, such as a flash memory. The communication control unit 80 may be composed of a processor, such as a microcomputer (also called a microprocessor), and performs various processes by executing firmware stored in the memory 81.

[0080] The left controller 4 has buttons 82 (specifically, buttons 32-34, etc.) and a joystick 42. Each button 82 and joystick 42 outputs information related to the operation performed on itself to the communication control unit 80.

[0081] The left controller 4 is equipped with inertial sensors. Specifically, the left controller 4 is equipped with an accelerometer 83 and an angular velocity sensor 84 as inertial sensors. The accelerometer 83 detects along three predetermined axes (e.g., ...). Figure 3 The magnitude of the acceleration in the x, y, and z axes is shown. Furthermore, the accelerometer 83 can also detect acceleration in one or two axial directions. The angular velocity sensor 84 detects angular velocities about the three axes specified above. Furthermore, the angular velocity sensor 84 can also detect angular velocities about one or two axes. Additionally, the angular velocity sensor can also be referred to as a "gyroscope sensor." The accelerometer 83 and the angular velocity sensor 84 are respectively connected to the communication control unit 80. Moreover, the detection results of the accelerometer 83 and the angular velocity sensor 84 are repeatedly output to the communication control unit 80 at appropriate times. Furthermore, the right controller 3 and the left controller 4 can be equipped with either the accelerometer or the angular velocity sensor as inertial sensors, or they can be equipped with other sensors as inertial sensors.

[0082] The left controller 4 is equipped with a mouse sensor 44. The mouse sensor 44 acquires data for calculating the movement of the left controller 4, which is placed on a mounting surface. The data acquired by the mouse sensor 44 is repeatedly output to the communication control unit 80 at appropriate times.

[0083] The communication control unit 80 acquires input-related information (specifically, information related to the operation of the buttons and joysticks, and the detection results of the sensors) from each input unit (specifically, each button 82, joystick 42, and each sensor 83, 84, and 44). The communication control unit 80 sends operation data containing the acquired information or information after predetermined processing of the acquired information to the main unit 2. Furthermore, the operation data is repeatedly sent at a predetermined interval.

[0084] By sending the aforementioned operation data to the main unit 2, the main unit 2 can receive input to the left controller 4. That is, the main unit 2 can determine the operation of each button 82 and joystick 42 based on the operation data. Furthermore, the main unit 2 can calculate information related to the action and / or posture of the left controller 4 based on the operation data (specifically, the detection results of the accelerometer 83 and / or the angular velocity sensor 84). Additionally, the main unit 2 can calculate information related to mouse operations performed on the left controller 4 based on the operation data (specifically, the detection results of the mouse sensor 44).

[0085] The left controller 4 includes an amplifier 85 and an oscillator 86. The amplifier 85 amplifies the control signal received from the communication control unit 80 to generate a drive signal. The oscillator 86 vibrates according to the drive signal generated by the amplifier 85 to make the left controller 4 vibrate.

[0086] The left controller 4 includes a power supply unit 87. The power supply unit 87 includes a battery and a power control circuit. The power control circuit is connected to the battery and supplies power to each part of the left controller 4 (specifically, to each part that receives power from the battery).

[0087] like Figure 5 As shown, the right controller 3 includes a communication control unit 91, which is composed of a processor and communicates with the main unit 2. The right controller 3 also includes a memory 94 connected to the communication control unit 91. The communication control unit 91 is connected to various components, including a terminal 92. The communication control unit 91 and the memory 94 have the same functions as the communication control unit 80 and the memory 81 of the left controller 4. Therefore, the communication control unit 91 can communicate with the main unit 2 through both wired communication via the terminal 92 and wireless communication without the terminal 92, and control the communication between the right controller 3 and the main unit 2.

[0088] Similar to the input units of the left controller 4, the right controller 3 also has input units. Specifically, it includes buttons 95 (A button 12, B button 13, X button 14, Y button 15, etc.), a joystick 22, inertial sensors (accelerometer 96 and angular velocity sensor 97), and a mouse sensor 24. These input units have the same functions as the input units of the left controller 4 and perform the same actions.

[0089] The right controller 3 includes an amplifier 98, an oscillator 99, and a power supply unit 100. The amplifier 98, oscillator 99, and power supply unit 100 have the same functions as the amplifier 85, oscillator 86, and power supply unit 87 of the left controller 4, and operate in the same manner.

[0090] The right controller 3 includes a processing unit 90 and an NFC antenna 93. The processing unit 90 controls the NFC antenna 93 according to instructions from the main device 2 via the communication control unit 91. The NFC antenna 93 performs short-range wireless communication based on the NFC (Near Field Communication) standard.

[0091] [Regarding the controller's handling method]

[0092] Figure 6 This is a schematic diagram illustrating an example of a user holding the right controller 3 with their right hand and placing it on a mounting surface to use it as a mouse, that is, performing mouse operations. Figure 6 As shown, from the user's perspective, the front of the right controller 3 faces forward, and the left side faces left. The user's right palm covers the upper side of the right controller 3. The user's right thumb is positioned on the left side of the right controller 3. The user's right thumb is, for example, placed on the A button 12. The user's right index finger is, for example, positioned on the R button 20, and the user's right middle finger is, for example, positioned on the ZR button 21. The user can operate the R button 20 and the ZR button 21 using their right index or middle finger. The user can operate the input sections located on the left side using their right thumb. Furthermore, when the user uses the left controller 4 as a mouse with their left hand, they can also hold the left controller 4 with their left hand and use it on the mounting surface using the same method. In this case, the right side of the left controller 4 faces right.

[0093] Figure 7 This is a schematic diagram illustrating an example of a user holding the right controller 3 with their right hand and operating it in the air. For example... Figure 7As shown, the right controller 3 can be held and used in the air with its long side facing up or down or forward and backward from the user's perspective, while detached from the main unit 2. The user can operate the joystick 22 (sometimes referred to as "joystick operation") with their right thumb, for example. Furthermore, the user can swing the held right controller 3 (sometimes referred to as "swing operation") and change their posture (sometimes referred to as "posture change operation"). The same method can also be used when the user uses the left controller 4, which is detached from the main unit 2, with their left hand.

[0094] [Summary of the processing in this embodiment]

[0095] Below, refer to Figures 8-13 Here is a detailed explanation of the processing in this embodiment. The following explanation will use the right controller 3 as an example. In the game of this embodiment, as an example, a crosshair is displayed as a virtual object. The crosshair moves in response to the operation, and bullets can be fired in the direction of the crosshair in accordance with a predetermined operation (e.g., a predetermined button operation). Furthermore, the same consideration can be made when using the left controller 4, therefore its explanation is omitted.

[0096] [Regarding the controller's operating modes]

[0097] Figure 8 This is a diagram used to illustrate the controller's operating modes and the transitions between them. For example... Figure 8 As shown, the controller's operating modes (sometimes simply referred to as "modes") include "mouse mode", "gyroscope mode", and "joystick mode".

[0098] In this embodiment, in mouse mode, when the inertial sensor detects a wobbling operation of the right controller 3 with a predetermined intensity (e.g., a wobbling operation of 0.2G or more; hereinafter sometimes simply referred to as a "wobbling operation"), the system switches to gyroscope mode. In mouse mode, when a joystick operation of the joystick 22 is detected, the system switches to joystick mode. In gyroscope mode, when a mouse operation of the right controller 3 is detected, the system switches to mouse mode. In gyroscope mode, when a joystick operation of the joystick 22 is detected, the system switches to joystick mode. In joystick mode, when the inertial sensor detects a wobbling operation of the right controller 3 with a predetermined intensity (e.g., a wobbling operation of 0.2G or more), the system switches to gyroscope mode. In joystick mode, when a mouse operation of the right controller 3 is detected, the system switches to mouse mode. Furthermore, the conditions for switching operating modes are not limited to these; other conditions may be used, or additional conditions may be added.

[0099] Figure 9This diagram illustrates the control of the crosshair 250 in mouse and joystick modes. First, refer to... Figure 9 This section explains how to control the crosshair 250 in mouse mode. Mouse mode is the operating mode that controls the crosshair 250 based on the output of the mouse sensor.

[0100] [Regarding crosshair control in each operating mode]

[0101] like Figure 9 As shown in (1) (b), when the right controller 3, which is placed on the mounting surface, is subjected to a mouse operation that moves it in the positive z-axis direction of the right controller coordinate system, as follows: Figure 9 As shown in (1) (a), the crosshair 250 (reference position A) displayed on the display 72 moves to the right by a distance corresponding to the mouse operation (reference position B). In addition, although not shown, when the right controller 3 on the mounting surface is operated in another direction, the crosshair 250 displayed on the display 72 also moves in the same direction and distance corresponding to the mouse operation.

[0102] However, in mouse mode, the crosshair 250 can also be restricted so that its center does not move outside the display area (sometimes simply referred to as the "display area") of the monitor 72. For example, in... Figure 9 As shown in (2) of (a), with the center of the crosshair 250 located at the right end of the display area, as Figure 9 As shown in (2) (b), even if the right controller 3, which is placed on the mounting surface, is moved in the positive z-axis direction of the right controller coordinate system by a mouse operation, the crosshair 250 is controlled to not move to the right. Thus, a portion of the crosshair 250 is always displayed on the display 72.

[0103] Next, refer to Figure 9 This section explains the control of the crosshair 250 in joystick mode. Joystick mode is the operation mode that controls the crosshair 250 based on the output of joystick operations.

[0104] For example, in such Figure 9 When the joystick 22 is tilted to the positive x-axis of the controller coordinate system as shown in (1) (c), the crosshair 250 displayed on the display 72 (reference position A) moves to the right a distance corresponding to the joystick operation (reference position B). Furthermore, although not shown, when the joystick is operated in other directions, the crosshair 250 displayed on the display 72 also moves in the same direction and distance corresponding to the joystick operation. Additionally, in joystick mode, similar to mouse mode, the movement of the crosshair 250 can be restricted so that the center of the crosshair 250 does not move outside the display area (reference position B). Figure 9(2) of (a)). Furthermore, in mouse mode and joystick mode, the movement restriction of the crosshair 250 is not limited to this. For example, the movement restriction can be such that the entire crosshair 250 is located within the display area, or the movement restriction can be such that a certain proportion (e.g., 1 / 4, 3 / 4) of the size of the crosshair 250 in the vertical or horizontal direction is located within the display area.

[0105] Figure 10 and Figure 11 This diagram illustrates the control of the crosshair 250 in gyroscope mode. Gyroscope mode is an operating mode that controls the crosshair 250 based on the output of the inertial sensor. In gyroscope mode, the posture of the right controller 3, as indicated by the output of the inertial sensor, corresponds to the display position of the crosshair 250, and the display position of the crosshair 250 is controlled according to the posture of the right controller 3. A detailed explanation follows.

[0106] In this embodiment, the posture of the right controller 3 when the crosshair 250 is displayed in the center of the display 72 is called the "reference posture". Furthermore, the correspondence that the right controller 3 becomes the reference posture when the crosshair 250 is displayed in the center of the display 72 is called the "crosshair posture correspondence". Then, based on the crosshair posture correspondence, the display position of the crosshair 250 is moved and controlled according to the posture change of the right controller 3. Furthermore, using... Figure 12 This will be described later, but the baseline posture may be reset at times.

[0107] For example, consider the following situation: Figure 10 As shown in (1) of (a), the posture (i.e., the reference posture) of the right controller 3 when the crosshair 250 is displayed at position A in the center of the display 72 is Figure 10 The posture shown in (1) and (b). In this case, for example, the posture of the right controller 3 becomes Figure 10 The posture shown in (2) of (b) (that is, from) Figure 10 In the case of the posture shown in (1) and (b) after rotating to the right in the positive z-axis direction, the collimator 250 is displayed at... Figure 10 The position shown in (2) of (a) (that is, compared to) Figure 10 (1) (a) shows position A to the right of position B). Moreover, for example, when the posture of the right controller 3 changes Figure 11 The posture shown in (b) (that is, from) Figure 10 In the case of the posture shown in (2) (b) after rotating to the right in the positive z-axis direction, the collimator 250 is Figure 11 The position shown in (a) (that is, compared to) Figure 10 Position B in (2) of (a) is further to the right of position C). Therefore, as shown in (2), Figure 11As shown in (b), corresponding to the rotation operation of the right controller 3, the crosshair 250 is not displayed on the display 72, but can be located outside the display area. Furthermore, at this time, either the processing of actually moving the crosshair 250 outside the display area can be performed, or only the processing of calculating the coordinates corresponding to the crosshair 250 can be performed. The same applies to the case where the right controller 3 changes to other postures, therefore, a detailed description of those is omitted.

[0108] As described above, in gyroscope mode, there is no restriction that the crosshair 250 is not located outside the display area. In gyroscope mode, where the crosshair 250 moves according to the posture of the right controller 3, if the restriction that the crosshair 250 is not located outside the display area is applied in the same way as in mouse mode, the relationship between the posture of the right controller 3 and the display position of the crosshair 250 may change significantly, making it difficult for the user to operate. Therefore, in gyroscope mode, the control is set to not restrict the crosshair 250 from being outside the display area.

[0109] [Regarding crosshair control when switching operating modes]

[0110] Figure 12 This diagram illustrates the control of the crosshair 250 when switching from mouse mode or joystick mode to gyroscope mode. Figure 13 This diagram illustrates the control of the crosshair 250 when switching between gyroscope mode and mouse or joystick mode. Furthermore, the operating mode, whether mouse or joystick, is sometimes referred to as "mouse / joystick mode." Additionally, in... Figure 12 and Figure 13 In the figure, reference numeral 260 indicates the calculated crosshair position used to display the crosshair 250. In addition, the center position of the crosshair 250 displayed on the display 72 is sometimes referred to as the "display crosshair position", and the calculated crosshair position mentioned above is sometimes referred to as the "target crosshair position".

[0111] First, refer to Figure 12 This section explains the control of the crosshair 250 when switching from mouse / joystick mode to gyroscope mode. Considering the mouse / joystick mode (refer to...),... Figure 12 (1) The case where the device swings and switches to gyroscope mode. In this case, if the target center position 260 corresponding to the posture of the right controller 3 at the time of switching to gyroscope mode is outside the display range (see reference). Figure 12 (2-1) updates the reference pose of the right controller 3 to the pose at the time of transition to gyroscope mode, and resets the target reticle position 260 and the display reticle position to the center of the display range (refer to...). Figure 12(3-1)). Thus, when switching to gyroscope mode, if the crosshair position corresponding to the posture of controller 3 meets a predetermined condition, such as being outside the display range, the crosshair 250 is displayed within the display range, thereby suppressing user confusion due to not being able to see the crosshair 250. Furthermore, as another example, the crosshair 250 may be displayed at a position within the display range, such as a predetermined position, when the target crosshair position during the switch to gyroscope mode is within a predetermined area of ​​the display range, or when it has moved more than a predetermined distance from the current displayed crosshair position. In this embodiment, the crosshair 250 is displayed in the center of the display range (see reference ). Figure 12 (3-1)) Therefore, it is possible to prevent the user from not seeing the crosshair 250. In addition, in this embodiment, the crosshair 250 moves instantaneously (not limited to high-speed movement, but also including instantaneous movement without intermediate passing) and is displayed, so that the user can comfortably start operating. Furthermore, the position and speed of the crosshair 250 after movement are not limited to the above situations.

[0112] In this embodiment, when switching to gyroscope mode, for example, if the crosshair 250 is displayed in the center of the display range because the target crosshair position is outside the display range, the reference posture is updated to the posture of the controller 3 when switching to gyroscope mode. This makes it easier for the user to operate the crosshair 250 in subsequent gyroscope modes. Alternatively, this reference posture update may not be performed, and a posture other than the controller 3's posture when switching to gyroscope mode, such as a predetermined posture, may be updated to the reference posture.

[0113] When the target center position 260 is within the display range corresponding to the posture of the right controller 3 at the time of switching to gyroscope mode after a wiggle operation in mouse / joystick mode (see reference). Figure 12 (2-2) will be explained. In this case, the crosshair 250 is moved toward the target crosshair position 260 and displayed (see reference). Figure 12 (3-2)). In this case, the movement speed of the crosshair 250 is, for example, a speed that the user can visually recognize, or it can be a speed that is instantly displayed within the display range when the crosshair 250 is outside the display range (see reference). Figure 12 (3-1)) The speed is low. In addition, the crosshair 250 can also supplement the position before moving (refer to) Figure 12 (2-2) and the position after movement (refer to) Figure 12 The position is moved between (3-2). By moving the crosshair 250 in this way, it is possible to prevent the user from seeing the crosshair 250. In addition, Figure 12 The crosshair 250 in (2-1) and (2-2) is inherited from the crosshair 250 in mouse / joystick mode. Additionally, Figure 12 The target crosshair position 260 of (3-1) and (2-2) can also be considered as the actual initial position of the crosshair in gyroscope mode.

[0114] Next, refer to Figure 13 This section explains the control of the crosshair 250 when switching from gyroscope mode to mouse / joystick mode. For example... Figure 13 As shown in (1), in gyroscope mode, consider the case where mouse or joystick operation is performed to switch to mouse / joystick mode. In this case, if the target center position 260 corresponding to the posture of the right controller 3 at the time of switching to mouse / joystick mode is outside the display range (see reference...), Figure 13 In (2-1), the target reticle position 260 and the display reticle position are set to positions within the display range. In this embodiment, as an example, the display reticle position is within the display range and is set to the position closest to the target reticle position 260 (see reference). Figure 13 (3-1)).

[0115] For example, consider the following situation: the display screen 72 is composed of points of 450 x 250, the lower left corner of the display area is set as the origin o(0, 0), and the xy coordinates of the display area are in the range of (0, 0) to (450, 250). In this embodiment, in this case, for example, if the target center position 260 is outside the display area at coordinates (600, 100), the target center position 260 and the display center position are set to the nearest coordinates (450, 100) within the display area; for example, if the target center position 260 is outside the display area at coordinates (100, 350), the target center position 260 and the display center position are set to the nearest coordinates (100, 250) within the display area; for example, if the target center position 260 is outside the display area at coordinates (600, 350), the target center position 260 and the display center position are set to the nearest coordinates (450, 250) within the display area. In other words, the x and y coordinates of the target reticle position 260 and the display reticle position are converted to the nearest x and y coordinates within the display range, respectively.

[0116] In this way, when switching to mouse / joystick mode, the crosshair 250 is displayed within the screen range, thus avoiding user confusion caused by the crosshair 250 not being displayed. Furthermore, since the crosshair 250 is displayed at the closest position within the screen range, users who previously operated in gyroscope mode can continue operating without any noticeable inconvenience.

[0117] Furthermore, in this embodiment, when switching from mouse / joystick mode to mouse / joystick mode after performing mouse or joystick operation in gyroscope mode, if the target center position 260 corresponding to the posture of the right controller 3 at the time of switching to mouse / joystick mode is within the display range (see reference...). Figure 13 (2-2)), the target reticle position 260 is set to display the reticle position (refer to...). Figure 13 (3-2)).

[0118] [Details regarding information processing in this embodiment]

[0119] Next, refer to Figures 14-18 The information processing of this embodiment will be described in detail. The following explanation will use the right controller 3 as an example. Furthermore, the same consideration can be given when using the left controller 4, therefore its description will be omitted.

[0120] [Regarding Data Usage]

[0121] Next, the various data stored in DRAM 69 will be explained. Figure 14 An example of data stored in the DRAM 69 of the main device 2 is shown. For example... Figure 14 As shown, at least a program storage area 301 and a data storage area 302 are provided in DRAM 69.

[0122] At least program 401 is stored in program storage area 301. At least operation mode data 402, mouse sensor data 403, joystick / button input data 404, inertial sensor data 405, target reticle position data 406, display reticle position data 407, reference posture data 408, reticle posture correspondence data 409, supplementary marker data 410, object data 411, image data 412, and virtual camera control data 413 are stored in data storage area 302.

[0123] Program 401 is a game program used to perform game processing.

[0124] Operation mode data 402 indicates which operation mode it is: mouse mode, gyroscope mode, or joystick mode. It contains historical data of the operation mode up to the previous specified frame.

[0125] Mouse sensor data 403 is data related to the output of mouse sensor 24, including dy / dz data. The dy / dz data is the output data of mouse sensor 24, representing the y-axis and z-axis directions (i.e., the yz plane) of the right controller coordinate system when the opening 23 of mouse sensor 24 is blocked by a mounting surface, etc. Figure 2The data is the distance traveled relative to the mounting surface, etc., per frame (sometimes referred to as "dy / dz"). Alternatively, dy / dz can be calculated by the communication control unit 91 or the processor 63, etc., based on the output data of the mouse sensor 24.

[0126] Joystick / Button Input Data 404 represents the data indicating the operations performed on the joystick 22 and each button 95 of the right controller 3.

[0127] Inertial sensor data 405 is data output from the inertial sensor of the right controller 3, such as data that can be used to calculate the xyz axis directions of the right controller coordinate system (see reference). Figure 2 The data includes the acceleration and angular velocity around the x, y, and z axes. Using inertial sensor data, it is possible to calculate the attitude and motion of the right controller 3, for example.

[0128] Target center position data 406 represents the target center position in the screen coordinate system obtained by converting the virtual space captured by the virtual camera into a planar coordinate system (see reference). Figure 12 and Figure 13 The target crosshair position (260) data. The target crosshair position moves based on dy / dz data in mouse mode, based on the operation of joystick 22 in joystick mode, and based on the posture change of right controller 3 in gyroscope mode.

[0129] The data 407 for displaying the crosshair position is the data representing the display position (i.e., the display crosshair position) of the crosshair 250 in the aforementioned screen coordinate system.

[0130] Reference posture data 408 is data representing the reference posture, which is the posture of the right controller 3 when the crosshair 250 is displayed in the center of the display 72.

[0131] The crosshair posture correspondence data 409 is data representing the crosshair posture correspondence, which is the correspondence of the reference posture shown in the reference posture data 408 when the crosshair 250 is displayed in the center of the display 72.

[0132] Supplementary flag data 410 indicates whether supplementary processing is performed to move the crosshair 250 toward the target crosshair position 260 (see reference). Figure 12 The marker data of (3-2)).

[0133] Object data 411 is data of virtual objects configured in the virtual space, such as data of virtual objects such as bullets fired in the direction of crosshair 250, player characters, opponent characters, and the ground.

[0134] Image data 412 includes image data such as the crosshair 250 (a virtual object), animated images, backgrounds, and virtual effects. The image of the crosshair 250 is configured in the aforementioned screen coordinate system and displayed on a monitor 72, etc. Alternatively, instead of configuring the crosshair 250 in the screen coordinate system and displaying it on the monitor 72, etc., the crosshair 250 can be configured in virtual space and displayed using a virtual camera. In other words, the crosshair 250 can be set as object data 411 instead of image data 412.

[0135] Virtual camera control data 413 is used to control a virtual camera configured in virtual space to capture images of virtual space.

[0136] In addition, DRAM 69 stores various data used in drawing processes, etc., as needed.

[0137] [Detailed examples of information processing]

[0138] Next, the processes involved in this embodiment will be explained with reference to the flowchart. Figures 15-18 This is an example of a flowchart illustrating the processing involved in this embodiment. Furthermore, the characteristic processing of this embodiment will be described primarily below, with other explanations such as drawing the processing largely omitted. Alternatively, the following processing may be performed at predetermined intervals (e.g., a processing frame interval performed every 1 / 60th of a second).

[0139] When the processing of this game begins, Figure 15 In step S101, the processor 63 determines whether the current processing is in gyroscope mode based on the operation mode data 402. If the determination in step S101 is "yes", the processing proceeds to step S102; if it is "no", the processing proceeds to step S104.

[0140] In step S102, the processor 63 determines whether the previous processing was in gyroscope mode based on the operation mode data 402. If the determination in step S102 is "yes", the processing proceeds to step S103; if it is "no", the processing proceeds to step S200 for the processing when switching to gyroscope mode.

[0141] In step S103, the processor 63 controls the target center position based on the current posture in gyroscope mode. Specifically, the processor 63 calculates the target center position corresponding to the current posture of the right controller 3 calculated based on the inertial sensor data 405, based on the crosshair posture correspondence data 409. Figure 10 (etc.). Afterwards, processing was transferred to... Figure 16 Step S111.

[0142] In step S200, processor 63 performs the processing required to switch to gyroscope mode. Figure 17 This is an example of a flowchart illustrating the process when switching to gyroscope mode.

[0143] exist Figure 17 In step S201, the processor 63 determines whether the target crosshair position based on the current posture is within the display range based on the target crosshair position data 406. If the determination in step S201 is "yes", the process proceeds to step S202; if it is "no", the process proceeds to step S203.

[0144] In step S202, processor 63 sets the supplementary flag of supplementary flag data 410 to ON. Then, processing proceeds to... Figure 16 Step S111.

[0145] In step S203, the processor 63 sets the target crosshair position represented by the target crosshair position data 406 and the display crosshair position represented by the display crosshair position data 407 to the center of the display range (see reference). Figure 12 (3-1)). After that, the process moves to step S204.

[0146] In step S204, the processor 63 sets the current posture of the right controller 3 calculated based on the inertial sensor data 405 as the reference posture represented by the reference posture data 408 (see reference). Figure 12 (3-1)). That is, processor 63 updates the reference pose to the current pose. After that, processing is transferred to (3-1). Figure 16 Step S111.

[0147] exist Figure 15 In step S104, the processor 63 determines whether the previous processing was in gyroscope mode based on the operation mode data 402. If the determination in step S104 is "yes", the processing proceeds to step S300 for processing when switching to mouse / joystick mode; if it is "no", the processing proceeds to step S105.

[0148] In step S300, processor 63 performs the processing required to switch to mouse / joystick mode. Figure 18 This is an example of a flowchart illustrating the process when switching to mouse / joystick mode.

[0149] exist Figure 18 In step S301, the processor 63 sets the supplementary flag of the supplementary flag data 410 to OFF. Then, the process proceeds to step S302.

[0150] In step S302, the processor 63 determines whether the target center position is within the display range based on the target center position data 406. If the determination in step S302 is "yes", the process proceeds to step S303; if it is "no", the process proceeds to step S304.

[0151] In step S303, the processor 63 sets the display crosshair position represented by the display crosshair position data 407 to the target crosshair position represented by the target crosshair position data 406 (see reference). Figure 13 (3-2)). Afterwards, processing is transferred to... Figure 16 Step S111.

[0152] In step S304, the processor 63 sets the display crosshair position represented by the display crosshair position data 407 and the target crosshair position represented by the target crosshair position data 406 as the nearest position within the display range (see reference). Figure 12 (3-1)). Afterwards, processing is transferred to... Figure 16 Step S111.

[0153] exist Figure 15 In step S105, the processor 63, in mouse mode or joystick mode, controls the target center position based on the mouse sensor output or joystick output (that is, joystick / button input data 404). Figure 9 After that, processing was transferred to... Figure 16 Step S111.

[0154] exist Figure 16 In step S111, the processor 63 determines whether a reset button operation has occurred. Specifically, the processor 63 determines whether, for example, the ZR button 21 of the right controller 3 has been operated based on the joystick / button input data 404. Furthermore, the reset button operation may vary depending on the operating mode. If the determination in step S111 is "yes," the process proceeds to step S112; otherwise, the process proceeds to step S114.

[0155] In step S112, with Figure 17 Similarly, in step S203, the processor 63 sets the target center position and the display center position to the center of the display range. Then, the process proceeds to step S113.

[0156] In step S113, with Figure 17 Similarly, in step S204, processor 63 sets the current pose as the reference pose. Then, processing proceeds to step S114.

[0157] In step S114, the processor 63 determines whether the supplementary flag represented by the supplementary flag data 410 is enabled. If the determination in step S114 is "yes", the process proceeds to step S115; if it is "no", the process proceeds to step S119.

[0158] In step S115, the processor 63 determines whether the difference between the target center position and the display center position is greater than a predetermined value (e.g., 10 points) based on the target center position data 406 and the display center position data 407. If the determination in step S115 is "yes", the process proceeds to step S116; if it is "no", the process proceeds to step S118. Furthermore, if the target center position is outside the display range, the process can also proceed to step S118.

[0159] In step S116, the processor 63 performs supplementary updates to bring the displayed crosshair position represented by the displayed crosshair position data 407 closer to the target crosshair position represented by the target crosshair position data 406 by a predetermined distance (e.g., 10 points). Figure 12 (3-2)). After that, the process moves to step S117.

[0160] In step S118, the processor 63 sets the supplementary flag of the supplementary flag data 410 to off. Then, the process proceeds to step S119.

[0161] In step S119, the processor 63 updates the displayed crosshair position represented by the display crosshair position data 407 to the target crosshair position represented by the target crosshair position data 406. Then, the process proceeds to step S117.

[0162] In step S117, the processor 63 displays the crosshair 250 at the crosshair position indicated by the crosshair position data 407. Afterwards, processing returns to... Figure 15 Step S101.

[0163] According to this embodiment, in both mouse / joystick mode and gyroscope mode, the crosshair position is reset based on button operations. Regardless of the mode in which the reset is performed, the reference posture is reset (see [reference]). Figure 16 (S111~S113). When a user resets the mouse / joystick mode via button operation, the user's posture and / or controller holding state may be easily operable, and the user may be able to switch to gyroscope mode without significantly changing this state. Therefore, by resetting the crosshair position and also resetting the reference posture, there is a possibility that the user can smoothly begin to perform posture change operations when switching to gyroscope mode afterwards.

[0164] Furthermore, in this embodiment, the controller's posture during the transition to gyroscope mode satisfies the conditions for transitioning to gyroscope mode, thus becoming a posture deviating from the reference posture, which could cause the crosshair position to shift significantly from the center of the display range. For example, this could occur if the conditions for transitioning to gyroscope mode are at least one of a swing operation, a specified angle condition, an angular velocity condition, or an acceleration condition. According to this embodiment, even in such cases, since the crosshair 250 is displayed within the display range, it is possible to prevent the user from not seeing the crosshair.

[0165] Furthermore, in this embodiment, for example, if a posture change operation is performed in mid-air in gyroscope mode, and then mouse operations are performed on a work surface such as a table or the user's thigh, and a posture change operation is performed again in mid-air in gyroscope mode, the controller's posture in the first and second gyroscope modes may be significantly different. According to this embodiment, control is performed to reset the crosshair position and reference posture (see [reference]). Figure 12 (2-1), (3-1), etc.), so it can suppress situations where the user cannot see the crosshair, such as when repeatedly switching between operation in gyroscope mode and operation in mouse mode.

[0166] [Variation Example]

[0167] The color, shape, and other display methods of the crosshair 250, which is a virtual object, can also change according to the operation mode. In such cases, it can be regarded as a substantially identical crosshair.

[0168] The virtual object that becomes the object of control is not limited. For example, it can also control cursors such as arrow shapes, pointers, player objects manipulated by the user, and other virtual objects.

[0169] In mouse / joystick mode, it can also be done in the same way as in gyroscope mode (see [reference]). Figure 11 Set the crosshair to be able to move outside the display range.

[0170] In the above embodiment, examples of having mouse mode, gyroscope mode and joystick mode are given. However, in other examples, the mouse mode or the joystick mode may not be present.

[0171] Furthermore, in the above-described embodiment, the following examples are given (see reference). Figure 12 , Figure 17When the condition that the target center position corresponding to the controller's posture when switching to gyroscope mode is outside the display range is met, the reference posture is updated to the controller's current posture. However, for example, the range of the controller's posture when the target center position is outside the display range can also be pre-calculated based on the reference posture. Moreover, when the condition that the controller's posture when switching to gyroscope mode is contained within the pre-calculated range is met, the reference posture can also be updated to the controller's current posture.

[0172] Furthermore, in the above-described embodiment, the following examples are given (see reference). Figure 12 , Figure 17 When the target centric position corresponding to the controller's posture when switching to gyroscope mode is outside the display range, the crosshair is displayed in the center of the display range and the reference posture is updated. Since the target centric position corresponds to the center of the crosshair, in this example, even if part of the crosshair is displayed within the display range but the target centric position (that is, the position corresponding to the center of the crosshair) is outside the display range, the crosshair is displayed in the center of the display range and the reference posture is updated. Therefore, when switching to gyroscope mode, even if the entire crosshair is outside the display range and is not displayed at all, control to display the crosshair in the center of the display range and update the reference posture can still be performed. Furthermore, when switching to gyroscope mode, control to display the crosshair in the center of the display range and update the reference posture can also be performed when the entire crosshair is located within a defined edge area of ​​the display range (e.g., an edge area accommodating the width of the entire crosshair or a portion thereof) and an area outside the display range. The phrase "the virtual object (e.g., the crosshair) is located within a certain range" can also include either the case where the entire virtual object is located within a certain range or the case where a portion of the virtual object is located within a certain range. When switching to gyroscope mode, even if the virtual object is located at the edge of the display area, control can still be performed to center the virtual object within the display area and update the reference posture. Similarly, when switching to gyroscope mode, even if the virtual object is not located within a defined area including the center of the display area, control can still be performed to center the virtual object within the display area and update the reference posture.

[0173] Even if the target reticle position corresponding to the controller's posture when switching to gyroscope mode is outside the display range, the reticle can still be controlled to display the reticle in a way that makes the center of the reticle the closest position to the display range.

[0174] Additionally, if the target's center of gravity is within the display range when switching to gyroscope mode, supplementary updates are not required (see [reference]). Figure 12(2-2), (3-2)), thus causing the crosshair to move instantly to the target's crosshair position.

[0175] Furthermore, the various data described in this embodiment are just one example; in each process, data that has been converted into other types of data may also be used appropriately.

[0176] Furthermore, a game system is an example of an information processing system, but an information processing system can also be a system that does not run games. Additionally, the main device can also be a general-purpose personal computer.

[0177] It can also be configured so that the user can operate the game by holding the right and left controllers with one hand each. In this case, for example, either either controller can operate in all three modes, only one controller can operate in all three modes, or the two controllers can share the operation. For example, the right controller can be used for mouse operation, and the left controller can be used for gyroscope and joystick operation. Alternatively, the right controller can be used for mouse and gyroscope operation, and the left controller can be used for mouse and joystick operation. That is, mode sharing includes the case where either controller supports some modes. In such a shared mode, if a controller cannot perform crosshair operation in a specified mode, it may not switch to that specified mode even if there is a device input corresponding to that specified mode. For example, if the right controller can perform mouse and gyroscope operation for crosshair operation but cannot perform joystick operation, the right controller may not switch to joystick mode even if joystick operation is present. Furthermore, devices not used for crosshair operation can also be used in other game processing. For example, in the above example, the joystick of the right controller can also be used for virtual camera and player object movement.

[0178] The controller in this embodiment is one example; its shape is not limited. The controller may also be non-detachable from the main unit. Neither controller may have a mouse sensor. Alternatively, only one of the two controllers may have a mouse sensor. Neither controller may have a joystick. Alternatively, only one of the two controllers may have a joystick. The controllers may not be in a pair. In this case, one controller may not have either a mouse sensor or a joystick. Furthermore, one controller may have two or more joysticks. In this case, for example, the crosshair may be manipulated based on the operation of one joystick, and the controller may switch to joystick mode; conversely, the crosshair may not be manipulated based on the operation of the other joystick, and the controller may not switch to joystick mode. In this case, the other joystick may also be used for the movement of the virtual camera or the player object.

[0179] The applications of the above processing are not limited to game processing. For example, they can also be applied to graphics applications, motion graphics editing applications, and operating systems. As an example, they can also be applied to menu operations in operating systems. Furthermore, when applied to game processing, they can also be applied to in-game menu operations.

[0180] At least some of the processes described above can also be executed by a server-side device in an information processing system that includes a terminal-side device and a server-side device capable of communicating via a network. Furthermore, the server can also consist of multiple information processing devices, with the processing being shared among them.

[0181] The above description of this embodiment and its variations is merely illustrative and not intended to limit its scope. Furthermore, it is self-evident that various modifications and variations can be made to this embodiment and its variations.

[0182] Explanation of reference numerals in the attached figures

[0183] 1: Information Processing System

[0184] 2: Main Unit

[0185] 3, 4: Controller

[0186] 22, 42: Joystick

[0187] 24, 44: Mouse sensor

[0188] 63: Processor

[0189] 68, 69, 81, 94: Storage section (memory)

[0190] 72: Monitor

[0191] 76, 77, 83, 84, 96, 97: Inertial sensors

[0192] 82, 95: Each button

[0193] 250: Crosshair (Displays the crosshair position)

[0194] 260: Target center position

Claims

1. An information processing method using a controller, the controller comprising at least one of a mouse sensor and a directional control unit operated by a user, and an inertial sensor, wherein, Set any one of multiple modes, including the first mode and the second mode. In the first mode, the position of the virtual object is determined based on the output of the direction operation unit or the mouse sensor. In the second mode, the position of the virtual object is determined based on the output of the inertial sensor. When determining the position of the virtual object in the second mode, the position of the virtual object is determined based on the correspondence between the virtual object's position within the display range when the controller is in a reference pose, and the position of the virtual object is determined according to the pose of the controller. When the controller's pose, i.e., the first pose, is switched from the first mode to the second mode, if at least the first condition is met, the reference pose is updated to a pose that makes the position of the virtual object corresponding to the first pose within the display range.

2. The information processing method according to claim 1, wherein, The first condition includes the virtual object corresponding to the first pose being located outside the display range.

3. The information processing method according to claim 2, wherein, In the information processing method, when the first pose satisfies the first condition, the reference pose is updated to the first pose.

4. The information processing method according to claim 3, wherein, The specified position is the center of the display area.

5. The information processing method according to claim 1, wherein, In the information processing method, when the first posture does not meet the first condition, compared with when the first posture meets the first condition, the virtual object is moved slowly from the position of the virtual object in the first mode toward the position of the virtual object in the second mode.

6. The information processing method according to claim 1, wherein, In the information processing method, when switching from the second mode to the first mode, if the virtual object is located outside the display range, the virtual object is moved into the display range.

7. The information processing method according to claim 6, wherein, In the information processing method, the position of the virtual object within the display range is determined based on the position of the virtual object outside the display range when switching from the second mode to the first mode.

8. The information processing method according to claim 1, wherein, In the information processing method, In both the first and second modes, the position of the virtual object is updated to the predetermined position based on button operations. When the position of the virtual object is updated to the specified position, the reference pose is updated to the pose of the controller at the time of the update.

9. The information processing method according to claim 1, wherein, In the information processing method, when the output of at least the inertial sensor satisfies the second condition, the method switches from the first mode to the second mode.

10. The information processing method according to claim 1, wherein, In the information processing method, in the first mode, the position of the virtual object is determined based on the output of the mouse sensor.

11. The information processing method according to claim 10, wherein, In the information processing method, Set any one of multiple modes, including the first mode, the second mode, and the third mode. When determining the position of the virtual object in the third mode, the position of the virtual object is determined based on the output of the direction operation unit. If the first pose when switching from the first mode or the third mode to the second mode at least satisfies the first condition, the reference pose is updated to a pose in which the position of the virtual object corresponding to the first pose is within the display range.

12. An information processing system comprising a controller and an information processing unit, the controller comprising at least one of a mouse sensor and a user-operated directional control unit, and an inertial sensor, wherein, Set any one of multiple modes, including the first mode and the second mode. In the first mode, the position of the virtual object is determined based on the output of the direction operation unit or the mouse sensor. In the second mode, the position of the virtual object is determined based on the output of the inertial sensor. When determining the position of the virtual object in the second mode, the position of the virtual object is determined based on the correspondence between the virtual object's position within the display range when the controller is in a reference pose, and the position of the virtual object is determined according to the pose of the controller. When the controller's pose, i.e., the first pose, is switched from the first mode to the second mode, if at least the first condition is met, the reference pose is updated to a pose that makes the position of the virtual object corresponding to the first pose within the display range.

13. The information processing system according to claim 12, wherein, The first condition includes the virtual object corresponding to the first pose being located outside the display range.

14. The information processing system according to claim 13, wherein, When the first pose satisfies the first condition, the information processing system updates the reference pose to the first pose.

15. The information processing system according to claim 14, wherein, The specified position is the center of the display area.

16. A computer-readable non-transient recording medium recording an information processing program, the information processing program being used to cause a processor of an information processing system having a controller and an information processing unit to perform processing, the controller having at least one of a mouse sensor and a user-operated directional control unit, and an inertial sensor, wherein... The information processing program causes the processor to perform the following processes: Set any one of multiple modes, including the first mode and the second mode; In the first mode, the position of the virtual object is determined based on the output of the direction operation unit or the mouse sensor; In the second mode, the position of the virtual object is determined based on the output of the inertial sensor; When determining the position of the virtual object in the second mode, the position of the virtual object is determined according to the posture of the controller, based on the correspondence between the virtual object's position within the display range when the controller is in a reference posture; and When the controller's pose, i.e., the first pose, is switched from the first mode to the second mode, if at least the first condition is met, the reference pose is updated to a pose that makes the position of the virtual object corresponding to the first pose within the display range.

17. The recording medium according to claim 16, wherein, The first condition includes the virtual object corresponding to the first pose being located outside the display range.

18. The recording medium according to claim 17, wherein, The information processing program is used to cause the processor to perform the following processing: when the first pose satisfies the first condition, update the reference pose to the first pose.

19. The recording medium according to claim 18, wherein, The specified position is the center of the display area.

20. The recording medium according to claim 16, wherein, The information processing program is a program that enables the processor to perform game processing.

21. A computer program product comprising an information processing program for causing a processor of an information processing system having a controller and an information processing unit to perform processing, the controller having at least one of a mouse sensor and a user-operated directional control unit, and an inertial sensor, wherein... The information processing program causes the processor to perform the following processes: Set any one of multiple modes, including the first mode and the second mode; In the first mode, the position of the virtual object is determined based on the output of the direction operation unit or the mouse sensor; In the second mode, the position of the virtual object is determined based on the output of the inertial sensor; When determining the position of the virtual object in the second mode, the position of the virtual object is determined according to the posture of the controller, based on the correspondence between the virtual object's position within the display range when the controller is in a reference posture; and When the controller's pose, i.e., the first pose, is switched from the first mode to the second mode, if at least the first condition is met, the reference pose is updated to a pose that makes the position of the virtual object corresponding to the first pose within the display range.

Images