Program, system and method

Abstract

To solve a problem that mouse sensor output that does not accurately reflect an actual mouse operation of a user may execute processing unintended by a user.SOLUTION: When an execution condition is satisfied including at least a matter that a controller having a mouse sensor comes into a state where it is grounded from a state where it is not grounded, a program causes a computer to control an object by using at least output of an inertial sensor of the controller.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to programs, systems, and methods. [Background technology]

[0002] Processing using mouse sensor output is known (for example, WO2025 / 079209 (Patent Document 1)). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2025 / 079209 Summary of the Invention [Problem to be solved by the invention]

[0004] Mouse sensor output that does not accurately reflect the user's actual mouse operation may result in the execution of an operation that the user does not intend. [Means for solving the problem]

[0005] (Configuration 1) A program according to one embodiment may cause a computer to function as: means for controlling an object using the output of a mouse sensor possessed by a controller; means for determining that the controller has changed from an ungrounded state to a grounded state; means for determining that an execution condition has been satisfied which includes at least the determination that the controller has changed from an ungrounded state to a grounded state; and means for controlling the object using at least the output of an inertial sensor possessed by the controller when the execution condition has been satisfied.

[0006] (Configuration 2) In configuration 1, the program may cause the computer to function as a means for calculating a parameter related to the velocity of the controller based on the output of the inertial sensor, and the execution condition may include determining that the parameter is greater than a first threshold value when the controller changes from a non-grounded state to a grounded state.

[0007] (Configuration 3) In configuration 2, the program may cause the computer to function as a means for terminating control of the object using the output of the inertial sensor based on the parameter being smaller than a second threshold value after the controller is grounded.

[0008] (Configuration 4) In configuration 3, the program may cause the computer to function as a means for calculating the second threshold value based on the parameters when the controller changes from a non-grounded state to a grounded state.

[0009] (Configuration 5) In any of configurations 1 to 4, the program may cause the computer to function as a means for terminating control of the object using the output of the inertial sensor based on the fact that the duration of contact between the controller and the ground after the controller has changed from a non-grounded state to a grounded state is greater than a third threshold.

[0010] (Configuration 6) In any of configurations 1 to 5, the execution condition may include that the duration of the state in which the controller is not grounded immediately before the controller is grounded is greater than a fourth threshold value.

[0011] (Configuration 7) In any of configurations 1 to 6, the program may cause the computer to function as means for calculating a parameter related to the velocity of the controller based on the output of the inertial sensor. The execution condition may include a condition in which the moving direction of the controller indicated by the parameter related to velocity indicates a first direction among a first direction and a second direction opposite to the first direction, while the moving direction of the controller indicated by the output of the mouse sensor indicates a fourth direction among a third direction corresponding to the first direction and a fourth direction opposite to the third direction and corresponding to the second direction, and may not include a condition in which the moving direction of the controller indicated by the parameter related to velocity indicates the second direction while the moving direction of the controller indicated by the output of the mouse sensor indicates the third direction.

[0012] (Configuration 8) In any of configurations 1 to 7, the program may cause the computer to function as a means for controlling the object in a direction corresponding to the direction indicated by the output of the mouse sensor, with a control amount corresponding to the amount of movement indicated by the output of the mouse sensor, and when the execution condition is satisfied, a means for controlling the object in a direction corresponding to the direction indicated by the output of the inertial sensor, with a control amount corresponding to the amount of movement indicated by the output of the mouse sensor.

[0013] (Configuration 9) A system according to another embodiment includes a controller and a device, wherein the controller has a mouse sensor and an inertial sensor and is configured to transmit the output of the mouse sensor and the output of the inertial sensor, and the device is configured to receive the output of the mouse sensor and the output of the inertial sensor, control an object using the output of the mouse sensor, determine that the controller has changed from an ungrounded state to a grounded state, determine that an execution condition has been satisfied including at least that the controller has changed from an ungrounded state to a grounded state, and when the execution condition is satisfied, control the object using at least the output of the inertial sensor.

[0014] (Configuration 10) A method according to another embodiment may include acquiring an output of a mouse sensor of a controller, acquiring an output of an inertial sensor of the controller, controlling an object using the output of the mouse sensor, determining that the controller has changed from an ungrounded state to a grounded state, determining that an execution condition has been satisfied including at least the determination that the controller has changed from an ungrounded state to a grounded state, and when the execution condition is satisfied, controlling the object using at least the output of the inertial sensor. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of an appearance of a controller according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a usage mode of a controller according to the present embodiment. [Figure 4] FIG. 2 is a diagram showing an example of data at least temporarily stored in a device according to the present embodiment. [Figure 5] 10 is a flowchart showing an example of a processing procedure in the device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present embodiment will be described with reference to the drawings. [A. System configuration example] 1 is a schematic diagram showing an example configuration of a system 1 according to the present embodiment. System 1 includes a device 10 and a controller 20. System 1 may also include a display 30. Device 10 is connected to controller 20 and display 30 via a wired or wireless connection. Device 10, controller 20, and display 30 are examples of a computer.

[0017] The device 10 includes, for example, a processor 11, a volatile storage medium 12, a non-volatile storage medium 13, and a communication unit 14. The device 10 may include a plurality of each of these. The device 10 may also include an image / audio output unit 15. The device 10 may also include only one of the volatile storage medium 12 and the non-volatile storage medium 13. The device 10 may also be, for example, a personal computer, a game console, a television, a smartphone, a server, etc.

[0018] The volatile storage medium 12 includes, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0019] The non-volatile storage medium 13 includes, for example, a flash memory or a hard disk, etc. The non-volatile storage medium 13 may also include, for example, a storage medium that is removable from the device 10, such as a cartridge or an optical disk.

[0020] Nonvolatile storage medium 13 stores, for example, system program 100 or an application program. In this embodiment, nonvolatile storage medium 13 stores system program 100 and game application program 200. In addition to or instead of the above programs, nonvolatile storage medium 13 may store other programs, such as a driver responsible for communication with controller 20, or an application program different from the game application. In this specification, the term "program" may refer not only to a single program but also to a group of programs.

[0021] The processor 11 executes various processes in the device 10. The processor 11 may be, for example, a central processing unit (CPU) or a graphics processing unit (GPU). As an example, the processor 11 loads a program stored in a non-volatile storage medium 13 into a volatile storage medium 12 and executes the program. In this specification, the term "processor" encompasses at least a processing circuit that executes processing according to computer-readable instructions, such as a CPU or a GPU, and a hardwired circuit, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0022] An SoC (System on Chip) that integrates functions such as a processor and a storage medium may be employed. Therefore, in this specification, the terms "processor" and "storage medium" encompass both independent configurations and integrated configurations.

[0023] The communication unit 14 receives data from at least the controller 20. The communication unit 14 may transmit data to the controller 20. The communication unit 14 may include a communication port and a communication IC (Integrated Circuit) for exchanging wired signals with the controller 20, or may include an antenna and a wireless communication circuit for exchanging wireless signals with the controller 20. The communication unit 14 may include a communication module for realizing wired communication or wireless communication. The communication unit 14 may include an interface circuit for exchanging data with the processor 11. The communication unit 14 may include a processor such as a microcomputer and a storage medium. The processor 11 may execute processing for realizing at least some of the functions of the communication unit 14.

[0024] The device 10 may output the image data and audio data generated by the processor 11 to a display 30 or the like via the image / audio output unit 15. The device 10 may include a display or a speaker.

[0025] 2 is a schematic diagram showing an example of the appearance of controller 20. As shown in FIG. 2, controller 20 has, as an example, a plate shape with the y-axis direction as the longitudinal direction (a rectangular parallelepiped or a shape similar thereto, in which the length in the z-axis direction is smaller than the lengths in the y-axis direction and the x-axis direction, and the thickness in the z-axis direction is smaller than the thickness in the x-axis direction) (see the xyz coordinate system in FIG. 2). Note that controller 20 may have other shapes. In this embodiment, controller 20 includes button 26 and direction input unit 27.

[0026] The controller 20 includes a mouse sensor 23. As shown in FIG. 2, the controller 20 includes a mouse sensor opening 28 on its bottom. The mouse sensor opening 28 is an opening in a light guide path that guides light to the mouse sensor 23 located inside the opening 28. The mouse sensor 23 may be, for example, an optical mouse sensor and may include a light emitter and a light receiver. The light detected by the light receiver may be visible light or light of an invisible wavelength. The mouse sensor 23 acquires data that enables calculation of the movement of the controller 20 on a work surface, with the bottom facing the work surface. This allows the controller 20 to be used as a mouse. This operation of using the controller 20 as a mouse is sometimes referred to as "mouse operation." FIG. 3 is a schematic diagram showing an example of a state in which a user holds the controller 20 in their right hand, places it on a work surface, and performs mouse operation. The controller 20 may be lifted up by the user and operated. For example, the controller 20 may be held in the air with both hands and used like a game controller 20.

[0027] Returning to FIG. 1 , the controller 20 includes a controller processor 21, an inertial sensor 22, a mouse sensor 23, and a controller communication unit 24. The controller 20 may include a storage medium. As shown in FIG. 2 , the controller 20 may include operation units such as a directional input unit 27 and buttons 26. The type of the controller 20 is not limited. The controller 20 may be called a mouse or a game controller. The controller 20 may also be a terminal such as a smartphone. The controller 20 may be configured to be detachably attached to the device 10.

[0028] The controller processor 21 executes the processing of the controller 20. The controller processor 21 may have the same configuration as the processor 11, or may have at least a partly different configuration. The controller processor 21 may be, for example, a microcomputer.

[0029] The mouse sensor 23 acquires mouse sensor data 310 including data that enables calculation of the movement of the controller 20 due to mouse operation, etc. The mouse sensor 23 is connected to the controller processor 21, and repeatedly outputs the mouse sensor data 310 to the controller processor 21 at appropriate timings.

[0030] The mouse sensor data 310 includes, for example, dy / dz data 311 and image clarity data 312.

[0031] As an example, the dy / dz data 311 is data indicating the movement distance of the controller 20 in the y-axis direction and the z-axis direction (see FIG. 2) per frame time relative to a work surface or the like when the opening 28 of the mouse sensor 23 is blocked by the work surface or the like. As an example, the mouse sensor 23 acquires a mouse sensor image showing the surface blocking the opening 28, and calculates the dy / dz data 311 according to the change between frames of the multiple image data.

[0032] The image clarity data 312 is, for example, data calculated by the mouse sensor 23 and indicates the clarity of the mouse sensor image. The image clarity data 312 is calculated, for example, based on the brightness of the mouse sensor image or the number of feature points in the mouse sensor image. The image clarity data 312 may also be calculated based on other factors. If the clarity indicated by the image clarity data 312 is equal to or greater than a predetermined value, it can be estimated that the opening 28 of the mouse sensor 23 is blocked by a mounting surface or the like. Note that, instead of the data indicating the clarity of the mouse sensor image, other data may be used as long as it can be used to estimate that the opening 28 is blocked. For example, the output of a distance sensor or the output of a contact detection mechanism provided on the bottom surface of the controller may be used.

[0033] The dy / dz data 311 or the image sharpness data 312 may be calculated by the controller 20 (eg, the mouse sensor 23 and / or the controller processor 21) or by the device 10.

[0034] The controller 20 includes an inertial sensor 22. In this embodiment, the controller 20 includes an acceleration sensor and an angular velocity sensor. The acceleration sensor detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in FIG. 2). The acceleration sensor may detect acceleration in one or two axial directions. The angular velocity sensor detects angular velocity around the three predetermined axes. The angular velocity sensor may detect angular velocity around one or two axes. The acceleration sensor and the angular velocity sensor are each connected to the controller processor 21, and repeatedly output inertial sensor data 313 to the controller processor 21 at appropriate timing. The controller 20 may include only one of the acceleration sensor and the angular velocity sensor, or may include other inertial sensors.

[0035] The controller communication unit 24 transmits at least data to the communication unit 14 of the device 10. The data is transmitted at appropriate timing. The data may be transmitted repeatedly, for example, at regular intervals. The data transmitted by the controller communication unit 24 includes mouse sensor data 310 and inertial sensor data 313. The data may include data indicating input to the operation unit. The controller processor 21 may, for example, acquire mouse sensor data 310 from the mouse sensor 23, process the data, and transmit the data to the controller communication unit 24. The controller communication unit 24 may also process and transmit data acquired from the controller processor 21. In this embodiment, when the mouse sensor data 310 output by the mouse sensor 23 is processed, transmitted, and stored in the storage medium of the device 10, all of the data is referred to as mouse sensor data 310. The same applies to the inertial sensor data 313 and other data.

[0036] When the controller 20 is operated like a mouse, the mouse sensor data 310 may indicate values ​​that do not correspond to the actual mouse movement. For example, such an inconsistency may occur when the controller 20 moves from an airborne state to a resting state on the work surface while moving in a direction parallel to the work surface.

[0037] As an example, a game implemented by the game application program 200 in this embodiment is a game in which an object moves forward when the user operates the controller 20 forward with the mouse. The rate at which the object moves forward is controlled according to the speed of the user's mouse operation. Note that the object may decelerate over time. When the user operates the controller 20 backward with the mouse, the object moves backward. In this embodiment, the object does not move even if the user operates the controller 20 left or right with the mouse. In this embodiment, the game application program 200 uses dy data indicating dy among the dy / dz data 311 of the mouse sensor data. In order to move the object forward faster or farther, the user may repeat the following operations: (1) operate the controller 20 forward with the mouse, (2) lift the controller 20 off the work surface and pull it toward them, and (3) place the controller 20 on the work surface again and operate the mouse forward with the mouse. While repeating these operations, the dy / dz data 311 may indicate that the controller 20 is moving backward, even though the controller 20 is actually being operated forward with the mouse. Therefore, in this embodiment, by correcting the mouse sensor data 310, object manipulation that conforms to the user's intention is realized.

[0038] FIG. 4 shows an example of various data stored in the storage medium of the device 10. As shown in FIG.

[0039] The non-volatile storage medium 13 stores a system program 100 and a game application program 200. The game application program 200 includes object data 211. The object data 211 is data relating to an object to be controlled.

[0040] The volatile storage medium 12 stores mouse sensor data 310 and inertial sensor data 313 received from the controller.

[0041] The other data will be explained later.

[0042] FIG. 5 is an example of a flowchart illustrating processing according to this embodiment. This processing may be executed constantly while the application program 200 is running, or may be executed during certain scenes. This processing may be executed, for example, during game play. This processing may not be executed, for example, during menu operation. This processing may be executed repeatedly at predetermined intervals (for example, at processing frame intervals executed every 1 / 60 seconds). The processing described in this embodiment may include other steps or may not include some steps. Furthermore, the execution order of each step is an example. For example, multiple steps may be executed in parallel, or some steps may be executed in the reverse order to that described. The steps may be divided in any way, and multiple steps may be integrated, or one step may be divided. The processing described in this embodiment is shown as a single flowchart, but may be realized by multiple processes shown in multiple flowcharts.

[0043] In step S101, it is determined whether the controller 20 is estimated to be in contact with the work surface in a state where mouse operation is possible.

[0044] As an example, the CPU 11 reads out the ground contact flag data 314 stored in the volatile storage medium 12. When the read ground contact flag data 314 indicates ON, the result of the determination in step S101 is YES.

[0045] As an example, the system program 100 is executed to control the ground contact flag data 314 based on the image clarity data 312. For example, when the value indicated by the image clarity data 312 is higher than a predetermined value, the ground contact flag data 314 indicating ON is stored in the volatile storage medium 12. When the value indicated by the image clarity data 312 is higher than the predetermined value, it indicates that the image is clear and that the opening 28 of the mouse sensor 23 is blocked by something. In this case, it is estimated that the controller 20 is grounded. Note that "storing" can also mean "updating."

[0046] When the value indicated by the image clarity data 312 is equal to or smaller than a predetermined value, the ground contact flag data 314 indicating OFF may be stored. Even in this case, for example, when the dy / dz data 311 indicates movement of the controller 20, the ground contact flag data 314 indicating ON may be stored. Furthermore, for example, the ground contact flag data 314 may be controlled based on the inertial sensor data 313 in addition to or in place of the image clarity data 312. For example, the ground contact of the controller 20 may be estimated based on the occurrence of acceleration equal to or greater than a predetermined value in the negative x-axis direction. Instead of or in addition to the image clarity data 312, appropriate data may be used to estimate the ground contact of the controller 20.

[0047] When it is estimated that the controller 20 has touched the ground, the controller 20 does not necessarily have to actually be touching the ground on a desk or other surface. For example, a user may lift the controller 20 so that the bottom is facing horizontally and block the opening 28 with their hand. In this case, it may be estimated that the controller 20 has touched the ground.

[0048] The ground contact flag data 314 may be updated by execution of the game application program 200. Note that the ground contact flag data 314 does not have to be provided. Ground contact may be estimated by other calculations or flags.

[0049] If step S101 is YES, then in step S102, the controller 20 determines whether touchdown has started. As an example, if the previous touchdown flag data 314 indicates OFF and the current touchdown flag data 314 indicates ON, step S102 is determined to be YES. To prevent chattering, step S102 may be determined to be YES based on whether the touchdown flag data 314 has been maintained at the value indicating ON for several consecutive times after switching from a value indicating OFF to a value indicating ON. In other processes, similar general control considerations may be adopted.

[0050] If step S102 is YES, in step S103, the velocity of the controller 20 is calculated based on the inertial sensor data 313. This velocity is a value expected to correspond to the velocity when the controller starts to slide on the work surface. Hereinafter, this velocity will be referred to as the "initial velocity." Initial velocity data 315 indicating the calculated initial velocity is stored in the volatile storage medium 12. Note that velocity data 316 indicating the velocity may be constantly calculated and stored based on the inertial sensor data 313, and the velocity data 316 stored in step S103 may be read out and used as the initial velocity data 315.

[0051] The method of calculating the initial velocity based on the inertial sensor data 313 is not limited. As an example, the acceleration for multiple frames based on the output values ​​of the acceleration sensor may be stored, and the acceleration for the most recent multiple frames may be added together to determine the initial velocity. Note that the acceleration at the moment when the ground contact flag data 314 changes from a value indicating OFF to a value indicating ON may be affected by the impact of touching down, and therefore may not be included in the addition. As an example, the acceleration for multiple past frames may be used for the addition, including the acceleration when the ground contact flag data 314 last had a value indicating OFF. The acceleration may be calculated without the effects of gravitational acceleration and centrifugal force. For example, the output value of the acceleration sensor may be corrected by the output value of an angular velocity sensor.

[0052] In step S104, it is determined whether the initial velocity is greater than a threshold value. In this embodiment, the initial velocity compared with the threshold value is an absolute value. The threshold value is included in threshold data 210 of the game application program 200, which is stored in the non-volatile storage medium 13. Other threshold values ​​mentioned later may also be included in the threshold data 210. In this specification, "greater" and "smaller" may mean the same thing. When the controller 20 is placed on a work surface and moved, a phenomenon in which the mouse sensor data 310 does not match the actual movement of the controller 20 is more likely to occur when the moving speed of the controller 20 is high than when the moving speed is low. Therefore, in this embodiment, when the initial velocity is greater than the threshold value, the mouse sensor data 310 is corrected based on the inertial sensor data 313.

[0053] If step S104 is YES, step S105 determines whether the non-ground contact period between the previous ground contact and the current ground contact is greater than a threshold. As an example, it determines whether the period in which the immediately preceding ground contact flag data 314 had a value indicating OFF was 15 frames or more. If the non-ground contact period is short, the accuracy of the initial velocity estimation based on the inertial sensor data 313 may be low. Therefore, in this embodiment, a correction is made when the non-ground contact period is long.

[0054] If step S105 is YES, mouse sensor data 310 is corrected (step S106). In this embodiment, when the initial velocity is positive (for example, when controller 20 is moving in the positive direction of the y-axis) but dy of dy / dz data 311 indicates a negative value (when controller 20 is moving in the negative direction of the y-axis), dy of dy / dz data 311 is corrected to a value indicating a positive value and used for object operation. For example, the direction indicated by initial velocity data 315 is compared with the direction indicated by dy / dz data 311, and if they differ, initial velocity data 315 is corrected. Note that, for example, "mouse sensor data 310 is corrected" does not necessarily mean that the positive or negative value of dy of dy / dz data 311 stored in volatile storage medium 12 is changed, but also means that dy of dy / dz data 311 read from volatile storage medium 12 is converted to a positive value and used for object control processing, or that the absolute value of dy of mouse sensor data 310 is read from volatile storage medium 12 and assigned a positive or negative value. For example, "the mouse sensor data 310 is corrected" may include the execution of an object control that would be executed when the dy of the mouse sensor data 310 is positive, even though the dy of the mouse sensor data 310 obtained from the controller 20 and stored in the volatile storage medium 12 is negative.

[0055] In this embodiment, the absolute value of dy is not corrected. That is, even when the moving direction of the object is controlled based on the inertial sensor data 313, the moving degree of the object is controlled based on the dy / dz data 311.

[0056] In this embodiment, the game application program 200 does not use dz, and therefore dz in the dy / dz data 311 is not corrected.

[0057] If step S104 is NO or if step S105 is NO, the above correction is not performed. The object is controlled based on the dy / dz data 311.

[0058] Returning to step S102, if the result is NO, that is, if the controller 20 continues to be in contact with the ground after the start of contact, the speed is calculated (step S107). The speed may be calculated by multiplying the initial speed by the acceleration after contact. Speed ​​data 316 indicating the speed is stored in the volatile storage medium 12.

[0059] In step S108, it is determined whether the direction of the current velocity indicated by the velocity data 316 is different from the direction of the initial velocity indicated by the initial velocity data 315. If they are different (step S108 = YES), the correction is terminated. This is because it is estimated that the way the user moves the controller 20 has changed.

[0060] If step S108 is NO, then in step S109 it is determined whether the current speed is less than half the initial speed. If it is less than half the initial speed (step S109=YES), the correction is terminated. This is because when the speed of the controller 20 is low, the accuracy of the mouse sensor data 310 is high. Note that half is just an example, and an appropriate rate can be set.

[0061] If step S109 is NO, it is determined whether the ground contact duration, which is the duration from the start of ground contact to the start of ground contact, is greater than the threshold. If the ground contact duration is greater than the threshold (step S110=YES), the correction is terminated. If the ground contact duration is not greater than the threshold (step S110=NO), the correction is continued (step S111).

[0062] The above embodiment is just an example. Other examples are given below.

[0063] The speed and initial velocity in each step are examples of parameters related to the movement of the controller 20. Instead of or in addition to the speed and initial velocity, for example, a value obtained by correcting the speed and initial velocity, acceleration based on the inertial sensor data 313, or the like may be used.

[0064] The condition for executing the correction when the controller 20 is grounded is not limited to step S104 or step S105. The correction may be executed when at least one of step S104 or step S105 is YES. The correction may be executed based on other conditions. At least one of step S104 or step S105 may not be included in the processing. As an example, step S104 and step S105 may not be included in the processing, and the correction may be executed when step S102 is YES. For example, in this case, determining that the controller 20 has changed from a non-grounded state to a grounded state and determining that the condition for executing the correction has been satisfied are achieved simultaneously.

[0065] In step S109, the value to be compared with the current speed may be a predetermined value.

[0066] The conditions for ending the correction are not limited to steps S108 to S110. Some steps may be omitted. The correction may be continued when at least one of steps S108 to S110 is NO. The correction may be ended based on other conditions.

[0067] When the controller 20 continues to be in a grounded state, and the correction is completed, the correction may be executed again. The condition for executing the correction at this time may be at least one of step S104 and step S105, or may be another condition. As an example, consider a case where the value of the dy / dz data 311 is, for example, 0 and the controller 20 is stationary. At this time, the correction is completed. When the controller 20 starts to move again while remaining in a grounded state from that state, it may be considered that the controller 20 has changed from a non-grounded state to a grounded state (i.e., step S102 = YES may be determined). Subsequently, the determinations of step S104 and step S105 may be performed, or another determination may be performed.

[0068] The game application program 200 may control an object based on the dz data in the dy / dz data 311. As an example, the object may move left or right based on the dz data. As an example, the object may be able to move freely in two dimensions based on the dy data and the dz data. When a predetermined condition is met, the object's two-dimensional movement direction may be controlled based on the inertial sensor data 313 (including, as an example, the initial velocity data 315 or the velocity data 316, and the same applies to others). In addition to or instead of the dy data, the dz data may be corrected based on the inertial sensor data 313.

[0069] Not only may the control direction of the object be controlled based on the inertial sensor data 313, but also the control amount of the object may be controlled based on the inertial sensor data 313. As an example, the object may be controlled with a control amount based on a value calculated from the dy / dz data 311 and the inertial sensor data 313. The control direction of the object does not have to be controlled based on the inertial sensor data 313. As another example, the object may be controlled based on the movement amount of the controller indicated by the inertial sensor data 313, instead of the movement amount of the controller 20 indicated by the dy / dz data 311. Furthermore, the object may be controlled based on the movement amount of the controller indicated by the inertial sensor data 313, instead of the movement direction of the controller 20 indicated by the dy / dz data 311. In this case, for example, the mouse sensor data 310 does not need to be corrected by the inertial sensor data 313.

[0070] For example, when the dy in the dy / dz data 311 is negative but the initial velocity data 315 is positive, dy may be corrected to be positive. Conversely, when the dy in the dy / dz data 311 is positive but the initial velocity data 315 is negative, dy may not be corrected to be positive. The opposite is also possible. Whether the correction is performed in either case may be set, for example, for each application or for each scene within the application. Whether the object is controlled based on the inertial sensor data 313 may be determined based on the direction indicated by the dy / dz data 311 and / or the direction indicated by the initial velocity data 315.

[0071] The mouse sensor data 310 may include data indicating the position of the controller 20 used to control the object in addition to or instead of the dy / dz data 311. The data indicating the position of the controller 20 may be obtained by integrating the dy / dz data 311, for example. The data indicating the position of the controller 20 may be corrected by the inertial sensor data 313.

[0072] 5 may be executed by the system program 100. The processing shown in Fig. 5 may be executed by a group of programs including the system program 100 and the game application program 200. The processing shown in Fig. 5 may be executed by the controller 20.

[0073] The object to be operated is not limited. The object may be a character, a bullet, a cursor, or a rotating option. For example, the entire screen or a field may move in response to mouse operation. The object operation realized by operating the controller 20, for example, forward with the mouse, is not limited. The object may move backward, rotate, expand or contract, disappear, extend, or move on to the next object.

[0074] If the system 1 includes a server or other device in addition to a device operated by a user, some or all of the processes shown in FIG. 5 or other processes may be performed by the server or other device.

[0075] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0076] 1 System 10 devices 11 processors 12 Volatile storage media 13 Non-volatile storage media 14 Communications Department 15. Video and audio output section 20 Controller 21 Controller Processor 22 Inertial Sensor 23 Mouse Sensor 24 Controller communication section 26 buttons 27 Directional input section 28 Aperture 30 Display 100 System Programs 200 Game Application Programs 210 Threshold Data 211 Object Data 310 Mouse Sensor Data 311 dy / dz data 312 Image clarity data 313 Inertial Sensor Data 314 Ground Flag Data 315 initial velocity data 316 Speed ​​Data

Claims

1. Computer, means for controlling an object using an output from a mouse sensor of the controller; means for determining whether the controller has changed from an ungrounded state to a grounded state; means for determining that an execution condition is satisfied, the execution condition including at least that the controller has changed from an ungrounded state to a grounded state; means for controlling the object using at least an output of an inertial sensor included in the controller when the execution condition is satisfied; A program that functions as a

2. The computer functioning as a means for calculating a parameter relating to the velocity of the controller based on the output of the inertial sensor; the execution condition includes a determination that the parameter is greater than a first threshold value when the controller changes from a non-grounded state to a grounded state; The program according to claim 1.

3. The computer and functioning as a means for terminating control of the object using the output of the inertial sensor based on the parameter being smaller than a second threshold value after the controller is in a grounded state. The program according to claim 2.

4. The computer causing the controller to function as a means for calculating the second threshold value based on the parameter when the controller changes from a non-grounded state to a grounded state; The program according to claim 3.

5. The computer and functioning as a means for terminating control of the object using the output of the inertial sensor, based on the fact that a duration of contact between the controller and the object after the controller has changed from a non-contact state to a contact state is greater than a third threshold value. The program according to claim 1.

6. the execution condition includes a condition that a duration of a state in which the controller is not grounded immediately before the controller is grounded is greater than a fourth threshold value; The program according to claim 1.

7. The computer functioning as a means for calculating a parameter relating to the velocity of the controller based on the output of the inertial sensor; The execution condition is: the moving direction of the controller indicated by the speed-related parameter indicates a first direction among a first direction and a second direction opposite to the first direction, while the moving direction of the controller indicated by the output of the mouse sensor indicates a fourth direction among a third direction corresponding to the first direction and a fourth direction opposite to the third direction and corresponding to the second direction, a case where the moving direction of the controller indicated by the speed-related parameter indicates the second direction, while the moving direction of the controller indicated by the output of the mouse sensor indicates the third direction, is not included. The program according to claim 1.

8. The computer means for controlling the object in a direction corresponding to the direction indicated by the output of the mouse sensor by a control amount corresponding to the amount of movement indicated by the output of the mouse sensor; means for controlling, when the execution condition is satisfied, the object in a direction corresponding to the direction indicated by the output of the inertial sensor, by a control amount corresponding to the amount of movement indicated by the output of the mouse sensor; To function as, The program according to any one of claims 1 to 7.

9. A system including a controller and a device, The controller It has a mouse sensor and an inertial sensor, configured to transmit the output of the mouse sensor and the output of the inertial sensor; The device comprises: receiving an output of the mouse sensor and an output of the inertial sensor; controlling an object using the output of the mouse sensor; determining whether the controller has transitioned from an ungrounded state to a grounded state; determining that an execution condition is satisfied, the execution condition including at least that the controller has transitioned from an ungrounded state to a grounded state; configured to control the object using at least an output of the inertial sensor when the execution condition is satisfied. system.

10. Obtaining the output of the controller's mouse sensor; acquiring an output of an inertial sensor of the controller; controlling an object using the output of the mouse sensor; determining that the controller has transitioned from an ungrounded state to a grounded state; determining that an execution condition has been met, the execution condition including at least determining that the controller has transitioned from an ungrounded state to a grounded state; When the execution condition is satisfied, controlling the object using at least an output of the inertial sensor; Methods including:

Images