Information processing device, information processing method, program

Abstract

To provide a technique capable of calculating a calibration parameter for an inertial sensor including other-axis sensitivities in a user environment.SOLUTION: An information processing device includes: first obtaining means for obtaining, from an inertial sensor, inertia information representing an inertia value of the inertial sensor; second obtaining means for obtaining information on a position and an attitude of the inertial sensor; control means for controlling displaying means so as to display a guide that instructs, to a user, each of a plurality of motion patterns including the translational movement or rotational movement of the inertial sensor corresponding to a first axis and a second axis that are perpendicular to each other; and parameter obtaining means for obtaining parameters including a correlation level between the actual inertia value of the inertial sensor corresponding to the first axis and the inertia information corresponding to the second axis on the basis of the inertia information obtained through the plurality of motion patterns of the inertial sensor and the information on the position and the attitude.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 and Non-Patent Document 1 describe techniques for calculating calibration parameters (such as gain or bias) to calibrate the measured values ​​(inertial information) of an inertial sensor. Patent Document 1 describes a technique for calculating calibration parameters for angular velocity using a camera and an angular velocity sensor. Non-Patent Document 1 discloses a technique for calculating calibration parameters by moving an inertial sensor (accelerometer or angular velocity sensor) without using a camera.

[0003] However, in the technologies described in Patent Document 1 and Non-Patent Document 1, when calculating calibration parameters in a user environment, the user needs to properly move the inertial sensor (accelerometer or angular velocity sensor). However, the average user does not have knowledge of how to properly move an inertial sensor.

[0004] In contrast, Patent Document 2 describes an example in which the user estimates the gain and bias by moving the orientation of the inertial sensor by referring to the display on the screen. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] International Publication No. 2014 / 058565 [Patent Document 2] U.S. Patent Application Publication No. 2015 / 0286279 [Non-patent literature]

[0006] [Non-Patent Document 1] A. Pretto and G. Grisetti, "Calibration and performance evaluation of low-cost IMUs", In Proceedings of the 20th IMEKO TC4 International Symposium, September 2014, p.429-434 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, the technology described in Patent Document 2 cannot estimate the cross-axis sensitivity, which represents the correlation between the output value (actual value) for one axis and the measured value for another axis. For an inertial sensor to estimate position or orientation more accurately, calibration parameters including cross-axis sensitivity must be calculated.

[0008] Therefore, the present invention aims to provide a technology that enables the calculation of calibration parameters for an inertial sensor, including multi-axis sensitivity, in the user's environment. [Means for solving the problem]

[0009] One aspect of the present invention is, A first acquisition means for acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition means for acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, The system displays a guide to instruct the user on each of several motion patterns, including two translational motions: the translational motion of the inertial sensor corresponding to the first axis and the translational motion of the inertial sensor corresponding to the second axis perpendicular to the first axis, or two rotational motions: the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. Department Control means for controlling, A parameter acquisition means that acquires parameters including the degree of correlation between the actual inertial value of the inertial sensor corresponding to the first axis and the inertial information corresponding to the second axis, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, It has, The inertial sensor is attached to the user's hand. The display unit shows a composite image in which virtual objects are placed on an image captured from real space. The control means controls the display unit to display the guide by superimposing the guide points indicating the position where the hand should be moved onto the virtual object in the composite image. This is an information processing device characterized by the following features.

[0010] One aspect of the present invention is, A first acquisition step of acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition step of acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, The system displays a guide to instruct the user on each of several motion patterns, including two translational motions: the translational motion of the inertial sensor corresponding to the first axis and the translational motion of the inertial sensor corresponding to the second axis perpendicular to the first axis, or two rotational motions: the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. Department A control step that controls, Based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, the actual inertia of the inertial sensor corresponding to the first axis A parameter acquisition step of acquiring a parameter that includes the degree of correlation between the value and the inertia information corresponding to the second axis, It has, The inertial sensor is attached to the user's hand. The display unit shows a composite image in which virtual objects are placed on an image captured from real space. In the control step, the display unit is controlled to display the guide by superimposing the guide point indicating the position where the hand should be moved onto the virtual object in the composite image. This is an information processing method characterized by the following features. [Effects of the Invention]

[0011] According to the present invention, calibration parameters for an inertial sensor, including multi-axis sensitivity, can be calculated in the user's environment. [Brief explanation of the drawing]

[0012] [Figure 1] This is a hardware configuration diagram of the information processing system according to Embodiment 1. [Figure 2] This is a software configuration diagram of the control device according to Embodiment 1. [Figure 3] This is a diagram illustrating the information processing system according to Embodiment 1. [Figure 4] This is a flowchart of the parameter calculation process according to Embodiment 1. [Figure 5] This is a diagram illustrating the information stored in the storage device according to Embodiment 1. [Figure 6] This is a flowchart of the guide display process according to Embodiment 1. [Figure 7] This diagram illustrates the display process of the guide according to Embodiment 1. [Figure 8] This is a diagram illustrating the guide for Embodiment 1. [Figure 9] This is a diagram illustrating the movement according to the guide for Embodiment 1. [Figure 10] This is a diagram illustrating the information obtained according to Embodiment 1. [Modes for carrying out the invention]

[0013] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0014] (Regarding calibration parameters) First, let's explain the parameters (calibration parameters) related to the calibration of the inertial sensor.

[0015] Inertial Measurement Units (IMUs) are sensors used in a wide range of electronic devices that enable virtual reality (VR) and mixed reality (MR). Electronic devices include, for example, cameras, game consoles, drones, vehicles, or smartphones. Inertial sensors generally consist of an accelerometer and an angular velocity (gyroscope) sensor. By using inertial sensors, it is possible to calculate the attitude of the inertial sensor (the electronic device containing the inertial sensor) and even estimate the position of the inertial sensor in space.

[0016] In principle, an acceleration sensor measures acceleration [m / s²]. 2 Since the acceleration is measured, the change in position [m] can be calculated by integrating the acceleration twice with respect to time. On the other hand, the angular velocity sensor measures the angular velocity [deg / s], so the change in angle (attitude) [deg] can be calculated by integrating the angular velocity with respect to time.

[0017] However, it is known that the measured values ​​of inertial sensors used in consumer applications have errors compared to actual values. Therefore, in order to perform measurements with inertial sensors as accurately as possible, it is desirable to calibrate these measured values ​​(errors) before use.

[0018] The calibration parameters (gain, bias, and multi-axis sensitivity) used to calibrate the measured acceleration values ​​(measurement errors) of an inertial sensor are explained using Equation 1.

number

[0019] In equation 1, a x ,a y ,a z This shows the measured acceleration values ​​in the three mutually perpendicular axes (X, Y, and Z axes) of the accelerometer. In contrast, a' x ,a' y ,a' zrepresents the acceleration (actual acceleration) obtained by calibrating (correcting) the measured value according to Equation 1. g axx , g ayy , g azz represents the gain (scale - factor) of each axis. The gain is the degree of correlation between the value to be output (actual value) of the acceleration in a certain axis direction and the measured value of the acceleration in that certain axis direction. b ax , b ay , b az represents the bias. The bias is a fixed value that does not depend on the measured value of the acceleration in the output value (actual value) of the acceleration in a certain axis direction.

[0020] Also, g axy , g axz , g { ayx, g ayz, g azx, g azy represents the cross - axis sensitivity (misalignment - factor, or cross - axis sensitivity). The cross - axis sensitivity is the degree of correlation between the actual value of the acceleration in a certain axis direction and the measured value of the acceleration in another axis direction. For example, the cross - axis sensitivity g axy represents the degree of correlation between the measured value of the acceleration in the Y - axis direction and the value to be output (actual value) of the acceleration in the X - axis direction. Ideally, the cross - axis sensitivity should be 0, but in consumer - grade acceleration sensors, it is rare for each of the three axes to be perfectly orthogonal, and due to slight inclinations and other reasons, the cross - axis sensitivity is not 0.

[0021] [[ID=

[34] ]According to Equation 1, it can be seen that the actual three - axis accelerations can be calculated from the measured values of the three - axis accelerations and the calibration parameters (gain, bias, and cross - axis sensitivity) (that is, the measured values can be accurately calibrated).

[0022] The calibration parameters (gain, bias, and cross - axis sensitivity) for calibrating the measured value (measurement error) of the angular velocity of the inertial sensor will be explained using Equation 2. ]

Number

[0023] In equation 2, ω x ,ω y ,ω z This shows the measured angular velocity of the angular velocity sensor around three mutually perpendicular axes (X, Y, and Z axes). In contrast, ω' x ,ω' y ,ω' z This shows the angular velocity (actual angular velocity) after calibrating (correcting) the measured value according to Equation 2. ωxx ,g ωyy ,g ωzz This indicates the gain for each axis. Gain is the degree of correlation between the output value (actual value) of the angular velocity in the rotational direction of a certain axis and the measured value of the angular velocity in the rotational direction of that axis. ωx ,b ωy ,b ωz This indicates bias. Bias is a fixed value that does not depend on the measured angular velocity in the direction of rotation of a certain axis, in the output value (actual value) of the angular velocity.

[0024] Also, g ωxy ,g ωxz ,g ωyx, g ωyz, g ωzx, g ωzy This indicates inter-axis sensitivity. Inter-axis sensitivity is the degree of correlation between the output value (actual value) of the angular velocity in the rotation direction of one axis and the measured value of the angular velocity in the rotation direction of other axes. In consumer-grade angular velocity sensors, it is rare for all three axes to be perfectly orthogonal, and there is a slight tilt, so the inter-axis sensitivity is not zero.

[0025] According to Equation 2, the actual angular velocity around the three axes can be calculated from the measured values ​​of the angular velocity around the three axes and the calibration parameters (gain, bias, and sensitivity of the other axes). angular velocity This indicates that it is possible to calculate (i.e., that the measured value can be accurately calibrated).

[0026] Furthermore, generally speaking, the acceleration gain is equal to the standard gravitational acceleration (gravitational acceleration at 45 degrees North / South latitude: 9.80665 m / s²). 2The calculation is based on [ ]). Therefore, if a user uses the inertial sensor in a location with a significantly different latitude after factory calibration, the acceleration gain value will no longer be appropriate. For this reason, it is desirable that the measured values ​​can be calibrated in the user's environment as well.

[0027] <Embodiment 1> Figure 1 is a configuration diagram showing an example of the hardware configuration of an information processing system 100 (calibration device) according to Embodiment 1. The information processing system 100 includes an inertial sensor 101, a position and attitude sensor 102, a storage device 103, a display device 104, a control device 105, and a transmission line 106.

[0028] The inertial sensor 101 is an IMU (Inertial Measurement Unit) that includes an acceleration sensor and an angular velocity sensor. The inertial sensor 101 is held, for example, by the user's hand (attached to the user's finger). The inertial sensor 101 measures the acceleration in each of three mutually orthogonal axes and the angular velocity around each of those three axes. The inertial sensor 101 continuously measures acceleration and angular velocity (inertial information) while in operation. The inertial sensor 101 can then convert the measured acceleration and angular velocity (inertial information) into digitized numerical data and output it.

[0029] The position and orientation sensor 102 calculates its own position and orientation (the position and orientation of the position and orientation sensor 102 itself).

[0030] The position and orientation sensor 102 includes, for example, an imaging device. The position and orientation sensor 102 uses the imaging device to recognize a characteristic pattern (such as a black and white marker) placed at a specific position within the range of the object to be imaged. The position and orientation sensor 102 then uses the imaging device to determine the marker's position in the captured image. Based on the position and orientation (position and orientation), the relative position and orientation of the imaging device (position and orientation sensor 102) with respect to the marker is calculated. Then, the position and orientation sensor 102 calculates its absolute position and orientation based on the calculated relative position and orientation and the predetermined absolute position of the marker. Alternatively, instead of a marker, the position and orientation sensor 102 may use the intersection of multiple lines or other features as feature points, and determine the relative position and orientation of the imaging device (position and orientation sensor 102) with respect to these feature points based on the feature points in the captured image. In other words, the position and orientation sensor 102 performs what is known as SLAM (Simultaneous Localization). You may also use the technology of (And Mapping).

[0031] Furthermore, the position and attitude sensor 102 may calculate its own position and attitude using, for example, an optical sensor (an optical sensor that measures the position of an optical marker) installed in the corner of a room. The position and attitude sensor 102 may also calculate its own position and attitude using a mechanical method. The position and attitude sensor 102 can be any device that can calculate its own position and attitude, such as a device that uses a mechanical method to read values ​​from an encoder or the like.

[0032] Furthermore, the position and orientation sensor 102 does not need to be included in the information processing system 100; it may be included in a separate device that is physically separated from the information processing system 100.

[0033] The storage device 103 is a storage medium (such as a semiconductor memory, hard disk, or solid-state drive). The storage device 103 stores (stores) information (programs and data) for the information processing system 100 to process. The information stored in the storage device 103 includes, for example, inertial information (information on acceleration and angular velocity measured by the inertial sensor 101) and position and orientation information (information acquired by the position and orientation sensor 102). In the following, some of the programs and data may be stored in an external storage medium (for example, a data server, network storage, or external memory) instead of the storage device 103.

[0034] The display device 104 is a display that shows images. The display device 104 displays a composite image created by combining a virtual object (CG) with an image captured by the camera of the real world.

[0035] The control device 105 has a processor. Specifically, the control device 105 has a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), etc. The control device 105 realizes various processes of the information processing system 100 by executing programs stored in the storage device 103 or other storage media. The control device 105 also generates a composite image by combining a virtual object (CG) with an image captured by the camera of the real space (the space in front of the user).

[0036] The transmission path 106 interconnects the inertial sensor 101, the position and orientation sensor 102, the storage device 103, the display device 104, and the control device 105. The transmission path 106 does not necessarily connect each component in close physical proximity. The transmission path 106 is not limited to internal wiring (bus) of a specific device, but may also be a transmission path using a LAN, WAN, or the Internet. Furthermore, information output from each component may be transmitted to other components via the transmission path 106 without delay, or the control device 105 may buffer (temporarily store) the information in a file or memory before transmitting it to other components at a desired timing.

[0037] (Internal configuration of the control unit) Figure 2 is a configuration diagram showing an example of the logical configuration (software configuration) of the control device 105. The control device 105 includes an inertial information acquisition unit 111, a position and orientation acquisition unit 112, a memory control unit 113, a guide information generation unit 114, a display control unit 115, and a parameter acquisition unit 116. At least a part of each logical configuration of the control device 105 may be included in any of the inertial sensor 101, position and orientation sensor 102, storage device 103, or display device 104.

[0038] The inertial information acquisition unit 111 acquires measured values ​​of inertia (acceleration and angular velocity) from the inertial sensor 101 as inertial information. Since the inertial sensor 101 measures acceleration and angular velocity for each of the three axes, the inertial information includes acceleration and angular velocity information (inertia value information) for each of the three axes.

[0039] The position and orientation acquisition unit 112 calculates (acquires) the position and orientation of the inertial sensor 101 based on the image (captured image) captured by the camera (imaging device) of the position and orientation sensor 102 and the position and orientation of the position and orientation sensor 102 calculated by the position and orientation sensor 102.

[0040] The memory control unit 113 stores in the storage device 103 information including the inertial information acquired by the inertial information acquisition unit 111 and the measurement time (timestamp) of the inertial information. The memory control unit 113 also stores in the storage device 103 information including the position and orientation information of the inertial sensor 101 acquired by the position and orientation acquisition unit 112 and the acquisition time (timestamp) of the position and orientation information. The memory control unit 113 may store the acquired information in the storage device 103 each time that each component of the control device 105 acquires (generates) information. Alternatively, the memory control unit 113 may store the information in the storage device 103 at arbitrary time intervals, or at arbitrary data amounts. In addition, the memory control unit 113 may store in the storage device 103 the temperature information of the inertial sensor 101 at the time the inertial sensor 101 measured the inertial information, along with the above information.

[0041] The guide information generation unit 114 generates guide information to instruct the user to perform multiple patterns of motion of the inertial sensor 101. These multiple motion patterns include translational motion of the inertial sensor 101 in multiple axial directions and rotational motion of the inertial sensor 101 around multiple axes. The guide information includes multiple guides to instruct the user (experiencer) to perform the multiple motion patterns. Each guide instructs the user on where to move the inertial sensor 101 (the hand wearing the inertial sensor 101) or what posture to change it to.

[0042] The display control unit 115 sequentially displays multiple guides included in the guide information generated by the guide information generation unit 114 on the display device 104. The user performs translational and rotational motion of the inertial sensor 101 according to the guides displayed on the display device 104.

[0043] The parameter acquisition unit 116 calculates calibration parameters for the inertial sensor 101 based on the inertial information acquired by the inertial information acquisition unit 111 and the position and orientation of the inertial sensor 101 acquired by the position and orientation acquisition unit 112. The calibration parameters include the multi-axis sensitivity (misalignment-factor) of acceleration and angular velocity. The calibration parameters are stored in the storage device 103. The calibration parameters are used when the parameter acquisition unit 116 calibrates the inertial information (for example, when calibrating based on the inertial information in equations 1 and 2).

[0044] Figure 3A is a conceptual diagram of the information processing system 100. The information processing system 100 includes an HMD 122 and a controller 124. The HMD 122 is a head-mounted display. The HMD 122 has, for example, a position and orientation sensor 102, a storage device 103, a display device 104, and a control device 105. Therefore, the HMD 122 can be said to be both a display device and an information processing device. At least one of the position and orientation sensor 102, storage device 103, display device 104, and control device 105 may be contained within a processing device (information processing device) that can communicate with the HMD 122.

[0045] The controller 124 is a device that can be worn on the user's finger. The shape of the controller 124 is, for example, a ring shape (finger ring shape) so that it can be worn on the user's finger. The controller 124 has an inertial sensor 101.

[0046] In Figure 3A, user 121 is wearing an HMD 122. The HMD 122 has one or more cameras that capture images of the area in front of user 121 (the original field of view). A controller 124 is attached to user 121's hand 123. Therefore, when the hand 123 moves, the controller 124 moves as well, and the inertial sensor 101 in the controller 124 acquires inertial information corresponding to the movement of the hand 123.

[0047] (Parameter calculation process) Referring to the flowchart shown in Figure 4, the process for calculating the calibration parameters of the inertial sensor 101 (parameter calculation process) will be explained. The parameter calculation process is performed in the user's environment when the user first uses (starts up) the inertial sensor 101 (controller 124).

[0048] In step S131, the inertial information acquisition unit 111 acquires inertial information (measured values) measured by the inertial sensor 101. The inertial information is obtained in three mutually perpendicular axes (X axis, Y axis, and) as shown in Figure 3B. Z This information includes acceleration in the axial direction and angular velocity around each of the three axes. The inertial information acquisition unit 111 periodically acquires inertial information, for example, every 1 / 200 second (at 200 Hz).

[0049] In step S132, the position and attitude acquisition unit 112 calculates (acquires) the position and attitude of the inertial sensor 101. For example, the position and attitude acquisition unit 112 calculates the position and attitude of the inertial sensor 101 by analyzing an image (captured image) taken of the inertial sensor 101 by a camera mounted on the HMD 122 (position and attitude sensor 102).

[0050] The position and attitude acquisition unit 112 can acquire the position and attitude (self-position and attitude) of the HMD 122 from the position and attitude sensor 102. Furthermore, if a specific marker is pre-installed on the inertial sensor 101, the position and attitude acquisition unit 112 can calculate the relative position and attitude of the HMD 122 (position and attitude sensor 102) with respect to the specific marker based on the captured image of the specific marker. Therefore, the position and attitude acquisition unit 112 can calculate the position and attitude of the inertial sensor 101 based on the position and attitude of the HMD 122, the relative position and attitude of the HMD 122 with respect to the specific marker, and the known relative position and attitude of the inertial sensor 101 with respect to the specific marker. In this way, the position and attitude acquisition unit 112 extracts (tracks) the marker in the captured image and calculates the position and attitude of the inertial sensor 101. The position and attitude acquisition unit 112 calculates the position and attitude of the inertial sensor 101, for example, every 1 / 60 second (at 60 Hz). Furthermore, the relative position and orientation of the HMD122 with respect to a specific marker can also be calculated based on matching a pre-stored model representing the appearance of the position and orientation sensor 102 with the position and orientation sensor 102 in the captured image. In addition, the relative position and orientation of the HMD122 (position and orientation sensor 102) with respect to a specific marker can also be calculated based on the determination of the size and shape of the hand in the captured image (hand tracking).

[0051] In step S133, the memory control unit 113 stores in the storage device 103 a combination of inertial information measured by the inertial sensor 101 and the measurement time (timestamp) of the inertial information. The memory control unit 113 also stores in the storage device 103 a combination of the position and orientation information of the inertial sensor 101 acquired by the position and orientation acquisition unit 112 and the measurement time of said position and orientation (imaging time of the captured image; timestamp). In Embodiment 1, as shown in Figures 5A and 5B, the memory control unit 113 stores information on the acceleration and angular velocity of the three axes, and information on the position and orientation of the inertial sensor 101.

[0052] The information stored in the storage device 103 will be explained in detail using Figures 5A and 5B. As shown in Figure 5A, inertial information (3-axis acceleration sensor values ​​142 and 3-axis angular velocity 143 The information (and the timestamp 141 of the inertial information) is stored in the storage device 103 each time inertial information is acquired (for example, at a frequency of 200 times per second). Also, as shown in Figure 5B, the position and orientation 145 of the inertial sensor 101 and the timestamp 144 of the time the camera took an image to calculate the position and orientation are stored in the storage device 103 each time the position and orientation is calculated (for example, at a frequency of 60 times per second).

[0053] The position and orientation 145 is represented by a 4x4 matrix. Specifically, the position and orientation 145 includes a 3x3 matrix 146 which is a three-dimensional rotation matrix (i.e., represents the orientation), and a 3x1 matrix 147 which represents the position in each of the three axes (the amount of movement from the reference position). Note that the information shown in Figures 5A and 5B does not necessarily have to be stored in the memory device 103 semi-permanently. The memory control unit 113 may delete unnecessary information from the memory device 103 sequentially, organize it by methods such as compression, or move it to another memory unit via communication.

[0054] Furthermore, the processes in steps S131 to S133 are repeatedly executed even while the processes from step S134 onward are being carried out.

[0055] In step S134, the guide information generation unit 114 generates information on multiple motion patterns (hereinafter referred to as "motion sets") that include translational motion in each of the three axes (X-axis, Y-axis, and Z-axis directions corresponding to the three-axis acceleration sensors) and rotational motion in each of the three axes. In the state shown in Figure 3A, for example, as shown in Figure 3B, the X-axis direction is the depth direction (front-to-back direction) as seen from the user, and the Y-axis direction is the lateral direction. The Z-axis direction is the vertical direction (vertical direction; direction of gravity). In Embodiment 1, the motion sets include translational motion in the X-axis, Y-axis, and Z-axis directions corresponding to the three-axis acceleration sensors, and rotational motion in the yaw direction (around the Z-axis), pitch direction (around the Y-axis), and roll direction (around the X-axis). The guide information generation unit 114 then generates guide information that includes multiple guides for instructing the user on each motion in the motion sets. Note that the guide information may be generated in advance before the start of this flowchart and stored in the storage device 103.

[0056] Figures 8A to 8H illustrate the multiple guides included in the guide information. Figures 8A to 8E each show a guide that instructs the movement of a user wearing the inertial sensor 101 by superimposing a round dot (a guide point indicating the position where the user moves their hand) and an arrow (an arrow indicating the direction in which the user moves their hand) onto the CG of a cube. In other words, in Figures 8A to 8E, the guide includes a cube, a guide point, and an arrow. Specifically, Figures 8A to 8E instruct the user to touch the guide point with their hand and then change the position and orientation of their hand according to the arrow. On the other hand, Figures 8F and 8G show a guide that instructs the user to move to a specific position using a message (text) representing the instruction and a cube. Note that even when displaying guide points and arrows, a message representing the instruction may also be displayed, as shown in Figure 8H. Note that in Embodiment 1, since the inertial sensor 101 moves when the user moves, it can also be said that the movement of the inertial sensor 101 is being instructed when the user's movement is instructed.

[0057] In step S135, the display control unit 115 sequentially displays multiple guides containing the guide information on the display device 104. At this time, the user performs the motion of the inertial sensor 101 according to the displayed guides.

[0058] In step S136, the parameter acquisition unit 116 calculates calibration parameters for the inertial sensor 101 based on the inertial information and position / orientation information of the inertial sensor 101 when the user moves the inertial sensor 101 according to the guide. In Embodiment 1, the calibration parameters for the inertial sensor 101 are the gains of acceleration and angular velocity, the multi-axis sensitivity and biaxial Includes as.

[0059] For example, the parameter acquisition unit 116 assumes that the angular velocity and acceleration obtained from the position and attitude information of the inertial sensor 101 acquired by the position and attitude acquisition unit 112 are accurate values, and calculates the correspondence between the angular velocity and acceleration indicated by the inertial information and their accurate values. Then, the parameter acquisition unit 116 calculates calibration parameters based on the calculated correspondence. A detailed example of the processing in step S136 will be described later.

[0060] (Guide display processing) Referring to the flowchart in Figure 6, the process in step S135 (guide display process) will be explained in detail.

[0061] In step S150, the display control unit 115 displays a composite image on the display device 104 in which the cube 161 (see Figure 7), which is a computer graphics used for calibration, is placed in real space (the space represented by the captured image). Here, the display control unit 115 places the cube 161 at the position in the three-dimensional space represented by the composite image based on the current position and orientation of the inertial sensor 101. The display control unit 115 also displays a guide on the display device 104 instructing the user to move to the front position 162 of the cube 161 as shown in Figure 7.

[0062] In step S151, the display control unit 115 selects the next guide to display from among the multiple guides included in the guide information. At this time, the guide information may include information indicating the order in which the multiple guides should be displayed, or the display control unit 115 may determine the order randomly. The display control unit 115 then displays the selected guide on the display device 104. In the following example, we will explain assuming that the guide information includes seven guides.

[0063] When step S151 is performed for the first time, the display control unit 115 selects a guide 171 that instructs the translational motion of the inertial sensor 101 in the Z-axis direction, as shown in Figure 8A. The guide 171 displays two guide points 171A and 171B on the front surface 161A of the cube 161. The guide 171 instructs the user to trace the arrow indicated by a dashed line with their hand from the lower guide point 171A to the upper guide point 171B. As shown in Figure 9A, with the inertial sensor 101 attached, the user touches the guide point 171A with their hand, moves upward, and stops at the guide point 171B, as instructed by the guide 171. This causes the translational motion of the inertial sensor 101 in the Z-axis direction.

[0064] If the user proceeds to step S151 for the second time, the display control unit 115 selects a guide 172 that instructs the inertial sensor 101 to perform a rotational movement in the roll direction, as shown in Figure 8B. The guide 172 instructs the user to perform one or more clockwise and counterclockwise rotations with their hand facing the front 161A of the cube 161. Following the guide 172, the user rotates their hand as if waving it between guide point 172A and guide point 172B, as shown in Figure 9B. This causes the inertial sensor 101 to perform a rotational movement in the roll direction.

[0065] If the user proceeds to step S151 for the third time, the display control unit 115 displays a guide 176, as shown in Figure 8F, which instructs the user to perform a rotational movement in the yaw direction of the inertial sensor 101. The guide 176 instructs the user to move from the front position 162 shown in Figure 7 to position 163 (the front position of the right side 161B) (instructing the user to move to a location where they can observe the cube from the right side). At this time, the user moves to a position with their hand pointed towards the cube 161 (while looking at the cube 161). This causes the inertial sensor 101 to perform a rotational movement in the yaw direction.

[0066] In this way, the device is placed in a fixed position at a distance greater than a certain distance from the user (for example, more than 1 meter). A guide is displayed (generated) that instructs the user to rotate around the placed cube 161. This allows calibration parameters to be calculated without relying on movements that make it difficult to guarantee the accuracy of calculating the position and orientation of the inertial sensor 101 using the camera (movement in which the user rotates around their own position). Therefore, even users who are not knowledgeable about movements suitable for calibration can calculate calibration parameters with guaranteed accuracy.

[0067] If the user proceeds to step S151 for the fourth time, the display control unit 115 selects a guide 173 that instructs the inertial sensor 101 to perform a rotational movement in the roll direction, as shown in Figure 8C. Guide 173, like guide 172, instructs the user to perform one or more clockwise and counterclockwise rotations with their hand facing the right side 161B of the cube 161. Following guide 173, the user rotates their hand by waving it between guide point 173A and guide point 173B, as shown in Figure 9C. This causes the inertial sensor 101 to perform a rotational movement in the roll direction.

[0068] If the user proceeds to step S151 for the fifth time, the display control unit 115 selects a guide 174 that instructs the Z-axis translational motion of the inertial sensor 101, as shown in Figure 8D. The guide 174 instructs the user to move their hand downwards according to the guide points 174A, 174B and arrows displayed on the right side 161B. As shown in Figure 9D, the user moves their hand to guide point 174A, and then moves their hand downwards toward guide point 174B. This causes the Z-axis translational motion of the inertial sensor 101 to occur.

[0069] If the user proceeds to step S151 for the sixth time, the display control unit 115 selects a guide 177 that instructs the inertial sensor 101 to rotate in the pitch direction, as shown in Figure 8G. As shown in Figure 7, the guide 177 moves from position 164 to position 163 (a position closer to the cube 161 than position 164) and instructs the user to look down at the cube 161 (top surface 161C). At this time, the user moves with their hand pointing to the center of the cube 161. This causes the inertial sensor 101 to rotate in the pitch direction.

[0070] If the user proceeds to step S151 for the seventh time, the display control unit 115 selects a guide 175 that instructs the X-axis and Y-axis translational motion of the inertial sensor 101, as shown in Figure 8E. The guide 175 instructs the user to sequentially touch the four corners of the top surface 161C of the cube 161 with their hand (finger). The user touches the guide points 175A, 175B, 175C, and 175D in that order, as shown in Figure 9E. In other words, the user moves their hand sequentially to the right, back, left, and front. This causes the X-axis and Y-axis translational motion of the inertial sensor 101 to occur.

[0071] In step S152, the memory control unit 113 stores the inertial information acquired by the inertial information acquisition unit 111 and the position and orientation information acquired by the position and orientation acquisition unit 112 during the processing of step S151 into the memory device 103 (see Figures 5A and 5B).

[0072] In step S153, the display control unit 115 determines whether the motion of the inertial sensor 101 corresponding to the guide displayed in step S151 has been completed. If it is determined that the motion of the inertial sensor 101 corresponding to the guide has been completed, the process proceeds to step S154. If it is determined that the motion of the inertial sensor 101 corresponding to the guide has not been completed, the process proceeds to step S152.

[0073] In step S154, the display control unit 115 determines whether the movements corresponding to all the guides (seven guides in this example) included in the guide information have been completed. If it is determined that the movements corresponding to all the guides have been completed, the process proceeds to step S136. If it is determined that any of the movements corresponding to any of the guides have not been completed, the process returns to step S151.

[0074] When the motion corresponding to all guides is completed, as shown in Figure 10, inertial information and information on the trajectory of the position and orientation of the inertial sensor 101 are obtained for each guide, during the motion corresponding to that guide. Based on the trajectory of the position and orientation of the inertial sensor 101, the acceleration and angular velocity of each axis of the inertial sensor 101 from the start to the end of the motion can be obtained. If the inertial sensor 101 performs motion corresponding to seven guides as in this example, translational motion and rotational motion corresponding to each of the three axes will be performed, as shown in Figure 10. Therefore, the calibration parameters of the inertial sensor 101 can be calculated appropriately.

[0075] (Detailed processing of step S136) An example of the calculation process for calibration parameters (gain, bias, and multi-axis sensitivity) performed by the parameter acquisition unit 116 in step S136 will be described below. Note that the following example is just one example of the calculation process (calculation method) for calibration parameters, and the calibration parameters may be calculated based on the inertial information and position / attitude trajectory information stored in the storage device 103 using other known methods.

[0076] First, the parameter acquisition unit 116 calculates the angular velocity bias for each axis direction by calculating the average of the difference between the angular velocity obtained from the change in position and attitude acquired by the position and attitude acquisition unit 112 and the angular velocity acquired by the inertial information acquisition unit 111, as shown in Equation 3. In Equation 3, b w ω represents the bias of the angular velocity of each axis. gyro ω represents the angular velocity value of each axis measured by the inertial sensor 101. cam This value is obtained by calculating the angular velocity from the difference between the position and attitude of the inertial sensor 101 at the start of the motion and at the end of the motion (or at each point in time during the motion), as calculated by the position and attitude acquisition unit 112.

number

[0077] The parameter acquisition unit 116 uses the Levenberg-Marquardt method, a type of nonlinear optimization method, to calculate calibration parameters other than the acceleration bias. Since the Levenberg-Marquardt method is publicly known (disclosed in Non-Patent Document 1, etc.), a detailed explanation will be omitted. However, to calculate the calibration parameters of the inertial sensor 101, an unknown parameter matrix needs to be defined. Then, a cost function that minimizes the error of each calibration parameter needs to be generated. For example, as shown in Equation 4, the parameter acquisition unit 116 defines an unknown parameter matrix θ for the acceleration sensor, which includes each calibration parameter as an element. acc Define . At this time, as mentioned above, g axx ,g ayy ,g azz g is the gain of acceleration, axy ,g axz ,g ayx ,g ayz ,g azx ,g azy This is multi-axis sensitivity, and b ax ,b ay ,b az This is a bias.

number

[0078] Subsequently, the parameter acquisition unit 116 calculates a cost function L(θ) to minimize the difference (error) between the acceleration calibrated by the calibration parameter and the actual acceleration, based on the inertial information. acc ) generates the cost function L(θ). acc The ) is determined based on the acceleration (inertial information) and the trajectory information of the position and orientation measured by the inertial sensor 101 for each motion pattern. The parameter acquisition unit 116 then uses the Levenberg-Marquardt method to determine the cost-effectiveness. The number L(θ acc The cost function L(θ) is minimized. acc The cost function L(θ) is optimized. acc The method for generating and optimizing the cost function L(θ) is publicly known (for example, described in Non-Patent Document 1), so it will not be described herein. The parameter acquisition unit 116 uses the optimized cost function L(θ) acc Calibration parameter corresponding to ) (i.e., g axx ,g ayy ,g azz, g axy ,g axz ,g ayx ,g ayz ,g azx ,g azy, b ax ,b ay ,b az ) obtain.

[0079] The parameter acquisition unit 116 can similarly acquire calibration parameters for angular velocity. First, the parameter acquisition unit 116 acquires an unknown parameter θ that includes calibration parameters other than the angular velocity bias as an element. gyro Define g as shown in Equation 5. In this case, as described above, ωxx ,g ωyy ,g ωzz This is the gain, and g ωxy ,g ωxz ,g ωyx ,g ωyz ,g ωzx ,g ωzy This is the multi-axis sensitivity.

number

[0080] The parameter acquisition unit 116 calculates the difference (error) between the value obtained by calibrating the angular velocity indicated by the inertial information with the calibration parameter and the actual angular velocity, and generates a cost function L(θ gyro ) for minimizing this difference. The cost function L(θ gyro ) is determined based on the angular velocity (inertial information) measured by the inertial sensor 101 and the angular velocity obtained from the trajectory information of the position and orientation for each motion pattern. Then, the parameter acquisition unit 116 optimizes the cost function L(θ gyro ) by the Levenberg-Marquardt method so that the cost function L(θ gyro ) becomes minimum. Note that since the method for generating the cost function L(θ gyro ) and the optimization are well-known (for example, described in Non-Patent Document 1), the description in this specification is omitted. The parameter acquisition unit 116 acquires the calibration parameter (that is, g gyro , g ωxx , g ωyy , g ωzz, g ωxy , g ωxz , g ωyx , g ωyz , g ωzx , g ωzy ) corresponding to the optimized cost function L(θ

[0081] The storage control unit 113 stores the calibration parameter acquired by the parameter acquisition unit 116 in the storage device 103. Thereby, the parameter acquisition unit 116 can use the calibration parameter when calibrating (correcting) the inertial information of the inertial sensor 101. Note that the process for calibrating the inertial information is the process as described using Expression 1 and Expression 2.

[0082] In Embodiment 1, the guide information includes guides that indicate translational motion in the three axes and rotational motion around the three axes. However, the guide information may include only guides that indicate translational motion in the three axes, or only guides that indicate rotational motion around the three axes. Furthermore, the guide information may include only guides that indicate translational motion in two of the three axes (translational motion in two axes perpendicular to each other). The guide information may include guides that indicate rotational motion around two of the three axes (translational motion around two axes perpendicular to each other). rotate It may include only guides that instruct each of the motions. In other words, any guide information can be used as long as it allows for the calculation of the multi-axis sensitivity of acceleration or angular velocity.

[0083] According to Embodiment 1, calibration parameters, including multi-axis sensitivity, can be calculated simply by the user moving the inertial sensor 101 according to the guide displayed on the display device 104. Therefore, calibration parameters of the inertial sensor, including multi-axis sensitivity, can be easily calculated in the user's environment.

[0084] In Embodiment 1, the position and orientation represent both position and orientation. However, if only the acceleration calibration parameters are calculated, the position and orientation may represent only the position. Also, if the angular velocity calibration parameters are calculated, the position and orientation may represent only the orientation. In other words, the position and orientation may represent either position or orientation.

[0085] <Example 1> Furthermore, the guide information generation unit 114 may generate a guide that instructs the inertial sensor 101 to move in a loop (to loop) in order to acquire the trajectory of the position and orientation of the inertial sensor 101. In this case, the position and orientation acquisition unit 112 calculates the position and orientation of the inertial sensor 101 by SLAM using a camera (i.e., by feature point tracking). When the position and orientation acquisition unit 112 detects that the motion is looping (loop motion) based on the captured image, it adjusts the information on the trajectory of the position and orientation of the inertial sensor 101 that it has acquired. Specifically, when the position and orientation acquisition unit 112 detects that the motion is looping, it adjusts (corrects) the trajectory of the position and orientation corresponding to the looping motion so that the trajectory of the position and orientation also forms a loop. Note that the trajectory of the orientation of the inertial sensor 101 forming a loop means that the position and orientation of the inertial sensor 101, which was a first position and orientation at the start of the motion or during the motion, returns to the first position and orientation again during the motion. Furthermore, the position and orientation acquisition unit 112 can detect, for example, that the motion between the acquisition times of two images is looping if the similarity between two images taken at different time intervals is above a threshold.

[0086] Then, the parameter acquisition unit 116 calculates (corrects) calibration parameters in accordance with the adjustment of the position and attitude trajectory (based on the adjusted position and attitude trajectory). According to Modification 1, since the accuracy of the position and attitude can be ensured by making the device perform a loop motion, the gain, multi-axis sensitivity, and bias can be calculated (corrected) with high accuracy.

[0087] <Modification 2> The guide information generation unit 114 may generate guide information such that a guide (first guide) showing motion including translational motion (and stationary motion) in the direction of gravity (vertical direction; Z-axis direction) of the inertial sensor is displayed before other guides (second guide). Other guides include, for example, translational motion in a direction perpendicular to the direction of gravity. In other words, the guide information generation unit 114 may instruct the display control unit 115 to display the first guide before other guides. The display control unit 115 may then display the first guide on the display device 104 before other guides. Here, the acceleration in the direction of gravity (gravitational acceleration) at a location differs depending on the latitude. Therefore, according to this variation 2, the approximate gain of the acceleration in the direction of gravity, which requires the most calibration, can be estimated (calculated) first, so the calculation accuracy of other calibration parameters performed later is less likely to decrease.

[0088] <Variation 3> When the position and orientation acquisition unit 112 calculates the position and orientation of the inertial sensor 101 using a camera, the guide information generation unit 114 may generate guide information that does not include guides indicating translational motion in the depth direction (front-back direction) as seen from the user (such as guide 175 in Figure 8E). In this case, the guide information includes guides indicating translational motion in the vertical and horizontal directions as seen from the user. According to Modification 3, when a camera is used to calculate the position and orientation of the inertial sensor 101, calibration parameters can be calculated without relying on information based on motion (translational motion in the depth direction as seen from the user) which makes it difficult to ensure the accuracy of position and orientation calculation. Therefore, calibration parameters can be calculated with high accuracy.

[0089] <Modification 4> Furthermore, the information processing system 100 determines that the current environment in which the inertial sensor 101 is installed is a suitable environment (measurement by the inertial sensor 101 and calculation of position and attitude by the position and attitude acquisition unit 112). The system may also have a monitoring unit that monitors (determines) whether the environment is suitable for the system. If the current environment is not suitable, for example, the inertial sensor 101 may have been subjected to a strong impact, or the ambient temperature around the inertial sensor 101 may have changed rapidly (when the rate of temperature change is above a predetermined value). If the current environment is not suitable, it may also be because the camera's movement or rotation speed is high, causing motion blur in the captured image, a moving object is traversing a wide area within the captured image, or the floor or wall where the user is located is not distinctive.

[0090] Furthermore, if the monitoring unit determines that the environment during the execution of multiple motion patterns is not suitable, it may instruct the display control unit 115 to display the guide again. In this case, the guide information generation unit 114 may generate new guide information according to the environment during the execution of multiple motion patterns. For example, if motion blur occurs in the captured image due to the camera's movement or rotation speed being fast, the guide information generation unit 114 generates guide information that includes a guide instructing the inertial sensor 101 to move slowly. In the case of an unsuitable environment, the parameter acquisition unit 116 may correct the calibration parameters or recalculate the calibration parameters based on the inertial information and position / orientation information acquired during the motion corresponding to the redisplayed guide. According to Modification 4, even if the environment at a particular time is considered to be an environment in which the accuracy of the calculated calibration parameters would be low, it is possible to calculate highly accurate calibration parameters.

[0091] Furthermore, in the above, the statement "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2" can be rephrased as "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2." Conversely, the statement "If A is greater than (higher than) B, proceed to step S1; if A is less than or equal to B, proceed to step S2" can be rephrased as "If A is greater than or equal to B, proceed to step S1; if A is less than (lower than) B, proceed to step S2." Therefore, as long as no contradiction arises, the expression "greater than or equal to A" can be replaced with "A or greater than (higher; longer; more)" or rephrased as "greater than (higher; longer; more)." On the other hand, the expression "less than or equal to A" can be replaced with "A or less than (lower; shorter; fewer)" or rephrased as "less than (lower; shorter; fewer)." Furthermore, "larger than A (higher; longer; more)" can be rephrased as "greater than or equal to A," and "smaller than A (lower; shorter; fewer)" can be rephrased as "less than or equal to A."

[0092] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0093] Furthermore, each functional unit in each of the above embodiments (each modified example) may or may not be individual hardware. The functions of two or more functional units may be implemented by common hardware. Each of the multiple functions of a single functional unit may be implemented by individual hardware. Two or more functions of a single functional unit may be implemented by common hardware. In addition, each functional unit may or may not be implemented by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. The functions of at least some of the functional units of the device may be implemented by the processor reading and executing the control program from the memory.

[0094] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0095] The above-disclosed embodiments include the following configurations, methods, and programs. [Configuration 1] A first acquisition means for acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition means for acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, Control means for controlling display means to display a guide instructing the user on each of a plurality of motion patterns, including two translational motions, such as the translational motion of the inertial sensor corresponding to a first axis and the translational motion of the inertial sensor corresponding to a second axis perpendicular to the first axis, or two rotational motions, such as the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. A parameter acquisition means that acquires parameters including the degree of correlation between the actual inertial value of the inertial sensor corresponding to the first axis and the inertial information corresponding to the second axis, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, An information processing device characterized by having the following features. [Configuration 2] The inertial sensor is attached to the user's hand. The display means displays a composite image in which virtual objects are placed on an image captured from real space. The control means controls the display means to display the guide by superimposing a guide point indicating the position where the hand should be moved onto the virtual object in the composite image. The information processing device according to configuration 1, characterized by the above. [Configuration 3] The parameter acquisition means acquires the parameter, which further includes at least one of the bias and gain of the inertial value of the inertial sensor, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns. An information processing device according to configuration 1 or 2, characterized by the above. [Structure 4] The second acquisition means acquires position and orientation information of the inertial sensor based on the captured image obtained by imaging the inertial sensor. An information processing device according to any one of configurations 1 to 3. [Composition 5] The aforementioned multiple patterns of motion include motion that traces a loop. When the second acquisition means detects the looping motion, it adjusts the position and orientation information of the inertial sensor so that the trajectory of the position and orientation of the inertial sensor corresponding to the looping motion traces a loop. An information processing apparatus according to any one of configurations 1 to 4, characterized by the above. [Composition 6] The aforementioned multiple motion patterns include a first translational motion that translates the inertial sensor in the vertical direction and a second translational motion that translates the inertial sensor in a direction perpendicular to the vertical direction. The control means controls the display means to display a guide instructing the user to perform the first translational motion, and then controls the display means to display a guide instructing the user to perform the second translational motion. An information processing device according to any one of configurations 1 to 5, characterized by the above. [Composition 7] The aforementioned multiple motion patterns do not include motion that translates the inertial sensor relative to the user's forward and backward direction. An information processing device according to any one of configurations 1 to 6. [Structure 8] The inertial sensor is attached to the user, The aforementioned multiple motion patterns include rotational motion, in which the user rotates around a fixed position located at a certain distance from the user, thereby rotating the inertial sensor. An information processing device according to any one of configurations 1 to 7, characterized by the above. [Composition 9] The system further includes monitoring means for determining whether the environment in which the inertial sensor is installed is a specific environment during the motion of the inertial sensor in the plurality of patterns, The control means controls the display means to display the guide again if the environment is the specific environment. The parameter acquisition means acquires the parameters based on the inertia information and position / orientation information acquired during the motion of the multiple patterns indicated by the guide that is displayed again. An information processing device according to any one of configurations 1 to 8. [method] A first acquisition step of acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition step of acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, A control step of controlling a display means to display a guide instructing the user on each of a plurality of motion patterns, including two translational motions, such as the translational motion of the inertial sensor corresponding to a first axis and the translational motion of the inertial sensor corresponding to a second axis perpendicular to the first axis, or two rotational motions, such as the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. A parameter acquisition step, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, to acquire parameters including the degree of correlation between the actual inertial value of the inertial sensor corresponding to the first axis and the inertial information corresponding to the second axis, An information processing method characterized by having the following features. [program] A program for causing a computer to function as one of the means of an information processing device described in any one of items 1 to 9 of the configuration. [Explanation of Symbols]

[0096] 100: Information processing system, 105: Control device, 111: Inertial information acquisition unit, 112: Position and orientation acquisition unit, 115: Display control unit, 116: Parameter acquisition unit

Claims

1. A first acquisition means for acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition means for acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, Control means for controlling a display unit to display a guide instructing the user on each of a plurality of motion patterns, including two translational motions, such as the translational motion of the inertial sensor corresponding to a first axis and the translational motion of the inertial sensor corresponding to a second axis perpendicular to the first axis, or two rotational motions, such as the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. Parameter acquisition means that acquires parameters including the degree of correlation between the actual inertial value of the inertial sensor corresponding to the first axis and the inertial information corresponding to the second axis, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, It has, The inertial sensor is attached to the user's hand. The display unit shows a composite image in which virtual objects are placed on an image captured from real space. The information processing apparatus is characterized in that the control means controls the display unit to display the guide by superimposing a guide point indicating the position where the hand should be moved onto the virtual object in the composite image.

2. The parameter acquisition means acquires the parameter, which further includes at least one of the bias and gain of the inertial value of the inertial sensor, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns. The information processing apparatus according to feature 1.

3. The second acquisition means acquires position and orientation information of the inertial sensor based on the captured image obtained by imaging the inertial sensor. The information processing apparatus according to feature 1.

4. The aforementioned multiple patterns of motion include motion that traces a loop. When the second acquisition means detects the looping motion, it adjusts the position and orientation information of the inertial sensor so that the trajectory of the position and orientation of the inertial sensor corresponding to the looping motion traces a loop. The information processing apparatus according to feature 1.

5. The aforementioned multiple motion patterns include a first translational motion that translates the inertial sensor in the vertical direction and a second translational motion that translates the inertial sensor in a direction perpendicular to the vertical direction. The control means controls the display unit to display a guide instructing the user to perform the first translational motion, and then controls the display unit to display a guide instructing the user to perform the second translational motion. The information processing apparatus according to feature 1.

6. The aforementioned multiple motion patterns do not include motion that translates the inertial sensor relative to the user's forward and backward direction. The information processing apparatus according to feature 1.

7. The inertial sensor is attached to the user, The aforementioned multiple motion patterns include rotational motion, in which the user rotates around a fixed position located at a certain distance from the user, thereby rotating the inertial sensor. The information processing apparatus according to feature 1.

8. The system further includes monitoring means for determining whether the environment in which the inertial sensor is installed is a specific environment during the motion of the inertial sensor in the plurality of patterns, The control means controls the display unit to display the guide again if the environment is the specific environment. The parameter acquisition means acquires the parameters based on the inertia information and position / orientation information acquired during the motion of the multiple patterns indicated by the guide that is displayed again. The information processing apparatus according to feature 1.

9. A first acquisition step of acquiring inertial information from the inertial sensor, which indicates the inertial value, which is the acceleration or angular velocity of the inertial sensor, A second acquisition step of acquiring position and orientation information indicating at least one of the position and orientation of the inertial sensor, A control step of controlling the display unit to display a guide instructing the user on each of a plurality of motion patterns, including two translational motions, such as the translational motion of the inertial sensor corresponding to a first axis and the translational motion of the inertial sensor corresponding to a second axis perpendicular to the first axis, or two rotational motions, such as the rotational motion of the inertial sensor corresponding to the first axis and the rotational motion of the inertial sensor corresponding to the second axis. A parameter acquisition step, based on the inertial information and position / orientation information acquired during the motion of the inertial sensor in the plurality of patterns, to acquire parameters including the degree of correlation between the actual inertial value of the inertial sensor corresponding to the first axis and the inertial information corresponding to the second axis, It has, The inertial sensor is attached to the user's hand. The display unit shows a composite image in which virtual objects are placed on an image captured from real space. In the control step, the display unit is controlled to display the guide by superimposing the guide point indicating the position where the hand should be moved onto the virtual object in the composite image. An information processing method characterized by the following:

10. A program for causing a computer to function as one of the means of an information processing apparatus according to any one of claims 1 to 8.

Images