Program, information processing device, and information processing method

Abstract

[Problem] Provided is a program, an information processing device, and an information processing method by which position information about a body part can be obtained with higher accuracy. [Solution] Provided is a program to cause a computer to perform a correction function of correcting position information about two or more installation parts of a body on which two or more motion sensors are installed, with reference to a first output obtained by performing a first process on sensor data acquired by a motion sensor and a second output obtained by performing a second process on the sensor data.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processor, and an information processing method. Background Art

[0002] In recent years, motion capture technology for acquiring body motion information has been actively developed. The acquired body motion information is used, for example, to improve exercise forms or forms in applications including VR (Virtual Reality) and AR (Augmented Reality).

[0003] In order to acquire body motion information, for example, the position of a motion sensor attached to the body is calculated from sensor data (sensing data) acquired by the motion sensor by using an inertial navigation system (INS) (eg, PTL 1 below).

[0004] Citation List

[0005] Patent Literature

[0006] PTL1: International Publication WO 2017 / 217050 Summary of the invention

[0007] Problems to be solved by the present invention

[0008] However, in an inertial navigation system, the calculation of the position is performed by integrating the acceleration included in the sensor data multiple times, so the error of the obtained position becomes larger over time. Therefore, a technology for obtaining position information about a part to which a motion sensor is attached with higher accuracy is required.

[0009] Means of solving the problem

[0010] According to the present disclosure, a program is provided that enables a computer to implement the following correction function, referring to a first output obtained by performing a first processing on sensor data acquired by two or more motion sensors attached to two or more parts of the body, and a second output obtained by performing a second processing on the sensor data, and correcting position information of the attachment part to which the motion sensor is attached.

[0011] In addition, according to the present disclosure, there is provided an information processor including a corrector that refers to a first output obtained by performing a first processing on sensor data acquired by two or more motion sensors attached to two or more parts of a body, and a second output obtained by performing a second processing on the sensor data, and corrects position information of the attachment part to which the motion sensor is attached.

[0012] In addition, according to the present disclosure, an information processing method is provided, including: a processor refers to a first output obtained by performing a first processing on sensor data acquired by two or more motion sensors attached to two or more parts of the body, and a second output obtained by performing a second processing on the sensor data; and the processor corrects the position information of the attachment part to which the motion sensor is attached.

[0013] Effects of the Invention

[0014] As described above, according to the present disclosure, position information on a portion to which a motion sensor is attached can be obtained with higher accuracy.

[0015] It should be noted that the above-described effects are not necessarily limitative, and any effects shown in the present specification or other effects that can be expected from the present specification can be obtained together with or instead of the above-described effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is an image diagram describing an overview of the first embodiment of the present disclosure.

[0017] Figure 2 is an explanatory diagram describing a conventional motion capture technique using forward kinematics calculation.

[0018] Figure 3 is an illustration describing the forward kinematics calculation.

[0019] Figure 4 This is an explanatory diagram describing a method of using inverse kinematics calculation.

[0020] Figure 5 is an explanatory diagram describing the inverse kinematics calculation.

[0021] Figure 6 is an explanatory diagram for describing the first step according to the embodiment.

[0022] Figure 7 is an explanatory diagram for describing the second step according to the embodiment.

[0023] Figure 8 FIG. 1 is a schematic diagram showing an overview of position estimation processing by an inertial navigation system.

[0024] Fig. 9 is a schematic diagram showing a time-varying image of a position error that may occur in a position estimation process based on an inertial navigation system.

[0025] Fig.10 is a schematic diagram showing an overview of position estimation processing based on a dynamic model.

[0026] Fig.11is a schematic diagram showing a time-varying image of a position error that may occur in a position estimation process based on a dynamic model.

[0027] Fig.12 is a schematic diagram showing an overview of a position estimation process according to the embodiment.

[0028] Fig.13 is a schematic diagram showing a time-varying image of a position error that may occur in a position estimation process according to an embodiment.

[0029] Fig.14 An example of a unit configuration of a system according to an embodiment is shown.

[0030] Fig.15 An example of a functional configuration of a system according to an embodiment is shown.

[0031] Fig.16 It is an explanatory diagram showing an example of a combination of an attachment site and a dynamics model.

[0032] Fig.17 An example of reliability identification performed by the reliability identifier 550 is shown.

[0033] Fig.18 is a flowchart illustrating an operation example of the system according to the embodiment.

[0034] Fig.19 A functional configuration example of a system according to a first modification example is shown.

[0035] Fig. 20 A functional configuration example of a system according to the second modification example is shown.

[0036] Fig.21 A functional configuration example of a system according to a third modification example is shown.

[0037] Fig. 22 is an explanatory diagram describing an overview of a non-tracking type kinetic model.

[0038] Fig.23 is an explanatory diagram describing an overview of a tracking type dynamics model.

[0039] Fig.24 A functional configuration example of a system according to a fourth modification example is shown.

[0040] Fig.25 A unit configuration example of a system according to a fifth modification example is shown.

[0041] Fig.26 : is an explanatory diagram (Part 1) describing the first ZUPT technique according to the second embodiment of the present disclosure.

[0042] Fig. 27 is an explanatory diagram (part 2) describing the first ZUPT technique according to the embodiment.

[0043] Fig.28 is an explanatory diagram (part 1) describing the second ZUPT technique according to the embodiment.

[0044] Fig.29 is an explanatory diagram (Part 2) describing the second ZUPT technique according to the embodiment.

[0045] Fig.30 An example of a functional configuration of a system according to an embodiment is shown.

[0046] Fig.31 is a flowchart illustrating an operation example of the system according to the embodiment.

[0047] Fig.32 : is an explanatory diagram describing the estimation of the front foot speed according to the embodiment.

[0048] Fig.33 is an explanatory diagram describing estimation of the forefoot position according to the embodiment.

[0049] Fig.34 : is an explanatory diagram for describing the ZUPT technique according to Modification Example 1 of the embodiment.

[0050] Fig.35 is an explanatory diagram for describing estimation of relative positions according to Modification Example 1 of the embodiment.

[0051] Fig.36 : is a flowchart showing an example of the operation of the system of Modification Example 1 of the embodiment.

[0052] Fig.37 An example of the unit configuration of the sensor unit 10 according to Modification Example 2 of the embodiment is shown.

[0053] Fig.38 : is an explanatory diagram describing the ZUPT technique according to Modification Example 2 of the embodiment.

[0054] Fig.39 is an explanatory diagram showing an example of a hardware configuration. DETAILED DESCRIPTION

[0055] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and thus redundant descriptions thereof are omitted.

[0056] In addition, in this specification and the drawings, sometimes different letters are added after the same reference numerals to distinguish multiple components having substantially the same functional configuration from each other. However, in the case where it is particularly unnecessary to distinguish multiple components having substantially the same functional configuration from each other, only the same reference numerals are assigned.

[0057] Note that the description will be given in the following order.

[0058] <<1. Overview of First Embodiment>>

[0059] <<2. Principles of the Prior Art>>

[0060] <<3. Configuration Example>>

[0061] <<4. Operation examples>>

[0062] <<5. Modification Example>>

[0063] <<6. Overview of Second Embodiment>>

[0064] <<7. Hardware Configuration Example>>

[0065] <<8. Conclusion>>

[0066] <<1. Overview of First Embodiment>>

[0067] For example, skeletal information represented by a skeletal structure representing the structure of a body is used to visualize information about the movement of a body of a person, an animal, or the like. The skeletal structure includes information about parts, and bones of line segments connecting the parts. The parts in the skeletal structure correspond to, for example, limb parts, joint parts, and the like, respectively. In addition, the bones in the skeletal structure may correspond to, for example, human bones; however, the positions and number of bones do not necessarily coincide with the positions and number of bones of an actual human body.

[0068] For example, the position of the part in the skeletal information can be obtained by attaching a marker or a motion sensor to each corresponding part of the body. For example, there is a technology of attaching a marker to each part of the body to obtain the position of the marker using an external camera, etc., and a technology of attaching a motion sensor to a part of the body to obtain position information of the motion sensor based on sensor data obtained by the motion sensor.

[0069] Compared with the technology using a marker attached to the body, the technology using a motion sensor attached to the body has the advantage of not requiring any external camera, light source or marker, making it easy to carry and set up. In addition, the technology using a motion sensor attached to the body is not limited by the image capture range of the camera, so it has an advantage in usability even in a wide outdoor space, a space with a complex shape that is prone to blind spots, a narrow and small space, etc.

[0070] However, in order to estimate the whole body skeleton using the technology using motion sensors attached to the body, the motion sensors must be attached to each joint, which sometimes puts a heavy burden on the user. In addition, although dedicated suits have also been developed to appropriately attach a large number of motion sensors to various parts of the user, the dedicated suits are sometimes expensive to manufacture.

[0071] Therefore, with the above situation in mind, the inventors have created the first embodiment of the present disclosure. The system according to the first embodiment of the present disclosure makes it possible to obtain position information of a greater number of parts than the number of attached motion sensors, while achieving a reduction in the number of motion sensors to be attached. It should be noted that in this specification, a motion sensor is a device that senses the movement of a body, and may include an inertial sensor (acceleration sensor, angular velocity sensor), a geomagnetic sensor, an air pressure sensor, an image sensor, and the like. In the following, an example in which at least an inertial sensor is attached to a body as a motion sensor will be mainly described.

[0072] Will refer to Figure 1 An overview of the present embodiment is described. Figure 1 is an image diagram describing the overview of this embodiment. Figure 1 In the example shown, six sensor units 10A to 10F are attached to six parts of the body of the user U1. The sensor units 10A to 10F include, for example, inertial sensors (IMU: Inertial Measurement Unit), such as an acceleration sensor that acquires acceleration (Acceleration) or a gyro sensor (angular velocity sensor) that acquires angular velocity (Angular speed). The sensor units 10A to 10F are preferably attached to a joint part used as a reference in the body (for example, the waist or the head) or to the proximal end of the body (wrist, ankle, head, etc.). Figure 1 In the example shown, sensor unit 10A is attached to the waist; sensor units 10B and 10E are attached to the wrist; sensor units 10C and 10D are attached to the ankles; and sensor unit 10F is attached to the head of user U1. Note that in the following description, sensor units 10A to 10F may be collectively referred to as sensor unit 10 without having to distinguish them from one another. Furthermore, hereinafter, the body part to which sensor unit 10 is attached may also be referred to as an attachment part. In addition, the number of sensor units 10 and the attachment positions (positions of attachment parts) are not limited to Figure 1 Those in the examples shown.

[0073] The system according to the present embodiment acquires information about the position (Position) and orientation (Orientation) of each attachment site based on the sensor data acquired by the sensor unit 10. As information about the attachment sites P101 to P106 to which the sensor units 10A to 10F are attached (hereinafter also referred to as attachment site information), Figure 1 The attachment location information PD100 including position information and orientation information is shown.

[0074] Furthermore, based on the position information and orientation information of the attachment sites P101 to P106 to which the sensor units 10A to 10F are attached, the system according to the present embodiment estimates skeletal information including position information and orientation information for each site of the skeletal structure. Here, in the present embodiment, among the sites of the skeletal structure, position information and orientation information are estimated not only for the attachment sites to which the sensor units 10 are attached (hereinafter also referred to as non-attachment sites) but also for sites to which the sensor units 10 are not attached.

[0075] exist Figure 1 In the example of FIG. 1 , skeleton information SD100 is shown, which includes position information and orientation information of each part in the skeleton structure. The skeleton information SD100 includes information of the non-attachment part SP107 and information of the attachment part SP101 corresponding to the attachment part P101 and the attachment part SP102 corresponding to the attachment part P102.

[0076] In addition to the information for the part, the skeleton information may also include information about the skeleton (position information, orientation information, etc.). Figure 1 In the example shown, the skeleton information SD100 may include information about the skeleton SB101. For example, based on the position information and orientation information of the parts in the skeleton structure, the information of the skeleton between the parts may be identified.

[0077] As described above, according to the present embodiment, information of non-attached parts to which no motion sensor is attached is estimated based on information of attached parts to which motion sensors are attached. This makes it possible to reduce the number of motion sensors to be attached while maintaining the number of parts whose information is to be acquired. The technical principle of the present embodiment for achieving the above-mentioned effect is described below.

[0078] <<2. Principles of the Prior Art>>

[0079] <2-1. Method of this embodiment>

[0080] As a conventional motion capture technology, there is a technology that acquires skeleton information through forward kinematics (FK) calculation. Forward kinematics calculation is a method of calculating the position of limb parts based on the orientation of joint parts.

[0081] Figure 2 An illustrative diagram depicting existing motion capture techniques using forward kinematics calculations. Figure 2 Skeletal information SD90 represented by a bone structure is shown.

[0082] exist Figure 2 In the skeleton information SD90 shown, the attachment part to which the sensor unit including the inertial sensor is attached is marked with a circle. Therefore, parts SP91 to SP96 are attachment parts. Here, in the case where the sensor unit includes an angular velocity sensor, the orientation information (angle information) of the attachment part can be identified by using an attitude measurement technology such as AHRS (Attitude and Heading Reference System).

[0083] For example, when the body coordinate system is set with the part SP91 corresponding to the waist of the body as the reference (origin), the position information of the parts SP92 to SP96 can be identified in the body coordinate system by performing forward kinematics calculation based on the orientation information of the parts SP91 to SP95. Note that in the following description, the part used as a reference in the body coordinate system is sometimes referred to as the root part.

[0084] Here, we will refer to Figure 3 Describes the forward kinematics calculation. Figure 3 is an illustration of the forward kinematics calculation. For simplicity, Figure 3 An example is shown where the arm is considered as a simple link with two degrees of freedom.

[0085] exist Figure 3 In the example shown, the part SP81 is the root part, and an xy coordinate system (plane coordinate system) with the part SP81 as a reference (origin) is shown. As described above, in the forward kinematics calculation, the orientation information of each part is known. Figure 3 In the example shown, the angle θ1 formed between the bone SB81 and the x-axis based on the orientation of the part SP81, and the angle θ2 formed between the bones SB81 and SB82 based on the orientation of the part SP82 are known. In addition, in the forward kinematics calculation, the distance between the parts, that is, the length of the bones, is also known. Figure 3 In the example shown, the length l1 of the bone SB81 and the length l2 of the bone SB82 are known.

[0086] According to the forward kinematics calculation, the position (x1, y1) of the part SP83 can be calculated using the known information as described above. Specifically, the position (x1, y1) of the part SP83 is represented by the following equations (1) and (2).

[0087] [Formula.1]

[0088] x1=l1cosθ1+l2cos(θ1+θ2)...(1)

[0089] y1=l1sinθ1+l2sin(θ1+θ2)...(2)

[0090] although Figure 3 An example of a planar and simple mechanism is shown, but even in the case of a three-dimensional and more complex mechanism, forward kinematics calculation can be performed in a similar manner. That is, by using the orientation information of the joint parts between the root part and the limb part as a reference and the information about the distance between the two parts, the position information of the limb part can be identified. In addition, by forward kinematics calculation, not only the position information of the limb part but also the bone structure (for example, in Figure 3 In the example shown, the position information of the part SP82 is the position information of the joint part located between the root part and the limb part.

[0091] An example of an existing motion capture technique using forward kinematics calculations has been described above. However, as already mentioned with reference to Figure 2 and Figure 3 As described, recognizing the position information of each part by using forward kinematics calculation requires orientation information of each joint part. Therefore, the use of forward kinematics calculation is a factor that causes an increase in the number of motion sensors to be attached to the body.

[0092] Therefore, in this embodiment, the skeleton information is acquired by using inverse kinematics (IK) calculation instead of the above forward kinematics calculation. Inverse kinematics calculation is a method of calculating the orientation of each joint part based on the position of the limb parts.

[0093] Figure 4 This is an explanatory diagram describing a method of using inverse kinematics calculation. Figure 4 Shown by a similar Figure 2 The bone structure represents the bone information SD90.

[0094] exist Figure 4 In the skeleton information SD90 shown, the attachment sites to which the sensor unit including the inertial sensor is attached are marked with circles. Therefore, sites SP91 to SP96 are attachment sites. Figure 4 In the skeleton information SD90 shown, the positions where any sensor unit including the inertial sensor is not attached are marked with a cross. Specifically, the parts SP92 to SP95 located between the part 91 and the part 96 are non-attached parts.

[0095] Here, it is assumed that the body coordinate system has been set with the part SP91 corresponding to the waist of the body as the reference (origin), and the position information of the part SP96 in the body coordinate system has been obtained. In this case, the orientation information and position information of the parts SP92 to SP95 can be identified by performing inverse kinematics calculation based on the position information of the part SP96 as the limb part.

[0096] Here, we will refer to Figure 5 Describes inverse kinematics calculations. Figure 5 is an illustration of the inverse kinematics calculation. For simplicity, Figure 5 An example is shown where the arm is considered as a simple link with two degrees of freedom.

[0097] exist Figure 5 In the example shown, the part SP81 is the root part, and an xy coordinate system (plane coordinate system) with the part SP81 as a reference (origin) is shown. Figure 5 As shown in the upper left corner, it is assumed that the length l1 of the bone SB81 with one endpoint being the part SP81 and the length l2 of the bone SP82 with one end being SP83 are known.

[0098] In the inverse kinematics calculation, as described above, the known information is used to calculate the angle formed between the bone SB81 and the x-axis and the angle formed between the bone SB81 and the bone SB82. However, in the inverse kinematics calculation, there may be a variety of solutions. Figure 5 In the example, the solution obtained is Figure 5 As shown in the upper right corner of FIG. , an angle θ is formed between the bone SB81 and the x-axis. 11 , and an angle θ is formed between bones SB81 and SB82 21 ; and the solution is, Figure 5 As shown in the lower right corner of FIG. , an angle θ is formed between the bone SB81 and the x-axis. 12 , forming an angle θ between bones SB81 and SB82 22 . It should be noted that Figure 5 Examples of flat and simple mechanisms are shown; therefore, a greater number of solutions may exist, for example in the case of three-dimensional mechanisms or in the case of a large number of joints.

[0099] The method using inverse kinematics calculation has been described above. Next, the overview of the processing according to the present embodiment will be described. In the present embodiment, as described above, inverse kinematics calculation is used to obtain skeleton information (position and orientation of joint parts). Therefore, the processing is as follows Figure 6 and Figure 7 The two general steps shown are performed.

[0100] Figure 6 is an explanatory diagram describing the first step according to this embodiment. Figure 1 As described, in the present embodiment, information about acceleration and angular velocity can be obtained from two or more inertial sensors attached to two or more parts of the body. In addition, information about acceleration (three-dimensional) and orientation (four-dimensional) in a global coordinate system can be obtained from the information about acceleration and angular velocity acquired from the inertial sensors. It should be noted that the global coordinate system is a coordinate system commonly used by, for example, multiple sensors, devices, etc., and can be a coordinate system corresponding to a real space. Figure 6 The attachment site information PD10 shown in φ includes information on the acceleration and orientation of the attachment sites P11 to P16 acquired by the inertial sensor, and thus includes 6×7=42-dimensional information in total.

[0101] In the first step, the position of the attachment part is estimated based on the attachment part information PD10 (S100), thereby obtaining the attachment part information PD12, which includes position information indicating the positions (three dimensions) of the attachment parts P11 to P16. It should be noted that in the case where the attachment part information PD12 is represented in the body coordinate system with the attachment part P11 as the root part, the position information for the attachment part P11 is not required, and therefore, the attachment part information PD12 includes a total of 5×3=15 dimensions of position information. In addition, the attachment part information PD12 may include the orientation information of each part in the global coordinate system included in the attachment part information PD10. In this case, the attachment part information PD12 includes a total of 6×4=24 dimensions of orientation information.

[0102] Figure 7 is an explanatory diagram for describing the second step according to this embodiment. Figure 7 The attachment site information PD12 shown is similar to Figure 6 The attachment location information PD12 is shown, so its description is omitted here.

[0103] In the second step, position estimation (interpolation) for non-attached parts is performed based on the attached part information PD12 (S200), thereby obtaining skeleton information SD10, which includes position information for non-attached parts to which no inertial sensors are attached, in addition to position information for attached parts. Figure 7 In the illustrated example, for example, the sites SP11 and SP12 are attachment sites, and the site SP20 is a non-attachment site.

[0104] Here, the position estimation for the non-attached part, i.e., the second step, can be performed by the inverse kinematics calculation as described above. In order to perform the position estimation for the non-attached part with high precision, it is desirable to obtain the attachment part information PD12 used as input with high precision in the first step. The first step, i.e., the position estimation of the attachment part, can be performed by, for example, an inertial navigation system. However, in the case of performing the position estimation of the attachment part by the inertial navigation system, the error of the estimated position will increase over time. Therefore, in the present embodiment, by correcting the position information of the attachment part estimated by the inertial navigation system, the position information of the attachment part can be obtained with high precision. The correction of the position information of the attachment part according to the present embodiment is described below.

[0105] <2-2. Correction of attachment site information>

[0106] First, a description will be given of a position estimation process based on an inertial navigation system. The inertial navigation system is a technique for calculating a sensor position by performing integration of angular velocity and acceleration multiple times, and is used in, for example, ships, airplanes, and the like. Figure 8 is a schematic diagram showing an overview of position estimation processing by an inertial navigation system.

[0107] In the inertial navigation system, first, the orientation of the sensor unit in the global coordinate system is calculated by integrating the angular velocity (an example of sensor data) in the local coordinate system acquired by the gyro sensor included in the sensor unit (S111). Next, in the inertial navigation system, based on the orientation of the sensor unit in the global coordinate system, the acceleration of the sensor unit in the local coordinate system (a coordinate system set for each sensor unit) acquired by the acceleration sensor included in the sensor unit (an example of sensor data) is converted into the acceleration of the sensor unit in the global coordinate system (S112). Then, the acceleration of the sensor unit in the global coordinate system generated by the coordinate system conversion is integrated (S113), thereby calculating the speed of the sensor unit in the global coordinate system. Next, the speed of the sensor unit in the global coordinate system is integrated (S114), thereby calculating the moving distance of the sensor unit. Here, by combining the moving distance of each subdivided point in the global coordinate system, the relative position information relative to the initial position (as the starting point) can be obtained. If the initial position is known, the absolute position information of the sensor unit (that is, the three-dimensional coordinates in the global coordinate system) can be calculated based on the above information. In the above-described manner, the orientation information and the position information of the sensor unit are output through the position estimation process based on the inertial navigation system.

[0108] based on Figure 8 The position estimation process of the inertial navigation system shown can be performed with a low processing load and at a high speed.

[0109] Here, in the inertial navigation system, the angular velocity is integrated only once in step S111 to obtain the orientation information, and further, by combining well-known techniques such as AHRS, the orientation information can be acquired with higher accuracy. On the other hand, in order to obtain the position information, the acceleration is integrated twice in steps S113 and S114. Therefore, if the acceleration acquired by the acceleration sensor includes an error, the error can be accumulated in the output position information.

[0110] Fig. 9 is a schematic diagram showing a time-varying image of position errors that may occur in a position estimation process based on an inertial navigation system. Fig. 9 As shown in , in the short time after the inertial navigation system starts estimating position information, the position error is small, so the position information can be estimated with high accuracy. Fig. 9 As shown, the error included in the position information estimated by the inertial navigation system increases with time. Therefore, if the estimation is performed continuously for a long time, the position information will include a very large error.

[0111] In order to solve this problem, in this embodiment, the position of the attachment site is estimated by regression based on a dynamics model. Fig.10 is a schematic diagram showing an overview of position estimation processing based on a dynamic model.

[0112] Fig.10 The processing in steps S121 and S122 shown in FIG. Figure 8 The processing in steps S111 and S112 described above is similar, so the description thereof is omitted here. Fig.10 As shown, in the position estimation process based on the dynamic model, the position information of the sensor unit is estimated by regression without integrating the acceleration (S123). In the regression estimation process in step S123, the position information is estimated by regression, wherein the orientation and acceleration of the sensor unit in the global coordinate system are suitable for the pre-prepared dynamic model. Here, for example, the dynamic model can be generated by pre-learning kinematic constraint information (for example, information including a plurality of samples of positions and orientations of different parts in a previously obtained posture or a series of movements). It should be noted that various regression analysis methods can be used for the regression estimation process in step S123, and for example, methods such as DNN (deep neural network) and random forest can be used alone or in combination.

[0113] Here, because Fig.10 The treatment shown does not involve integration of acceleration, so it is similar to reference Figure 8 and Fig. 9As is the case with the described inertial navigation system, the position error may grow larger over time. Fig.11 FIG. 1 is a schematic diagram showing a time-varying image of a position error that may occur in a position estimation process based on a dynamic model. Fig.10 As shown in FIG. 1 , although position error also occurs in the position estimation process based on the dynamic model, it does not become larger over time. Therefore, even if the process is continuously executed for a long time, it is unlikely to cause a large error.

[0114] Therefore, it is believed that based on Fig.10 The position estimation process of the dynamic model shown makes it possible to estimate position information with higher accuracy. However, since the position estimation process based on the dynamic model performs statistical estimation by regression, even if the input changes continuously, the output change (estimation result) may be discontinuous. As a result, for example, the visualized skeletal information finally obtained may tend to cause a sense of strangeness. In addition, when compared with the position estimation process based on the dynamic model, the position estimation process based on the dynamic model may be more accurate. Figure 8 The position estimation process of the inertial navigation system shown is compared to that based on Fig.10 The processing load of the position estimation process of the dynamic model shown is high, and it is difficult to achieve a higher execution speed than the position estimation process based on the inertial navigation system.

[0115] As described above, the position estimation process based on the inertial navigation system and the position estimation process based on the dynamic model have their own characteristics. Therefore, in the present embodiment, by combining the position estimation process based on the inertial navigation system and the position estimation process based on the dynamic model, and correcting the estimated position information, the position information is estimated with higher accuracy. Hereinafter, the position estimation process based on the inertial navigation system may be referred to as the first process, and the position estimation process based on the dynamic model may be referred to as the second process.

[0116] Fig.12 is a schematic diagram showing an overview of the position estimation process according to the present embodiment. It should be noted that Fig.12 The overview is shown, so the position estimation process according to this embodiment may further include Fig.12 Processing not shown.

[0117] like Fig.12As shown, in addition to the first processing (S111 to S114) based on the inertial navigation system and the second processing (S121 to S123) based on the dynamic model, the position estimation processing according to the present embodiment also includes a correction processing (S130). In the correction processing of step S130, correction is performed by referring to the output of the first processing (hereinafter, also referred to as the first output) and the output of the second processing (hereinafter, also referred to as the second output). It should be noted that the first output includes orientation information and position information for the attachment site, and the second output includes position information for it. Then, in step S130, the position information included in the first output is corrected based on the orientation information included in the first output and the position information included in the second output. Note that in step S130, the orientation information included in the first output can be used to correct the position information and output as it is. In addition, for example, the correction processing in step S130 can be implemented by a Kalman filter.

[0118] In addition, as described above, the first process is executed faster than the second process. Therefore, the correction process (S130) can be performed when the second output is obtained, and if the second output is not obtained and only the first output is obtained, the first output can be output as it is as the output of the position estimation process according to the present embodiment.

[0119] Fig.13 FIG. 1 is a schematic diagram showing a time-varying image of a position error that may occur in the position estimation process according to the present embodiment. Fig.13 In the example shown, the second output is obtained, and at time t 11 A correction based on the second output is performed. Fig.13 As shown, in the position estimation process according to the present embodiment, from the start of the process to time t 11 During the period of time, the position error increases with time; however, as based on the 11 As a result of the correction of the second output being performed, the position error is suppressed.

[0120] Therefore, according to the position estimation processing of the present embodiment, the position error is suppressed whenever correction is performed. Therefore, the error is less likely to become larger over time, and even if the processing is continuously performed for a long time, it is less likely to cause a large error. In addition, in the position estimation processing according to the present embodiment, in the case where the second output cannot be obtained, the first output is output as it is. This makes it possible to estimate position information more frequently compared to the case where position estimation is performed using only the second processing based on the dynamic model.

[0121] <<3. Configuration Example>>

[0122] The principle of the technology according to the present embodiment has been described above. Next, a configuration example of the present embodiment will be described. Hereinafter, a unit configuration example and a functional configuration example of the present embodiment will be described in sequence.

[0123] <3-1. Unit configuration example>

[0124] Fig.14 An example of a unit configuration of a system according to this embodiment is shown. Fig.14 As shown, the system according to the present embodiment includes sensor units 10A to 10F, a hub unit 20 , an information processor 30 , and an output unit 40 .

[0125] The sensor units 10 are units each including at least one motion sensor and to be attached to a body part. Figure 1 The parts to which the sensor units 10A to 10F are attached are described, and thus individual descriptions will be omitted. Fig.14 As shown, the sensor unit 10 includes a gyro sensor 12, an acceleration sensor 14, a controller 16, and a communicator 18. Note that the motion sensor included in the sensor unit 10 is not limited to the inertial sensor (gyro sensor 12 and acceleration sensor 14). For example, the sensor unit 10 may include a motion sensor such as a geomagnetic sensor, an air pressure sensor, an image sensor, etc. instead of or in addition to the inertial sensor. In addition, although Fig.14 Only the configuration of the sensor unit 10A is shown, but the sensor units 10A to 10F may have the same configuration.

[0126] The gyro sensor 12 is an inertial sensor that acquires an angular velocity as sensor data. The angular velocity acquired by the gyro sensor 12 may be an angular velocity of the sensor unit 10 in a local coordinate system.

[0127] In addition, the acceleration sensor 14 is an inertial sensor that acquires acceleration as sensor data. The acceleration acquired by the acceleration sensor 14 may be the acceleration of the sensor unit 10 in a local coordinate system set for each sensor unit 10 .

[0128] The controller 16 controls the operation of the sensor unit 10. For example, the controller 16 may control the communication performed by the communicator 18, and may cause the sensor data (angular velocity and acceleration) acquired by the gyro sensor 12 and the acceleration sensor 14 to be transmitted to the hub unit 20. Alternatively, the controller 16 may perform processing on the sensor data acquired by the gyro sensor 12 and the acceleration sensor 14, and may cause the processing result obtained by the processing to be transmitted to the hub unit 20.

[0129] The communicator 18 is a communication module for transmitting and receiving data to and from other units in a wired manner or wirelessly. For example, the communicator 18 wirelessly communicates with external devices directly or via a network access point in a scheme such as a wired LAN (Local Area Network), a wireless LAN, Wi-Fi (registered trademark) (Wireless Fidelity), infrared communication, Bluetooth (registered trademark), short-range / contactless communication, etc.

[0130] The hub unit 20 is an information processor that receives information from the plurality of sensor units 10 and aggregates the information. Note that the sensor units 10 and the hub unit 20 may be connected to each other in a wired manner or wirelessly. Fig.14 As shown, the hub unit 20 includes a communicator 22 and a controller 24 .

[0131] The communicator 22 is a communication module for sending and receiving data to and from other units in a wired or wireless manner. For example, the communicator 22 wirelessly communicates with external devices directly or via a network access point in a scheme such as a wired LAN, a wireless LAN, Wi-Fi, infrared communication, Bluetooth, short-range / contactless communication, etc.

[0132] The controller 24 controls the operation of the hub unit 20. For example, the controller 24 may control the communication performed by the communicator 22, and may cause the information received from the sensor unit 10 to be transmitted as it is to the information processor 30. Alternatively, the controller 24 may perform processing on the information received from the sensor unit 10, and may cause the processing result obtained by the processing to be transmitted to the information processor 30.

[0133] The information processor 30 is an information processor that receives information from the hub unit 20 and processes the information. Note that the hub unit 20 and the information processor 30 may be connected to each other in a wired manner or wirelessly. Fig.14 As shown, the information processor 30 includes, for example, a communicator 32 and a controller 34 .

[0134] The communicator 32 is a communication module for transmitting and receiving data to and from other units in a wired manner or wirelessly. For example, the communicator 32 wirelessly communicates with external devices directly or via a network access point in a scheme such as a wired LAN, a wireless LAN, Wi-Fi, infrared communication, Bluetooth, short-range / contactless communication, etc.

[0135] The controller 34 controls the operation of the information processor 30. For example, the controller 34 performs processing on the information received by the communicator 32. In addition, the controller 34 is able to control the display of the output unit 40 by controlling and causing the communicator 32 to send the aforementioned processing results to the output unit 40 connected to the information processor 30. For example, the controller 34 can cause the output unit 40 to display visualized skeleton information, a 3D model reflecting the skeleton information, etc. based on the skeleton information obtained as a result of the processing.

[0136] The output unit 40 is a unit having at least a display function, and performs display according to the control of the controller 34 of the information processor 30. The output unit 40 may be, for example, a mounted display or an HMD (head mounted display) to be worn by a user. In addition, the output unit 40 and the information processor 30 may be an integral unit.

[0137] <3-2. Functional Configuration Example>

[0138] The unit configuration example of the system according to the present embodiment has been described above. Next, a functional configuration example of the system according to the present embodiment will be described. Fig.15 An example of the functional configuration of the system according to the present embodiment is shown. Note that Fig.15 Only the functional configuration mainly representing the features of the present embodiment is shown, and the system according to the present embodiment may therefore include Fig.15 Functional configuration not shown in FIG.

[0139] Reference Fig.15 According to the present embodiment, the system has the functions of serving as a global motion acceleration orientation calculator 510, an integral calculator 520, a selector 530, an attachment position regression estimator 540, a reliability identifier 550, a ground contact detector 560, a corrector 570 and an interpolator 580.

[0140] Note that the following description Fig.15 Each function in the configuration can be Fig.14 For example, the controller 16 of the sensor unit 10 or the controller 24 of the hub unit 20 may have a function as the global motion acceleration orientation calculator 510, and the controller 34 of the information processor 30 may have other functions as the integral calculator 520 to the interpolator 580. However, which unit has which function is not limited to such an example.

[0141] The global motion acceleration direction calculator 510 has a Fig.14The sensor data (angular velocity and acceleration) acquired by the illustrated sensor units 10A to 10F are processed. Note that the global motion acceleration orientation calculator 510 can process a plurality of pieces of sensor data acquired by the sensor units 10A to 10F, respectively, independently of one another.

[0142] For example, the global motion acceleration orientation calculator 510 performs a process of integrating the angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data to calculate the orientation of the sensor unit 10 in the global coordinate system (corresponding to Fig.12 The global motion acceleration orientation calculator 510 may calculate the orientation of the sensor unit 10 through the AHRS. Note that in the case where the sensor unit 10 includes a geomagnetic sensor, the global motion acceleration orientation calculator 510 may use sensor data acquired by the geomagnetic sensor when calculating the orientation of the sensor unit 10 through the AHRS.

[0143] In addition, the global motion acceleration orientation calculator 510 performs a process of converting the orientation of the sensor unit 10 calculated in the global coordinate system into the orientation of the attachment site to which those sensor units 10 are attached in the global coordinate system. The global motion acceleration orientation calculator 510 can convert the orientation of the sensor unit 10 into the orientation of the attachment site by using the preliminary correction result.

[0144] Furthermore, based on the orientation of the attachment site in the global coordinate system, the global motion acceleration orientation calculator 510 performs a process of converting the acceleration in the local coordinate system contained in the sensor data into the acceleration in the global coordinate system (corresponding to Fig.12 1 and 122 in step S112 and S122).

[0145] In addition, the global motion acceleration orientation calculator 510 performs a process of removing the gravity component from the acceleration in the global coordinate system obtained as described above. For example, the global motion acceleration orientation calculator 510 may remove the gravity component from the acceleration in the global coordinate system based on the orientation of the attachment site in the global coordinate system.

[0146] The global motion acceleration orientation calculator 510 outputs the orientation information of the attachment part in the global coordinate system and the acceleration in the global coordinate system from which the gravity component has been removed obtained by the above processing to the integral calculator 520, the selector 530 and the attachment position regression estimator 540.

[0147] The integral calculator 520 performs a process of integrating twice the acceleration in the global coordinate system from which the gravity component is removed, which is input from the global motion acceleration direction calculator 510 (corresponding to Fig.12The integral calculator 520 outputs the calculated position information for the attachment part in the global coordinate system and the orientation information for the attachment part in the global coordinate system input from the global motion acceleration orientation calculator 510 to the corrector 570.

[0148] Note that the aforementioned first process based on the inertial navigation system can be performed by the above-mentioned global motion acceleration orientation calculator 510 and the integral calculator 520. In addition, the position information and orientation information output from the integral calculator 520 correspond to the first output.

[0149] The selector 530 selects a dynamic model for the attachment position regression estimator 540 to estimate the position of the attachment site by regression. For example, a plurality of dynamic models may be prepared according to the pattern of the site to which the sensor unit 10 is attached, and may be stored in a memory section (not shown). From such a plurality of dynamic models, the selector 530 selects a dynamic model to be used by the attachment position regression estimator 540.

[0150] For example, a dynamic model may be prepared in advance for each combination of attachment sites. The selector 530 may select an appropriate dynamic model by identifying a combination of attachment sites based on information input from the global motion acceleration orientation calculator 510. For example, even in a case where a user detaches and reattaches one or more sensor units 10 at some point during system operation, or in a case where power from a battery (not shown) is lost in one or more sensor units 10, such a configuration allows the selector 530 to dynamically select an appropriate dynamic model. Note that Fig.15 An example of identifying a combination of attachment sites based on information input from the global motion acceleration orientation calculator 510 is shown; however, the selector 530 may identify a combination of attachment sites by using sensor data itself as input or by using other information.

[0151] Fig.16 is an explanatory diagram showing an example of a combination of an attachment site and a dynamic model. Fig.16 In each of the skeleton information SD101 to SD105 shown, the attachment site to which the sensor unit 10 is attached is marked with a circle. Similarly, different root sites can be set for different dynamic models.

[0152] For example, in the skeleton information SD101 corresponding to the whole body model, the attachment site SP101 corresponding to the waist is the root site, and the sensor unit 10 is attached to six attachment sites SP101 to SP106. In the skeleton information SD102 corresponding to the upper limb model, the attachment site SP106 corresponding to the head is the root site, and the sensor unit 10 is attached to three attachment sites SP102, SP105, and SP106. In addition, in the skeleton information D103 corresponding to the head-to-arm model, the attachment site SP106 corresponding to the head is the root site, and the sensor unit 10 is attached to two attachment sites SP102 and SP106. In addition, in the skeleton information D104 corresponding to the waist-to-arm model, the attachment site SP101 corresponding to the waist is the root site, and the sensor unit 10 is attached to two attachment sites SP101 and SP105. Furthermore, in the skeleton information D105 corresponding to the lower limb model, the attachment site SP101 corresponding to the waist is the root site, and the sensor unit 10 is attached to the three attachment sites SP101 , SP103 , and SP104 .

[0153] Note that not only the attachment position regression estimator 540 to be described later but also the corrector 570 and the interpolator 580 to be described later perform processing according to the combination of attachment parts, and therefore, the combination of non-attachment parts to be estimated may differ depending on the combination of attachment parts (i.e., depending on the dynamic model). Parts for which the system according to the present embodiment can estimate position information and orientation information may be attachment parts, and non-attachment parts located between a plurality of attachment parts in a predetermined bone structure. Fig.16 In the skeleton information SD101 to SD105, the parts (including attached parts and non-attached parts) whose position information and orientation information can be estimated in this embodiment according to the combination of attached parts are indicated by hatching.

[0154] It should be noted that Fig.16 An example is shown, and the combination of attachment sites in the kinetic model available in this embodiment is not limited to Fig.16 The example shown in .

[0155] Using the dynamic model selected by the selector 530, the attachment position regression estimator 540 performs a regression estimation process (corresponding to Fig.12The process further comprises the following steps: (1) estimating the position information of the attachment part in the body coordinate system by regression based on the orientation information of the attachment part in the global coordinate system and the acceleration in the global coordinate system from which the gravity component has been deleted. Note that the attachment position regression estimator 540 according to the present embodiment estimates the position information of the attachment part in the body coordinate system with reference to the root part set for each dynamic model. The attachment position regression estimator 540 outputs the estimated position information for the attachment part in the body coordinate system to the reliability identifier 550 and the corrector 570.

[0156] Note that the above-mentioned second process based on the dynamic model may be performed by the above-mentioned global motion acceleration orientation calculator 510 and the attachment position regression estimator 540. In addition, the position information output from the attachment position regression estimator 540 corresponds to the second output.

[0157] The reliability identifier 550 has a reliability identification function that identifies the reliability of the position information of the attachment site estimated by the attachment position regression estimator 540 (which will also be referred to as the position information included in the second output). The reliability identified by the reliability identifier 550 can be used in the correction of the corrector 570 to be described later. For example, in the case where the reliability of the position information included in the second output is high, the corrector 570 to be described later attaches more importance (higher reliability) to the position information included in the second output than to the position information included in the first output, and performs correction; in the case where the aforementioned reliability is low, the corrector 570 attaches more importance to the position information included in the first output than to the position information included in the second output, and performs correction.

[0158] For example, in a case where the position information included in the second output exhibits a narrower likelihood distribution, the reliability identifier 550 may set a higher reliability. Fig.17 An example of reliability identification performed by the reliability identifier 550 is shown. Fig.17 , the position estimated by the attachment position regression estimator 540 and its likelihood distribution R10 are shown. For example, in the case where the difference between the upper limit L1 and the lower limit L2 of the likelihood distribution R10 is small, a higher reliability is identified. Fig.17 In the example shown, at time t 21 and time t 22 The interval and time t 23 and time t 24 During the interval between , high reliability is identified, that is, more emphasis is placed on the position information included in the second output. Fig.17 In the example shown, at time t 22 With time t 23 During the interval between, low reliability is identified, ie, more emphasis is placed on the position information included in the first output.

[0159] The reliability identifier 550 outputs the identified reliability to the corrector 570 .

[0160] The ground contact detector 560 performs processing for detecting contact of the foot of the body with the ground based on the sensor data (angular velocity and acceleration) acquired by the sensor units 10A to 10F. This configuration enables the corrector 570 described later to convert the position information in the body coordinate system estimated by the attachment position regression estimator 540 into the position information in the global coordinate system.

[0161] The ground contact detector 560 can detect contact with the ground, for example, in a so-called zero speed update (ZUPT: Zero Speed ​​Update). It should be noted that ZUPT is a technique for correcting a position by detecting a zero speed state as a contact state. In the case of using ZUPT, ZUPT can also realize a function of correcting position information by a corrector 570 to be described later. In addition, the method by which the ground contact detector 560 detects contact with the ground is not limited to this example, and another technique may be used to detect contact of a foot of a body with the ground.

[0162] The corrector 570 refers to the first output input from the integral calculator 520 and the second output input from the attachment position regression estimator 540, and corrects the position information of the attachment part included in the first output. Note that, hereinafter, the function of the corrector 570 may be referred to as a correction function. With this correction function, as shown in FIG. Fig.12 and Fig.13 As described, the error is less likely to grow larger over time, and thus the position information of the attachment site can be estimated with higher accuracy.

[0163] As described above, the corrector 570 can correct the position information of the attachment site based on the reliability identified by the reliability identifier 550. Such a configuration makes it possible to estimate the position information of the attachment site with higher accuracy.

[0164] Furthermore, the corrector 570 converts the position information (second output) of the attachment part in the body coordinate system estimated by the attachment position regression estimator 540 into the position information of the attachment part in the global coordinate system based on the result of the ground contact detection performed by the ground contact detector 560. With such a configuration, correction can be performed in a coordinate system unified as the global coordinate system.

[0165] Note that the correction function of the corrector 570 can be implemented by a Kalman filter. In addition, the integration function of the above-mentioned integral calculator 520 and the correction function of the corrector 570 can be implemented by a combination of a Kalman filter. For example, it can be assumed that the Kalman filter takes the speed and position of each attachment part as an internal state. Such a Kalman filter integrates the acceleration of the attachment part in the global coordinate system output from the global motion acceleration orientation calculator 510, thereby predicting the speed and position of the attachment part. Then, such a Kalman filter is able to correct the position information by updating the internal state using the position obtained by converting the position information contained in the second output into the position information in the global coordinate system as the observation value.

[0166] Based on the position information of the attachment part corrected by the correction function of the corrector 570, the interpolator 580 estimates the position information of the non-attachment part and generates the skeleton information. Fig.16 As described above, the interpolator 580 can estimate the position information of the non-attachment part between the plurality of attachment parts in the predetermined bone structure. Note that, hereinafter, the function of the interpolator 580 may be referred to as an interpolation function. Such an interpolation function may be, for example, referred to as Figure 4 and Figure 5 The inverse kinematics calculation described is implemented.

[0167] If already referred to Figure 4 and Figure 5 As described, in inverse kinematics calculations, there are usually multiple solutions, and for example, depending on the number of joint parts, there may be a large number of solutions. Therefore, in order to obtain more accurate skeletal information, a mechanism is required that makes it possible to identify a more appropriate solution from a large number of existing solutions. Therefore, the interpolator 580 according to the present embodiment estimates the position information of the non-attached part by regression by fitting to a pre-prepared inverse kinematics model.

[0168] Here, for example, the inverse kinematics model can be generated by pre-learning kinematic constraint information (e.g., information including a plurality of samples of positions and orientations of different parts in a previously acquired posture or a series of motions). It should be noted that various regression analysis methods can be used for the interpolation function of the interpolator 580, and for example, methods such as DNN and random forest can be used alone or in combination.

[0169] Note that although it is described above that the interpolator 580 estimates the position information of each part, the present embodiment is not limited to such an example. If any one of the position information and the orientation information of each part is obtained, the other can be identified based on the bone structure. Therefore, the interpolator 580 only needs to estimate at least one of the position information or the orientation information of the non-attached part.

[0170] <<4. Operation examples>>

[0171] The unit configuration example and the functional configuration example of the present embodiment have been described above. Next, an operation example of the present embodiment will be described. Fig.18 is a flowchart showing an example of the operation of the system according to the present embodiment. Note that Fig.18 Only the processing that mainly represents the features of the present embodiment is shown, and the system according to the present embodiment can therefore perform Fig.18 Processing not shown.

[0172] like Fig.18 As shown, first, the global motion acceleration orientation calculator 510 calculates the orientation of the sensor unit 10 in the global coordinate system through the AHRS based on the angular velocity obtained by the gyro sensor 12 (S502). Note that step S502 corresponds to Fig.12 Steps S111 and S121 in .

[0173] Subsequently, the global motion acceleration orientation calculator 510 converts the orientations of the sensor units 10 in the global coordinate system obtained in step S502 into the orientations of the attachment sites to which those sensor units 10 are attached in the global coordinate system (S504)).

[0174] Subsequently, the global motion acceleration orientation calculator 510 converts the acceleration in the local coordinate system acquired by the acceleration sensor 14 into the acceleration in the global coordinate system based on the orientation of the attachment part in the global coordinate system obtained in step S504 (S506). Note that step S506 corresponds to Fig.12 Steps S112 and S122 in .

[0175] Subsequently, the global motion acceleration orientation calculator 510 removes the gravity component from the acceleration in the global coordinate system obtained in step S506 based on the orientation of the attachment part in the global coordinate system obtained in step S504 (S508).

[0176] Subsequently, the integral calculator 520 calculates the position information of the attachment part in the global coordinate system by integrating twice the acceleration in the global coordinate system from which the gravity component is removed obtained in step S508 (S510). Note that step S510 corresponds to Fig.12 Steps S113 and S114 in .

[0177] Subsequently, it is determined whether to perform the correction of step S510 (S512). As described above, the processing load of the position estimation process based on the dynamic model is higher than that of the position estimation process based on the inertial navigation system. Therefore, the execution frequency of the position estimation process based on the dynamic model (S514) is lower than the execution frequency of the position estimation process based on the inertial navigation system (S502 to S510). Therefore, in step S512, it can be determined that each time the position estimation process based on the dynamic model is Fig.18 The processing shown in is performed a predetermined number of times and correction is performed once. However, the present embodiment is not limited to such an example. For example, in the case where it is determined based on the sensor data that the attachment part is not fixed, it can be determined that correction is to be performed, and in the case where it is determined that the attachment part is fixed, it can be determined that no correction is to be performed.

[0178] In the case where it is determined that correction is to be performed (Yes in S512), the attachment position regression estimator 540 estimates the position information of the attachment part in the body coordinate system by regression estimation using the dynamic model (S514). Note that step S514 corresponds to Fig.12 Step S123.

[0179] Subsequently, the corrector 570 converts the position information of the attachment part in the body coordinate system estimated in step S514 into the position information of the attachment part in the global coordinate system based on the ground contact detection result of the ground contact detector 560 ( ​​S516 ).

[0180] Then, based on the position information for the attachment site in the global coordinate system obtained in step S516 , the corrector 570 corrects the position information for the attachment site in the global coordinate system obtained in step S510 ( S518 ).

[0181] Subsequently, the interpolator 580 estimates (interpolates) the position information for the non-attached part by regression based on the position information for the attached part in the global coordinate system by using the fitted inverse kinematics model (S520). Note that the position information of the attached part for the regression estimation of the position information for the non-attached part in step S520 may be different depending on the determination result in step S512. In the case where it is determined in step S512 that correction is to be performed, the position information of the attached part corrected in step S518 is used for regression estimation in step S520. Meanwhile, in the case where it is determined in step S512 that correction is not to be performed, the position information for the attached part in the global coordinate system obtained in step S510 is used for regression estimation in step S520.

[0182] The operation of the system according to the present embodiment has been described above. Note that the operation may be repeatedly performed every time or at a predetermined frequency when sensor data is acquired by the gyro sensor 12 and the acceleration sensor 14. Fig.18 Processing shown.

[0183] <<5. Modification Example>>

[0184] An embodiment of the present disclosure has been described above. Below, some modified examples of the present embodiment will be described. Note that the modified examples described below can be applied to the present embodiment individually or in combination. In addition, instead of the configuration described in the present embodiment, the above-mentioned modified examples can also be applied.

[0185] <5-1. First Modification Example>

[0186] For the aforementioned embodiment, an example has been described in which the attachment position regression estimator 540 performs regression estimation processing based on the orientation information of the attachment part in the global coordinate system calculated by the global motion acceleration orientation calculator 510 and the acceleration in the global coordinate system from which the gravity component has been removed. However, the present technology is not limited to such an example. For example, the angular velocity and acceleration acquired by the gyro sensor 12 and the acceleration sensor 14 can be used as inputs of such regression estimation processing. Such an example will be described as a first modified example.

[0187] Fig.19 An example of a functional configuration of a system according to a first modified example is shown. Fig.19 In the configuration shown, Fig.15 Components substantially identical to those shown are denoted by the same reference numerals, and thus description thereof will be omitted here. Fig.15 Description of the different points of the system shown. Fig.19 , the system according to this modified example and Fig.15 The system shown differs in that it includes an attachment position regression estimator 541 instead of the attachment position regression estimator 540 .

[0188] The attachment position regression estimator 541 according to the present modification example performs regression estimation processing using the angular velocity and acceleration acquired by the gyro sensor 12 and the acceleration sensor 14 as inputs in addition to or instead of the information calculated by the global motion acceleration orientation calculator 510. A dynamic model for such regression estimation can be prepared in advance by learning kinematic constraint information including the same type of information (angular velocity and acceleration) as that used for regression estimation.

[0189] The first modification example has been described above. According to the present modification example, using more information makes it possible to estimate the position information of the attachment site with higher accuracy. Note that the above description deals with an example in which angular velocity and acceleration are used as inputs of the regression estimation process by the attachment position regression estimator 541; however, other information may be further used as inputs. For example, in the case where the sensor unit 10 includes a geomagnetic sensor or an air pressure sensor in addition to the gyro sensor 12 and the acceleration sensor 14, information about geomagnetism or air pressure may be used as inputs of such regression estimation process.

[0190] <5-2. Second Modification Example>

[0191] Next, an example of corrector identification reliability will be described as a second modification example. Fig. 20 An example of a functional configuration of a system according to this modification example is shown. Fig. 20 In the configuration shown, Fig.15 Components substantially identical to those shown are denoted by the same reference numerals, and thus description thereof will be omitted here. Fig.15 Description of the different points of the system shown. Fig. 20 , the system according to this modification example and Fig.15 The illustrated system differs in that it includes a corrector 572 in place of the reliability identifier 550 and the corrector 570 .

[0192] In addition to reference Fig.15 In addition to the functions of the corrector 570 described above, the corrector 572 has a function of recognizing the reliability of the position information of the attachment part input from the integral calculator 520 or the attachment part information input from the attachment position regression estimator 540, or both. For example, such a function may be implemented by a Kalman filter, may be performed based on a predetermined rule, or may be implemented by regression estimation using a model prepared in advance by learning.

[0193] <5-3. Third Modification Example>

[0194] Next, an example in which correction processing is performed after position estimation (interpolation) processing of a non-attachment part will be described as a third modification example. Fig.21 An example of a functional configuration of a system according to this modification example is shown. Fig.21 In the configuration shown, Fig.15 Components substantially identical to those shown are denoted by the same reference numerals, and thus description thereof will be omitted here. Fig.15 Description of the different points of the system shown. Fig.21 , the system according to this modification example and Fig.15The system shown differs in that it includes a corrector 573 and an interpolator 583 instead of the reliability identifier 550 , the corrector 570 , and the interpolator 580 .

[0195] According to the interpolator 583 of the present modification example, the position information or the orientation information of the non-attached part, or both, is estimated by regression based on the orientation information of the attached part in the global coordinate system calculated by the global motion acceleration orientation calculator 510 and the position information of the attached part estimated by the attached position regression estimator 540. Note that the regression estimation of the interpolator 583 can be similar to the regression estimation of the non-attached part by the interpolator 580 except that the information to be input is different. The position information or the orientation information of the non-attached part estimated by the interpolator 583 is output to the corrector 573.

[0196] It should be noted that the second processing according to the present modification example includes an interpolation process that estimates the position information or orientation information or both of the non-attached parts by the aforementioned interpolator 583. Then, the second output (the output of the second processing) according to the present modification example includes the position information or orientation information, or both, of the non-attached parts estimated by the interpolator 583. In addition, the second output according to the present modification example may include the position information or orientation information of the attached parts obtained by the global motion acceleration orientation calculator 510 or the attached position regression estimator 540. That is, the second output according to the present modification example includes the position information or orientation information, or both, of all parts used to estimate the target.

[0197] Based on the first output (the orientation information and the position information for the attachment part) input from the integral calculator 520, the corrector 573 according to the present modified example corrects the corrected position information or the orientation information, or both, of the non-attached part included in the second output input from the interpolator 583. In the present modified example, the speed and the position of the attachment part have a nonlinear relationship with the position or the orientation of the non-attached part, and therefore, the correction function of the corrector 573 of the present modified example can be implemented, for example, by an extended Kalman filter. Hereinafter, an example in which the correction function of correcting the orientation information of the non-attached part is implemented by an extended Kalman filter will be described.

[0198] This extended Kalman filter takes the velocities and positions of all attachment parts and the orientations of all target parts to be estimated (including attachment parts and non-attachment parts) as internal states. This extended Kalman filter predicts the velocity and position of the attachment parts by integrating the acceleration of the attachment parts in the global coordinate system output from the global motion acceleration orientation calculator 510. The extended Kalman filter then updates the internal state by using the orientation information of all parts for estimating the target included in the second output as observations, thereby being able to correct the orientation information. It should be noted that the covariance matrix of the error that the extended Kalman filter has internally can be predicted by using the Jacobian matrix, which describes the relationship between the attachment part velocity and the non-attachment part orientation used in the inverse kinematics calculation.

[0199] Notice, Fig.21 The attachment position regression estimator 540 and the interpolator 583 are shown as separate functional configurations; however, the functions of the attachment position regression estimator 540 and the interpolator 583 may be unified. In this case, the position information and orientation information of all parts for estimating the target can be directly estimated by regression based on the information input from the global motion acceleration orientation calculator 510. In this case, there is an effect of reducing the number of models to be prepared in advance for regression estimation.

[0200] <5-4. Fourth Modification Example>

[0201] In the aforementioned embodiment, an example has been described in which the attachment position regression estimator 540 performs regression estimation using a non-tracking type dynamic model; however, a tracking type dynamic model may be used. Fig. 22 and Fig.23 Describe an overview of non-tracking type kinetic models and tracking type kinetic models.

[0202] Fig. 22 is an illustration that describes an overview of the non-tracking type kinetic model. Fig. 22 As shown, in the regression estimation using the non-tracking type dynamic model, information about acceleration and orientation is input to allow position information to be output.

[0203] Fig.23 is an illustration that describes an overview of the tracking type kinetic model. Fig.23 As shown, in the regression estimation using the tracking type dynamic model, information about acceleration and orientation is also input to allow position information to be output. However, in the regression estimation using the tracking type dynamic model, in addition to information about acceleration and orientation, position information delayed by one sample by the delayer D (i.e., the output of the previous regression estimation process) is input to the next regression estimation process. The tracking type dynamic model has the advantage that the tracking capability can be improved.

[0204] An example in which the position information of the attachment site is estimated by regression using such a tracking type dynamic model will be described as a fourth embodiment. Fig.24 An example of a functional configuration of a system according to this modification example is shown. Fig.24 In the configuration shown, Fig.15 Components substantially identical to those shown are denoted by the same reference numerals, and thus description thereof will be omitted here. Fig.15 Description of the different points of the system shown. Fig.24 , the system according to this modification example and Fig.24 The system shown differs in that it includes an attachment position regression estimator 544 and a corrector 574 instead of the attachment position regression estimator 540 and the corrector 570 .

[0205] The attachment position regression estimator 544 performs a regression estimation process by using the position information of the attachment part estimated in the previous regression estimation process, using the position information of the attachment part corrected by the corrector 574 (as an input), in addition to the information calculated by the global motion acceleration orientation calculator 510. The dynamic model used for this regression estimation is the tracking type dynamic model described above, and can be prepared in advance by, for example, learning motion constraint information including position information delayed by one sample.

[0206] The above-described fourth modification example provides an improved capability of tracking various motions, thereby making it possible to estimate the position information of the attachment site with higher accuracy.

[0207] <5-5. Fifth Modification Example>

[0208] In the foregoing embodiments, reference has been made to Fig.14 The unit configuration example is described; however, the present technology is not limited to such an example. Hereinafter, a unit configuration example of a system that does not include the hub unit 20 is described as a fifth modified example. Fig.25 A unit configuration example of a system according to the present modification example is shown.

[0209] like Fig.25 As shown, the system according to the present modification example includes sensor units 10A to 10F, an information processor 30, and an output unit 40. Fig.25 As shown, in this modified example, the sensor unit 10 is directly connected to the information processor 30 without passing through the hub unit 20. Note that the sensor unit 10 and the information processor 30 may be connected to each other in a wired manner or wirelessly. In addition to this connection relationship, Fig.25 Each unit shown in can have Fig.14 The configurations of corresponding units denoted by the same reference numerals in the drawings are similar.

[0210] <5-6. Sixth Modification Example>

[0211] In the aforementioned embodiment, an example has been described in which regression estimation is performed using a dynamic model and an inverse kinematic model. The estimation result of this regression estimation may vary greatly depending on the kinematic constraint information used for learning to generate the dynamic model and the inverse kinematic model. For example, in the case where kinematic constraint information obtained by sensing a characteristic posture or motion is used for learning, the estimation result tends to have characteristics closer to the motion used for learning than the actual motion of the body.

[0212] Examples of characteristic poses or movements include, but are not limited to, poses unique to characters such as zombies, dynamic movements typical of fighting scenes, poses frequently appearing in games such as shooters, choreographed movements for each dance type, movements based on individual rules for each sport, etc.

[0213] By appropriately preparing kinematic constraint information suitable for the application as described above and using it for learning to generate a dynamic model and an inverse kinematic model, skeletal information suitable for the application can be estimated. As a result, for example, in the case of generating an animation based on skeletal information, a feature animation can be generated without post-processing the skeletal information estimated by the above series of processes.

[0214] <<6. Overview of Second Embodiment>>

[0215] In the aforementioned embodiment of the present disclosure, as described above, in order to convert the position information in the body coordinate system estimated by the attachment position regression estimator 540 or the like into the position information in the global coordinate system, the process of detecting the contact of the foot of the body with the ground is performed by the ground contact detector 560. For example, the ground contact detector 560 is able to utilize a technique of correcting the position by using ZUPT, in other words, by detecting a state where the velocity is zero (predetermined velocity) as a contact state.

[0216] <6-1. First ZUPT Technology>

[0217] First, refer to Fig.26 and 27 A description is given of the first ZUPT technique. Fig.26 and 27 : is an explanatory diagram for describing the first ZUPT technique according to the present embodiment.

[0218] like Fig.26 As shown, the first ZUPT technique uses sensor data acquired by a sensor unit 10 including an inertial sensor (eg, an acceleration sensor and a gyro sensor) attached to a forefoot (specifically, a toe or an instep).

[0219] Specifically, in the first ZUPT technology, as Fig. 27 As shown, sensor data is acquired from the aforementioned sensor unit 10, and the acceleration of the sensor unit 10 in the local coordinate system and the angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data are acquired. Then, in the first ZUPT technique, the acquired acceleration and angular velocity are subjected to the above-mentioned processing based on the inertial navigation system, thereby acquiring the acceleration, velocity and position in the global coordinate system (S600).

[0220] Next, in this technology, the aforementioned sensor data is used to detect a state where the velocity of the forefoot is zero (S601). In this detection, in this technology, the discreteness of the acceleration included in the aforementioned sensor data, the norm of the angular velocity, etc. can be compared with a threshold value, or the state where the velocity is zero can be detected from the acceleration or angular velocity by machine learning, so there is no particular limitation.

[0221] Then, in this technology, if the speed of the front foot is detected to be zero, assuming that the speed of the front foot in the global coordinate system acquired in step S600 based on the processing of the inertial navigation system is zero, the position of the front foot in the global coordinate system acquired by the processing based on the inertial navigation system in step S600 is corrected (S602). A Kalman filter or the like can be used for the above correction. For example, using such a Kalman filter, the position of the front foot in the global coordinate system can be predicted by integrating the acceleration of the front foot in the global coordinate system. In addition, the Kalman filter makes it possible to correct the position of the front foot in the global coordinate system by updating the internal state using the observation value in which the speed of the front foot is assumed to be zero.

[0222] <6-2. Second ZUPT Technology>

[0223] Next, we will refer to Fig.28 and Fig.29 A second ZUPT technique is described. Fig.28 and Fig.29 : is an explanatory diagram for describing the second ZUPT technique according to the present embodiment.

[0224] Incidentally, in motion capture, the aforementioned sensor unit 10 is not always as Fig.26 Attached to the front foot as shown, but can be Fig.28 As shown, the sensor unit 10 is attached to, for example, an ankle, a calf, etc. As a characteristic of human motion, when the forefoot contacts the ground, the velocity of the forefoot will surely become zero within a certain period of time, while the velocity of the ankle, etc. is less likely to become zero. In particular, in the case of running motion in motion capture, for example, the above tendency may become apparent. Therefore, in the case of performing ZUPT using sensor data of the sensor unit 10 attached to the ankle, etc., it is sometimes difficult to correct the position in the global coordinate system with high accuracy because the velocity is less likely to become zero within a certain period of time.

[0225] Furthermore, according to the research of the present inventors, in the case of the movement of the body obtained by motion capture, in the configuration in which the sensor unit 10 is attached to the ankle and the sensor unit 10 is not attached to the forefoot, the skeletal structure representing the body structure is reproduced, and accurate reproduction can be achieved for the area from the waist to the ankle of the person. However, since the forefoot or the like constrains the human body less than the knee or the like, its movement is difficult to reproduce with high accuracy. In particular, in the case where the body movement obtained by motion capture is applied to an animation in which an avatar performs movement, etc., the forefoot of the avatar is sometimes unstable in the animation, resulting in insufficient weight of the avatar.

[0226] In view of the above, the inventors came up with an idea to estimate the state where the speed of the forefoot is zero using the sensor data of the sensor unit 10 attached to the ankle or the like, and thus created the second ZUPT technique according to the present embodiment described below. The second ZUPT technique indirectly estimates the speed of the forefoot. Therefore, even when a running motion is captured, the state where the speed of the forefoot is zero can be stably detected, which can more reliably correct the position of the forefoot. In addition, according to the second ZUPT technique, the forefoot of the above-mentioned avatar can be controlled by using the estimated forefoot speed or forefoot position. Therefore, in the case of reproducing the motion of the forefoot of the avatar contacting the ground in animation, a stable forefoot behavior can be reproduced.

[0227] Specifically, in the second ZUPT technique, as Fig.29 As shown, sensor data is acquired from the sensor unit 10 attached to the ankle, calf, etc., for acquiring the acceleration of the sensor unit 10 in the local coordinate system and the angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data. Then, in the second ZUPT technique, the acquired acceleration and angular velocity are subjected to the aforementioned processing based on the inertial navigation system, thereby acquiring the acceleration and orientation of the sensor unit 10 in the global coordinate system (S610).

[0228] Next, in this technology, the speed of the forefoot is estimated by using the sensor data (acceleration and angular velocity of the sensor unit 10 in the local coordinate system) and the acceleration and orientation of the sensor unit 10 in the global coordinate system based on the inertial navigation system (S611). Then, in this technology, the position of the forefoot viewed from the ankle is estimated by using the sensor data (acceleration and angular velocity of the sensor unit 10 in the local coordinate system) and the acceleration and orientation of the sensor unit 10 in the global coordinate system based on the inertial navigation system (S612). In addition, in this technology, a fusion process of fusing the acceleration and orientation of the sensor unit 10 in the global coordinate system obtained by the above steps, the estimation result of the forefoot speed, and the estimation result of the forefoot position is performed (S613). As a result, in this technology, the corrected positions of the ankle and forefoot in the global coordinate system can be obtained. The processing details in each step will be described later.

[0229] <6-3. Functional configuration example>

[0230] The overview of the first and second ZUPT techniques according to the present embodiment has been described above. Next, as an example, reference will be made to Fig.30 A system functional configuration of the second ZUPT technique according to the present embodiment is described. Fig.30 An example of the functional configuration of the system according to the present embodiment is shown. Note that Fig.30 Only the functional configuration mainly representing the features of the present embodiment is shown, and the system according to the present embodiment may include Fig.15 Functional configuration shown.

[0231] Reference Fig.30 The system according to the present embodiment has functions as a global motion acceleration orientation calculator 510, an integral calculator 520, a selector 530, an attachment position regression estimator 540, a reliability identifier 550, a ground contact detector 560, a corrector 570, an interpolator 580, and an estimator 561. It should be noted that Fig.30 The interpolator 580, the global motion acceleration orientation calculator 510, the integral calculator 520, the selector 530, the attachment position regression estimator 540, the reliability identifier 550, the ground contact detector 560, the corrector 570 and the reference Fig.15 Therefore, the description thereof will be omitted here, and only the estimator 561 which is different from the first embodiment will be described. Fig.30 Each function configuration can be Fig.14 The present invention can be implemented by any of the units shown and is not particularly limited.

[0232] The estimator 561 estimates the velocity and position of the forefoot as viewed from the ankle by using the sensor data from the integral calculator 520 (the acceleration and angular velocity of the sensor unit 10 in the local coordinate system) and the acceleration and orientation of the sensor unit 10 in the global coordinate system, and outputs the estimation result to the corrector 570.

[0233] <6-4. Operation example>

[0234] The functional configuration example according to the present embodiment has been described above. Fig.31 An operation example of the second ZUPT technique of the present embodiment is described. Fig.31 is a flowchart showing an example of the operation of the system according to the present embodiment. Note that Fig.31 Only the processing that mainly represents the features of this embodiment is shown, and the system according to this embodiment can perform Fig.31 Processing not shown.

[0235] like Fig.31 As shown, the global motion acceleration orientation calculator 510 and the estimator 561 acquire sensor data (acceleration and angular velocity of the sensor unit 10 in the local coordinate system) from the sensor unit 10 attached to the ankle, calf, etc. (S700). Then, the global motion acceleration orientation calculator 510 and the integral calculator 520 calculate the acceleration and orientation of the sensor unit 10 in the global coordinate system by performing the above-mentioned processing on the acceleration and angular velocity acquired in step S700 based on the inertial navigation system (S702).

[0236] Subsequently, the estimator 561 estimates the velocity of the forefoot using the sensor data (acceleration and angular velocity of the sensor unit 10 in the local coordinate system) acquired in step S700 and the acceleration and orientation of the sensor unit 10 in the global coordinate system based on the inertial navigation system calculated in step S702 (S704). In addition, the estimator 561 estimates the position of the forefoot viewed from the ankle by using the sensor data (acceleration and angular velocity of the sensor unit 10 in the local coordinate system) acquired in step S700 and the acceleration and orientation of the sensor unit 10 in the global coordinate system based on the inertial navigation system calculated in step S702 (S706).

[0237] Next, the corrector 570 performs correction on the velocity of the front foot estimated in step S704 by the ZUPT process, and performs fusion processing of the front foot position estimated in step S706 ( S708 ).

[0238] In addition, reference will be made to Fig.32 and Fig.33 describe Fig.31The details of each of steps S704, S706 and S708 are shown in FIG. Fig.32 is an explanatory diagram describing the estimation of the front foot speed according to the present embodiment, Fig.33 is an explanatory diagram describing estimation of the forefoot position according to the present embodiment.

[0239] (Estimated front foot speed)

[0240] The estimator 516 estimates the speed of the front foot in the above step S704. Specifically, first, Fig.32 As shown, the acceleration and angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data acquired by the sensor unit 10 attached to the ankle are input to the estimator 516. In addition, the acceleration and orientation of the sensor unit 10 in the global coordinate system obtained by performing the above-mentioned processing based on the inertial navigation system on the acceleration and angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data are input to the estimator 516. It is to be noted that when estimating the speed of the forefoot, any one or two of the aforementioned sensor data (acceleration and angular velocity in the local coordinate system), the aforementioned acceleration in the global coordinate system, or the aforementioned orientation in the global coordinate system may be input to the estimator 516. In addition, when estimating the speed of the forefoot, the sensor data of the sensor unit 10 attached to other parts, or the acceleration and orientation in the global coordinate system obtained by placing the sensor data in the processing based on the inertial navigation system may be input to the estimator 516, so there is no specific limitation.

[0241] Based on the above input, the estimator 516 then estimates the speed of the forefoot according to a predetermined rule. In the present embodiment, the estimator 516 can estimate the speed of the forefoot by using classification and regression of machine learning. In this case, the estimator 516 is able to output a binary label in which the speed of the forefoot is zero or non-zero as the estimation result. In addition, the present embodiment is not limited to outputting such a binary label as the estimation result. In the present embodiment, for example, the forefoot speed can be output as "0m / s 2 Above and less than 0.01m / s 2 ”, “0.01m / s 2 Above and less than 0.10m / s 2 ","0.10m / s 2 In addition, in the present embodiment, instead of such labeling, the speed of the front foot itself can be estimated and output.

[0242] (Estimated front foot position)

[0243] In the above step S706, the estimator 516 estimates the position of the forefoot viewed from the ankle. Specifically, Fig.33As shown, the acceleration and angular velocity in the local coordinate system included in the sensor data acquired by the sensor unit 10 attached to the ankle are input to the estimator 516. In addition, the acceleration and orientation of the sensor unit 10 in the global coordinate system obtained by performing the above-mentioned processing based on the inertial navigation system on the acceleration and angular velocity of the sensor unit 10 in the local coordinate system included in the sensor data are input to the estimator 516. It is to be noted that when estimating the position of the forefoot, one or two of the aforementioned sensor data (acceleration and angular velocity in the local coordinate system), the aforementioned acceleration in the global coordinate system, or the aforementioned orientation in the global coordinate system may be input to the estimator 516. In addition, when estimating the position of the forefoot, the sensor data of the sensor unit 10 attached to other parts, or the acceleration and orientation in the global coordinate system obtained by placing the sensor data in the processing based on the inertial navigation system may be input to the estimator 516, so there is no specific limitation. In addition, as Fig.33 As shown, body parameters such as the height or length of the leg (eg, leg length above the knee or leg length below the knee) may be input to the estimator 516 .

[0244] Then, based on the aforementioned input, the estimator 516 estimates the position of the front foot according to a predetermined rule. In this embodiment, the estimator 516 can estimate the speed of the front foot by using classification and regression of machine learning.

[0245] (Fusion Processing)

[0246] In the above-mentioned step S708, the corrector 570 performs a fusion process that fuses the acceleration and orientation in the global coordinate system, the estimated speed of the forefoot, and the estimated position of the forefoot. The corrector 570 is able to perform the fusion process by using, for example, a Kalman filter that fuses multiple data, thereby placing a higher weight on data with higher reliability. Using such a Kalman filter, for example, in the case of a binary label in which the speed of the forefoot estimated in step S704 is zero or non-zero, the observed value of the zero speed label can be corrected. In addition, for example, using a Kalman filter, in the case of a multi-value label having three or more values ​​of the estimated forefoot speed, the reliability of the zero speed vector can be reduced by appropriately performing the fusion process in a manner that sequentially reduces the reliability of labels close to zero speed.

[0247] As described above, according to the second ZUPT technique of the present embodiment, the speed of the forefoot can be estimated indirectly. Therefore, even when a running motion is captured, a state in which the speed of the forefoot is zero can be stably detected, which can more reliably correct the position of the forefoot. In addition, according to the second ZUPT technique, the forefoot of the above-mentioned avatar can be controlled by using the estimated forefoot speed or forefoot position. Therefore, in the case of reproducing the motion of the forefoot of the avatar contacting the ground with animation, a stable forefoot behavior can be reproduced.

[0248] <6-5. Modification Example 1>

[0249] In the aforementioned embodiment, the sensor data of the sensor unit 10 attached to one ankle is used; however, in the present embodiment, this is non-limiting, and the sensor data of the sensor unit 10 attached to both ankles may be used. By doing so, ZUPT can be performed with higher accuracy. Therefore, as a modification example 1 of the present embodiment, reference will be made to Figure 34 to Figure 36 A modified example using sensor data of the sensor units 10 attached to both ankles is explained. Fig.34 : is an explanatory diagram for describing the ZUPT technique according to Modification Example 1. Fig.35 is an explanatory diagram describing estimation of relative positions according to Modification Example 1. In addition, Fig.36 is a flowchart illustrating an operation example of the system according to Modification Example 1.

[0250] Specifically, in this modification example 1, as Fig.34 As shown, sensor data is acquired from the sensor unit 10 attached to the right ankle, the sensor unit 10 attached to the left ankle, and the sensor unit 10 attached to another part. In addition, in this modified example 1, the acceleration and angular velocity of the right ankle in the local coordinate system, the acceleration and angular velocity of the left ankle in the local coordinate system, and the acceleration and angular velocity of other parts in the local coordinate system are acquired. Then, in this modified example 1, the aforementioned processing based on the inertial navigation system is performed on the acquired acceleration and angular velocity of the right ankle, thereby acquiring the acceleration and orientation of the right ankle in the global coordinate system (S800).

[0251] In addition, similar to the above, in this modification example 1, the above-mentioned processing based on the inertial navigation system is performed on the acquired acceleration and angular velocity of the left ankle, thereby acquiring the acceleration and orientation of the left ankle in the global coordinate system (S802). In addition, in this modification example 1, the above-mentioned processing based on the inertial navigation system is performed on the acquired acceleration and angular velocity of another part, thereby acquiring the acceleration and orientation of another part in the global coordinate system (S804).

[0252] Next, in the present modified example, the respective velocities of the two forefoots are estimated by using the acceleration and orientation of the right ankle, the left ankle, and another part in the global coordinate system based on the inertial navigation system (S806). Then, in the present modified example, the respective positions of the two forefoots are estimated by using the acceleration and orientation of the right ankle, the left ankle, and another part in the global coordinate system based on the inertial navigation system (S808). Furthermore, in the present modified example, the relative positions of the two ankles are estimated by using the acceleration and orientation of the right ankle, the left ankle, and another part in the global coordinate system based on the inertial navigation system (S809). Fig.35 R20 in)(S810).

[0253] In this modified example, the relative positions of the two ankles are re-estimated, which makes it possible to obtain the positions of the ankles and forefoot in a more constrained global coordinate system. In the estimation, similar to the estimation in the aforementioned embodiment, machine learning can be used. In addition, when estimating the speed of the two ankles, the positions of the two ankles, and the relative positions of the two ankles, not only the acceleration and orientation of the right ankle, left ankle, and other parts in the global coordinate system based on the inertial navigation system can be used, but also the acceleration and angular velocity of the right ankle, left ankle, and other parts in the local coordinate system contained in the sensor data can be used, or body parameters such as leg height or length can be used, so there is no particular limitation.

[0254] Furthermore, in this modified example, a fusion process (S812) is performed to fuse the estimation results of the speeds of the two front feet, the estimation results of the positions of the two front feet, and the relative positions of the two ankles. Note that, as described above, in the fusion process in this modified example, the processing in the fusion process can also be performed by using a Kalman filter or the like that estimates the states of the speeds and positions of the two ankles and the two front feet. As a result, in this modified example, the corrected positions of the ankles and the front feet in the global coordinate system can be acquired.

[0255] The operation of the above-mentioned modification example 1 is summarized as follows. Fig.36 As shown, the global motion acceleration orientation calculator 510 and the estimator 561 acquire sensor data (acceleration and angular velocity of the two ankles and another part in the local coordinate system) from the sensor unit 10 attached to the two ankles, etc. (S820). Then, the global motion acceleration orientation calculator 510 and the integral calculator 520 perform the aforementioned processing on the acceleration and angular velocity acquired in step S820 based on the inertial navigation system, thereby calculating the acceleration and orientation of the two ankles and other parts in the global coordinate system (S822).

[0256] Subsequently, the estimator 561 estimates the speed of the two forefoots by using the acceleration and orientation of the two ankles and another part in the global coordinate system based on the inertial navigation system calculated in step S822 (S824). In addition, the estimator 561 estimates the respective positions of the two forefoots viewed from the two ankles by using the two ankles and another part in the global coordinate system based on the inertial navigation system calculated in step S822.

[0257] Next, the estimator 561 estimates the relative positions of the two ankles by using the acceleration and orientation of the two ankles and another part in the global coordinate system based on the inertial navigation system ( Fig.35 In addition, the corrector 570 corrects the speeds of the two forefoots estimated in step S824 through ZUPT processing, and performs fusion processing (S830) to fuse the positions of the two forefoots estimated in step S826 and the relative positions of the two ankles estimated in step S828.

[0258] In the above-mentioned present modification example, by using the sensor data of the sensor unit 10 attached to the two ankles, etc. to newly estimate the relative positions of the two ankles, it is possible to obtain the positions of the ankles and forefoot in the global coordinate system with stronger constraints. As a result, in the present modification example, ZUPT can be performed with higher accuracy.

[0259] <6-6. Modification Example 2>

[0260] In the aforementioned modification example 1, the relative positions of the two ankles are estimated; however, the sensor data from the distance sensor can be further used to measure the relative positions. By doing so, ZUPT can be performed with higher accuracy. Therefore, reference Fig.37 and 38 A modification example in which sensor data from a distance sensor is used to measure a relative position is described as modification example 2 of the present embodiment. Fig.37 An example of a unit configuration of the sensor unit 10 according to Modification Example 2 is shown, Fig.38 : is an explanatory diagram describing the ZUPT technique according to Modification Example 2.

[0261] like Fig.37 As shown, the system according to the present modification example 2 may include a sensor unit 10G. The sensor unit 10G may be, for example, Fig.14 As shown in the sensor unit 10, it has a gyro sensor 12, an acceleration sensor 14, a controller 16 and a communicator 18. In addition, the sensor unit 10G may also include a distance sensor 50 capable of measuring the relative distance between the two ankles. The distance sensor 50 may be, for example, an ultrasonic sensor capable of measuring the distance by reflecting ultrasonic waves.

[0262] Specifically, in this modification example 2, as Fig.34 As shown, sensor data is acquired from the sensor unit 10 attached to the right ankle, the sensor unit 10 attached to the left ankle, and the sensor unit 10 attached to another part. In addition, in the present modification example 2, sensor data about the relative distance between the two ankles is acquired from the distance sensor 50. Then, in the present modification example 2, the aforementioned processing based on the inertial navigation system is performed on the acquired accelerations and angular velocities of the two ankles, thereby obtaining the accelerations and orientations of the two ankles in the global coordinate system (S900, S902). In addition, in the present modification example 2, the aforementioned processing based on the inertial navigation system is performed on the accelerations and angular velocities acquired in other parts, thereby obtaining the accelerations and orientations of other parts in the global coordinate system (S904).

[0263] Next, in this modified example, based on the inertial navigation system, the acceleration and orientation of the right ankle, left ankle and other parts in the global coordinate system are used to estimate the speed and position of each of the two forefoots, and further estimate the relative positions of the two ankles (S906, S908, S910), as well as the relative distance between the two ankles contained in the sensor data.

[0264] In addition, in this modified example, fusion processing is performed to fuse the estimated results of the speed of the two front feet, the estimated results of the positions of the two front feet, the relative positions of the two ankles, and further, the relative distance between the two ankles included in the sensor data (S912).

[0265] As described above, the above-described Modification Example 2 makes it possible to perform ZUPT with higher accuracy by reusing the sensor data of the distance sensor that measures the relative position.

[0266] <7. Hardware Configuration Example>

[0267] The embodiments of the present disclosure have been described above. Fig.39 , a description will be given of the hardware configuration of an information processor according to an embodiment of the present disclosure. Fig.39 is a block diagram showing an example of the hardware configuration of an information processor according to an embodiment of the present disclosure. Note that Fig.39 The information processor 900 shown is capable of implementing, for example, Fig.14 The sensor unit 10, the hub unit 20 and the information processor 30 are shown. The information processing of the sensor unit 10, the hub unit 20 and the information processor 30 according to the embodiment of the present disclosure is realized through the cooperation of the software and hardware described below.

[0268] like Fig.39As shown, the information processor 900 includes a CPU (central processing unit) 901, a ROM (read only memory) 902, a RAM (random access memory) 903, and a host bus 904a. In addition, the information processor 900 includes a bridge 904, an external bus 904b, an interface 905, an input unit 906, an output unit 907, a storage unit 908, a driver 909, a connection port 911, a communication unit 913, and a sensor 915. The information processor 900 may include a processing circuit such as a DSP or an ASIC instead of the CPU 901 or in combination with the CPU 901.

[0269] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation in the information processor 900 according to various programs. In addition, the CPU 901 may be a microprocessor. The ROM 902 stores programs used by the CPU 901, arithmetic parameters, and the like. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters appropriately changed in the execution thereof, and the like. The CPU 901 may form, for example, the controller 16, the controller 24, and the controller 34.

[0270] The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a host bus 904a including a CPU bus, etc. The host bus 904a is connected to an external bus 904b such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 904. Note that the host bus 904a, the bridge 904, and the external bus 904b do not have to be separately configured, and these functions may be packaged in a single bus.

[0271] The input unit 906 is implemented by, for example, a unit through which a user inputs information, such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a joystick. In addition, the input unit 906 may be a remote control unit using infrared or other electromagnetic waves, or may be an external connection device compatible with the operation of the information processor 900, such as a mobile phone or a PDA. In addition, the input unit 906 may include, for example, an input control circuit, etc., which generates an input signal based on the information input by the user using the above-mentioned input device, and outputs the generated input signal to the CPU 901. The user of the information processor 900 can input various data to the information processor 900 or provide instructions for processing operations by operating the input unit 906.

[0272] The output unit 907 is formed of a unit capable of notifying the user of the acquired information in a visual or auditory manner. Examples of such a unit include display units such as a CRT display unit, a liquid crystal display unit, a plasma display unit, an EL display unit, and a lamp, a sound output unit such as a speaker and a headphone, a printer unit, and the like. The output unit 907 outputs, for example, a result obtained by various processes performed by the information processor 900. Specifically, the display visually displays the result obtained by various processes performed by the information processor 900 in various formats such as text, images, tables, graphs, and the like. At the same time, the sound output unit converts an audio signal including reproduced sound data, sound data, and the like into an analog signal, and outputs the analog signal in an audible manner.

[0273] The storage unit 908 is a data storage unit formed as an example of a storage portion of the information processor 900. The storage unit 908 is implemented by, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like. The storage unit 908 may include a storage medium, a recording unit for recording data on the storage medium, a reading unit for reading data from the storage medium, a deleting unit for deleting data recorded on the storage medium, and the like. The storage unit 908 stores a program to be executed by the CPU 901, various data, various externally acquired data, and the like.

[0274] The drive 909 is a reader / writer for a storage medium, and is incorporated in the information processor 900 or externally attached to the information processor 900. The drive 909 reads information recorded on a removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory mounted thereon, and outputs the information to the RAM 903. The drive 909 can also write information on the removable storage medium.

[0275] The connection port 911 is an interface to be connected to an external device, and is a connector of an external device capable of transmitting data by, for example, USB (Universal Serial Bus) or the like.

[0276] The communication unit 913 is a communication interface for connecting to the network 920 formed by, for example, a communication device or the like. The communication unit 913 may be, for example, a communication card for a wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB), etc. In addition, the communication unit 913 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various types of communication, etc. The communication unit 913 is capable of transmitting or receiving signals to or from the Internet or other communication devices according to a predetermined protocol such as TCP / IP, etc. The communication unit 913 may form, for example, the communicator 18, the communicator 22, and the communicator 32.

[0277] The sensor 915 is any type of sensor, for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, a sound sensor, a distance sensor, or a force sensor. The sensor 915 acquires information about the state of the information processor 900 itself, such as the orientation, movement speed, etc. of the information processor 900, and information about the surrounding environment of the information processor 900, such as the brightness or noise around the information processor 900. In addition, the sensor 915 may include a GPS sensor that receives a GPS signal to measure the latitude, longitude, and altitude of the unit. The sensor 915 may be formed, for example, of the gyro sensor 12 and the acceleration sensor 14.

[0278] Note that the network 920 is a wired or wireless transmission path for information transmitted from a unit connected to the network 920. For example, the network 920 may include a public network such as the Internet, a telephone network or a satellite communication network, various LANs (Local Area Networks) including Ethernet (registered trademark), WANs (Wide Area Networks), etc. In addition, the network 920 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

[0279] An example of a hardware configuration that can implement the functions of the information processor 900 according to an embodiment of the present disclosure has been described above. Each of the above components can be implemented by using a general-purpose component, or can be implemented by hardware dedicated to the function of the component. Therefore, the hardware configuration to be used can be appropriately changed according to the technical level when executing the embodiment of the present disclosure.

[0280] It should be noted that, as described above, a computer program for realizing each function of the information processor 900 may be created and installed in a PC or the like. In addition, a computer-readable recording medium storing such a computer program may be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. In addition, the above-mentioned computer program may be distributed, for example, via a network without using a recording medium.

[0281] <<8. Conclusion>>

[0282] As described above, according to the embodiments of the present disclosure, the position information of the body part to which the motion sensor is attached can be obtained with higher accuracy. In addition, according to the embodiments of the present disclosure, the position information and orientation information of the part to which the motion sensor is not attached can also be obtained. This makes it possible to obtain bone information using a smaller number of motion sensors.

[0283] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings; however, the technical scope of the present disclosure is not limited to such examples. It is obvious that a person skilled in the art of the present disclosure can conceive of various changes or modifications within the scope of the technical ideas described in the claims, and it should be understood that such changes and modifications are also within the technical scope of the present disclosure.

[0284] For example, in the aforementioned embodiments, an example in which an inertial sensor is attached to a body as a motion sensor is mainly described; however, the present technology is not limited to such an example. The motion sensor only needs to be a sensor that senses the motion of the body, and may be a geomagnetic sensor, an air pressure sensor, an image sensor, etc. as described above.

[0285] In addition, according to the order described as a flowchart in this article, the steps in the aforementioned embodiments may not necessarily be processed in chronological order. For example, the steps in the processing of the aforementioned embodiments may be processed in an order different from the order described in the flowchart, or may be performed in parallel.

[0286] In addition, the effects described herein are only illustrative or exemplary and are non-limiting. That is, the technology according to the present disclosure can achieve other effects that are obvious to those skilled in the art based on the description herein in addition to or instead of the above effects.

[0287] It is to be noted that the following configurations are also covered by the technical scope of the present disclosure. (1)

[0289] A program that causes a computer to implement the following correction function, referring to a first output obtained by performing a first processing on sensor data acquired by two or more motion sensors attached to two or more parts of a body, and a second output obtained by performing a second processing on the sensor data, and correcting position information of the attachment parts to which the motion sensors are attached. (2)

[0291] According to the procedure of (1),

[0292] Sensor data includes acceleration,

[0293] The first process involves the integration of the acceleration, and

[0294] The first output includes the position information of the attachment site obtained by integration. (3)

[0296] A procedure according to (1) or (2), wherein

[0297] Sensor data includes acceleration,

[0298] The second process includes a regression estimation process of estimating position information of the attachment site by regression, and

[0299] A regression estimation process is performed based on acceleration. (4)

[0301] According to the procedure of (3),

[0302] The sensor data also includes angular velocity, and

[0303] A regression estimation process is further performed based on the angular velocity. (5)

[0305] The program according to (4), wherein the acceleration is used as an input to the regression estimation process after the coordinate system conversion is performed based on the angular velocity. (6)

[0307] The program according to (4) or (5), wherein the orientation information of the attachment site obtained based on the angular velocity is used as an input of the regression estimation process. (7)

[0309] The program according to any one of (4) to (6), wherein acceleration or angular velocity acquired by the motion sensor is used as an input of the regression estimation process. (8)

[0311] The program according to any one of (3) to (7), wherein the regression estimation process is performed based on position information of the attachment site previously estimated by the regression estimation process. (9)

[0313] The program according to any one of (1) to (8) further causes the computer to implement an interpolation function of estimating position information or orientation information of a non-attachment site to which the motion sensor is not attached based on the position information of the attachment site corrected by the correction function. (10)

[0315] The program according to (9), wherein the interpolation function estimates position information or orientation information for a non-attachment site between a plurality of attachment sites in a predetermined skeletal structure. (11)

[0317] A program according to (9) or (10), wherein the interpolation function estimates the position information or orientation information of the non-attached part by regression. (12)

[0319] The program according to any one of (9) to (11), further causing the computer to implement a reliability identification function of identifying reliability indicating reliability of position information of the attachment site included in the first output or the second output. (13)

[0321] The program according to (12), wherein the correction function corrects the position information of the attachment site based on the reliability identified by the reliability identification function. (14)

[0323] The program according to any one of (3) to (8), wherein the second processing further includes an interpolation processing of estimating position information or orientation information of a non-attachment portion to which the motion sensor is not attached, based on the position information of the attachment portion estimated by the regression estimation processing. (15)

[0325] The program according to (14), wherein the correction function further corrects position information or orientation information of the non-attached part included in the second output based on the first output. (16)

[0327] The program according to any one of (9) to (15), wherein the correction function is implemented by a Kalman filter. (17)

[0329] The program according to (1), wherein the correction function detects a ground contact state and corrects position information of the attachment part based on the detection result. (18)

[0331] A program according to (17), wherein the correction function estimates the speed and position of the front foot using the acceleration and angular velocity included in the sensor data, and based on the estimation result, detects a ground contact state, the ground contact state being a state in which the front foot has a predetermined speed. (19)

[0333] An information processor includes a corrector that refers to a first output obtained by performing a first process on sensor data acquired by two or more motion sensors attached to more than two parts of a body, and a second output obtained by performing a second process on the sensor data, and corrects position information of attachment parts to which the motion sensors are attached. (20)

[0335] An information processing method includes: a processor referring to a first output obtained by performing a first processing on sensor data obtained by two or more motion sensors attached to more than two parts of a body, and a second output obtained by performing a second processing on the sensor data; and the processor correcting position information of the attachment part to which the motion sensor is attached.

[0336] Reference Numbers List

[0337] 10, 10A, 10B, 10C, 10D, 10E, 10F, 10G: Sensor unit

[0338] 12: Gyro sensor

[0339] 14: Acceleration sensor

[0340] 16: Controller

[0341] 18, 32: Communicator

[0342] 20: Hub unit

[0343] 22: Communicator

[0344] 24, 34: Controller

[0345] 30: Information Processor

[0346] 40, 907: Output unit

[0347] 50: Distance sensor

[0348] 510: Global Motion Acceleration Direction Calculator

[0349] 520: Points Calculator

[0350] 530: Selector

[0351] 540, 541, 544: Attachment location regression estimator

[0352] 550: Reliability Identifier

[0353] 560: Ground contact detector

[0354] 561: Estimator

[0355] 570, 572, 573, 574: Corrector

[0356] 580, 583: Interpolator

[0357] 900: Information Processor

[0358] 901: CPU

[0359] 902: ROM

[0360] 903: RAM

[0361] 904: Bridge

[0362] 904a: Host bus

[0363] 904b: External bus

[0364] 905: Interface

[0365] 906: Input unit

[0366] 908: Storage unit

[0367] 909: Drive

[0368] 911: Connection port

[0369] 913: Communication unit

[0370] 915: Sensor

[0371] 920: Network

[0372] D: Delay

[0373] D103, D104, D105, SD10, SD90, SD100, SD101, SD102, SD103, SD104, SD105: Bone structure

[0374] L1: upper limit

[0375] L2: Lower limit

[0376] P11, P12, P13, P14, P15, P16, P101, P102, P103, P104, P105, P106, SP101, SP102, SP103, SP105, SP106, SP107: Attachment site

[0377] PD10, PD12, PD100: Attachment site information

[0378] R10: Probability distribution

[0379] R20: Relative position

[0380] S100, S111, S112, S113, S114, S121, S122, S123, S130, S200, S502, S504, S506, S508, S510, S512, S514, S516, S518, S520, S600, S601, S602, S610, S611, S612, S613, S700, S702, S704, S706, S708, S710, S800, S802, S804, S806, S808, S810, S812, S900, S902, S904, S906, S906, S908, S910, S912: Steps

[0381] S53, S54, S55, S56, S57, S58, S59: Position estimation processing

[0382] SB101, SB81, SB82: Skeleton

[0383] SP11, SP12, SP20, SP81, SP82, SP83, SP91, SP92, SP93, SP94, SP95, SP96: Part

[0384] U1: User

Claims

1. A storage medium, the storage medium comprising a program, which, when executed by a computer comprising the storage medium, enables the computer to implement the following correction function: referring to first orientation information and first position information of an attachment site obtained by performing a first process on sensor data acquired by two or more motion sensors attached to two or more parts of a body, and second position information estimated by performing a dynamic model on the sensor data, and correcting the first position information of the attachment site to which the motion sensor is attached by using the first orientation information and the second position information, The computer is further caused to implement an interpolation function of estimating position information or orientation information of a non-attachment portion to which the motion sensor is not attached, excluding the attachment portion, based on the first orientation information and first position information of the attachment portion corrected by the correction function.

2. The storage medium according to claim 1, wherein The sensor data includes acceleration, The first processing includes integration of acceleration, and The first position information includes position information of the attachment site obtained by integration.

3. The storage medium according to claim 1, wherein The sensor data includes acceleration, The second process includes a regression estimation process of estimating second position information of the attachment site by regression, and The regression estimation process is performed based on the acceleration.

4. The storage medium according to claim 3, wherein The sensor data also includes angular velocity, and The regression estimation process is further performed based on the angular velocity.

5. The storage medium according to claim 4, wherein: The acceleration is used as an input to the regression estimation process after being coordinate-system-converted based on the angular velocity.

6. The storage medium according to claim 4, wherein: The first orientation information of the attachment part obtained based on the angular velocity is used as an input of the regression estimation process.

7. The storage medium according to claim 4, wherein: The acceleration or the angular velocity acquired by the motion sensor is used as an input of the regression estimation process.

8. The storage medium according to claim 3, wherein: The regression estimation process is performed based on the position information of the attachment site previously estimated by the regression estimation process.

9. The storage medium according to claim 1, wherein: The interpolation function estimates the position information or the orientation information for the non-attachment site between a plurality of the attachment sites in a predetermined bone structure.

10. The storage medium according to claim 1, wherein: The interpolation function estimates the position information or the orientation information of the non-attached part by regression.

11. The storage medium according to claim 1, further causing the computer to implement a reliability identification function of identifying reliability indicating reliability of the position information of the attachment site included in the first position information or the second position information.

12. The storage medium according to claim 11, wherein: The correction function corrects the first position information of the attachment site based on the reliability identified by the reliability identification function.

13. The storage medium according to claim 3, wherein: The second process further includes an interpolation process of estimating position information or orientation information of a non-attachment portion to which the motion sensor is not attached, based on the second position information of the attachment portion estimated by the regression estimation process.

14. The storage medium according to claim 13, wherein: The correction function also corrects the position information or the orientation information of the non-attachment site included in the second position information based on the first position information.

15. The storage medium according to claim 1, wherein: The correction function is implemented by a Kalman filter.

16. The storage medium according to claim 1, wherein: The correction function detects a ground contact state and corrects the first position information of the attachment part based on the detection result.

17. The storage medium according to claim 16, wherein: The correction function estimates the speed and position of the front foot using the acceleration and angular velocity included in the sensor data, and based on the estimation result, detects the ground contact state, which is a state in which the front foot has a predetermined speed.

18. An information processor, comprising a corrector, the corrector referring to first orientation information and first position information of an attachment site obtained by performing a first process on sensor data acquired by two or more motion sensors attached to two or more sites of a body, and second position information estimated by performing a dynamic model on the sensor data, and correcting the first position information of the attachment site to which the motion sensor is attached by using the first orientation information and the second position information, Position information or position information of a non-attachment site to which the motion sensor is not attached is estimated based on the first orientation information and the first position information of the attachment site corrected by the correction function, the non-attachment site not including the attachment site.

19. An information processing method, comprising: The processor refers to first orientation information and first position information of the attachment site obtained by performing a first process on sensor data acquired by two or more motion sensors attached to two or more parts of the body, and second position information estimated by performing a dynamic model on the sensor data; and the processor corrects the first position information of the attachment site to which the motion sensor is attached by using the first orientation information and the second position information, Position information or position information of a non-attachment site to which the motion sensor is not attached is estimated based on the first orientation information and the first position information of the attachment site corrected by the correction function, the non-attachment site not including the attachment site.

Images