Information processing device, information processing method, and program

Abstract

To achieve reproduction of interaction between an acting entity and an object by a simpler configuration.SOLUTION: An information processing device includes: a time synchronization part for performing time synchronization related to motion data on an object in an actual space and motion data on an acting entity in the actual space; a contact time detection part for detecting the contact time of the object and the acting entity on the basis of the motion data on the object and the motion data on the acting entity that have been synchronized; and a contact position estimation part for estimating a contact position of the object and the acting entity on the basis of the motion data on the object and the motion data on the acting entity that have been synchronized, and the contact time.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, technologies for acquiring motion data related to a user's motion using sensors have been developed. Further, for example, as disclosed in Patent Document 1, there is a technology for detecting an interaction between a user and an object different from the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technology disclosed in Patent Document 1 only detects the above-described interaction and does not reach the reproduction of the interaction. Further, when attempting to reproduce the interaction between a user and an object, generally high-precision sensing is required, and the equipment cost and the calculation cost increase.

Means for Solving the Problems

[0005] According to an aspect of the present disclosure, there is provided an information processing apparatus including: a time synchronization unit that performs time synchronization on motion data of an object in a real space and motion data of an acting body in the real space; a contact time detection unit that detects a contact time between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body; and a contact position estimation unit that estimates a contact position between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and the contact time.

[0006] Also, according to another aspect of the present disclosure, a processor performs time synchronization for motion data of an object in real space and motion data of an acting body in the real space, detects a contact time between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and estimates a contact position between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and the contact time. An information processing method is provided.

[0007] Also, according to another aspect of the present disclosure, a computer is caused to function as an information processing apparatus including a time synchronization unit that performs time synchronization for motion data of an object in real space and motion data of an acting body in the real space, a contact time detection unit that detects a contact time between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and a contact position estimation unit that estimates a contact position between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and the contact time. A program is provided.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Embodiments for Carrying Out the Invention

[0009] With reference to the accompanying drawings, preferred embodiments of the present disclosure will be described in detail below. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] In addition, in the present specification and drawings, when describing a plurality of components of the same type separately, an alphabet or the like may be appended to the end of the reference numeral. On the other hand, when there is no need to distinguish a plurality of components of the same type, the above-mentioned alphabet or the like is omitted, and there may be cases where a description common to all of the plurality of components of the same type is given.

[0011] Note that the description will be made in the following order. 1. Embodiment 1.1. Overview 1.2. Example of functional configuration 1.3. Functional details 1.4. Application examples 2. Example of hardware configuration 3. Summary

[0012] <1. Embodiment> <<1.1. Overview>> As described above, when attempting to reproduce the interaction between an acting entity such as a user and an object different from the acting entity, it is required to accurately measure the motions of the acting entity and the object and perform drawing based on the measurement results.

[0013] Examples of methods for accurately measuring the motions of the acting entity and the object include, for example, optical motion capture in which markers are attached to the acting entity and the object for measurement, and video motion capture in which the skeleton of the acting entity and the behavior of the object are measured from a plurality of moving images.

[0014] However, when performing measurement using the above methods, high costs for measurement equipment are required.

[0015] In addition, when measurement is performed using the above-described method, sufficient computing resources for processing the highly accurate motion data are required.

[0016] Furthermore, when measurement is performed using the above-described method, since the environment where the measurement equipment can be installed is limited, the environment where measurement is possible is also limited.

[0017] The technical idea according to an embodiment of the present disclosure was conceived by paying attention to the above points, and realizes the reproduction of the interaction between the operating entity and the object with a simpler configuration.

[0018] FIG. 1 is a diagram for explaining an outline of an information processing method according to an embodiment of the present disclosure.

[0019] On the left side of FIG. 1, a measurement environment in the real space 50 is illustrated.

[0020] In the real space 50, a first sensor group 10N including N first sensors 10 is attached to an operating entity 52 such as a user.

[0021] Note that in FIG. 1, a case where the first sensor group 10N includes the following six first sensors 10a to 10f is illustrated, but the number of the first sensors 10 included in the first sensor group 10N and the attachment locations are not limited to such an example.

[0022] First sensor 10a: Attached to the head of the operating entity 52 First sensor 10b: Attached to the waist of the operating entity 52 First sensor 10c: Attached to the right wrist of the operating entity 52 First sensor 10d: Attached to the left wrist of the operating entity 52 First sensor 10e: Attached to the right ankle of the operating entity 52 First sensor 10f: Attached to the left ankle of the operating entity 52

[0023] The first sensor group 10N acquires motion data of the operating entity 52. Each of the first sensors 10 included in the first sensor group 10N may be an inertial measurement unit (IMU).

[0024] Also, in the real space 50, at least one second sensor 20 is attached to an object 54 different from the operating entity 52.

[0025] For example, when the object 54 is a ball, the second sensor 20 may be embedded inside the ball.

[0026] The second sensor 20 acquires motion data of the object 54. The second sensor 20 may be an inertial measurement device.

[0027] In the information processing method according to an embodiment of the present disclosure, based on the motion data of the operating entity 52 and the motion data of the object 54 acquired as described above, as shown on the right side of FIG. 1, the avatar 72 and the virtual object 74 in the virtual space 70 are drawn.

[0028] The avatar 72 is a computer graphics (CG) corresponding to the operating entity 52. Also, the virtual object 74 is a CG corresponding to the object 54.

[0029] A processor that implements the information processing method according to an embodiment of the present disclosure may draw the avatar 72 and the virtual object 74 so as to reproduce the interaction between the operating entity 52 and the object 54 in the real space 50.

[0030] For this purpose, the information processing method according to an embodiment of the present disclosure includes performing time synchronization between the motion data of the object 54 in the real space 50 and the motion data of the operating entity 52 in the real space 50.

[0031] In addition, an information processing method according to an embodiment of the present disclosure includes detecting a contact time between the object 54 and the acting body 52 based on the motion data of the synchronized object 54 and the motion data of the acting body 52.

[0032] Further, an information processing method according to an embodiment of the present disclosure includes estimating a contact position between the object 54 and the acting body 52 based on the synchronized motion data of the object 54 and the motion data of the acting body 52, and the contact time.

[0033] Furthermore, an information processing method according to an embodiment of the present disclosure may further include rendering an interaction between the avatar 72 and the virtual object 74 in the virtual space 70 based on the contact time and the contact position.

[0034] According to the above configuration, it is possible to realize the reproduction of the interaction between the acting body 52 and the object 54 with a simpler configuration.

[0035] Hereinafter, an example of a functional configuration for implementing the information processing method according to an embodiment of the present disclosure will be described.

[0036] <<1.2. Example of Functional Configuration>> FIG. 2 is a block diagram showing an example of the functional configuration of the information processing system 1 according to an embodiment of the present disclosure.

[0037] As shown in FIG. 2, the information processing system 1 according to the present embodiment may include a first sensor group 10N, at least one second sensor 20, and an information processing device 30.

[0038] (First Sensor Group 10N) As described above, the first sensor group 10N according to the present embodiment includes N first sensors 10.

[0039] Each of the N first sensors 10 is attached to a predetermined location on the body of the acting body 52.

[0040] In addition, in the present embodiment, it is assumed that the operation subject 52 is a person as the main example, but the operation subject 52 is not limited to such an example.

[0041] The operation subject 52 may be an animal other than a person or an autonomous mobile body.

[0042] (First Sensor 10) The first sensor 10 according to the present embodiment acquires motion data of the operation subject 52. The first sensor 10 according to the present embodiment may be an inertial measurement device.

[0043] As shown in FIG. 2, the first sensor 10 according to the present embodiment includes a wireless communication unit 110 and a sensor unit 120.

[0044] (Wireless Communication Unit 110) The wireless communication unit 110 performs wireless communication compliant with a predetermined communication standard with the information processing device 30.

[0045] For example, the wireless communication unit 110 transmits the motion data acquired by the sensor unit 120 to the information processing device 30.

[0046] Examples of the predetermined communication standard include Bluetooth (registered trademark) Low Energy (BLE). However, the predetermined communication standard is not limited to such an example.

[0047] (Sensor Unit 120) The sensor unit 120 acquires motion data of the operation subject 52.

[0048] The motion data according to the present embodiment may include, for example, three-axis acceleration and three-axis angular velocity.

[0049] The sensor unit 120 includes various sensors for acquiring motion data, such as an acceleration sensor and a gyro sensor.

[0050] (Second Sensor 20) The second sensor 20 according to this embodiment is attached to a predetermined position of the object 54.

[0051] In this embodiment, it is mainly assumed that the object 54 is a ball, but the object 54 is not limited to such an example. The object 54 may be any object.

[0052] As shown in FIG. 2, the second sensor 20 according to this embodiment includes a wireless communication unit 210 and a sensor unit 220.

[0053] (Wireless communication unit 210) The wireless communication unit 210 performs wireless communication compliant with a predetermined communication standard with the information processing device 30.

[0054] For example, the wireless communication unit 210 transmits the motion data acquired by the sensor unit 220 to the information processing device 30.

[0055] (Sensor unit 220) The sensor unit 220 acquires the motion data of the object 54.

[0056] The sensor unit 220 includes various sensors for acquiring motion data, such as an acceleration sensor and a gyro sensor.

[0057] (Information processing device 30) The information processing device 30 is a computer that implements the information processing method according to this embodiment.

[0058] As shown in FIG. 2, the information processing device 30 according to this embodiment may include a wireless communication unit 310, a processing unit 320, and a display unit 330.

[0059] (Wireless communication unit 310) The wireless communication unit 310 performs wireless communication compliant with a predetermined communication standard with each of the first sensor 10 and the second sensor 20.

[0060] For example, the wireless communication unit 310 receives motion data from each of the first sensor 10 and the second sensor 20.

[0061] (Processing unit 320) The processing unit 320 according to the present embodiment performs various processes based on the motion data of the moving body 52 and the motion data of the object 54 received by the wireless communication unit 310.

[0062] The functions of the processing unit 320 according to the present embodiment are realized by various processors.

[0063] FIG. 3 is a block diagram showing a functional configuration example of the processing unit 320 according to the present embodiment.

[0064] As shown in FIG. 3, the processing unit 320 according to the present embodiment may include a calibration unit 321, a time synchronization unit 322, a contact time detection unit 323, a contact position estimation unit 324, and a drawing unit 325.

[0065] (Calibration unit 321) The calibration unit 321 according to the present embodiment performs calibration related to the first sensor group 10N.

[0066] In the calibration according to the present embodiment, in order to realize more accurate motion capture, the orientation of each of the first sensors 10, the height of the moving body 52, etc. are acquired, and various adjustments are made based on the acquired various information.

[0067] By performing the calibration as described above, the skeleton information of the moving body 52, which will be described later, can be acquired.

[0068] (Time synchronization unit 322) The time synchronization unit 322 according to the present embodiment performs time synchronization related to the motion data of the object 54 in the real space 50 and the motion data of the moving body 52 in the real space 50.

[0069] Details of the functions of the time synchronization unit 322 according to this embodiment will be described later.

[0070] (Contact time detection unit 323) The contact time detection unit 323 according to this embodiment detects the time (contact time) when the object 54 and the operating body 52 come into contact based on the motion data of the synchronized object 54 and the motion data of the operating body 52.

[0071] Details of the functions of the contact time detection unit 323 according to this embodiment will be described later.

[0072] (Contact position estimation unit 324) The contact position estimation unit 324 according to this embodiment estimates the position (contact position) where the object 54 and the operating body 52 come into contact based on the motion data of the synchronized object 54, the motion data of the operating body 52, and the contact time.

[0073] Details of the functions of the contact position estimation unit 324 according to this embodiment will be described later.

[0074] (Drawing unit 325) The drawing unit 325 according to this embodiment draws the interaction between the avatar 72 corresponding to the operating body 52 in the virtual space 70 and the virtual object 74 corresponding to the object 54 in the virtual space 70 based on the contact time and the contact position.

[0075] Details of the functions of the drawing unit 325 according to this embodiment will be described later.

[0076] (Display unit 330) The display unit 330 according to this embodiment displays various information according to the control by the processing unit 320.

[0077] For example, the display unit 330 according to this embodiment displays the result of drawing by the drawing unit 325.

[0078] The display unit 330 includes various displays.

[0079] The above described the functional configuration example of the information processing system 1 according to the present embodiment. Note that the above-described functional configuration described with reference to FIGS. 2 and 3 is merely an example, and the functional configuration of the information processing system 1 according to the present embodiment is not limited to such an example.

[0080] For example, the above-described calibration, acquisition of skeletal information based on calibration, etc. may be performed by a device (such as a smartphone) separate from the information processing device 30.

[0081] Also, for example, time synchronization, contact time detection, contact position estimation, drawing, and display may be realized by cooperation of a plurality of devices.

[0082] The functional configuration of the information processing system 1 according to the present embodiment can be flexibly deformed according to specifications, operations, etc.

[0083] <<1.3. Functional Details>> Next, the functions of the processing unit 320 according to the present embodiment will be described in detail. In the following, as an interaction between the moving body 52 and the object 54, lifting in soccer or the like will be mainly described as an example.

[0084] Also, in the following, the description will be made with reference to FIGS. 4A and 4B. FIGS. 4A and 4B are diagrams showing examples of data input and output in the processing unit 320 according to the present embodiment.

[0085] First, time synchronization by the time synchronization unit 322 according to the present embodiment will be described.

[0086] As shown in FIG. 4A, motion data of the moving body 52 and motion data of the object 54 are input to the time synchronization unit 322 according to the present embodiment.

[0087] The motion data of the moving body 52 input to the time synchronization unit 322 may include skeletal information acquired based on acceleration and angular velocity sensed by the first sensor group 10N.

[0088] In addition, the motion data of the object 54 input to the time synchronization unit 322 includes at least acceleration.

[0089] The time synchronization unit 322 performs time synchronization based on the input data, and outputs the motion data of the operating body 52 after time synchronization and the motion data of the object 54 after time synchronization.

[0090] FIG. 5 is a diagram for explaining the time synchronization according to the present embodiment.

[0091] FIG. 5 shows an image of the motion data of the object 54 and the motion data of the operating body 52 acquired in time series.

[0092] In FIG. 5, the motion data of the object 54 at the 0th frame before time synchronization is indicated by a dotted circle, and the motion data of the operating body 52 at the 0th frame before time synchronization is indicated by a hatched circle.

[0093] In addition, in FIG. 5, the motion data of the object 54 at the 0th frame after time synchronization and the motion data of the operating body 52 at the 0th frame after time synchronization are both indicated by a solid white circle.

[0094] In the time synchronization according to the present embodiment, for example, before measuring lifting, an instruction is given via the display unit 330 or the like so that the operating body 52 jumps while holding the object 54.

[0095] In this case, after the above instruction is given, the time synchronization unit 322 synchronizes the time when an impact exceeding a threshold value in the object 54 is detected and the time when it is detected that both feet of the operating body 52 leave the ground and then re-contact the ground.

[0096] The time synchronization unit 322 can detect the time (indicated by a solid white arrow) when an impact exceeding a threshold value occurs in the object 54 based on the acceleration of the object 54 or the like.

[0097] For example, when the time at which an impact exceeding a threshold value in the object 54 is detected is the 171st frame before time synchronization, the time synchronization unit 322 may set the frame as the 0th frame after time synchronization.

[0098] In addition, the time synchronization unit 322 can detect the time when both feet of the motion main body 52 leave the ground (indicated by a symbol combining a double circle and a cross) and then re - contact the ground based on the skeletal information of the motion main body 52 and the like.

[0099] For example, when the time at which it is detected that both feet of the motion main body 52 have left the ground and then re - contacted the ground is the 130th frame before time synchronization, the time synchronization unit 322 may set the frame as the 0th frame after time synchronization.

[0100] According to the processing as described above, it is possible to synchronize the acquisition times of the motion data of the motion main body 52 and the motion data of the object 54. However, the time synchronization is not limited to the above - described processing, and any method may be adopted.

[0101] Next, the contact time detection by the contact time detection unit 323 according to the present embodiment will be described.

[0102] As shown in FIG. 4A, the motion data of the motion main body 52 and the motion data of the object 54 after time synchronization are input to the contact time detection unit 323 according to the present embodiment.

[0103] The contact time detection unit 323 according to the present embodiment may perform contact time detection based on the input data and output the motion data of the motion main body 52 and the motion data of the object 54 after time synchronization, and the detected contact time.

[0104] FIG. 6 is a diagram for explaining the contact time detection according to the present embodiment.

[0105] FIG. 6 shows an image of the motion data of the object 54 after time synchronization, acquired in time series.

[0106] In an interaction involving physical contact such as lifting, it is assumed that an impact corresponding to the interaction occurs on the object 54 at the time when the interaction is performed.

[0107] Therefore, the contact time detection unit 323 according to the present embodiment may detect, as the contact time, the time (indicated by the diagonal arrow) when an impact exceeding a certain threshold value occurs on the object 54 after time synchronization.

[0108] In FIG. 6, the motion data of the object 54 at the 0th frame after time synchronization is shown by a white circle. Also, in FIG. 6, the motion data of the object 54 at the Nth frame detected as the contact time is shown by a double circle.

[0109] Further, the contact time detection unit 323 according to the present embodiment may estimate the detected contact time intervals as the hovering time of the object 54 in lifting.

[0110] The estimated hovering time is used for the drawing of the virtual object 74 by the drawing unit 325. Note that the hovering time may be estimated by the drawing unit 325.

[0111] Next, the contact position estimation by the contact position estimation unit 324 according to the present embodiment will be described.

[0112] As shown in FIG. 4A, the motion data of the acting body 52 after time synchronization, the motion data of the object 54, and the detected contact time are input to the contact position estimation unit 324 according to the present embodiment.

[0113] The contact position estimation unit 324 according to the present embodiment may perform contact position estimation based on the input data, and output the motion data of the acting body 52 after time synchronization, the motion data of the object 54, the contact time, and the result of the contact position estimation.

[0114] For example, the contact position estimation unit 324 according to the present embodiment may estimate, as the contact position, the body part (also referred to as the contact part) of the motion subject 52 that has come into contact with the object 54.

[0115] In this case, the contact position estimation unit 324 according to the present embodiment outputs to the drawing unit 325 the name of the contact part (or information capable of identifying the contact part) and the coordinates of the contact part.

[0116] First, the contact position estimation unit 324 according to the present embodiment acquires the three-dimensional coordinates (simply referred to as coordinates) of each joint from the skeletal information included in the motion data of the motion subject 52 based on the contact time.

[0117] FIG. 7 is a diagram for explaining an example of the skeletal information according to the present embodiment.

[0118] In the case of an example shown in FIG. 7, the skeletal information includes information regarding a plurality of joints J0 to J26 and a plurality of bones connecting the joints.

[0119] FIG. 8 is a diagram showing a detailed example of each joint defined in the skeletal information according to the present embodiment.

[0120] Index in FIG. 8 corresponds to the numerical part of the reference numeral shown in FIG. 7.

[0121] As shown in FIG. 8, the skeletal information according to the present embodiment may include an Index, a name, etc. for specifying the defined joints.

[0122] Further, the skeletal information according to the present embodiment includes the three-dimensional coordinates of the joints estimated from the acceleration and angular velocity acquired by the first sensor group 10N.

[0123] The contact position estimation unit 324 according to the present embodiment acquires the three-dimensional coordinates of a plurality of joints at the contact time, and estimates the contact position based on the positional relationship between the joints.

[0124] The contact position estimation unit 324 according to the present embodiment may estimate the contact position using, for example, a rule-based method.

[0125] FIG. 9 is a flowchart showing an example of the flow of contact position estimation using the rule-based method according to the present embodiment.

[0126] The contact position estimation unit 324 first acquires the coordinates of a plurality of joints at the contact time (S101).

[0127] In the case of the example shown in FIG. 9, the contact position estimation unit 324 acquires the coordinates of 10 joints indicated by dots in FIG. 8.

[0128] Next, the contact position estimation unit 324 determines whether one leg of the operating body 52 is floating based on the coordinates of the 10 joints acquired in step S101 (S102).

[0129] When the contact position estimation unit 324 determines that one leg is not floating (S102: NO), subsequently, based on the coordinates of the 10 joints acquired in step S101, it determines whether the face of the operating body is facing upward (S103).

[0130] When the contact position estimation unit 324 determines that the face is facing upward (S103: YES), it determines that the operating body 52 is in contact with the object 54 by the head (S104), and ends the contact position estimation.

[0131] On the other hand, when the contact position estimation unit 324 determines that the face is not facing upward (S103: NO), it assumes that the operating body 52 touches the object 54 with only one foot slightly floating, and proceeds to the process of step S106.

[0132] Also, when the contact position estimation unit 324 determines that one leg is floating (S102: YES), or when it determines that the face is not facing upward (S103: NO), it identifies the floating leg based on the coordinates of the 10 joints acquired in step S101 (S106).

[0133] Next, based on the coordinates of the 10 joints acquired in step S101, the contact position estimation unit 324 determines whether the area formed by the tip of the floating foot side, the knee, and the hip joint is equal to or greater than a threshold value (S107). More specifically, the area may be the area of a triangle with the tip of the floating foot side, the knee, and the hip joint as vertices respectively.

[0134] When the contact position estimation unit 324 determines that the area formed by the tip of the floating foot side, the knee, and the hip joint is less than the threshold value (S107: NO), it determines that the moving body 52 is in contact with the object 54 by the instep of the floating foot side (S108), and ends the contact position estimation.

[0135] On the other hand, when the contact position estimation unit 324 determines that the area formed by the tip of the floating foot side, the knee, and the hip joint is equal to or greater than the threshold value (S107: YES), subsequently, it determines whether the distance between the ankle and the waist on the floating foot side is equal to or greater than the threshold value (S109).

[0136] When the contact position estimation unit 324 determines that the distance between the ankle and the waist on the floating foot side is less than the threshold value (S109: NO), it determines that the moving body 52 is in contact with the object 54 by the inside or outside of the floating foot side (S110), and ends the contact position estimation.

[0137] On the other hand, when the contact position estimation unit 324 determines that the distance between the ankle and the waist on the floating foot side is equal to or greater than the threshold value (S109: YES), it determines that the moving body 52 is in contact with the object 54 by the thigh on the floating foot side (S111), and ends the contact position estimation.

[0138] As described above, the contact position estimation unit 324 according to the present embodiment may estimate the contact position using a rule-based method.

[0139] On the other hand, the contact position estimation unit 324 according to the present embodiment may estimate the contact position using a machine learning method.

[0140] FIG. 10 is a diagram showing the data flow of contact position estimation using the machine learning method according to the present embodiment.

[0141] When performing contact position estimation using the machine learning method, as shown in FIG. 10, the contact position estimation unit 324 includes a contact position classifier 326.

[0142] The contact position classifier 326 according to the present embodiment is generated in advance by supervised learning based on, for example, a set of learning data including the coordinates of 27 points of all joints at the contact time and information (e.g., contact site name) regarding the correct contact site given by a developer or the like for the learning data.

[0143] The contact position classifier 326 generated by the above learning can output information regarding the contact site based on the coordinates of 27 points of all joints at the input contact time.

[0144] In addition, the contact position estimation unit 324 according to the present embodiment can acquire the coordinates of the contact site from the skeleton information based on the information regarding the contact site output by the contact position classifier 326.

[0145] According to the contact position estimation using the machine learning method according to the present embodiment, similar to the contact position estimation using the rule-based method, for example, classification such as head / thigh / instep / inside or outside can be performed.

[0146] However, the above classification is merely an example for both the contact position estimation using the rule-based method and the contact position estimation using the machine learning method, and the number of classifications and the setting of the contact site can be flexibly deformed.

[0147] Next, drawing of the avatar 72, virtual object 74, etc. by the drawing unit 325 according to the present embodiment will be described.

[0148] As shown in FIG. 4B, the drawing unit 325 according to the present embodiment receives the motion data of the moving entity 52, the motion data of the object 54, the contact time, the contact part name, and the contact part coordinates after time synchronization output from the contact position estimation unit 324.

[0149] In addition, the drawing unit 325 according to the present embodiment receives avatar data, virtual object data, and virtual space data from a storage unit (not shown).

[0150] The above avatar data includes CG materials and the like required for drawing the avatar 72. The above virtual object data includes CG materials and the like required for drawing the virtual object 74. Also, the above virtual space data includes CG materials and the like required for drawing the virtual space 70.

[0151] The drawing unit 325 according to the present embodiment can draw the motion of the avatar 72 based on the skeleton information included in the motion data of the moving entity 52 and the above avatar data.

[0152] In addition, the drawing unit 325 according to the present embodiment performs a physical calculation based on the contact position and the hang time estimated based on the contact time, estimates the trajectory of the virtual object 74, and draws the motion of the virtual object 74 based on the estimated trajectory and the above virtual object data.

[0153] FIG. 11 is a diagram for explaining the estimation of the trajectory of the virtual object 74 according to the present embodiment.

[0154] FIG. 11 illustrates the skeleton information Ma and the contact position at the contact time Ta, the skeleton information Mb and the contact position at the contact time Tb, and the skeleton information Mc and the contact position at the contact time Tc.

[0155] When drawing the motion of the virtual object 74 between the contact time Ta and the contact time Tb, the drawing unit 325 sets the coordinates of the contact part estimated based on the skeleton information Ma (the coordinates of Ma) as the launch coordinates of the virtual object 74.

[0156] Further, the drawing unit 325 sets the coordinates of the contact part estimated based on the skeleton information Mb (the coordinates of Mb) as the landing coordinates of the virtual object 74.

[0157] Based on the above launch coordinates, the above landing coordinates, and the flight time estimated from the contact times Ta and Tb, the drawing unit 325 can estimate the trajectory of the virtual object 74 between the contact times Ta and Tb.

[0158] Further, the drawing unit 325 can draw the motion of the virtual object 74 based on the estimated trajectory of the virtual object 74 and the virtual object data.

[0159] When drawing the motion of the virtual object 74 between the contact times Tb and Tc, the drawing unit 325 sets the coordinates of the contact part estimated based on the skeleton information Mb (the coordinates of Mb) as the launch coordinates of the virtual object 74.

[0160] Further, the drawing unit 325 sets the coordinates of the contact part estimated based on the skeleton information Mc (the coordinates of Mc) as the landing coordinates of the virtual object 74.

[0161] Based on the above launch coordinates, the above landing coordinates, and the flight time estimated from the contact times Tb and Tc, the drawing unit 325 can estimate the trajectory of the virtual object 74 between the contact times Tb and Tc.

[0162] Further, the drawing unit 325 can draw the motion of the virtual object 74 based on the estimated trajectory of the virtual object 74 and the virtual object data.

[0163] The estimation of the trajectory of the virtual object 74 according to the present embodiment has been described above. According to the above processing, it is possible to reproduce the interaction between the acting body 52 and the object 54 in the real space 50 as the interaction between the avatar 72 and the virtual object 74 in the virtual space 70.

[0164] <<1.4. Application Example>> Next, an application example of the information processing method according to the present embodiment will be described.

[0165] As described above, the information processing method according to the present embodiment is applicable to the reproduction of lifting in soccer or the like.

[0166] FIG. 12 is a flowchart showing an example of the flow of reproducing lifting according to the present embodiment.

[0167] In the case of the example shown in FIG. 12, first, the time synchronization unit 322 performs time synchronization processing (S201).

[0168] After the time synchronization processing in step S201, the measurement of lifting is started (S202).

[0169] The start of lifting may be started by the operation entity 52 (hereinafter referred to as the user in this example) releasing the object 54 (hereinafter referred to as the ball in this example) from the hand.

[0170] After the start of measuring the lifting in step S202, the contact time detection unit 323 first determines whether the object 54, that is, the ball, is located on the ground (S203).

[0171] The contact time detection unit 323 may estimate the approximate height of the ball based on inertial navigation using the motion data acquired by the second sensor 20 attached to the ball, and determine whether the ball is located on the ground based on the result of the estimation. For example, the contact time detection unit 323 determines whether the ball is located on the ground based on whether the approximate height of the ball is smaller than a predetermined height.

[0172] When the contact time detection unit 323 determines that the ball is located on the ground (S203: YES), the measurement of lifting ends (S209).

[0173] On the other hand, when the contact time detection unit 323 determines that the ball is not located on the ground (S203: NO), it attempts to detect the contact between the user and the ball (S204).

[0174] If the contact between the user and the ball is not detected (S204: NO), the process may return to step S203.

[0175] On the other hand, if the contact between the user and the ball is detected (S204: YES), the contact position estimation unit 324 estimates the contact position based on the detected contact time (S205).

[0176] The drawing unit 325 estimates the trajectory of the ball based on the contact position estimated in step S205, etc. (S206).

[0177] Subsequently, the drawing unit 325 performs drawing processing based on the estimated trajectory of the ball, etc. (S207).

[0178] The processing unit 320 attempts to detect an end operation such as the user recovering the ball by hand (S208). If the end operation is detected (S208: YES), the measurement of lifting is terminated (S209), and a series of processes also ends.

[0179] On the other hand, if the end operation is not detected (S208: NO), the process may return to step S203.

[0180] The flow of reproducing lifting according to the present embodiment has been described above.

[0181] As described above, according to the information processing method according to the present embodiment, it is possible to reproduce the interaction between the user and the ball in the real space 50 in the virtual space 70.

[0182] Note that the application range of the information processing method according to the present embodiment is not limited to interactions where the operating entity 52 and the object 54 have a one-to-one relationship, such as lifting.

[0183] For example, the information processing method according to the present embodiment is also applicable to interactions where the operating entity 52 and the object 54 have an N-to-one relationship.

[0184] FIG. 13 is a diagram for explaining a case where the information processing method according to the present embodiment is applied to the reproduction of a volleyball game.

[0185] As shown in FIG. 13, a volleyball game is performed by a plurality of acting bodies 52 (players). In FIG. 13, four players of acting bodies 52a to 52d are illustrated, but the number of acting bodies 52 is not limited. The information processing method according to the present embodiment can be similarly applied to the reproduction of volleyball games with six players or nine players.

[0186] A first sensor group 10N (not shown) is attached to each of the plurality of acting bodies 52, and motion data related to each of the plurality of acting bodies 52 is acquired.

[0187] In addition, a second sensor 20 (not shown) is attached to the object 54, and motion data related to the object 54 is acquired.

[0188] The information processing apparatus 30 according to the present embodiment estimates the contact time and contact position between the object 54 and each of the plurality of acting bodies 52 based on the acquired motion data.

[0189] In addition, the information processing apparatus 30 according to the present embodiment estimates the trajectory of the object 54 based on the estimated contact time and contact position.

[0190] According to the above processing, it becomes possible to reproduce an interaction between a certain object 54 and a plurality of acting bodies 52, such as a volleyball game, in the virtual space 70.

[0191] In addition, the information processing method according to the present embodiment can also be applied to the reproduction of the interaction between the acting body 52 and the object 54 via the tool 56.

[0192] FIG. 14 is a diagram for explaining a case where the information processing method according to the present embodiment is applied to the reproduction of a table tennis game.

[0193] In a table tennis game, the acting body 52 (player) interacts with the object 54 (ball) via the tool 56 (racket).

[0194] A first sensor group 10N (not shown) is attached to each of the plurality of acting bodies 52, and motion data related to each of the plurality of acting bodies 52 is acquired.

[0195] In addition, a second sensor 20 (not shown) is attached to the object 54, and motion data related to the object 54 is acquired.

[0196] In addition, the information processing device 30 detects the contact time between the object 54 and the tool 56.

[0197] In addition, the information processing device 30 detects the contact time between the object 54 and the table 58.

[0198] The information processing device 30 may detect the contact time between the object 54 and the tool 56 and the contact time between the object 54 and the table 58 based on the sensing data acquired by the inertial measurement device attached to the tool 56 and the table 58.

[0199] For example, the information processing device 30 may detect the time when an impact exceeding a threshold value is detected in the sensing data acquired by the inertial measurement device attached to the tool 56 as the contact time between the object 54 and the tool 56.

[0200] Similarly, the information processing device 30 may detect the time when an impact exceeding a threshold value is detected in the sensing data acquired by the inertial measurement device attached to the table 58 as the contact time between the object 54 and the table 58.

[0201] On the other hand, the information processing device 30 may detect the contact time between the object 54 and the tool 56 and the contact time between the object 54 and the table 58 based on the sensing data acquired by the microphone installed around the table 58.

[0202] In this case, for example, the information processing apparatus 30 may detect the time when the sound pressure exceeding the threshold value is detected as the contact time between the tool 56 or the table 58 and the object 54.

[0203] In this case, for example, the information processing apparatus 30 may estimate the contact position based on the motion data of the operating body 52.

[0204] Further, the information processing apparatus 30 may estimate the sound source position based on the sensing data acquired by a plurality of microphones, and may estimate the object and the contact position with which the object 54 is in contact.

[0205] The information processing apparatus 30 can estimate the trajectory of the object 54 based on the contact time and the contact position estimated as described above.

[0206] According to the processing as described above, for example, in a table tennis game, it is possible to reproduce the interaction between the operating body 52 and the object 54 via the tool 56 in the virtual space 70.

[0207] Further, when reproducing the interaction between the operating body 52 and the object 54 via the tool 56 in the virtual space 70, it is also possible to perform motion analysis based on the detected or estimated information and present the motion analysis result.

[0208] FIG. 15 is a diagram showing an example of presenting a motion analysis result according to the present embodiment. FIG. 15 shows an example of presenting a motion analysis result related to a tennis swing.

[0209] The information processing apparatus 30 can reproduce the operations of the operating body 52, the object 54, and the tool 56 in the real space 50 as the operations of the avatar 72, the virtual object 74, and the virtual tool 76 in the virtual space 70 by the processing described with reference to FIG. 14.

[0210] Further, the information processing apparatus 30 may perform various motion analyses during the swing based on the contact time, the motion data of the operating body 52, etc., and present the analysis result 79 to the user.

[0211] The analysis result 79 may include, for example, the angle of the hand, the angle of the elbow, the swing speed, etc., as shown in FIG. 15.

[0212] As described above, according to the information processing method according to the present embodiment, it is also possible to perform motion analysis based on the detected or estimated information and present the motion analysis result.

[0213] Note that the information processing method according to the present embodiment is applicable to, for example, motion analysis such as at what timing the racket and the ball are in contact.

[0214] Next, a case where the information processing method according to the present embodiment is applied to an interaction in which the acting body 52 and the object 54 are in a one-to-N relationship will be described.

[0215] FIG. 16 is a diagram for explaining a case where the information processing method according to the present embodiment is applied to the reproduction of juggling.

[0216] FIG. 16 illustrates an acting body 52 that performs juggling using a plurality of objects 54a to 54b.

[0217] A first sensor group 10N (not shown) is attached to the acting body 52, and motion data related to the acting body 52 is acquired.

[0218] In addition, a second sensor 20 (not shown) is attached to each of the plurality of objects 54, and motion data related to each of the plurality of objects 54 is acquired.

[0219] The information processing apparatus 30 according to the present embodiment estimates the contact time and contact position between each of the plurality of objects 54 and the acting body 52 based on the acquired motion data.

[0220] In addition, the information processing apparatus 30 according to the present embodiment estimates the trajectory of each of the plurality of objects 54 based on the estimated contact time and contact position.

[0221] According to the above-described processing, it becomes possible to reproduce, in the virtual space 70, an interaction between a plurality of objects 54 and an acting body 52, such as in a volleyball game.

[0222] Next, with reference to FIG. 17, description will be given of drawing of an interaction that is difficult in reality according to the present embodiment.

[0223] In the above, an application example of reproducing lifting in the real space 50 in the virtual space 70 has been described. However, in reality, there also exists an acting body 52 that cannot perform lifting.

[0224] For the acting body 52 that cannot perform lifting, it can be said that lifting is an interaction that is difficult in reality.

[0225] According to the information processing method according to the present embodiment, it becomes possible to pseudo-reproduce such an interaction that is difficult in reality in the virtual space 70.

[0226] First, in the real space 50, the acting body 52 that cannot perform lifting performs an action imitating lifting without the object 54.

[0227] The first sensor group 10N is attached to the acting body 52, and motion data related to the acting body 52 is acquired.

[0228] In addition to the above-described motion data, the contact time and contact position specified by the user are input to the drawing unit 325.

[0229] The above contact time is the time when the user wants the avatar 72 and the virtual object 74 to come into contact, and the above contact position may be the position where the user wants the avatar 72 and the virtual object 74 to come into contact.

[0230] The drawing unit 325 performs drawing of the avatar 72 and the virtual object 74 in the virtual space 70 based on the input motion data, contact time, contact position, and the like.

[0231] According to the above-described processing, the operating entity 52 can pseudo-reproduce in the virtual space 70 a lifting that is actually difficult for the operating entity 52, and it becomes possible to provide a new user experience to the operating entity 52.

[0232] <2. Hardware Configuration Example> Next, a hardware configuration example of the information processing apparatus 30 according to an embodiment of the present disclosure will be described. FIG. 18 is a block diagram showing a hardware configuration example of the information processing apparatus 90 according to an embodiment of the present disclosure. The information processing apparatus 90 may be an apparatus having a hardware configuration equivalent to that of the information processing apparatus 30.

[0233] As shown in FIG. 18, the information processing apparatus 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Further, the information processing apparatus 90 may further include components other than the components shown here.

[0234] (Processor 871) The processor 871 functions as, for example, an arithmetic processing device or a control device, and controls the overall operation or a part of the operation of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable storage medium 901.

[0235] (ROM 872, RAM 873) The ROM 872 is a means for storing programs read by the processor 871, data used for calculations, and the like. In the RAM 873, for example, programs read by the processor 871 and various parameters that change as appropriate when executing the programs are stored temporarily or permanently.

[0236] (Host bus 874, bridge 875, external bus 876, interface 877) The processor 871, ROM 872, and RAM 873 are interconnected via a host bus 874 capable of high-speed data transmission, for example. On the other hand, the host bus 874 is connected to an external bus 876 with a relatively low data transmission speed via a bridge 875, for example. Further, the external bus 876 is connected to various components via an interface 877.

[0237] (Input device 878) For the input device 878, for example, a mouse, keyboard, touch panel, button, switch, lever, etc. are used. Further, as the input device 878, a remote controller (hereinafter, remote control) capable of transmitting a control signal using infrared rays or other radio waves may be used. Also, the input device 878 includes a voice input device such as a microphone.

[0238] (Output device 879) The output device 879 is a device capable of notifying the user visually or auditorily of the acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. Also, the output device 879 according to the present disclosure includes various vibration devices capable of outputting a tactile stimulus.

[0239] (Storage 880) The storage 880 is a device for storing various data. As the storage 880, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device is used.

[0240] (Drive 881) The drive 881 is a device that reads information recorded on a removable storage medium 901 such as, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable storage medium 901.

[0241] (Removable storage medium 901) The removable storage medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor memory media, etc. Of course, the removable storage medium 901 may be, for example, an IC card equipped with a contactless IC chip, or an electronic device, etc.

[0242] (Connection port 882) The connection port 882 is a port for connecting an external connection device 902 such as, for example, a USB (Universal Serial Bus) port, an IEEE1394 port, an SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0243] (External connection device 902) The external connection device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder, etc.

[0244] (Communication device 883) The communication device 883 is a communication device for connecting to a network, and is, for example, a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications, etc.

[0245] <3. Summary> As described above, the information processing apparatus 30 according to an embodiment of the present disclosure includes a time synchronization unit 322 that performs time synchronization on the motion data of the object 54 in the real space 50 and the motion data of the acting body 52 in the real space 50, a contact time detection unit 323 that detects the contact time between the object 54 and the acting body 52 based on the synchronized motion data of the object 54 and the motion data of the acting body 52, and a contact position estimation unit 324 that estimates the contact position between the object 54 and the acting body 52 based on the synchronized motion data of the object 54 and the motion data of the acting body 52, and the contact time.

[0246] According to the above configuration, it is possible to realize the reproduction of the interaction between the acting body and the object with a simpler configuration.

[0247] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and these are also naturally understood to belong to the technical scope of the present disclosure.

[0248] Also, each step related to the processing described in the present disclosure does not necessarily need to be processed in time series in the order described in the flowchart or sequence diagram. For example, each step related to the processing of each device may be processed in an order different from the described order or may be processed in parallel.

[0249] In addition, a series of processes performed by each apparatus described in the present disclosure may be realized by a program stored in a non-transitory computer readable storage medium. Each program is, for example, read into a RAM when executed by a computer and executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above program may be distributed via a network, for example, without using a storage medium.

[0250] Also, the effects described in this specification are illustrative or exemplary only and not limiting. That is, the technology according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description of this specification, in addition to or instead of the above effects.

[0251] Note that the following configurations also belong to the technical scope of the present disclosure. (1) A time synchronization unit that performs time synchronization on motion data of an object in real space and motion data of an acting body in the real space, A contact time detection unit that detects a contact time between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, A contact position estimation unit that estimates a contact position between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and the contact time, An information processing apparatus comprising: (2) A drawing unit that draws an interaction between an avatar corresponding to the acting body in a virtual space and a virtual object corresponding to the object in the virtual space based on the contact time and the contact position, The information processing apparatus according to (1), further comprising: (3) ​​The contact time detection unit detects, as the contact time, the time when an impact exceeding a threshold value occurs on the object based on the motion data of the object. The information processing apparatus according to any one of (1) or (2) above. (4) The contact position estimation unit estimates the contact position based on the skeleton information of the acting body included in the motion data of the acting body. The information processing apparatus according to any one of (1) to (3) above. (5) The skeleton information includes the coordinates of a plurality of joints of the acting body. The information processing apparatus according to (4) above. (6) The contact position estimation unit estimates the contact position based on the positional relationship of the plurality of joints at the contact time. The information processing apparatus according to (5) above. (7) The contact position estimation unit estimates the contact position using a rule-based method. The information processing apparatus according to (6) above. (8) The contact position estimation unit estimates the contact position using a machine learning method. The information processing apparatus according to (6) above. (9) The contact position estimation unit estimates the body part of the acting body as the contact position and outputs information regarding the estimated body part. The information processing apparatus according to any one of (4) to (8) above. (10) The drawing unit performs a physical calculation based on the contact position and the flight time estimated between the contact times, and draws the trajectory of the virtual object. The information processing apparatus according to (2) above. (11) The drawing unit draws the motion of the avatar based on the skeleton information of the acting body included in the motion data of the acting body. The information processing apparatus according to (2) above. (12) The time synchronization unit synchronizes the time when an impact exceeding a threshold value is detected on the object and the time when re - contact between the moving body and the ground is detected. The information processing apparatus according to any one of (1) to (11) above. (13) The contact position estimation unit estimates the contact position between the object and each of the plurality of moving bodies. The information processing apparatus according to any one of (1) to (12) above. (14) The contact position estimation unit estimates the contact position between each of the plurality of objects and the moving body. The information processing apparatus according to any one of (1) to (13) above. (15) The motion data of the moving body is acquired by a plurality of inertial measurement devices attached to the moving body. The information processing apparatus according to any one of (1) to (14) above. (16) The motion data of the object is acquired by an inertial measurement device attached to the object. The information processing apparatus according to any one of (1) to (15) above. (17) A processor performs time synchronization on the motion data of the object in the real space and the motion data of the moving body in the real space, detects the contact time between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body, estimates the contact position between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body, and the contact time, including an information processing method. (18) A computer A time synchronization unit that performs time synchronization for motion data of an object in real space and motion data of an acting body in the real space, A contact time detection unit that detects a contact time between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, A contact position estimation unit that estimates a contact position between the object and the acting body based on the synchronized motion data of the object and the motion data of the acting body, and the contact time, Comprising An information processing device A program for causing it to function as

Explanation of Signs

[0252] 10 First sensor 10N First sensor group 20 Second sensor 30 Information processing device 320 Processing unit 321 Calibration unit 322 Time synchronization unit 323 Contact time detection unit 324 Contact position estimation unit 325 Drawing unit 326 Contact position classifier 50 Real space 52 Acting body 54 Object 55 Tool 72 Avatar 74 Virtual object 76 Virtual tool

Claims

1. A time synchronization unit that performs time synchronization on motion data of an object in real space and motion data of an acting entity in the real space, A contact time detection unit that detects a contact time between the object and the acting entity based on the synchronized motion data of the object and the motion data of the acting entity, A contact position estimation unit that estimates a contact position between the object and the acting entity based on the synchronized motion data of the object and the motion data of the acting entity, and the contact time, Comprising, An information processing apparatus.

2. A drawing unit that draws an interaction between an avatar corresponding to the acting entity in a virtual space and a virtual object corresponding to the object in the virtual space based on the contact time and the contact position, Further comprising, The information processing apparatus according to claim 1.

3. The contact time detection unit detects, as the contact time, a time when an impact exceeding a threshold value occurs on the object based on the motion data of the object, The information processing apparatus according to claim 1.

4. The contact position estimation unit estimates the contact position based on skeleton information of the acting entity included in the motion data of the acting entity, The information processing apparatus according to claim 1.

5. The skeleton information includes coordinates of a plurality of joints of the acting entity, The information processing apparatus according to claim 4.

6. The contact position estimation unit estimates the contact position based on a positional relationship of the plurality of joints at the contact time, The information processing apparatus according to claim 5.

7. The contact position estimation unit estimates the contact position using a rule-based method, The information processing apparatus according to claim 6.

8. The contact position estimation unit estimates the contact position using a machine learning method, The information processing apparatus according to claim 6.

9. The contact position estimation unit estimates a body part of the acting entity as the contact position and outputs information regarding the estimated body part, The information processing apparatus according to claim 4.

10. The drawing unit performs physical calculations based on the contact position and a hang time estimated based on the contact time, and draws a trajectory of the virtual object, The information processing apparatus according to claim 2.

11. The drawing unit draws the motion of the avatar based on skeleton information of the acting entity included in the motion data of the acting entity, The information processing apparatus according to claim 2.

12. The time synchronization unit synchronizes the time when an impact exceeding a threshold value is detected on the object and the time when re - contact between the moving body and the ground is detected. The information processing apparatus according to claim 1.

13. The contact position estimation unit estimates the contact position between the object and each of the plurality of moving bodies. The information processing apparatus according to claim 1.

14. The contact position estimation unit estimates the contact position between each of the plurality of objects and the moving body. The information processing apparatus according to claim 1.

15. The motion data of the moving body is acquired by a plurality of inertial measurement devices attached to the moving body. The information processing apparatus according to claim 1.

16. The motion data of the object is acquired by an inertial measurement device attached to the object. The information processing apparatus according to claim 1.

17. A processor performs time synchronization related to the motion data of the object in the real space and the motion data of the moving body in the real space; detects the contact time between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body; estimates the contact position between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body, and the contact time; including an information processing method.

18. A computer is caused to function as an information processing apparatus comprising: a time synchronization unit that performs time synchronization related to the motion data of the object in the real space and the motion data of the moving body in the real space; a contact time detection unit that detects the contact time between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body; and a contact position estimation unit that estimates the contact position between the object and the moving body based on the synchronized motion data of the object and the motion data of the moving body, and the contact time. The information processing apparatus according to claim 1.

14. The contact position estimation unit estimates the contact position between each of the plurality of objects and the moving body. The information processing apparatus according to claim 1. A program for causing it to function as such.

Images