MOTION CAPTURE DEVICE, MOTION CAPTURE METHOD, AND MOTION CAPTURE PROGRAM

Abstract

The present invention enables acquisition of variations in motion of an object of motion acquisition without detecting the object from outside thereof. This motion acquisition device comprises two acceleration sensors, one angular rate sensor, an information acquisition unit, and a movement analysis unit. The two acceleration sensors and the one angular rate sensor are disposed in an object of motion acquisition that is rotated around a rotational axis. The information acquisition unit acquires acceleration information detected by the two acceleration sensors and angular rate information detected by the one angular rate sensor. The movement analysis unit estimates the distance from the object of motion acquisition to the rotational axis and the attitude angle of the object of motion acquisition on the basis of the acceleration information and the angular rate information acquired by the information acquisition unit.

Description

[Technical Field]

[0001] One aspect of the present invention relates to a motion capture device, a motion capture method, and a motion capture program. [Background technology]

[0002] At remote online events, it is more difficult to share reactions between performers and audience members, or between audience members, compared to on-site events.

[0003] Existing video streaming services allow users to share their reactions via text chat, but the problem is that performers reading text while performing and audience members typing and reading text can distract from the content itself.

[0004] To address this issue, one possible method is to share audience reactions through non-verbal physical actions. For example, the movement of penlights is one of the elements that best reflects audience movement in a dark concert venue. If the movement of such objects, such as penlights, can be captured and reproduced, it is expected that reactions can be shared between performers and audience members, and among audience members.

[0005] For example, Non-Patent Document 1 proposes a method in which spectators hold a VR (Virtual Reality) controller, which is the object of motion acquisition, and the spectators' movements are estimated based on the movements of this VR controller and reproduced in a VR space. In this method, the absolute position and attitude angle information of the VR controller are sensed by an infrared transmitter / receiver installed in the environment. Therefore, problems arise in that transmitter / receiver equipment is required, and the installation and calibration of this equipment are costly, a place to install it must be secured, and the usable range is limited to the sensing range.

[0006] Therefore, a simple implementation method for capturing spectator movements using only the sensors in the object held by the spectator, without requiring external sensors in the environment, is to install a single 6-axis sensor (acceleration + angular velocity) on the object and estimate the posture angle of the object. However, simply reproducing the estimated posture angle in VR space as is makes it impossible to capture the difference in movement between when the spectator swings the object around their wrist and when they swing it around their elbow, and therefore makes it impossible to accurately present the movement of the object.

[0007] In principle, it is possible to compensate for the above by calculating the absolute position of the moving object by integrating the acceleration acquired by the acceleration sensor in addition to the attitude angle, but noise makes it impossible to acquire the absolute position with sufficient accuracy using only the acceleration sensor. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Yamashita et al., "KSA2: A Remote Music Live System Using Virtual Reality," Information Processing Society of Japan Research Report 2018 Summary of the Invention [Problem to be solved by the invention]

[0009] This invention has been made in light of the above-mentioned circumstances, and its purpose is to provide a motion capture device, a motion capture method, and a motion capture program that are capable of capturing differences in the movement of a motion capture object without detecting the motion capture object from outside. [Means for solving the problem]

[0010] In order to solve the above problems, a motion capture device according to one aspect of the present invention includes two acceleration sensors, one angular velocity sensor, an information acquisition unit, and a motion analysis unit. The two acceleration sensors and one angular velocity sensor are disposed on a motion capture target that rotates around a rotation axis. The information acquisition unit acquires acceleration information detected by the two acceleration sensors and angular velocity information detected by the one angular velocity sensor. The motion analysis unit analyzes the acceleration information acquired by the information acquisition unit. to Based on the motion acquisition object distance to the rotation axis Estimate , Based on at least one of the acceleration information and the angular velocity information acquired by the information acquisition unit, Attitude angle of the object to be measured of Estimate. Furthermore, the motion analysis unit calculates the swing direction of the motion acquisition object in the reference coordinate system based on one piece of acceleration information, calculates the attitude angle of the motion acquisition object based on one piece of acceleration information and at least one piece of angular velocity information, and calculates the distance from the motion acquisition object to the rotation axis based on the calculated swing direction of the motion acquisition object, the acceleration information, and the distance between the two acceleration sensors. [Effects of the Invention]

[0011] According to one aspect of the present invention, the rotation axis as well as the attitude angle of the motion acquisition object can be estimated using only two acceleration sensors and one angular velocity sensor placed on the motion acquisition object, thereby providing a motion acquisition device, a motion acquisition method, and a motion acquisition program that make it possible to acquire differences in the movement of the motion acquisition object without detecting it from outside the motion acquisition object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of an outline of a distribution system to which a motion acquisition device according to a first embodiment of the present invention is applied. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of the movement acquisition device according to the first embodiment. [Figure 3] FIG. 3 is a schematic diagram showing an example of a motion acquisition object serving as an input device in FIG. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the distribution server in FIG. [Figure 5] FIG. 5 is a schematic diagram showing the movement of the penlight when the penlight is swung from an attitude angle of 0 degrees to an attitude angle of 45 degrees around a rotation axis outside the penlight. [Figure 6]FIG. 6 is a schematic diagram showing the movement of the penlight when the penlight is swung around the rotation axis within the penlight from an attitude angle of 0 degrees to an attitude angle of 45 degrees. [Figure 7] FIG. 7 is a flowchart showing a processing routine in the movement acquisition device. [Figure 8] FIG. 8 is a schematic diagram showing the relationship between the position of the rotation axis and the acceleration vector of each acceleration sensor provided in the motion acquisition object. [Figure 9] FIG. 9 is a diagram showing each acceleration vector in the same coordinate system. [Figure 10] FIG. 10 is a diagram showing the relationship between the world coordinate system and the coordinate system of the motion acquisition object. [Figure 11] FIG. 11 is a diagram for explaining the swing direction when the coordinate system of the motion acquisition object is fixed relative to the world coordinate system. [Figure 12] FIG. 12 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is outside the moving acquisition object. [Figure 13] FIG. 13 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is within the moving acquisition object. [Figure 14] FIG. 14 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis is outside the moving acquisition object. [Figure 15] FIG. 15 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis is within the moving acquisition object. [Figure 16] FIG. 16 is a diagram for explaining a method of reflecting the position of the rotation axis in the image display of the motion acquisition object when the rotation axis is outside the motion acquisition object. [Figure 17] FIG. 17 is a diagram for explaining a method of reflecting the position of the rotation axis in the image display of the motion acquisition object when the rotation axis is within the motion acquisition object. [Figure 18] FIG. 18 is a diagram for explaining a method of reflecting the posture angle in the image display of the motion acquisition object. [Figure 19] FIG. 19 is a block diagram showing an example of the configuration of a movement acquisition device according to the second embodiment of the present invention. [Figure 20] FIG. 20 is a schematic diagram showing an example of a motion acquisition object serving as an input device in FIG. [Figure 21] FIG. 21 is a flowchart showing a processing routine in the movement acquisition device according to the second embodiment. [Figure 22] FIG. 22 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is outside the moving acquisition object. [Figure 23] FIG. 23 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is within the moving acquisition object. [Figure 24] FIG. 24 is a block diagram showing an example of the configuration of a movement acquisition device according to the third embodiment of the present invention. [Figure 25] FIG. 25 is a schematic diagram showing an example of the arrangement position of the geomagnetic sensor in the motion acquisition object. [Figure 26] FIG. 26 is a block diagram showing another example of the configuration of the movement acquisition device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [First embodiment] FIG. 1 is a block diagram showing an example of an outline of a distribution system to which a motion capture device according to a first embodiment of the present invention is applied. In the distribution system, a distribution server SV distributes video of a performer PE to multiple audience members AU1, AU2, AU3, ..., AUn via a network NW such as the Internet. A filming device PC and a display device PD are installed at a live venue where the performer PE is performing. Here, the filming device PC may include multiple cameras. The audience members AU1, AU2, AU3, ..., AUn each include display devices AD1, AD2, AD3, ..., ADn and input devices AI1, AI2, AI3, ..., AIn. In the following description, the audience members AU1, AU2, AU3, ..., AUn, display devices AD1, AD2, AD3, ..., ADn, and input devices AI1, AI2, AI3, ..., AIn will be referred to as the audience members AU, display devices AD, and input devices AI, without any particular distinction.

[0014] Live video of the performers PE is captured by the camera PC and transmitted to the distribution server SV via the network NW. The distribution server SV distributes the live video captured by the camera PC to the display devices AD of each audience AU via the network NW and displays it on the display devices AD. Here, the distribution server SV may create and distribute VR video based on the live video captured by the camera PC. In this case, the display devices AD of the audience AU may be HMDs (Head Mounted Displays) worn on the heads of the audience AU.

[0015] Furthermore, the input device AI of the audience AU transmits an input signal to the distribution server SV via the network NW. The distribution server SV analyzes the movement of the input device AI based on the input signal. Then, the distribution server SV transmits an image that reproduces the movement of the input device AI based on the analyzed movement to the display device PD of the performer PE. For example, the display device PD may be multiple large displays surrounding the performer PE, or may be AR (Augmented Reality) glasses. Furthermore, for audience AUs watching VR video, the distribution server SV can include images of the input devices AI of other audience AUs in the VR video.

[0016] 2 is a block diagram showing an example of the configuration of a motion capture device 1 according to the first embodiment. As shown in FIG. 2, the motion capture device 1 includes an input device AI for each audience member AU, a distribution server SV, and a display device PD for a performer PE and / or a display device AD ​​for each audience member AU. Here, the input device AI is a motion capture target and includes two acceleration sensors 2A and 2B. The distribution server SV also includes an information acquisition unit 3, a motion analysis unit 4, and a video display unit 5.

[0017] FIG. 3 is a schematic diagram illustrating an example of a motion acquisition target as an input device AI. As shown in FIG. 3, in this embodiment, the input device AI is provided in the form of a penlight 6 held by a member of the audience AU. Two acceleration sensors 2A and 2B are spaced apart from each other on the elongated, cylindrical rigid body that constitutes the penlight 6. While the acceleration sensors 2A and 2B may be attached to the surface of the penlight 6, considering that the penlight 6 will be swung by the member of the audience AU, i.e., rotated around a certain rotation axis AX, it is preferable that the acceleration sensors 2A and 2B be housed inside the penlight 6. The spacing between the acceleration sensors is the radial direction of the rotation, i.e., the longitudinal direction of the cylindrical penlight 6. Furthermore, since the greater the distance, the better the accuracy of motion analysis, so it is desirable to make the spacing as wide as possible. In this embodiment, the two acceleration sensors 2A and 2B are located at both ends of the longitudinal axis of the cylindrical penlight 6. The two acceleration sensors 2A and 2B are arranged on the penlight 6 so that their three detection axes (x-axis, y-axis, z-axis) are aligned and the z-axis direction is the longitudinal direction of the cylindrical penlight 6.

[0018] The information acquisition unit 3 has a function of acquiring acceleration information detected by the two acceleration sensors 2A and 2B of each penlight 6 via the network NW.

[0019] The motion analysis unit 4 has a function of estimating whether the rotation axis AX is between the two acceleration sensors 2A and 2B, i.e., whether the rotation axis AX is inside or outside the penlight 6, the distance from the acceleration sensor to the rotation axis AX, and the attitude angle of each penlight 6, based on the acceleration information acquired by the information acquisition unit 3. Note that the acceleration sensor used to estimate the distance to the rotation axis AX may be any one of the two acceleration sensors 2A and 2B of each penlight 6. As indicated by the dashed-dotted arrow and the dashed-two-dotted arrow in FIG. 3 , the direction of the acceleration vector from the two acceleration sensors 2A and 2B differs depending on whether the rotation axis AX is inside or outside the penlight 6. That is, when the rotation axis AX is inside the penlight 6, the two acceleration sensors 2A and 2B are accelerated in opposite directions, as indicated by the dashed-dotted arrow. Conversely, when the rotation axis AX is outside the penlight 6, the two acceleration sensors 2A and 2B are accelerated in the same direction, as indicated by the dashed-dotted arrow. Therefore, the motion analysis unit 4 can estimate the position of the rotation axis AX based on the acceleration information. Details of the method for estimating the rotation axis AX and the attitude angle in the motion analysis unit 4 will be described later.

[0020] The video display unit 5 has a function to generate a video for displaying the image of each penlight 6 based on the distance to the rotation axis AX analyzed by the motion analysis unit 4, the attitude angle of each penlight 6, and whether the rotation axis AX is inside or outside the penlight 6. Furthermore, the video display unit 5 has a function to transmit the generated video via the network NW to the display device PD of the performer PE and / or the display device AD ​​of each audience member AU, and display it thereon.

[0021] FIG. 4 is a block diagram showing an example of the configuration of the distribution server SV. As shown in the figure, the distribution server SV is, for example, a PC (Personal Computer) or the like, and has a processor 11A such as a CPU (Central Processing Unit). The processor 11A may be multi-core / multi-threaded and can execute multiple processes in parallel. The motion capture device is configured by connecting a program memory 11B, a data memory 12, and a communication interface 13 to the processor 11A via a bus 14.

[0022] The program memory 11B is a storage medium that combines a nonvolatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), with a nonvolatile memory such as a read-only memory (ROM). The program memory 11B stores programs necessary for the processor 11A to execute various processes. The programs include an operating system (OS) and a motion capture program according to the first embodiment. When the processor 11A executes this motion capture program, the information acquisition unit 3, the motion analysis unit 4, and the video display unit 5 can be realized as software-based processing function units. These processing function units may also be realized in various other forms, including integrated circuits such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0023] The data memory 12 is a storage device that uses a combination of a nonvolatile memory, such as an HDD or SSD, which can be written to and read from at any time, and a volatile memory, such as a RAM (Random Access Memory), as a storage medium. The data memory 12 is used to store data acquired and created in the course of various processes. The storage area of ​​the data memory 12 includes, for example, a setting information storage unit 121, a received information storage unit 122, a rotation axis information storage unit 123, an attitude angle information storage unit 124, an image storage unit 125, and a temporary storage unit 126.

[0024] The setting information storage unit 121 is a storage area for storing setting information previously acquired by the processor 11 A. The setting information includes, for example, the virtual position of each audience member AU in the live venue where the performer PE is performing, i.e., the positional relationship between the performer PE and the audience member AU, the relationship between the coordinate system of the screen of the display device AD ​​and the coordinate system of the penlight 6 in each audience member AU, the distance between the two acceleration sensors 2A and 2B in each input device AI, etc.

[0025] The received information storage unit 122 is a storage area for storing the acquired acceleration information when the processor 11A functions as the information acquisition unit 3 and acquires acceleration information from the acceleration sensors 2A, 2B arranged in the penlight 6 of each spectator AU.

[0026] The rotation axis information storage unit 123 is a memory area for storing the analysis results when the processor 11A functions as the motion analysis unit 4 and analyzes information about the rotation axis AX for each spectator AU, i.e., the distance to the rotation axis AX and whether the rotation axis AX is inside or outside the penlight 6.

[0027] The attitude angle information storage unit 124 is a storage area for storing the analysis results when the processor 11A functions as the motion analysis unit 4 and analyzes the attitude angle of the penlight 6 for each audience member AU.

[0028] The video storage unit 125 is a storage area for storing the generated video when the processor 11A functions as the video display unit 5 and generates a video for displaying the image of the penlight 6 of each audience member AU.

[0029] The temporary storage unit 126 is a storage area for temporarily storing various data such as intermediate data generated while the processor 11A functions as the information acquisition unit 3, the motion analysis unit 4, and the video display unit 5 and performs various processing.

[0030] As described above, each processing function unit of the motion capture device 1 can be realized by the processor 11A, which is a computer, and a motion capture program pre-stored in the program memory 11B. However, this motion capture program can also be recorded on a non-transitory computer-readable medium or provided to the motion capture device 1 via the network NW. The motion capture program provided in this manner can be stored in the program memory 11B. Alternatively, the provided motion capture program can be stored in the data memory 12, which is a storage, and executed by the processor 11A as needed, allowing the processor 11A to function as each processing function unit.

[0031] The communication interface 13 is a wired or wireless communication unit for connecting to the network NW.

[0032] Although not specifically shown, the distribution server SV may be provided with an input / output interface that interfaces with an input device and an output device. The input device may include, for example, a keyboard or pointing device that allows the administrator of the distribution server SV to input instructions to the processor 11A. The input device may also include a reader for reading data to be stored in the data memory 12 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium. The output device may also include a display for displaying output data from the processor 11A to be presented to the user, a printer for printing the data, or the like.

[0033] Next, the processing operations of the movement capture device 1 configured as above will be described. FIG. 5 is a schematic diagram showing the movement of the penlight 6 when it is swung around the outer rotation axis AX of the penlight 6 from an attitude angle of 0 degrees to an attitude angle of 45 degrees. FIG. 6 is a schematic diagram showing the movement of the penlight 6 when it is swung around the inner rotation axis AX of the penlight 6 from an attitude angle of 0 degrees to an attitude angle of 45 degrees. FIG. 5 shows the case where the penlight 6 is swung around, for example, the elbow as the rotation axis AX, and FIG. 6 shows the case where the penlight 6 is swung around, for example, the wrist as the rotation axis AX. As shown by the dashed arrows in FIGS. 5 and 6, even if the attitude angle is the same, the size of the trajectory of the movement of the penlight 6 differs depending on the position of the rotation axis AX. Therefore, the image of the penlight that the image display unit 5 displays on the display device PD and / or AD needs to reproduce this difference in trajectory and provide the performer PE and / or the audience AU with an image that gives a different visual impression.

[0034] FIG. 7 is a flowchart showing a processing routine in the motion capture device 1 according to the first embodiment. The processor 11A of the motion capture device 1 can perform the processing shown in this flowchart by executing a motion capture program stored in advance in, for example, the program memory 11B. The processor 11A executes the motion capture program in response to receiving a broadcast viewing start instruction from the audience AU via the network NW through the communication interface 13. Note that the processing routine shown in this flowchart shows processing corresponding to one input device AI, and the processor 11A can perform similar processing in parallel for each of multiple input devices AI.

[0035] The processor 11A operates as the information acquisition unit 3 to acquire acceleration information (step S11). That is, the processor 11A receives the acceleration information transmitted from the two acceleration sensors 2A and 2B arranged in the penlight 6, which is the input device AI, via the network NW, through the communication interface 13, and stores the information in the received information storage unit 122 of the data memory 12.

[0036] The processor 11A determines whether or not the audience AU is waving the penlight 6 based on the acceleration information stored in the received information storage unit 122 (step S12). For example, the processor 11A can make this determination by determining whether or not the sum of the squares of the acceleration in the x and y directions exceeds a threshold. If it is determined that the audience AU is not waving the penlight 6, the processor 11A proceeds to the processing of step S11.

[0037] If it is determined that the audience member AU is waving the penlight 6, the processor 11A determines whether the rotation axis AX is inside or outside the penlight 6 (step S13). For example, the processor 11A can make this determination based on the angle formed by the acceleration vector. This determination method will be described in detail later. The processor 11A stores the determination result in the rotation axis information storage unit 123 of the data memory 12.

[0038] The processor 11A calculates the rotation plane, which is the swing direction of the penlight 6, using the setting information stored in the setting information storage unit 121 and the acceleration information stored in the received information storage unit 122 (step S14). The details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0039] The processor 11A calculates the distance from the acceleration sensor 2A or 2B to the rotation axis AX using the setting information stored in the setting information storage unit 121, the acceleration information stored in the received information storage unit 122, and the determination result of whether the rotation axis AX is inside or outside the penlight 6, stored in the rotation axis information storage unit 123 (step S15). The method of calculating this distance differs depending on whether the rotation axis AX is inside or outside the penlight 6. The details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0040] The processor 11A calculates the attitude angle α of the penlight 6 using the setting information stored in the setting information storage unit 121 and the distance from the acceleration sensor 2A or 2B to the rotation axis AX stored in the rotation axis information storage unit 123 (step S16). The calculation method will be described in detail later. The processor 11A stores the calculation result in the attitude angle information storage unit 124 of the data memory 12.

[0041] The processor 11A displays the image of the penlight 6 on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU (step S17). That is, the processor 11A generates an image for displaying the penlight 6 based on the information about the rotation axis AX stored in the rotation axis information storage unit 123 and the attitude angle α stored in the attitude angle information storage unit 124, and stores the image in the image storage unit 125. At this time, the processor 11A generates an image that reflects not only the penlight 6, the object whose movement is to be acquired by the processing shown in this flowchart, but also the movements of the penlights 6 of the other audience members AU. The processor 11A then transmits the image stored in the image storage unit 125 to the display device PD and / or AD via the communication interface 13 and the network NW, thereby displaying the image thereon.

[0042] The processor 11A determines whether or not to end the processing (step S18). The processor 11A can make this determination based on whether or not it has received a distribution viewing end instruction from the audience AU via the network NW through the communication interface 13. If it has determined not to end the processing, the processor 11A proceeds to the processing of step S11 above. On the other hand, if it has determined to end the processing, the processor 11A ends the processing routine shown in this flowchart.

[0043] The details of the processing at each step will be explained below. <Jump axis in / out determination> In step S13, the processor 11A determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B, and in this embodiment, whether the rotation axis AX is inside or outside the penlight 6. Fig. 8 is a schematic diagram showing the relationship between the position of the rotation axis AX and the acceleration vectors of the acceleration sensors 2A and 2B provided in the penlight 6, which is the object whose motion is to be acquired. Fig. 9 is a diagram showing each acceleration vector in the same coordinate system.

[0044] When a member of the audience AU swings the penlight 6 around, for example, their wrist, the rotation axis AX is located inside the penlight 6. In this case, the acceleration vector a detected by the acceleration sensor 2A is A and the acceleration vector a detected by the acceleration sensor 2B. B In addition, if the audience member AU swings the penlight 6 around the elbow, the rotation axis AX is on the outside of the penlight 6. In this case, the acceleration vector a detected by the acceleration sensor 2A is A and the acceleration vector a detected by the acceleration sensor 2B. B is in the same direction.

[0045] Acceleration vector a A and the acceleration vector is a B The angle θ is as follows:

[0046]

number

[0047] The processor 11A determines whether the rotation axis AX is inside or outside the penlight 6 based on the value of θ.

[0048]

number

[0049] In this case, the processor 11A determines that the rotation axis AX is outside the penlight 6,

[0050]

number

[0051] In this case, the processor 11A determines that the rotation axis AX is inside the penlight 6.

[0052] <Pivot plane calculation> In step S14, the processor 11A calculates the rotation plane, which is the swing direction of the penlight 6 projected onto the XY plane in the world coordinate system (XYZ). There are two calculation methods for this calculation.

[0053] (Pattern 1) 10 is a diagram showing the relationship between the world coordinate system (XYZ) and the coordinate system (xyz) of the penlight 6, which is the object for acquiring movement. Before starting the process shown in FIG. 7, the processor 11A predefines the conversion between the screen coordinate system, which is the world coordinate system (XYZ), and the penlight coordinate system by having the audience AU wave the penlight 6 vertically or horizontally. For example, by having the audience AU wave the penlight 6 horizontally, i.e., in the x-axis direction, the processor 11A calculates the angle T between the x-axis of the world coordinate system and the penlight 6 at that time, and stores it in the setting information storage unit 121 as one of the setting values.

[0054] In step S14, processor 11A calculates the swing direction in the penlight coordinate system, i.e., the angle S with the x-axis, based on the x and y components of the acceleration vector. Processor 11A then converts the swing direction in the penlight coordinate system (xyz) into a swing direction β in the screen coordinate system (XYZ). Specifically, the swing direction β is calculated using β=ST.

[0055] (Pattern 2) FIG. 11 is a diagram for explaining the swing direction when the coordinate system (xyz) of the penlight 6, which is the object whose movement is to be acquired, is fixed relative to the world coordinate system (XYZ). Consider a situation in which the front of the penlight 6 is fixed relative to the screen of the display device AD, and the vertical and horizontal swing planes are fixed on the yz and xz planes. Roll RO, which is rotation around the x-axis of the penlight coordinate system, is a vertical swing, and pitch PI, which is rotation around the y-axis, is a horizontal swing. Here, rotation around the z-axis, which is twisting of the penlight 6, is not taken into account.

[0056] Processor 11A compares the x-direction acceleration with the y-direction acceleration, and if the x-axis acceleration is small, it calculates that the penlight 6 is being swung vertically, i.e., the y-axis direction is the direction of the swing. If the y-axis acceleration is small, processor 11A calculates that the penlight 6 is being swung horizontally, i.e., the x-axis direction is the direction of the swing.

[0057] <Pivot axis distance calculation> In step S15, the processor 11A calculates the distance from the acceleration sensor 2A or 2B to the rotation axis AX. The calculation method differs depending on whether the rotation axis AX is inside or outside the penlight 6.

[0058] (When the rotation axis AX is outside the penlight 6) FIG. 12 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is outside the penlight 6, which is the object whose motion is to be acquired. P A [t]: Position of acceleration sensor 2A at time t P B [t]: Position of acceleration sensor 2B at time t r: Length of penlight 6 (known) r X : Length to the rotation axis AX (variable to be found) That is, here, the processor 11A calculates the length r from the acceleration sensor 2B of the two acceleration sensors 2A and 2B to the rotation axis AX. XThe length r of the penlight 6 is stored in the setting information storage unit 121 as one of the setting values.

[0059] where: D A : The distance traveled by the acceleration sensor 2A after Δt [sec] D B : The distance traveled by the acceleration sensor 2B after Δt [sec] Then, the triangle <AX·P A [t]·P A [t+1]> and triangle <AX·P B [t]·P B Since [t+1]> is a similarity relation, D A :D B =(r+r X ):r X D B (r+r X )=D A r X D B r+D B r X =D A r X (D A -D B )r X =D B r ∴r X =D B r / (D A -D B ) Therefore, the distance D A ,D B Once this is found, the length r X is obtained.

[0060] where: gamma A [t]: Linear acceleration of acceleration sensor 2A observed at time t gamma B [t]: Linear acceleration of acceleration sensor 2B observed at time t V A : The velocity of the acceleration sensor 2A at time t VB : The velocity of the acceleration sensor 2B at time t The linear acceleration is the acceleration excluding the gravitational acceleration, and can be calculated by applying a high-pass filter to acceleration data obtained by an acceleration sensor, for example.

[0061] These gamma A ,γ B ,V A ,V B Using this, the distance D A ,D B teeth, D A =V A Δt+(γ A [t]Δt 2 ) / 2 D B =V B Δt+(γ B [t]Δt 2 ) / 2 This becomes:

[0062] speed V A ,V B It is difficult to accurately determine the velocity V from the acceleration sensors 2A and 2B. However, when the timing of the turning of the penlight 6 is taken into consideration, the velocity V A ,V B Both can be considered to be 0.

[0063] Here, the turning time t0 is defined as the acceleration γ A [t0-1] and γ A [t0], γ B [t0-1] and γ B It is defined as the time when the sign of [t0] is reversed. Also, when starting to move from a stationary state, γ A [t0-1]=γ B [t0-1]=0: Penlight 6 is stationary, γ A [t0]γ B It can also be calculated when [t0] is not 0.

[0064] At time t0, V A =0, V B =0, so D A=(γ A [t0]Δt 2 ) / 2 D B =(γ B [t0]Δt 2 ) / 2 The processor 11A then r X =D B r / (D A -D B ) The length r from the acceleration sensor 2B to the rotation axis AX is X can be calculated.

[0065] (When the rotation axis AX is inside the penlight 6) FIG. 13 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is inside the penlight 6, which is the object whose motion is to be acquired. P A [t]: Position of acceleration sensor 2A at time t P B [t]: Position of acceleration sensor 2B at time t r: Length of penlight 6 (known) r X : Length to the rotation axis AX (variable to be found) Here, the processor 11A also calculates the length r from the acceleration sensor 2B of the two acceleration sensors 2A and 2B to the rotation axis AX. X This will be explained as a case where

[0066] where: D A : The distance traveled by the acceleration sensor 2A after Δt [sec] D B : The distance traveled by the acceleration sensor 2B after Δt [sec] Then, the triangle <AX·P A [t]·P A [t+1]> and triangle <AX·P B [t]·P B Since [t+1]> is a similarity relation, D A :D B =(rr X):r X D B (rr X )=D A r X D B rD B r X =D A r X (D A +D B )r X =D B r ∴r X =D B r / (D A +D B ) Therefore, the distance D A ,D B Once this is found, the length r X is obtained.

[0067] where: gamma A [t]: Linear acceleration of acceleration sensor 2A observed at time t gamma B [t]: Linear acceleration of acceleration sensor 2B observed at time t V A : The velocity of the acceleration sensor 2A at time t V B : The velocity of the acceleration sensor 2B at time t Then, the distance D A ,D B teeth, D A =V A Δt+(γ A [t]Δt 2 ) / 2 D B =V B Δt+(γ B [t]Δt 2 ) / 2 This becomes:

[0068] At time t0, V A =0, V B If we set =0, D A =(γ A[t0]Δt 2 ) / 2 D B =(γ B [t0]Δt 2 ) / 2 The processor 11A then r X =D B r / (D A +D B ) The length r from the acceleration sensor 2B to the rotation axis AX is X can be calculated.

[0069] <Attitude angle calculation> In step S16, the processor 11A calculates the attitude angle α of the penlight 6. Fig. 14 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis AX is outside the penlight 6, which is the object whose motion is to be acquired, and Fig. 15 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis AX is inside the penlight 6.

[0070] Here, the attitude angle α is defined as the angle between the XY plane of the world coordinate system (XYZ) and the longitudinal direction of the penlight 6. In addition, the acceleration in the x-axis direction and the acceleration in the y-axis direction are compared, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is large.

[0071] The sum of the gravitational acceleration and the acceleration due to motion is detected by the acceleration sensors 2A and 2B, respectively. The acceleration in the x-axis direction acquired by acceleration sensor 2A is a Ax ,z-axis direction is a Az , The acceleration in the x-axis direction acquired by acceleration sensor 2B is a Bx ,z-axis direction is a Bz , The attitude angle of the penlight on the XZ plane (rotation around the Y axis, pitch PI) is α Let's say.

[0072] As shown in FIG. 14, when the rotation axis AX is located outside the penlight 6, the distances from the rotation axis AX to the acceleration sensors 2A and 2B are (r + r X ),r X Therefore, the processor 11A calculates the attitude angle α as follows:

[0073]

number

[0074] It is calculated as follows.

[0075] In addition, when the rotation axis AX is located inside the penlight 6 as shown in FIG. 15, the distances from the rotation axis AX to the acceleration sensors 2A and 2B are (rr X ),r X Therefore, the processor 11A calculates the attitude angle α as follows:

[0076]

number

[0077] It is calculated as follows.

[0078] <Video display> In step S17, the processor 11A displays the image of the penlight 6 on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU.

[0079] (Reflecting the rotation axis AX position) 16 is a diagram for explaining how to reflect the rotation axis position in the image display of the penlight 6 when the rotation axis AX is outside the penlight 6, which is the object of movement acquisition, and FIG. 17 is a diagram for explaining how to reflect the rotation axis position in the image display of the penlight 6 when the rotation axis AX is inside the penlight 6. In FIG. 16 and FIG. 17, the position AX of the rotation axis AX, which is not actually displayed in the image display, is D A penlight image 6 is an image of the penlight 6 drawn with respect to D This shows:

[0080] The processor 11A determines the position AX of the rotation axis AX in the image display. D is fixed, and the distance r to the rotation axis AX calculated in the z-axis direction of the penlight coordinate system X Depending on the penlight statue 6 D As a result, the drawing position of the penlight image 6 changes depending on whether the audience member AU rotates the penlight 6 around the wrist or the elbow. D The movements of the

[0081] (Reflecting attitude angles (α, β)) FIG. 18 is a diagram for explaining a method of reflecting the attitude angle in the image display of the penlight 6, which is an object for which motion is to be acquired.

[0082] The processor 11A calculates the pitch angle (the attitude angle α calculated in step S16) and the yaw angle (the swing direction β calculated in step S14) in the world coordinate system (XYZ). D This allows the attitude angle of the penlight 6 to be reproduced.

[0083] As described above in detail, in the first embodiment of the present invention, two acceleration sensors 2A and 2B are arranged on the penlight 6, which is an object whose movement is to be acquired and which rotates around the rotation axis AX. The information acquisition unit 3 acquires acceleration information detected by these two acceleration sensors 2A and 2B. The motion analysis unit 4 estimates the distance from one of the acceleration sensors 2A or 2B to the rotation axis AX and the attitude angle of the penlight 6 based on the acceleration information acquired by the information acquisition unit 3. Therefore, according to the first embodiment, the rotation axis AX as well as the attitude angle of the penlight 6 are estimated using only two acceleration sensors 2A and 2B, so that it is possible to obtain differences in the movement of the penlight 6 without detecting them from outside the penlight 6.

[0084] The two acceleration sensors 2A and 2B are arranged on the penlight 6, spaced apart in the radial direction of the rotation. Therefore, based on the difference in the direction of the acceleration information from acceleration sensor 2A and the acceleration information from acceleration sensor 2B, it becomes possible to determine whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0085] In addition, the motion analysis unit 4 calculates the swing direction of the penlight 6 in the world coordinate system, which is a reference coordinate system, based on the acceleration information, calculates the distance from one of the acceleration sensors to the rotation axis AX based on the acceleration information and the distance between the two acceleration sensors 2A, 2B, and calculates the attitude angle of the penlight 6 based on the distance between the two acceleration sensors 2A, 2B, the calculated swing direction of the penlight 6, and the calculated distance to the rotation axis AX. Therefore, the distance to the rotation axis AX and the attitude angle of the penlight 6 can be calculated based on the acceleration information from the two acceleration sensors 2A and 2B.

[0086] In addition, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B based on the acceleration information acquired by the information acquisition unit 3, and the calculation method for calculating the distance to the rotation axis AX and the attitude angle of the penlight 6 differs depending on whether the rotation axis AX is between the two acceleration sensors 2A and 2B or not. Therefore, by using a calculation method according to the position of the rotation axis AX, the distance to the rotation axis AX and the attitude angle of the penlight 6 can be calculated with high accuracy.

[0087] Furthermore, in this first embodiment, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B based on the acceleration information acquired by the information acquisition unit 3, and the image display unit 5 displays an image of the penlight 6 on the display device PD and / or AD based on the distance to the rotation axis AX, the attitude angle of the penlight 6, and the determination result of whether the rotation axis AX is between the two acceleration sensors 2A and 2B. Therefore, it is possible to provide a video display that reproduces the movement of the penlight 6.

[0088] [Second embodiment] Next, a second embodiment of the present invention will be described. In the following description, the same parts as those in the first embodiment will be denoted by the same reference numerals as those used in the first embodiment, and the description thereof will be omitted.

[0089] 19 is a block diagram showing an example of the configuration of a movement acquisition device 1 according to a second embodiment of the present invention. In the second embodiment, in addition to the configuration of the first embodiment, the input device AI includes a gyro sensor 7 that detects angular velocity.

[0090] FIG. 20 is a schematic diagram showing an example of a motion acquisition target object serving as the input device AI. In the second embodiment, the input device AI is also provided in the form of a penlight 6 held by an audience member AU. The gyro sensor 7 is installed in the same position as one of the two acceleration sensors 2A, 2B. For example, as shown in FIG. 20, the gyro sensor 7 is installed in the same position as the acceleration sensor 2A. In this case, the gyro sensor 7 is installed so that its three axes (x-axis, y-axis, z-axis) are aligned with the three axes of the acceleration sensor 2A.

[0091] FIG. 21 is a flowchart showing a processing routine in the motion capture device 1 according to the second embodiment. The processor 11A of the motion capture device 1 can perform the processing shown in this flowchart by executing a motion capture program stored in advance in, for example, the program memory 11B. The processor 11A executes the motion capture program in response to receiving a broadcast viewing start instruction from the audience AU via the network NW through the communication interface 13. Note that the processing routine shown in this flowchart shows processing corresponding to one input device AI, and the processor 11A can perform similar processing in parallel for each of multiple input devices AI.

[0092] The processor 11A operates as the information acquisition unit 3 and acquires acceleration information and angular velocity information (step S21). That is, the processor 11A receives, via the network NW, the acceleration information from the two acceleration sensors 2A and 2B respectively arranged in the penlight 6 serving as the input device AI and the angular velocity information from the gyro sensor 7, and stores the information in the received information storage unit 122 of the data memory 12.

[0093] As in the first embodiment, the processor 11A determines from the acceleration information whether or not the audience AU is waving the penlight 6 (step S12). If it is determined that the audience AU is not waving the penlight 6, the processor 11A proceeds to the processing of step S21.

[0094] If it is determined that the audience member AU is waving the penlight 6, the processor 11A determines, as in the first embodiment, whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, in this embodiment as well, whether the rotation axis AX is inside or outside the penlight 6 (step S13).

[0095] As in the first embodiment, the processor 11A calculates the rotation plane, which is the swing direction of the penlight 6, using the setting information stored in the setting information storage unit 121 and the acceleration information stored in the received information storage unit 122 (step S14).

[0096] The processor 11A calculates the attitude angle α of the penlight 6 using the acceleration information and angular velocity information stored in the received information storage unit 122 (step S22). The calculation method will be described in detail later. The processor 11A stores the calculation result in the attitude angle information storage unit 124 of the data memory 12.

[0097] The processor 11A calculates the distance from the penlight 6 to the rotation axis AX using the setting information stored in the setting information storage unit 121, the acceleration information stored in the received information storage unit 122, and the determination result of whether the rotation axis AX is inside or outside the penlight 6, stored in the rotation axis information storage unit 123 (step S23). The method of calculating this distance differs depending on whether the rotation axis AX is inside or outside the penlight 6. Details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0098] The processor 11A displays the image of the penlight 6 on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU, as in the first embodiment (step S17). The processor 11A determines whether or not to end the process, as in the first embodiment (step S18). If the processor 11A determines not to end the process, it proceeds to the process of step S11, and if it determines to end the process, it ends the processing routine shown in this flowchart.

[0099] The processing in steps S22 and S23 will be described in detail below. <Attitude angle calculation> In step S22, the processor 11A calculates the attitude angle α of the penlight 6. Here, the attitude angle α is defined as the angle between the XY plane of the world coordinate system (XYZ) and the longitudinal direction of the penlight 6.

[0100] (When calculating using only acceleration information) The acceleration in the x-axis direction acquired by acceleration sensor 2A or 2B is a x ,z-axis direction is a z Then, the processor 11A calculates the pitch rotation angle p as follows:

[0101]

number

[0102] It can be calculated as follows.

[0103] The calculated pitch rotation angle p corresponds to the attitude angle α on the XZ plane. This method of calculating the attitude angle α based on acceleration information can calculate the attitude angle α with higher accuracy when the object whose motion is to be acquired moves at a low frequency compared to when the object moves at a high frequency.

[0104] (When using only angular velocity information) The angular velocity information obtained by the gyro sensor 7 is (ω x ,ω y ,ω z ), the rotation angles (δφ, δθ, δψ) around the x, y, and z axes at a certain short time δt are (δφ,δθ,δψ)=(ω x δt,ω y δt,ω z δt) This can be expressed as:

[0105] The processor 11A calculates the roll, pitch, and yaw rotation angles as (δ r ,δ p ,δ y ) as

[0106]

number

[0107] Therefore, the pitch rotation angle δ p can be calculated, where C θ = cosθ, S θ =sinθ.

[0108] Calculated pitch rotation angle δ p corresponds to the attitude angle α in the XZ plane. This method of calculating the attitude angle α based on angular velocity information can calculate the attitude angle α with higher accuracy when the object to be motion-obtained moves at a high frequency compared to when the object moves at a low frequency.

[0109] (When calculating using acceleration and angular velocity information) Since the accuracy of the attitude angle α is poor when only the acceleration information or only the angular velocity information is used, the accuracy can be improved by sensor fusion.

[0110] For example, a complementary filter is used that calculates a weighted sum of an angle calculated by applying a low-pass filter to acceleration information and an angle calculated by applying a high-pass filter to angular velocity information. Corrected attitude angle = k * (attitude angle calculated from angular velocity information) + (1-k) * (attitude angle calculated from acceleration information) The corrected attitude angle α is calculated by the above.

[0111] In this way, the processor 11A can accurately calculate the attitude angle α both when the object is stationary and when the object is moving by using the acceleration information and the angular velocity information.

[0112] It should be noted that this embodiment is not limited to the complementary filter, and other filters such as a Kalman filter and a gradient filter may be used.

[0113] <Pivot axis distance calculation> In step S23, the processor 11A calculates the distance from the penlight 6, which is the movement acquisition target, to the rotation axis AX in this embodiment, from the bottom end of the penlight 6 where the acceleration sensor 2B is located. In the second embodiment as well, the calculation method differs depending on whether the rotation axis AX is between the two acceleration sensors 2A and 2B or not, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0114] (When the rotation axis AX is outside the penlight 6) 22 is a diagram illustrating variables used to estimate the rotation axis when the rotation axis AX is outside the penlight 6, which is the object whose motion is being acquired. Here, the acceleration in the x-axis direction is compared with the acceleration in the y-axis direction, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is larger.

[0115] The longitudinal axis of the penlight 6 during movement is taken as the xz coordinate system, which is the instantaneous stationary coordinate system. When the coordinate system is moved by a small angle ε, the acceleration in the xz coordinate system (ε=0) and gravity converted into the xz coordinate system are added together, and the acceleration a in the x-axis and z-axis directions from the acceleration sensor 2A or 2B is obtained. x ,a z is obtained.

[0116] where:

[0117]

number

[0118] Then, if ε=0,

[0119]

number

[0120] is.

[0121] Taking into account the gravitational acceleration g, the acceleration a obtained by the acceleration sensor 2A is Ax ,a Az and acceleration a acquired by acceleration sensor 2B Bx ,a Bz Considering this,

[0122]

number

[0123] This becomes:

[0124] Therefore, the processor 11A calculates the following from the above formulas (3) and (5):

[0125]

number

[0126] The length r from the acceleration sensor 2B located at the bottom of the penlight 6 to the rotation axis AX is X can be calculated.

[0127] (When the rotation axis AX is inside the penlight 6) 23 is a diagram illustrating variables used to estimate the rotation axis when the rotation axis AX is inside the penlight 6, which is the object of motion acquisition. Here, the acceleration in the x-axis direction is compared with the acceleration in the y-axis direction, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is larger.

[0128] The longitudinal axis of the penlight 6 during movement is taken as the xz coordinate system, which is the instantaneous stationary coordinate system. When the coordinate system is moved by a small angle ε, the acceleration in the xz coordinate system (ε=0) and gravity converted into the xz coordinate system are added together, and the acceleration a in the x-axis and z-axis directions from the acceleration sensor 2A or 2B is obtained. x ,a z is obtained.

[0129] Here, from the above formulas (1) and (2), the acceleration a acquired by the acceleration sensor 2A is calculated by taking into consideration the gravitational acceleration g. Ax ,a Az and acceleration a acquired by acceleration sensor 2B Bx ,a Bz Considering this,

[0130]

number

[0131] This becomes:

[0132] Therefore, the processor 11A calculates the following from the above formulas (8) and (10):

[0133]

number

[0134] The length r from the acceleration sensor 2B located at the bottom of the penlight 6 to the rotation axis AX is X can be calculated.

[0135] As described above in detail, in the second embodiment of the present invention, two acceleration sensors 2A, 2B and one angular velocity sensor, the gyro sensor 7, are arranged on the penlight 6, which is an object whose movement is to be acquired and which rotates around the rotation axis AX. The information acquisition unit 3 acquires acceleration information detected by the two acceleration sensors 2A, 2B and angular velocity information detected by the one gyro sensor 7. The motion analysis unit 4 estimates the distance from the penlight 6 to the rotation axis AX and the attitude angle of the penlight 6 based on the acceleration information and angular velocity information acquired by the information acquisition unit 3. Therefore, according to the second embodiment, the rotation axis AX as well as the attitude angle of the penlight 6 are estimated using only two acceleration sensors 2A, 2B and one gyro sensor 7, so that it is possible to obtain differences in the movement of the penlight 6 without detecting them from outside the penlight 6.

[0136] The two acceleration sensors 2A and 2B are arranged on the penlight 6 at a distance from each other in the radial direction of the rotation, and the one gyro sensor 7 is arranged at the same position as one of the two acceleration sensors 2A and 2B. Therefore, based on the difference in the direction of the acceleration information from acceleration sensor 2A and the acceleration information from acceleration sensor 2B, it becomes possible to determine whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0137] In addition, the motion analysis unit 4 calculates the swing direction of the penlight 6 in the world coordinate system, which is a reference coordinate system, based on the acceleration information, calculates the attitude angle of the penlight 6 based on the acceleration information and angular velocity information, and calculates the distance from the penlight 6 to the rotation axis AX based on the calculated swing direction of the penlight 6, the acceleration information, and the distance between the two acceleration sensors 2A and 2B. Therefore, based on the acceleration information from the two acceleration sensors 2A and 2B and the angular velocity information from the one gyro sensor 7, the distance to the rotation axis AX and the attitude angle of the penlight 6 can be calculated.

[0138] In addition, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B based on the acceleration information acquired by the information acquisition unit 3, and the calculation method for calculating the distance to the rotation axis AX differs depending on whether the rotation axis AX is between the two acceleration sensors 2A and 2B or not. Therefore, by using a calculation method according to the position of the rotation axis AX, the distance to the rotation axis AX can be calculated with high accuracy.

[0139] Also, in this second embodiment, as in the first embodiment, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B based on the acceleration information acquired by the information acquisition unit 3, and the image display unit 5 displays an image of the penlight 6 on the display device PD and / or AD based on the distance to the rotation axis AX, the attitude angle of the penlight 6, and the determination result of whether the rotation axis AX is between the two acceleration sensors 2A and 2B. Therefore, it is possible to provide a video display that reproduces the movement of the penlight 6.

[0140] [Third embodiment] Next, a third embodiment of the present invention will be described. In the following description, the same parts as those in the first embodiment will be denoted by the same reference numerals as those used in the first embodiment, and the description thereof will be omitted.

[0141] 24 is a block diagram showing an example of the configuration of a movement acquisition device 1 according to a third embodiment of the present invention. In the third embodiment, in addition to the configuration of the first embodiment, the input device AI includes a geomagnetic sensor 8 which is a direction sensor.

[0142] 25 is a schematic diagram showing an example of a motion acquisition target object serving as an input device AI. In the third embodiment, the input device AI is also provided in the form of a penlight 6 held by an audience member AU. As shown in FIG. 25, one geomagnetic sensor 8 is installed at the end of the penlight 6, oriented so that the xy plane of the geomagnetic sensor 8 is on a plane perpendicular to the longitudinal direction of the penlight 6.

[0143] The geomagnetic sensor 8 acquires the strength of the geomagnetic field. The center of the circle of the output distribution map when the geomagnetic sensor 8 is rotated horizontally is (P x ,P y ), and the strength of the geomagnetic field acquired by the geomagnetic sensor 8 is (X, Y), the angle from magnetic north can be calculated as follows:

[0144]

number

[0145] It should be noted that the above formula can only be used when the xy plane of the geomagnetic sensor 8 is perpendicular to the vertical direction, so the processor 11A selects the measurement timing using one of the following methods. When the pitch angle (attitude angle α) is 90 degrees (after attitude angle calculation) The midpoint between the two timings for swinging the penlight 6 - The timing of maximum speed (calculated from acceleration) The processor 11A operating as the motion analysis unit 4 can determine the direction of the penlight 6 based on the strength of the geomagnetic field acquired by the geomagnetic sensor 8. By acquiring the direction in which the screen of the display device AD ​​is located in advance and storing it in the setting information storage unit 121, the processor 11A can determine the angle (angle T in FIG. 10) between the front of the screen and the front of the penlight 6. In other words, it becomes possible to define the conversion between the world coordinate system (XYZ) of the screen and the coordinate system (xyz) of the penlight 6.

[0146] This eliminates the need for an operation (calibration) to convert the screen coordinate system and the coordinate system of the penlight 6 by having the audience AU wave the penlight 6, as described in the rotation plane calculation process of the first embodiment, or an operation to fix the front direction of the penlight 6, thereby reducing the burden on the audience AU.

[0147] 26 is a block diagram showing another example of the configuration of the movement acquisition device 1 according to the third embodiment of the present invention. In this way, the geomagnetic sensor 8 is applicable not only to the first embodiment but also to the movement acquisition device 1 according to the second embodiment.

[0148] As described above in detail, in addition to the configuration of the first or second embodiment, the third embodiment of the present invention further includes a geomagnetic sensor 8, which is a direction sensor, arranged at the end of the penlight 6 in the longitudinal direction so that the xy plane of detection is in a direction perpendicular to the longitudinal direction of the penlight 6, which is the radial direction of rotation. Therefore, according to the third embodiment, the burden on the audience AU can be reduced by using the output of the geomagnetic sensor 8, which is a direction sensor.

[0149] [Other embodiments] The present invention is not limited to the above-described embodiment.

[0150] For example, the above embodiment has been described using an example of a swinging movement centered on the wrist and elbow, but it goes without saying that it is possible to detect not only swinging movements, but also movements such as raising the arm and making a circular motion above the head centered on the shoulder.

[0151] Furthermore, the movement acquisition object is not limited to the shape of the penlight 6, and may have any shape as long as it can be held by the spectator AU. Furthermore, the movement acquisition object may take a shape other than one that can be held by the spectator AU. For example, the movement acquisition object may be in a shape that is worn on the body of the spectator AU, such as on the arm. In this case, the elbow or shoulder serves as the rotation axis or axis of rotation for turning or rotating the movement acquisition object, and therefore, it can be treated in the same way as when the rotation axis AX described in the above embodiment is located outside the penlight 6.

[0152] Furthermore, the above embodiment has been described using the example of live streaming between a performer PE and an audience AU, but it goes without saying that the present invention can be applied to a variety of applications, such as a virtual kendo match in which the penlights 6 are used as bamboo swords.

[0153] Furthermore, the flow of each process described with reference to the flowchart is not limited to the procedures described, and the order of some steps may be changed, some steps may be performed simultaneously in parallel, or the processing content of some steps may be modified.

[0154] The methods described in each embodiment can be stored as a processing program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only an execution program but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0155] In short, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0156] 1...Movement acquisition device 2A, 2B...Acceleration sensor 3…Information acquisition department 4...Motion analysis section 5...Video display section 6... Penlight 6 D ...Penlight statue 7...Gyro sensor 8...Geomagnetic sensor 11A...Processor 11B...Program memory 12...Data memory 121...Setting information storage unit 122... Received information storage unit 123... Rotation axis information storage unit 124…Attitude angle information storage unit 125...Video storage unit 13...Communication interface 14...Bus AD,AD1,AD2,AD3,ADn,PD…Display device AI, AI1, AI2, AI3, AIn...input device AU,AU1,AU2,AU3,AYn...Audience AX...rotation axis AX D …Position of the rotation axis PC: Imaging device PE…Performer SV...Distribution server NW...Network

Claims

1. Two acceleration sensors and one angular velocity sensor are disposed on a motion acquisition object that is rotated around a rotation axis; an information acquisition unit that acquires acceleration information detected by the two acceleration sensors and angular velocity information detected by the one angular velocity sensor; a motion analysis unit that estimates a distance from the motion acquisition object to the rotation axis based on the acceleration information acquired by the information acquisition unit, and estimates a posture angle of the motion acquisition object based on at least one of the acceleration information and the angular velocity information acquired by the information acquisition unit; Equipped with The motion analysis unit is Calculating a swing direction of the motion acquisition object in a reference coordinate system based on one of the acceleration information; Calculating the attitude angle of the motion acquisition object based on at least one of the acceleration information and the angular velocity information; a motion capture device that calculates a distance from the motion capture object to the rotation axis based on the calculated swing direction of the motion capture object, the acceleration information, and a distance between the two acceleration sensors.

2. the two acceleration sensors are disposed on the motion acquisition object, spaced apart in a radial direction of the rotation; The motion acquisition device according to claim 1 , wherein the one angular velocity sensor is disposed at the same position as one of the two acceleration sensors.

3. The motion analysis unit is determining whether the rotation axis is between the two acceleration sensors based on the acceleration information acquired by the information acquisition unit; 3. The motion acquisition device according to claim 1, wherein a calculation method for calculating the distance to the rotation axis differs depending on whether the rotation axis is between the two acceleration sensors or not.

4. 4. The motion acquisition device according to claim 1, further comprising an orientation sensor disposed on the motion acquisition object such that an xy plane of detection is in a direction perpendicular to the radial direction of the rotation.

5. The motion analysis unit determines whether the rotation axis is between the two acceleration sensors based on the acceleration information acquired by the information acquisition unit, The motion acquisition device according to any one of claims 1 to 4, further comprising an image display unit that displays an image of the motion acquisition object based on the distance to the rotation axis, the attitude angle of the motion acquisition object, and a determination result of whether the rotation axis is between the two acceleration sensors.

6. A motion acquisition method for an apparatus for acquiring motion of a motion acquisition object rotated around a rotation axis, comprising: acquiring acceleration information detected by two acceleration sensors and angular velocity information detected by one angular velocity sensor, the two acceleration sensors being disposed on the motion acquisition object; estimating a distance from the motion acquisition object to the rotation axis based on the acquired acceleration information, and estimating a posture angle of the motion acquisition object based on at least one of the acquired acceleration information and the acquired angular velocity information; and estimating the distance and the attitude angle includes: Calculating a swing direction of the motion acquisition target object in a reference coordinate system based on one of the acceleration information; Calculating the attitude angle of the motion acquisition target object based on at least one of the acceleration information and the angular velocity information; calculating a distance from the motion acquisition object to the rotation axis based on the calculated swing direction of the motion acquisition object, the acceleration information, and a distance between the two acceleration sensors; A motion acquisition method comprising:

7. A motion capture program that causes a computer to execute processing by each unit of the motion capture device according to any one of claims 1 to 5.

Images