MOTION CAPTURE DEVICE, MOTION CAPTURE METHOD, AND MOTION CAPTURE PROGRAM

Abstract

The present invention enables acquisition of variation in motion of an motion acquisition subject without detecting the motion acquisition subject from outside thereof. This motion acquisition apparatus comprises: two acceleration sensors; an information acquisition unit; and a kinematic analysis unit. The two acceleration sensors are disposed in a motion acquisition subject which is moved rotationally about a rotary shaft. The information acquisition unit acquires acceleration information detected by the two acceleration sensors. The kinematic analysis unit estimates the distance from one of the acceleration sensors to the rotary shaft and the attitude angle of the motion acquisition subject on the basis of the acceleration information acquired by the information acquisition unit.

Description

[Technical field]

[0001] One aspect of the present invention relates to a motion capture device, a motion capture method, and a motion capture program. [Background technology]

[0002] In remote online events, it is more difficult to share reactions between performers and audience members, or between audience members themselves, compared to on-site events.

[0003] Existing video streaming services allow users to share their reactions through text chat. However, there is a problem that performers reading text during a performance and audience members typing and reading text can distract viewers from the content itself.

[0004] To address this issue, a method of sharing the audience's reactions through non-verbal physical actions is being considered. For example, the movement of a penlight is one of the elements that most closely reflects the movement of an audience in a dark concert hall. If it were possible to capture and reproduce the movement of such objects as penlights, it would be possible to share reactions between performers and audience members, and between audience members themselves.

[0005] For example, Non-Patent Document 1 proposes a method in which an audience member is made to carry a VR (Virtual Reality) controller, the object of which movement is to be acquired, and the audience's movements are estimated based on the movements of the VR controller and reproduced in a VR space. In this method, the absolute position and attitude angle information of the VR controller are sensed by an infrared transmitter / receiver installed in the environment. Therefore, problems arise in that a transmitter / receiver device is required, and the installation and calibration of the device are costly, a place for installation must be secured, and the range of use is limited to the sensing range, etc.

[0006] Therefore, as a simple implementation method to acquire the spectator's movement using only the sensors of the object to be captured that the spectator holds, without requiring external sensors in the environment, a method of estimating the attitude angle of the object to be captured by installing one 6-axis sensor (acceleration + angular velocity) on the object to be captured, is considered. However, simply reproducing the estimated attitude angle in VR space as it is cannot acquire the difference in movement between when the spectator swings the object to be captured around the wrist and when the spectator swings the object to be captured around the elbow, and it is therefore not possible to accurately present the movement of the object to be captured.

[0007] In principle, it is possible to compensate for the above by calculating the absolute position of the moving object by integrating the acceleration acquired by the acceleration sensor in addition to the attitude angle. However, the absolute position cannot be acquired with sufficient accuracy using only the acceleration sensor due to noise. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Yamashita et al., “KSA2: A Remote Music Live System Using Virtual Reality,” IPSJ Research Report 2018 Summary of the Invention [Problem to be solved by the invention]

[0009] This invention has been made in light of the above-mentioned circumstances, and its object is to provide a motion capture device, a motion capture method, and a motion capture program that are capable of capturing differences in the movement of a motion capture object without detecting the motion capture object from outside. [Means for solving the problem]

[0010] In order to solve the above problems, a motion acquisition device according to one aspect of the present invention includes two acceleration sensors, an information acquisition unit, and a motion analysis unit. The two acceleration sensors are disposed on a motion acquisition target that is rotated around a rotation axis. The information acquisition unit acquires acceleration information detected by the two acceleration sensors. The motion analysis unit estimates a distance from one of the acceleration sensors to the rotation axis and an attitude angle of the motion acquisition target based on the acceleration information acquired by the information acquisition unit. Furthermore, the motion analysis unit calculates the swing direction of the motion acquisition object in the reference coordinate system based on one of the acceleration information, calculates the distance from one of the acceleration sensors to the rotation axis based on the acceleration information and the distance between the two acceleration sensors, and calculates the attitude angle of the motion acquisition object based on the distance between the two acceleration sensors, the calculated swing direction of the motion acquisition object, and the calculated distance to the rotation axis. Effect of the Invention

[0011] According to one aspect of the present invention, a motion capture device, a motion capture method, and a motion capture program can be provided that estimate the rotation axis as well as the attitude angle of the motion capture object using only two acceleration sensors placed on the motion capture object, making it possible to capture differences in the movement of the motion capture object without detecting it from outside the motion capture object. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of an outline of a distribution system to which a motion acquisition device according to a first embodiment of the present invention is applied. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the movement acquisition device according to the first embodiment. [Diagram 3] FIG. 3 is a schematic diagram showing an example of a motion acquisition target serving as the input device in FIG. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the distribution server in FIG. [Diagram 5] FIG. 5 is a schematic diagram showing the movement of the penlight when the penlight is swung from an attitude angle of 0 degrees to an attitude angle of 45 degrees around a rotation axis outside the penlight. [Figure 6] FIG. 6 is a schematic diagram showing the movement of the penlight when the penlight is swung around a rotation axis within the penlight from an attitude angle of 0 degrees to an attitude angle of 45 degrees. [Figure 7] FIG. 7 is a flowchart showing a processing routine in the movement acquisition device. [Figure 8] FIG. 8 is a schematic diagram showing the relationship between the position of the rotation axis and the acceleration vectors of the acceleration sensors included in the motion acquisition object. [Figure 9] FIG. 9 is a diagram showing each acceleration vector in the same coordinate system. [Figure 10] FIG. 10 is a diagram showing the relationship between the world coordinate system and the coordinate system of the motion acquisition object. [Figure 11] FIG. 11 is a diagram for explaining the swing direction when the coordinate system of the motion acquisition object is fixed with respect to the world coordinate system. [Figure 12] FIG. 12 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis moves and is outside the acquisition target object. [Figure 13] FIG. 13 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is within a moving acquisition object. [Figure 14] FIG. 14 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis moves and is outside the acquisition target object. [Figure 15] FIG. 15 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis is within the moving acquisition target object. [Figure 16] FIG. 16 is a diagram for explaining a method of reflecting the position of a rotation axis in an image display of a motion acquisition target when the rotation axis is outside the motion acquisition target. [Figure 17] FIG. 17 is a diagram for explaining a method of reflecting the position of a rotation axis in an image display of a moving acquisition target when the rotation axis is within the moving acquisition target. [Figure 18] FIG. 18 is a diagram for explaining a method of reflecting a posture angle in a video display of an object whose motion is to be acquired. [Figure 19] FIG. 19 is a block diagram showing an example of the configuration of a movement acquisition device according to the second embodiment of the present invention. [Figure 20] FIG. 20 is a schematic diagram showing an example of a motion acquisition object serving as the input device in FIG. [Figure 21]FIG. 21 is a flowchart showing a processing routine in the movement acquisition device according to the second embodiment. [Figure 22] FIG. 22 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis moves and is outside the acquisition target object. [Figure 23] FIG. 23 is a diagram for explaining variables used for estimating the rotation axis when the rotation axis is within a moving acquisition object. [Figure 24] FIG. 24 is a block diagram showing an example of the configuration of a movement acquisition device according to the third embodiment of the present invention. [Diagram 25] FIG. 25 is a schematic diagram showing an example of the arrangement position of the geomagnetic sensor in a motion acquisition object. [Figure 26] FIG. 26 is a block diagram showing another example of the configuration of the movement acquisition device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [First embodiment] FIG. 1 is a block diagram showing an example of an outline of a distribution system to which a motion acquisition device according to a first embodiment of the present invention is applied. In the distribution system, a distribution server SV distributes video of a performer PE to a plurality of audience members AU1, AU2, AU3, ..., AUn via a network NW such as the Internet. A filming device PC and a display device PD are arranged in a live venue where the performer PE is performing. Here, the filming device PC may include a plurality of cameras. The audience members AU1, AU2, AU3, ..., AUn each include a display device AD1, AD2, AD3, ..., ADn, and an input device AI1, AI2, AI3, ..., AIn. In the following description, the audience members AU1, AU2, AU3, ..., AUn, the display devices AD1, AD2, AD3, ..., ADn, and the input devices AI1, AI2, AI3, ..., AIn are described as the audience members AU, the display devices AD, and the input devices AI, without any particular distinction therebetween.

[0014] Live video of the performer PE is captured by the camera device PC and transmitted to the distribution server SV via the network NW. The distribution server SV distributes the live video captured by the camera device PC to the display device AD ​​of each audience member AU via the network NW and displays it on the display device AD. Here, the distribution server SV may create and distribute VR video based on the live video captured by the camera device PC. In this case, the display device AD ​​of the audience member AU may be an HMD (Head Mounted Display) that the audience member AU wears on their head.

[0015] Furthermore, the input device AI of the audience AU transmits an input signal to the distribution server SV via the network NW. The distribution server SV analyzes the movement of the input device AI based on the input signal. The distribution server SV then transmits an image that reproduces the movement of the input device AI based on the analyzed movement to the display device PD of the performer PE. For example, the display device PD may be multiple large displays surrounding the performer PE, or may be AR (Augmented Reality) glasses. Furthermore, for an audience AU watching a VR image, the distribution server SV can include images of the input devices AI of other audience AUs in the VR image.

[0016] Fig. 2 is a block diagram showing an example of the configuration of the motion acquisition device 1 according to the first embodiment. As shown in Fig. 2, the motion acquisition device 1 includes an input device AI of each audience member AU, a distribution server SV, and a display device PD of a performer PE and / or a display device AD ​​of each audience member AU. Here, the input device AI is a motion acquisition target, and includes two acceleration sensors 2A and 2B. In addition, the distribution server SV includes an information acquisition unit 3, a motion analysis unit 4, and a video display unit 5.

[0017] FIG. 3 is a schematic diagram showing an example of a motion acquisition target object as an input device AI. As shown in FIG. 3, in this embodiment, the input device AI is provided in the form of a penlight 6 held by an audience member AU. Two acceleration sensors 2A and 2B are arranged at a distance from each other on a long and thin cylindrical rigid body constituting the penlight 6. The arrangement may be such that the acceleration sensors 2A and 2B are attached to the surface of the penlight 6, but considering that the penlight 6 is swung by the audience member AU, that is, rotated around a certain rotation axis AX, it is preferable that the acceleration sensors 2A and 2B are accommodated inside the penlight 6. The direction of the distance is the radial direction of the rotation, that is, the longitudinal direction of the cylindrical penlight 6. Furthermore, it is preferable that the distance between the acceleration sensors 2A and 2B is as wide as possible because the greater the distance, the better the accuracy of the motion analysis. In this embodiment, the two acceleration sensors 2A and 2B are arranged at both ends on the longitudinal axis of the cylindrical penlight 6. The two acceleration sensors 2A, 2B are arranged on the penlight 6 such that the three axes (x-axis, y-axis, z-axis) of detection are aligned and the z-axis direction is the longitudinal direction of the cylindrical penlight 6.

[0018] The information acquisition unit 3 has a function of acquiring acceleration information detected by the two acceleration sensors 2A, 2B of each penlight 6 via the network NW.

[0019] The motion analysis unit 4 has a function of estimating whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6, the distance from the acceleration sensor to the rotation axis AX, and the attitude angle of each penlight 6, based on the acceleration information acquired by the information acquisition unit 3. The acceleration sensor that is the subject of estimation of the distance to the rotation axis AX may be any one of the two acceleration sensors 2A and 2B of each penlight 6. As shown by the dashed arrow and the double-dashed arrow in FIG. 3, the direction of the acceleration vector from the two acceleration sensors 2A and 2B differs depending on whether the rotation axis AX is inside or outside the penlight 6. That is, when the rotation axis AX is inside the penlight 6, the two acceleration sensors 2A and 2B are accelerated in opposite directions, as shown by the dashed arrow. On the other hand, when the rotation axis AX is outside the penlight 6, the two acceleration sensors 2A and 2B are accelerated in the same direction, as shown by the dashed arrow. Therefore, the motion analysis unit 4 can estimate the position of the rotation axis AX based on the acceleration information. The method of estimating the rotation axis AX and the attitude angle in the motion analysis unit 4 will be described in detail later.

[0020] The video display unit 5 has a function of generating a video for displaying the image of each penlight 6 based on the distance to the rotation axis AX analyzed by the motion analysis unit 4, the attitude angle of each penlight 6, and whether the rotation axis AX is inside or outside the penlight 6. Furthermore, the video display unit 5 has a function of transmitting the generated video to the display device PD of the performer PE and / or the display device AD ​​of each audience member AU via the network NW to display it thereon.

[0021] FIG. 4 is a block diagram showing an example of the configuration of the distribution server SV. As shown in the figure, the distribution server SV is, for example, a PC (Personal Computer) or the like, and has a processor 11A such as a CPU (Central Processing Unit). The processor 11A may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The motion acquisition device is configured by connecting a program memory 11B, a data memory 12, and a communication interface 13 to the processor 11A via a bus 14.

[0022] The program memory 11B is a combination of a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD) that can be written and read at any time, and a non-volatile memory such as a read only memory (ROM). The program memory 11B stores programs required for the processor 11A to execute various processes. The programs include an operating system (OS) and a motion acquisition program according to the first embodiment. The processor 11A executes the motion acquisition program, thereby realizing the information acquisition unit 3, the motion analysis unit 4, and the video display unit 5 as software-based processing function units. These processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) and a field-programmable gate array (FPGA).

[0023] The data memory 12 is a storage using a combination of a non-volatile memory such as an HDD or SSD, which can be written and read at any time, and a volatile memory such as a RAM (Random Access Memory), as a storage medium. The data memory 12 is used to store data acquired and created in the course of performing various processes. The storage area of ​​the data memory 12 includes, for example, a setting information storage unit 121, a received information storage unit 122, a rotation axis information storage unit 123, an attitude angle information storage unit 124, an image storage unit 125, and a temporary storage unit 126.

[0024] The setting information storage unit 121 is a storage area for storing setting information previously acquired by the processor 11A. The setting information includes, for example, the virtual position of each audience member AU in a live performance venue where the performer PE is performing, that is, the positional relationship between the performer PE and the audience member AU, the relationship between the coordinate system of the screen of the display device AD ​​in each audience member AU and the coordinate system of the penlight 6, the distance between the two acceleration sensors 2A and 2B in each input device AI, and the like.

[0025] The received information storage unit 122 is a memory area for storing the acquired acceleration information when the processor 11A functions as the information acquisition unit 3 and acquires acceleration information from the acceleration sensors 2A, 2B arranged in the penlight 6 of each spectator AU.

[0026] The rotation axis information storage unit 123 is a memory area for storing the analysis results when the processor 11A functions as the motion analysis unit 4 and analyzes information about the rotation axis AX for each spectator AU, i.e., the distance to the rotation axis AX and whether the rotation axis AX is inside or outside the penlight 6.

[0027] The attitude angle information storage unit 124 is a storage area for storing the analysis results when the processor 11A functions as the motion analysis unit 4 and analyzes the attitude angle of the penlight 6 for each spectator AU.

[0028] The video storage unit 125 is a storage area for storing the generated video when the processor 11A functions as the video display unit 5 and generates a video for displaying the image of the penlight 6 of each audience member AU.

[0029] The temporary memory unit 126 is a memory area for temporarily storing various data such as intermediate data generated while the processor 11A functions as the information acquisition unit 3, the motion analysis unit 4, and the video display unit 5 and performs various processing.

[0030] As described above, each processing function unit of the motion capture device 1 can be realized by the processor 11A, which is a computer, and a motion capture program pre-stored in the program memory 11B. However, it is also possible to provide the motion capture program to the motion capture device 1 by recording it on a non-transitory computer-readable medium or through the network NW. The motion capture program thus provided can be stored in the program memory 11B. In addition, the provided motion capture program can be stored in the data memory 12, which is a storage, and executed by the processor 11A as necessary, so that the processor 11A can function as each processing function unit.

[0031] The communication interface 13 is a wired or wireless communication unit for connecting to the network NW.

[0032] Although not specifically shown, the distribution server SV may be provided with an input / output interface that is an interface with an input device and an output device. The input device includes, for example, a keyboard or a pointing device that allows the administrator of the distribution server SV to input instructions to the processor 11A. Furthermore, the input device may include a reader for reading data to be stored in the data memory 12 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium. Moreover, the output device includes a display for displaying output data to be presented to the user from the processor 11A, a printer for printing the output data, or the like.

[0033] Next, the processing operation of the movement acquisition device 1 configured as above will be described. FIG. 5 is a schematic diagram showing the movement of the penlight 6 when the penlight 6 is swung from an attitude angle of 0 degrees to an attitude angle of 45 degrees around the outer rotation axis AX of the penlight 6. FIG. 6 is a schematic diagram showing the movement of the penlight when the penlight 6 is swung from an attitude angle of 0 degrees to an attitude angle of 45 degrees around the inner rotation axis AX of the penlight 6. FIG. 5 shows a case where the penlight 6 is swung around, for example, the elbow as the rotation axis AX, and FIG. 6 shows a case where the penlight 6 is swung around, for example, the wrist as the rotation axis AX. As shown by the dashed arrows in FIG. 5 and FIG. 6, even if the attitude angle is the same, the size of the trajectory of the movement of the penlight 6 differs depending on the position of the rotation axis AX. Therefore, it is necessary for the image of the penlight that the image display unit 5 displays on the display device PD and / or AD to reproduce the difference in the trajectory and provide the performer PE and / or the audience AU with an image that gives a different visual impression.

[0034] FIG. 7 is a flowchart showing a processing routine in the motion acquisition device 1 according to the first embodiment. The processor 11A of the motion acquisition device 1 can perform the processing shown in this flowchart by executing a motion acquisition program stored in advance in, for example, the program memory 11B. The processor 11A executes the motion acquisition program in response to receiving a distribution viewing start instruction from the audience AU via the network NW through the communication interface 13. Note that the processing routine shown in this flowchart shows processing corresponding to one input device AI, and the processor 11A can perform similar processing in parallel for each of the multiple input devices AI.

[0035] The processor 11A operates as the information acquisition unit 3 to acquire acceleration information (step S11). That is, the processor 11A receives the acceleration information transmitted from the two acceleration sensors 2A and 2B arranged in the penlight 6, which is the input device AI, via the network NW, through the communication interface 13, and stores the information in the received information storage unit 122 of the data memory 12.

[0036] The processor 11A determines whether or not the audience AU is waving the penlight 6 from the acceleration information stored in the received information storage unit 122 (step S12). For example, the processor 11A can make this determination by determining whether or not the sum of the squares of the acceleration in the x and y directions exceeds a threshold value. If it is determined that the audience AU is not waving the penlight 6, the processor 11A proceeds to the process of step S11.

[0037] When it is determined that the audience member AU is waving the penlight 6, the processor 11A determines whether the rotation axis AX is inside or outside the penlight 6 (step S13). For example, the processor 11A can make this determination according to the angle of the acceleration vector. The details of this determination method will be described later. The processor 11A stores the determination result in the rotation axis information storage unit 123 of the data memory 12.

[0038] The processor 11A calculates the rotation plane, which is the swing direction of the penlight 6, by using the setting information stored in the setting information storage unit 121 and the acceleration information stored in the received information storage unit 122 (step S14). The details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0039] The processor 11A calculates the distance from the acceleration sensor 2A or 2B to the rotation axis AX using the setting information stored in the setting information storage unit 121, the acceleration information stored in the received information storage unit 122, and the determination result of whether the rotation axis AX is inside or outside the penlight 6 stored in the rotation axis information storage unit 123 (step S15). The method of calculating this distance differs depending on whether the rotation axis AX is inside or outside the penlight 6. The details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0040] The processor 11A calculates the attitude angle α of the penlight 6 using the setting information stored in the setting information storage unit 121 and the distance from the acceleration sensor 2A or 2B to the rotation axis AX stored in the rotation axis information storage unit 123 (step S16). The details of this calculation method will be described later. The processor 11A stores the calculation result in the attitude angle information storage unit 124 of the data memory 12.

[0041] The processor 11A displays the image of the penlight 6 on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU (step S17). That is, the processor 11A generates an image for displaying the penlight 6 based on the information about the rotation axis AX stored in the rotation axis information storage unit 123 and the attitude angle α stored in the attitude angle information storage unit 124, and stores the image in the image storage unit 125. At this time, the processor 11A generates an image that reflects the movements of the penlights 6 of the other audience members AU in addition to the penlight 6, which is the object of movement acquisition by the process shown in this flowchart. The processor 11A then transmits the image stored in the image storage unit 125 to the display device PD and / or AD via the network NW through the communication interface 13, thereby displaying the image thereon.

[0042] The processor 11A determines whether or not to end the process (step S18). The processor 11A can make this determination based on whether or not a distribution viewing end instruction is received from the audience AU via the network NW through the communication interface 13. If it is determined not to end the process, the processor 11A proceeds to the process of step S11. On the other hand, if it is determined to end the process, the processor 11A ends the processing routine shown in this flowchart.

[0043] The processing of each step will be described in detail below. <Jump axis in / out determination> In step S13, the processor 11A determines whether the rotation axis AX is between the two acceleration sensors 2A and 2B, and in this embodiment, whether the rotation axis AX is inside or outside the penlight 6. Fig. 8 is a schematic diagram showing the relationship between the position of the rotation axis AX and the acceleration vectors of the acceleration sensors 2A and 2B provided in the penlight 6, which is the object of motion acquisition. Fig. 9 is a diagram showing each acceleration vector in the same coordinate system.

[0044] When a member of the audience AU swings the penlight 6 around his or her wrist, the rotation axis AX is on the inside of the penlight 6. In this case, the acceleration vector a detected by the acceleration sensor 2A is A and the acceleration vector a detected by the acceleration sensor 2B. B In addition, when a member of the audience AU swings the penlight 6 around his / her elbow, the rotation axis AX is on the outside of the penlight 6. In this case, the acceleration vector a detected by the acceleration sensor 2A is A and the acceleration vector a detected by the acceleration sensor 2B. B The directions are the same.

[0045] Acceleration vector a A and the acceleration vector is a B The angle θ is as follows:

[0046]

number

[0047] The processor 11A determines whether the rotation axis AX is inside or outside the penlight 6 based on the value of θ.

[0048]

number

[0049] In this case, the processor 11A determines that the rotation axis AX is outside the penlight 6.

[0050]

number

[0051] In this case, the processor 11A determines that the rotation axis AX is inside the penlight 6.

[0052] <Rotation plane calculation> In step S14, the processor 11A calculates a rotation plane, which is the swing direction of the penlight 6 projected onto the XY plane in the world coordinate system (XYZ). There are two calculation methods for this calculation.

[0053] (Pattern 1) 10 is a diagram showing the relationship between the world coordinate system (XYZ) and the coordinate system (xyz) of the penlight 6, which is a motion acquisition target. Before starting the process shown in FIG. 7, the processor 11A predefines the conversion between the screen coordinate system, which is the world coordinate system (XYZ), and the penlight coordinate system by having the audience AU swing the penlight 6 vertically or horizontally. For example, by having the audience AU swing the penlight 6 horizontally, i.e., in the x-axis direction, the processor 11A calculates the angle T between the world coordinate system and the x-axis at that time and stores it in the setting information storage unit 121 as one of the setting values.

[0054] In step S14, the processor 11A calculates the swing direction in the penlight coordinate system, that is, the angle S with the x-axis, based on the x and y components of the acceleration vector. The processor 11A then converts the swing direction in the penlight coordinate system (xyz) into a swing direction β in the screen coordinate system (XYZ). Specifically, the swing direction β is calculated by β=ST.

[0055] (Pattern 2) Fig. 11 is a diagram for explaining the swing direction when the coordinate system (xyz) of the penlight 6, which is the object for acquiring movement, is fixed relative to the world coordinate system (XYZ). Consider a situation in which the front of the penlight 6 is fixed relative to the screen of the display device AD, and the vertical and horizontal swing planes are fixed on the yz and xz planes. Roll RO, which is rotation around the x-axis of the penlight coordinate system, is a vertical swing, and pitch PI, which is rotation around the y-axis, is a horizontal swing. Here, rotation around the z-axis, which is the twist of the penlight 6, is not taken into account.

[0056] Processor 11A compares the x-direction acceleration with the y-direction acceleration, and if the x-axis direction acceleration is smaller, it calculates that the penlight 6 is being swung vertically, i.e., the y-axis direction is the swing direction. If the y-axis direction acceleration is smaller, processor 11A calculates that the penlight 6 is being swung horizontally, i.e., the x-axis direction is the swing direction.

[0057] <Pivot axis distance calculation> In step S15, the processor 11A calculates the distance from the acceleration sensor 2A or 2B to the rotation axis AX. The calculation method differs depending on whether the rotation axis AX is inside or outside the penlight 6.

[0058] (When the rotation axis AX is outside the penlight 6) FIG. 12 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is outside the penlight 6, which is the moving acquisition target. P A [t]: position of acceleration sensor 2A at time t P B [t]: position of acceleration sensor 2B at time t r: Length of Penlight 6 (known) r X : Length to the rotation axis AX (variable to be found) That is, in this example, the processor 11A calculates the length r from the acceleration sensor 2B of the two acceleration sensors 2A and 2B to the rotation axis AX. XThe following description will be given assuming that the following is obtained. The length r of the penlight 6 is stored in the setting information storage unit 121 as one of the setting values.

[0059] Where: D A : The distance traveled by the acceleration sensor 2A after Δt [sec] D B : The distance traveled by the acceleration sensor 2B after Δt [sec] Then, the triangle <AX·P A [t] P A [t+1]> and triangle <AX·P B [t] P B Since [t+1]> is a similarity relation, D A :D B =(r+r X ):r X D B (r+r X )=D A r X D B r+D B r X =D A r X (D A -D B )r X =D B r ∴r X =D B r / (D A -D B ) Therefore, the distance D A ,D B Once this is found, the length r X is required.

[0060] Where: Gamma A [t]: Linear acceleration of acceleration sensor 2A observed at time t Gamma B [t]: Linear acceleration of acceleration sensor 2B observed at time t V A : The velocity of the acceleration sensor 2A at time t VB : The velocity of the acceleration sensor 2B at time t It should be noted that the linear acceleration is acceleration excluding the gravitational acceleration, and can be obtained, for example, by applying a high-pass filter to acceleration data obtained by an acceleration sensor.

[0061] These gamma A ,γ B ,V A ,V B Using this, the distance D A ,D B teeth, D A =V A Δt+(γ A [t]Δt 2 ) / 2 D B =V B Δt+(γ B [t]Δt 2 ) / 2 It becomes.

[0062] speed V A ,V B It is difficult to accurately obtain the velocity V from the acceleration sensors 2A and 2B. However, when the timing of the switching of the penlight 6 is taken into consideration, the velocity V A ,V B Both can be considered to be 0.

[0063] Here, the time t0 of the turning is defined as the acceleration γ A [t0-1] and γ A [t0], γ B [t0-1] and γ B This is defined as the time when the sign of [t0] is reversed. Also, when starting to move from a stationary state, γ A [t0-1]=γ B [t0-1]=0: Penlight 6 is stationary, γ A [t0]γ B It can also be calculated when [t0] is not 0.

[0064] At time t0, V A =0, V B = 0, so D A=(γ A [t0]Δt 2 ) / 2 D B =(γ B [t0]Δt 2 ) / 2 Then, the processor 11A performs the following: r X =D B r / (D A -D B ) The length r from the acceleration sensor 2B to the rotation axis AX is X can be calculated.

[0065] (When the rotation axis AX is inside the penlight 6) FIG. 13 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is inside the penlight 6, which is the moving acquisition target. P A [t]: position of acceleration sensor 2A at time t P B [t]: position of acceleration sensor 2B at time t r: Length of Penlight 6 (known) r X : Length to the rotation axis AX (variable to be found) Here, the processor 11A also calculates the length r from the acceleration sensor 2B of the two acceleration sensors 2A and 2B to the rotation axis AX. X This will be explained as seeking the following.

[0066] Where: D A : The distance traveled by the acceleration sensor 2A after Δt [sec] D B : The distance traveled by the acceleration sensor 2B after Δt [sec] Then, the triangle <AX·P A [t] P A [t+1]> and triangle <AX·P B [t] P B Since [t+1]> is a similarity relation, D A :D B =(rr X):r X D B (rr X )=D A r X D B rD B r X =D A r X (D A +D B )r X =D B r ∴r X =D B r / (D A +D B ) Therefore, the distance D A ,D B Once this is found, the length r X is required.

[0067] Where: Gamma A [t]: Linear acceleration of acceleration sensor 2A observed at time t Gamma B [t]: Linear acceleration of acceleration sensor 2B observed at time t V A : The velocity of the acceleration sensor 2A at time t V B : The velocity of the acceleration sensor 2B at time t Then, the distance D A ,D B teeth, D A =V A Δt+(γ A [t]Δt 2 ) / 2 D B =V B Δt+(γ B [t]Δt 2 ) / 2 It becomes.

[0068] At time t0, V A =0, V B If we set =0, D A =(γ A[t0]Δt 2 ) / 2 D B =(γ B [t0]Δt 2 ) / 2 Then, the processor 11A performs the following: r X =D B r / (D A +D B ) The length r from the acceleration sensor 2B to the rotation axis AX is X can be calculated.

[0069] <Attitude angle calculation> In step S16, the processor 11A calculates the attitude angle α of the penlight 6. Fig. 14 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis AX is outside the penlight 6, which is the moving acquisition target, and Fig. 15 is a diagram for explaining variables used in calculating the attitude angle when the rotation axis AX is inside the penlight 6.

[0070] Here, the attitude angle α is defined as the angle between the XY plane of the world coordinate system (XYZ) and the longitudinal direction of the penlight 6. In addition, the acceleration in the x-axis direction and the acceleration in the y-axis direction are compared, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is large.

[0071] The sum of the gravitational acceleration and the acceleration due to motion is detected by the acceleration sensors 2A and 2B, respectively. The acceleration in the x-axis direction obtained by the acceleration sensor 2A is a Ax ,z-axis direction is a Az , The acceleration in the x-axis direction acquired by the acceleration sensor 2B is a Bx ,z-axis direction is a Bz , The attitude angle of the penlight on the XZ plane (rotation around the Y axis, pitch PI) is α Let us assume that.

[0072] As shown in FIG. 14, when the rotation axis AX is located outside the penlight 6, the distances from the rotation axis AX to the acceleration sensors 2A and 2B are (r+r X ),r X Therefore, the processor 11A calculates the attitude angle α as follows:

[0073]

number

[0074] It is calculated as follows.

[0075] In addition, when the rotation axis AX is located inside the penlight 6 as shown in FIG. 15, the distances from the rotation axis AX to the acceleration sensors 2A and 2B are (rr X ),r X Therefore, the processor 11A calculates the attitude angle α as follows:

[0076]

number

[0077] It is calculated as follows.

[0078] <Video display> In step S17, the processor 11A causes the image of the penlight 6 to be displayed on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU.

[0079] (Reflecting the rotation axis AX position) Fig. 16 is a diagram for explaining a method of reflecting the rotation axis position in the image display of the penlight 6 when the rotation axis AX is outside the penlight 6, which is the moving acquisition target, and Fig. 17 is a diagram for explaining a method of reflecting the rotation axis position in the image display of the penlight 6 when the rotation axis AX is inside the penlight 6. In Figs. 16 and 17, the position AX of the rotation axis AX, which is not actually displayed, in the image display is D A penlight image 6, which is an image of the penlight 6, is drawn with respect to D This shows that.

[0080] The processor 11A detects the position AX of the rotation axis AX in the image display. D is fixed, and the distance r to the rotation axis AX calculated in the z-axis direction of the penlight coordinate system is X Depending on the number, 6 penlight statues D In this way, the drawing position of the penlight image 6 is changed depending on whether the audience member AU rotates the penlight 6 around the wrist or the elbow. D The movements of the

[0081] (Reflecting attitude angles (α, β)) FIG. 18 is a diagram for explaining a method of reflecting the attitude angle in the image display of the penlight 6, which is a motion acquisition target object.

[0082] The processor 11A calculates the pitch angle (the attitude angle α calculated in step S16) and the yaw angle (the swing direction β calculated in step S14) in the world coordinate system (XYZ). D This allows the attitude angle of the penlight 6 to be reproduced.

[0083] As described above in detail, in the first embodiment of the present invention, two acceleration sensors 2A, 2B are arranged on the penlight 6, which is a motion acquisition target object that rotates around the rotation axis AX, and the information acquisition unit 3 acquires acceleration information detected by these two acceleration sensors 2A, 2B. Based on the acceleration information acquired by the information acquisition unit 3, the motion analysis unit 4 estimates the distance from one of the acceleration sensors 2A or 2B to the rotation axis AX and the attitude angle of the penlight 6. Therefore, according to the first embodiment, the rotation axis AX as well as the attitude angle of the penlight 6 are estimated using only two acceleration sensors 2A, 2B, so that it is possible to obtain differences in the movement of the penlight 6 without detecting them from outside the penlight 6.

[0084] The two acceleration sensors 2A and 2B are disposed on the penlight 6 and spaced apart from each other in the radial direction of rotation. Therefore, based on the difference in the direction of the acceleration information from the acceleration sensor 2A and the acceleration information from the acceleration sensor 2B, it becomes possible to determine whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0085] In addition, the motion analysis unit 4 calculates the swing direction of the penlight 6 in the world coordinate system, which is a reference coordinate system, based on the acceleration information, calculates the distance from one of the acceleration sensors to the rotation axis AX based on the acceleration information and the distance between the two acceleration sensors 2A, 2B, and calculates the attitude angle of the penlight 6 based on the distance between the two acceleration sensors 2A, 2B, the calculated swing direction of the penlight 6, and the calculated distance to the rotation axis AX. Therefore, the distance to the rotation axis AX and the attitude angle of the penlight 6 can be calculated based on the acceleration information from the two acceleration sensors 2A, 2B.

[0086] In addition, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A, 2B based on the acceleration information acquired by the information acquisition unit 3, and the calculation method for calculating the distance to the rotation axis AX and the attitude angle of the penlight 6 differs depending on whether the rotation axis AX is between the two acceleration sensors 2A, 2B or not. Therefore, by using a calculation method according to the position of the rotation axis AX, it is possible to calculate the distance to the rotation axis AX and the attitude angle of the penlight 6 with high accuracy.

[0087] Furthermore, in this first embodiment, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A, 2B based on the acceleration information acquired by the information acquisition unit 3, and the image display unit 5 displays an image of the penlight 6 on the display device PD and / or AD based on the distance to the rotation axis AX, the attitude angle of the penlight 6, and the determination result of whether the rotation axis AX is between the two acceleration sensors 2A, 2B. Therefore, an image display that reproduces the movement of the penlight 6 can be provided.

[0088] [Second embodiment] Next, a second embodiment of the present invention will be described. In the following description, the same reference numerals as those used in the first embodiment are used to denote the same parts as those in the first embodiment, and the description thereof will be omitted.

[0089] 19 is a block diagram showing an example of the configuration of a movement acquisition device 1 according to a second embodiment of the present invention. In the second embodiment, in addition to the configuration of the first embodiment, the input device AI includes a gyro sensor 7 that detects angular velocity.

[0090] FIG. 20 is a schematic diagram showing an example of a motion acquisition target object serving as the input device AI. In the second embodiment, the input device AI is also provided in the form of a penlight 6 held by an audience member AU. The gyro sensor 7 is installed in the same position as one of the two acceleration sensors 2A, 2B. For example, as shown in FIG. 20, the gyro sensor 7 is installed in the same position as the acceleration sensor 2A. In this case, the gyro sensor 7 is installed in a direction such that its three axes (x-axis, y-axis, z-axis) are aligned with the three axes of the acceleration sensor 2A.

[0091] FIG. 21 is a flowchart showing a processing routine in the motion acquisition device 1 according to the second embodiment. The processor 11A of the motion acquisition device 1 can perform the processing shown in this flowchart by executing a motion acquisition program stored in advance in, for example, the program memory 11B. The processor 11A executes the motion acquisition program in response to receiving a distribution viewing start instruction from the audience AU via the network NW through the communication interface 13. Note that the processing routine shown in this flowchart shows processing corresponding to one input device AI, and the processor 11A can perform similar processing in parallel for each of the multiple input devices AI.

[0092] The processor 11A operates as the information acquisition unit 3 and acquires acceleration information and angular velocity information (step S21). That is, the processor 11A receives the acceleration information from the two acceleration sensors 2A and 2B respectively arranged in the penlight 6 serving as the input device AI and the angular velocity information from the gyro sensor 7, which are transmitted via the network NW, through the communication interface 13, and stores the information in the received information storage unit 122 of the data memory 12.

[0093] As in the first embodiment, the processor 11A determines from the acceleration information whether or not the audience AU is waving the penlight 6 (step S12). If it is determined that the audience AU is not waving the penlight 6, the processor 11A proceeds to the process of step S21.

[0094] If it is determined that the spectator AU is waving the penlight 6, the processor 11A determines, as in the first embodiment, whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, in this embodiment as well, whether the rotation axis AX is inside or outside the penlight 6 (step S13).

[0095] As in the first embodiment, the processor 11A calculates the rotation plane, which is the swinging direction of the penlight 6, using the setting information stored in the setting information storage unit 121 and the acceleration information stored in the received information storage unit 122 (step S14).

[0096] The processor 11A calculates the attitude angle α of the penlight 6 using the acceleration information and angular velocity information stored in the reception information storage unit 122 (step S22). The details of this calculation method will be described later. The processor 11A stores the calculation result in the attitude angle information storage unit 124 of the data memory 12.

[0097] The processor 11A calculates the distance from the penlight 6 to the rotation axis AX using the setting information stored in the setting information storage unit 121, the acceleration information stored in the received information storage unit 122, and the determination result of whether the rotation axis AX is inside or outside the penlight 6 stored in the rotation axis information storage unit 123 (step S23). The method of calculating this distance differs depending on whether the rotation axis AX is inside or outside the penlight 6. The details of this calculation method will be described later. The processor 11A stores the calculation result in the rotation axis information storage unit 123 of the data memory 12.

[0098] The processor 11A, like the first embodiment, displays the image of the penlight 6 on the display device PD of the performer PE and / or the display device AD ​​of each audience member AU (step S17). The processor 11A judges whether or not to end the process, similarly to the first embodiment (step S18). If the processor 11A judges not to end the process, it proceeds to the process of step S11, and if it judges to end the process, it ends the process routine shown in this flowchart.

[0099] The processes in steps S22 and S23 will be described in detail below. <Attitude angle calculation> In step S22, the processor 11A calculates the attitude angle α of the penlight 6. Here, the attitude angle α is defined as the angle between the XY plane of the world coordinate system (XYZ) and the longitudinal direction of the penlight 6.

[0100] (When calculating only acceleration information) The acceleration in the x-axis direction obtained by the acceleration sensor 2A or 2B is a x ,z-axis direction is a z Then, the processor 11A calculates the pitch rotation angle p as follows:

[0101]

number

[0102] It can be calculated as follows.

[0103] The calculated pitch rotation angle p corresponds to the attitude angle α in the XZ plane. This method of calculating the attitude angle α based on acceleration information can calculate the attitude angle α with higher accuracy when the object whose motion is to be acquired moves at a low frequency than when the object moves at a high frequency.

[0104] (When only angular velocity information is used) The angular velocity information acquired by the gyro sensor 7 is (ω x ,ω y ,ω z ), the rotation angles (δφ, δθ, δψ) around the x, y, and z axes at a certain small time δt are (δφ, δθ, δψ)=(ω x δt,ω y δt,ω z δt) This can be expressed as:

[0105] The processor 11A calculates the roll, pitch and yaw rotation angles as (δ r , δ p , δ y ) as

[0106]

number

[0107] Therefore, the pitch rotation angle δ p can be calculated, where C θ = cosθ, S θ =sinθ.

[0108] Calculated pitch rotation angle δ p corresponds to the attitude angle α in the XZ plane. This method of calculating the attitude angle α based on angular velocity information can calculate the attitude angle α with higher accuracy when the object to be moved has a high frequency of movement, compared to when the object has a low frequency of movement.

[0109] (When calculating using acceleration and angular velocity information) Since the accuracy of the attitude angle α is poor when only the acceleration information or only the angular velocity information is used, the accuracy can be improved by sensor fusion.

[0110] For example, a complementary filter is used that calculates a weighted sum of an angle calculated by applying a low-pass filter to acceleration information and an angle calculated by applying a high-pass filter to angular velocity information. Corrected attitude angle = k * (attitude angle calculated from angular velocity information) + (1-k) * (attitude angle calculated from acceleration information) The corrected attitude angle α is calculated by the above.

[0111] In this way, the processor 11A can use the acceleration information and the angular velocity information to accurately calculate the attitude angle α both in a stationary state and in a moving state.

[0112] It should be noted that this embodiment is not limited to the complementary filter, and other filters such as a Kalman filter and a gradient filter may be used.

[0113] <Pivot axis distance calculation> In step S23, the processor 11A calculates the distance from the lower end of the penlight 6, which is the object of movement acquisition, where the acceleration sensor 2B is located, to the rotation axis AX in this embodiment. In the second embodiment, the calculation method also differs depending on whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0114] (When the rotation axis AX is outside the penlight 6) 22 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is outside the penlight 6, which is the moving acquisition target. Here, the acceleration in the x-axis direction is compared with the acceleration in the y-axis direction, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is large.

[0115] The longitudinal axis of the penlight 6 during movement is taken as the xz coordinate system, which is the instantaneous stationary coordinate system. If we add the acceleration in the xz coordinate system (ε=0) when it moves at an infinitesimal angle ε in that coordinate system, and the acceleration converted into the xz coordinate system by gravity, we get the acceleration a in the x-axis direction and the z-axis direction from the acceleration sensor 2A or 2B. x ,a z is obtained.

[0116] Where:

[0117]

number

[0118] Then, if ε=0,

[0119]

number

[0120] It is.

[0121] Taking into account the gravitational acceleration g, the acceleration a obtained by the acceleration sensor 2A is Ax ,a Az and the acceleration a acquired by the acceleration sensor 2B Bx ,a Bz Considering that,

[0122]

number

[0123] It becomes.

[0124] Therefore, the processor 11A calculates the following from the above formulas (3) and (5):

[0125]

number

[0126] The length r from the acceleration sensor 2B disposed at the bottom end of the penlight 6 to the rotation axis AX is X can be calculated.

[0127] (When the rotation axis AX is inside the penlight 6) 23 is a diagram for explaining variables used in estimating the rotation axis when the rotation axis AX is inside the penlight 6, which is the moving acquisition target. Here, the acceleration in the x-axis direction is compared with the acceleration in the y-axis direction, and the larger value is used. The following describes the case where the acceleration in the x-axis direction is larger.

[0128] The longitudinal axis of the penlight 6 during movement is taken as the xz coordinate system, which is the instantaneous stationary coordinate system. If we add the acceleration in the xz coordinate system (ε=0) when it moves at an infinitesimal angle ε in that coordinate system, and the acceleration converted into the xz coordinate system by gravity, we get the acceleration a in the x-axis direction and the z-axis direction from the acceleration sensor 2A or 2B. x ,a z is obtained.

[0129] Here, from the above formulas (1) and (2), taking into account the gravitational acceleration g, the acceleration a acquired by the acceleration sensor 2A is Ax ,a Az and the acceleration a acquired by the acceleration sensor 2B Bx ,a Bz Considering that,

[0130]

number

[0131] It becomes.

[0132] Therefore, the processor 11A calculates the following from the above formulas (8) and (10):

[0133]

number

[0134] The length r from the acceleration sensor 2B disposed at the bottom end of the penlight 6 to the rotation axis AX is X can be calculated.

[0135] As described above in detail, in the second embodiment of the present invention, two acceleration sensors 2A, 2B and one angular velocity sensor, the gyro sensor 7, are arranged on the penlight 6, which is an object for acquiring motion that rotates around the rotation axis AX, and the information acquisition unit 3 acquires acceleration information detected by the two acceleration sensors 2A, 2B and angular velocity information detected by the one gyro sensor 7. The motion analysis unit 4 estimates the distance from the penlight 6 to the rotation axis AX and the attitude angle of the penlight 6 based on the acceleration information and angular velocity information acquired by the information acquisition unit 3. Therefore, according to the second embodiment, the rotation axis AX as well as the attitude angle of the penlight 6 is estimated using only two acceleration sensors 2A, 2B and one gyro sensor 7, so that it is possible to obtain differences in the movement of the penlight 6 without detecting them from outside the penlight 6.

[0136] The two acceleration sensors 2A, 2B are disposed on the penlight 6, spaced apart in the radial direction of the rotation, and the one gyro sensor 7 is disposed at the same position as one of the two acceleration sensors 2A, 2B. Therefore, based on the difference in the direction of the acceleration information from the acceleration sensor 2A and the acceleration information from the acceleration sensor 2B, it becomes possible to determine whether the rotation axis AX is between the two acceleration sensors 2A and 2B, that is, whether the rotation axis AX is inside or outside the penlight 6.

[0137] In addition, the motion analysis unit 4 calculates the swing direction of the penlight 6 in the world coordinate system, which is a reference coordinate system, based on the acceleration information, calculates the attitude angle of the penlight 6 based on the acceleration information and the angular velocity information, and calculates the distance from the penlight 6 to the rotation axis AX based on the calculated swing direction of the penlight 6, the acceleration information, and the distance between the two acceleration sensors 2A, 2B. Therefore, based on the acceleration information from the two acceleration sensors 2A, 2B and the angular velocity information from the one gyro sensor 7, the distance to the rotation axis AX and the attitude angle of the penlight 6 can be calculated.

[0138] In addition, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A, 2B based on the acceleration information acquired by the information acquisition unit 3, and the method for calculating the distance to the rotation axis AX differs depending on whether the rotation axis AX is between the two acceleration sensors 2A, 2B or not. Therefore, by using a calculation method according to the position of the rotation axis AX, the distance to the rotation axis AX can be calculated with high accuracy.

[0139] Also in this second embodiment, as in the first embodiment, the motion analysis unit 4 determines whether the rotation axis AX is between the two acceleration sensors 2A, 2B based on the acceleration information acquired by the information acquisition unit 3, and the image display unit 5 displays an image of the penlight 6 on the display device PD and / or AD based on the distance to the rotation axis AX, the attitude angle of the penlight 6, and the determination result of whether the rotation axis AX is between the two acceleration sensors 2A, 2B. Therefore, an image display that reproduces the movement of the penlight 6 can be provided.

[0140] [Third embodiment] Next, a third embodiment of the present invention will be described. In the following description, the same reference numerals as those used in the first embodiment are used to denote the same parts as those in the first embodiment, and the description thereof will be omitted.

[0141] 24 is a block diagram showing an example of the configuration of a movement acquisition device 1 according to a third embodiment of the present invention. In the third embodiment, in addition to the configuration of the first embodiment, the input device AI includes a geomagnetic sensor 8 which is a direction sensor.

[0142] Fig. 25 is a schematic diagram showing an example of a motion acquisition target object serving as an input device AI. In the third embodiment, the input device AI is also provided in the form of a penlight 6 held by an audience member AU. As shown in Fig. 25, one geomagnetic sensor 8 is installed at an end of the penlight 6 in an orientation such that the xy plane of the geomagnetic sensor 8 is on a plane perpendicular to the longitudinal direction of the penlight 6.

[0143] The geomagnetic sensor 8 acquires the strength of the geomagnetic field. The center of the circle of the output distribution map when the geomagnetic sensor 8 is rotated horizontally is (P x ,P y ), and the strength of the geomagnetic field acquired by the geomagnetic sensor 8 is (X, Y), the angle from magnetic north can be calculated as follows.

[0144]

number

[0145] It should be noted that the above formula can be used only when the xy plane of the geomagnetic sensor 8 is perpendicular to the vertical direction, so the processor 11A selects the measurement timing by one of the following methods. When the pitch angle (attitude angle α) is 90 degrees (after attitude angle calculation) The midpoint between the two timings of the Penlight 6 swing - The timing when the speed is at its maximum (calculated from acceleration) The processor 11A operating as the motion analysis unit 4 can know the direction of the penlight 6 based on the strength of the geomagnetic field acquired by the geomagnetic sensor 8. By acquiring the direction in which the screen of the display device AD ​​is located in advance and storing it in the setting information storage unit 121, the processor 11A can determine the angle (angle T in FIG. 10) between the front of the screen and the front of the penlight 6. In other words, it becomes possible to define the conversion between the world coordinate system (XYZ) of the screen and the coordinate system (xyz) of the penlight 6.

[0146] This eliminates the need for an operation (calibration) to convert between the screen coordinate system and the coordinate system of the penlight 6 by having the audience AU wave the penlight 6, as described in the rotation plane calculation process of the first embodiment, or an operation to fix the front direction of the penlight 6, thereby reducing the burden on the audience AU.

[0147] 26 is a block diagram showing another example of the configuration of the movement acquisition device 1 according to the third embodiment of the present invention. In this manner, the geomagnetic sensor 8 is applicable not only to the first embodiment but also to the movement acquisition device 1 according to the second embodiment.

[0148] As described above in detail, in addition to the configuration of the first or second embodiment, the third embodiment of the present invention further includes a geomagnetic sensor 8, which is a direction sensor, arranged at the end of the penlight 6 in the longitudinal direction so that the xy plane of detection is in a direction perpendicular to the longitudinal direction of the penlight 6, which is the radial direction of rotation. Therefore, according to the third embodiment, the burden on the audience AU can be reduced by utilizing the output of the geomagnetic sensor 8, which is a direction sensor.

[0149] [Other embodiments] It should be noted that the present invention is not limited to the above-described embodiment.

[0150] For example, the above embodiment has been described using an example of a swinging movement centered on the wrist and elbow, but it goes without saying that it is also possible to detect not only swinging movements, but also movements such as raising the arm and making a circular motion above the head centered on the shoulder.

[0151] Furthermore, the motion acquisition object is not limited to the shape of the penlight 6, and may be in any form as long as it can be held by the spectator AU. Furthermore, the motion acquisition object may take a form other than that held by the spectator AU. For example, the motion acquisition object may be in a form worn on the body of the spectator AU, such as the arm. In this case, the elbow or shoulder serves as a rotation axis or a rotation axis for rotating the motion acquisition object, and therefore, it can be handled in the same manner as the case where the rotation axis AX described in the above embodiment is on the outside of the penlight 6.

[0152] Furthermore, the above embodiment has been described using the example of live streaming between a performer PE and an audience AU, but it goes without saying that the present invention can be applied to a variety of applications, such as a virtual kendo match in which the penlights 6 are used as bamboo swords.

[0153] In addition, the flow of each process described with reference to the flowchart is not limited to the procedures described, and the order of some steps may be changed, some steps may be performed simultaneously in parallel, or the processing content of some steps may be modified.

[0154] The methods described in each embodiment can be stored as a processing program (software means) that can be executed by a calculator (computer) on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), a semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The programs stored on the medium side include a setting program that configures the software means (including not only execution programs but also tables and data structures) that the calculator executes within the computer. The computer that realizes this device reads the program recorded on the recording medium, and in some cases, configures the software means by the setting program, and executes the above-mentioned processing by controlling the operation of the software means. The recording medium referred to in this specification is not limited to a recording medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory provided inside the computer or in a device connected via a network.

[0155] In short, this invention is not limited to the above-mentioned embodiment as it is, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components across different embodiments may be appropriately combined. [Explanation of symbols]

[0156] 1...Motion acquisition device 2A, 2B...Acceleration sensor 3…Information acquisition department 4...Motion analysis section 5. Video display section 6. Penlight 6 D … Penlight statue 7...Gyro sensor 8...Geomagnetic sensor 11A…Processor 11B…Program memory 12…Data memory 121...Setting information storage unit 122...received information storage unit 123... Rotation axis information storage unit 124…Attitude angle information storage unit 125...Video storage section 13...Communication interface 14…Bus AD,AD1,AD2,AD3,ADn,PD…Display device AI, AI1, AI2, AI3, AIn...input device AU,AU1,AU2,AU3,AYn...Audience AX...rotation axis AX D …Position of rotation axis PC: Imaging device PE…Performer SV: Distribution server NW…Network

Claims

1. Two acceleration sensors are arranged on a motion acquisition object that is rotated around a rotation axis; an information acquisition unit that acquires acceleration information detected by the two acceleration sensors; a motion analysis unit that estimates a distance from one acceleration sensor to the rotation axis and an attitude angle of the motion acquisition target object based on the acceleration information acquired by the information acquisition unit; Equipped with The motion analysis unit is Calculating a swing direction of the motion acquisition object in a reference coordinate system based on one of the acceleration information; calculating a distance from the one acceleration sensor to the rotation axis based on the acceleration information and a distance between the two acceleration sensors; a motion capture device that calculates the attitude angle of the motion capture object based on the distance between the two acceleration sensors, the calculated swing direction of the motion capture object, and the calculated distance to the rotation axis.

2. The motion acquisition device according to claim 1 , wherein the two acceleration sensors are disposed on the motion acquisition object and spaced apart in a radial direction of the rotation.

3. The motion analysis unit is determining whether the rotation axis is between the two acceleration sensors based on the acceleration information acquired by the information acquisition unit; 3. The motion capture device according to claim 1, wherein a calculation method for calculating the distance to the rotation axis and the attitude angle of the motion capture object is different depending on whether the rotation axis is between the two acceleration sensors or not.

4. 4. The motion acquisition device according to claim 1, further comprising an orientation sensor disposed on the motion acquisition object such that an xy plane of detection is in a direction perpendicular to the radial direction of the rotation.

5. The motion analysis unit determines whether the rotation axis is between the two acceleration sensors based on the acceleration information acquired by the information acquisition unit, The motion acquisition device according to any one of claims 1 to 4, further comprising an image display unit that displays an image of the motion acquisition object based on the distance to the rotation axis, the attitude angle of the motion acquisition object, and a determination result of whether the rotation axis is between the two acceleration sensors.

6. A motion acquisition method for an apparatus for acquiring motion of a motion acquisition object rotated around a rotation axis, comprising: acquiring acceleration information detected by two acceleration sensors disposed on the motion acquisition target; estimating a distance from one acceleration sensor to the rotation axis and an attitude angle of the motion acquisition object based on the acquired acceleration information; and estimating the distance and the attitude angle includes: Calculating a swing direction of the motion acquisition target object in a reference coordinate system based on one of the acceleration information; calculating a distance from the one acceleration sensor to the rotation axis based on the acceleration information and a distance between the two acceleration sensors; calculating the attitude angle of the motion acquisition target object based on a distance between the two acceleration sensors, the calculated swing direction of the motion acquisition target object, and the calculated distance to the rotation axis; A motion acquisition method comprising:

7. A motion capture program that causes a computer to execute processing by each unit of the motion capture device according to any one of claims 1 to 5.

Images