Apparatus, method, and program for generating training information related to human movement

Abstract

This method generates training information about human movement by utilizing the influence of motion phase on primitives. [Solution] The device 100 includes an imaging unit 102 for imaging motion, a recording unit 104 for generating and storing a record including imaging data 10A related to motion and the period in which the motion is performed, a phase identification unit 106 for identifying the phase in the period of each of the one or more actions included in the motion from the record, a training information generation unit 110 for generating training information including the phase of each of the one or more actions and the target phase, and an output unit 114 for outputting the training information.

Description

Technical Field

[0001] The present disclosure relates to a technique for generating training information related to human exercise.

Background Art

[0002] It is widely known that primitives, which are composed of groups of neurons in the human brain and are responsible for the memory of motor learning, act in a manner dependent on the body's motion state such as position and velocity (state-dependent primitive). This finding indicates that when performing a plurality of motions with different positions or velocities of the moving body, different primitives are active in each of these motions, and the neuron groups can acquire different motor memories from each motion. On the other hand, when the motion with the same position and velocity of the moving body is repeated, the same primitives are always active, so the neuron groups can only acquire one motor memory.

[0003] Regarding primitives responsible for motor learning memory, for example, Non-Patent Document 1 shows that according to the difference in the lead-in movement performed before the main motion to be learned, even if the state of the main motion is exactly the same, the neuron groups can acquire different motor memories. In addition, Non-Patent Document 2 shows that according to the difference in the follow-through movement performed after the main motion to be learned, even if the state of the main motion is exactly the same, the neuron groups can acquire different motor memories.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

[0005] Non-patent documents 1 and 2 suggest that adding some motion before or after the main motion to be learned changes the overall motion state, and that different primitives are recruited depending on that motion state. However, non-patent documents 1 and 2 do not suggest the effect that the phase in the overall motion at which the main motion is performed has on the primitives.

[0006] This disclosure is made in view of the above-mentioned background, and in one aspect, its purpose is to provide a technology for generating training information about human movement by utilizing the influence of factors other than the movement state on primitives. More specifically, it is to provide a technology for generating training information about human movement by utilizing the influence of the phase of movement on primitives. [Means for solving the problem]

[0007] According to one embodiment, an apparatus is provided. The apparatus comprises an imaging unit for imaging motion; a recording unit for generating and storing a record including imaging data relating to motion and the period in which the motion is performed; a phase identification unit for identifying the phase in the period of each of one or more actions included in the motion from the record; a training information generation unit for generating training information including the phase and target phase of each of one or more actions; and an output unit for outputting the training information.

[0008] In a given scenario, the record includes performance information associated with the movement. The recording unit is configured to store multiple records relating to the movement of the same person in an associated manner. The device further includes a comparison unit for generating comparison information between a first record and a second record contained in the multiple records. The comparison information includes the phase and performance information of each of one or more movements contained in the first record, and the phase and performance information of each of one or more movements contained in the second record. The output unit is configured to output the comparison information.

[0009] In a given scenario, the training information generation unit determines changes in performance information from comparison information, adjusts the target phase of one or more actions included in the previous training information based on the changes in performance information, and is configured to generate new training information.

[0010] In a given scenario, the record is configured to store multiple training records related to the same person's movements in an associated manner. The comparison unit is configured to extract the history of changes in performance information related to movement and the history of changes in the phases of one or more movements from the multiple training records and multiple records, and to generate analysis results. The output unit is configured to output the analysis results.

[0011] In a given scenario, the device further includes an action identification unit for extracting one or more actions from imaging data. The action identification unit extracts one or more actions based on receiving inputs specifying each of the one or more actions, or based on predetermined settings.

[0012] In a given scenario, the training information generation unit is configured to extract some actions from the imaging data and generate training information based on those actions.

[0013] In a given scenario, imaging motion includes imaging motion and performance routines. The period is the period during which the motion and performance routines are executed. The phase identification unit is configured to identify the phase of each of one or more actions based on the period, which includes the execution period of the performance routine and motion.

[0014] In other embodiments, a method is provided that is performed by the apparatus. The method includes imaging motion; generating and storing a record including imaging data relating to the motion and the period in which the motion is performed; identifying the phase in the period for each of one or more actions included in the motion from the record; generating training information including the phase and target phase for each of the one or more actions; and outputting the training information.

[0015] In yet another embodiment, a program is provided which is executed by the device. The program causes the device to image motion, generate and store a record including image data of the motion and the period in which the motion is performed, identify the phase in the period of each of the one or more actions included in the motion from the record, generate training information including the phase and target phase of each of the one or more actions, and output the training information. [Effects of the Invention]

[0016] According to one embodiment, training information about human movement can be generated by utilizing the effect of the phase of motion on primitives.

[0017] The above and other purposes, features, aspects and advantages of this disclosure will become apparent from the following detailed description of this disclosure, which will be understood in conjunction with the attached drawings. [Brief explanation of the drawing]

[0018] [Figure 1] This is a diagram showing an example of the functional configuration of the device 100 according to the present embodiment. [Figure 2] This is a diagram showing an example of the hardware configuration of the device 100. [Figure 3] This is a diagram showing an example of the first function of the device 100. [Figure 4] This is a diagram showing an example of the second function of the device 100. [Figure 5] This is a diagram showing an example of the third function of the device 100. [Figure 6] This is a diagram showing an example of the processing procedure of the device 100. [Mode for Carrying Out the Invention]

[0019] Hereinafter, embodiments of the technical idea according to the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also identical. Therefore, detailed descriptions thereof will not be repeated. Further, each embodiment, each modified example, the configuration of each piece of software or program, each hardware configuration, each function, each process and the like may be selectively combined as appropriate.

[0020] <Device Configuration> Figure 1 is a diagram showing an example of the functional configuration of the device 100 according to the present embodiment. With reference to Figure 1, the use case of the device 100, terms necessary for describing the technology of the present disclosure, and the configuration and operation of the device 100 will be described.

[0021] (Use Case of Device) As an example, the device 100 provides training information in sports, music performance, rehabilitation, and any other optional physical activities. As an example, the device 100 may provide training information for mastering new movements or techniques in sports, music performance and the like. As another example, the device 100 may provide training information for breaking through a situation where mastering new movements has become difficult in sports, music performance and the like.

[0022] More specifically, the device 100 can provide training information that utilizes the effect of motion phase on primitives. By repeatedly performing the motion specified by the training information, the user can efficiently activate the target primitive and acquire a new movement or skill.

[0023] (term) In this specification, "primitive" refers to a group of neurons in the human brain. For example, the primitive related to a golf club swing refers to the group of neurons that are active when a golf club is swung. The primitives associated with each movement may include the same neurons. For example, a tennis racket swing and a badminton racket swing involve similar movements, and the primitives associated with each of these movements may include partially the same neurons.

[0024] In this specification, "movement" refers to any physical activity performed by a human being. For example, basic movements such as reaching out and grasping are considered movements. Other examples include complex movements involving multiple actions, such as swinging a golf club. Movement also includes discrete movements and continuous movements. Discrete movements are physical activities that are completed in one step, such as a tennis serve. Continuous movements are physical activities that involve repeating the same action, such as walking and swimming. Continuous movements include the repetition of discrete movements. In this specification, a person who is being trained in some kind of movement may also be referred to as a "trainee." In this specification, the movement that a trainee is training in may also be referred to as a "trainee movement."

[0025] In this specification, "movement" refers to individual movements, postures, and stances that are part of a movement. For example, a golf swing includes movements and stances called address, backswing, top of swing, downswing, impact, follow-through, and finish. These movements and stances are movements that make up the golf swing. Furthermore, a movement includes the transition from one stance or posture to another. How a movement is divided into multiple movements can be determined appropriately depending on the training objectives. There can be countless movements within a movement.

[0026] In this specification, "period" refers to the period from the start to the end of motion. The period from the start to the end of discrete motion can be expressed as one period. That is, discrete motion is motion with one period. Continuous motion is a repetition of discrete motion and is motion with multiple periods. As an example, the period is expressed in radians. The period may also be expressed in any unit other than radians, such as a percentage. When the period is expressed in radians, the start of the motion is "0", the midpoint is "π", and the end of the motion is "2π". When the period is expressed in percentages, the start of the motion is "0", the midpoint is "50", and the end of the motion is "100".

[0027] In this specification, "phase" refers to the degree of progression within the overall motion. Phase indicates the position within a period. For example, suppose we define a period for a series of actions, "golf club swing." Suppose the action "impact" occurs at the midpoint of this period. In this case, the phase of the action "impact" is "π" in radians. The granularity of the period can be determined as appropriate. When the period is expressed in radians, for example, the granularity may be in units of 1 degree or 0.001 degrees. Furthermore, the phase may be expressed in any unit other than radians, such as a percentage.

[0028] In this specification, "training information" refers to information used by the user to train for movement. The training information includes the phase of each of one or more movements included in the movement. The phase of each of these one or more movements may be calculated from imaging data obtained by imaging (or photographing) the user's movement one or more times. That is, these phases are the current phase of each movement included in the user's movement. The training information also includes the target phase of each of the one or more movements included in the movement. The user refers to the training information and trains to bring the phase of each of the one or more movements closer to the target phase. In addition to phase information, the training information may also include training advice, photographs or diagrams of each movement included in the movement, etc. Furthermore, the training information may include a history of the phase of each of the one or more movements that has changed through past training, and a history of the target phase specified in past training.

[0029] (The effect of phase on primitives) In this research, a new finding was obtained: the primitives responsible for motor learning memory do not depend solely on the state of the movement, but also on the phase of the movement. Let's explain the effect of phase on primitives using walking as an example. When walking, the left and right feet make contact with the ground. In such movements where parts of the body make contact with objects, walkers need to predict the external force and move their bodies appropriately, which requires learning. If the ground contact occurs at the same phase each time, the same primitives will always be active, allowing walkers to learn walking efficiently. Conversely, if the ground contact occurs at different phases each time, different primitives will be active each time, which may cause walkers to take longer to learn walking.

[0030] Device 100 utilizes the above knowledge to measure the user's movements and calculate the phase at which each movement occurs. Device 100 then provides training information with a target phase set for each movement. Device 100 may also evaluate the variability in the phase at which each movement occurs and include this variability in the training information.

[0031] (Functional blocks of the device) The device 100 includes, as functional blocks, an imaging unit 102, a recording unit 104, a phase identification unit 106, an operation identification unit 108, a training information generation unit 110, a comparison unit 112, an output unit 114, a storage area 120, a recording DB (Database) 122, and a training information DB 124. Some or all of these functional blocks may be implemented as programs. In this case, these functional blocks are implemented by executing programs on the hardware shown in Figure 2. Alternatively, some or all of these functional blocks may be implemented as hardware. The device 100 may also have a UI (User Interface) for using each function.

[0032] The imaging unit 102 acquires and outputs imaging data 10A of the movements performed by the trainee. In the example in Figure 1, the imaging data 10A is imaging data of the movement of swinging a golf club. The imaging data 10A also includes actions 11A, 12A, 13A, 14A, 15A, and 16A. The imaging unit 102 may include a camera. In this case, the imaging unit 102 records the imaging data 10A of the movements performed by the trainee. The imaging unit 102 does not have to include a camera. In this case, the imaging unit 102 receives the imaging data 10A of the movements performed by the trainee from a wired or wirelessly connected camera. The imaging unit 102 may repeatedly acquire and output the imaging data 10A of the movements performed by the trainee multiple times. The imaging unit 102 outputs one or more pieces of imaging data 10A to the recording unit 104. In this specification, "imaging data 10A" includes video, sequential images, and combinations thereof.

[0033] The recording unit 104 generates a record that includes imaging data 10A related to motion and the period during which the motion is performed. The recording unit 104 may generate a record from one imaging data 10A. The recording unit 104 may generate a record from multiple imaging data 10A. The recording unit 104 may also accept performance information associated with the imaged motion via the UI and include such performance information in the record. For example, if the motion is a golf club swing, the performance information may include the speed of the golf club head. The recording unit 104 may also calculate performance information by analyzing the imaging data 10A. As an example, the recording unit 104 can calculate the speed of the golf club head by analyzing the imaging data 10A. The recording unit 104 may also save the record in the storage area 120. The recording unit 104 may output the record to the phase identification unit 106. The recording unit 104 may output the recording to both the storage area 120 and the phase identification unit 106, or to just one of them.

[0034] The phase identification unit 106 identifies the phase during the period of each of the one or more actions included in the motion from the recording. The phase identification unit 106 may acquire the recording from the imaging unit 102 or read the recording from the storage area 120. For example, the phase identification unit 106 may identify the phase of each of the actions 11A, 12A, 13A, 14A, 15A, and 16A from one or more imaging data 10A included in the recording. The phase identification unit 106 may identify the phase of each action from the timing at which each action occurred between the start time and the end time (i.e., one period) of the imaging data 10A. The phase identification unit 106 adds the identified phase of each action to the recording. If the recording includes multiple imaging data 10A, the phase identification unit 106 identifies the phase of each action for each imaging data 10A. Then, the phase identification unit 106 adds the phase of each action in each imaging data 10A to the recording. The phase of each action in each imaging data 10A may indicate the variation in the phase of each action. The phase identification unit 106 may also calculate the average value of the phase of each operation and add the average value of the phase of each operation to the record. The phase identification unit 106 stores the record with the phase of each operation added in the storage area 120. The phase identification unit 106 may also output the record with the phase of each operation added to the training information generation unit 110.

[0035] The operation identification unit 108 extracts one or more operations from the imaging data 10A. The operation identification unit 108 may also accept input via the UI to select one or more operations from the imaging data 10A. The operation identification unit 108 may output information of the selected one or more operations to the phase identification unit 106. Based on the information of the selected one or more operations, the phase identification unit 106 identifies the phase of each of the one or more operations. For example, suppose the operation identification unit 108 accepts input to select operations 11A, 12A, 13A, 14A, 15A, and 16A. In this case, the operation identification unit 108 outputs information to the phase identification unit 106 indicating that operations 11A, 12A, 13A, 14A, 15A, and 16A have been selected. Based on this information, the phase identification unit 106 identifies the phase of each of the operations 11A, 12A, 13A, 14A, 15A, and 16A.

[0036] The training information generation unit 110 generates training information 150A that includes the phase and target phase of one or more movements based on the added records of the phase of each movement. The training information generation unit 110 may accept input operations for the target phase of one or more movements via the UI. In this case, the training information generation unit 110 generates training information 150A that includes the target phase of the one or more movements that were input. For example, suppose the training information generation unit 110 accepts input operations for the target phase 156A of movement 14A "impact" via the UI. In this case, the training information generation unit 110 generates training information 150A that includes the current phase 154A of movement 14A and the input target phase 156A. The training information 150A may also include the phase variation 160A of the selected movement 14A. The training information 150A may also include the period 152A of the movement. The training information 150A may also include the current phases of other actions (actions 11A, 12A, 13A, 15A, 16A). The training information generation unit 110 may also read target phase setting information from the memory area 120 and determine the target phases of one or more actions based on that setting information. In this case, the training information generation unit 110 generates training information 150A that includes the target phases of the one or more actions determined. For example, the setting information may be generated based on the results of an analysis of the movements of famous athletes in the past. The training information generation unit 110 stores the generated training information 150A in the memory area 120. The training information generation unit 110 may also output the generated training information 150A to the comparison unit 112. The training information generation unit 110 may also output the generated training information 150A to the output unit 114.

[0037] The comparison unit 112 generates comparison information for two or more records of the same movement performed by the same person. In this specification, "comparison information" refers to information showing the differences in performance information and the phase of each movement in two or more records of the same movement performed by the same person. For example, suppose a trainee trained on some movement based on training information 150A. In this case, it is desirable for the trainee to check the changes in performance information and the phase of each movement before and after training. The trainee can check these changes by referring to the comparison information. The comparison unit 112 may generate comparison information based on receiving an instruction input to generate comparison information via the UI. Also, if past records exist when the training information 150A is generated, the comparison unit 112 may add comparison information to the training information 150A. In this case, the training information 150A may include feedback information to the trainee and a history of the trainee's performance and changes in the phase of each movement. The comparison unit 112 may store the generated comparison information in the storage area 120. Furthermore, each time the comparison unit 112 receives an instruction to generate comparison information, it may read two or more records from the memory area 120 and generate comparison information. If the comparison unit 112 adds comparison information to the training information 150A, it saves the training information 150A to the memory area 120. The comparison unit 112 may output the generated comparison information to the output unit 114.

[0038] The output unit 114 outputs various types of information. The output unit 114 is configured to output imaging data 10A, recording and training information 150A, feedback information, comparison information, and analysis information thereof. The output unit 114 may also output a UI for using the various functions of the device 100. The output unit 114 may display the various types of information on a display connected to or built into the device 100. The output unit 114 may transmit the various types of information to other devices via a network.

[0039] The storage area 120 stores various programs and data referenced by the device 100. The storage area 120 includes a record DB 122 and a training information DB 124. The storage area 120 encompasses relational databases, hierarchical databases, network databases, NoSQL databases, or any other database. Furthermore, the storage area 120 includes storage and file servers.

[0040] The record database 122 stores the records. Each record is associated with a trainee and a training exercise. For example, suppose trainee X is training in the exercise "bat swing" and the exercise "soccer ball dribbling" using device 100. In this case, one or more records of trainee X's bat swing are associated with trainee X and the exercise "bat swing". Similarly, one or more records of trainee X's soccer ball dribbling are associated with trainee X and the exercise "soccer ball dribbling". Comparison information is generated based on multiple records associated with the same trainee and training exercise.

[0041] The training information DB124 stores training information 150A. Each training information 150A, like a record, is associated with the trainee and the exercise being trained. One or more training information 150A related to trainee X's bat swing are associated with trainee X and the exercise "bat swing".

[0042] Device 100 includes personal computers, tablets, smartphones, smartwatches, wearable glasses, server devices, virtual machines built on a cloud environment, and any other information processing devices, or any combination thereof.

[0043] As described above, the device 100 can provide the trainee with training information that focuses on the influence of phase on primitives. Based on the training information, the trainee can perform training that focuses on the phase of one or more movements. The human brain has a function to correct discrepancies. The device 100 facilitates the correction of the trainee's movements by presenting the discrepancy between the current phase and the target phase of one or more movements included in the trainee's movement based on the training information.

[0044] Furthermore, phase can be defined not only in discrete motion but also in continuous motion. Therefore, device 100 can be used not only for training discrete movements such as baseball pitching and tennis serves, but also for training continuous movements such as walking and swimming. Moreover, phase can be defined in musical performance, which involves complex sequential movements, and in everyday movements. In other words, device 100 can be used for training any physical activity of a human being.

[0045] (Device hardware configuration) Figure 2 shows an example of the hardware configuration of device 100. Device 100 does not necessarily have to include some of the configurations shown in Figure 2. Furthermore, device 100 may include configurations not shown in Figure 2. In addition, device 100 may include two or more of each configuration. The various functions of device 100 described herein can be realized by executing a program on the hardware shown in Figure 2.

[0046] Device 100 includes a processor 201, memory 202, storage 203, external device interface 204, input interface 205, output interface 206, and communication interface 207. Device 100 also includes a bus 208 that connects these components.

[0047] The processor 201 can execute programs to realize various functions of the device 100. The processor 201 is composed of, for example, at least one integrated circuit. According to one embodiment, the integrated circuit may include at least one CPU (Central Processing Unit), at least one GPU (Graphics Processing Unit), at least one FPGA (Field Programmable Gate Array), at least one ASIC (Application Specific Integrated Circuit), at least one AI (Artificial Intelligence) chip, or a combination thereof.

[0048] Memory 202 functions as a workspace for processor 201. Memory 202 stores programs executed by processor 201 and data referenced by processor 201. Memory 202 can be implemented using DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory), etc.

[0049] Storage 203 is non-volatile memory that stores programs executed by processor 201 and data referenced by processor 201. Processor 201 executes programs read from storage 203 to memory 202 and references data read from storage 203 to memory 202. Storage 203 can be implemented by HDD (Hard Disk Drive), SSD (Solid State Drive), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory, etc.

[0050] The external device IF204 can be connected to any external device such as a printer, scanner, and external HDD. The external device IF204 can be implemented using a USB (Universal Serial Bus) terminal or the like.

[0051] The input interface 205 can be connected to any input device such as a keyboard, mouse, touchpad, or gamepad. The input interface 205 can be implemented by a USB terminal, PS / 2 terminal, and a Bluetooth® module, etc. Device 100 may include an input device.

[0052] Output IF206 can be connected to any output device such as a cathode ray tube display, liquid crystal display, or organic EL display. Output IF206 can be implemented using a USB terminal, D-sub terminal, DVI (Digital Visual Interface) terminal, HDMI® (High-Definition Multimedia Interface) terminal, and DisplayPort terminal, etc. Device 100 may include an output device.

[0053] The communication IF207 is connected to other devices via a wired or wireless network. The communication IF207 can be implemented using a wired LAN (Local Area Network) port and a Wi-Fi® (Wireless Fidelity) module, etc. The communication IF207 can send and receive data using communication protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) and UDP (User Datagram Protocol).

[0054] <Device Functions> The first, second, and third functions of device 100 will be described with reference to Figures 3 to 5. The first, second, and third functions can be used in combination. Furthermore, the functions of device 100 described with reference to one figure can also be used in the examples in other figures. For example, in the example in Figure 3, device 100 is configured to communicate with a generated AI (Artificial Intelligence) model (not shown), and this is also the case in Figures 4 and 5.

[0055] Figure 3 shows an example of the first function of the device 100. Referring to Figure 3, the first function of the device 100 will be explained using the example of improving the golf club swing (training target movement) of a trainee. In the first function, the device 100 generates training information 150A for the trainee. After the trainee has trained based on the training information 150A, the device 100 generates training information 150B again for the trainee after training. In other words, the device 100 can update the training information as appropriate according to the trainee's training results.

[0056] First, the device 100 images the training movement performed by the trainee and acquires image data 10A of the training movement. The device 100 also stores the record, including the image data 10A, in the storage area 120. The device 100 may image the training movement multiple times and acquire multiple sets of image data 10A. In this case, the trainee performs the training movement multiple times within the camera's shooting range.

[0057] Next, the device 100 extracts one or more actions from the imaging data 10A. The device adds the information of the extracted one or more actions to the record and saves the record in the storage area 120. As an example, the device 100 may extract important actions from the imaging data 10A. Important actions may include actions that involve contact with the outside world, actions that involve a large change in body movement, etc. In the example in Figure 3, the device 100 extracts actions 11A, 12A, 13A, 14A, 15A, and 16A from the imaging data 10A.

[0058] The device 100 may accept a selection input for an action to be extracted from the UI and extract one or more actions from the imaging data 10A based on the selection input. Alternatively, the device 100 may read action extraction settings for each movement from the storage area 120 and extract one or more actions from the imaging data 10A based on the extraction settings. The action extraction settings for each movement may include extraction settings for actions included in any movement. For example, the action extraction settings for each movement may include extraction settings for actions in any movement such as swinging a golf club, playing the piano, or opening a door. The device 100 may also be configured to communicate with a trained AI model capable of extracting one or more actions from the imaging data 10A based on the extraction settings. The device 100 may also include such a trained AI model.

[0059] Next, the device 100 identifies the phase of one or more movements from the imaging data 10A included in the recording. The device adds the identified phase information to the recording and saves the recording in the storage area 120. The device 100 may identify the phase of all extracted movements. Alternatively, the device 100 may identify the phase of some of the movements included in the multiple extracted movements. The device 100 may accept a selection input for the movement whose phase to be identified from the UI and identify the phase of the selected movement based on that selection input. Furthermore, the movement extraction setting for each motion may include information about the movement whose phase should be identified. In this case, the device 100 identifies the phase of one or more movements based on the movement extraction setting for each motion. In the example in Figure 3, the device 100 identifies the current phase 154A of movement 14A (impact).

[0060] Next, the device 100 sets the target phase for one or more operations. The device 100 may set the target phase based on operation input via the UI or target phase setting information. In the example in Figure 3, the device 100 sets the target phase 156A for operation 14A (impact).

[0061] Next, the device 100 generates and outputs training information 150A. The training information 150A includes the current phase 154A and target phase 156A of one or more movements. The training information 150A may also include the period of the entire movement 152A, performance information 300A, the variation in the period of one or more movements 160A, and one or more images or illustrations of each movement. The current phase 154A may be identified from the most recent (last captured) imaging data. The variation in the period of one or more movements 160A may include the phase of one or more movements in all or some of the imaging data captured to date. The training information 150A displays each timing from the start to the end of the entire movement as a phase rather than time.

[0062] By referring to training information 150A, trainees can recognize the difference between the current phase 154A of the action of interest 14A and the target phase 156A. By training with this recognition of the difference, trainees can bring the phase of the action of interest 14A closer to the target phase 156A.

[0063] Next, the device 100 again images the training target movement performed by the training subject and acquires the image data 10B of the training target movement. The image data 10B is the image data obtained after the training subject has completed the training. The device 100 also saves the record, including the image data 10B, to the storage area 120. In the example in Figure 3, the device 100 extracts the actions 11B, 12B, 13B, 14B, 15B, and 16B from the image data 10B.

[0064] Next, the device 100 generates training information 150B using the same procedure as when generating training information 150A. Training information 150B includes the current phase 154B and the target phase 156B of one or more movements. Training information 150B may also include the overall period of the movement 152B, performance information 300B, the variation in the period of one or more movements 160B, and one or more images or illustrations of each movement.

[0065] The trainee can recognize the difference between the current phase 154B of the movement of interest 14B and the target phase 156B, while referring to training information 150B. By training with this recognition of the difference, the trainee can further improve the performance of the movement being trained.

[0066] Training information 150B may include information contained in training information 150A. More specifically, training information 150B may include comparison information between the information contained in training information 150B and the information contained in training information 150A. Furthermore, training information 150B may include detailed analysis information obtained from the comparison information. The comparison information may include at least one of the following: the phase of each operation before and after training, the change in the variation of the phase of each operation, performance information, and the target phase. In addition, the device 100 may generate and output feedback information that includes comparison information obtained from the imaging data 10A and 10B before and after training, instead of training information 150B.

[0067] By referring to comparative information, trainees can recognize changes in the phase of their action of interest and changes in performance information. Furthermore, supervisors of trainees' training can use comparative information as a basis for determining the next target phase. For example, if an improvement in performance information is observed by increasing the phase of the action of interest, the supervisor may set the target phase even higher. Conversely, if a decrease in performance information is observed by increasing the phase of the action of interest, the supervisor may set the target phase lower. Additionally, device 100 may adjust the previous target phase and determine a new target phase based on the changes in performance information included in the comparative information.

[0068] As explained with reference to Figures 1-3, the apparatus 100 comprises an imaging unit 102 for imaging motion, a recording unit 104 for generating and storing a record including imaging data 10A, 10B related to motion and the period in which the motion is performed, a phase identification unit 106 for identifying the phase in the period of each of the one or more actions included in the motion from the record, a training information generation unit 110 for generating training information including the phase of each of the one or more actions and the target phase, and an output unit 114 for outputting the training information.

[0069] Furthermore, the records include performance information associated with the movements. The recording unit 104 is configured to store multiple records relating to the movements of the same person in an associated manner. The device 100 further includes a comparison unit 112 for generating comparison information between a first record (a record including imaging data 10A) and a second record (a record including imaging data 10B) included in the multiple records. The comparison information includes the phase and performance information of each of one or more movements included in the first record and the phase and performance information of each of one or more movements included in the second record. The output unit 114 is configured to output the comparison information.

[0070] Furthermore, the training information generation unit 110 is configured to determine changes in performance information from comparison information, adjust the target phase of one or more actions included in the previous training information based on the changes in performance information, and generate new training information.

[0071] Furthermore, the record is configured to store multiple training information related to the same person's movements in an associated manner. The comparison unit 112 is configured to extract the history of changes in performance information related to movement and the history of changes in the phase of one or more movements from the multiple training information and multiple records, and to generate analysis results. The output unit 114 is configured to output the analysis results.

[0072] The device 100 also includes an operation identification unit 108 for extracting one or more operations from the imaging data 10A and 10B. The operation identification unit 108 extracts one or more operations based on the input specifying each of the one or more operations, or based on predetermined settings.

[0073] As described above, the device 100 can provide the trainee with training information that focuses on the influence of phase on primitives. Furthermore, the device 100 can provide the trainee with updated training information based on changes in performance information before and after training. The device 100 also facilitates the maintenance of the trainee's motivation and the generation of new training menus by providing comparative information. By training according to the training information, the trainee can bring the phase of one or more movements closer to the ideal phase. Furthermore, the trainee can reduce the variability in the phase of each movement. As a result, the trainee can improve their motor performance.

[0074] Figure 4 shows an example of the second function of device 100. Referring to Figure 4, the second function of device 100 will be explained using the example of improving the golf club swing (training target movement) of a trainee. The second function is a function for improving the movement of a trainee who has fallen into a local optimum. A local optimum is a solution that is effective in a specific range but is not optimal when viewed as a whole. For example, suppose a person has some kind of habit in their golf club swing and is unable to correct that habit. In such a case, that person can be said to be stuck in a local optimum.

[0075] First, the device 100 images the training movement performed by the training subject and acquires image data 40A of the training movement. The device 100 also stores the record, including the image data 40A, in the storage area 120. The image data 40A captures the movement of the training subject who is stuck in a local optimum during a golf club swing.

[0076] Next, the device 100 extracts one or more actions from the imaging data 10A. The device adds the information of the extracted one or more actions to the record and saves the record in the storage area 120. In the example in Figure 4, the device 100 extracts actions 41A, 42A, 43A, 44A, 45A, and 46A from the imaging data 40A.

[0077] Next, the device 100 identifies the phase of one or more movements from the imaging data 10A included in the recording. The device adds the identified phase information to the recording and saves the recording in the storage area 120.

[0078] Next, the device 100 generates and outputs training information 410, which includes the target phase. The training information 410 may also include training information 450 of movements extracted from the original movement. In this case, the period 452 of the extracted movements may be redefined as the period of the movement containing only the extracted movements. The device 100 may also define the target phase of the movement of interest as the phase in period 452. In the example in Figure 4, the device 100 extracts movements 43A, 44A, 45A, and 46A, and defines the period 452 of the movement containing movements 43A, 44A, 45A, and 46A. As an example, the device 100 defines the target phase of movement 44A of interest as the phase "0.2π" in period 452. In this case, the trainee can refer to the training information 410A and train to bring the current phase of movement 44A of interest closer to the target phase "0.2π". As another example, training information 450 does not have to include the target phase. In this case, the trainee can focus on training movements that include actions 43A, 44A, 45A, and 46A included in training information 450. The device 100 may also add other movements before or after the original movement, or to one or both. Adding other movements to the original movement changes the phase of each movement included in the original movement.

[0079] Next, the device 100 images the training target movement performed by the training target and acquires imaging data 40B of the training target movement. The imaging data 40B is imaging data obtained after the training target has completed the training. The device 100 also stores the record, including the imaging data 40B, in the storage area 120. In the example in Figure 4, the device 100 extracts actions 41B, 42B, 43B, 44B, 45B, and 46B from the imaging data 40B.

[0080] Next, the device 100 generates training information 410B using the same procedure as when generating training information 410A. Training information 410B includes the current phase information of one or more movements and a new target phase. Training information 410B may also include the period of the entire motion, performance information 400B, the variation in the periods of one or more movements, and one or more images or illustrations of each movement. Training information 410B may also include the information contained in training information 410A. More specifically, training information 410B may include comparison information between the information contained in training information 410B and the information contained in training information 410A. By referring to the comparison information, the trainee can recognize the phase change of their movement of interest, the change in performance information, and whether or not they have moved out of a local optimum. Furthermore, the trainee's supervisor can use the comparison information as a basis for deciding the next target phase. In addition, the device 100 may generate and output feedback information that includes comparison information obtained from the imaging data 10A and 10B before and after training, instead of training information 410B.

[0081] Thus, the device 100 can also be used for training to escape local optima. To this end, the training information generation unit 110 is configured to extract some movements from the imaging data 40A and generate training information 450 based on those movements. This allows the trainee to perform training that focuses on specific movements.

[0082] Figure 5 shows an example of the third function of device 100. Referring to Figure 5, the third function of device 100 will be explained using the example of improving the golf club swing (training target movement) of a trainee. The third function is a function for training movements that include performance routines. A performance routine is an action or thought performed before or after an exercise. A performance routine performed before an exercise is called a pre-performance routine (PPR). A performance routine performed after an exercise is called a post-performance routine (PPR).

[0083] First, the device 100 images the training target movement performed by the training target and acquires imaging data 50A of the training target movement. Unlike imaging data 10A and 40A, imaging data 50A includes a performance routine 58A.

[0084] Next, the device 100 extracts one or more operations from the imaging data 50A. The device adds the information of the extracted one or more operations to the record and saves the record in the storage area 120. In the example in Figure 5, the device 100 extracts operations 58A1, 58A2, 51A, 52A, 53A, 54A, 55A, and 56A from the imaging data 50A. Operations 58A1 and 58A2 are operations included in the performance routine 58A.

[0085] Next, the device 100 identifies phase variations of one or more operations from the imaging data 50A included in the recording. The device adds the identified phase information to the recording and saves the recording in the storage area 120. In the example in Figure 5, the device 100 identifies the phase variations 558A1, 558A2, 552A, 553A, and 554A for each of the operations 58A1, 58A2, 52A, 53A, and 54A. The device 100 also identifies the current phases 568A1, 568A2, 562A, 563A, and 564A for each phase.

[0086] Next, the device 100 generates and outputs training information 550A. The training information 550A includes phase variations 558A1, 558A2, 552A, 553A, 554A of one or more movements, and the current phase 568A1, 568A2, 562A, 563A, 564A of each of the one or more movements. The training information 550A may also include the period 570A of the entire movement including the performance routine, performance information, and one or more images or illustrations of each movement. Furthermore, the training information 550A may include the target phase of each of the one or more movements. The current phase may be identified from the most recent (last captured) imaging data. The phase variations of one or more movements may include the phase of one movement in all or some of the imaging data captured to date. The training information 550A displays each timing from the start to the end of the entire movement including the performance routine as a phase rather than time.

[0087] The trainee may, while referring to training information 550A, train to reduce the variation in the phases of each of the actions of interest 58A1, 58A2, 52A, 53A, and 54A. Furthermore, if training information 550A includes the target phases for each action, the trainee may train to bring the phases of each of the actions of interest 58A1, 58A2, 52A, 53A, and 54A closer to their respective target phases.

[0088] Next, the device 100 images the training target movement performed by the training target and acquires image data 50B of the training target movement. Image data 40B is image data obtained after the training target has completed the training. The device 100 also saves the record, including the image data 50B, in the storage area 120. In the example in Figure 5, the device 100 extracts actions 58B1, 58B2, 51B, 52B, 53B, 54B, 55B, and 56B from the image data 50B.

[0089] Next, the device 100 generates training information 550B using the same procedure as when generating training information 550A. Training information 550B includes phase variations 558B1, 558B2, 552B, 553B, 554B of one or more movements, and the current phases 568B1, 568B2, 562B, 563B, 564B of each of the one or more movements. Training information 550B may also include the period 570B of the entire movement including the performance routine, performance information, and one or more images or illustrations of each movement.

[0090] Training information 550B may include information contained in training information 550A. More specifically, training information 550B may include comparative information between the information contained in training information 550B and the information contained in training information 150A. The trainee can recognize the effect of the training by referring to the comparative information. In the example in Figure 5, it can be seen that the variation in the phase of each movement has decreased before and after training. Furthermore, the device 100 may generate and output feedback information that includes comparative information obtained from the imaging data 50A and 50B before and after training, instead of training information 550B.

[0091] As explained with reference to Figure 5, imaging motion includes imaging motion and performance routines. The period includes the period in which motion and performance routines are executed. The phase identification unit 106 is configured to identify the phase of each of one or more movements based on the period including the execution period of the performance routine and motion. By utilizing training information based on motion including the performance routine and main motion, the trainee can bring the phase difference between each movement in the performance routine and each movement in the main motion closer to a constant interval. Furthermore, the trainee can reduce the variability in the phase of each movement. As a result, the trainee can achieve high performance in the main motion. Here, "main motion" refers to motion other than the performance routine. To explain using a baseball bat swing as an example, practice swings are the performance routine, and the swing aimed at actually hitting the ball is the main motion.

[0092] <Device Processing Procedure> Figure 6 shows an example of the processing procedure of the device 100. The processor 201 may read a program for performing the processing shown in Figure 6 from the storage 203 into the memory 202 and execute the program. Part or all of the processing can also be implemented as a combination of circuit elements configured to perform the processing. The following steps may be performed in any order.

[0093] In step S610, the device 100 images the movements of the person being trained. The device 100 may image the movements of the person being trained multiple times. In step S620, the device 100 generates a record from one or more image data and stores it in the memory area 120.

[0094] In step S630, the device 100 identifies the phase of each of one or more operations included in the recorded or imaging data. The device 100 may accept selection input for one or more operations via the UI. In this case, the device 100 identifies the phase of each of the selected one or more operations. The device 100 adds the identified phases to the recording.

[0095] In step S640, the device 100 determines whether or not it has received an input for a training information generation instruction. The device 100 may receive an input for a training information generation instruction via the UI. The training information generation instruction may include an instruction for selecting the type of training information. For example, such an instruction may include the selection of normal training information, training information for escaping local minima, and training information for movements including performance routines. If the device 100 determines that it has received an input for a training information generation instruction (YES in step S640), it transfers control to step S650. Otherwise (NO in step S640), the device 100 transfers control to step S660.

[0096] In step S650, the device 100 generates training information. As described with reference to Figures 3 to 5, the device 100 may generate normal training information, training information for exiting local minima, and training information for movements including performance routines.

[0097] In step S660, the device 100 determines whether or not it has received an input for a comparison information generation instruction. The device 100 may receive an input for a comparison information generation instruction via the UI. If the device 100 determines that it has received an input for a comparison information generation instruction (YES in step S660), it transfers control to step S670. Otherwise (NO in step S660), the device 100 transfers control to step S680.

[0098] In step S670, the device 100 generates comparison information. If there is an input instruction to generate training information, the device 100 may include the comparison information in the training information. If there is no input instruction to generate training information, the device 100 may generate feedback information that includes the comparison information. The feedback information may include, for example, the differences in phase and performance information of one or more operations before and after training.

[0099] In step S680, the device 100 outputs various types of generated information. The device 100 may output one or more of the following: training information, training information including comparison information, feedback information including comparison information, and analysis information.

[0100] As explained with reference to Figure 6, the device 100 can perform the following by executing a program: to image motion; to generate and store a record including imaging data related to the motion and the period in which the motion is performed; to identify the phase in the period of each of the one or more actions included in the motion from the record; to generate training information including the phase and target phase of each of the one or more actions; and to output the training information.

[0101] <Summary> As explained above, the device 100 can provide trainees with training information that focuses on the influence of phase on primitives. Furthermore, the device 100 can provide trainees with updated training information based on changes in performance information before and after training. Additionally, by providing comparative information, the device 100 facilitates the maintenance of trainees' motivation and the generation of new training menus.

[0102] Furthermore, the device 100 can provide training information to escape local optima. This allows the trainee to escape local optima by referring to this training information during training. For example, the trainee can correct bad habits in their golf swing by referring to training information to escape local optima.

[0103] Furthermore, the device 100 can provide training information based on exercise, including a performance routine. By training with reference to this training information, the trainee can bring the phase difference between each movement in the performance routine and each movement in the main exercise closer to a constant interval. As a result, the trainee can achieve high performance in the main exercise.

[0104] The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications are intended to be equivalent to the claims. Furthermore, the disclosures described in the embodiments and each variation are intended to be implemented, as far as possible, individually or in combination. [Explanation of symbols]

[0105] 10A,10B,40A,40B,50A,50B Imaging data, 11A,11B,12A,12B,13A,13B,14A,14B,15A,15B,16A,16B,41A,41B,42A,42B,43A,43B,44A,44B,45A,45B,46A,46B,51A,52A,53A,54A,55A,56A,58A1,58A2,51B,52B,53B,54B,55B,56B,58B1,58B2 Operation, 58A Performance routine, 100 Device, 102 Imaging unit, 104 Recording unit, 106 Phase identification unit, 108 Operation identification unit, 110 Training information generation unit, 112 comparison unit, 114 output unit, 120 memory area, 122 recording DB, 124 training information DB, 150A, 150B, 410, 410A, 410B, 450, 550A, 550B training information, 152A, 152B, 452, 570A, 570B period, 154A, 154B phase, 156A, 156B, 568A1, 568A2, 568B2, 568B1, 562A, 562B, 563A, 563B, 564A, 564B target phase, 201 processor, 202 memory, 203 storage, 204 external device IF, 205 input IF, 206 output IF, 207 communication IF, 208 bus.

Claims

1. An imaging unit for capturing motion, A recording unit that generates and stores a record including imaging data relating to the motion and the period during which the motion is performed. A phase identification unit for identifying the phase during the period in each of the one or more movements included in the motion, from the aforementioned record, A training information generation unit for generating training information including the phase and target phase of each of the one or more operations, A device comprising an output unit that outputs the aforementioned training information.

2. The aforementioned record includes performance information associated with the exercise, The recording unit is configured to store multiple records relating to the exercise of the same person in an associated manner. The system further comprises a comparison unit for generating comparison information between the first record and the second record included in the plurality of records, The comparison information includes the phase and performance information of each of the one or more operations included in the first record, and the phase and performance information of each of the one or more operations included in the second record. The apparatus according to claim 1, wherein the output unit is configured to output the comparison information.

3. The aforementioned training information generation unit, Based on the comparison information, the change in the performance information is determined. The apparatus according to claim 2, configured to generate new training information by adjusting the target phase of each of the one or more operations included in the previous training information based on the changes in the performance information.

4. The record is configured to store multiple pieces of training information relating to the exercise of the same person in an associated manner. The comparison unit is configured to extract from the plurality of training information and the plurality of records the history of changes in the performance information related to the movement and the history of changes in the phase of each of the one or more movements, and to generate analysis results. The apparatus according to claim 2, wherein the output unit is configured to output the analysis results.

5. The system further comprises an action identification unit for extracting one or more actions from the aforementioned imaging data, The apparatus according to any one of claims 1 to 4, wherein the operation identification unit extracts one or more operations based on receiving an input specifying each of the one or more operations, or based on a predetermined setting.

6. The apparatus according to any one of claims 1 to 4, wherein the training information generation unit is configured to extract some actions from the imaging data and generate the training information based on the some actions.

7. Image capture of the aforementioned motion includes image capture of the aforementioned motion and performance routine, The aforementioned period is the period in which the exercise and the performance routine are executed. The apparatus according to any one of claims 1 to 4, wherein the phase identification unit is configured to identify the phase of each of the one or more operations based on a period including the execution period of the performance routine and the motion.

8. A method by which the device performs Imaging motion, To generate and store a record including imaging data relating to the motion and the period during which the motion is performed, From the aforementioned record, identify the phase in the period for each of the one or more movements included in the motion, To generate training information including the phase and target phase of each of the one or more operations, A method comprising outputting the aforementioned training information.

9. A program that the device executes, Imaging motion, To generate and store a record including imaging data relating to the motion and the period during which the motion is performed, From the aforementioned record, identify the phase in the period for each of the one or more movements included in the motion, To generate training information including the phase and target phase of each of the one or more operations, A program that causes the device to output the aforementioned training information.

Images