Learning data generation system, and learning system

Abstract

To provide a learning data generating system and a learning system that can obtain a high learning effect.SOLUTION: A learning data generating system comprises: action data acquisition means of acquiring action data as information showing the action of a learner; corrective data acquisition means 322 of acquiring corrective data showing a corrective action for correcting an action shown by the action data; and learning data generation means 32 of generating learning data based on the action data acquired by the action data acquisition means and the corrective data acquired by the corrective data acquisition means 322.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a learning data generation system and a learning system for assisting the learning of operations.

Background Art

[0002] Conventionally, various systems for assisting the acquisition of operations have been provided. For example, Patent Document 1 discloses a system that expresses the physical operation of a learner in animation so that the difference from a model operation can be understood.

Prior Art Documents

Patent Documents

[0003] <了

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the above conventional system, since the physical operation itself of the modeler (skilled person) is used as the model operation, there are cases where the model operation does not suit the learner and a high learning effect cannot be obtained.

[0005] Therefore, in view of such circumstances, an object of the present invention is to provide a learning data generation system and a learning system that can obtain a high learning effect.

Means for Solving the Problems

[0006] The learning data generation system of the present invention includes operation data acquisition means for acquiring operation data, which is information representing the operations of a learner, correction data acquisition means for acquiring correction data, which represents correction operations for correcting the operations indicated by the operation data, The system includes a learning data generation means that generates learning data based on the operation data acquired by the operation data acquisition means and the correction data acquired by the correction data acquisition means.

[0007] In the learning data generation system with the above configuration, learning data is generated by modifying motion data that represents the learner's own actions. Therefore, learning data representing the actions that the learner aspires to emulate can be generated in a way that suits the learner's individuality, thereby achieving a high learning effect.

[0008] In the learning data generation system of the present invention, The operation data acquisition means includes a correction location generation means that generates correction location information indicating the correction locations of the operation data acquired by the operation data acquisition means, The correction data acquisition means acquires correction data for the correction location indicated by the correction location information, The learning data generation means may be configured to generate the learning data by combining the modified data with the modified locations indicated by the modified location information in the operation data.

[0009] In this way, training data can be generated using corrected data that is appropriate for the parts of the operation data that need to be modified, making it possible to generate training data that is more likely to yield high learning effectiveness.

[0010] In the learning data generation system and learning system of the present invention, The motion data acquisition means acquires the motion data based on the output of the motion sensor attached to the learner. The modification location generation means may generate the modification location information based on the output of the motion sensor.

[0011] In this way, training data can be generated by appropriately identifying and correcting aspects of the learner's actions that need modification, tailored to the individual learner. This makes it possible to generate training data that is more likely to yield high learning effectiveness.

[0012] The learning system of the present invention is A means for acquiring motion data, which is information representing the actions of the learner, Correction data acquisition means for acquiring correction data representing a correction operation to correct the operation indicated by the aforementioned operation data, A learning data generation means generates learning data based on the operation data acquired by the operation data acquisition means and the correction data acquired by the correction data acquisition means, The system may also include a display processing means that displays the learning data and the current action data regarding the learner's current actions in a comparable manner.

[0013] According to the learning system configured as described above, by modifying the motion data representing the learner's own actions, learning data representing the actions the learner aspires to is generated. This generates learning data that is suitable for the learner, and furthermore, because the learner can learn actions by comparing the learning data representing the exemplary actions with their own current actions, a high learning effect can be achieved. [Effects of the Invention]

[0014] As described above, the learning data generation system and learning system of the present invention can achieve the excellent effect of obtaining a high learning effect. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a block diagram of a learning data generation system according to one embodiment of the present invention. [Figure 2] Figure 2 is an explanatory diagram of the operation screen of the learning data generation system according to the embodiment of the same. [Figure 3] Figure 3 is an explanatory diagram of the operation screen of the learning data generation system according to the same embodiment, and is an explanatory diagram of the display modes of the synchronization model and the learning model. [Modes for carrying out the invention]

[0016] Hereinafter, a learning system according to an embodiment of the present invention will be described with reference to the accompanying drawings. In the learning system, learning data representing exemplary actions is generated based on the actions of the learner himself / herself, and the learner can learn the actions he / she wants to acquire while checking this learning data and his / her own movements.

[0017] As shown in FIG. 1, the learning system 1 of the present embodiment includes an action measurement device 2 capable of measuring the action state of a learner, a learning device 3 that generates learning data using information on the action state of the learner measured by the action measurement device 2, and an output device 4 on which a 3D model (hereinafter referred to as a synchronization model) that operates in synchronization with the action state of the learner and a 3D model (hereinafter referred to as a learning model) that operates in accordance with the action state indicated by the learning data are displayed.

[0018] The action measurement device 2 is a means for acquiring action data, which is information representing the actions of the learner.

[0019] The action measurement device 2 may be configured to create action data from, for example, images or videos, or may be configured to have a motion sensor used in a state attached to the learner.

[0020] The motion sensor is configured to be able to acquire information such as position change information, acceleration, angular velocity, and orientation by moving together with the learner. The action measurement device 2 can be configured using, for example, optical markers, sensor devices such as acceleration sensors and angular velocity sensors, GPS, etc.

[0021] The learning device 3 includes an action data receiving means 30 that receives action data indicating the learner's action state measured by the action measurement device 2 as action measurement information, a synchronization output means 31 that outputs the action measurement information received by the action data receiving means 30 as synchronization action information for operating the synchronization model, a learning data generation means 32 that executes a process to generate learning data, a display processing means 33 that executes a process to display and operate the synchronization model and the learning model on the output device 4, and a playback operation means 34 for performing an operation to play back the learning data.

[0022] The learning device 3 is comprised of a computer. The computer has a control device C1 that performs information processing and a storage device C2 that stores information. The program that causes the computer (control device C1) to function as an operation data receiving means 30, a synchronization output means 31, a learning data generation means 32, and a display processing means 33 only needs to be stored in the storage device C2.

[0023] Furthermore, in the learning device 3, the operation data receiving means 30 and the learning data generation means 32 may be configured as a learning data generation system for generating learning data.

[0024] The motion measurement information received by the motion data receiving means 30 from the motion measurement device 2 is acquired by the synchronization output means 31, the base data acquisition means 320 (described later) and the correction data acquisition means 322 of the learning data generation means 32.

[0025] The synchronous output means 31 is configured to repeatedly perform the following processes while outputting (displaying) the synchronous model to the output device 4: receiving motion measurement information from the motion measurement device 2 and transmitting the received motion measurement information to the synchronous display means.

[0026] The synchronization operation information output by the synchronization output means 31 is used as an operation command to operate the synchronization model.

[0027] The learning data generation means 32 includes a base data acquisition means 320 that acquires motion measurement information when the learner is performing an action they want to learn (learning action) and stores this motion measurement information as base data; a modification location generation means 321 that identifies modification locations in the base data that can be modified and generates the identification result as modification location identification information; a modification data acquisition means 322 that stores motion measurement information acquired from the motion data receiving means 30 as modification data in order to modify the motion state of the modification locations identified by the modification location generation means 321 in the base data; and a generation processing execution means 323 that generates learning data in which the motion state of the base data has been modified based on the modification location identification information and the modification data.

[0028] The motion measurement information acquired as base data by the base data acquisition means 320 is information that shows a series of motion states from when the learner starts performing the learning motion they want to acquire until when they finish performing the learning motion, and it shows the current series of motion states of the learner before any modifications are made.

[0029] The modification location generation means 321 includes a modification location identification means 3210 that identifies modification locations and generates modification location identification information indicating the modification locations, and a modification location association means 3211 that associates the modification location identification information generated by the modification location identification means 3210 with base data.

[0030] The modification location identification means 3210 uses operation feature information that indicates the characteristics of the operation state of the modification location to identify the location in the base data as a modification location where the operation state matches or nearly matches the characteristics of the operation state indicated by the operation feature information.

[0031] The characteristics of a movement state can be defined by the body part and the static movement information (position, etc.) and dynamic movement information (velocity, acceleration, rotational speed, etc.) of that part.

[0032] For example, if the learned motion is a golf swing, motion feature information indicating the characteristics of the motion state at address, motion feature information indicating the characteristics of the motion state at the top of the swing, motion feature information indicating the characteristics of the motion state at impact, and motion feature information indicating the characteristics of the motion state at the finish would be used.

[0033] In this case, the motion characteristic information indicating the characteristics of the movement state at address, the motion characteristic information indicating the characteristics of the movement state at the top of the swing, and the motion characteristic information indicating the characteristics of the movement state at the finish can be represented by information indicating the rotational speed of the learner's body and hips.

[0034] Furthermore, motion characteristic information, which describes the characteristics of the motion state at the time of impact, can be represented by information showing the angular velocity and acceleration of the learner's hand.

[0035] The information used to identify the modified area should include details such as the playback time and number of frames of the base data corresponding to the modified area.

[0036] The modification location association means 3211 is configured to store the modification location identification information in the storage device C2 in association with the base data.

[0037] The correction data acquisition means 322 includes a correction target acquisition means 3220 that selects and acquires base data to be corrected in terms of operating state, a correction location selection means 3221 that selects correction locations that are set (specified) for the base data acquired by the correction target acquisition means 3220, and a correction acquisition processing means 3222 that acquires operation measurement information from the operation data receiving means 30 to be used as correction data for correcting the operating state of the correction locations selected by the correction location selection means 3221.

[0038] The modification target acquisition means 3220 is configured to select the target for modification of the operating state from the base data stored in the memory device C2, and to acquire the selected base data.

[0039] As shown in Figure 2, the modification location selection means 3221 is comprised of a selection operation button B1 displayed on the operation screen U.

[0040] The number of selection buttons B1 displayed on the operation screen U will be the same as the number of correction locations set in the base data.

[0041] Furthermore, the modification location selection means 3221 is configured to output an operation display B2, which indicates the operational status of the modification location to be modified, to the operation screen U, together with the selection operation button B1.

[0042] The operation display B2 is configured to represent the operation status of the modified area using text and icons. If the learned operation is a golf swing, for example, it would consist of text such as address, top, impact, and finish.

[0043] As shown in Figure 1, the correction acquisition processing means 3222 is configured to perform the following processes when a selection operation button is selected: acquiring operation measurement information from the operation data receiving means 30; converting the operation measurement information acquired from the operation data receiving means 30 into correction data for correcting the operation state of the correction location corresponding to the selection operation button (correction location identification information); and storing the correction data in association with the correction location identification information.

[0044] Correction data is operational measurement information used to correct the operational state of the correction location indicated by the correction location identification information in the base data.

[0045] Since the correction acquisition processing means 3222 in this embodiment is configured to acquire the learner's operational state at the time the correction location selection means 3221 is selected, the operational measurement information will indicate the learner's posture while in motion or while stationary as the operational state.

[0046] The generation processing execution means 323 includes a processing target acquisition means 3230 that acquires base data to be used for generating training data, a processing location confirmation means 3231 that confirms correction location identification information associated with the base data selected by the processing target acquisition means 3230, a correction data confirmation means 3232 that confirms whether or not there is correction data associated with the correction location identification information confirmed by the processing location confirmation means 3231, and a correction processing execution means 3233 that, if the correction data confirmation means 3232 determines that there is correction data associated with the correction location identification information, synthesizes the correction data to the base data at the location indicated by the correction location identification information.

[0047] When the processing target acquisition means 3230 acquires base data to be used for generating learning data, the processing target acquisition means 3230 acquires base data to which the correction location identification information and the correction data are associated. In this embodiment, the learning device 3 is configured so that the base data is stored in the storage device C2, so the processing target acquisition means 3230 is configured to acquire the base data from the storage device C2.

[0048] The correction data verification means 3232, when multiple correction location identification pieces are associated with the base data, performs a process to check for the presence or absence of associated correction data for all base data.

[0049] The correction processing execution means 3233 is configured to perform the following actions when the processing location confirmation means 3231 determines that there is correction data associated with the correction location identification information: to perform the action of acquiring such correction location identification information and correction data, and to perform the action of correcting the base data based on the correction location identification information and correction data.

[0050] The correction processing execution means 3233, in the process of correcting the base data based on the correction location identification information and the correction data, synthesizes the correction data with the correction location indicated by the correction location identification information in the base data.

[0051] The correction processing execution means 3233 may be configured to synthesize the correction data into the base data by replacing the operating state of the correction location with the operating state indicated by the correction data, or it may be configured to synthesize the correction data into the base data by changing the operating state of the correction location based on the operating state indicated by the correction data.

[0052] Furthermore, the correction processing execution means 3233 is configured to correct (compensate) the operating state before and after the correction point to match the operating state of the corrected point after the correction.

[0053] More specifically, if the location of the base data is represented by the number of frames, the correction processing execution means 3233 sets the correction range of the base data by selecting the correction location and a frame after or before the correction location. The correction processing execution means 3233 may also select another correction location before or after the correction location used as the basis for correction as a frame after or before the above correction location.

[0054] Next, the correction processing execution means 3233 obtains the frame number and information representing the orientation of the operating state (specifically, quaternions or angles of the X, Y, and Z axes) for the correction location and the location selected in the frame after or before the correction location (hereinafter referred to as the selected location).

[0055] Then, the correction processing execution means 3233 calculates the difference between the operating state posture of the correction location and the operating state posture of the selected location, and the difference between the number of frames of the correction location and the number of frames of the selected location. Based on this difference information, it derives the amount of change in posture per frame from the frame of the correction location to the frame of the selected location, and replaces (corrects) the operating state posture in each frame from the correction location to the selected location based on the derived amount of change in posture.

[0056] The modification processing execution means 3233 in this embodiment is configured to execute a process to generate training data when the execution button B3 displayed on the operation screen U is pressed (see Figure 2).

[0057] The display processing means 33 includes a synchronization display means 330 that performs processing for displaying the synchronization model on the output device 4 while it is running, and a learning display means 331 that performs processing for displaying the learning model on the output device 4 while it is running.

[0058] The synchronization display means 330 includes a synchronization model display means 3300 that performs the process of displaying the synchronization model on the output device 4, a synchronization operation receiving means 3301 that receives synchronization operation information output from the synchronization output means 31, and a synchronization operation reflecting means 3302 that operates the synchronization model based on the operation state indicated by the synchronization operation information received by the synchronization operation receiving means 3301.

[0059] The learning display means 331 includes a learning model display means 3310 that performs the process of displaying the learning model on the output device 4, a learning data acquisition means 3311 that acquires learning data, and a learning operation reflection means 3312 that operates the learning model based on the learning data acquired by the learning data acquisition means 3311.

[0060] The learning model display means 3310 is configured to display the learning model M2 on the output device 4 (operation screen U) simultaneously with the synchronization model M1 (see Figure 3).

[0061] Furthermore, the learning model display means 3310 is configured to display the learning model M2 on the output device 4, superimposed on the synchronization model M1.

[0062] Thus, the display processing means 33 is configured to display synchronization operation information on the output device 4 in a state that can be compared with the learning data as current operation data representing the learner's current actions.

[0063] The learning data playback means 34 consists of a playback operation button (not shown) displayed on the operation screen. When pressed, the learning data playback means 34 is configured to send a command to the learning action reflection means 3302 to execute a process to operate the learning model based on the learning data.

[0064] Output device 4 can consist of a monitor or similar device.

[0065] The configuration of the learning system 1 according to this embodiment is as described above. Next, the method of using the learning system 1 will be explained.

[0066] In the learning system 1 of this embodiment, learners can check their own movements, create learning data, and learn (practice) movements using the learning data.

[0067] When the motion measurement device 2 is measuring the learner's movements and the synchronization model display processing means 33 is displaying the synchronization model on the output device 4, the synchronization output means 31 outputs the motion measurement information acquired from the motion data receiving means 30 as synchronization motion information, and this synchronization motion information is received by the synchronization motion receiving means 3301.

[0068] Then, the synchronization operation reflection means 3302 operates the synchronization model based on the synchronization operation information received by the synchronization operation receiving means 3301.

[0069] Therefore, learners can learn their movements while confirming their own actions by viewing the synchronization model displayed on the output device 4, or by recognizing the difference between the training data and their own movements as information such as sound or vibration.

[0070] When creating training data, the base data acquisition means 320 generates base data, the correction location identification means 321 generates correction location identification information, the correction data acquisition means 322 acquires correction data, and the generation processing execution means 323 generates training data based on the base data, correction location identification information, and correction data.

[0071] The base data acquisition means 320 continuously acquires motion measurement information from the motion data receiving means 30 from the time the learner starts performing the learning motion they wish to acquire until they finish performing the learning motion, and stores this motion measurement information as base data in the storage device C2.

[0072] The modification location generation means 321 executes a process to generate modification location identification information based on the base data generated by the base data acquisition means 320, and stores the generated modification location identification information in the storage device C2 in association with the base data.

[0073] If the target acquisition means 3220 is acquiring base data, the correction data acquisition means 322 can acquire correction data for correcting the operating state of the correction location by operating the correction location selection means 3221.

[0074] The modification location selection means 3221 of this embodiment has a selection operation button and an operation display B2 that is displayed together with the selection operation button. Therefore, a learner looks at the operation display B2, takes the operation state shown in the operation display B2, and while looking at the synchronization model M1, adjusts their posture so that the operation state is correct, and then another person operates the selection operation button to acquire the modification data (see Figure 2).

[0075] When acquiring corrected data, if the learner is unable to achieve the correct behavior on their own, the educator will correct the learner's behavior before acquiring the corrected data.

[0076] If the base data has multiple pieces of information identifying correction locations associated with it, correction data may be obtained for each of the multiple pieces of information identifying correction locations, or correction data may be obtained only for the pieces of information identifying correction locations that require correction.

[0077] Once the acquisition of the corrected data is complete, the generation processing execution means 323 generates training data based on the base data, the information identifying the corrected parts, and the corrected data.

[0078] When a learner operates the learning data playback means 34, the learning data playback means 34 sends a command to the learning operation reflection means 3302 to execute a process to operate the learning model based on the learning data, and the learning model displayed on the output device 4 operates according to the learning data (the operating state indicated by the learning data).

[0079] Therefore, learners can learn the operation by observing the operation of the learning model displayed on the output device 4 and using that operation as a model.

[0080] Furthermore, if the learning model is displayed on the output device 4 overlaid on the synchronized model by the learning model display means 3310, the learner can learn their actions by comparing their current actions with the actions of the learning data.

[0081] As described above, the learning system 1 according to this embodiment is configured to generate learning data by modifying motion data that represents the learner's own actions. Therefore, learning data representing the actions that the learner aspires to emulate can be generated in accordance with the learner's individuality (such as their habits in using their body), thereby enabling high learning effectiveness.

[0082] Furthermore, the learning system can generate learning data using modified data appropriate to the areas of the operational data that need correction, thanks to the learning data generation system. This allows for the generation of learning data that is more likely to yield high learning effectiveness.

[0083] Furthermore, in the learning system of this embodiment, since both the base data and the modified data are generated from the learner's actions by the learning data generation system, learning data can be generated by appropriately identifying and correcting parts of the learner's actions that need to be modified according to the learner, thus enabling the generation of learning data that is more likely to yield high learning effectiveness.

[0084] Furthermore, since the system is configured to acquire and use information representing the learner's posture as correction data, it becomes easier to acquire correction data and obtain learning data suitable for learning when learning movements that include a static state within a series of movements, such as a golf swing.

[0085] Furthermore, since the learning data of this embodiment can also be modified to match the operating state of the modified part, including the movements before and after the modified part, it is possible to generate learning data that learners will find less unnatural and will be able to acquire more easily.

[0086] As described above, the learning system can generate learning data suitable for the learner using a learning data quiescence system. Furthermore, since the system is configured to allow learners to learn actions by comparing their current actions with learning data representing exemplary actions, a higher learning effect can be achieved.

[0087] It should be noted that the learning data generation system and learning system according to the present invention are not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.

[0088] In the description of the learning system in the above embodiment, a golf swing was used as an example of a learning action, but the learning system and the learning data generation system can also be used for actions other than a golf swing.

[0089] In the description of the learning data generation system of the above embodiment, the learning data generation means 32 was described as being included in the learning device 3. However, for example, the learning data generation means 32 can also be configured as an independent learning data generation system.

[0090] Although not specifically mentioned in the description of the modification location identification means 3210 in the above embodiment, the modification location identification means 3210 may be configured to allow manual identification of modification locations in the base data.

[0091] In the above embodiment, the correction data acquisition means 332 (correction acquisition processing means 3222) was configured to acquire correction data from the learner's actions, but it may also be configured to acquire correction data obtained from the actions of someone other than the learner.

[0092] In the above embodiment, the correction acquisition processing means 3222 was configured to acquire the learner's operating state at the time the correction location selection means 3221 was selected, but the configuration is not limited to this. The correction acquisition processing means 3222 may be configured, for example, to continue acquiring the learner's operating state for a predetermined period of time from the time the correction location selection means 3221 was selected.

[0093] In this case, the motion measurement information will represent the learner's movements during the action as an action state. [Explanation of Symbols]

[0094] 1...Learning system, 2...Motion measurement device, 3...Learning device, 4...Output device, 30...Motion data receiving means, 31...Synchronized output means, 32...Learning data generation means, 33...Display processing means, 34...Playback operation means, 320...Base data acquisition means, 321...Modification location generation means, 322...Modification data acquisition means, 323...Generation processing execution means, 330...Synchronization display means, 331...Learning display means, 332...Modification data acquisition means, 3210...Modification location identification means, 3211...Modification location association means, 3220...Modification target acquisition means, 3221...Modification location selection B1...Means, 3222...Modification acquisition processing means, 3230...Processing target acquisition means, 3231...Processing location confirmation means, 3232...Modification data confirmation means, 3233...Modification processing execution means, 3300...Display means for synchronization model, 3301...Synchronization operation reception means, 3302...Synchronization operation reflection means, 3310...Display means for learning model, 3311...Learning operation reception means, 3312...Learning operation reflection means, B1...Selection operation button, B2...Operation display, B3...Execute button, C1...Control device, C2...Storage device, M1...Synchronization model, M2...Learning model, U...Operation screen

Claims

1. A motion data acquisition means that acquires motion data, which is information representing the learner's actions, as base data having multiple frames, A correction location generation means for identifying the time point of a correction location in the aforementioned base data that allows for modification of the operation, based on the number of frames, Correction data acquisition means acquires correction data representing a correction action to correct the action indicated by the correction location identified by the correction location generation means from the base data, based on a model action. The system comprises a learning data generation means that generates learning data based on the operation data acquired by the operation data acquisition means and the correction data acquired by the correction data acquisition means, The learning data generation means is configured to perform the following processes: modify the operation indicated by the modified portion in the base data by replacing it with the modified data or by making changes based on the modified data; and modify the operation state before and after the modified portion to match the operation state of the modified portion after the modification. A system for generating training data.

2. The motion data acquisition means acquires the motion data based on the output of the motion sensor attached to the learner. The modification location generation means generates modification location identification information indicating the modification location based on the output of the motion sensor. The learning data generation system according to claim 1.

3. A motion measurement device capable of measuring the learner's movement state, Motion data acquisition means that acquires motion data, which is information representing the learner's movements measured by the motion measurement device, as base data having multiple frames, A correction location generation means for identifying the time point of a correction location in the aforementioned base data that allows for modification of the operation, based on the number of frames, Correction data acquisition means acquires correction data representing a correction action to correct the action indicated by the correction location identified by the correction location generation means from the base data, based on a model action. A learning data generation means generates learning data based on the operation data acquired by the operation data acquisition means and the correction data acquired by the correction data acquisition means, The system includes a display processing means that displays the learning data and the current action data indicating the learner's current actions, acquired by the action measurement device, in a comparable manner. The learning data generation means is configured to perform the following processes: modify the operation indicated by the modified portion in the base data by replacing it with the modified data or by making changes based on the modified data; and modify the operation state before and after the modified portion to match the operation state of the modified portion after the modification. Learning system.

Images