Learning support device, learning support method, and computer program
The learning support device addresses the challenge of imitating complex movements by constructing a low-dimensional map and providing visual feedback, enhancing learner motivation through precise cursor adjustment.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- NIPPON TELEGRAPH & TELEPHONE CORP
- Filing Date
- 2023-01-31
- Publication Date
- 2026-06-23
AI Technical Summary
Learners find it difficult to accurately imitate complex movements due to the high dimensionality of movement information, leading to a decrease in learning motivation as they struggle to discover the correspondence between motor information and cursor movement in conventional methods.
A learning support device that constructs a low-dimensional map by combining multidimensional information datasets of the learner and the imitation target, acquires real-time data, mixes and maps it onto a Self-Organizing Map (SOM) with a mixing ratio parameter, and provides visual feedback.
Maintains a high sense of task accomplishment and prevents a decline in learning motivation by reflecting the learner's motion information in stages, allowing for precise adjustment of cursor movement relative to the imitation target.
Smart Images
Figure 0007878468000024 
Figure 0007878468000025 
Figure 0007878468000026
Abstract
Description
[Technical Field]
[0001] This invention relates to a learning support device, a learning support method, and a computer program. [Background technology]
[0002] In sports and musical instrument performance, the method of learning by imitating the movements of experts or other exemplary individuals is known for acquiring correct form. However, when the target movement consists of complex movements (movements with many degrees of freedom), it is necessary to recognize and correct the differences between the learner's movements and those of the imitation target based on the movement information of each body part of both the learner and the imitation target (e.g., hip joint angle, knee joint angle, etc.). Furthermore, the more degrees of freedom there are in the movement, the more movement information there is for each body part, making it more difficult to recognize the differences between the learner and the imitation target. For these reasons, it has sometimes been difficult for the learner to accurately imitate the movements of the imitation target.
[0003] For example, a method has been proposed that uses a Self-Organization Map (SOM) to map the motion information of the learning subject and the object being imitated to arbitrary positions on a two-dimensional map, thereby providing visual feedback (FB) on the map showing the difference between the actions of the learning subject and the actions of the object being imitated (see Non-Patent Document 1).
[0004] The system disclosed in Non-Patent Document 1 specifically presents the actions of the learning subject as a cursor on a map and the actions of the target to be imitated as a target trajectory. The learning subject can learn the actions of the target to be imitated by adjusting its own actions and manipulating the cursor to follow the target trajectory (reducing the difference between the cursor and the target trajectory). [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Yokota, H., Naito, M., Mizuno, N., & Ohshima, S. (2020). Framework for visual-feedback training based on a modified self-organizing map to imitate complex motion. Proceedings of the Institution of Mechanical Engineers, Part P: Journal of Sports Engineering and Technology, 234(1), 49-58. [Overview of the project] [Problems that the invention aims to solve]
[0006] However, with conventional methods, it is difficult for learners to immediately discover the correspondence between motor information and cursor movement, and continuous trial and error is required to bring the cursor closer to the target trajectory, which can lead to a decrease in learning motivation.
[0007] This invention has been made in view of the above circumstances, and aims to provide a learning support device, a learning support method, and a computer program that maintain a high sense of task accomplishment for the learner and suppress a decline in motivation to learn through action imitation. [Means for solving the problem]
[0008] A learning support device according to a first aspect of the present invention includes: a low-dimensional map construction unit that generates a multidimensional information dataset by combining a multidimensional information dataset to be imitated and a multidimensional information dataset of a learning subject, and constructs a map with reduced dimensions from the multidimensional information dataset; a first acquisition unit that acquires real-time learning subject multidimensional information in a time series; a second acquisition unit that acquires the multidimensional information to be imitated corresponding to the real-time learning subject multidimensional information at a predetermined time from the multidimensional information dataset to be imitated after preprocessing in the low-dimensionality reduction; a mixing unit that generates mixed multidimensional information by mixing the real-time learning subject multidimensional information and the multidimensional information to be imitated corresponding to the real-time learning subject multidimensional information based on a mixing ratio parameter; a mapping unit that generates coordinate data by mapping the mixed multidimensional information and the preprocessed multidimensional information dataset to be imitated onto the map; and a drawing unit that draws the coordinate data as an image.
[0009] A learning support method according to a second aspect of the present invention includes generating a multidimensional information dataset by combining a multidimensional information dataset to be imitated and a multidimensional information dataset of a learning subject, constructing a map with reduced dimensionality from the multidimensional information dataset, acquiring real-time learning subject multidimensional information in a time series, acquiring the multidimensional information to be imitated corresponding to the real-time learning subject multidimensional information at a predetermined time from the multidimensional information dataset to be imitated after preprocessing in the reduction of dimensionality, generating mixed multidimensional information by mixing the real-time learning subject multidimensional information and the multidimensional information to be imitated corresponding to the real-time learning subject multidimensional information based on a mixing ratio parameter, generating coordinate data by mapping the mixed multidimensional information and the preprocessed multidimensional information dataset to be imitated onto the map, and drawing the coordinate data as an image.
[0010] A computer program according to a third aspect of the present invention is a computer program that causes a computer to function as a learning support device according to the first aspect. [Effects of the Invention]
[0011] According to the present invention, it is possible to provide a learning support device, a learning support method, and a computer program that maintain a high sense of task accomplishment for the learner and suppress a decline in motivation to learn through action imitation. [Brief explanation of the drawing]
[0012] [Figure 1] Figure 1 is a schematic block diagram showing one example of the configuration of a learning support device according to one embodiment. [Figure 2] Figure 2 is a schematic block diagram showing one example of the configuration of the SOM map construction unit of a learning support device according to one embodiment. [Figure 3] Figure 3 is a flowchart showing an example of the SOM map construction process of the map construction unit of a learning support device according to one embodiment. [Figure 4] Figure 4 is a schematic block diagram showing one example of the configuration of the mapping FB section of a learning support device according to one embodiment. [Figure 5] Figure 5 is a schematic block diagram showing another example of the configuration of the mapping FB section of a learning support device according to one embodiment. [Figure 6] Figure 6 is a flowchart showing an example of the mixing ratio parameter update process in the mixing ratio adjustment unit of a learning support device according to one embodiment. [Modes for carrying out the invention]
[0013] The learning support device according to the embodiment of this invention will now be described with reference to the drawings. In the following embodiments, parts with the same number will be assumed to perform the same operation, and therefore will not be described again. [First Embodiment] Figure 1 is a schematic block diagram showing one example of the configuration of the learning support device 1 according to the first embodiment.
[0014] The learning support device 1 of this embodiment is a device that reduces the dimensionality of motor information such as sports and playing musical instruments and provides visual feedback to the motor subject. For example, it constructs multidimensional motor information data of an imitation target and a learning subject (subject, participant) acquired in advance as a two-dimensional map representation called an SOM map, and maps (maps) the subject's data in real time based on the constructed SOM map and provides feedback.
[0015] The learning support device 1 comprises a processor 2, a display unit 5, an input unit 6, a memory 7, and a communication unit 8. The processor 2, the display unit 5, the input unit 6, the memory 7, and the communication unit 8 are interconnected via a bus communication line, enabling the transmission and reception of data between these components.
[0016] Processor 2 includes at least one CPU (Central Process Unit), MPU (microprocessing unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), etc., and can realize various functions of the learning support device 1 using programs such as system software, application software, or firmware stored in memory 7.
[0017] Processor 2 comprises an SOM map construction unit (low-dimensional map construction unit) 3 and a mapping FB unit 4, which perform SOM map construction processing and mapping FB processing, respectively. Details of these processes will be described later.
[0018] The display unit 5 may include, for example, display means such as a monitor, and audio output means such as a speaker. The display unit 5 may also be configured to output image information and audio information to externally connected display means and audio output means. For example, the display unit 5 outputs visually recognizable image information, etc., output by the processor 2, and audio information, etc., output by the processor 2.
[0019] The input unit 6 may include, for example, a user interface such as a mouse or keyboard, or a microphone, touch panel, camera, various sensors, etc. The input unit 6 transmits information acquired through user operation to the processor 2 via a bus communication line.
[0020] In this embodiment of the learning support device 1, the movement to be learned is assumed to be the pedaling motion of a bicycle. For example, the input unit 6 may include various sensors for measuring the crank angle of the bicycle, and various sensors for measuring the joint angles of the hip joint, knee joint, and ankle joint, which are specific body parts of the target and subject. In this embodiment, for illustrative purposes, the crank angle of the bicycle and the angles of the three joints are measured by sensors, but the values measured for the learning of the learning subject are not limited to these.
[0021] Memory 7 includes, for example, a main memory section and an auxiliary memory section. The main memory section may include, for example, ROM (read-only memory) and RAM (random-access memory). ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values used by the processor 2 in performing various processes. RAM can be used as a so-called work area to temporarily store data when the processor 2 performs various processes. In this embodiment, the main memory section is, for example, RAM and is used as memory.
[0022] The auxiliary storage unit is a non-temporary computer-readable storage medium for a computer centered around processor 2. Examples of auxiliary storage units include EEPROM (electric erasable programmable read-only memory), HDD (hard disk drive), or SSD (solid state drive).
[0023] The auxiliary storage unit can store data used by the processor 2 in performing various processes, data generated by processing in the processor 2, or various setting values. For example, the auxiliary storage unit is a memory for storing various information and includes a target motion capture data storage unit 71, a pre-processed target data storage unit 72, a subject motion capture data storage unit 73, a subject data normalization parameter storage unit 74, a pre-processed subject data storage unit 75, a constructed SOM map 76, and a mixing ratio parameter storage unit 77. The target motion capture data storage unit 71, the pre-processed target data storage unit 72, the subject motion capture data storage unit 73, the subject data normalization parameter storage unit 74, the pre-processed subject data storage unit 75, the constructed SOM map 76, and the mixing ratio parameter storage unit 77 will be described later. Note that the memory 7 is not an essential component inside the learning support device 1 and may be, for example, an externally connected server. In that case, various processing functions of the learning support device 1 become possible by reading or writing information sequentially from the server that stores the above-mentioned multiple pieces of information.
[0024] The communication unit 8 has the function of sending and receiving the above information stored in the memory 7 with an external party. For example, when uploading such information to the cloud or downloading such information from the cloud, the communication unit 8 sends and receives information with the cloud via the internet.
[0025] Next, we will describe the details of the SOM map construction unit included in processor 2. Figure 2 is a schematic block diagram showing one example of the configuration of the SOM map construction unit of a learning support device according to one embodiment.
[0026] The SOM map construction unit 3 comprises a data preprocessing unit 31, a data preprocessing unit 32, a data merging unit 33, and a map construction unit 34.
[0027] The data preprocessing unit 31 performs preprocessing on the time-series 3D coordinate dataset input from the target motion capture data storage unit 71 and outputs it to the preprocessed target data storage unit 72 and the data merging unit 33.
[0028] The data preprocessing unit 31 includes a joint angle calculation unit 311 and a normalization processing unit 312.
[0029] The joint angle calculation unit 311 acquires a time-series 3D coordinate dataset for N markers measured by motion capture sensors, etc., from the target motion capture data storage unit 71. This time-series 3D coordinate dataset is represented by the following equation 1.
number
number
number
[0030] The normalization processing unit 312 obtains the M time-series joint angles output by the joint angle calculation unit 311, and the angle θ j The angle θ is normalized and output by the normalization processing unit 312. jThe normalization process is represented by Equation 4 below.
Equation
[0031] That is, the data preprocessing unit 31 acquires the target motion capture data MC target from the target motion capture data storage unit 71, and outputs the preprocessed target data D target to the preprocessed target data storage unit 72 and the data combining unit 33.
[0032] The data preprocessing unit 32 performs preprocessing on the time-series three-dimensional coordinate data set input from the subject motion capture data storage unit 73, outputs the preprocessed data set to the preprocessed subject data storage unit 75 and the data combining unit 33, and outputs the subject data normalization parameters to the subject data normalization parameter storage unit 74.
[0033] The data preprocessing unit 32 includes a joint angle calculation unit 321 and a normalization processing unit 322.
[0034] The joint angle calculation unit 321 acquires a time-series three-dimensional coordinate data set for N markers measured by a motion capture sensor or the like from the subject motion capture data storage unit 73. The time-series three-dimensional coordinate data set is represented by Equation 1 above.
[0035] Next, the joint angle calculation unit 321 calculates the vector v jk = p k - p j from the marker of joint j to the marker of joint k, and calculates the angle (joint angle) θ jThe joint angle calculation unit 321 assumes that the operating body performs a bicycle pedaling motion rotating on the XZ plane, and uses the above equation 2 to calculate the angle θ. j We seek.
[0036] The joint angle calculation unit 321 outputs M time-series joint angles to be used as motion information to the normalization processing unit 322. These time-series joint angles are represented by the above equation 3. In this embodiment, the angles of four joints—hip joint, knee joint, ankle joint, and bicycle crank—are calculated using the above equation 3 and output to the normalization processing unit 322.
[0037] The normalization processing unit 322 obtains the M time-series joint angles output by the joint angle calculation unit 321, and the angle θ j The angle θ is normalized and output by the normalization processing unit 322. j The normalization process is represented by the above number 4. Furthermore, the normalization processing unit 322 uses the subject data normalization parameter {maxθ} used in the above normalization process. j ,minθ j The result is output to the subject data normalization parameter storage unit 74.
[0038] In other words, the data preprocessing unit 32 receives subject motion capture data MC from the subject motion capture data storage unit 73. subject Obtain preprocessed subject data D subject The data is output to the pre-processed subject data storage unit 75 and the data merging unit 33.
[0039] In this embodiment, target motion capture data MC target This is used as the target pedaling motion data, and subject motion capture data MC subject This will be used as the subject's pedaling motion data, and the target motion capture data MC target And, Subject Motion Capture Data MC subject This refers to data sampled at 10-degree intervals of crank angle (36 samples).
[0040] The data merging unit 33 receives the pre-processed target data D output from the data pre-processing unit 31 and the data pre-processing unit 32. target And, pre-processed subject data D subject The pre-processed target data D is obtained and target And, pre-processed subject data D subject The data is combined synchronously in the column direction (time axis t direction), and the combined data D is output to the map construction unit 34. The combined data D is represented by the following equation 5.
number
[0041] When the map construction unit 34 obtains the combined data D, the SOM map construction process begins. The map construction unit 34 may start the SOM map construction process when the combined data D is input, or it may start the SOM map construction process at any time after input.
[0042] When the SOM map construction process starts, the map construction unit 34 generates an H×W×D dimensional matrix (map) (H: height, W: width, D: element) and initializes the elements (nodes) with random values (step S1). In this embodiment, the H×W×D dimensional matrix (map) is a 50×50×4 square map (50 dimensions for each of the height and width, and 4 dimensions for each element), but it is not limited to this.
[0043] In step S2, the map construction unit 34 repeatedly performs the processes from steps S3 to S9 until it reaches a predetermined number of iterations. After the number of iterations reaches the predetermined number, the map construction unit 34 terminates the iteration process in step S10 (SOM learning loop). The processes from steps S4 to S5 are considered circular arrangement processing, and the processes from steps S6 to S8 are considered overfitting prevention processing.
[0044] The map construction unit 34 first performs a circular update process, which updates each element x of D. i (x i ∈D target or x i ∈D subject ) for (Step S3), element x of D i o On the circle corresponding to (D target In this case, on the inner circle, D subject In this case, the coordinates (corresponding to the crank angle) r (on the outer circle) c Find (Step S4). Any coordinate r k node n k For the element x of D, i o The error is weighted h ck The result multiplied by the original node n k Add to node n k Update (step S5). Node n k The formula for calculating this is expressed by the following number 6.
number
number
[0045] Next, the map construction unit 34, as an overfitting prevention process, after the circular arrangement process from step S4 to step S5, processes element x of D. i o Adding white noise ε with mean 0 and variance 1 to this gives a i =x i +qε o (q: noise multiplier) (step S6), a i for nearest neighbor node n c This is calculated using equation 8 below (step S7). It is recommended to set the noise magnification q to approximately 0.05.
number
number
[0046] Next, we will describe the details of the mapping FB unit 4 provided by processor 2. Figure 4 is a schematic block diagram showing one example of the configuration of the mapping FB unit 4 of the learning support device 1 according to one embodiment.
[0047] The mapping FB unit 4 includes an motion measurement unit 411, a real-time subject data acquisition unit (first acquisition unit) 41, a target data acquisition unit (second acquisition unit) 42 corresponding to the crank angle, a data mixing unit (mixing unit) 43, a target data mapping (mapping) unit 44, a mixed data mapping (mapping) unit 45, and a drawing unit 46.
[0048] The real-time subject data acquisition unit 41 preprocesses the subject's movement measured in real time by a motion capture sensor or the like, and outputs it to the target data acquisition unit 42 and the data mixing unit 43, which correspond to the crank angle. In this embodiment, considering the influence on the drawing process by the drawing unit 46, which will be described later, the real-time subject data acquisition unit 41 acquires subject data at 18 Hz.
[0049] The real-time subject data acquisition unit 41 includes a joint angle calculation unit 412 and a normalization processing unit 413.
[0050] The motion measurement unit 411 measures the 3D coordinates P(t) for N markers at time t of the subject in real time using motion capture sensors, etc., and outputs them to the joint angle calculation unit. The 3D coordinates P(t) are represented by the following equation 10.
number
number
number
[0051] The normalization processing unit 413 obtains the M joint angles θ(t) output by the joint angle calculation unit 412, normalizes the joint angles θ(t), and calculates the element x subject (t) is output. The normalization processing unit 413 obtains subject data normalization parameters from the subject data normalization parameter storage unit 74 as part of the normalization process, and performs Min-Max Normalization using said subject data normalization parameters.
[0052] The target data acquisition unit 42, which corresponds to the crank angle, receives element x from the real-time subject data acquisition unit 41. subject (t) is obtained, and the preprocessed target data D is obtained from the preprocessed target data storage unit 72. target Obtain pre-processed target data D. target This is represented by the following number 13.
number
number
number
number
number
number
[0053] The data mixing unit 43 processes the elements x output by the real-time subject data acquisition unit 41. subject (t) and the element x output by the target data acquisition unit 42 corresponding to the crank angle target (t) is obtained. The data mixing unit 43 also simultaneously obtains the mixing ratio parameter α from the mixing ratio parameter storage unit 77.
[0054] The data mixing unit 43 is composed of the above element x subject (t) and the above element x target The mixed x(t) (mixed multidimensional information) obtained by adjusting (t) with the mixing ratio parameter α is calculated using the following equation 19 and output to the mixed data mapping unit 45.
number
[0055] The target data mapping unit 44 acquires the pre-processed target data D target stored in the pre-processed target data storage unit 72 and the map stored in the constructed SOM map 76. Note that the pre-processed target data D target is represented by the following equation (21). [Number] The target data mapping unit 44 searches for the node n (i,j) closest to the target element x k target among the elements (nodes) n of the map using the following equation (22). (c,d) [Number] The target data mapping unit 44 uses the position (c, d) on the map of n (c,d) as the mapping coordinates of the target element x k target and outputs the mapping coordinate series data T = [(c, d) k (k = 1,..., T) obtained for all k = 1,..., T to the drawing unit 46.
[0056] The drawing unit 46 acquires the map stored in the constructed SOM map 76, the mapping coordinates (a, b) output by the mixed data mapping unit 45, and the mapping coordinate series data T = [(c, d) k (k = 1,..., T) output by the target data mapping unit 44.
[0057] The drawing unit 46 selects an appropriate channel (vector n (i,j)Select some elements from the map and draw them as an image. In this embodiment, the map channels will have four values corresponding to the hip joint, knee joint, ankle joint, and bicycle crank, and three of these channels, excluding the bicycle crank, will be visualized as RGB values. Then, a cursor centered at the mapping coordinates (a,b) will be drawn on the same image. The size of the cursor can be freely set by the subject, etc., and it is recommended to specify an appropriate size that allows the subject to be visible.
[0058] The drawing unit 46 uses mapping coordinate series data T=[(c,d) k The point cloud (k=1,...,T) is connected by lines and drawn on the image as a trajectory. The line width of the trajectory can be freely set by the subject, etc., and it is recommended to specify an appropriate size so that the subject is visible.
[0059] The drawing unit 46 outputs the drawn image to the display unit 5, and the display unit 5 displays the image using the image display means.
[0060] According to the learning support device 1 of this embodiment, by mixing the motion information of the object to be imitated and the motion information of the learning subject in an arbitrary ratio, it becomes possible to adjust the coordinates of the cursor corresponding to the motion information of the learning subject on the SOM map. Therefore, according to the learning support device 1 of this embodiment, the motion information of the learning subject can be reflected in the cursor movement in stages, and if the motion information of the learning subject deviates greatly from the motion information of the object to be imitated in the early stages of learning, it becomes possible to reduce the degree to which the motion information of the learning subject is reflected in the cursor, thereby supporting the learning subject in maintaining a high sense of task accomplishment.
[0061] In other words, according to this embodiment, it is possible to provide a learning support device, a learning support method, and a computer program that maintain a high sense of task accomplishment for the learner and suppress a decline in motivation to learn through action imitation. [Second Embodiment] Next, the learning support device, learning support method, and computer program of the second embodiment will be described in detail with reference to the drawings.
[0062] Note that since a part of the configuration of the learning support device 1 in the present embodiment is the same as that of the learning support device 1 in the first embodiment, the description of the part having the same configuration is omitted.
[0063] FIG. 5 is a block diagram schematically showing an example of another configuration of the mapping FB unit 4 of the learning support device 1 according to an embodiment.
[0064] In the learning support device 1 of the present embodiment, the auxiliary storage unit of the memory 7 is different from the learning support device 1 of the first embodiment described above in that it further includes a motion error memory 78 and an update rate parameter accumulation unit 79. The motion error memory 78 and the update rate parameter accumulation unit 79 will be described later.
[0065] Also, in the learning support device 1 of the present embodiment, the mapping FB unit 4 is different from the learning support device 1 of the first embodiment described above in that it includes a mixing rate adjustment unit 47 including an error calculation unit 471 and a parameter update unit 472 in addition to the configuration of the mapping FB unit 4 of the first embodiment.
[0066] The details of the mixing rate parameter update process of the mixing rate adjustment unit 47 will be described with reference to FIG. 6. FIG. 6 is a flowchart showing an example of the mixing rate parameter update process of the mixing rate adjustment unit 47 of the learning support device 1 according to an embodiment.
[0067] The mixing rate adjustment unit 47 starts the mixing rate parameter update process at the timing when the element x subject (t) output by the real-time subject data acquisition unit 41 is input to the error calculation unit 471 and the element x target (t) output by the target data acquisition unit 42 corresponding to the crank angle. Note that the mixing rate parameter update process may be started at the timing when the above element x subject (t) and / or the element x target (t) is input, or may be started at an arbitrary timing after the input.
[0068] When the mixing ratio parameter update process starts, the error calculation unit 471 calculates the motion information error e(t) at the current time using the formula e(t) = x subject (t)-x target The result is calculated based on (t) (step S11) and output (stored) in the motion error memory 78 (step S12).
[0069] Next, the parameter update unit 472 obtains the error e(t) output by the motion error memory 78 and the update rate parameter η stored in the update rate parameter storage unit 79. The parameter update unit 472 updates the mixing rate parameter α according to the amount of change in the error e(t). When α=1 is reached, the parameter update unit 472 terminates the mixing rate parameter update process. In other words, the parameter update unit 472 adjusts the elements x included in the mixed multidimensional information x(t) according to the decrease (or increase) of the error e(t). subject The mixing ratio parameter α is updated to increase (or decrease) the ratio of (t).
[0070] In this embodiment, since the movement to be learned is assumed to be bicycle pedaling, the parameter update unit 472 can perform the mixing ratio parameter update process in the following steps S13 to S16. In this embodiment, an example will be described in which the update frequency by the parameter update unit 472 is set to one pedaling cycle (one trial).
[0071] The parameter update unit 472 reads the error e(t) from the motion error memory 78 in time series for each trial (step S13), and calculates the average E of the error e(t) in that trial (step S14). The parameter update unit 472 calculates the average E of the error e(t) in the most recent trial. T-1 and the average error e(t) from the previous trial E T-2 The difference between these two results in a performance improvement of P. T =E T-1 -E T-2 Calculate (Step S15).
[0072] The parameter update unit 472 is P TFeature average E[P T Based on ], update the mixing ratio parameter α according to the following number 23 (step S16). Note that up to the second trial, E T-2 Since this cannot be calculated, the parameter update unit 472 sets an appropriate initial value such as 0.1 for the mixing ratio parameter α.
number
[0073] It should be noted that using the average value of the error e(t) over one trial as an indicator in the parameter update process described above is just one example and is not limited to this. For example, the parameter update unit 472 may perform the parameter update process using other aggregated values of the error e(t) over any period (such as the sum of errors e(t), minimum value, maximum value, median value, approximate value of the change) or combinations thereof as indicators. Furthermore, as another example of parameter update processing, the parameter update unit 472 may update the mixing ratio parameter α using the error e(t) at the time of pedal depression. In this case, the parameter update unit 472 may be configured to read the error e(t) in time series from the motion error memory 78 for each trial and update the mixing ratio parameter α according to the amount of change in the error e(t) at a specific point in the time series (for example, when the imitation target presses the pedal, when the learning subject presses the pedal, or when both the imitation target and the learning subject press the pedal).
[0074] According to the learning support device 1 of this embodiment, the mixing ratio parameter is set to increase the mixing ratio of the learning subject's motion information with respect to the movement information of the object to be imitated, based on the amount of reduction in the error between the learning subject's motion information and the object's motion information. In other words, the degree to which the learning subject's motion information is reflected in the cursor is increased. As a result, the learning support device 1 of this embodiment can contribute to improving the learning subject's motion imitation while maintaining a high sense of task accomplishment for the learning subject.
[0075] In other words, according to this embodiment, it is possible to provide a learning support device, a learning support method, and a computer program that maintain a high sense of task accomplishment for the learner and suppress a decline in motivation to learn through action imitation.
[0076] In addition, in the first and second embodiments described above, the learning support device 1 may be configured as a learning support system that includes a device equipped with an SOM map construction unit 3, a device equipped with a mapping FB unit 4, and a storage device that includes various storage units. In that case, the multiple devices included in the learning support system can share various information stored in the various storage units of the storage device.
[0077] The programs according to the first and second embodiments may be transferred while stored in an electronic device, or they may be transferred while not stored in an electronic device. In the latter case, the programs may be transferred via a network, or they may be transferred while stored in a storage medium. The storage medium is a non-temporary tangible medium. The storage medium is a computer-readable medium. The storage medium can be any medium that is capable of storing programs and is readable by a computer, such as a CD-ROM or memory card, and its form is not limited.
[0078] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention. [Explanation of Symbols]
[0079] 1…Learning support device 2…Processor 3…SOM Map Construction Department 4…Mapping FB section 5...Display section 6...Input section 7…Memory 8… Communications Department 31...Data preprocessing unit 32...Data preprocessing unit 33...Data merging section 34... Map Construction Department 41...Real-time subject data acquisition unit 42...Target data acquisition unit 43...Data mixing section 44…Target Data Mapping Department 45…Mixed Data Mapping Section 46... Drawing section 47...Mixing ratio adjustment section 71...Target motion capture data storage unit 72... Pre-processed target data storage unit 73... Subject motion capture data storage unit 74... Subject data normalization parameter storage unit 75... Pre-processed subject data storage unit 76... Completed SOM map 77... Mixing ratio parameter storage section 78…Motion Error Memory 79... Update rate parameter storage unit 311... Joint angle calculation unit 312...Normalization Processing Unit 321... Joint angle calculation unit 322...Normalization Processing Unit 411...Motion Measurement Unit 412... Joint angle calculation unit 413...Normalization Processing Unit 471...Error calculation section 472...Parameter update section
Claims
1. A low-dimensional map construction unit generates a multi-dimensional information dataset by combining a multi-dimensional information dataset to be imitated and a multi-dimensional information dataset of the learning subject, and constructs a map with reduced dimensionality from the multi-dimensional information dataset. A first acquisition unit that acquires real-time learning-oriented multidimensional information in a time series, A second acquisition unit acquires the multidimensional information to be imitated that corresponds to the real-time learning subject multidimensional information at a predetermined time from the multidimensional information dataset to be imitated after preprocessing in the aforementioned reduction in dimensionality, A mixing unit that generates mixed multidimensional information by mixing the real-time learning subject multidimensional information and the imitation target multidimensional information corresponding to the real-time learning subject multidimensional information based on a mixing ratio parameter, A mapping unit that generates coordinate data by mapping the mixed multidimensional information and the pre-processed multidimensional information dataset to be imitated onto the map, A drawing unit that draws the aforementioned coordinate data as an image, A learning support device equipped with the following features.
2. An error calculation unit calculates the error between the real-time learning subject multidimensional information and the imitation target multidimensional information corresponding to the real-time learning subject multidimensional information as error information. A parameter update unit updates the mixing ratio parameter to increase the ratio of the real-time learning subject multidimensional information included in the mixed multidimensional information in accordance with the amount of decrease in the error information, A learning support device according to claim 1, comprising:
3. The aforementioned low-dimensional map construction unit, A circular arrangement process is performed to calculate the coordinates on a circle corresponding to the elements of the multidimensional information dataset, and to update the nodes using the error between the nodes at the coordinates and the elements of the multidimensional information dataset. The process involves calculating a value obtained by adding noise to the elements of the multidimensional information dataset, and then performing an overfitting prevention process to update the node using the error between the calculated value and the node. The learning support device according to claim 1.
4. Each of the multiple elements included in the aforementioned multidimensional information dataset to be imitated is associated with time, The aforementioned second acquisition unit is, From the aforementioned multidimensional information dataset to be imitated, the element with the time closest to the predetermined time is obtained. The learning support device according to claim 1.
5. A multidimensional information dataset is generated by combining the multidimensional information dataset of the target to be imitated and the multidimensional information dataset of the learning subject, and a map is constructed that reduces the dimensionality of the said multidimensional information dataset. Real-time learning acquires multidimensional information in a time series, From the multidimensional data set of the target to be imitated after preprocessing in the aforementioned reduction in dimensionality, the multidimensional information of the target to be imitated corresponding to the real-time learning subject multidimensional information at a predetermined time is obtained. Mixed multidimensional information is generated by mixing the real-time learning subject multidimensional information and the imitation target multidimensional information corresponding to the real-time learning subject multidimensional information based on the mixing ratio parameter. Coordinate data is generated by mapping the mixed multidimensional information and the pre-processed multidimensional information dataset to be imitated onto the map. A learning support method that visualizes the aforementioned coordinate data as an image.
6. A computer program that causes a computer to function as a learning support device according to any one of claims 1 to 4.