Student practice effect feedback method and system based on multi-dimensional data driving

Abstract

The invention provides a student practice effect feedback method and system based on multi-dimensional data driving, and relates to the technical field of training feedback systems. According to the method, the expert standard action library is constructed, the joint point activity frequency is dynamically analyzed based on project types, core and secondary capture points are intelligently divided, and the collected data are subjected to grading processing in combination with a weighting mechanism, so that the system efficiency is improved while the key action information precision is guaranteed; a difference value is calculated by performing weighted comparison on an action track of a student and a standard track, layered feedback is generated according to a preset threshold value, differentiated tactile prompts are output through wearable equipment, and closed-loop control from data acquisition and intelligent evaluation to real-time feedback is realized; according to the whole scheme, the accuracy and scientificity of action evaluation are improved, students are stimulated and helped to quickly correct errors and consolidate correct actions in a non-intrusive real-time tactile interaction mode, and the training efficiency, concentration degree and autonomy are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of training feedback system, and particularly relates to a student exercise effect feedback method and system based on multi-dimensional data driving. BACKGROUND

[0002] In the current field of sports training, rehabilitation therapy and skill learning, how to scientifically, timely and effectively feedback the exercise effect of students has been a key challenge in technical application. The traditional action guidance method mainly relies on the naked eye observation or video playback analysis of the coach, which is highly subjective and has a lagging feedback, and it is difficult to capture subtle action deviations, especially for high-speed or complex coordination actions, manual judgment is easy to miss key details, resulting in untimely correction or even misdirection. With the development of sensing technology, some systems begin to introduce inertial sensors or optical motion capture devices to collect the motion data of the joints of students, so as to realize quantitative analysis. However, the existing technology generally has the following problems: Firstly, most systems use a unified standard mode for action comparison, that is, the same weight is given to all joints, ignoring the difference in the role of each joint in different training projects. For example, in the practice of Tai Chi, the stability of the trunk and core joints is much more critical than the action of the finger tip; while in the piano fingering training, the accurate trajectory of the finger joint becomes the evaluation focus. If the primary and secondary are not distinguished, the evaluation result may deviate from the actual training goal, affecting the pertinence of the feedback.

[0003] Secondly, the data processing strategy lacks hierarchical design, and the same filtering and analysis method is often used for all joint data, resulting in unreasonable allocation of computing resources, either over-processing non-critical nodes causing system delay, or insufficient processing of core action characteristics, reducing the recognition accuracy.

[0004] More importantly, the existing feedback mechanism relies on screen display or voice prompt, and such visual or auditory feedback is easy to interfere with the concentration of students, and destroy the coherence of the action, especially in training scenarios that require immersive experience. Although some wearable devices have tried to introduce vibration feedback, the triggering logic is simple, and usually only makes completion prompts or error alarms, lacking a differentiated tactile incentive mechanism based on action quality grading.

[0005] Therefore, it is necessary to provide a student exercise effect feedback method and system based on multi-dimensional data driving to solve the above technical problems. SUMMARY

[0006] To solve the above technical problems, the present application provides a student exercise effect feedback method and system based on multi-dimensional data driving to solve the problems that the existing technology ignores the difference in the role of each joint in different training projects, the data processing strategy lacks hierarchical design, and the feedback lacks differentiation.

[0007] The application provides a student practice effect feedback method based on multi-dimensional data driving, and the feedback method comprises the following steps: S1, for different practice projects, an expert standard action library containing multiple project types is established; S2, according to the project type of the current practice of the student, the joint activity frequency of all standard actions of the project type is counted from the expert standard action library, and the core capture point and the secondary capture point are defined according to the joint activity frequency, and the core capture point and the secondary capture point are respectively assigned weights; S3, the joint node data of the student is synchronously collected through a sensor, and the core capture point and the secondary capture point are marked, and the data of the core capture point and the data of the secondary capture point are processed in stages according to the weights assigned to the core capture point and the secondary capture point, so as to obtain the core capture point trajectory and the secondary capture point trajectory; S4, the standard action fixed-point trajectory of the current project type is extracted from the expert standard action library, the core capture point trajectory and the secondary capture point trajectory are compared with the standard action fixed-point trajectory respectively, and the difference value is calculated; S5, a hierarchical feedback is generated according to the calculated difference value, and a real-time tactile feedback is given to the student through a wearable device.

[0008] Preferably, the specific steps of step S1 are as follows: S101, the practice project types to be covered are determined, and the standard action names corresponding to each project are listed; S102, the three-dimensional coordinate data of the joint nodes of the expert standard action are collected through an inertial sensor or an optical capture system device, and the standard action data of each project is obtained; S103, the standard action data of each project is stored according to the project type, so as to construct the expert standard action library, wherein each standard action data comprises a joint trajectory sequence, an action duration and a key frame mark.

[0009] Preferably, the specific steps of step S2 are as follows: S201, the project type of the current practice of the student is determined, and all standard action data under the project type of the current practice of the student is extracted from the expert standard action library; S202, all standard action data under the project type of the current practice of the student is analyzed, specifically including analyzing and counting the activity frequency of each joint node of the student's body in the process of completing the standard action; S203, according to the joint activity frequency obtained by counting, the joint activity frequency is arranged in descending order, the joints corresponding to the front 30% joint activity frequency in the descending order are defined as the core capture points, and the joints corresponding to the rear 30% joint activity frequency in the descending order are defined as the secondary capture points. S204, respectively set the core capture point weight to 0.7 and the secondary capture point weight to 0.3.

[0010] Preferably, the specific steps of the step S3 are: S301, synchronously collecting three-dimensional coordinate data of the key joints of the trainee during the practice through the sensor; S302, based on the three-dimensional coordinate data of the key joints, respectively labeling the three-dimensional coordinate data of the corresponding key joints according to the defined core capture point and the secondary capture point, to obtain the labeled core capture point data and the secondary capture point data; S303, according to the weight set for the core capture point and the weight set for the secondary capture point, performing hierarchical processing on the labeled core capture point data and the secondary capture point data, wherein the hierarchical processing specifically includes that the core capture point data is processed in a high-precision filtering and time alignment manner, and the secondary capture point data is processed in a lightweight denoising manner, to respectively obtain the core capture point trajectory and the secondary capture point trajectory.

[0011] Preferably, the specific steps of the step S4 are: S401, extracting the standard action fixed-point trajectory of the current practice project type of the trainee from the expert standard action library, and determining the comparison time point; S402, using the comparison method of calculating the distance between two points, respectively comparing and calculating the obtained core capture point trajectory and the secondary capture point trajectory with the corresponding key joint trajectory in the standard action fixed-point trajectory, to obtain the difference values of each key joint obtained by comparison and calculation; S403, according to the weight set for the core capture point and the weight set for the secondary capture point, and through the calculation method of weighted summation, comprehensively calculating the difference values of each key joint obtained by comparison and calculation, to respectively obtain the final difference value of the core capture point and the final difference value of the secondary capture point.

[0012] Preferably, the specific steps of the step S5 are: S501, pre-setting the difference value threshold, and respectively comparing the final difference value of the core capture point and the final difference value of the secondary capture point with the set difference value threshold, to generate hierarchical feedback according to the comparison result, wherein the hierarchical feedback includes that if the final difference value exceeds the set difference value threshold, the action is qualified, and the feedback information is to maintain the action; if the final difference value does not exceed the set difference value threshold, the action is unqualified, and the feedback information is to adjust the action. S502, according to the generated hierarchical feedback, set the corresponding haptic feedback mode, and carry out haptic feedback through the wearable device, wherein the haptic feedback mode includes: for the action qualified, the haptic feedback mode is soft, short vibration feedback, and for the action unqualified, the haptic feedback mode is strong, continuous vibration feedback.

[0013] The student practice effect feedback system based on multi-dimensional data driving comprises: A database construction module is configured to establish an expert standard action library containing multiple project types for different practice projects; An identification definition module is configured to count the joint node activity frequency of all standard actions of the project type from the expert standard action library according to the project type currently practiced by the student, and define the core capture point and the secondary capture point according to the joint node activity frequency, and respectively assign weights to the core capture point and the secondary capture point; A hierarchical processing module is configured to synchronously collect student joint node data through a sensor, mark the core capture point and the secondary capture point, and perform hierarchical processing on the data of the core capture point and the secondary capture point according to the weights assigned to the core capture point and the secondary capture point, to obtain the core capture point trajectory and the secondary capture point trajectory; A comparative analysis module is configured to extract the standard action fixed-point trajectory of the current project type from the expert standard action library, compare the core capture point trajectory and the secondary capture point trajectory with the standard action fixed-point trajectory respectively, and calculate the difference value; A feedback interaction module is configured to generate hierarchical feedback according to the calculated difference value, and perform real-time haptic feedback to the student through a wearable device.

[0014] Compared with the related art, the student practice effect feedback method and system based on multi-dimensional data driving provided by the application have the following beneficial effects: The application constructs an expert standard action library and dynamically analyzes the joint node activity frequency based on the project type, intelligently divides the core and secondary capture points, and performs hierarchical processing on the collected data in combination with the weighting mechanism, thereby improving the system efficiency while ensuring the accuracy of the key action information; the student action trajectory is compared with the standard trajectory in combination with the weighting mechanism, the difference value is calculated, hierarchical feedback is generated according to the preset threshold, and the differential haptic prompt is output through the wearable device, thereby realizing the closed-loop control from data collection, intelligent evaluation to real-time feedback; the overall scheme not only improves the accuracy and scientificity of the action evaluation, but also stimulates and helps the student to quickly correct errors and consolidate correct actions in a non-intrusive real-time haptic interaction manner, thereby significantly improving the training efficiency, concentration and autonomy. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1The flowchart shows the student practice effect feedback method based on multi-dimensional data driven by the present invention. Figure 2 This is a system block diagram of the student practice effect feedback system based on multi-dimensional data driven by the present invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Example 1 like Figure 1 As shown, the multi-dimensional data-driven feedback method for student practice effectiveness includes the following steps: S1. Establish an expert standard movement library containing multiple exercise types for different practice projects; S2. Based on the type of project the student is currently practicing, calculate the joint activity frequency of all standard movements of that project type from the expert standard movement library, and define core capture points and secondary capture points based on the joint activity frequency, and assign weights to the core capture points and secondary capture points respectively. S3. Collect trainee joint data synchronously through sensors and mark core capture points and secondary capture points. Based on the weights assigned to core capture points and secondary capture points, perform hierarchical processing on the data of core capture points and secondary capture points to obtain the trajectory of core capture points and the trajectory of secondary capture points. S4. Extract the standard motion fixed-point trajectory of the current project type from the expert standard motion library, compare the core capture point trajectory and the secondary capture point trajectory with the standard motion fixed-point trajectory respectively, and calculate the difference value. S5. Generate layered feedback based on the calculated difference values ​​and provide real-time tactile feedback to trainees through wearable devices.

[0018] In the specific implementation process, the specific steps of step S1 are as follows: S101. Identify the types of practice items that need to be covered and list the standard movement names corresponding to each item.

[0019] Specifically, understand the common types of exercises in specific fields such as fitness, dance, and martial arts. For example, in the fitness field, common types of exercises include strength training such as barbell squats and dumbbell bench presses; aerobic exercises such as running and rope skipping; and flexibility training such as yoga postures and Pilates movements.

[0020] S102. Collect the three-dimensional coordinate data of the joints of the expert standard movements through inertial sensors or optical capture system equipment to obtain the standard movement data of each project.

[0021] Specifically, the experts are required to repeat the standard movements for multiple times, and the motion data of the joints are collected by the selected inertial sensor or optical capture system device, and the collected three-dimensional coordinate data of the joints are recorded in real time and stored in the computer or special storage device. The data recording format can adopt a common file format, such as CSV (Comma Separated Values) format, to facilitate subsequent data processing and analysis.

[0022] S103, store the standard movement data of each project according to the project type to construct an expert standard movement library, wherein each standard movement data comprises a joint trajectory sequence, a movement duration and a key frame mark.

[0023] Specifically, according to the exercise project type specified in step S101, the collected standard movement data is classified. For example, all standard movement data related to barbell squat is classified into one category, and all standard movement data related to plank support is classified into another category, and so on. The classified data is stored in a folder structure, and different project type folders are created on the computer hard disk, such as "strength training", "aerobic exercise", "flexibility training", etc. Then, specific exercise project sub-folders are created under each project type folder, such as "barbell squat", "dumbbell bench press", etc. under the "strength training" folder. For each classified standard movement data, the joint trajectory sequence, movement duration and key frame mark are integrated and encapsulated. It should be noted that the joint trajectory sequence refers to the sequence of three-dimensional coordinates of each joint changing with time during the movement; the movement duration refers to the time from the start to the end of the movement; and the key frame mark refers to the time point with important significance during the movement, such as the time point of squatting to the lowest point in barbell squat, the starting time point of standing up, etc. All the integrated standard movement data is stored in a unified storage system to construct an expert standard movement library. It should be noted that the expert standard movement library can be implemented by using a relational database or a non-relational database.

[0024] In the specific implementation process, the specific steps of step S2 are: S201, specify the project type of the current exercise of the student, and extract all standard movement data of the project type of the current exercise of the student from the expert standard movement library.

[0025] Specifically, the type of the current exercise of the trainee is determined in various ways. In this embodiment, a project selection interface is set in the training system, and the trainee selects the project of the current exercise before starting the exercise. The system records the selection information of the trainee. For example, in a fitness training APP, the interface displays categories such as strength training, aerobic exercise, and flexibility training. After determining the type of the current exercise of the trainee, the expert standard motion library is queried and extracted according to the determined type. For example, all standard motion data records related to the type of the project are filtered from the database using an SQL statement.

[0026] S202, analyze all standard motion data under the type of the current exercise of the trainee, specifically including analyzing and counting the activity frequency of each joint of the trainee's body during the completion of the standard motion.

[0027] Specifically, the extracted standard motion data is preprocessed to ensure the accuracy and consistency of the data. For example, check if there are missing values or abnormal values in the joint trajectory sequence. For missing values, interpolation method can be used for filling. For abnormal values, correction or rejection is performed. At the same time, the time scale of the data is unified to make the time intervals of different standard motion data consistent, which is convenient for subsequent analysis. For each joint, the number of activities during the completion of all standard motions is counted. The number of activities can be counted by traversing the joint trajectory sequence and recording the time points at which the joint position changes significantly. According to the counted number of activities and the motion duration, the activity frequency of each joint is calculated. The calculation formula of the activity frequency is: activity frequency = number of activities / motion duration.

[0028] S203, according to the joint activity frequency obtained by statistics, and arranging the joint activity frequency in descending order, defining the joints corresponding to the top 30% joint activity frequency in the descending order as core capture points, and defining the joints corresponding to the last 30% joint activity frequency in the descending order as secondary capture points.

[0029] Specifically, the activity frequencies of the joints obtained by statistics are arranged in descending order. The joint activity frequency array is sorted according to the sorting result. According to the sorting result, 30% of the total number of joints is calculated. If the total number of joints is n, then 30%n is the basis for determining the number of core and secondary capture points. The joints corresponding to the top 30% joint activity frequency in the descending order are defined as core capture points, and the joints corresponding to the last 30% joint activity frequency in the descending order are defined as secondary capture points.

[0030] S204, respectively setting the weight of the core capture point to 0.7 and the weight of the secondary capture point to 0.3.

[0031] In the implementation process, the specific steps of step S3 are: S301, synchronously collect the three-dimensional coordinate data of the key joints of the trainee in the practice process through the sensors.

[0032] Specifically, the sensors used to collect the three-dimensional coordinate data of the key joints include inertial measurement units (IMUs), optical motion capture cameras, etc., to ensure that all sensors are synchronized in time to accurately record the position information of each key joint at different times; then, the three-dimensional coordinate data of the key joints of the trainee in the practice process is synchronously collected through the sensors, and the data format of the three-dimensional coordinate data is that each time point corresponds to a set of three-dimensional coordinates of the key joints, wherein each key joint has a unique identifier.

[0033] S302, based on the three-dimensional coordinate data of the key joints, the three-dimensional coordinate data of the corresponding key joints is respectively labeled according to the defined core capture points and secondary capture points, to obtain the labeled core capture point data and secondary capture point data.

[0034] Specifically, by reading the identifier of the three-dimensional coordinate data of the key joints, the three-dimensional coordinate data of the key joints is compared with the list of core capture points and secondary capture points to determine whether each data point belongs to a core capture point or a secondary capture point, and then, the matched core capture point and secondary capture point data are added with corresponding labeling information, which can include capture point type (core or secondary), key joint name, timestamp, etc.

[0035] S303, according to the weight set for the core capture points and the weight set for the secondary capture points, the labeled core capture point data and secondary capture point data are processed in stages, wherein the core capture point data is processed in a high-precision filtering and time alignment manner, while the secondary capture point data is processed in a lightweight denoising manner, to respectively obtain the core capture point trajectory and the secondary capture point trajectory.

[0036] Specifically, for the core capture point data, a high-precision filtering algorithm in the prior art is used to filter the core capture point data to remove noise and interference in the data and improve the accuracy and smoothness of the data, wherein the high-precision filtering algorithm used includes Kalman filtering or particle filtering; then, since there may be a certain time error in the data collected by the sensors, the core capture point data needs to be time-aligned to ensure that the data at different time points accurately correspond on the time axis, which can be achieved by using time interpolation method to interpolate the missing or inaccurate data in the middle according to the known time point data, so that the data is continuous and accurate in time; for example, if the data at a certain time point is missing, the core capture point position at that time point can be estimated by linear interpolation according to the data at the two time points before and after it.

[0037] For the secondary capture point data, a relatively simple denoising method in the prior art is used for processing to reduce the amount of calculation and improve the processing efficiency. In this embodiment, the simple denoising method includes a moving average filter or a median filter.

[0038] In the specific implementation process, the specific steps of step S4 are as follows: S401, extracting the standard action fixed-point trajectory of the type of the current practice project of the trainee from the expert standard action library, and determining the comparison time points.

[0039] Specifically, according to the type of the project that the trainee is currently practicing, the standard action data matched with the current practice action of the trainee is searched out in the constructed expert standard action library. Each standard action data contains information such as the node trajectory sequence, the action duration, and the key frame marker, and the fixed-point trajectory in the node trajectory sequence is extracted. The fixed-point trajectory is usually the node position information corresponding to some key moments in the execution process of the standard action, for example, the node positions at the key moments such as the starting posture, the force, and the closing posture in martial arts action. Then, the comparison time points are determined, and the key frame-based method is used to determine the comparison time points according to the key frame markers in the standard action data.

[0040] S402, using the comparison method of calculating the distance between two points, the obtained core capture point trajectory and secondary capture point trajectory are compared and calculated with the corresponding node trajectories in the standard action fixed-point trajectory respectively, and the difference values obtained by the comparison and calculation of each node are obtained.

[0041] Specifically, for each comparison time point, the distance between the positions of the corresponding nodes in the trainee's core capture point trajectory and secondary capture point trajectory and the positions of the same nodes in the standard action fixed-point trajectory is calculated, and the Euclidean distance in the prior art can be used for calculation. The distance calculation is performed for each comparison time point and each node in turn, and the differences of each node at different comparison time points are obtained.

[0042] S403, according to the weight set for the core capture point and the weight set for the secondary capture point, and through the calculation method of weighted summation, the difference values obtained by the comparison and calculation of each node are comprehensively calculated to obtain the final difference value of the core capture point and the final difference value of the secondary capture point respectively.

[0043] Specifically, in the basketball shooting action in the embodiment, the core capture points are shoulder joint and elbow joint, and the secondary capture points are wrist joint and ankle joint. The difference values of the shoulder joint are 0.2 and 0.3 respectively, the difference values of the elbow joint are 0.15 and 0.25 respectively, the difference values of the wrist joint are 0.1 and 0.2 respectively, and the difference values of the ankle joint are 0.05 and 0.15 respectively. The weight of the core capture point and the weight of the secondary capture point are 0.7 and 0.3 respectively. The final difference value of the core capture point is calculated by weighted summation as follows: 0.7 x (0.2 + 0.3 + 0.15 + 0.25) = 0.7 x 0.9 = 0.63. The final difference value of the secondary capture point is calculated by weighted summation as follows: 0.3 x (0.1 + 0.2 + 0.05 + 0.15) = 0.3 x 0.5 = 0.15.

[0044] In the specific implementation process, the specific steps of step S5 are as follows: S501, a difference value threshold is set in advance, and the final difference value of the core capture point and the final difference value of the secondary capture point are compared with the set difference value threshold respectively, and a hierarchical feedback is generated according to the comparison result, wherein the hierarchical feedback includes that if the final difference value exceeds the set difference value threshold, the action is qualified, and the feedback information is to maintain the action; if the final difference value does not exceed the set difference value threshold, the action is unqualified, and the feedback information is to adjust the action.

[0045] Specifically, the difference value threshold is set in advance. In practice, a difference value threshold can be determined comprehensively in combination with the teaching target and the actual training effect. It should be noted that the difference value threshold can be dynamically adjusted according to the actual situation. As the level of the trainee improves, the threshold can be appropriately reduced to improve the precision requirement of the training. After obtaining the final difference value of the core capture point and the final difference value of the secondary capture point, they are compared with the pre-set difference value threshold respectively. The comparison process is independent, that is, the difference value of the core capture point is compared with the threshold corresponding to the core capture point, and the difference value of the secondary capture point is compared with the threshold corresponding to the secondary capture point. If the final difference value of the core capture point or the secondary capture point exceeds the set difference value threshold, it is determined that the part of the action is qualified, and the feedback information is to maintain the action, which indicates that the performance of the trainee in this part of the action has reached or exceeded the expected standard, and no substantial adjustment is needed, only the current action state needs to be maintained for continuous practice. If the final difference value of the core capture point or the secondary capture point does not exceed the set difference value threshold, it is determined that the part of the action is unqualified, and the feedback information is to adjust the action, which indicates that the trainee has deviation in this part of the action, and the action needs to be corrected and improved according to the feedback information.

[0046] For example, a threshold value of 0.5 is preset for the difference value of the core capture point, and 0.3 is preset for the difference value of the secondary capture point. After the trainee completes a movement, the final difference value of the core capture point is calculated to be 0.6, and the final difference value of the secondary capture point is calculated to be 0.2. Comparing the difference value of 0.6 for the core capture point with the threshold value of 0.5, 0.6 > 0.5, so the movement corresponding to the core capture point is qualified, and the feedback information is to maintain the movement; comparing the difference value of 0.2 for the secondary capture point with the threshold value of 0.3, 0.2 < 0.3, so the movement corresponding to the secondary capture point is unqualified, and the feedback information is to adjust the movement.

[0047] S502. Based on the generated layered feedback, set the corresponding tactile feedback mode and provide tactile feedback through the wearable device. The tactile feedback mode includes: for qualified actions, the tactile feedback mode is a gentle and short vibration feedback, while for unqualified actions, the tactile feedback mode is a strong and continuous vibration feedback.

[0048] Specifically, in providing haptic feedback through wearable devices, it should be noted that these wearable devices can be smart bracelets, smart belts, smart insoles, etc.; these devices have built-in haptic feedback components such as vibration motors. After the haptic feedback mode is set, the system sends feedback commands to the wearable device, which then controls the vibration motor to vibrate according to the corresponding mode. For example, for gentle, short vibration feedback, the vibration motor is controlled to vibrate at a lower intensity for a shorter duration; for strong, continuous vibration feedback, the vibration motor is controlled to vibrate at a higher intensity for a longer duration.

[0049] Example 2 like Figure 2 As shown, the multi-dimensional data-driven student practice performance feedback system, applied to a multi-dimensional data-driven student practice performance feedback method, specifically includes: The database building module is used to create a library of expert standard movements for different practice projects, containing multiple project types. The identifier definition module is used to count the joint activity frequency of all standard movements of the current training project from the expert standard movement library, and define core capture points and secondary capture points according to the joint activity frequency, and assign weights to the core capture points and secondary capture points respectively. The hierarchical processing module is used to synchronously collect trainee joint point data through sensors and mark the core capture points and secondary capture points. Based on the weights assigned to the core capture points and secondary capture points, the module performs hierarchical processing on the data of the core capture points and secondary capture points to obtain the trajectories of the core capture points and secondary capture points. The comparative analysis module is configured to extract a standard action fixed-point track of the current project type from the expert standard action library, compare the core capture point track and the secondary capture point track with the standard action fixed-point track respectively, and calculate a difference value; The feedback interaction module is configured to generate hierarchical feedback according to the calculated difference value, and provide real-time tactile feedback to the learner through the wearable device.

[0050] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0051] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage, a magnetic tape storage, or any other computer readable medium capable of carrying or storing data.

[0052] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without further restriction, preclude the existence of additional elements of the same name in the process, method, article, or apparatus.

Claims

1. A method for providing feedback on student practice effectiveness based on multi-dimensional data-driven approaches, characterized in that: The feedback method comprises the following steps: S1, for different exercise projects, an expert standard action library containing multiple project types is established; S2, according to the project type of the current exercise of the student, the joint activity frequency of all standard actions of the project type is counted from the expert standard action library, and the core capture point and the secondary capture point are defined according to the joint activity frequency, and the core capture point and the secondary capture point are respectively assigned weights; S3, the joint data of the student is synchronously collected by the sensor, and the core capture point and the secondary capture point are marked, and the data of the core capture point and the data of the secondary capture point are processed in stages according to the weights assigned to the core capture point and the secondary capture point, so as to obtain the core capture point trajectory and the secondary capture point trajectory; S4, the standard action fixed point trajectory of the current project type is extracted from the expert standard action library, the core capture point trajectory and the secondary capture point trajectory are compared with the standard action fixed point trajectory respectively, and the difference value is calculated; S5, hierarchical feedback is generated according to the calculated difference value, and real-time tactile feedback is given to the student through the wearable device.

2. The multi-dimension data driven student practice effectiveness feedback method of claim 1, wherein, The specific steps of step S1 are: S101, the exercise project types to be covered are determined, and the standard action names corresponding to each project are listed; S102, the three-dimensional coordinate data of the joints of the expert standard action is collected by an inertial sensor or an optical capture system device, and the standard action data of each project is obtained; S103, the standard action data of each project is stored according to the project type to construct an expert standard action library, wherein each standard action data includes a joint trajectory sequence, an action duration and a key frame mark.

3. The multi-dimension data driven student practice effectiveness feedback method of claim 1, wherein, The specific steps of step S2 are: S201, the project type of the current exercise of the student is determined, and all standard action data under the project type of the current exercise of the student is extracted from the expert standard action library; S202, all standard action data under the project type of the current exercise of the student is analyzed, which specifically includes analyzing and counting the joint activity frequency of the student's body in completing the standard action; S203, according to the joint activity frequency obtained by statistics, the joint activity frequency is arranged in descending order, the joints corresponding to the top 30% joint activity frequency in the descending order are defined as the core capture point, and the joints corresponding to the last 30% joint activity frequency in the descending order are defined as the secondary capture point; S204, the weight of the core capture point is set to 0.7, and the weight of the secondary capture point is set to 0.

3.

4. The multi-dimension data driven student practice effectiveness feedback method of claim 1, wherein, The specific steps of step S3 are: S301, the three-dimensional coordinate data of the joints of the student in the exercise process is synchronously collected by the sensor; S302, based on the three-dimensional coordinate data of the joints, the three-dimensional coordinate data of the corresponding joints is respectively marked according to the defined core capture point and secondary capture point, and the marked core capture point data and secondary capture point data are obtained; S303、According to the weight of the core capture point setting and the weight of the secondary capture point setting, the labeled core capture point data and the secondary capture point data are processed in a hierarchical manner, wherein the hierarchical processing specifically includes that the core capture point data is processed in a high-precision filtering and time alignment manner, and the secondary capture point data is processed in a light denoising manner, so as to respectively obtain the core capture point trajectory and the secondary capture point trajectory.

5. The multi-dimension data driven student practice effectiveness feedback method of claim 1, wherein, The specific steps of the step S4 are: S401, extracting the standard action fixed-point trajectory of the current practice project type of the student from the expert standard action library, and determining the comparison time point; S402, using the comparison method of calculating the distance between two points, comparing and calculating the obtained core capture point trajectory and secondary capture point trajectory with the corresponding joint node trajectory in the standard action fixed-point trajectory respectively, and obtaining the difference value obtained by comparing and calculating each joint node; S403, according to the weight of the core capture point setting and the weight of the secondary capture point setting, and through the weighted summation calculation method, the difference values obtained by comparing and calculating each joint node are comprehensively calculated to obtain the final difference value of the core capture point and the final difference value of the secondary capture point respectively.

6. The multi-dimensional data-driven based student practice effectiveness feedback method of claim 1, wherein, The specific steps of the step S5 are: S501, setting a difference value threshold in advance, and comparing the final difference value of the core capture point and the final difference value of the secondary capture point with the set difference value threshold respectively, and generating hierarchical feedback according to the comparison result, wherein the hierarchical feedback includes that if the final difference value exceeds the set difference value threshold, the action is qualified, and the feedback information is to maintain the action; if the final difference value does not exceed the set difference value threshold, the action is unqualified, and the feedback information is to adjust the action; S502, according to the generated hierarchical feedback, setting the corresponding haptic feedback mode, and performing haptic feedback through the wearable device, wherein the haptic feedback mode includes: for the qualified action, the haptic feedback mode is soft and short vibration feedback, and for the unqualified action, the haptic feedback mode is strong and continuous vibration feedback.

7. A student practice effect feedback system based on multi-dimension data driving, applying the student practice effect feedback method based on multi-dimension data driving as claimed in any one of claims 1-6, characterized in that, The feedback system comprises: A database construction module is configured to establish an expert standard action library containing multiple project types for different practice projects; An identification definition module is configured to count the joint node activity frequency of all standard actions of the project type from the expert standard action library according to the project type currently practiced by the student, and define the core capture point and the secondary capture point according to the joint node activity frequency, and assign weights to the core capture point and the secondary capture point respectively; A hierarchical processing module is configured to synchronously collect the joint node data of the student through a sensor, mark the core capture point and the secondary capture point, and process the data of the core capture point and the secondary capture point according to the weights of the core capture point and the secondary capture point to obtain the core capture point trajectory and the secondary capture point trajectory; A comparison and analysis module is configured to extract the standard action fixed-point trajectory of the current project type from the expert standard action library, compare the core capture point trajectory and the secondary capture point trajectory with the standard action fixed-point trajectory respectively, and calculate the difference value. The feedback interaction module is configured to generate hierarchical feedback according to the calculated difference value, and to provide real-time tactile feedback to the learner through the wearable device.

Images