Tai Chi teaching system based on motion perception

Abstract

The invention belongs to the technical field of Tai Chi teaching, and particularly relates to a Tai Chi teaching system based on motion perception, which comprises a video acquisition module, a central processing module and an interaction module. A central processing module of the system performs action recognition on a user video by adopting an OvR strategy CNN fusion full-connection neural network algorithm, and performs secondary judgment and gives evaluation suggestions in combination with a Tai Chi teaching rule base. The accuracy of the deep learning model is improved by priori knowledge introduced by the rule base, and the expandability and interpretability of the program are further improved while the action recognition accuracy is improved. The hardware with low cost and low volume improves the economical efficiency and portability of the whole product, effectively solves the problem that the existing Tai Chi teaching system is expensive and heavy, and enables the old to learn Tai Chi anytime and anywhere at low cost.

Description

Technical Field

[0001] The invention belongs to the field of teaching systems, and specifically refers to a Tai Chi teaching system based on motion perception. Background Art

[0002] With the aging of the population, there are more and more elderly people. Many elderly people pay great attention to physical exercise, and they are particularly fond of Tai Chi, because practicing Tai Chi can cultivate one's body, cultivate one's temperament, and strengthen one's body, which plays an extremely important role in promoting human physical and mental health. However, there are two main ways to learn Tai Chi. The first is to learn with a group or a teacher indoors or outdoors, and the learning time is limited. The second is to learn through videos indoors through computers and the Internet, but it is difficult to get feedback from the videos, and the teaching equipment that can provide feedback is generally too heavy and costly.

[0003] How to enable the elderly to learn Tai Chi and get feedback anytime and anywhere at low cost is of great significance. Summary of the invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a Tai Chi teaching system based on motion perception. The adoption of lightweight streamlined hardware design and rule library-assisted OvR strategy CNN fusion fully connected neural network algorithm effectively solves the high cost, heavy weight, feedback and recognition deficiencies of the existing Tai Chi teaching system. The prior knowledge introduced by the rule library improves the accuracy of the deep learning model, improves the accuracy of motion recognition, and further improves the scalability and interpretability of the program. The flexible algorithm structure design can adapt to different hardware scales and launch different models to meet the needs of different consumer levels by adjusting the number of CNN-based learners according to different hardware performance and teaching needs.

[0005] The Tai Chi teaching system based on motion perception includes a video acquisition module, a central processing module and an interaction module;

[0006] The video acquisition module and the interaction module are electrically connected to the central processing module respectively;

[0007] The interactive module includes a touch screen, a speaker and a Tai Chi teaching knowledge material library, which includes Tai Chi teaching videos with marked movement nodes and Tai Chi teaching theoretical knowledge;

[0008] The action nodes are key time points for performing standard Tai Chi actions in Tai Chi teaching videos;

[0009] The action set corresponding to the action node includes a standard Tai Chi action state and all error action states corresponding to the standard Tai Chi action state;

[0010] The video acquisition module acquires user videos in real time and transmits them to the central processing module;

[0011] The central processing module uses the OvR strategy CNN fusion fully connected neural network algorithm to perform action recognition on the user video. The central processing module builds a Tai Chi teaching rule library based on the Tai Chi teaching knowledge material library, and uses the Tai Chi teaching rule library to give judgment guidance and action evaluation opinions on the recognition results of the OvR strategy CNN fusion fully connected neural network algorithm;

[0012] The OvR strategy CNN fusion fully connected neural network algorithm structure consists of an OvR strategy CNN part, a second fusion layer and a three-layer fully connected neural network. The structure of the OvR strategy CNN part consists of N CNN base learners and a first fusion layer.

[0013] The structure of the CNN-based learner is composed of three layers of two-dimensional convolutional layers and two layers of fully connected layers. The recognition result is the action state probability of the current user performing the action corresponding to the CNN-based learner, which is converted into a total set of action probabilities after being spliced ​​by the first fusion layer;

[0014] The Tai Chi teaching rule library includes action evaluation opinions corresponding to each action state and a judgment condition set for providing judgment guidance opinions based on the total set of action probabilities.

[0015] The process of the central processing module performing action recognition on the user video specifically includes the following steps:

[0016] Step S1: training the OvR strategy CNN fusion fully connected neural network algorithm;

[0017] Step S2: deploying the OvR strategy CNN fusion fully connected neural network algorithm on the central processing module;

[0018] Step S3: the video acquisition module inputs the user video to the central processing module in real time;

[0019] Step S4: the central processing module samples each action node in the user video once to generate a key frame, and the central processing module inputs the key frame into the OvR strategy CNN part;

[0020] Step S5: the CNN base learner in the OvR strategy CNN part processes the key frames independently and splices them in the first fusion layer to obtain the output of the OvR strategy CNN part, and then fills the output of the OvR strategy CNN part with 0 and splices it in the second fusion layer to obtain the splicing result;

[0021] Step S6: The splicing result is passed into the three-layer fully connected neural network, and the total set of action probabilities is output to the Tai Chi teaching rule library;

[0022] Step S7: The Tai Chi teaching rule library searches for the corresponding action set in the action probability total set according to the current action node, and traverses the action state probability in the action set;

[0023] Step S8: When the probability of at least one of the traversed action states is higher than the threshold, the Tai Chi teaching rule library selects an action state with the highest probability and gives a corresponding action evaluation opinion to the interactive module; otherwise, the Tai Chi teaching rule library gives a judgment guidance opinion according to the judgment condition corresponding to the total probability set;

[0024] Step S9: the central processing module performs sentence vectorization on the judgment guidance opinions to generate a hint vector to the second fusion layer for a second judgment;

[0025] Step S10: When the probability of each action state in the action set corresponding to the current action node in the total action probability set in the second judgment result is still lower than the threshold, the highest action state probability is set to 1 and the corresponding action evaluation opinion is given to the interaction module according to its corresponding action state. Otherwise, the Tai Chi teaching rule library selects the action state with the highest probability in the current action set and gives the corresponding action evaluation opinion to the interaction module.

[0026] Further, in step S1, training the OvR strategy CNN fusion fully connected neural network algorithm includes the following steps:

[0027] Step S11: extract key frames from the Tai Chi teaching video according to the action nodes, label the action states with one-hot coding, and prepare a training picture set;

[0028] Step S12: Use the training picture set to train the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm, and freeze some parameters of the OvR strategy CNN after the training is completed;

[0029] Step S13: Combine the key frame samples in the training picture set with the zero-filled input for data enhancement and combine them with the hint vectors corresponding to the action state labels to form an enhanced data set; combine the OvR strategy CNN part after the parameters are frozen with the second fusion layer and the three-layer fully connected neural network;

[0030] Step S14: Use the enhanced data set to train the OvR strategy CNN fusion fully connected neural network algorithm. Due to parameter freezing, the OvR strategy CNN part is not affected at this time;

[0031] Step S15: Obtain the OvR strategy CNN fusion fully connected neural network algorithm after training.

[0032] Further, in step S12, the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm is trained, specifically including the following steps:

[0033] Step S121: define N CNN-based learners according to N action states. According to different hardware performance and teaching requirements, the value range of N is 24-240. The task of each CNN-based learner is to perform binary classification learning on the action state corresponding to itself. The output of the CNN-based learner is the action state probability of the action state corresponding to this CNN-based learner.

[0034] Step S122: split the training picture set according to the action state labels and input them into each CNN base learner for training;

[0035] Step S123: concatenate the outputs of each CNN base learner in the first fusion layer to obtain the OvR strategy CNN part that can output the total set of action probabilities.

[0036] The beneficial effects of the present invention are as follows:

[0037] (1) The introduction of prior knowledge improves the accuracy of the deep learning model, which improves the accuracy of motion recognition and further improves the scalability and interpretability of the program. The low-cost and low-volume hardware improves the overall economy and portability of the product, effectively solving the problem of the expensive and bulky existing Tai Chi teaching system, allowing the elderly to learn Tai Chi anytime and anywhere at a low cost.

[0038] (2) The pioneering use of sentence vectorization technology enables the rule base to assist the deep learning algorithm in making secondary judgments, introduces prior knowledge, and further improves the accuracy of action recognition.

[0039] (3) The movement evaluation opinions in the rule library, which are pre-prepared based on Tai Chi teaching theory knowledge, provide accurate and easy-to-understand learning feedback for the elderly.

[0040] (4) According to the hardware performance and teaching needs, the number of CNN-based learners can be adjusted to adapt to different hardware scales, and different models can be launched to meet the needs of different consumer levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a schematic diagram of the motion recognition principle of the Tai Chi teaching system based on motion perception proposed by the present invention.

[0042] Figure 2 This is a schematic diagram of the CNN-based learner of the Tai Chi teaching system based on motion perception proposed in the present invention.

[0043] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the examples given are only used to explain the present invention, and are not used to limit the scope of the present invention.

[0045] Embodiment 1, the present invention provides a Tai Chi teaching system based on motion perception, including a video acquisition module, a central processing module and an interaction module;

[0046] The video acquisition module and the interaction module are electrically connected to the central processing module respectively;

[0047] The interactive module includes a touch screen, a speaker and a Tai Chi teaching knowledge material library, which includes Tai Chi teaching videos with marked movement nodes and Tai Chi teaching theoretical knowledge;

[0048] The action nodes are key time points for performing standard Tai Chi actions in Tai Chi teaching videos;

[0049] The action set corresponding to the action node includes a standard Tai Chi action state and all error action states corresponding to the standard Tai Chi action state;

[0050] The video acquisition module acquires user videos in real time and transmits them to the central processing module;

[0051] The central processing module uses the OvR strategy CNN fusion fully connected neural network algorithm to perform action recognition on the user video. The central processing module builds a Tai Chi teaching rule library based on the Tai Chi teaching knowledge material library, and uses the Tai Chi teaching rule library to give judgment guidance and action evaluation opinions on the recognition results of the OvR strategy CNN fusion fully connected neural network algorithm;

[0052] The OvR strategy CNN fusion fully connected neural network algorithm structure consists of an OvR strategy CNN part, a second fusion layer and a three-layer fully connected neural network. The structure of the OvR strategy CNN part consists of N CNN base learners and a first fusion layer.

[0053] The structure of the CNN-based learner is composed of three layers of two-dimensional convolutional layers and two layers of fully connected layers. The recognition result is the action state probability of the current user performing the action corresponding to the CNN-based learner, which is converted into a total set of action probabilities after being spliced ​​by the first fusion layer;

[0054] The Tai Chi teaching rule library includes action evaluation opinions corresponding to each action state and a judgment condition set for providing judgment guidance opinions based on the total set of action probabilities.

[0055] Example 2, see Figure 2 This embodiment is based on the above embodiment. The structure of the CNN-based learner consists of three layers of two-dimensional convolutional layers and two layers of fully connected layers. The recognition result is the action state probability of the current user making the corresponding action of the CNN-based learner, which becomes the total set of action probabilities after being spliced ​​by the first fusion layer.

[0056] Example 3, see Figure 1 This embodiment is based on the above embodiment, and the process of the central processing module performing action recognition on the user video includes the following steps:

[0057] Step S1: training the OvR strategy CNN fusion fully connected neural network algorithm;

[0058] Step S2: deploying the OvR strategy CNN fusion fully connected neural network algorithm on the central processing module;

[0059] Step S3: the video acquisition module inputs the user video to the central processing module in real time;

[0060] Step S4: the central processing module samples each action node in the user video once to generate a key frame, and the central processing module inputs the key frame into the OvR strategy CNN part;

[0061] Step S5: the CNN base learner in the OvR strategy CNN part processes the key frames independently and splices them in the first fusion layer to obtain the output of the OvR strategy CNN part, and then fills the output of the OvR strategy CNN part with 0 and splices it in the second fusion layer to obtain the splicing result;

[0062] Step S6: The splicing result is passed into the three-layer fully connected neural network, and the total set of action probabilities is output to the Tai Chi teaching rule library;

[0063] Step S7: The Tai Chi teaching rule library searches for the corresponding action set in the action probability total set according to the current action node, and traverses the action state probability in the action set;

[0064] Step S8: If the probability of at least one of the traversed action states is higher than the threshold, the Tai Chi teaching rule library selects the action state with the highest probability and gives the corresponding action evaluation opinion to the interactive module. Otherwise, the Tai Chi teaching rule library gives the judgment guidance opinion according to the judgment condition corresponding to the total probability set.

[0065] Step S9: the central processing module performs sentence vectorization on the judgment guidance opinions to generate a hint vector to the second fusion layer for a second judgment;

[0066] Step S10: If the probability of each action state in the action set corresponding to the current action node in the total action probability set in the second judgment result is still lower than the threshold, the highest action state probability is set to 1 and the corresponding action evaluation opinion is given to the interaction module according to its corresponding action state, otherwise the Tai Chi teaching rule library selects the action state with the highest probability in the current action set and gives the corresponding action evaluation opinion to the interaction module;

[0067] Embodiment 4, based on the above embodiment, in step S1, training the OvR strategy CNN fusion fully connected neural network algorithm includes the following steps:

[0068] Step S11: extract key frames from the Tai Chi teaching video according to the action nodes, label the action states with one-hot coding, and prepare a training picture set;

[0069] Step S12: Use the training picture set to train the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm, and freeze some parameters of the OvR strategy CNN after the training is completed;

[0070] Step S13: Perform data enhancement on the samples in the training picture set and the zero-filled input or on the hint vectors corresponding to the action state labels to form an enhanced data set; combine the OvR strategy CNN part after parameter freezing with the second fusion layer and the three-layer fully connected neural network;

[0071] Step S14: Use the enhanced data set to train the OvR strategy CNN fusion fully connected neural network algorithm. Due to parameter freezing, the OvR strategy CNN part is not affected at this time;

[0072] Step S15: Obtain the OvR strategy CNN fusion fully connected neural network algorithm after training.

[0073] Embodiment 5, this embodiment is based on the above embodiment. In step S12, the number of CNN base learners is 24, and the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm is trained, specifically including the following steps:

[0074] Step S121: 24 CNN-based learners are defined according to the 24 action states. The minimum 24 CNN-based learners are sufficient to run in most of the poor hardware environments and still meet the minimum recognition and feedback of the 24-style simplified Tai Chi. The task of each CNN-based learner is to perform binary classification learning on the action state corresponding to itself. The output of the CNN-based learner is the action state probability of the user video corresponding to the action state of this CNN-based learner.

[0075] Step S122: split the training picture set according to the action state labels and input them into each CNN base learner for training;

[0076] Step S123: concatenate the outputs of each CNN base learner in the first fusion layer to obtain the OvR strategy CNN part that can output the total set of action probabilities.

[0077] Embodiment 6, this embodiment is based on Embodiment 1 to Embodiment 4. In step S12, the number of CNN base learners is 240, and the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm is trained, specifically including the following steps:

[0078] Step S121: define 240 CNN-based learners according to 24*10 action states. The maximum 240 CNN-based learners are sufficient to identify any situation actually encountered in Tai Chi teaching, laying a good foundation for the subsequent further processing. The task of each CNN-based learner is to perform binary classification learning on the action state corresponding to itself. The output of the CNN-based learner is the action state probability of the user video corresponding to the action state of this CNN-based learner.

[0079] Step S122: split the training picture set according to the action state labels and input them into each CNN base learner for training;

[0080] Step S123: concatenate the outputs of each CNN base learner in the first fusion layer to obtain the OvR strategy CNN part that can output the total set of action probabilities.

[0081] Example 7. This example is based on the above example. This example runs in the Windows operating system environment. The experiment relies on the Thonny environment and uses Scikit and TensorFlow as the algorithm framework. For the data processing stage, the Pandas library is used in combination with the Numpy tool to complete data preprocessing, and the original data is converted into a matrix format that is easy to analyze. The optimizer uses Adam.

[0082] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0083] The present invention and its implementation are described above, and such description is not restrictive. The drawings show only one implementation of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by the present invention, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should fall within the protection scope of the present invention.

Claims

1. A Tai Chi teaching system based on motion perception, characterized in that: The Tai Chi teaching system based on motion perception includes: a video acquisition module, a central processing module and an interaction module; The video acquisition module and the interaction module are electrically connected to the central processing module respectively; The interactive module has a built-in Tai Chi teaching knowledge material library, which contains Tai Chi teaching theoretical knowledge and Tai Chi teaching videos with marked movement nodes; The action nodes are key time points for performing standard Tai Chi actions in Tai Chi teaching videos; The action set corresponding to the action node includes a standard Tai Chi action state and an error action state; The video acquisition module acquires user videos in real time and transmits them to the central processing module; The central processing module uses the OvR strategy CNN fusion fully connected neural network algorithm to perform action recognition on the user video. The central processing module builds a Tai Chi teaching rule library based on the Tai Chi teaching knowledge material library, and uses the Tai Chi teaching rule library to give judgment guidance and action evaluation opinions on the recognition results of the OvR strategy CNN fusion fully connected neural network algorithm; The OvR strategy CNN fusion fully connected neural network algorithm structure consists of an OvR strategy CNN part, a second fusion layer and a three-layer fully connected neural network. The structure of the OvR strategy CNN part consists of N CNN base learners and a first fusion layer. The recognition result of the CNN-based learner is the action state probability of the current user performing the action corresponding to the CNN-based learner, which becomes the total set of action probabilities after being spliced ​​by the first fusion layer; The Tai Chi teaching rule library includes action evaluation opinions corresponding to each action state and a judgment condition set for providing judgment guidance opinions based on the total set of action probabilities.

2. A Tai Chi teaching system based on motion perception according to claim 1, characterized in that: The process of the central processing module performing action recognition on the user video includes the following steps: Step S1: training the OvR strategy CNN fusion fully connected neural network algorithm; Step S2: deploying the OvR strategy CNN fusion fully connected neural network algorithm on the central processing module; Step S3: the video acquisition module inputs the user video to the central processing module in real time; Step S4: the central processing module samples each action node in the user video once to generate a key frame, and the central processing module inputs the key frame into the OvR strategy CNN part; Step S5: the CNN base learner in the OvR strategy CNN part processes the key frames independently and splices them in the first fusion layer to obtain the output of the OvR strategy CNN part, and then fills the output of the OvR strategy CNN part with 0 and splices it in the second fusion layer to obtain the splicing result; Step S6: The splicing result is passed into the three-layer fully connected neural network, and the total set of action probabilities is output to the Tai Chi teaching rule library; Step S7: The Tai Chi teaching rule library searches for the corresponding action set in the action probability total set according to the current action node, and traverses the action state probability in the action set; Step S8: When the probability of at least one of the traversed action states is higher than the threshold, the Tai Chi teaching rule library selects an action state with the highest probability and gives a corresponding action evaluation opinion to the interactive module; otherwise, the Tai Chi teaching rule library gives a judgment guidance opinion according to the judgment condition corresponding to the total probability set; Step S9: the central processing module performs sentence vectorization on the judgment guidance opinions to generate a hint vector to the second fusion layer for a second judgment; Step S10: When the probability of each action state in the action set corresponding to the current action node in the total action probability set in the second judgment result is still lower than the threshold, the highest action state probability is set to 1 and the corresponding action evaluation opinion is given to the interaction module according to its corresponding action state. Otherwise, the Tai Chi teaching rule library selects the action state with the highest probability in the current action set and gives the corresponding action evaluation opinion to the interaction module.

3. A Tai Chi teaching system based on motion perception according to claim 2, characterized in that: In step S1, training the OvR strategy CNN fusion fully connected neural network algorithm includes the following steps: Step S11: extract key frames from the Tai Chi teaching video according to the action nodes, label the action states with one-hot coding, and prepare a training picture set; Step S12: Use the training picture set to train the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm, and freeze some parameters of the OvR strategy CNN after the training is completed; Step S13: Combine the key frame samples in the training picture set with the zero-filled input for data enhancement and combine them with the hint vectors corresponding to the action state labels to form an enhanced data set; combine the OvR strategy CNN part after the parameters are frozen with the second fusion layer and the three-layer fully connected neural network; Step S14: Use the enhanced data set to train the OvR strategy CNN fusion fully connected neural network algorithm. Due to parameter freezing, the OvR strategy CNN part is not affected at this time; Step S15: Obtain the OvR strategy CNN fusion fully connected neural network algorithm after training.

4. The Tai Chi teaching system based on motion perception according to claim 3 is characterized in that: In step S12, the OvR strategy CNN part in the OvR strategy CNN fusion fully connected neural network algorithm is trained, specifically including the following steps: Step S121: define N CNN-based learners according to N action states. According to different hardware performance and teaching requirements, the value range of N is 24-240. The task of each CNN-based learner is to perform binary classification learning on the action state corresponding to itself. The output of the CNN-based learner is the action state probability of the action state corresponding to this CNN-based learner. Step S122: split the training picture set according to the action state labels and input them into each CNN base learner for training; Step S123: concatenate the outputs of each CNN base learner in the first fusion layer to obtain the OvR strategy CNN part that can output the total set of action probabilities.

Images