Digital evaluation method and device for shadowboxing training actions

Abstract

The invention provides a digital evaluation method and device for shadowboxing training actions, and belongs to the field of image recognition, and the method comprises the steps: extracting time sequence-based 2D skeleton node data of a plurality of preset skeleton nodes in a to-be-evaluated shadowboxing image; performing depth expansion on the 2D skeleton node data to obtain a 3D skeleton node matrix; according to the 3D skeleton node matrix, determining space geometric features and time motion features of preset skeleton nodes; classifying the spatial geometric features and the time motion features through RF to obtain a classification result of each action; and based on the action classification result, determining a deviation degree between each action and a preset standard action, and determining an evaluation result of the to-be-evaluated shadowboxing image according to the deviation degree. In this way, the Tai Chi actions are evaluated by extracting the action data of the key skeleton nodes, the evaluation result of the Tai Chi actions can be quantified, the accuracy and reliability of evaluation are improved, and a scientific basis is provided for teaching and training of Tai Chi.

Description

Technical Field

[0001] The invention belongs to the field of image recognition, and in particular relates to a digital evaluation method and device for Tai Chi training movements. Background Art

[0002] Tai Chi is a traditional martial art with profound cultural heritage and philosophical thoughts. It is rapidly gaining popularity among the younger generation through the maturity of sports digital technology. With the help of online courses, social media live broadcasts, virtual reality experiences and other forms, Tai Chi breaks the limitations of geography and time, allowing more people, especially young people, to learn conveniently. Its advantages in fitness, such as strengthening physical fitness, improving flexibility, and improving cardiopulmonary function, have attracted a large number of young people who pursue a healthy life. Tai Chi is no longer limited to the middle-aged and elderly groups, but has crossed age boundaries and become a fashionable lifestyle.

[0003] Tai Chi is soft, gentle, slow and even, with orderly opening and closing, light and flexible, and a combination of hardness and softness. However, it is often difficult for learners to grasp the key points of Tai Chi through videos alone, and they cannot understand their own mastery of Tai Chi. Therefore, how to evaluate the learning of Tai Chi has become an important issue.

[0004] At present, people who practice Tai Chi mainly rely on their own feelings to determine whether the movements are in place, which does not meet the standards of scientific and effective fitness. For beginners, it is difficult to master the correct posture and rhythm of the movements through self-feeling. Without the coach's repeated reminders, beginners cannot correctly judge whether the postures are in place and the rhythm is accurate.

[0005] With the development of artificial intelligence technology, theoretical research and achievements in human motion recognition based on image and video processing continue to emerge. However, a notable feature of Tai Chi is the slow and seamless transition between postures. Different people and learning stages may bring differences in time and motion. Different skill levels, heights, weights, and gaits also vary, resulting in variations in the performance of each subject, leading to technical difficulties in Tai Chi motion recognition. At the same time, existing research on human motion evaluation is still in its infancy, especially for complex movements, such as Tai Chi and traditional arts such as Peking Opera, which currently lack widely recognized motion evaluation indicators. Summary of the invention

[0006] In order to solve the problem of Tai Chi movement evaluation, the present invention provides a digital evaluation method and device for Tai Chi training movements.

[0007] In order to achieve the above object, the present invention provides the following technical solutions: Firstly, a digital evaluation method for Tai Chi training movements is provided, the method comprising: Extracting 2D bone node data based on time series of multiple preset bone nodes of the target object to be evaluated in the Tai Chi image; Performing geometric projection on the 2D bone node data to determine the bone length, and performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix; A feature vector is constructed according to the positional relationship of adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone nodes are determined according to the vector inclination of the feature vector; the temporal motion features of each preset bone node are determined based on the position change of the preset bone nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; The spatial geometric features and temporal motion features are classified by using a random forest algorithm RF to obtain a classification result for each action; Based on the action classification result, the deviation of each action from the preset standard action is determined, and the evaluation result of the Tai Chi training action of the target object to be evaluated is determined according to the deviation.

[0008] Optionally, geometrically projecting the 2D bone node data to determine the bone length, and then deeply expanding the 2D bone node data based on the bone length to obtain a 3D bone node matrix includes: Perform geometric projection on the 2D skeleton node data in a single frame, determine the distance between adjacent nodes in a single frame, and form a single frame node distance set; Traverse the multi-frame node distance set and determine the maximum value of the distance between two adjacent nodes as the bone length, thereby constructing the bone length matrix; Construct a projection triangle based on the single-frame node distance set and the bone length to obtain the depth value of the adjacent nodes; The 2D bone node data is expanded into 3D using the adjacent node depth values ​​to obtain a 3D bone node matrix.

[0009] Optionally, after obtaining the 3D skeleton node matrix, the method further includes: Determine a first-order two-dimensional difference matrix of multiple frames of 3D skeleton node data, and perform outlier screening based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix; For the missing data that has been filtered out, the mean value of the corresponding preset bone nodes of the adjacent frames before and after is obtained for filling.

[0010] Optionally, constructing a feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, and determining the spatial geometric features of the preset bone node according to the vector inclination of the feature vector includes: Constructing a feature vector based on two adjacent nodes in the 3D skeleton node matrix, and determining the coordinates of the feature vector based on the coordinates of the two adjacent nodes; The inclination angles of the feature vector in the depth direction, the horizontal direction and the vertical direction are determined respectively, and the inclination angles are used as the spatial geometric features of two adjacent preset bone nodes.

[0011] Optionally, the classifying the spatial geometric features and the temporal motion features by RF to obtain a classification result for each action includes: The spatial geometric features and the temporal motion features are optimized by dimensionality reduction through RF, and the target motion features are extracted according to the importance; the target motion features include spatial geometric features and temporal motion features; A decision tree classifier is generated based on RF, and the target motion features are classified into action categories by the decision tree classifier to obtain a classification result for each action.

[0012] Optionally, before performing dimension reduction optimization on the spatial geometric features and the temporal motion features by using RF, feature sequence recognition is performed on the spatial geometric features and the temporal motion features by using a DTW algorithm, including: Determine the Euclidean distance between the spatial geometric feature and the temporal motion feature and the preset standard feature respectively through DTW; Based on the Euclidean distance, feature sequence recognition is performed on each frame of the Tai Chi image, and each frame is standardized on the time axis based on the recognition result.

[0013] Optionally, based on the action classification result, determining the deviation of each action from a preset standard action, and determining the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation includes: Obtaining a deviation index for each action, and determining a mean of multiple deviation indexes and a standard deviation of the deviation index for each action; Determine a coefficient of variation corresponding to each action based on the mean and standard deviation; Based on the coefficient of variation, a weighted sum of the deviation index of each action is performed to obtain a comprehensive score; The evaluation result of the Tai Chi training action of the target object to be evaluated is determined according to the comprehensive score.

[0014] Secondly, a digital evaluation device for Tai Chi training movements is provided, the device comprising: A node extraction module is used to extract 2D bone node data based on time series of multiple preset bone nodes of the target object to be evaluated in the Tai Chi image; geometrically project the 2D bone node data to determine the bone length, and then deeply expand the 2D bone node data based on the bone length to obtain a 3D bone node matrix; A feature extraction module, used to construct a feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, and determine the spatial geometric features of the preset bone nodes according to the vector inclination of the feature vector; determine the temporal motion features of each preset bone node based on the position change of the preset bone node between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; A classification module, used for classifying the spatial geometric features and the temporal motion features through RF to obtain a classification result for each action; The evaluation module is used to determine the deviation of each action from a preset standard based on the action classification result, and determine the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation.

[0015] In addition, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the digital evaluation method of Tai Chi training movements described above is implemented.

[0016] Finally, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned digital evaluation method for Tai Chi training movements when executing the program.

[0017] The digital evaluation method for Tai Chi training movements provided by the present invention has the following beneficial effects: First, the 2D bone node data based on the timing of the preset bone nodes is extracted, so that the position information of the key bone nodes in the Tai Chi movements can be captured to avoid the interference of redundant information such as height and weight. Secondly, the 2D bone node data is converted into 3D data through geometric projection and depth expansion, so as to more realistically reflect the spatial characteristics of the Tai Chi movements, which helps to more accurately evaluate the accuracy and standardization of the movements; then, by constructing feature vectors and calculating vector inclinations, the spatial geometric characteristics of the Tai Chi movements can be fully extracted. By analyzing the position changes of the bone nodes between adjacent frames, the temporal motion characteristics of the movements can also be accurately captured. These features together constitute a complete description of Tai Chi movements, which can accurately complete the recognition of Tai Chi and provide rich information for subsequent evaluation. Then, the extracted features are classified through RF, which divides the complete Tai Chi movements into individual moves. This movement segmentation further improves the recognition effect of Tai Chi. Finally, each movement is evaluated based on the deviation, which can quantify the evaluation results of Tai Chi movements, helping the evaluator to more intuitively understand the advantages and disadvantages of the Tai Chi training movements of the target object to be evaluated, so that targeted improvement suggestions can be put forward, which improves the accuracy and reliability of the evaluation and provides a scientific basis for the teaching and training of Tai Chi. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 The present invention is a flowchart of a digital evaluation method for Tai Chi training movements provided according to an exemplary embodiment of the present invention.

[0020] Figure 2 A yolov8s-pose network structure diagram provided according to an exemplary embodiment of the present invention.

[0021] Figure 3 A human skeleton node model provided by the present invention according to an exemplary embodiment; wherein Figure 3 (a) is the human skeleton node model diagram before optimization. Figure 3 (b) in the figure is the optimized human skeleton node model diagram.

[0022] Figure 4 A schematic diagram of a vector inclination angle provided by the present invention according to an exemplary embodiment; wherein Figure 4 (a) is a schematic diagram of the inclination angle in the depth direction. Figure 4 (b) is a schematic diagram of the horizontal inclination. Figure 4 (c) is a schematic diagram of the inclination angle in the vertical direction.

[0023] Figure 5 A cumulative distance matrix grid diagram is provided according to an exemplary embodiment of the present invention.

[0024] Figure 6 The present invention is a block diagram of a digital evaluation device for Tai Chi training movements provided according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0026] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0027] First, the present invention provides a digital evaluation method for Tai Chi training movements, specifically, Figure 1 As shown, the following steps are included: S101, extracting time-series 2D bone node data of multiple preset bone nodes of a target object to be evaluated in a Tai Chi image.

[0028] In the present invention, a 2D skeleton node extraction algorithm based on yolov8s-pose can be used. Yolov8s-pose is a model in yolov8 that simultaneously realizes target frame detection and human key point detection. It consists of four parts: input, backbone network, neck network and detection head output layer. The network structure is as follows Figure 2 As shown in the figure, the input end adjusts the size of the input image, the backbone network performs pre-training and feature extraction, the neck network obtains richer feature information by fusing features with position information, and the head part performs the final classification based on these features and outputs a three-layer feature map.

[0029] Specifically, a Yolov8s-pose algorithm model can be constructed first, and then the sample images of labeled bone nodes are input into Yolov8s-pose, and training is performed with the goal of minimizing the deviation between the predicted results and the labeled bone nodes to obtain a bone node extraction algorithm.

[0030] Taking the preset 17 skeleton nodes as an example, each key point is represented by pixel coordinates (x, y) in the image, and the corresponding relationship between the serial number and the body part is shown in Table 1.

[0031] Table 1 Preset bone node list

[0032] The image frames in the Tai Chi video are intercepted and input into the 2D bone node extraction algorithm to generate 2D bone node data based on time series. Assume that n frames of RGB images are intercepted, and 17 bone nodes are extracted from each frame. Each node contains two-dimensional position information, y represents width information, and z represents height information, so we get a The two-dimensional bone node matrix P is as follows: ; Among them, p1-p 17 The two-dimensional bone node matrix corresponding to different parts 0-16 respectively. Thus, the two-dimensional bone node data of n frames of image samples is obtained, which includes The node data is as follows: .

[0033] S102, geometrically projecting the 2D bone node data to determine the bone length, and then deeply expanding the 2D bone node data based on the bone length to obtain a 3D bone node matrix.

[0034] Since the human body will inevitably have visual overlap and visual occlusion of bone nodes during movement, it will interfere with the extraction of bone nodes and the subsequent motion recognition and motion quality evaluation based on bone node information. Therefore, 3D bone node reconstruction is needed to improve the quality of bone node extraction by increasing the depth information of the human body.

[0035] In one embodiment, the 2D bone node data in a single frame can be geometrically projected to determine the distance between adjacent nodes in a single frame to form a single-frame node distance set; the multi-frame node distance set is traversed to determine the maximum value of the distance between two adjacent nodes as the bone length, thereby constructing a bone length matrix; a projection triangle is constructed based on the single-frame node distance set and the bone length to obtain the adjacent node depth value; the 2D bone node data is 3D expanded through the adjacent node depth value to obtain a 3D bone node matrix.

[0036] For example, firstly, the distances between adjacent nodes in a single frame are calculated through geometric projection to form a single frame node distance set: ,in , ; in, is the distance between adjacent nodes in a single frame.

[0037] Then traverse the n-frame node distance set and find the maximum distance between two adjacent nodes , this value is the actual length of the bone between these two nodes, thus constructing the bone length matrix: .

[0038] Construct a projection triangle based on the single-frame node distance array and the node distance extreme value to find the depth value of the adjacent nodes : , ; in, is the node distance extreme value, Array of node distances for a single frame.

[0039] Taking the right pelvis (node ​​11) as the reference point, setting the depth value of node 11 to 0, and then calculating the cumulative depth information of each bone node . Combined with the two-dimensional bone node matrix P, we get a 3D bone node matrix containing width, height and depth, as follows: .

[0040] In another embodiment, in order to further improve the accuracy of the data, the 3D bone node data can also be screened. Specifically, the first-order two-dimensional difference matrix of multiple frames of 3D bone node data can be determined, and outlier screening can be performed based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix; for the screened missing data, the mean of the corresponding preset bone nodes of the adjacent frames before and after is obtained for filling.

[0041] For example, according to the kinematic statistical characteristics, the position information of the skeletal nodes of adjacent frames is strongly correlated, so the first-order two-dimensional difference matrix of the node data of n frames is statistically calculated as , in order to remove abnormal data, as follows: .

[0042] in, and They are the two-dimensional bone node matrices of the next frame and the previous frame respectively.

[0043] First, calculate the first-order two-dimensional difference matrix The mathematical expectation EX matrix is ​​as follows: .

[0044] Then calculate the bone nodes The first difference of The standard deviation matrix , the calculation formula is as follows: .

[0045] According to statistics Guidelines, in The data within the interval is considered normal and is retained. Data outside the interval are considered as abnormal data and are removed.

[0046] After the abnormal data is eliminated in the previous step, the missing nodes need to be repaired. Since the data between adjacent frames are continuous, the median is the average value between them. The present invention proposes a three-dimensional median interpolation method based on time series for data repair.

[0047] Assume that the missing node data is the jth node data of the i-th frame , take the jth node of the i-1th frame adjacent to the previous and next and the jth node in the i+1th frame The missing node data is filled with the average value of the data. The specific formula is as follows: ; ; .

[0048] In addition, the human skeleton node model is Figure 3 As shown in (a), considering that different parts of the human body participate in different degrees during exercise, the recognition and evaluation of sports mainly focus on the movement of the limbs and have nothing to do with the head and face. Therefore, the nose, ears and eyes bone nodes (serial numbers 0-4) are removed, and the shoulders are used to abstractly represent the position information of the head.

[0049] According to the characteristics of Tai Chi, the limbs have large fluctuations during the whole movement, while the trunk is relatively stable. Therefore, the present invention adds the midpoints of the two shoulder joints (big vertebra) and the two hip joints (coccyx) to evaluate and analyze the changes of the limbs during the whole movement. Thus, the key nodes are reduced to 14. The optimized human skeleton node model is as follows Figure 3 The optimized skeleton node list is shown in Table 2.

[0050] Table 2 Optimized skeleton node list

[0051] In order to eliminate the influence of human body shape differences on action recognition, normalization processing is performed to obtain a normalized 3D skeleton node dataset.

[0052] S103, extracting the spatial geometric features and temporal motion features of the preset bone node according to the 3D bone node matrix.

[0053] After the human body posture estimation is completed, it is necessary to determine the characteristic parameters for Tai Chi action recognition and action quality evaluation. The present invention extracts the spatial geometric characteristics and temporal motion characteristics of the human body respectively.

[0054] In this step, feature extraction is required as the basis for subsequent Tai Chi movement recognition and evaluation. The extracted feature parameters need to meet the following four requirements: 1. The extracted feature parameters can represent the unique state of the action; 2. The feature parameters are continuous and stable throughout the entire action process, and the amount of calculation is reduced by dimensionality reduction; 3. When the video shooting position and angle change, the extracted features must meet certain invariance; 4. The differences in height and body shape of different people should be considered when extracting feature parameters.

[0055] Specifically, a feature vector can be constructed according to the positional relationship between adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone node can be determined according to the vector inclination of the feature vector; the temporal motion features of each preset bone node can be determined based on the position change of the preset bone node between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time.

[0056] For example, a feature vector can be constructed based on two adjacent nodes in the 3D bone node matrix, and the coordinates of the feature vector can be determined based on the coordinates of the two adjacent nodes; the inclination angles of the feature vector in the depth direction, horizontal direction and vertical direction are determined respectively, and the inclination angle is used as the spatial geometric feature of two adjacent preset bone nodes.

[0057] Spatial geometric relationships are static features, which are reflected in the information directly expressed by the limbs in a single picture, such as node coordinates, distances and angles between nodes, etc. Considering that the body proportions of different individuals are not consistent, the present invention constructs 10 sets of feature vectors according to the positional relationship of adjacent nodes as shown in Table 3.

[0058] Table 3 Spatial geometric characteristics

[0059] The present invention defines the vector inclination as a spatial geometric feature to avoid the influence of individual body shape inconsistency on feature description, such as Figure 4 As shown, the inclination angle is divided into the vertical inclination angle (the angle between the vector connecting nodes 1 and 2 and the vertical direction, as shown in Figure 4 (c) in the figure), horizontal inclination (the angle between the vector connecting nodes 1 and 2 and the horizontal direction, such as Figure 4 (b)) and the depth inclination (the angle between the vector connecting nodes 1 and 2 and the depth direction, as Figure 4 (a) in the figure.

[0060] For another example, suppose the coordinates of nodes 1, 2, and 3 are , , , the vector between nodes is , The vectors in the depth direction, horizontal direction, and vertical direction are represented by unit vectors , and Depth inclination , horizontal inclination and vertical inclination Determined by the following formula: ; ; .

[0061] The temporal motion feature reflects the changes of the human body at consecutive moments, such as the swing speed of certain joints and limbs. The present invention selects the motion acceleration of each skeletal node as the temporal motion feature to reflect the "force" feature in motion. Since the time interval between adjacent frames is fixed, the time interval between adjacent frames of a skeletal node is not necessarily the same as the time interval between adjacent frames. arrive The acceleration formulas on x, y, and z are as follows: ; ; .

[0062] Wherein, t is the interval between adjacent frames. When the video is shot at a frequency of 25 frames per second, t is 0.04s.

[0063] In summary, the motion feature parameters extracted from N frames of RGB images are shown in Table 4: Table 4 Characteristic parameter list

[0064] S104, classifying the spatial geometric features and temporal motion features by using a random forest algorithm RF to obtain a classification result for each action.

[0065] In order to test the effectiveness of the 3D skeleton node reconstruction algorithm and lay the foundation for the subsequent action quality evaluation, the random forest algorithm (RF) is used for feature optimization, dimensionality reduction and multi-classification recognition. Random forest is an ensemble learning algorithm that improves the prediction ability of the model by constructing multiple decision trees. It was proposed by Leo Breiman in 2001. The present invention uses RF to calculate the feature importance to remove features with lower importance, thereby reducing the complexity of human skeleton features, and then establishing a Tai Chi action classification model.

[0066] Specifically, RF can be used to first perform dimensionality reduction optimization on the spatial geometric features and temporal motion features, and then extract the target motion features according to their importance; the target motion features include spatial geometric features and temporal motion features; a decision tree classifier is generated based on RF, and the target motion features are used to classify actions to obtain the classification results of each action.

[0067] Since some features in the constructed human skeleton feature set are strongly correlated, in order to prevent model overfitting and reduce computing time, the random forest algorithm is used for feature dimensionality reduction. RF consists of multiple decision trees, where the root node is where the decision tree starts, the trunk is the decision that needs to be made, and the decision tree divides the node into child nodes according to given conditions by splitting. Each main branch is an option in the decision process, and the smaller branches represent potential results. The leaf node is the final output node. Each tree randomly selects some samples from all samples, and it is a sampling with replacement. The probability that a sample is not selected in N decision trees is , when N is large enough, the probability is The samples that are not selected by a tree are called out-of-bag (OOB) samples of the tree. OOB error can not only evaluate the accuracy of classification, but also serve as an important indicator of features. Assuming that N decision trees are generated, each decision tree contains M features. First, the out-of-bag error of each decision tree is calculated, and the value of the i-th feature variable in the out-of-bag data is randomly changed, and then a new out-of-bag error is calculated. The specific formula is as follows: ; in, is the out-of-bag error after random change of the i-th feature variable; N is the total number of decision trees, For the sample N The probability of not being selected in any decision tree is N When it is large enough, the probability approaches a certain value. is the probability that the i-th sample is not selected in a single sampling. The error increment is obtained by recalculating the OOB error after randomly changing the value of the i-th feature variable in the out-of-bag data. The larger the out-of-bag error (OOB) caused by the change, the more important the i-th feature is. Sort the features by importance, extract the features with the highest importance, and achieve feature optimization and feature dimensionality reduction.

[0068] Each decision tree classifier is defined as , x represents the input feature, Represents independent and identically distributed feature sets, X is the same type of feature, and Y is the classification label set. Each decision tree independently performs classification voting, and finally obtains the classification result through RF voting. The classification process of RF is as follows: (1) Generate a decision tree classifier, the calculation formula is: .

[0069] (2) Generate RF, the calculation formula is: ; in, H Represents the classification result set obtained by RF through voting of multiple decision tree classifiers. h k is the kth decision tree classifier.

[0070] (3) The result with the most votes is taken as the classification result. The calculation formula is: .

[0071] in, I is an indicator function. When the classification label matches the true label, the parameter is 1, otherwise it is 0. t is the number of votes corresponding to the input feature x, The category that receives the most votes.

[0072] In this way, through the above process, the spatial geometric features and temporal motion features can be classified to obtain the classification results of each action, that is, the overall image of Tai Chi is divided into the image of each move.

[0073] In another embodiment, before the spatial geometric features and the temporal motion features are optimized by dimensionality reduction through RF, the spatial geometric features and the temporal motion features are also identified by the DTW algorithm, and the spatial geometric features and the temporal motion features are standardized on the time axis, and then the standardized spatial geometric features and the temporal motion features are optimized by dimensionality reduction through RF. Specifically, the Euclidean distance between the spatial geometric features and the temporal motion features and the preset standard features is determined by DTW, and the preset standard features include preset standard spatial geometric features and preset standard temporal motion features; based on the Euclidean distance, each frame of the Tai Chi image is identified by feature sequence, and each frame is standardized on the time axis based on the identification result, such as mapping the feature sequence of each frame according to the time axis of the preset standard feature.

[0074] Tai Chi routines consist of a series of "moves", each of which is composed of multiple frames of continuous skeleton node data and has continuity in time. It is necessary to extract the "moves" action sequence in Tai Chi routines through action similarity evaluation and establish a standard action feature sequence library. Common methods for solving sequence similarity problems include calculating Euclidean distance, cosine similarity, and dynamic time warping (DTW) algorithm. Since each person's action speed is different, the length of the action sequence obtained is also different. For time sequences of different lengths, the DTW algorithm can more effectively obtain similarity to better perform action sequence recognition. Therefore, the present invention adopts the DTW algorithm to compare sequences.

[0075] The idea of ​​the DTW algorithm is to scale the sequences of different lengths representing the same action on the time axis so that the two sequences have the same length, thereby ensuring the smooth progress of subsequent action recognition and action evaluation. It was first proposed by Sakoe to solve the optimization problem of two time series that are irregular on the time axis. The principle of the DTW algorithm is to re-regularize the two sequences to find the matching path with the shortest distance, and solve the cumulative minimum distance of the two sequences, that is, the true similarity of the two sequences. This matching path is the mapping relationship between the points on the two sequences, which eliminates the differences between the sequences on the time axis.

[0076] The present invention uses the expert feature parameter sequence as the standard sample, that is, the preset standard feature, and the student feature parameter sequence as the test sample. As an example, suppose the expert The characteristic parameter sequence is , student No. The characteristic parameter sequence is , with lengths of m frames and n frames respectively, construct an m-row and n-column cumulative distance matrix grid, such as Figure 5 shown.

[0077] Figure 5 The elements of the matrix are shown in the following four formulas. The value of the i-th row and j-th column in the grid represents g i With h j The Euclidean distance between two points is denoted by . (D represents the cumulative distance, d represents the distance between two points): ; ; ; .

[0078] In the formula, m is the length of the expert feature parameter sequence; n is the length of the student feature parameter sequence. According to the cumulative distance between the first i points of any feature parameter sequence G in the expert feature parameter sequence and the first j points of any feature parameter sequence H in the student feature parameter sequence, that is, the current grid point distance The sum of the cumulative distances to the smallest neighboring element that can reach the point. This distance represents the similarity between two sequences. The smaller the distance, the higher the similarity.

[0079] The last element of the cumulative distance matrix That is, the distance between the two parameter sequences. Count the deviation of all eigenvalues : .

[0080] in, is the number of feature parameters, that is, the number of preset bone nodes.

[0081] Find a shortest path through several grid points in this grid matrix. The grid points passed are the points of the corresponding frames of the two sequences. Define this shortest path as a regular path . Calculate the optimal path of the distance matrix. If the sequence A frame in and sequence A frame in , corresponding to the same element in the optimal path, then and That is, the corresponding frame, and then according to the corresponding relationship, the image frame where the student's feature parameter sequence is located can be standardized according to the time axis of the image frame where the expert's feature parameter sequence is located.

[0082] S105. Based on the action classification result, determine the deviation of each action from the preset standard action, and determine the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation.

[0083] In order to explore the importance of each characteristic parameter to the quality of Tai Chi movements and to comprehensively evaluate Tai Chi movements, it is necessary to determine the weight allocation method for each indicator. There are two types of weighting methods: subjective weighting and objective weighting. Common subjective weighting methods include expert judgment method, hierarchical analysis method, etc., and objective weighting methods include principal component analysis method, coefficient of variation method, entropy weight method, etc. This study introduces the method of determining weights by coefficient of variation to evaluate the quality of Tai Chi movements. The coefficient of variation is used to determine the weights of each indicator, which more objectively reflects the relative importance of the evaluation indicators, avoids the preference of experts in weighting, and weakens the influence of extreme value indicators on the evaluation results.

[0084] In one embodiment, the deviation index of each action can be obtained first, and the mean of multiple deviation indexes and the standard deviation of each action deviation index can be determined; the coefficient of variation corresponding to each action can be determined based on the mean and standard deviation; the deviation index of each action can be weighted summed based on the coefficient of variation to obtain a comprehensive score; and the evaluation result of the Tai Chi training action of the target object to be evaluated can be determined based on the comprehensive score. The deviation index can be the Euclidean distance between the feature parameter sequence of the action and the feature parameter sequence of the expert.

[0085] The overall deviation is defined as the evaluation index of Tai Chi movement quality, as follows: Step 1: Calculate the standard deviation of each indicator to reflect the absolute variation of each indicator. The specific calculation formula is: ; ; in, represents the standard deviation of the i-th deviation index, Indicates its mean.

[0086] Step 2: Calculate the coefficient of variation of each indicator to reflect the relative degree of variation of each indicator. The specific calculation formula is: .

[0087] Step 3: Normalize the coefficient of variation of each indicator to obtain the weight of each indicator. The specific calculation formula is: .

[0088] Step 4: Calculate the deviation and comprehensive score. The specific calculation formula is: .

[0089] in, is the number of characteristic parameters; when the coefficient of variation of the data is small, it means that the discrete degree of the data is low and the data points are more concentrated; and when the coefficient of variation of the data is large, it means that the discrete degree of the data is high and the data is more scattered. In the present invention, the coefficient of variation of each student's Tai Chi movements is calculated to measure their consistency with the expert's movements, so as to score the movements. The larger the coefficient of variation, the poorer the consistency with the expert's movements, and the lower the score; the smaller the coefficient of variation. It indicates that the consistency with the expert's movements is good, and the higher the score.

[0090] The scoring formula is as follows: .

[0091] Finally, a comprehensive evaluation is conducted on the Tai Chi images to be evaluated based on the scores, and they can be divided into excellent, good, qualified and unqualified according to the scores.

[0092] In addition, as a traditional martial arts sport selected as a world cultural heritage with a development history of more than 600 years, Tai Chi has distinct characteristics of its own movements. Based on the overall quality evaluation of sports movements, the present invention further explores the digital expression and excavation of the connotation of Tai Chi. Taking the four essentials of Tai Chi: Qi sinking into Dantian, upper and lower following each other, strong and fierce, and upright as the movement evaluation standards, the correlation between the movement characteristic parameters and the movement characteristics of Tai Chi is explored, and the digital expression and comprehensive evaluation method of Tai Chi is explored.

[0093] Based on the movement requirements of Tai Chi, the Tai Chi images to be evaluated can be comprehensively evaluated according to the spatial geometric characteristics, acceleration characteristics, position information and deviation of the corresponding position bone nodes; when the results of the comprehensive evaluation meet the movement requirements, it is determined that the Tai Chi training movements of the target object to be evaluated in the Tai Chi image embody the charm and connotation of Tai Chi.

[0094] Tai Chi requires the body to stand upright, that is, the head, neck, shoulders, torso, etc. should be neither biased nor tilted to the left or right. Based on the above requirements, determine the vertical inclination angle of the spine vector. is the characteristic parameter expressing "positive". The specific formula is as follows: .

[0095] Tai Chi is characterized by softness and hardness, and fast and slow movements. It emphasizes the speed and explosiveness of the moves. The points of force are concentrated in the arms, elbows, hands, hips, knees, and feet. Therefore, the acceleration characteristic value of the bone node is determined as the characteristic parameter to express the "fast and fierce force". The specific formula is as follows: .

[0096] Where n represents the number of frames. It represents the vertical acceleration value of the experimenter's right wrist joint. It represents the vertical acceleration value of the right wrist joint of a professional athlete.

[0097] Tai Chi is a highly coordinated and well-coordinated body language art that emphasizes "everything moves, hands and feet follow each other, and left and right are coordinated." Therefore, the position information of the wrist and knee joints is determined as the characteristic parameters to express "hands and feet follow each other." The specific formula is as follows: .

[0098] Tai Chi movements require "Qi sinking into Dantian", and "Dantian" is the center of the pelvis. During Tai Chi movements, the center of gravity is always required to be in the center of the pelvis. According to this requirement, the vertical deviation of the coccyx node is used as the characteristic parameter to express "Qi sinking into Dantian". The specific formula is as follows: .

[0099] Where n represents the number of frames. is the vertical displacement of the 7th bone node in the i-th frame, Indicates the mean vertical displacement of the 7th bone node.

[0100] The evaluation formula obtained based on the above four Tai Chi boxing requirements can be further used to evaluate the quality of Tai Chi images to be evaluated. It is not only limited to the comparison of the similarity of external movements. For professional fields such as Tai Chi and Peking Opera that emphasize "using form to convey spirit, and combining form and spirit", it can better reflect the quantitative evaluation research of its inner "charm", so as to make the evaluation results more comprehensive and accurate.

[0101] Using the above method, firstly, the 2D bone node data based on the timing of the preset bone nodes is extracted, so as to capture the position information of the key bone nodes in the Tai Chi movements and avoid the interference of redundant information such as height and weight. Secondly, the 2D bone node data is converted into 3D data through geometric projection and depth expansion, so as to more realistically reflect the spatial characteristics of the Tai Chi movements and help to more accurately evaluate the accuracy and standardization of the movements; then, by constructing feature vectors and calculating vector inclinations, the spatial geometric characteristics of the Tai Chi movements can be fully extracted, and by analyzing the position changes of the bone nodes between adjacent frames, the temporal motion characteristics of the movements can also be accurately captured. These features together constitute a complete description of Tai Chi movements, which can accurately complete the recognition of Tai Chi and provide rich information for subsequent evaluation. Then, RF is used to classify the extracted features, so that the complete Tai Chi movements are divided into individual moves. This movement segmentation further improves the recognition effect of Tai Chi. Finally, each movement is evaluated based on the deviation, which can quantify the evaluation results of Tai Chi movements, helping the evaluator to more intuitively understand the advantages and disadvantages of the Tai Chi images to be evaluated, so that targeted improvement suggestions can be put forward, which improves the accuracy and reliability of the evaluation and provides a scientific basis for the teaching and training of Tai Chi.

[0102] Secondly, the present invention also provides a digital evaluation device for Tai Chi training movements, such as Figure 6 As shown, including: The node extraction module 601 is used to extract the 2D bone node data of multiple preset bone nodes in the Tai Chi image to be evaluated based on the time series; perform geometric projection on the 2D bone node data to determine the bone length, and then perform depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix; The feature extraction module 602 is used to construct a feature vector according to the position relationship of adjacent nodes in the 3D bone node matrix, and determine the spatial geometric features of the preset bone nodes according to the vector inclination of the feature vector; determine the temporal motion features of each preset bone node based on the position change of the preset bone node between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; A classification module 603 is used to classify the spatial geometric features and the temporal motion features by using a random forest algorithm RF to obtain a classification result for each action; The evaluation module 604 is used to determine the deviation of each action from a preset standard based on the action classification result, and determine the evaluation result of the Tai Chi image to be evaluated according to the deviation.

[0103] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides steps for a digital assessment method for Tai Chi training movements.

[0104] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provides steps for a digital assessment method for Tai Chi training movements.

[0105] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0109] It should be noted that the above specific implementation method can enable those skilled in the art to understand the invention more comprehensively, but does not limit the invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

Claims

1. A digital evaluation method for Tai Chi training movements, characterized in that: The method comprises: Extracting 2D bone node data based on time series of multiple preset bone nodes of the target object to be evaluated in the Tai Chi image; Performing geometric projection on the 2D bone node data to determine the bone length, and performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix; A feature vector is constructed according to the positional relationship of adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone nodes are determined according to the vector inclination of the feature vector; the temporal motion features of each preset bone node are determined based on the position change of the preset bone nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; The spatial geometric features and temporal motion features are classified by using a random forest algorithm RF to obtain a classification result for each action; Based on the action classification result, the deviation of each action from the preset standard action is determined, and the evaluation result of the Tai Chi training action of the target object to be evaluated is determined according to the deviation.

2. A digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: The 2D bone node data is geometrically projected to determine the bone length, and then the 2D bone node data is deeply expanded based on the bone length to obtain a 3D bone node matrix including: Perform geometric projection on the 2D skeleton node data in a single frame, determine the distance between adjacent nodes in a single frame, and form a single frame node distance set; Traverse the multi-frame node distance set and determine the maximum value of the distance between two adjacent nodes as the bone length, thereby constructing the bone length matrix; Construct a projection triangle based on the single-frame node distance set and the bone length to obtain the depth value of the adjacent nodes; The 2D bone node data is expanded into 3D using the adjacent node depth values ​​to obtain a 3D bone node matrix.

3. A digital evaluation method for Tai Chi training movements according to claim 2, characterized in that: Before constructing the feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, the 3D bone node matrix is ​​also preprocessed, and the preprocessing includes: Determine a first-order two-dimensional difference matrix of multiple frames of 3D skeleton node data, and perform outlier screening based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix; For the missing data that has been screened, the mean of the corresponding preset bone nodes of the previous and next adjacent frames is obtained to fill in the missing data, and the preprocessed 3D bone node matrix is ​​obtained.

4. The digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: Constructing a feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, and determining the spatial geometric features of the preset bone node according to the vector inclination of the feature vector includes: Constructing a feature vector based on two adjacent nodes in the 3D skeleton node matrix, and determining the coordinates of the feature vector based on the coordinates of the two adjacent nodes; The inclination angles of the feature vector in the depth direction, the horizontal direction and the vertical direction are determined respectively, and the inclination angles are used as the spatial geometric features of two adjacent preset bone nodes.

5. The digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: The spatial geometric features and temporal motion features are classified by the random forest algorithm RF, and the classification results of each action are obtained including: The spatial geometric features and the temporal motion features are optimized by dimensionality reduction through RF, and the target motion features are extracted according to the importance; the target motion features include spatial geometric features and temporal motion features; A decision tree classifier is generated based on RF, and the target motion features are classified into action categories by the decision tree classifier to obtain a classification result for each action.

6. A digital evaluation method for Tai Chi training movements according to claim 5, characterized in that: Before performing dimension reduction optimization on the spatial geometric features and the temporal motion features by using RF, feature sequence recognition is performed on the spatial geometric features and the temporal motion features by using a DTW algorithm, and the feature sequence recognition includes: Determine the Euclidean distance between the spatial geometric feature and the temporal motion feature and the preset standard feature respectively through DTW; Based on the Euclidean distance, feature sequence recognition is performed on each frame of the Tai Chi image, and based on the recognition result, each frame is standardized on the time axis to obtain the spatial geometric features and time motion features standardized on the time axis.

7. A digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: Based on the action classification result, determining the deviation between each action and the preset standard action, and determining the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation includes: Obtaining a deviation index for each action, and determining a mean of multiple deviation indexes and a standard deviation of the deviation index for each action; Determine a coefficient of variation corresponding to each action based on the mean and standard deviation; Based on the coefficient of variation, a weighted sum of the deviation index of each action is performed to obtain a comprehensive score; The evaluation result of the Tai Chi training action of the target object to be evaluated is determined according to the comprehensive score.

8. A digital evaluation device for Tai Chi training movements, characterized in that: The device comprises: A node extraction module is used to extract 2D bone node data based on time series of multiple preset bone nodes of the target object to be evaluated in the Tai Chi image; geometrically project the 2D bone node data to determine the bone length, and then deeply expand the 2D bone node data based on the bone length to obtain a 3D bone node matrix; A feature extraction module, used to construct a feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, and determine the spatial geometric features of the preset bone nodes according to the vector inclination of the feature vector; determine the temporal motion features of each preset bone node based on the position change of the preset bone node between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; A classification module, used for classifying the spatial geometric features and temporal motion features by using a random forest algorithm RF to obtain a classification result for each action; The evaluation module is used to determine the deviation of each action from a preset standard based on the action classification result, and determine the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

Images