A digital evaluation method and device for Tai Chi training movements

Abstract

The present invention provides a digital evaluation method and device for Tai Chi training movements, belonging to the field of image recognition. The method comprises: extracting 2D skeletal node data based on time series of multiple preset skeletal nodes in a Tai Chi image to be evaluated; performing deep expansion on the 2D skeletal node data to obtain a 3D skeletal node matrix; determining the spatial geometric features and temporal motion features of the preset skeletal nodes based on the 3D skeletal node matrix; classifying the spatial geometric features and temporal motion features using RF to obtain a classification result for each movement; and determining the deviation of each movement from a preset standard movement based on the movement classification result, and determining the evaluation result of the Tai Chi image to be evaluated based on the deviation. In this way, by extracting the movement data of key skeletal nodes to evaluate Tai Chi movements, the evaluation results of Tai Chi movements can be quantified, thereby improving the accuracy and reliability of the evaluation and providing a scientific basis for Tai Chi teaching and training.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, and in particular relates to a digital evaluation method and device for Tai Chi training movements. Background Art

[0002] Tai Chi, a traditional martial art steeped in rich cultural heritage and philosophical thought, is rapidly gaining popularity among younger generations thanks to the advancement of digital sports technology. Through diverse formats such as online courses, live social media broadcasts, and virtual reality experiences, Tai Chi has broken down geographical and time constraints, making it accessible to a wider audience, especially young people. Its fitness benefits, such as enhanced physical fitness, increased flexibility, and improved cardiopulmonary function, have attracted a large number of young people seeking a healthier lifestyle. Tai Chi is no longer limited to the elderly; it has become a fashionable lifestyle that transcends age boundaries.

[0003] Tai Chi is characterized by its gentle, slow, and even movements, orderly opening and closing, lightness, agility, and a balance of strength and flexibility. However, it is often difficult for learners to grasp the key points of Tai Chi through videos alone, nor can they clearly understand their own mastery of the technique. Therefore, how to evaluate Tai Chi learning has become a key issue.

[0004] Currently, Tai Chi practitioners rely primarily on self-perception to determine if their movements are in place, which doesn't meet the standards of scientific and effective fitness. For beginners, it's difficult to master the correct posture and rhythm through self-perception. Without a coach to constantly remind them, they can't accurately judge whether their posture and rhythm are in place.

[0005] With the development of artificial intelligence technology, theoretical research and achievements in human motion recognition based on image and video processing have continued to emerge. However, a notable characteristic of Tai Chi is the slow and seamless transitions between postures. Different individuals and learning stages can lead to differences in timing and movement. Differences in skill level, height, weight, and gait also exist, resulting in variations in performance for each subject, leading to technical difficulties in Tai Chi motion recognition. Furthermore, existing human motion evaluation research is still in its infancy, especially for complex movements, such as those in traditional arts like Tai Chi and Peking Opera, which currently lack widely recognized motion evaluation metrics. Summary of the Invention

[0006] In order to solve the problem of Tai Chi movement evaluation, the present invention provides a digital evaluation method and device for Tai Chi training movements.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] First, a digital evaluation method for Tai Chi training movements is provided, the method comprising:

[0009] Extracting 2D skeletal node data based on time series of multiple preset skeletal nodes of the target object to be evaluated in the Tai Chi image;

[0010] Performing geometric projection on the 2D bone node data to determine the bone length, and performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix;

[0011] A feature vector is constructed based on the positional relationship between adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone nodes are determined based on the vector inclination of the feature vector; the temporal motion features of each preset bone node are determined based on the position change of the preset bone nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time;

[0012] The spatial geometric features and temporal motion features are classified by the random forest algorithm RF to obtain a classification result for each action;

[0013] Based on the action classification result, the deviation of each action from the preset standard action is determined, and the evaluation result of the Tai Chi training action of the target object to be evaluated is determined according to the deviation.

[0014] Optionally, performing geometric projection on the 2D bone node data to determine the bone length, and then performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix includes:

[0015] Perform geometric projection on the 2D skeleton node data in a single frame, determine the distance between adjacent nodes in a single frame, and form a single-frame node distance set;

[0016] Traverse the multi-frame node distance set and determine the maximum distance between two adjacent nodes as the bone length, thereby constructing the bone length matrix;

[0017] Construct a projection triangle based on the single-frame node distance set and bone length to obtain the depth value of the adjacent nodes;

[0018] The 2D skeleton node data is expanded into 3D using the adjacent node depth values ​​to obtain a 3D skeleton node matrix.

[0019] Optionally, after obtaining the 3D skeleton node matrix, the method further includes:

[0020] Determine a first-order two-dimensional difference matrix of multiple frames of 3D skeleton node data, and perform outlier screening based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix;

[0021] For the missing data that has been filtered out, the mean value of the corresponding preset bone nodes of the adjacent frames before and after is obtained to fill it.

[0022] Optionally, constructing a feature vector according to the positional relationship between adjacent nodes in the 3D skeleton node matrix, and determining the spatial geometric features of the preset skeleton node according to the vector inclination of the feature vector includes:

[0023] Constructing a feature vector based on two adjacent nodes in the 3D skeleton node matrix, and determining the coordinates of the feature vector based on the coordinates of the two adjacent nodes;

[0024] The inclination angles of the feature vector in the depth direction, the horizontal direction and the vertical direction are determined respectively, and the inclination angles are used as the spatial geometric features of two adjacent preset bone nodes.

[0025] Optionally, the classifying the spatial geometric features and the temporal motion features by RF to obtain a classification result for each action includes:

[0026] Performing dimensionality reduction optimization on the spatial geometric features and temporal motion features through RF, and extracting target motion features according to importance; the target motion features include spatial geometric features and temporal motion features;

[0027] A decision tree classifier is generated based on RF, and the target motion features are classified into action categories by the decision tree classifier to obtain a classification result for each action.

[0028] Optionally, before performing dimensionality reduction optimization on the spatial geometric features and the temporal motion features by using RF, feature sequence recognition is further performed on the spatial geometric features and the temporal motion features by using a DTW algorithm, including:

[0029] Determine the Euclidean distance between the spatial geometric features and the temporal motion features and the preset standard features respectively through DTW;

[0030] A feature sequence is identified for each frame of the Tai Chi image based on the Euclidean distance, and each frame is standardized on a time axis based on the identification result.

[0031] Optionally, based on the action classification result, determining the deviation of each action from a preset standard action, and determining the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation includes:

[0032] Obtaining a deviation index for each action, and determining a mean of the plurality of deviation indexes and a standard deviation of the deviation index for each action;

[0033] determining a coefficient of variation corresponding to each action based on the mean and standard deviation;

[0034] Based on the coefficient of variation, the deviation index of each action is weighted and summed to obtain a comprehensive score;

[0035] An evaluation result of the Tai Chi training movements of the target object to be evaluated is determined based on the comprehensive score.

[0036] Secondly, a digital evaluation device for Tai Chi training movements is provided, the device comprising:

[0037] A node extraction module is used to extract 2D skeletal node data based on time series of multiple preset skeletal nodes of the target object to be evaluated in the Tai Chi image; geometrically project the 2D skeletal node data to determine the bone length, and then deeply expand the 2D skeletal node data based on the bone length to obtain a 3D skeletal node matrix;

[0038] A feature extraction module is configured to construct a feature vector based on the positional relationship between adjacent nodes in the 3D skeletal node matrix, and determine the spatial geometric features of the preset skeletal nodes based on the vector inclination of the feature vector; and determine the temporal motion features of each preset skeletal node based on the positional changes of the preset skeletal nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset skeletal nodes, and the temporal motion features are used to characterize the motion features of the preset skeletal nodes that change over time;

[0039] A classification module, configured to classify the spatial geometric features and temporal motion features through RF to obtain a classification result for each action;

[0040] The evaluation module is used to determine the deviation of each action from a preset standard based on the action classification result, and determine the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation.

[0041] In addition, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned digital evaluation method for Tai Chi training movements.

[0042] Finally, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned digital evaluation method for Tai Chi training movements is implemented.

[0043] The digital evaluation method for Tai Chi training movements provided by the present invention has the following beneficial effects:

[0044] First, the 2D bone node data based on the timing of the preset bone nodes is extracted. This can capture the position information of the key bone nodes in the Tai Chi movement and avoid the interference of redundant information such as height and weight. Secondly, the 2D bone node data is converted into 3D data through geometric projection and depth expansion, so as to more realistically reflect the spatial characteristics of the Tai Chi movement and help to more accurately evaluate the accuracy and standardization of the movement; then, by constructing feature vectors and calculating vector inclinations, the spatial geometric characteristics of the Tai Chi movement can be fully extracted. By analyzing the position changes of the bone nodes between adjacent frames, the temporal motion characteristics of the movement can also be accurately captured. These features together constitute a complete description of Tai Chi movements, which can accurately complete the recognition of Tai Chi and provide rich information for subsequent evaluation. The extracted features are then used to classify movements through RF, so that the complete Tai Chi movements are divided into individual moves. This movement splitting further improves the recognition effect of Tai Chi. Finally, each movement is evaluated based on the deviation, which can quantify the evaluation results of Tai Chi movements, helping the evaluator to more intuitively understand the advantages and disadvantages of the Tai Chi training movements of the target object to be evaluated, so that targeted improvement suggestions can be put forward, which improves the accuracy and reliability of the evaluation and provides a scientific basis for the teaching and training of Tai Chi. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the embodiments of the present invention and its design, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0046] Figure 1 The figure is a flow chart of a digital evaluation method for Tai Chi training movements provided by the present invention according to an exemplary embodiment.

[0047] Figure 2 This is a network structure diagram of yolov8s-pose provided according to an exemplary embodiment of the present invention.

[0048] Figure 3 A human skeleton node model provided by the present invention according to an exemplary embodiment; wherein Figure 3 (a) is the human skeleton node model diagram before optimization. Figure 3 (b) in the figure is the optimized human skeleton node model diagram.

[0049] Figure 4 A schematic diagram of a vector inclination angle provided according to an exemplary embodiment of the present invention; wherein Figure 4 (a) is a schematic diagram of the inclination angle in the depth direction. Figure 4 (b) is a schematic diagram of the horizontal inclination. Figure 4 (c) in the figure is a schematic diagram of the inclination angle in the vertical direction.

[0050] Figure 5 A cumulative distance matrix grid diagram is provided according to an exemplary embodiment of the present invention.

[0051] Figure 6 This is a block diagram of a digital evaluation device for Tai Chi training movements provided by the present invention according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the technical solution of the present invention and to be able to implement it, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0054] First, the present invention provides a digital evaluation method for Tai Chi training movements, specifically Figure 1 As shown, the following steps are included:

[0055] S101, extracting time-series-based 2D skeletal node data of multiple preset skeletal nodes of a target object to be evaluated in a Tai Chi image.

[0056] In the present invention, a 2D skeleton node extraction algorithm based on yolov8s-pose can be used. Yolov8s-pose is a model in yolov8 that simultaneously realizes target frame detection and human key point detection. It consists of four parts: input, backbone network, neck network and detection head output layer. The network structure is as follows Figure 2 As shown in the figure, the input end adjusts the size of the input image, the backbone network performs pre-training and feature extraction, the neck network obtains richer feature information by fusing features with position information, and the head part performs the final classification based on these features and outputs a three-layer feature map.

[0057] Specifically, we can first build a Yolov8s-pose algorithm model, and then input the labeled skeleton node sample image into Yolov8s-pose, and train it with the goal of minimizing the deviation between the predicted results and the labeled skeleton nodes to obtain the skeleton node extraction algorithm.

[0058] Taking the preset 17 skeleton nodes as an example, each key point is represented by pixel coordinates (x, y) in the image, and the correspondence between the serial number and the body part is shown in Table 1.

[0059] Table 1 Preset Skeleton Node List

[0060]

[0061] Intercept the image frames in the Tai Chi video and input them into the 2D skeleton node extraction algorithm to generate 2D skeleton node data based on time series. Assume that n frames of RGB images are intercepted and 17 skeleton nodes are extracted from each frame. Each node contains two-dimensional position information, y represents width information and z represents height information, so we get a The two-dimensional bone node matrix P is as follows:

[0062] ;

[0063] Among them, p1-p 17 The two-dimensional bone node matrix corresponding to different parts 0-16 is obtained. The node data is as follows:

[0064] .

[0065] S102: Perform geometric projection on the 2D bone node data to determine the bone length, and then perform depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix.

[0066] Since the human body will inevitably have problems such as visual overlap and visual occlusion of bone nodes during movement, which interferes with the extraction of bone nodes and the subsequent motion recognition and motion quality evaluation based on bone node information, 3D bone node reconstruction is required to improve the quality of bone node extraction by increasing the depth information of the human body.

[0067] In one embodiment, the 2D bone node data in a single frame can be geometrically projected to determine the distance between adjacent nodes in a single frame to form a single-frame node distance set; the multi-frame node distance set is traversed to determine the maximum value of the distance between two adjacent nodes as the bone length, thereby constructing a bone length matrix; a projection triangle is constructed based on the single-frame node distance set and the bone length to obtain the adjacent node depth value; the 2D bone node data is 3D expanded using the adjacent node depth value to obtain a 3D bone node matrix.

[0068] For example, firstly, the distances between adjacent nodes in a single frame are calculated through geometric projection to form a single-frame node distance set:

[0069] ,in , ;

[0070] in, is the distance between adjacent nodes in a single frame.

[0071] Then traverse the n-frame node distance set and find the maximum distance between two adjacent nodes , this value is the actual length of the bone between these two nodes, thus constructing the bone length matrix:

[0072] .

[0073] Construct a projection triangle based on the single-frame node distance array and the node distance extreme value to calculate the depth value of the adjacent nodes :

[0074] , ;

[0075] in, is the extreme value of node distance, Array of node distances for a single frame.

[0076] Taking the right pelvis (node ​​11) as the reference point, setting the depth value of node 11 to 0, and then calculating the cumulative depth information of each bone node Combined with the two-dimensional bone node matrix P, we get a 3D bone node matrix containing width, height, and depth, as follows:

[0077] .

[0078] In another embodiment, to further improve data accuracy, the 3D skeletal node data can be filtered. Specifically, the first-order two-dimensional difference matrix of multiple frames of 3D skeletal node data can be determined, and outlier screening can be performed based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix. For missing data that has been filtered out, the mean value of the corresponding preset skeletal nodes in the adjacent frames is obtained to fill the missing data.

[0079] For example, according to the kinematic statistical characteristics, the position information of the skeleton nodes of adjacent frames is strongly correlated, so the first-order two-dimensional difference matrix of the node data of n frames is statistically calculated as , in order to remove abnormal data, as follows:

[0080] .

[0081] in, and The two-dimensional bone node matrices of the next frame and the previous frame respectively.

[0082] First, calculate the first-order two-dimensional difference matrix The mathematical expectation EX matrix is ​​as follows:

[0083] .

[0084] Then calculate the bone nodes The first difference of The standard deviation matrix , the calculation formula is as follows:

[0085] .

[0086] According to statistics Guidelines, in The data within the interval are considered normal data and are retained. Data outside the interval are considered as abnormal data and are eliminated.

[0087] After removing abnormal data in the previous step, data restoration is required for the missing nodes. Since the data between adjacent frames is continuous, the median is the average value between them. This paper proposes a time-series-based three-dimensional median interpolation method for data restoration.

[0088] Assume that the missing node data is the jth node data of the i-th frame , take the jth node of the i-1th frame adjacent to the front and back and the jth node in the i+1th frame The missing node data is filled with the average value of the data. The specific formula is as follows:

[0089] ;

[0090] ;

[0091] .

[0092] In addition, the human skeleton node model is as follows Figure 3 As shown in (a), considering that different parts of the human body have different degrees of participation during movement, the recognition and evaluation of sports mainly focus on the movement of the limbs and have nothing to do with the head and face. Therefore, the nose, ears, and eyes bone nodes (serial numbers 0-4) are removed, and the shoulders are used to abstractly represent the position information of the head.

[0093] According to the characteristics of Tai Chi, the limbs have large fluctuations throughout the movement, while the trunk is relatively stable. Therefore, the present invention adds the midpoints of the two shoulder joints (big vertebra) and the two hip joints (coccyx) to evaluate and analyze the changes of the limbs throughout the movement. This reduces the number of key nodes to 14. The optimized human skeleton node model is as follows: Figure 3 The optimized skeleton node list is shown in Table 2.

[0094] Table 2 Optimized skeleton node list

[0095]

[0096] In order to eliminate the influence of human body shape differences on action recognition, normalization processing is performed to obtain a normalized 3D skeleton node dataset.

[0097] S103: Extracting the spatial geometric features and temporal motion features of the preset bone node according to the 3D bone node matrix.

[0098] After the human body posture estimation is completed, it is necessary to determine the characteristic parameters for Tai Chi action recognition and action quality evaluation. The present invention extracts the spatial geometric characteristics and temporal motion characteristics of the human body respectively.

[0099] In this step, feature extraction is required as the basis for subsequent Tai Chi movement recognition and evaluation. The extracted feature parameters must meet the following four requirements: 1. The extracted feature parameters can represent the unique state of the movement; 2. The feature parameters must be continuous and stable throughout the entire movement process, and dimensionality reduction can be used to reduce the amount of computation; 3. The extracted features must maintain a certain degree of invariance when the video shooting position and angle change; 4. Differences in height and body shape of different people should be considered when extracting feature parameters.

[0100] Specifically, a feature vector can be constructed based on the positional relationship between adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone node can be determined based on the vector inclination of the feature vector; the temporal motion features of each preset bone node can be determined based on the position changes of the preset bone nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time.

[0101] For example, a feature vector can be constructed based on two adjacent nodes in the 3D bone node matrix, and the coordinates of the feature vector can be determined based on the coordinates of the two adjacent nodes; the inclination angles of the feature vector in the depth direction, horizontal direction and vertical direction are determined respectively, and the inclination angles are used as the spatial geometric features of the two adjacent preset bone nodes.

[0102] Spatial geometric relationships are static features, reflecting the information directly expressed by the limbs in a single image, such as node coordinates, distances between nodes, and angles between nodes. Considering that body proportions vary among individuals, this paper constructs 10 sets of feature vectors based on the positional relationships of adjacent nodes, as shown in Table 3.

[0103] Table 3 Spatial geometric characteristics

[0104]

[0105] The present invention defines the vector inclination as a spatial geometric feature to avoid the influence of individual body shape inconsistency on feature description, such as Figure 4 As shown, the inclination angle is divided into the vertical inclination angle (the angle between the vector connecting nodes 1 and 2 and the vertical direction, as shown in Figure 4 (c) in the figure), horizontal inclination (the angle between the vector connecting nodes 1 and 2 and the horizontal direction, such as Figure 4 (b)) and the depth inclination (the angle between the vector connecting nodes 1 and 2 and the depth direction, as shown in Figure 4 (a) in the figure.

[0106] For another example, suppose the coordinates of nodes 1, 2, and 3 are 、 、 , the vector between nodes is , The vectors in the depth direction, horizontal direction and vertical direction are represented by unit vectors 、 and Depth inclination , horizontal inclination and vertical inclination Determined by the following formula:

[0107] ;

[0108] ;

[0109] .

[0110] The temporal motion feature reflects the changes of the human body at consecutive moments, such as the swing speed of certain joints and limbs. The present invention selects the motion acceleration of each skeletal node as the temporal motion feature to reflect the "force" feature in the motion. Since the time interval between adjacent frames is fixed, the time interval between adjacent frames of a skeletal node is not constant. arrive The acceleration formulas on x, y, and z are as follows:

[0111] ;

[0112] ;

[0113] .

[0114] Wherein, t is the interval between adjacent frames. When the video is shot at a frequency of 25 frames per second, t is 0.04s.

[0115] In summary, the motion feature parameters extracted from N frames of RGB images are shown in Table 4:

[0116] Table 4 Characteristic parameter list

[0117]

[0118] S104 , classifying the spatial geometric features and temporal motion features using a random forest algorithm (RF) to obtain a classification result for each action.

[0119] To test the effectiveness of the 3D skeletal node reconstruction algorithm and lay the foundation for subsequent movement quality assessment, the random forest (RF) algorithm was used for feature optimization, dimensionality reduction, and multi-classification recognition. Random forest, proposed by Leo Breiman in 2001, is an ensemble learning algorithm that improves the model's predictive power by constructing multiple decision trees. This paper uses RF to calculate feature importance and remove less important features, thereby reducing the complexity of human skeletal features and subsequently establishing a Tai Chi movement classification model.

[0120] Specifically, RF can be used to first perform dimensionality reduction optimization on the spatial geometric features and temporal motion features, and then extract the target motion features according to their importance; the target motion features include spatial geometric features and temporal motion features; a decision tree classifier is generated based on RF, and the target motion features are used to perform action classification to obtain the classification results of each action.

[0121] Since there is a strong correlation between some features in the constructed human skeleton feature set, in order to prevent the model from overfitting and reduce the calculation time, the random forest algorithm is used for feature dimensionality reduction. RF consists of multiple decision trees, where the root node is where the decision tree starts, the trunk is the decision that needs to be made, and the decision tree divides the node into child nodes according to given conditions by splitting. Each main branch is an option in the decision process, and the smaller branches represent potential results. The leaf node is the final output node. Each tree randomly selects some samples from all samples, and it is a sampling with replacement. The probability that a sample is not selected in N decision trees is , when N is large enough, the probability is The samples that are not selected by a tree are called out-of-bag (OOB) samples of the tree. OOB error can not only evaluate the accuracy of classification, but also serve as an important indicator of features. Assuming that N decision trees are generated, each decision tree contains M features. First, the out-of-bag error of each decision tree is calculated, and the value of the i-th feature variable in its out-of-bag data is randomly changed, and then the new out-of-bag error is calculated. The specific formula is as follows:

[0122] ;

[0123] in, The out-of-bag error after random change of the i-th feature variable; N is the total number of decision trees, For the sample N The probability of not being selected in any decision tree is N When it is large enough, the probability approaches a certain value. is the probability that the i-th sample is not selected in a single sampling. The error increment is obtained by recalculating the OOB error after randomly changing the value of the i-th feature variable in the out-of-bag data. The larger the out-of-bag error (OOB) caused by this change, the more important the i-th feature is. Features are sorted by importance and the top-ranked features are extracted to achieve feature optimization and feature dimensionality reduction.

[0124] Each decision tree classifier is defined as , x represents the input feature, Represents an independent set of identically distributed features, X represents features of the same class, and Y represents a set of classification labels. Each decision tree independently votes for classification, and the classification result is finally obtained through the voting of the RF. The classification process of RF is as follows:

[0125] (1) Generate a decision tree classifier. The calculation formula is:

[0126] .

[0127] (2) Generate RF, the calculation formula is:

[0128] ;

[0129] in, H Represents the classification result set obtained by RF through voting of multiple decision tree classifiers. h k is the kth decision tree classifier.

[0130] (3) The result with the most votes is taken as the classification result. The calculation formula is:

[0131] .

[0132] in, I is an indicator function. When the classification label matches the true label, the parameter is 1, otherwise it is 0. t is the number of votes corresponding to the input feature x, The category that receives the most votes.

[0133] In this way, through the above process, the spatial geometric features and temporal motion features can be classified to obtain the classification results of each action, that is, the overall image of Tai Chi is divided into the image of each move.

[0134] In another embodiment, before performing dimensionality reduction optimization on the spatial geometric features and temporal motion features through RF, the spatial geometric features and temporal motion features are also subjected to feature sequence recognition through the DTW algorithm, and the spatial geometric features and temporal motion features are standardized on the time axis, and then the standardized spatial geometric features and temporal motion features are subjected to dimensionality reduction optimization through RF. Specifically, the Euclidean distance between the spatial geometric features and temporal motion features and the preset standard features is determined respectively through DTW, and the preset standard features include preset standard spatial geometric features and preset standard temporal motion features; based on the Euclidean distance, each frame of the Tai Chi image is subjected to feature sequence recognition, and based on the recognition result, each frame is standardized on the time axis, such as mapping the feature sequence of each frame according to the time axis of the preset standard feature.

[0135] Tai Chi routines consist of a series of "moves", each of which is composed of multiple frames of continuous skeletal node data and is continuous in time. It is necessary to extract the "moves" action sequences in Tai Chi routines through action similarity evaluation and establish a standard action feature sequence library. Common methods for solving sequence similarity problems include calculating Euclidean distance, cosine similarity, and Dynamic Time Warping (DTW) algorithm. Since everyone's movements are different, the length of the resulting action sequences is also different. For time sequences of different lengths, the DTW algorithm can more effectively calculate the similarity to better perform action sequence recognition. Therefore, the present invention adopts the DTW algorithm to perform sequence comparison.

[0136] The DTW algorithm is based on scaling sequences of different lengths representing the same action on the timeline to make both sequences of the same length, thus ensuring smooth subsequent action recognition and evaluation. It was first proposed by Sakoe to solve the optimization problem of two time series that are irregular on the timeline. The DTW algorithm works by re-regularizing the two sequences to find the matching path with the minimum distance, calculating the cumulative minimum distance between the two sequences, or the true similarity between the two sequences. This matching path is the mapping relationship between the points in the two sequences, eliminating the differences between the sequences on the timeline.

[0137] The present invention uses the expert feature parameter sequence as the standard sample, that is, the preset standard feature, and the student feature parameter sequence as the test sample. As an example, let the expert The characteristic parameter sequence is , student No. The characteristic parameter sequence is , with lengths of m frames and n frames respectively, construct an m-row and n-column cumulative distance matrix grid, such as Figure 5shown.

[0138] Figure 5 The elements of the matrix are represented by the following four formulas. The value of the i-th row and j-th column in the grid represents g i With h j The Euclidean distance between two points is denoted as . (D represents the cumulative distance, d represents the distance between two points):

[0139] ;

[0140] ;

[0141] ;

[0142] .

[0143] Where m is the length of the expert feature parameter sequence; n is the length of the student feature parameter sequence. According to the cumulative distance between the first i points of any feature parameter sequence G in the expert feature parameter sequence and the first j points of any feature parameter sequence H in the student feature parameter sequence, that is, the current grid point distance The sum of the cumulative distances to the smallest neighboring element that can reach the point. This distance represents the similarity between two sequences. The smaller the distance, the higher the similarity.

[0144] The last element of the cumulative distance matrix That is the distance between the two parameter sequences. Statistical deviation of all eigenvalues :

[0145] .

[0146] in, is the number of feature parameters, that is, the number of preset bone nodes.

[0147] Find a shortest path through several grid points in this grid matrix, and the grid points passed through are the points of the corresponding frames of the two sequences. Define this shortest path as a regular path . Calculate the optimal path of the distance matrix, if the sequence A frame in and sequence A frame in , corresponding to the same element in the optimal path, then and That is, the corresponding frame, and then according to the corresponding relationship, the image frame where the student's feature parameter sequence is located can be standardized according to the time axis of the image frame where the expert's feature parameter sequence is located.

[0148] S105. Based on the action classification result, determine the deviation of each action from the preset standard action, and determine the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation.

[0149] To explore the importance of various characteristic parameters to the quality of Tai Chi movements and comprehensively evaluate them, it is necessary to determine a method for assigning weights to each indicator. Weighting methods can be categorized into two types: subjective and objective. Common subjective weighting methods include expert judgment and the analytic hierarchy process, while objective weighting methods include principal component analysis, the coefficient of variation method, and the entropy weight method. This study introduces a method for determining weights based on the coefficient of variation to evaluate the quality of Tai Chi movements. Using the coefficient of variation to determine the weights of each indicator objectively reflects the relative importance of the evaluation indicators, avoids biased weighting by experts, and mitigates the influence of extreme values ​​on the evaluation results.

[0150] In one embodiment, a deviation index for each movement can be obtained, and the mean of the multiple deviation indexes and the standard deviation of each movement deviation index can be determined. A coefficient of variation corresponding to each movement can be determined based on the mean and standard deviation. A weighted summation of the deviation index for each movement can be performed based on the coefficient of variation to obtain a comprehensive score. An evaluation result of the Tai Chi training movement of the target subject can be determined based on the comprehensive score. The deviation index can be the Euclidean distance between the feature parameter sequence of the movement and the feature parameter sequence of the expert.

[0151] The overall deviation is defined as an indicator for evaluating the quality of Tai Chi movements, as follows:

[0152] Step 1: Calculate the standard deviation of each indicator to reflect the absolute degree of variation of each indicator. The specific calculation formula is:

[0153] ;

[0154] ;

[0155] in, represents the standard deviation of the i-th deviation index, Indicates its mean.

[0156] Step 2: Calculate the coefficient of variation of each indicator to reflect the relative degree of variation of each indicator. The specific calculation formula is:

[0157] .

[0158] Step 3: Normalize the coefficient of variation of each indicator to obtain the weight of each indicator. The specific calculation formula is:

[0159] .

[0160] Step 4: Calculate the deviation and comprehensive score. The specific calculation formula is:

[0161] .

[0162] in, is the number of characteristic parameters; when the coefficient of variation of the data is small, it means that the data is less discrete and the data points are more concentrated; when the coefficient of variation of the data is large, it means that the data is more discrete and the data is more dispersed. In the present invention, the coefficient of variation of each student's Tai Chi movements is calculated to measure their consistency with the expert's movements, and this is used to score the movements. The larger the coefficient of variation, the poorer the consistency with the expert's movements, and the lower the score; the smaller the coefficient of variation. It indicates that the consistency with the expert's movements is better, and the higher the score.

[0163] The scoring formula is as follows:

[0164] .

[0165] Finally, a comprehensive evaluation is conducted on the Tai Chi images to be evaluated based on the scores, and they can be divided into excellent, good, qualified and unqualified according to the scores.

[0166] Furthermore, as a traditional martial art designated a World Cultural Heritage and boasting a history of over 600 years, Tai Chi possesses distinct characteristics. This study, based on an overall quality assessment of movement, further explores the digital expression and exploration of Tai Chi's essence. Using the four essential principles of Tai Chi—sinking Qi into the Dantian, harmony between upper and lower parts, swift and fierce power, and upright posture—as movement evaluation criteria, this study explores the correlation between movement characteristic parameters and Tai Chi's movement characteristics, exploring methods for digital expression and comprehensive evaluation of Tai Chi.

[0167] Based on the movement requirements of Tai Chi, a comprehensive evaluation can be performed on the Tai Chi image to be evaluated based on the spatial geometric characteristics, acceleration characteristics, position information and deviation of the corresponding skeletal nodes; when the results of the comprehensive evaluation meet the movement requirements, it is determined that the Tai Chi training movements of the target object to be evaluated in the Tai Chi image reflect the charm and connotation of Tai Chi.

[0168] Tai Chi requires the body to be upright, that is, the head, neck, shoulders, and torso should be neither leaning nor tilting. Based on the above requirements, the vertical inclination angle of the spine vector is determined. is the characteristic parameter expressing “positive”. The specific formula is as follows:

[0169] .

[0170] Tai Chi is characterized by softness combined with strength, alternating between fast and slow movements, and emphasizes the swiftness and explosiveness of its attacks. Force is concentrated in the arms, elbows, hands, hips, knees, and feet. Therefore, the acceleration eigenvalues ​​of skeletal nodes are determined as characteristic parameters expressing the "velocity and ferocity of force." The specific formula is as follows:

[0171] .

[0172] Where n represents the number of frames, represents the vertical acceleration value of the experimenter's right wrist joint, Indicates the vertical acceleration value of the right wrist joint of a professional athlete.

[0173] Tai Chi, as a highly coordinated and integrated body language art, emphasizes "every movement is complete, hands and feet follow each other, and left and right are coordinated." Therefore, the position information of the wrist and knee joints is used as the characteristic parameters to express "hands and feet follow each other." The specific formula is as follows:

[0174] .

[0175] Tai Chi movements require "sinking Qi into the Dantian," the center of the pelvis. During Tai Chi, the center of gravity must always be centered in the pelvis. Based on this requirement, the vertical deviation of the coccyx node is used as a characteristic parameter to express "sinking Qi into the Dantian." The specific formula is as follows:

[0176] .

[0177] Where n represents the number of frames, is the vertical displacement of the 7th bone node in the i-th frame, Indicates the mean vertical displacement of the 7th bone node.

[0178] The evaluation formula derived from the above four Tai Chi boxing requirements can be further used to evaluate the quality of Tai Chi images to be evaluated. It is not only limited to the comparison of the similarity of external movements. For professional fields such as Tai Chi and Peking Opera that emphasize "using form to convey spirit, and combining form and spirit", it can better reflect the quantitative evaluation research of their inner "charm", so that the evaluation results are more comprehensive and accurate.

[0179] Using the above method, we first extract the 2D bone node data based on the time series of the preset bone nodes, so as to capture the position information of the key bone nodes in the Tai Chi movement and avoid the interference of redundant information such as height and weight. Secondly, the 2D bone node data is converted into 3D data through geometric projection and depth expansion, so as to more realistically reflect the spatial characteristics of the Tai Chi movement and help to more accurately evaluate the accuracy and standardization of the movement; then, by constructing feature vectors and calculating vector inclinations, we can comprehensively extract the spatial geometric characteristics of the Tai Chi movement, and by analyzing the position changes of the bone nodes between adjacent frames, we can also accurately capture the temporal motion characteristics of the movement. These features together constitute a complete description of Tai Chi movements, which can accurately complete the recognition of Tai Chi and provide rich information for subsequent evaluation. The extracted features are then used to classify movements through RF, thus dividing the complete Tai Chi movements into individual moves. This movement splitting further improves the recognition effect of Tai Chi. Finally, each movement is evaluated based on the deviation, which can quantify the evaluation results of Tai Chi movements, helping the evaluator to more intuitively understand the advantages and disadvantages of the Tai Chi images to be evaluated, so that targeted improvement suggestions can be put forward, which improves the accuracy and reliability of the evaluation and provides a scientific basis for the teaching and training of Tai Chi.

[0180] Secondly, the present invention also provides a digital evaluation device for Tai Chi training movements, such as Figure 6 As shown, including:

[0181] The node extraction module 601 is used to extract 2D skeletal node data based on time series of multiple preset skeletal nodes in the Tai Chi image to be evaluated; perform geometric projection on the 2D skeletal node data to determine the bone length, and then perform depth expansion on the 2D skeletal node data based on the bone length to obtain a 3D skeletal node matrix;

[0182] Feature extraction module 602 is configured to construct a feature vector based on the positional relationship between adjacent nodes in the 3D skeletal node matrix, and determine the spatial geometric features of the preset skeletal nodes based on the vector inclination of the feature vector; determine the temporal motion features of each preset skeletal node based on the positional changes of the preset skeletal nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset skeletal nodes, and the temporal motion features are used to characterize the motion features of the preset skeletal nodes that change over time;

[0183] A classification module 603 is configured to classify the spatial geometric features and temporal motion features using a random forest algorithm (RF) to obtain a classification result for each action;

[0184] The evaluation module 604 is used to determine the deviation of each action from a preset standard based on the action classification result, and determine the evaluation result of the Tai Chi image to be evaluated according to the deviation.

[0185] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides steps for a digital assessment method of Tai Chi training movements.

[0186] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provides steps for a digital assessment method of Tai Chi training movements.

[0187] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0191] It should be noted that the above specific embodiments can enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although this specification has described the present invention in detail, those skilled in the art should understand that the present invention can still be modified or replaced with equivalents; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are included in the scope of protection of the patent for the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. A digital evaluation method for Tai Chi training movements, characterized in that: The method comprises: Extracting 2D skeletal node data based on time series of multiple preset skeletal nodes of the target object to be evaluated in the Tai Chi image; Performing geometric projection on the 2D bone node data to determine the bone length, and performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix; A feature vector is constructed based on the positional relationship between adjacent nodes in the 3D bone node matrix, and the spatial geometric features of the preset bone nodes are determined based on the vector inclination of the feature vector; the temporal motion features of each preset bone node are determined based on the position change of the preset bone nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset bone nodes, and the temporal motion features are used to characterize the motion features of the preset bone nodes that change over time; The spatial geometric features and temporal motion features are classified by the random forest algorithm RF to obtain a classification result for each action; Based on the action classification results, determining the deviation of each action from a preset standard action, and determining an evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation; Performing geometric projection on the 2D bone node data to determine the bone length, and then performing depth expansion on the 2D bone node data based on the bone length to obtain a 3D bone node matrix includes: Perform geometric projection on the 2D skeleton node data in a single frame, determine the distance between adjacent nodes in a single frame, and form a single-frame node distance set; Traverse the multi-frame node distance set and determine the maximum distance between two adjacent nodes as the bone length, thereby constructing the bone length matrix; Construct a projection triangle based on the single-frame node distance set and bone length to obtain the depth value of the adjacent nodes; Expand the 2D skeleton node data into 3D using the adjacent node depth values ​​to obtain the 3D skeleton node matrix; Constructing a feature vector according to the positional relationship of adjacent nodes in the 3D bone node matrix, and determining the spatial geometric features of the preset bone node according to the vector inclination of the feature vector includes: Constructing a feature vector based on two adjacent nodes in the 3D skeleton node matrix, and determining the coordinates of the feature vector based on the coordinates of the two adjacent nodes; Determine the inclination angles of the feature vector in the depth direction, the horizontal direction, and the vertical direction respectively, and use the inclination angles as spatial geometric features of two adjacent preset bone nodes; Determine the motion acceleration of each bone node based on the position change of the bone nodes in adjacent frames, and use the motion acceleration of each bone node as the temporal motion feature; The spatial geometric features and temporal motion features are classified by the random forest algorithm RF, and the classification results of each action are obtained including: Performing dimensionality reduction optimization on the spatial geometric features and temporal motion features through RF, and extracting target motion features according to importance; the target motion features include spatial geometric features and temporal motion features; A decision tree classifier is generated based on RF, and the target motion features are classified into action categories by the decision tree classifier to obtain a classification result for each action.

2. A digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: Before constructing the feature vector according to the positional relationship between adjacent nodes in the 3D skeleton node matrix, the 3D skeleton node matrix is ​​preprocessed. The preprocessing includes: Determine a first-order two-dimensional difference matrix of multiple frames of 3D skeleton node data, and perform outlier screening based on the mathematical expectation and standard deviation matrix of the first-order two-dimensional difference matrix; For the missing data that has been filtered out, the mean value of the preset bone nodes corresponding to the adjacent frames is obtained to fill the missing data and obtain the preprocessed 3D bone node matrix.

3. The digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: Before performing dimensionality reduction optimization on the spatial geometric features and the temporal motion features by using RF, feature sequence recognition is performed on the spatial geometric features and the temporal motion features by using the DTW algorithm. The feature sequence recognition includes: Determine the Euclidean distance between the spatial geometric features and the temporal motion features and the preset standard features respectively through DTW; Based on the Euclidean distance, each frame of the Tai Chi image is subjected to feature sequence recognition, and based on the recognition result, each frame is standardized on the time axis to obtain the spatial geometric features and temporal motion features standardized on the time axis.

4. The digital evaluation method for Tai Chi training movements according to claim 1, characterized in that: Based on the action classification results, determining the deviation of each action from a preset standard action, and determining the evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation includes: Obtaining a deviation index for each action, and determining a mean of the plurality of deviation indexes and a standard deviation of the deviation index for each action; determining a coefficient of variation corresponding to each action based on the mean and standard deviation; Based on the coefficient of variation, the deviation index of each action is weighted and summed to obtain a comprehensive score; An evaluation result of the Tai Chi training movements of the target object to be evaluated is determined based on the comprehensive score.

5. A digital evaluation device for Tai Chi training movements, characterized in that: The device comprises: A node extraction module is used to extract 2D skeletal node data based on time series of multiple preset skeletal nodes of the target object to be evaluated in the Tai Chi image; geometrically project the 2D skeletal node data to determine the bone length, and then deeply expand the 2D skeletal node data based on the bone length to obtain a 3D skeletal node matrix; A feature extraction module is configured to construct a feature vector based on the positional relationship between adjacent nodes in the 3D skeletal node matrix, and determine the spatial geometric features of the preset skeletal nodes based on the vector inclination of the feature vector; and determine the temporal motion features of each preset skeletal node based on the positional changes of the preset skeletal nodes between adjacent frames; the spatial geometric features are used to characterize the spatial position information of the preset skeletal nodes, and the temporal motion features are used to characterize the motion features of the preset skeletal nodes that change over time; A classification module is used to classify the spatial geometric features and temporal motion features using a random forest algorithm (RF) to obtain a classification result for each action; An evaluation module, configured to determine a degree of deviation of each action from a preset standard based on the action classification result, and determine an evaluation result of the Tai Chi training action of the target object to be evaluated according to the deviation; The node extraction module is further used to perform geometric projection on the 2D skeleton node data in a single frame, determine the distance between adjacent nodes in a single frame, and form a single-frame node distance set; traverse the multi-frame node distance set to determine the maximum value of the distance between two adjacent nodes as the bone length, thereby constructing a bone length matrix; construct a projection triangle based on the single-frame node distance set and the bone length to obtain the depth value of the adjacent node; and perform 3D expansion on the 2D skeleton node data using the adjacent node depth values ​​to obtain a 3D skeleton node matrix; The feature extraction module is further configured to construct a feature vector based on two adjacent nodes in the 3D skeleton node matrix, and determine the coordinates of the feature vector based on the coordinates of the two adjacent nodes; determine the inclination angles of the feature vector in the depth direction, the horizontal direction, and the vertical direction, respectively, and use the inclination angles as the spatial geometric features of two adjacent preset skeleton nodes; determine the motion acceleration of each skeleton node based on the position change of the skeleton nodes in adjacent frames, and use the motion acceleration of each skeleton node as the temporal motion feature; The classification module is also used to perform dimensionality reduction optimization on the spatial geometric features and temporal motion features through RF, and extract target motion features according to importance; the target motion features include spatial geometric features and temporal motion features; a decision tree classifier is generated based on RF, and the target motion features are classified into action classes through the decision tree classifier to obtain a classification result for each action.

6. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

7. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 4 when executing the program.

Images