A method and system for evaluation visualization of a cheerleading video

Abstract

The application discloses a kind of evaluation visualization method and system of cheerleading video, wherein the evaluation visualization method of cheerleading video includes: extracting three-dimensional human posture in cheerleading video based on human posture estimation;Three-dimensional human posture is standardized as standardization posture skeleton information based on limb length and rotation angle hierarchical representation;Similarity data is obtained by comparing standardization posture skeleton information with reference posture skeleton information and calculating;Nonlinear dimensionality reduction clustering analysis is carried out based on similarity data, and the group characteristics after dimensionality reduction clustering are explained based on the characteristic expression obtained by calculation;Three-dimensional human posture, similarity data, nonlinear dimensionality reduction clustering analysis result is visualized and displayed.It improves the accuracy of human posture estimation, and the standardization processing of three-dimensional human posture, overcome the error of limb and shooting error, extract group characteristics and provide readable analysis results, provide a complete solution for cheerleading online education.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and visualization, in particular to a method and system for evaluation visualization of cheerleading videos. BACKGROUND

[0002] The online teaching of sports is more challenging than other theoretical disciplines.

[0003] In particular, for the online education of cheerleading, there is currently no complete system process and method for analyzing and evaluating the level of cheerleading, so there are still the following challenges in the online evaluation and teaching of cheerleading. First, the existing evaluation method of cheerleading lacks accurate evaluation scales, and the existing standards are mostly descriptive words rather than quantitative values, resulting in weak objective quantification and weak repeatability of individual evaluation. Second, the online cheerleading teaching mode makes the manual evaluation of teachers a heavy workload, and teachers urgently need an efficient, accurate and scientific system to reduce the workload. Third, the one-to-many teaching mode makes it difficult for teachers to accurately grasp the overall teaching quality and individual level.

[0004] Moreover, the reasons for the above-mentioned defects in the prior art are as follows: First, the existing human body posture estimation method for cheerleading is a two-dimensional human body posture estimation method, which cannot provide more rich information for subsequent scoring. Second, this method is based only on single image information, and the accuracy is poor. Third, the existing posture similarity evaluation method needs to eliminate errors through complex standardization processing to obtain the evaluation result. Fourth, it is impossible to obtain group characteristics from multi-dimensional individual scores, and there is no intuitive data to understand individual characteristics and overall trends. Fifth, it provides readable explanation for group characteristics. Although the t-SNE dimension reduction method has been proven to be scientific in long-term practice, the numerical results obtained by dimension reduction still cannot give human-readable explanations, which makes it difficult for teachers to know which body movements mainly affect the group characteristics, and is not conducive to developing targeted improvement programs. SUMMARY

[0005] The present application aims to overcome the above technical deficiencies, and provides a systematic process for the online evaluation of cheerleading, obtains data from monocular videos taken by students and performs objective, accurate, efficient and explainable analysis of the results, and finally visualizes the results, providing a complete solution for online cheerleading education. The objective and efficient quantitative evaluation method is introduced into this discipline, thereby improving the quality and level of online teaching, and providing a new analysis means for online cheerleading education.

[0006] To overcome its technical problems, the present invention proposes a visualization method for evaluating cheerleading videos, comprising: S1, extracting the three-dimensional human posture of each frame in the cheerleading video based on human posture estimation; S2, standardizing the three-dimensional human posture into standardized posture skeleton information based on a hierarchical representation of limb length and rotation angle; S3, comparing the standardized posture skeleton information with reference posture skeleton information and calculating similarity data; S4, performing nonlinear dimensionality reduction clustering analysis based on the similarity data of the group, and interpreting the group characteristics after dimensionality reduction clustering based on the calculated feature expressions; S5, visualizing at least the three-dimensional human posture, similarity data, and nonlinear dimensionality reduction clustering analysis results.

[0007] Furthermore, the step of extracting the three-dimensional human pose of each frame in the cheerleading video based on human pose estimation specifically includes: S11, obtaining the two-dimensional human pose sequence of each frame in the cheerleading video based on the two-dimensional human pose estimation method; S12, estimating the two-dimensional human pose sequence based on the three-dimensional human pose estimation method containing temporal information to obtain the corresponding three-dimensional human pose.

[0008] Furthermore, S2 specifically includes: S21, determining limb length based on the three-dimensional human pose in a preset number of video frames; S22, converting the three-dimensional human pose to BVH format based on the baseline human pose to obtain the hierarchical expression of the rotation angle of each joint.

[0009] Furthermore, S22 specifically includes: S221, calculating the baseline human pose and each of the three-dimensional human poses.

[0010] S222: Obtain the local coordinate system of joint k relative to the reference human pose based on the local coordinate system; S223: Select any joint as the root node, and calculate the transformation matrix of the child node relative to the parent node in the tree structure based on the rotation matrix of the root node and the child node; S224: Calculate the rotation angle of joint k based on the transformation matrix, thereby obtaining the three-dimensional human pose representation in BVH format.

[0011] Furthermore, the similarity data includes at least joint similarity, local similarity, and overall similarity.

[0012] Furthermore, S4 specifically includes: S41, performing nonlinear dimensionality reduction clustering on the similarity data as high-dimensional data; S42, performing set operations on a single feature based on the nonlinear dimensionality reduction clustering result to obtain a feature expression; S43, interpreting the group features after dimensionality reduction clustering based on the feature expression.

[0013] Further, the S5 specifically includes: visualizing the individual local similarity and the overall similarity based on a radar chart; visualizing the difference of the similarity scores generated by the individual in all joint dimensions in different time periods based on a column chart; representing the nonlinear dimensionality reduction clustering result based on a dot chart; visualizing the average and variance of the group similarity scores based on a k box chart; expressing the characteristics possessed by the group based on a characteristic expression; and visualizing the different joint similarity scores at each time point based on a line chart.

[0014] Another aspect of the present application provides a la la exercise video evaluation visualization system, comprising a three-dimensional pose estimation and standardization module for estimating and standardizing the three-dimensional human body pose in each frame of the captured la la exercise video; a three-dimensional pose similarity calculation module for comparing the standardized three-dimensional human body pose with a reference human body pose and calculating the similarity therebetween; and a visualization module for feature extraction, data analysis and interpretation of the high-dimensional la la exercise video evaluation visualization system data composed of the similarity, and for interactive visual presentation.

[0015] Further, the three-dimensional pose estimation and standardization module comprises: a two-dimensional pose estimation unit for estimating the two-dimensional human body pose in each frame of the captured la la exercise video to obtain a sequence of two-dimensional human body poses; a time series three-dimensional pose estimation unit for estimating the three-dimensional human body pose from the sequence of two-dimensional human body poses to obtain a three-dimensional human body pose; and a three-dimensional pose standardization unit for standardizing the obtained three-dimensional human body pose for subsequent comparison.

[0016] Further, the visualization module comprises: an individual analysis unit for presenting the score of different joint dimensions based on a radar chart, and presenting the change of the joint dimension scores in different time periods based on a column chart; a group analysis unit for representing the nonlinear dimensionality reduction clustering result based on a dot chart, expressing the average and variance of the group similarity scores based on a k box chart, and expressing the characteristics possessed by the current group based on a characteristic expression; and a video analysis unit for expressing the different joint similarity scores at each time point based on a line chart, and synchronously comparing the current video and the reference video, and synchronously comparing the standard three-dimensional human body pose skeletons extracted from the current video and the reference video.

[0017] The present application has the following beneficial effects:

[0018] 1. The three-dimensional human body pose is estimated using a sequence of images, which fully utilizes the time series information, improves the human body pose estimation accuracy, and provides more accurate information compared to two-dimensional human body pose estimation;

[0019] 2. The three-dimensional human body pose is standardized, thereby becoming a pose based on limb and joint rotation angle, thereby overcoming the error of the limb and the error of the shooting;

[0020] 3. Non-linear dimensionality reduction clustering processing is performed on high-dimensional data, operations between sets containing single features are used to approximate non-linear dimensionality reduction clustering results, and readable representations are provided for group features;

[0021] 4. Group features are extracted and readable analysis results are provided, and an efficient, accurate, fine-grained interactive visualization system. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flow chart of a la la exercise video evaluation visualization method according to an embodiment of the present application;

[0023] Figure 2 A principle schematic diagram of a la la exercise video evaluation visualization method according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of converting a pose skeleton into a tree structure according to an embodiment of the present application;

[0025] Figure 4 A visualization schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to facilitate those skilled in the art to better understand the present application, the following explains the special terms appearing in the text as follows.

[0027] T-pose: T-shaped posture or called bind pose, is a preset posture for binding the skeleton of a three-dimensional model.

[0028] BVH: A file containing joint rotation data of the skeleton and limbs of a character. BVH is a general human feature animation file format.

[0029] Hip joint: Hip joint.

[0030] The present application will be further described in detail below in combination with the drawings and specific embodiments, and the following is only exemplary and does not limit the protection scope of the present application.

[0031] As Figure 1As shown, the flow chart of the video-based cheerleading evaluation visualization method comprises the following steps: S1, extracting three-dimensional human body poses of each frame in the cheerleading video based on human body pose estimation; S2, standardizing the three-dimensional human body poses into standardized pose skeleton information based on limb length and rotation angle hierarchical representation; S3, comparing the standardized pose skeleton information with reference pose skeleton information and calculating similarity data; S4, performing nonlinear dimensionality reduction clustering analysis based on the similarity data of the groups, and explaining the group features after dimensionality reduction clustering based on the calculated characteristic expressions; and S5, visualizing at least the three-dimensional human body poses, the similarity data, and the nonlinear dimensionality reduction clustering analysis results.

[0032] In an embodiment of the present application, a video with a resolution of 1280*720 and a frame rate of 30 frames per second is captured by using any device, and the object of the video is about 3-5 meters away from the lens, and the whole body should be clearly exposed during the whole video capturing process, the light should be moderate, and should not be too dark or too bright. It should be pointed out that the method of the present application has strong scalability for cameras used as data input devices, and image sequences captured by conventional mobile devices (mobile phones, tablets), computer cameras, and professional cameras can be processed by using the method.

[0033] The following will describe each step of the cheerleading video evaluation visualization method proposed in the present application in detail. Figure 2 The method principle schematic diagram is taken as an example.

[0034] S1, extracting three-dimensional human body poses of each frame in the cheerleading video based on human body pose estimation.

[0035] S11, obtaining a two-dimensional human body pose sequence of each frame in the cheerleading video based on a two-dimensional human body pose estimation method.

[0036] In an embodiment of the present application, the two-dimensional human body pose estimation can be performed by using a cascaded pyramid network (CPN).

[0037] In another embodiment, the two-dimensional human body pose can also be estimated by using a mask region-based convolutional neural network (Mask R-CNN).

[0038] S12, estimating the two-dimensional human body pose sequence to obtain corresponding three-dimensional human body poses based on a three-dimensional human body pose estimation method containing time sequence information.

[0039] In an embodiment of the present application, the extracted two-dimensional human posture estimation information is directly estimated into three-dimensional human posture key points using a temporal dilated convolutional model. The model takes all two-dimensional human postures in a certain time range as input, and fully utilizes the time sequence information.

[0040] S2, standardizing the three-dimensional human posture into standardized posture skeleton information based on limb length and rotation angle hierarchical representation.

[0041] The three-dimensional human posture key points estimated in step 1 are standardized into standardized posture skeleton information based on limb length and joint rotation angle hierarchical representation.

[0042] In an embodiment of the present application, the standardized posture skeleton information is represented and stored using BVH format.

[0043] Specifically, the method comprises the following steps:

[0044] S21, determining the limb length based on the three-dimensional human postures in a preset number of video frames.

[0045] A three-dimensional human posture can be extracted from each frame of the video. n frames of three-dimensional human postures are randomly selected from all video frames, and the length of the limb L in the n three-dimensional human postures is averaged to obtain the limb length of the limb L in the three-dimensional human posture. The formula is

[0046]

[0047] In an embodiment of the present application, n = 50.

[0048] The above method can reduce errors in a single frame or a certain number of consecutive frames and accumulated errors.

[0049] In some embodiments, the position of the jth joint K of the ith frame is initially expressed as The length of the limb L of the ith frame is the length between two adjacent joints The average length of the same limb in different frames is calculated as the final unified posture average limb length

[0050] S22, BVH format conversion of the three-dimensional human posture based on the reference human posture, thereby obtaining the rotation angle hierarchical representation of each joint.

[0051] First, in the reference posture T-Pose, a local coordinate system of a joint is defined by a vector formed by three-dimensional posture key points. When a posture changes, the world coordinate system does not change, but the local coordinate system of the joint changes. By using the change of the local coordinate system of the joint, a rotation matrix of the joint can be calculated. Since the joint is defined hierarchically, if there is an upper joint of the joint, the rotation matrix of the current joint needs to be multiplied by the rotation matrix of the upper joint, and thus the rotation information of the current joint relative to the upper joint is obtained. Then, the rotation matrix of the joint is converted into Euler angles, which is more convenient for subsequent calculation of posture similarity.

[0052] The specific implementation is as follows. To obtain the joint rotation angle expression, for joint k, find two joints adjacent in semantics around the joint, and form two non-parallel vectors with their world coordinate positions P k =(x k , y k , z k ) respectively, and obtain the base of the local coordinate system by calculation.

[0053] In an embodiment of the present application, the local coordinate system of the hip joint is defined as The base thereof is specifically:

[0054]

[0055]

[0056]

[0057] The local coordinate system has a rotation matrix R relative to the base of the world coordinate system, which satisfies

[0058]

[0059] Formulas (1)-(4) are used to calculate the reference human posture T-Pose and the three-dimensional human posture, respectively, and thus two rotation matrices and are obtained for each joint k. Thus, the rotation matrix of joint k in the current frame i relative to the reference human posture T-Pose is obtained as which satisfies

[0060] In an embodiment of the present application, the hip joint with a relatively stable position is selected as the root node, the posture skeleton is converted into a tree structure as shown in Figure 3 , the obtained rotation matrix of the child node is calculated, and the rotation matrix of relative to the parent node in the tree structure is calculated. The transformation matrix is then calculated as The rotation angle is calculated using formula (5) The rotation matrix of the joint K in the current frame i is expressed as:

[0061]

[0062] The corresponding Euler angle is expressed as formula (5) as follows.

[0063] θ K = atan2(R 32 , R 33 )

[0064]

[0065] φ K = atan2(R 21 , R 11 )

[0066] Therefore, the BVH format three-dimensional human posture representation is obtained, and the rotation level expression of all joints can be obtained in the same way.

[0067] It should be noted that the conventional three-dimensional human posture representation method records the three-dimensional coordinates of each joint, which is an absolute distance-based representation method, and there is an error in the absolute distance when calculating the similarity. The present application converts the conventional representation method into BVH format, and standardizes the posture based on the BVH representation method, determines the fixed representation length of the limbs, and records the current posture action using the rotation angle, thereby eliminating the absolute difference of the limb length of different people and the limb length difference caused by the shooting picture.

[0068] S3, comparing the standardized posture skeleton information and the reference posture skeleton information and calculating the similarity data.

[0069] The standardized posture skeleton information obtained in the above steps is compared with the reference posture skeleton information, wherein the reference posture skeleton information is obtained by processing the professional performer's cheerleading video through S1 and S2.

[0070] The method of comparison is to calculate the difference between the rotation angles of each joint in the two posture skeletons at the same time. Among them, the same time is generally obtained relative to the background music in the cheerleading video. In particular, the posture transformation based on the rotation angle of each joint under the same limb length overcomes the error of the limbs.

[0071] It should be noted that in the previous application, comparison is made by absolute image pixels, thus absolute error and shooting error are caused. In the present application, however, through the steps of S1 and S2, the posture based on each joint rotation angle and the posture based on each joint rotation angle under the same limb length are obtained, thus overcoming the error of the limb.

[0072] In an embodiment of the present application, the similarity data includes joint similarity, local similarity and overall similarity. The calculation is made using L2 norm of the corresponding rotation angle difference. Considering that the overall influence of each joint on the human posture is different, different weights are multiplied when calculating the rotation angle difference of different joints to conform to the actual situation. The specific weight is specified by experts according to the type of dance. In addition to the overall posture similarity comparison, similarity comparison can also be made for specific part joints to evaluate the accuracy of hand, foot and other movements. When calculating, only the corresponding joint rotation angle needs to be selected for calculation.

[0073] In an embodiment of the present application, the specific calculation of the joint i of a certain frame is taken as an example to illustrate the calculation of the similarity data,

[0074] For the joint i of the current posture, the rotation angle is and the rotation angle of the joint i of the standard posture P is The joint similarity S is calculated. The weight of the joint is α i , which satisfies The calculation formula is shown in formula (6).

[0075]

[0076] In the overall similarity S total calculation, n is the number of all joints in the whole body. In the local similarity S j calculation, the value of n varies according to the number of joints constituting the limb part, but it satisfies formula (6). At the same time, it satisfies m is the number of limbs.

[0077] S4, nonlinear dimensionality reduction clustering analysis is made on the similarity data, and the characteristics of the clustered groups after dimensionality reduction are explained based on the calculated characteristic expression.

[0078] This step performs nonlinear dimensionality reduction clustering on the same group and obtains the characteristic expression of each sub-group, so as to facilitate the subsequent explanation and visualization processing of the characteristics of the sub-groups.

[0079] Specifically, the following steps are included:

[0080] S41, nonlinear dimensionality reduction clustering is made on a plurality of similarity data as high-dimensional data.

[0081] The obtained individual joint similarity s of person n i And the overall similarity s total Form a one-dimensional vector S n ={s total , s0, s1,..., s i}. Combine the one-dimensional vectors of all people N in the current class to form a high-dimensional matrix data M N×(i+1) , and perform t-SNE nonlinear dimensionality reduction on the data to obtain a new matrix M N×2 , that is, extract the first 2 features that can best distinguish the entire group from the high-dimensional data. Then perform K-Means clustering on the dimensionality reduction result to divide N people into 6 groups.

[0082] It should be noted that the number of groups divided can be changed according to actual needs, and through expert investigation and experimental results, the embodiment selects 6 groups, and considers that it can both retain the detailed differences of the groups and not too many.

[0083] S42, calculate the mean and variance based on the scores of the clustered groups to obtain overall statistical features. In an embodiment of the present application, the collected data includes video data of two time periods of in-class and after-class periods, and the above steps S1-S4 are also performed for calculation and analysis, and the performance of the same person at two times is visually compared in time sequence, as shown in Figure 4 , which is a visualization diagram. The statistical features of the same group changing with the exercise time can be discovered, helping the teacher to deeply understand the training characteristics of the group, so as to guide the teacher to develop a training improvement scheme suitable for the group.

[0084] S43, perform set operation based on the defined feature set, so that the calculated result is closest to the clustered result, thereby abstracting the set operation process into a feature expression, providing understandable features for the dimensionality reduction result

[0085] We define a set of single features, calculate the Jaccard distance between the set obtained by set operation on these single feature sets and the group set obtained in step 41, and make the Jaccard distance minimum, so as to approximate the dimensionality reduction clustering result. Abstract the above set operation process into a feature expression, thereby providing understandable features for the dimensionality reduction result.

[0086] In an embodiment of the present application, the specific calculation process is as follows.

[0087] Define a single feature set F, whose elements are the low score of joint i i_low, the medium score of joint i i_mid and the high score of joint i i-high, that is, F={s0_low s 0_mid S 0_high ,,...,s i_low， s i_mid s i_high The division of scores into low, medium, and high tiers is determined by the teacher, who specifies the score range.

[0088] For example, a single feature could be "high similarity score for the left elbow joint" or "low similarity score for the right knee joint".

[0089] Define node N i To meet feature F i The set of all individuals.

[0090] Define node N a With N b The operations between them include: intersection N a∩b and N a∪b 、Left minus N a-b And right minus N b-a Four types. Among them, N a∩b =N a ∩N b To meet feature F a And simultaneously meets feature F b A collection of individuals; N a∪b =N a ∪N b To meet feature F a Or meets characteristic F b A collection of individuals; N a-b =N a -N b To meet feature F a And does not meet feature F b A collection of individuals; N b-a =N b -N a To meet feature F b Does not conform to feature F a A collection of individuals.

[0091] Define a characteristic expression E, which is a set operation formula that achieves the final result. For example, N E =N a∪b∩c-a∪e Then E = a∪b∩ca∪e.

[0092] Define target node N t The algorithm is defined as making the result N of the node expression... o With target node N tJaccard distance is minimum, and the expression is the shortest. At this time, the node expression is the result we require. The node expression can provide a readable explanation for the characteristics of the population after dimensionality reduction clustering. Thus, the teacher can learn the strengths and weaknesses of the population and propose a more targeted training path.

[0093] S5, at least the three-dimensional human posture, similarity data, nonlinear dimensionality reduction clustering analysis results are visualized.

[0094] As shown in Figure 4 In addition to the analysis of the population, the analysis and presentation of individual data are also provided, including the scores of each joint dimension, the scores of the same dance at different time nodes, the score changes in a dance in the time dimension, and the presentation of three-dimensional human posture estimation and detection results.

[0095] In an embodiment of the present application, the local similarity and overall similarity of an individual are visualized based on a radar chart; the similarity score differences of an individual in all joint dimensions in different time periods are visualized based on a column chart; the nonlinear dimensionality reduction clustering results are represented based on a point chart; the average and variance of the group similarity scores are visualized based on a k-box chart; the characteristics of the group are expressed based on a feature expression; and the different joint similarity scores at each time point are visualized based on a line chart.

[0096] Another aspect of the present application also provides a cheerleading video evaluation visualization system for running the cheerleading video evaluation visualization method described above, including a three-dimensional posture estimation and standardization module for estimating and standardizing the three-dimensional human posture in each frame of the captured cheerleading video; a three-dimensional posture similarity calculation module for comparing and calculating the similarity of the standardized three-dimensional human posture and the reference human posture; a visualization module for feature extraction, data analysis and interpretation of the similarity composed of the high-dimensional cheerleading video evaluation visualization system data, and interactive visualization presentation.

[0097] The three-dimensional posture estimation and standardization module includes: a two-dimensional posture estimation unit for estimating the two-dimensional human posture in each frame of the captured cheerleading video to obtain a sequence of two-dimensional human postures; a time series three-dimensional posture estimation unit for estimating the three-dimensional human posture from the sequence of two-dimensional human postures to obtain a three-dimensional human posture; and a three-dimensional posture standardization unit for standardizing the obtained three-dimensional human posture for subsequent comparison.

[0098] The visualization module includes: a personal analysis unit for presenting scores of different joint dimensions based on a radar chart, and presenting changes in joint dimension scores of different time periods based on a column chart; a group analysis unit for presenting nonlinear dimensionality reduction clustering results based on a scatter plot, presenting average values and variances of group similarity scores based on a k-box chart, and presenting characteristics possessed by the current group based on a characteristic expression table; and a video analysis unit for presenting different joint similarity scores at different time points based on a line chart, and synchronously comparing the current video and the reference video, and synchronously comparing standard three-dimensional human posture skeletons extracted from the current video and the reference video.

[0099] The group analysis unit is further configured to perform nonlinear dimensionality reduction clustering on the same group and obtain characteristic expressions of each subgroup, and compare similarity high-dimensional vectors of the same group in different time periods, and calculate average increase values and standard increase variances.

[0100] In summary, by extracting three-dimensional human postures and converting them into BVH format, and on this basis, normalizing the postures, determining fixed representation lengths of limbs, and using rotation angles to record current postures, the absolute differences in limb lengths of different people and the limb length differences caused by shooting pictures are eliminated, so that the similarity of three-dimensional human postures is more accurately calculated. Similarity data is obtained by calculating joint similarity, local similarity and overall similarity, and the group characteristics are expressed in an interpretable manner through the similarity data, so as to provide a complete solution for online cheerleading education, introduce an objective and efficient quantitative evaluation method into the subject, and thereby improve the quality and level of online teaching.

[0101] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in the present specification. In some other embodiments, the steps included in the method can be more or less than described in the present specification. In addition, a single step described in the present specification can be divided into multiple steps for description in other embodiments, and multiple steps described in the present specification can be combined into a single step for description in other embodiments.

[0102] The various embodiments described in this specification are described using a numbering of embodiments so that relative pendency of numbers may indicate other embodiments that can incorporate aspects of the various embodiments. Each feature disclosed in this specification, and / or the claims, may be incorporated in the form of the various embodiments unless the context explicitly indicates otherwise. Moreover, features of each embodiment are to be considered as individual embodiments that can be combined with features of the other embodiments as appropriate, unless the context explicitly indicates otherwise. Moreover, unless the context indicates otherwise, embodiments are contemplated covering any and all combinations of the various features described herein.

Claims

1. A method for visualizing the evaluation of cheerleading videos, characterized in that, include: S1, extract the three-dimensional human pose of each frame in the cheerleading video based on human pose estimation; S2 standardizes the three-dimensional human posture into standardized posture skeleton information based on the hierarchical representation of limb length and rotation angle; S3, compare the standardized posture skeleton information with the reference posture skeleton information and calculate the similarity data; S4. Nonlinear dimensionality reduction clustering analysis is performed based on the similarity data of the groups, and the characteristics of the groups after dimensionality reduction clustering are interpreted based on the calculated feature expressions. S5 provides a visual representation of 3D human pose, similarity data, and nonlinear dimensionality reduction clustering analysis results. S2 specifically includes: S21, determine limb length based on the three-dimensional human posture in a preset number of video frames; S22, based on the baseline human pose, the three-dimensional human pose is converted into BVH format to obtain the hierarchical expression of the rotation angle of each joint. S22 specifically includes: S221, calculates the local coordinate system of each joint in the baseline human pose and the three-dimensional human pose; S222, the rotation matrix of joint k relative to the reference human posture is obtained based on the local coordinate system; S223, select any joint as the root node, and calculate the transformation matrix of the child node relative to the parent node in the tree structure based on the rotation matrix of the root node and the child node. S224, based on the transformation matrix, calculates the rotation angle of joint k, thereby obtaining a three-dimensional human pose representation in BVH format.

2. The evaluation visualization method for cheerleading videos according to claim 1, characterized in that, The extraction of the 3D human pose from each frame of the cheerleading video based on human pose estimation specifically includes: S11, Based on the two-dimensional human pose estimation method, the two-dimensional human pose sequence of each frame in the cheerleading video is obtained; S12, the three-dimensional human pose estimation method with temporal information is used to estimate the two-dimensional human pose sequence to obtain the corresponding three-dimensional human pose.

3. The evaluation visualization method for cheerleading videos according to claim 1, characterized in that, The similarity data includes at least joint similarity, local similarity, and overall similarity.

4. The evaluation visualization method for cheerleading videos according to claim 1, characterized in that, S4 specifically includes: S41, perform non-linear dimensionality reduction clustering on similarity data as high-dimensional data; S42, perform set operations on a single feature based on the nonlinear dimensionality reduction clustering results to obtain the feature expression; S43, interpreting the group characteristics after dimensionality reduction and clustering based on feature expressions.

5. The evaluation visualization method for cheerleading videos according to claim 4, characterized in that, S5 specifically includes: Radar charts are used to visualize the local and overall similarity of individuals. The bar chart is used to visualize the differences in similarity scores of individuals across all joint dimensions at different time periods; the dot plot is used to represent the results of nonlinear dimensionality reduction clustering. The mean and variance of group similarity scores are visualized using k-box diagrams. The characteristics of a group are expressed based on feature expressions; The similarity scores of different joints at each time point are visualized using line charts.

Images