Taijiquan action coordination evaluation method, device, equipment and medium based on gravity center height detection

Abstract

The invention discloses a shadowboxing action coordination evaluation method and device based on gravity center height detection, equipment and a medium, and the method comprises the following steps: obtaining a shadowboxing action video, and recognizing the three-dimensional coordinates of human joints; based on the three-dimensional coordinates, identifying the motion style of the shadowboxing; calculating gravity center data corresponding to each action style based on the three-dimensional coordinates; the gravity center data comprises a human body gravity center point and a human body gravity center height difference; and performing coordination scoring according to the constructed standard data and the gravity center data of the shadowboxing action video. According to the method, subjective errors in manual scoring are effectively reduced, the scientificity and fairness of action scoring are improved, meanwhile, a practicer is helped to find and improve unstable points in actions, and the practice effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Tai Chi movements, and in particular to a method, device, equipment and medium for evaluating the coordination of Tai Chi movements based on center of gravity height detection. Background Art

[0002] As an important part of Chinese traditional culture, Tai Chi is one of the most popular sports with a strong mass base. Tai Chi practitioners hope to get feedback through an objective scoring system, improve the standardization and coherence of their movements, and master the essence of Tai Chi. In addition, the scoring of Tai Chi competitions also needs to improve efficiency and reduce human errors to achieve fair and just judging.

[0003] Most of the existing Tai Chi movement scoring methods rely on the personal experience of the referee, lack a systematic and scientific scoring standard, and are easily affected by subjective factors, resulting in inaccurate and unfair scoring. At the same time, the existing scoring algorithms often focus on a single point and ignore important indicators such as the continuity and stability of the movement. Therefore, the existing scoring system cannot fully meet the needs of Tai Chi practitioners and competition referees. There is an urgent need for an intelligent and systematic movement coordination evaluation method to help Tai Chi practitioners improve the quality of their movements and obtain more accurate scoring results. Summary of the invention

[0004] Based on the above deficiencies in the prior art, the present invention provides a method, device, equipment and medium for evaluating the coordination of Tai Chi movements based on center of gravity height detection.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a method for evaluating Tai Chi movement coordination based on center of gravity height detection, and the method comprises the following steps: Obtain Tai Chi action videos and identify the three-dimensional coordinates of human joints; Recognize Tai Chi movements based on the three-dimensional coordinates; Calculate the center of gravity data corresponding to each action based on the three-dimensional coordinates; the center of gravity data includes the center of gravity of the human body and the height difference of the center of gravity of the human body; Coordination scores were made based on the constructed standard data and the center of gravity data from the Tai Chi movement video.

[0006] In some embodiments, identifying the three-dimensional coordinates of a human joint includes: Extracting human body posture data of each frame image in the Tai Chi action video, wherein the human body posture data includes two-dimensional coordinates of human body joints, and constructing a two-dimensional coordinate sequence of the human body joints; Based on the multi-head attention mechanism, the features of the two-dimensional coordinate sequence are extracted, and the three-dimensional posture data of the human joints are output; the three-dimensional posture data includes the three-dimensional coordinates (x, y, z) of the key points in each frame image of the Tai Chi action video, where x and y represent the horizontal position and vertical position of the key point on the image, and z represents the distance between the key point and the shooting point.

[0007] In some implementations, calculating the center of gravity data corresponding to each action move based on the three-dimensional coordinates includes: Divide the body parts of the human body based on the key points, and calculate the body segment center of gravity of each body segment in each frame of the image; The center of gravity of the human body is calculated for each frame according to the moment synthesis algorithm, and the height of the human body and the height change of the center of gravity of each action are calculated based on the center of gravity of the human body.

[0008] In some embodiments, the calculation of the height of the center of gravity of the human body and the change of the center of gravity height of each action form based on the center of gravity of the human body includes: Calculate the height of the center of gravity of the human body in each frame of image by combining the first distance and the second distance, wherein the first distance is the distance between the center of gravity of the human body and the key point of the pelvis in the vertical direction; and the second distance is the distance between the key point of the left foot, the key point of the right foot and the key point of the pelvis in the vertical direction; Constructing a two-dimensional array of the time of the Tai Chi action video and the height change of the center of gravity of the human body, and obtaining a center of gravity height change curve based on the two-dimensional array; Determine the highest and lowest points of the center of gravity for each move, and calculate the height difference of the center of gravity for each move.

[0009] In some embodiments, calculating the body segment centroid of each body segment in each frame of the image includes: Obtaining the proximal coefficient and the distal coefficient of the body segment, and calculating the body segment centroid of each body segment in combination with the three-dimensional coordinates of the key points in each body segment; The center of gravity of the human body is calculated for each frame of the image according to the moment synthesis algorithm, including: The body segment mass coefficient is obtained, and the body segment center of gravity of each body segment and the body segment mass coefficient are combined to calculate the body center of gravity of each frame of the image.

[0010] In some embodiments, the coordination score is performed based on the constructed standard data and the center of gravity data of the Tai Chi action video, including: Construct standard data; the standard data is the center of gravity height, center of gravity height difference and center of gravity height change trend of each action move in the standard video, and determine the score range of each action move according to the standard data; The DTW algorithm is used to align the time axis of the center of gravity height change curves of the standard video and the Tai Chi action video, and the coordination score of each action in the Tai Chi action video is determined according to the score range.

[0011] In some embodiments, identifying Tai Chi movements based on the three-dimensional coordinates includes: Constructing a training data set, wherein the training data set includes time series data of annotated action moves and key points; Taking the training data set as input and Tai Chi movements as output, training a long short-term memory model; The Tai Chi movements corresponding to the three-dimensional coordinates are identified through a long short-term memory model.

[0012] In a second aspect, the present invention discloses a Tai Chi movement coordination evaluation device based on center of gravity height detection, comprising: The three-dimensional coordinate recognition module obtains Tai Chi action videos and recognizes the three-dimensional coordinates of human joints; An action and gesture recognition module, which recognizes Tai Chi action and gestures based on the three-dimensional coordinates; A center of gravity data calculation module calculates the center of gravity data corresponding to each action based on the three-dimensional coordinates; the center of gravity data includes the center of gravity point of the human body and the height difference of the center of gravity of the human body; The coordination scoring module performs coordination scoring based on the constructed standard data and the center of gravity data of the Tai Chi movement video.

[0013] In a third aspect, the present invention discloses a computer device, comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of any of the above methods for evaluating the coordination of Tai Chi movements based on center of gravity height detection.

[0014] In a fourth aspect, the present invention discloses a computer storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for evaluating the coordination of Tai Chi movements based on center of gravity height detection as described in any one of the above items is implemented.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method, device, equipment, and medium for evaluating the coordination of Tai Chi movements based on center of gravity height detection. By calculating the center of gravity height and center of gravity height change of the human body in Tai Chi movement videos, and combining the center of gravity height difference and change trend in the standard video for coordination evaluation, the stability and coordination of Tai Chi practitioners' movements can be objectively and accurately evaluated. The invention effectively reduces the subjective errors in manual scoring, improves the scientificity and fairness of movement scoring, and helps practitioners discover and improve unstable points in movements, thereby improving the effect of practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic flow chart of a method for evaluating Tai Chi movement coordination based on center of gravity height detection provided by the present invention; Figure 2 A schematic diagram of key points in the Tai Chi movement coordination evaluation method based on center of gravity height detection provided by the present invention; Figure 3 A schematic diagram of body segments in the Tai Chi movement coordination evaluation method based on center of gravity height detection provided by the present invention; Figure 4 This is a schematic diagram of the score ranges corresponding to different movements of the Tai Chi movement coordination evaluation method based on center of gravity height detection provided by the present invention. DETAILED DESCRIPTION

[0017] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or apparatus.

[0019] The embodiment of the present invention discloses a Tai Chi movement coordination evaluation method based on center of gravity height detection, which solves the problem of scoring errors caused by subjective factors, so as to help Tai Chi practitioners improve the quality of their movements and obtain more accurate scoring results.

[0020] Specifically, the method comprises the following steps: Step S1, obtaining Tai Chi action video and identifying the three-dimensional coordinates of human joints.

[0021] Obtain the Tai Chi action video to be scored. The Tai Chi action video needs to capture the complete body of the subject to be scored, so as to better capture the movements of the human joints. By preprocessing the Tai Chi action video, the quality of the video is optimized to ensure that the video data can be used for posture recognition and subsequent scoring algorithms.

[0022] Among them, the preprocessing includes converting the Tai Chi action video into a frame sequence, timestamp synchronization, image adjustment, denoising, background removal, etc. Among them, the frame sequence is to decompose the Tai Chi action video frame by frame, and each frame represents a static image. The frequency of the frame depends on the frame rate (FPS) of the video. For example, a 30 FPS video has 30 frames per second, which is converted into 30 static images.

[0023] In step S1, the three-dimensional coordinates of the human body joints are identified, including: Step S11, extracting human body posture data of each frame image in the Tai Chi action video, wherein the human body posture data includes two-dimensional coordinates of human body joints, and constructing a two-dimensional coordinate sequence of the key points.

[0024] Yolov3 is used to frame the object to be rated in the static image. Yolov3 is a fast and accurate target detection algorithm that can identify various objects in the image. In this process, Yolov3 is mainly used to detect the object to be rated from each frame of the video and frame its position. Only the detection results of the person are retained, and the rest of the detected objects such as objects in the background will be ignored.

[0025] After extracting the object to be scored, the 2D coordinates of the human joints are obtained through the 2D pose estimator. Usually, a pre-trained 2D pose estimator (such as OpenPose or AlphaPose) is used to detect the position of the human joints of the object to be scored, including the head, shoulders, elbows, hands, knees, ankles, etc. In this application, the position of each human joint is represented by the two-dimensional coordinates of the key point, and the coordinate form is (x, y), where x and y represent the horizontal and vertical positions of the human joint in the image, respectively. In this application, Figure 2 As shown, the key points include pelvis, head, left hip, right hip, left knee, right knee, left foot, right foot, spine, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, and chest.

[0026] The human posture data of all frames in the Tai Chi action video are connected in series to form a two-dimensional coordinate sequence of human joints including time for subsequent action analysis. If the Tai Chi action video has N frames and each frame has J human joints, the data format obtained is: N * J * 2, where 2 represents the two-dimensional matrix (x, y) composed of the two-dimensional coordinate sequence.

[0027] Step S12: extract the features of the two-dimensional coordinate sequence based on the multi-head attention mechanism, and output a three-dimensional posture hypothesis of the human joints; the three-dimensional posture hypothesis includes the three-dimensional coordinates (x, y, z) of the key points in each frame image in the Tai Chi action video, where x and y represent the horizontal position and vertical position of the key point on the image, and z represents the distance between the key point and the shooting point.

[0028] Based on the multi-head attention mechanism, the required features are extracted from the two-dimensional coordinate sequence and converted into a group of initial three-dimensional posture hypotheses. Each group of three-dimensional posture hypotheses is a possible 3D interpretation of the human body posture. Assuming that there are M three-dimensional posture hypotheses, it can be represented as a list including M elements, each of which is a posture hypothesis. Each 3D posture hypothesis is usually a matrix, the rows of the matrix correspond to the frames in the Tai Chi action video, and the columns of the matrix correspond to the three-dimensional coordinates of the human body joints. Exemplarily, there are N frames in a Tai Chi action video, each frame contains the three-dimensional coordinates of J human body joints, then the shape of each three-dimensional posture hypothesis is N * (J · 2). Among them, 2 represents a two-dimensional matrix, and N * (J · 2) as a whole represents that the shape of each three-dimensional posture hypothesis is N two-dimensional matrices of J joints.

[0029] The 3D posture hypothesis is input into the multi-head attention model, and the dependencies between different joints of the human body are captured by parallelizing multiple attention heads. Each head is responsible for processing different joint interaction characteristics, such as the relationship between the left shoulder and the left elbow, the relationship between the left leg and the pelvis, the relationship between the left knee and the left foot, etc.

[0030] The specific process is as follows: Concatenate the key point coordinates (x, y) of the human joints in each frame in the two-dimensional coordinate sequence to form a new matrix , where each row represents the 2D coordinates of a key point in a frame.

[0031] Introducing spatial position vector , in order to preserve the spatial information of human joints, which helps the multi-head attention model understand the relative position and displacement of each human joint between different frames.

[0032] Normalize the concatenated matrix and embed the spatial position vector to obtain the initialized 3D pose hypothesis. .

[0033] Input to the multi-head attention model, processed by the multi-head attention mechanism (MSA) in the first layer , and normalized by LayerNorm, and then the result is compared with the original 3D pose hypothesis Add together and get , realizing residual connection. Multilayer perceptron (MLP) is used to further output the 3D pose hypothesis processed by MSA mechanism and normalized by LayerNorm, plus itself, again realizing the residual connection.

[0034] In the last layer L1, the processed features are normalized by LayerNorm and compared with the initial 3D pose hypothesis. Add together to get the final 3D posture hypothesis, that is, the 3D posture data . Where m is the number of different three-dimensional posture data, L1 is the number of layers in the MHG module (Multi-Head Graph Module); N is the number of frames in the Tai Chi action video, and J is the number of human joints. The output three-dimensional posture assumes that the human pelvis (0) is set as the coordinate origin. The pelvis is usually a stable point in the center of the human body and is suitable as a reference point to calculate and describe the positions of other joints. X is the left and right direction in the human posture, that is, the position change of the human joints in the horizontal direction, and the Y axis is the vertical direction in the human posture, that is, the position change of the human joints in the vertical direction. The Z axis represents the front and back direction, that is, the position change of the human joints in depth. The general pelvic coordinates are (0,0,0), and other joint points are vectors relative to the pelvis.

[0035] The multi-head attention mechanism processes the two-dimensional coordinate sequence, extracts the complex relationship between the joints in the human body posture, and generates a variety of three-dimensional posture data. In this way, a more accurate three-dimensional posture can be inferred based on the two-dimensional plane information, providing key data support for subsequent center of gravity calculation, motion analysis and scoring. After processing based on the multi-head attention mechanism, the final 3D posture data is output , that is, three-dimensional posture data. Among them, three-dimensional posture data It includes the three-dimensional coordinates (x, y, z) of the key points in each frame of the Tai Chi action video, where x and y represent the horizontal position and vertical position of the key point on the image, and z represents the distance between the key point and the shooting point.

[0036] Since the vertical distance between the camera and the subject to be scored will affect the coordinates of the subject in the Tai Chi action video, in order to eliminate the scoring error caused by this factor, the following are also included: Step S13, determining a scaling ratio, and scaling the three-dimensional coordinates in proportion according to the scaling ratio; the scaling ratio is determined according to a vertical distance between the key point and the shooting point or a distance between preset key points.

[0037] For example, in the three-dimensional coordinates (x, y, z) of the key point, z represents the distance between the key point and the shooting point, so the distances of all key points are scaled proportionally. Set a target distance z_target, which can be an ideal standard distance, such as the standard distance between the human body and the camera. All subsequent joint coordinates will be scaled based on the target distance.

[0038] Use the formula to calculate the scaling ratio scale = z_target / z, where z is the actual distance between the key point and the camera in the current frame, and z_target is the standard distance. Apply the scaling formula to each key point (x, y) coordinate in each frame, and the position of the adjusted key point in the image will be consistent with the coordinates at the standard distance.

[0039] In another embodiment, the scaling ratio can also be determined based on the height. By estimating the height of the subject to be scored, the scaling ratio of the three-dimensional posture data can be calculated, thereby standardizing the three-dimensional coordinates.

[0040] For example, the distance d between the key point head and the key point neck is calculated, and the estimated height is obtained according to the human body proportion. Generally, the distance from the head to the neck is 1 / 6 of the total height, and the estimated height is h=6*d. This height represents the relative height of the object to be rated in the current video or image.

[0041] Based on this, a standard height h_standard is assumed, which represents the ideal height in this method and can be the standard height in the training data. The scaling ratio scale can be calculated by the following formula: scale = h_standard / h. By estimating the height and scaling the ratio, the human joints in all videos are standardized, which helps to make consistent comparisons when scoring postures. All key points are scaled in turn to ensure that all key points are scaled proportionally to eliminate the deviation caused by the change in the distance of the shooting device. By scaling the coordinate points proportionally, the coordinate deviation caused by the different distances between the shooting device and the key points is eliminated, and the three-dimensional posture data is kept consistent between different video frames or different videos, which effectively improves the accuracy and reliability of this method.

[0042] For example, the coordinates of the key point 0 pelvis are ( , , and scaled proportionally using the following formula: ( , ) = scale( ,

[0043] After proportional scaling, the final coordinates of the pelvis are ( , ) Step S2, identifying Tai Chi movements based on the three-dimensional coordinates; Step S21, constructing a training data set, wherein the training data set includes time series data of annotated action moves and key points; Step S22, using the training data set as input and Tai Chi movements as output to train a long short-term memory model; Step S23, identifying the Tai Chi movements corresponding to the three-dimensional coordinates through the trained long short-term memory model.

[0044] In order to build a 24-style Tai Chi move recognition model, experts first annotate the moves according to the standard video and frame number, and process these annotated 24 Tai Chi moves through step S1 to obtain three-dimensional posture data. The time series data of the three-dimensional coordinates of 17 key points in the three-dimensional posture data and the annotated Tai Chi moves constitute the training data set. The training data set is input into the long short-term memory (LSTM) model, and the action moves are output to train the corresponding long short-term memory model. The LSTM model is a recursive neural network that can process time series data. It is good at capturing temporal dependencies and is particularly suitable for identifying continuous action changes.

[0045] By training the LSTM model, we can identify different Tai Chi movements from the time series data of key points of the human body. When new three-dimensional coordinate data is input, the LSTM model can predict the Tai Chi movements corresponding to the time series data based on the knowledge it has learned, thereby realizing the recognition of different Tai Chi movements.

[0046] Step S3, calculating the center of gravity data corresponding to each action form based on the three-dimensional coordinates; the center of gravity data includes the center of gravity of the human body and the height of the center of gravity of the human body.

[0047] Calculating the center of gravity data corresponding to each action based on the three-dimensional coordinates includes: Step S31: Divide the human body into segments based on the key points, and calculate the centroid of each segment in each frame of the image.

[0048] Since the human body is a complex model composed of different body parts, the corresponding operations are achieved through the cooperation between different body parts during daily exercise. Figure 3As shown in the figure, based on the anatomical structure of the human body, the human body is divided into 11 different body segments according to 17 key points of the human joints. The body segments include the head and neck, chest and abdomen, pelvis, right thigh, right calf, left thigh, left calf, right upper arm, right forearm, left upper arm, and left forearm. Each body segment is connected by two key points, the one close to the heart is called the proximal end of the body segment, and the one far from the heart is called the distal end of the body segment.

[0049] In some embodiments, the body parts can be divided according to the actual situation, for example, the upper arm and the forearm can be combined into the arm, the thigh and the calf can be combined into the leg, or the head and neck can be combined with the chest and abdomen, etc., so that the adjacent body parts can be combined into a larger body part. The simplified division method can be adjusted according to specific needs to adapt to different analysis scenarios and improve calculation efficiency.

[0050] Calculating the body segment center of gravity of each body segment in each frame of the image, including: The proximal coefficient and the distal coefficient of the body segment are obtained, and the body segment centroid of each body segment is calculated in combination with the three-dimensional coordinates of the key points of each body segment, wherein the proximal coefficient and the distal coefficient respectively represent the relative weight of a body segment in the centroid calculation.

[0051] Table 1 Structural parameters of human body segments

[0052] Note: k is the mass coefficient of the body segment, and The proximal coefficient and the distal coefficient are With reference to the human body segment structural parameters in Table 1, the body segment center of gravity of 11 body segments is calculated by the body segment center of gravity calculation formula, that is, the three-dimensional coordinate point of the body segment center of gravity is calculated.

[0053] The formula for calculating the center of gravity of a body segment is as follows: X = +

[0054] Y = +

[0055] Z = +

[0056] Where (X, Y, Z) is the three-dimensional coordinate of the centroid of the body segment, ( , , ) is the coordinate of one of the key points constituting the centroid of the body segment, ( , , ) is the coordinate of another key point constituting the body segment; and are the corresponding proximal and distal coefficients. Taking the right calf as an example, the proximal coordinates are the coordinates of the right knee key point, and the distal coordinates are the coordinates of the right foot key point. Then multiply them with the corresponding proximal and distal coefficients of the calf to obtain the coordinates of the center of gravity of the right calf body segment in the current state. The coordinates of the center of gravity of other body segments are calculated in the same way as the right calf.

[0057] Step S32, calculating the center of gravity of the human body in each frame of image according to the moment synthesis algorithm, and calculating the height of the center of gravity of the human body and the height change of the center of gravity of each action and form based on the center of gravity of the human body.

[0058] If each part is regarded as an independent rigid body model, the whole human body is a multi-rigid body model structure. According to the moment composition theorem, the resultant moment of the body's center of gravity is the algebraic sum of the moment of each part of the body. Therefore, in this application, the moment composition method is used to calculate the center, which is more accurate than the multiplication coefficient method to reasonably simplify the structure of the human body.

[0059] Specifically, the center of gravity of the human body of each frame image is calculated according to the moment synthesis algorithm, including: The body segment mass coefficient is obtained, and the body segment center of gravity of each body segment and the body segment mass coefficient are combined to calculate the center of gravity of the human body in each frame of image. The body segment mass coefficient is used to represent the mass ratio of each body segment relative to the whole body. The mass coefficient of each body segment reflects the importance of the body segment in the calculation of the total center of gravity. The body segment mass coefficient is used to weightedly calculate the contribution of each body segment to the total center of gravity of the human body. Body segments with larger mass play a more important role in the center of gravity calculation, which is reflected in the center of gravity position in the human posture and motion analysis.

[0060] The formula for calculating the center of gravity of the human body is as follows: =

[0061] =

[0062] =

[0063] Among them: The center of gravity of the human body is ( , ), i= {0,1,2,…,10}, representing the serial number of the body segment. , , ) represents i The coordinates of the centroid of the body segment of the individual segment; k is the body segment coefficient.

[0064] Calculate the height of the human body's center of gravity and the height change of the center of gravity of each action based on the center of gravity of the human body, including: Step S321, calculate the height of the human body center of gravity for each frame image by combining the first distance and the second distance, wherein the first distance is the distance between the human body center of gravity and the pelvic key point in the vertical direction; and the second distance is the distance between the left foot key point, the right foot key point and the pelvic key point in the vertical direction.

[0065] Calculate the height of the human body's center of gravity using the formula for the height of the human body's center of gravity: = – ( + ) / 2 in, is the height of the center of gravity, is the distance between the center of gravity of the human body and the key points of the pelvis in the vertical direction, that is, the distance between the center of gravity of the human body and the pelvis on the Y axis, that is, the ordinate of the center of gravity of the human body, The left and right feet are the distances from the pelvis on the Y axis, i.e. the vertical coordinates of the left and right feet. The foot is the point closest to the ground, and the average value of the left and right feet can represent the height from the ground. The human body's center of gravity height is standardized to the distance based on the foot through the human body center of gravity height calculation formula, that is, the plane where the foot is located is regarded as the ground, and the vertical distance of the human body's center of gravity relative to the ground is calculated from the ground, i.e. the human body center of gravity height.

[0066] Step S322, constructing a two-dimensional array of the time of the Tai Chi action video and the height change of the center of gravity of the human body, and obtaining a center of gravity height change curve based on the two-dimensional array; Match the height of the human body's center of gravity with the timeline of the Tai Chi action video, and first calculate the total duration of the video. The total number of video frames is frame_number, and the number of frames per second (FPS) is used to correspond the number of frames to the duration of the video, usually 30 or 60 FPS.

[0067] The specific relationship is as follows: total_seconds = frame_number / FPS minutes = int(total_seconds / / 60) seconds = int(total_seconds % 60) Convert the total seconds into minutes and seconds to get the total duration of the video so that the change of the center of gravity of the human body can be matched with time later. A two-dimensional array of changes, which is used to represent the changes in the center of gravity height during the Tai Chi action video time. Based on this two-dimensional array, a center of gravity height change curve is drawn to clearly show how the center of gravity of the human body changes over time during Tai Chi, and intuitively show the stability and coordination of the action. If the center of gravity change curve is smooth and stable, it means that the coordination of the action is good and the action is coherent. If the center of gravity change curve fluctuates greatly and the center of gravity fluctuates frequently, it may mean that the action is not stable enough and the coordination is poor.

[0068] Step S323, determine the highest point and the lowest point of the center of gravity of each action and calculate the height difference of the center of gravity of each action and move.

[0069] According to the action moves identified in step S2, determine the highest point of the center of gravity and the lowest point of the center of gravity of each move, and calculate the height difference of each move height : height = －

[0070] Arrange the center of gravity height differences according to the action moves, and also generate a two-dimensional array of moves-height differences.

[0071] Step S4: Scoring the coordination based on the constructed standard data and the center of gravity data of the Tai Chi action video.

[0072] Coordination scoring is performed based on the constructed standard data and the center of gravity data of the Tai Chi action video, including: Step S41, constructing standard data; the standard data is the center of gravity height, center of gravity height difference and center of gravity height change trend of each action move in the standard video, and determining the score interval of each action move according to the standard data; Standard videos are Tai Chi teaching videos or practice videos demonstrated by experts, with highly standardized movements. The standard videos are constructed into standard data through the above steps. The standard data includes movement moves, calculated center of gravity height, center of gravity height difference, and then the score range of each movement move is determined.

[0073] For example, for the same action, the corresponding scoring area is set according to the height difference of the center of gravity. For example, the height difference of the center of gravity is between 9.0 and 15.0: the score is 10 points; the height difference of the center of gravity is between 15.1 and 20.0: the score is 8 points; the height difference of the center of gravity is between 20.1 and 25.0: the score is 6 points; the height difference of the center of gravity is greater than 25.0: the score is 4 points or less.

[0074] like Figure 4 As shown in the figure, different action moves are provided with corresponding scoring areas. Among them, "'23.0': [(9.0, 15.0, 10) ...]" means that when the height difference of the center of gravity corresponding to the 23rd action move is greater than 9.0 and not greater than 15.0, the score is 10 points.

[0075] Step S42: align the time axis of the center of gravity height change curves of the standard video and the Tai Chi action video using the DTW algorithm, and determine the coordination score of each action form in the Tai Chi action video according to the score range.

[0076] In order to more accurately compare the center of gravity change curves, the Dynamic Time Warping (DTW) algorithm is used. The DTW algorithm can align time series of different lengths so that the center of gravity changes in different videos can be matched on the timeline to view trend changes.

[0077] For two time series of different videos: Tai Chi action video Q =（ ,…, ） and standard video C=（ ,…, ） , lengths are n and m respectively. Construct an n*m matrix D ,in D[i,j] express and and the Euclidean distance between them, find a path from [1,1] to [n,m] so that the cumulative Euclidean distance of the path is minimized. At this time, the two videos are considered to be aligned.

[0078] After DTW alignment, the difference between the object to be rated and the standard video can be compared more accurately, the coordination of movements can be evaluated, and the accuracy of movement evaluation can be improved.

[0079] This method processes Tai Chi action videos, calculates the height of the human body's center of gravity and the analysis of its height changes, and combines the height difference and change trend of the center of gravity in the standard video to establish a clear scoring range, which can objectively and accurately evaluate the movement stability and coordination of Tai Chi practitioners. It effectively reduces the subjective errors in manual scoring, improves the scientificity and fairness of action scoring, and helps practitioners discover and improve unstable points in their movements, thereby improving the effect of their practice.

[0080] Based on the same inventive concept, the present application also provides a Tai Chi movement coordination evaluation device based on center of gravity height detection, comprising: The three-dimensional coordinate recognition module obtains Tai Chi action videos and recognizes the three-dimensional coordinates of human joints; An action and gesture recognition module, which recognizes Tai Chi action and gestures based on the three-dimensional coordinates; A center of gravity data calculation module calculates the center of gravity data corresponding to each action based on the three-dimensional coordinates; the center of gravity data includes the center of gravity of the human body and the height of the center of gravity of the human body; The coordination scoring module performs coordination scoring based on the constructed standard data and the center of gravity data of the Tai Chi movement video.

[0081] The processing method of the device can refer to the description of the above method and will not be repeated here.

[0082] The present application also provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute the steps of the Tai Chi movement coordination evaluation method based on center of gravity height detection of the present invention.

[0083] An embodiment of the present application also provides a non-transitory machine-readable storage medium, on which an executable program is stored. When the executable program is executed by a processor, the processor executes the method provided in the above embodiment.

[0084] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the described method.

[0085] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the described method.

[0086] The embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without paying creative labor.

[0087] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0088] Finally, it should be noted that the embodiments disclosed in the present invention are only preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A Tai Chi movement coordination evaluation method based on center of gravity height detection, characterized in that: The method comprises the following steps: Obtain Tai Chi action videos and identify the three-dimensional coordinates of human joints; Recognize Tai Chi movements based on the three-dimensional coordinates; Calculate the center of gravity data corresponding to each action based on the three-dimensional coordinates; the center of gravity data includes the center of gravity point of the human body and the height difference of the center of gravity of the human body; Coordination scores were made based on the constructed standard data and the center of gravity data from the Tai Chi movement video.

2. The Tai Chi movement coordination evaluation method based on center of gravity height detection according to claim 1 is characterized in that: Identify the 3D coordinates of human joints, including: Extracting human body posture data of each frame image in the Tai Chi action video, wherein the human body posture data includes two-dimensional coordinates of human body joints, and constructing a two-dimensional coordinate sequence of the human body joints; Based on the multi-head attention mechanism, the features of the two-dimensional coordinate sequence are extracted, and the three-dimensional posture data of the human joints are output; the three-dimensional posture data includes the three-dimensional coordinates (x, y, z) of the key points in each frame image in the Tai Chi action video, where x and y represent the horizontal position and vertical position of the key point on the image, and z represents the distance between the key point and the shooting point.

3. The Tai Chi movement coordination evaluation method based on center of gravity height detection according to claim 2 is characterized in that: Calculating the center of gravity data corresponding to each action based on the three-dimensional coordinates includes: Divide the body parts of the human body based on the key points, and calculate the body segment center of gravity of each body segment in each frame of the image; The center of gravity of the human body is calculated for each frame according to the moment synthesis algorithm, and the height of the human body and the height change of the center of gravity of each action are calculated based on the center of gravity of the human body.

4. The Tai Chi movement coordination evaluation method based on center of gravity height detection according to claim 3 is characterized in that: Calculate the height of the human body's center of gravity and the height change of the center of gravity of each action based on the center of gravity of the human body, including: Calculate the height of the center of gravity of the human body in each frame of image by combining the first distance and the second distance, wherein the first distance is the distance between the center of gravity of the human body and the key point of the pelvis in the vertical direction; and the second distance is the distance between the key point of the left foot, the key point of the right foot and the key point of the pelvis in the vertical direction; Constructing a two-dimensional array of the time of the Tai Chi action video and the height change of the center of gravity of the human body, and obtaining a center of gravity height change curve based on the two-dimensional array; Determine the highest and lowest points of the center of gravity for each move, and calculate the height difference of the center of gravity for each move.

5. The Tai Chi movement coordination evaluation method based on center of gravity height detection according to claim 3 is characterized in that: Calculating the body segment center of gravity of each body segment in each frame of the image, including: Obtaining the proximal coefficient and the distal coefficient of the body segment, and calculating the body segment centroid of each body segment in combination with the three-dimensional coordinates of the key points in each body segment; The center of gravity of the human body is calculated for each frame of the image according to the moment synthesis algorithm, including: The body segment mass coefficient is obtained, and the body segment center of gravity of each body segment and the body segment mass coefficient are combined to calculate the body center of gravity of each frame of the image.

6. The method for evaluating Tai Chi movement coordination based on center of gravity height detection according to claim 4, characterized in that: Coordination scoring is performed based on the constructed standard data and the center of gravity data of the Tai Chi action video, including: Construct standard data; the standard data is the center of gravity height, center of gravity height difference and center of gravity height change trend of each action move in the standard video, and determine the score range of each action move according to the standard data; The DTW algorithm is used to align the time axis of the center of gravity height change curves of the standard video and the Tai Chi action video, and the coordination score of each action in the Tai Chi action video is determined according to the score range.

7. The Tai Chi movement coordination evaluation method based on center of gravity height detection according to claim 1 is characterized in that: Recognizing Tai Chi movements based on the three-dimensional coordinates includes: Constructing a training data set, wherein the training data set includes time series data of annotated action moves and key points; Taking the training data set as input and Tai Chi movements as output, training a long short-term memory model; The Tai Chi movements corresponding to the three-dimensional coordinates are identified through a long short-term memory model.

8. A Tai Chi movement coordination evaluation device based on center of gravity height detection, characterized in that: include: The three-dimensional coordinate recognition module obtains Tai Chi action videos and recognizes the three-dimensional coordinates of human joints; An action and gesture recognition module, which recognizes Tai Chi action and gestures based on the three-dimensional coordinates; A center of gravity data calculation module calculates the center of gravity data corresponding to each action based on the three-dimensional coordinates; the center of gravity data includes the center of gravity point of the human body and the height difference of the center of gravity of the human body; The coordination scoring module performs coordination scoring based on the constructed standard data and the center of gravity data of the Tai Chi movement video.

9. A computer device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the Tai Chi movement coordination evaluation method based on center of gravity height detection as claimed in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for evaluating Tai Chi movement coordination based on center of gravity height detection as described in any one of claims 1 to 7 is implemented.

Images