Eight-section brocade action evaluation method and system based on dynamic planning and deep learning

Abstract

The invention provides an eight-segment brocade action evaluation method and system based on dynamic planning and deep learning. The method comprises the steps of collecting eight-segment brocade action video data; a matching path, a DTW distance and a single frame score are obtained through an eight-segment brocade action recognition model based on MediaPipe and DTW, and a feature sequence of hub actions output by the eight-segment brocade action recognition model is subjected to uniform frame extraction processing and then is input into an eight-segment brocade action evaluation model based on LSTM to obtain classification labels of user hub actions. And reflecting errors of the user on key actions according to the classification labels. According to the method, skeleton points are extracted through a Mediape framework, the total score is obtained according to the DTW distance, the similarity between the overall recruitment of the user and the standard action is evaluated according to the rhythm score, and the rhythm stability of the action can be reflected in detail according to the single-frame score; the problem that the length of a video sequence is unstable is solved by adopting a uniform frame extraction method, and the model is ensured to effectively learn and recognize action features.

Description

Technical Field

[0001] The present invention relates to the technical field of motion recognition, and in particular to a method and system for evaluating a Ba Duan Jin motion based on dynamic programming and deep learning. Background Art

[0002] As an ancient traditional Chinese exercise, Ba Duan Jin has unique health-preserving value and is simple and easy to perform. However, despite its many benefits, it is difficult to provide practitioners with real-time and accurate posture assessment and guidance due to the lack of effective technical means.

[0003] Traditional teaching methods mainly rely on the coach's experience and personal observation, which is not only inefficient but also prone to inconsistent guidance due to subjective factors. In order to solve the problems of traditional technologies, people have tried to use smartphones and wearable devices to assist in the practice of Ba Duan Jin, but these devices still need to be improved in terms of accuracy and real-time performance. With the development of computer vision and machine learning technology, motion capture and data analysis can now be used to assist sports training and improve the standardization of movements and learning efficiency. However, existing technologies have not yet provided a complete recognition and evaluation method for complex sequence movements such as Ba Duan Jin. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a method and system for evaluating Ba Duan Jin movements based on dynamic programming and deep learning. The present invention obtains a matching path, a DTW distance and a single-frame score through a Ba Duan Jin movement recognition model, and obtains a feature sequence of a pivot movement according to the matching path; the feature sequence of the pivot movement output by the Ba Duan Jin movement recognition model is uniformly subjected to frame extraction processing and then input into a Ba Duan Jin movement evaluation model to obtain a classification label of the user's pivot movement, and reflects the user's errors in key movements according to the classification label.

[0005] The technical solution of the present invention is: a method for evaluating the movements of Ba Duan Jin based on dynamic programming and deep learning, comprising the following steps:

[0006] S1), collecting Ba Duan Jin action video data, and dividing it into a training set and a validation set after annotating it;

[0007] S2) Build and train the Ba Duan Jin action recognition model based on MediaPipe and the dynamic time warping algorithm DTW, obtain the matching path, DTW distance and single frame score through the trained Ba Duan Jin action recognition model, and obtain the feature sequence of the pivot action according to the matching path;

[0008] S3) construct and train a Ba Duan Jin action evaluation model based on a long short-term memory network (LSTM), perform uniform frame extraction on the feature sequence of the pivot action output by the Ba Duan Jin action recognition model, and then input it into the trained Ba Duan Jin action evaluation model to obtain the classification label of the user's pivot action, and reflect the user's errors in key actions according to the classification label;

[0009] S4), deploying the trained Ba Duan Jin motion recognition model and Ba Duan Jin motion evaluation model to the cloud server to recognize and evaluate the user's Ba Duan Jin motion.

[0010] Preferably, in step S1), the collected video data of the user's Eight-Section Brocade movements are segmented into continuous image segments according to the frame rate, and each image segment represents the posture state of the user at a specific time point.

[0011] Preferably, in step S2), the three-dimensional spatial coordinate data of the human skeleton points are identified from the Ba Duan Jin action video data by the Pose method based on MediaPipe, and a human motion model is constructed according to the human skeleton points.

[0012] Preferably, in step S2), the angle index of each joint of the human body and the time series of the angle index are obtained according to the human motion model.

[0013] Preferably, in step S2), the matching path, DTW distance and single-frame score between the time series of the user action and the time series of the standard action video are calculated by the dynamic time warping algorithm DTW, the matching path is used to identify the user action, and the feature sequence of the pivot action is returned, the DTW distance is output as the overall similarity score of the moves through the scoring function, and the scoring result of the user action rhythm is obtained through the single-frame scoring.

[0014] Preferably, in step S2), the matching path is used for action recognition, and a feature sequence of the pivot action is returned.

[0015] The calculation method of the single frame score is as follows: the cosine distance calculation is performed on each frame of the user's action and the standard frame that matches it, and the scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain the single frame score of each frame of the user's action. d ,Right now:

[0016]

[0017] Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action.

[0018] As a preferred step, in step S2), the DTW distance is converted into the overall similarity score of the moves using a scoring function. dwt , the overall similarity score dwt The calculation formula is as follows:

[0019]

[0020] Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great The action is excellent.

[0021] As a preferred embodiment, in step S2), the rhythm score result score of the user action is calculated by scoring the single frame. r , expressed as:

[0022]

[0023] In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

[0024] Preferably, in step S3), the uniform frame extraction process is as follows: for a given target sequence length and original feature sequence length, a frame extraction interval is calculated, and then the original feature sequence is uniformly extracted according to the frame extraction interval.

[0025] Preferably, in step S3), the length of the original feature sequence is seq_len, and the given target sequence length is target_len; then the frame extraction interval interval = seq_len / target_len.

[0026] Preferably, in step S3), for the frame extraction interval interval, starting from the 0th frame of the original feature sequence, one frame is extracted every interval frame until the target sequence length target_len is reached, and finally the remaining frames are processed. If the original feature sequence length seq_len cannot be divided by the target sequence length target_len, the last frame of the original feature sequence is ignored.

[0027] Preferably, in step S3), the single-frame features after uniform frame extraction are input into the Eight-Section Brocade action evaluation model based on the long short-term memory network LSTM in chronological order, and the hidden state is learned and updated through the internal gating mechanism to capture the time dependency of the Eight-Section Brocade action and output the classification of the user action.

[0028] Preferably, the present invention further provides a Ba Duan Jin action evaluation system based on dynamic programming and deep learning, comprising:

[0029] A data collection module is used to collect the user's Ba Duan Jin action video data;

[0030] The human skeleton point recognition module uses the Pose method based on MediaPipe to recognize the three-dimensional spatial coordinate data of the human skeleton points from the Ba Duan Jin action video data, and builds a human action model based on the human skeleton points;

[0031] The dynamic time warping module is used to calculate the matching path, DTW distance and single-frame score between the time series of the user's actions and the time series of the standard action video, and obtain the feature sequence of the user's pivot action based on the matching path, and obtain the overall similarity score of the user's moves based on the DTW distance, and obtain the score result of the user's action rhythm through the single-frame score;

[0032] The action evaluation module is used to capture the time dependency of the Ba Duan Jin action from the single-frame features after uniform frame extraction according to the Ba Duan Jin action evaluation model based on the long short-term memory network (LSTM), and obtain the classification result of the user action.

[0033] Preferably, the single frame score is calculated as follows:

[0034] The cosine distance is calculated for each frame of the user's action and the standard frame that matches it. The scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain the single-frame score of each frame of the user's action. d ,Right now:

[0035]

[0036] Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action.

[0037] As a preferred method, a scoring function is used to convert the DTW distance into an overall similarity score of the moves. dwt , the overall similarity score dwt The calculation formula is as follows:

[0038]

[0039] Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great The action is excellent.

[0040] As a preferred method, the rhythm score of the user's action is calculated by scoring a single frame. r , expressed as:

[0041]

[0042] In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

[0043] Preferably, the uniform frame extraction process is as follows: for a given target sequence length and original feature sequence length, a frame extraction interval is calculated, and then the original feature sequence is uniformly extracted according to the frame extraction interval.

[0044] The beneficial effects of the present invention are:

[0045] 1. The present invention extracts skeleton points through the Mediapipe framework, takes joint angle indicators as timing elements, performs timing regularization calculation with the sequence of standard videos, takes the matching path, and reversely assigns user video action labels according to the index of the standard action video timing to achieve the recognition effect;

[0046] 2. The present invention obtains a total score based on the DTW distance, and evaluates the similarity between the user's overall moves and the standard moves based on the rhythm score. The cosine distance calculation is performed for each frame of the user's moves and the standard frame that matches it. According to the property that the maximum value of the cosine distance is 2, a scoring function is set to obtain a single-frame score for each frame, and statistics are performed according to the score interval; thereby, a single-frame score segment distribution is obtained, which can further reflect the rhythm stability of the action in detail, and further use entropy as a normalized feature to evaluate the rhythm of the user's action, thereby objectively quantifying the consistency and stability of the action;

[0047] 3. The present invention adopts a uniform frame extraction method to solve the problem of unstable length of video sequences, ensuring that the model can effectively learn and recognize action features. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the method of the present invention;

[0049] Figure 2 A schematic diagram of 33 skeleton points identified by MediaPipe of the present invention;

[0050] Figure 3 A schematic diagram of a human motion model constructed for the present invention;

[0051] Figure 4 It is a schematic diagram of the process of uniformly extracting frames of the present invention;

[0052] Figure 5 It is a schematic diagram of the structure outputted by the present invention;

[0053] Figure 6 It is a structural framework diagram of the system of the present invention. DETAILED DESCRIPTION

[0054] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0055] Example 1

[0056] like Figure 1 As shown, this embodiment provides a method for evaluating the movements of Ba Duan Jin based on dynamic programming and deep learning, comprising the following steps:

[0057] S1), collecting Ba Duan Jin action video data, and dividing it into a training set and a validation set after annotating it;

[0058] In this embodiment, video data of Eight Section Brocade movements are collected from different groups of people and in different environments, wherein the annotated content includes information such as the movement category, start time, end time, and standard degree of the movement; and the collected video data of Eight Section Brocade movement are divided into continuous image segments according to the frame rate, and each image segment represents the posture state of the user at a specific time point.

[0059] In this embodiment, the eight moves of Ba Duan Jin are: holding up both hands to regulate the triple burner, opening the bow with both hands like shooting an eagle, regulating the spleen and stomach by lifting one hand, looking back to relieve the five kinds of fatigue and seven kinds of injuries, shaking the head and tail to relieve the heart fire, climbing the feet with both hands to strengthen the kidneys and waist, clenching the fists and glaring to increase the energy, and shaking the back seven times to eliminate all diseases, as shown in Table 1:

[0060] Table 1 Eight moves of Ba Duan Jin

[0061]

[0062]

[0063] The number of frames of the extracted standard video is shown in Table 2.

[0064] Table 2 Frame numbers of standard videos

[0065]

[0066] S2) Build and train the Ba Duan Jin motion recognition model based on MediaPipe and dynamic time warping algorithm DTW, obtain the matching path, DTW distance and single frame score through the trained Ba Duan Jin motion recognition model, and obtain the feature sequence of the pivot action according to the matching path.

[0067] like Figure 2 and 3As shown, this embodiment analyzes each frame of image by the Pose method based on MediaPipe, and extracts 33 bone point information of the user's arms, elbows, knees, ankles and shoulders from the image, and forms three-dimensional coordinate data; then calculates the angle between the skeletons formed by the bone points of the hands and legs of each image segment. And constructs a human motion model based on the 33 human bone points. According to the human motion model, the angle index of each joint of the human body and the time series of the angle index are obtained.

[0068] Then, the dynamic time warping algorithm DTW is used to calculate the matching path, DTW distance and single-frame score between the time series of the user's actions and the time series of the standard action video. The matching path is used to identify the user's actions and return the feature sequence of the pivot action. The DTW distance is output as the overall similarity score of the moves through the scoring function, and the scoring result of the user's action rhythm is obtained through the single-frame scoring.

[0069] This embodiment takes "drawing the bow left and right like shooting an eagle" as an example, and identifies the three actions of "holding the wrist", "drawing the bow" and "stepping side by side" through matching paths, among which the "drawing the bow" action is used as the pivotal action to define the key points of the error.

[0070] The matching path is used for action recognition and returns a feature sequence of the pivot action.

[0071] The DTW distance calculation method is:

[0072]

[0073] Where d represents the DTW distance between the current user action time series and the standard action time series; A represents the angle index set of the standard action; B represents the angle index set of the user action; a i is the i-th joint angle in the standard action; b i represents the i-th joint angle in the user action; Φ represents the frame number set of the standard video and the user action video.

[0074] The calculation method of the single frame score is as follows: the cosine distance calculation is performed on each frame of the user's action and the standard frame that matches it, and the scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain the single frame score of each frame of the user's action. d ,Right now:

[0075]

[0076] Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action;

[0077] Among them, the expression of cosine similarity sim(A,B) is:

[0078] sim(a i ,b j )=1-cos(a i ,b j )

[0079] A={a1,a2,a3,…,a n}

[0080] B={b1,b2,b3,…,b m}

[0081] In the formula, a i is the i-th joint angle in the standard action; b i represents the i-th joint angle in the user action, n represents the number of frames of the standard action video, and m represents the number of frames of the user action.

[0082] Use the scoring function to convert the DTW distance into the overall similarity score of the moves dwt , the overall similarity score dwt The calculation formula is as follows:

[0083]

[0084] Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great When , the action is very good. Table 3 shows the total score results of the four output videos.

[0085] This embodiment calculates the rhythm score of the user's action by scoring a single frame. r , expressed as:

[0086]

[0087] In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

[0088] Table 3 Total score results of four videos

[0089]

[0090]

[0091] S3) construct and train a Ba Duan Jin action evaluation model based on a long short-term memory network (LSTM), perform uniform frame extraction on the feature sequence of the pivot action output by the Ba Duan Jin action recognition model, and then input it into the trained Ba Duan Jin action evaluation model to obtain the classification label of the user's pivot action, and reflect the user's errors in key actions according to the classification label;

[0092] In this embodiment, video category encoding is implemented through One-hot encoding. The encoding is not only the label of the training data, but also the encoding method is binary encoding. In the four error points of the "drawing the bow" in the "drawing the bow like shooting a eagle", the encoding is expressed as a 4-bit binary number, 1 means that the error point has an error, and 0 means that the error point has not an error. In the annotation of the data, the annotator arranges it from left to right in order to form a binary number, and converts it into a decimal number as a label. Correspondingly, the output classification result of the Eight-Section Brocade action evaluation model based on the long short-term memory network LSTM is a tensor of (1,4), in which each element represents the confidence of classification as 1, and the value is between 0 and 1. When calculating the loss function, the label is converted into a one-hot encoding, and the cross entropy is calculated with the output tensor, so as to realize back propagation and training.

[0093] In this embodiment, the uniform frame extraction process is: for a given target sequence length and original feature sequence length, the frame extraction interval is calculated, and then the original feature sequence is uniformly extracted according to the frame extraction interval, specifically:

[0094] like Figure 4 As shown in the figure, for the original feature sequence length seq_len, the given target sequence length is target_len; then the frame extraction interval interval = seq_len / target_len; for the frame extraction interval interval, starting from the 0th frame of the original feature sequence, extract a frame every interval frame until the target sequence length target_len is reached, and finally process the remaining frames. If the original feature sequence length seq_len cannot be divided by the target sequence length target_len, the last frame of the original feature sequence is ignored. For example, for the "bow drawing" action, uniform frame extraction is used to reduce the number of frames to 10 frames.

[0095] Then, the single-frame features after uniform frame extraction are input into the Eight-Section Brocade action evaluation model based on the long short-term memory network (LSTM) in chronological order. The hidden state is learned and updated through the internal gating mechanism to capture the temporal dependency of the Eight-Section Brocade actions and output the classification of user actions.

[0096] In this embodiment, the parameters of the Ba Duan Jin action evaluation model based on the long short-term memory network LSTM are shown in Table 4.

[0097] Table 4 Parameters of the Ba Duan Jin action evaluation model based on the long short-term memory network LSTM

[0098]

[0099] During the training process, the output form of the Eight-Section Brocade action evaluation model is a tensor with a shape of (1,16) after normalization by the Softmax function, and the cross entropy calculation is performed with the label encoding after one-hot encoding as the loss function. Through the back propagation algorithm, the weight and bias parameters of the LSTM network are continuously adjusted to minimize the loss function so that the model can better fit the training data. Combined with the Adam optimizer to accelerate the model training process and improve the convergence speed. Finally, the output result of the Eight-Section Brocade action evaluation model is the classification encoding of the video, and the encoding corresponds to a text content, and the corresponding text content data is fed back to the practitioner. This embodiment uses accuracy, sensitivity, recall rate, and F1 value to evaluate the model performance, and the performance index test results of the Eight-Section Brocade action evaluation model are shown in Table 5.

[0100] Table 5 Performance index test results of the Ba Duan Jin action evaluation model

[0101]

[0102]

[0103] S4), deploy the trained Ba Duan Jin action recognition model and Ba Duan Jin action evaluation model to the cloud server to recognize and evaluate the user's Ba Duan Jin action. The final output of this embodiment is as follows Figure 5 shown.

[0104] Example 2

[0105] like Figure 6 As shown, this embodiment provides a Ba Duan Jin action evaluation system based on dynamic programming and deep learning, including:

[0106] A data collection module is used to collect the user's Ba Duan Jin action video data;

[0107] The human skeleton point recognition module uses the Pose method based on MediaPipe to recognize the three-dimensional spatial coordinate data of 33 human skeleton points from the Ba Duan Jin action video data, and builds a human motion model based on the 33 human skeleton points;

[0108] The dynamic time warping module is used to calculate the matching path, DTW distance and single-frame score between the time series of the user's actions and the time series of the standard action video, and obtain the feature sequence of the user's pivot action based on the matching path, and obtain the overall similarity score of the user's moves based on the DTW distance, and obtain the score result of the user's action rhythm through the single-frame score;

[0109] The action evaluation module is used to capture the time dependency of the Ba Duan Jin action from the single-frame features after uniform frame extraction according to the Ba Duan Jin action evaluation model based on the long short-term memory network (LSTM), and obtain the classification result of the user action.

[0110] As a preferred embodiment of this invention, the single frame score is calculated as follows:

[0111] The cosine distance is calculated for each frame of the user's action and the standard frame that matches it. The scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain the single-frame score of each frame of the user's action. d ,Right now:

[0112]

[0113] Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action.

[0114] As a preferred embodiment of this invention, the DTW distance is converted into the overall similarity score of the moves by using a scoring function. dwt , the overall similarity score dwt The calculation formula is as follows:

[0115]

[0116] Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great The action is excellent.

[0117] As a preferred embodiment of this invention, the rhythm score result of the user action is calculated by scoring a single frame. r , expressed as:

[0118]

[0119] In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

[0120] As a preferred embodiment of the present invention, the uniform frame extraction process is as follows: for a given target sequence length and original feature sequence length, a frame extraction interval is calculated, and then the original feature sequence is uniformly extracted according to the frame extraction interval.

[0121] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. The method for evaluating the movements of Ba Duan Jin based on dynamic programming and deep learning is characterized by: The following steps are involved: S1), collecting Ba Duan Jin action video data, and dividing it into a training set and a validation set after annotating it; S2) Build and train the Ba Duan Jin action recognition model based on MediaPipe and the dynamic time warping algorithm DTW, obtain the matching path, DTW distance and single frame score through the trained Ba Duan Jin action recognition model, and obtain the feature sequence of the pivot action according to the matching path; S3) construct and train a Ba Duan Jin action evaluation model based on a long short-term memory network (LSTM), perform uniform frame extraction on the feature sequence of the pivot action output by the Ba Duan Jin action recognition model, and then input it into the trained Ba Duan Jin action evaluation model to obtain the classification label of the user's pivot action, and reflect the user's errors in key actions according to the classification label; S4), deploying the trained Ba Duan Jin motion recognition model and Ba Duan Jin motion evaluation model to the cloud server to recognize and evaluate the user's Ba Duan Jin motion.

2. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 1, characterized in that: In step S2), the three-dimensional spatial coordinate data of the human skeleton points are identified from the Ba Duan Jin action video data through the Pose method based on MediaPipe, and a human action model is constructed according to the human skeleton points.

3. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 2 is characterized in that: In step S2), the angle index of each joint of the human body and the time series of the angle index are obtained according to the human body motion model.

4. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 3 is characterized in that: In step S2), the matching path, DTW distance and single-frame score between the time series of the user action and the time series of the standard action video are calculated through the dynamic time warping algorithm DTW. The matching path is used to identify the user action and return the feature sequence of the pivot action. The DTW distance is output as the overall similarity score of the moves through the scoring function, and the scoring result of the user action rhythm is obtained through the single-frame scoring.

5. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 4 is characterized in that: In step S2), the cosine distance is calculated for each frame of the user's action and the standard frame that matches it, and a scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain a single frame score for each frame of the user's action. d ,Right now: Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action.

6. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 4 is characterized in that: In step S2), the DTW distance is converted into the overall similarity score of the moves using the scoring function dwt , the overall similarity score dwt The calculation formula is as follows: Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great The action is excellent.

7. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 6 is characterized in that: In step S2), the rhythm score of the user's action is calculated by scoring the single frame. r , expressed as: In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

8. The method for evaluating the movement of Ba Duan Jin based on dynamic programming and deep learning according to claim 1, characterized in that: In step S3), the uniform frame extraction process is as follows: for a given target sequence length and an original feature sequence length, a frame extraction interval is calculated, and then the original feature sequence is uniformly extracted according to the frame extraction interval; The single-frame features after uniform frame extraction are input into the Ba Duan Jin action evaluation model based on the long short-term memory network (LSTM) in chronological order. The hidden state is learned and updated through the internal gating mechanism to capture the temporal dependency of the Ba Duan Jin actions and output the classification of user actions.

9. The Ba Duan Jin movement evaluation system based on dynamic programming and deep learning is characterized by: The system evaluates the user's Ba Duan Jin movements using the method described in any one of claims 1 to 8, and the system comprises: A data collection module is used to collect the user's Ba Duan Jin action video data; The human skeleton point recognition module uses the Pose method based on MediaPipe to recognize the three-dimensional spatial coordinate data of the human skeleton points from the Ba Duan Jin action video data, and builds a human action model based on the human skeleton points; The dynamic time warping module is used to calculate the matching path, DTW distance and single-frame score between the time series of the user's actions and the time series of the standard action video, and obtain the feature sequence of the user's pivot action based on the matching path, and obtain the overall similarity score of the user's moves based on the DTW distance, and obtain the score result of the user's action rhythm through the single-frame score; The action evaluation module is used to capture the time dependency of the Ba Duan Jin action from the single-frame features after uniform frame extraction according to the Ba Duan Jin action evaluation model based on the long short-term memory network (LSTM), and obtain the classification result of the user action.

10. The Ba Duan Jin movement evaluation system according to claim 9, characterized in that: The calculation method of the single frame score is: The cosine distance is calculated for each frame of the user's action and the standard frame that matches it. The scoring function is set according to the property that the maximum value of the cosine distance is 2 to obtain the single-frame score of each frame of the user's action. d ,Right now: Where sim(A,B) represents the cosine similarity between the angle index A of the standard action and the angle index B of the user action; Use the scoring function to convert the DTW distance into the overall similarity score of the moves dwt , the overall similarity score dwt The calculation formula is as follows: Where d represents the DTW distance between the current user action time series and the standard action time series; d max Indicates the cosine distance when the similarity is the lowest; d great is approximated by the DTW distance of the best test action, when d <d great When , the action is very excellent; Calculate the rhythm score of the user's action by scoring the single frame r , expressed as: In the formula, S represents the user's single frame score d The frequency greater than 60 points; D The total number of frames of the user's pivot action.

Images