Eight-section brocade rehabilitation training action recognition method

Abstract

The invention relates to the technical field of image recognition, in particular to an eight-segment brocade rehabilitation training action recognition method, which comprises the following steps of: based on an input eight-segment brocade training action image sequence, extracting human skeleton key point coordinates in each frame of image, and establishing multi-dimensional limb movement track data; according to the invention, through frame-by-frame extraction of human skeleton key points in an image sequence, multi-dimensional limb movement track data with space and time characteristics is established, and a basic structure from single-frame static analysis to cross-frame dynamic modeling is formed. By means of mirror image mapping of a left key point on the basis of a body midline, item-by-item vector operation is carried out on a mapping point and a right actual coordinate, an instantaneous deviation vector field used for measuring symmetry is constructed, and real-time quantitative evaluation of movement consistency of the left limb and the right limb is achieved. And in combination with periodic integral operation, an accumulated symmetry deviation value covering a complete action process is generated, and full-period feedback on the attitude stability and the action specification degree is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to a Baduanjin rehabilitation training action recognition method. BACKGROUND

[0002] The technical field of image recognition focuses on enabling computers to automatically understand and analyze visual information obtained from cameras, videos or other sensors, and the core task is to convert pixel-level data in images or image sequences into semantic information with practical significance.

[0003] The prior art focuses on single-frame posture analysis or key point detection in the image recognition process, and the processing logic mainly relies on the independent recognition of each frame of image, and the recognition result is matched with the preset action template in terms of angle, position or category. This frame-level static recognition method is difficult to capture the trend deviation in the complete action cycle, and cannot perceive the accumulated error and symmetry destruction of the motion quality. At the same time, the recognition result in the prior art is limited to shape matching, and it is difficult to quantify the action control degree and the exerciser's input. The coordination in the left and right open bow type action is difficult to model by the frame-level static key point judgment method. Therefore, improvement is needed. SUMMARY

[0004] The purpose of the present application is to solve the shortcomings in the prior art and propose a Baduanjin rehabilitation training action recognition method.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme, a Baduanjin rehabilitation training action recognition method, comprising the following steps:

[0006] Based on the input Baduanjin training action image sequence, the human body skeleton key point coordinates are extracted in each frame of image, and multi-dimensional limb motion trajectory data is established;

[0007] Based on the multi-dimensional limb motion trajectory data, for each frame of coordinate point in the left limb key point sequence, the mirror image mapping point coordinate set is calculated, and the mirror image mapping point coordinate set and the corresponding right limb key point coordinates in the multi-dimensional limb motion trajectory data are operated to obtain an instantaneous symmetry deviation vector field;

[0008] The modulus of all deviation vectors in the instantaneous symmetry deviation vector field is operated within the action cycle to obtain a cumulative symmetry deviation value, and the task group and non-task group key points are divided for the left and right open bow action, and the limb motion energy distribution feature is calculated and obtained;

[0009] The accumulated symmetry deviation value is compared with a preset Baguazhang standard movement threshold value to determine the standard degree of movement, and the ratio of task group energy to non-task group energy in the limb movement energy distribution feature is calculated to establish a Baguazhang rehabilitation training movement concentration quantitative index.

[0010] Preferably, the step of acquiring the multi-dimensional limb movement trajectory data is:

[0011] Based on the input Baguazhang training movement image sequence, three-dimensional space coordinate values of all labeled nodes in the whole body skeletal structure are analyzed and extracted frame by frame, arranged and summarized in frame sequence to generate a skeletal key point coordinate set;

[0012] Based on the skeletal key point coordinate set, a straight line fitting operation is performed on the nose tip coordinate, neck coordinate and trunk midpoint coordinate, and three-dimensional trajectories of left shoulder coordinate, left elbow coordinate, left wrist coordinate, right shoulder coordinate, right elbow coordinate and right wrist coordinate are recorded synchronously in a continuous time window to generate a limb key point time sequence coordinate matrix;

[0013] Based on the limb key point time sequence coordinate matrix, coordinate splicing and dimension expansion are performed on all key point three-dimensional trajectories according to time sequence, and central straight line direction vector information is added to form multi-dimensional limb movement trajectory data.

[0014] Preferably, the step of acquiring the mirror mapping point coordinate set is:

[0015] Based on the multi-dimensional limb movement trajectory data, left shoulder coordinate, left elbow coordinate and left wrist coordinate are read frame by frame, and orthogonal projection operation is performed on the central straight line direction vector in the same frame multi-dimensional limb movement trajectory data and the position of the symmetric point is translated reversely along the central straight line to form a mirror mapping point coordinate set.

[0016] Preferably, the step of acquiring the instantaneous symmetry deviation vector field is:

[0017] Based on the mirror mapping point coordinate set, right shoulder coordinate, right elbow coordinate and right wrist coordinate are matched frame by frame, three-dimensional coordinate components are subtracted item by item and difference vectors are summarized to generate a left-right symmetry difference vector set;

[0018] Based on the left-right symmetry difference vector set, all difference vectors are arranged in time sequence and corresponding key point labels are labeled to form an instantaneous symmetry deviation vector field.

[0019] Preferably, the step of acquiring the accumulated symmetry deviation value is:

[0020] Based on the instantaneous symmetry deviation vector field, the deviation vectors corresponding to the right shoulder, right elbow and right wrist key points in all frames in the action period are extracted respectively, the three-dimensional difference vectors of the three key points in each frame are recorded, combined in frame order and key point type order, and the deviation vector time sequence set of the three types of key points is obtained;

[0021] According to the deviation vector time sequence set of the three types of key points, the square of the length of each frame deviation vector is calculated one by one according to the key point type, and the square of the length of the difference vector of the current frame and the last frame deviation vector is constructed, while the key point type index and frame index are marked, and the cumulative symmetry deviation value is calculated.

[0022] Preferably, the acquisition step of the limb movement energy distribution feature is:

[0023] Based on the left and right open bow action grouping rule, the three-dimensional coordinates of the right shoulder, right elbow, right wrist, left shoulder, left elbow and left wrist key points in the action period are read, the velocity vector and velocity change vector between each frame adjacent coordinates of each key point are calculated in turn, and the virtual mass value of the corresponding key point is recorded;

[0024] According to the velocity vector and the velocity change vector, the control kinetic energy value is calculated combined with the virtual mass value;

[0025] Based on the control kinetic energy value, the control kinetic energy values of the task group and the non-task group key points in the whole action period are accumulated respectively, and the total amount of control kinetic energy of the two groups is aggregated to form the limb movement energy distribution feature.

[0026] Preferably, the acquisition step of the Baduanjin rehabilitation training action concentration quantitative index is:

[0027] Based on the corresponding relationship between the cumulative symmetry deviation value and the Baduanjin standard action threshold value, the standard action threshold value is set as a static reference limit, the cumulative symmetry deviation values of the current action period are traversed in turn, and it is judged whether each item exceeds the standard action threshold value range to form a judgment result set of whether it meets the standard action condition or not;

[0028] According to the judgment result set, the limb movement energy distribution feature corresponding to the current action period is extracted, the total values of the task group energy and the non-task group energy are read respectively, the quantity ratio of the task group energy value to the non-task group energy value is calculated, and the energy concentration ratio index set is formed.

[0029] Preferably, the acquisition step of the Baduanjin rehabilitation training action concentration quantitative index further includes: based on the energy concentration ratio index set, combining each action standard judgment label in the judgment result set, using label screening to extract the corresponding energy concentration ratio item, and establishing the Baduanjin rehabilitation training action concentration quantitative index.

[0030] Compared with the prior art, the application has the advantages and positive effects that:

[0031] The application establishes multi-dimensional limb movement trajectory data with spatial and temporal characteristics by frame-by-frame extraction of human body skeleton key points in the image sequence, forming a basic structure from single-frame static analysis to cross-frame dynamic modeling. By means of mirror mapping of the left key points based on the body midline, the mapped points are subjected to item-by-item vector operation with the right actual coordinates to construct an instantaneous deviation vector field for measuring symmetry, realizing real-time quantitative evaluation of left and right limb movement consistency. Combined with periodic integration operation, the cumulative symmetry deviation value covering the complete action process is generated to provide full-cycle feedback on posture stability and action specification. On the basis of symmetry, a distinction strategy of task group and non-task group is introduced, the product of the speed and virtual mass of each group of key points is calculated to construct fine-grained energy distribution features, and the analysis dimension of movement performance difference is introduced at the energy level. The symmetry evaluation result is compared with the standard action threshold to output a structured judgment label, and the degree of concentration is further measured by the energy ratio of the task group and the non-task group, so that the recognition result not only stays at the static classification level, but also has dynamic process understanding and training control ability. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The application is a step schematic diagram. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the application clearer, further detailed description will be made below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0034] Please refer to Figure 1 The application provides a technical scheme, a Ba Duan Jin rehabilitation training action recognition method, comprising the following steps:

[0035] Based on the input Ba Duan Jin training action image sequence, human body skeleton key point coordinates are extracted in each frame of image, and multi-dimensional limb movement trajectory data is established;

[0036] Based on the multi-dimensional limb movement trajectory data, for each frame of coordinate point in the left limb key point sequence, a mirror mapping point coordinate set is calculated, and then the mirror mapping point coordinate set is operated with the corresponding key point coordinates of the right limb in the multi-dimensional limb movement trajectory data to obtain an instantaneous symmetry deviation vector field;

[0037] The lengths of all deviation vectors in the instantaneous symmetry deviation vector field are operated in the action period to obtain a cumulative symmetry deviation value, and the task group and non-task group key points are divided for the left and right bow opening actions to calculate and obtain the limb movement energy distribution features;

[0038] The accumulated symmetry deviation value is compared with a preset Baguazhang standard action threshold value, the standard degree of the action is determined, the ratio of the task group energy to the non-task group energy in the limb movement energy distribution feature is calculated, and a Baguazhang rehabilitation training action concentration quantitative index is established.

[0039] The acquisition step of the multi-dimensional limb movement trajectory data is:

[0040] Based on the input Baguazhang training action image sequence, three-dimensional space coordinate values of all labeled nodes in the whole body skeletal structure are analyzed and extracted frame by frame, are arranged and summarized in sequence, and a skeletal key point coordinate set is generated;

[0041] Based on the skeletal key point coordinate set, straight line fitting operation is performed on the nose tip coordinate, neck coordinate and trunk midpoint coordinate, three-dimensional trajectories of the left shoulder coordinate, left elbow coordinate, left wrist coordinate, right shoulder coordinate, right elbow coordinate and right wrist coordinate are synchronously recorded in a continuous time window, and a limb key point time sequence coordinate matrix is generated.

[0042] Based on the limb key point time sequence coordinate matrix, coordinate splicing and dimension expansion are performed on all key point three-dimensional trajectories according to the time sequence, and central straight line direction vector information is added, and multi-dimensional limb movement trajectory data is formed.

[0043] Specifically, based on the input Ba Duan Jin training action image sequence, first, a deep convolutional neural network model based on an encoder-decoder structure is used to process each frame of image, the model is specifically composed of a ResNet-50 backbone network for feature extraction as an encoder, and a decoder composed of three deconvolution layers, the model is pre-trained, the training data set is collected by a multi-camera motion capture system, more than 500 standard Ba Duan Jin action sequences of different body types of exercisers are collected, and three-dimensional space coordinate true value labels containing 25 key points of the whole body are generated, during training, the image frame is input into the model, the model outputs two groups of results, the first group is the 2D heat map for each key point, and the second group is the depth value of each key point, the loss function is defined as the weighted sum of the mean square error loss of the heat map and the L1 loss of the depth value, the weights are set to 0.6 and 0.4 respectively according to experience, 50 cycles of iterative training are carried out through the Adam optimizer until the loss function converges, in the analysis stage, for each frame of image in the video sequence, the trained model is used to infer the 2D coordinates and depth values of all 25 key points, wherein the 2D coordinates are determined by finding the position of the maximum response value in the corresponding heat map, then the 2D pixel coordinates of each key point are combined with the corresponding depth value to convert the three-dimensional space coordinate values in the camera coordinate system, repeat the process for each frame, organize all the key point three-dimensional coordinate values extracted from each frame in the order of the frames, and finally form a time sequence set, wherein each time point corresponds to a complete set of whole body key point three-dimensional coordinates, and a key point coordinate set is generated.

[0044] Based on the set of skeletal key point coordinates, for each frame in the sequence, first calculate the torso midpoint coordinate, which is obtained by extracting the left hip joint coordinate and the right hip joint coordinate of the current frame, and then calculating the arithmetic mean of the corresponding components of the two three-dimensional coordinate vectors, then select the nose tip coordinate, neck coordinate and the just calculated torso midpoint coordinate of the current frame, and use the least squares method to perform three-dimensional space straight line fitting, this method aims to find a straight line, so that the sum of the squared perpendicular distances of the three points to the straight line is minimized, thereby calculating the central straight line representing the body center axis, then set a fixed length of 15 frames of sliding time window, the window moves on the whole action sequence in a step of 1 frame, in the 15 frames covered by each time window, synchronously extract and record the three-dimensional trajectory data of the six upper limb key points of left shoulder coordinate, left elbow coordinate, left wrist coordinate, right shoulder coordinate, right elbow coordinate and right wrist coordinate, this process will continuously capture the coordinates of the six key points of 15 frames, finally, after processing all the frames of the whole action sequence according to the above method, organize all the recorded upper limb key point three-dimensional trajectory data in time sequence into a multi-dimensional array, the dimension of the array is the total number of frames multiplied by the number of key points (6) multiplied by the coordinate dimension (3 dimensions), generate the limb key point time sequence coordinate matrix.

[0045] Based on the limb key point time sequence coordinate matrix, for each frame in the action sequence, first perform coordinate splicing operation, specifically, read the three-dimensional coordinates of the six key points (left shoulder, left elbow, left wrist, right shoulder, right elbow, right wrist) corresponding to the current frame from the limb key point time sequence coordinate matrix, and according to the preset fixed order, such as "left shoulder X, left shoulder Y, left shoulder Z, left elbow X, left elbow Y, left elbow Z, …, right wrist X, right wrist Y, right wrist Z", flatten and splice the 18 coordinate values into an 18-dimensional feature vector, then, dimensionally extend the 18-dimensional vector, retrieve the central straight line direction vector calculated for the same frame in the previous step, which is a three-dimensional unit vector representing the main direction of the human body trunk in the frame image, directly append the components of this three-dimensional direction vector (for example, the X, Y, Z components of the direction vector) to the end of the 18-dimensional feature vector, thereby forming a 21-dimensional extended feature vector, which not only contains the accurate spatial position information of the upper limb key points of the current frame, but also integrates the overall posture orientation information of the body, repeat the splicing and extension operation for each frame in the whole eight-section exercise training action image sequence, stack the 21-dimensional extended feature vectors generated by all frames in time sequence, finally form a two-dimensional matrix, the number of rows of the matrix is equal to the total number of frames of the action sequence, and the number of columns is 21, forming a multi-dimensional limb motion trajectory data.

[0046] The mirror mapping point coordinate set acquisition step is:

[0047] Based on the multi-dimensional limb motion trajectory data, the left shoulder coordinates, the left elbow coordinates and the left wrist coordinates are read frame by frame, the central straight line direction vector in the same frame of multi-dimensional limb motion trajectory data is called to perform orthogonal projection operation and reverse translation of the symmetric point position along the central straight line, and the mirror image mapping point coordinate set is formed.

[0048] Specifically, based on the multi-dimensional limb motion trajectory data, the system iteratively processes each frame of image in the action sequence. In the current frame being processed, first, the three-dimensional coordinates of the left shoulder, the three-dimensional coordinates of the left elbow and the three-dimensional coordinates of the left wrist are extracted from the multi-dimensional limb motion trajectory data. At the same time, the human body center straight line direction vector corresponding to the frame is called. This vector, as a three-dimensional unit vector, defines the symmetry axis of the human body trunk. For each key point on the left side, such as the left shoulder, the mirror image mapping point thereof relative to the central straight line is calculated. The calculation process first performs orthogonal projection operation. Specifically, a positioning vector is constructed with a reference point (such as the neck coordinates) on the central line as the starting point and the left shoulder coordinates as the ending point. Then, the positioning vector is projected onto the central straight line direction vector. The projection calculation is achieved by calculating the dot product of the two vectors and multiplying the dot product result by the central straight line direction vector itself, thereby obtaining a projection vector. The ending point of the projection vector is taken as the orthogonal projection point of the left shoulder on the central straight line. Next, a vector from the original left shoulder coordinate point to the orthogonal projection point is calculated, and twice the vector is added to the original left shoulder coordinate point, thereby obtaining the mirror image mapping point coordinates of the left shoulder. This process is geometrically equivalent to flipping the left shoulder coordinate point with the central straight line as the symmetry axis. The system also repeatedly performs the same orthogonal projection and reverse translation operation on the left elbow coordinates and the left wrist coordinates in the same frame to calculate their respective mirror image mapping point coordinates. After completing the single frame processing, the three mirror image mapping point coordinates (corresponding to the ideal right shoulder, right elbow and right wrist positions respectively) are collected. The above calculation is performed for each frame of the entire action sequence to form the mirror image mapping point coordinate set.

[0049] The acquisition step of the instantaneous symmetry deviation vector field is:

[0050] Based on the mirror image mapping point coordinate set, the right shoulder coordinates, the right elbow coordinates and the right wrist coordinates are matched frame by frame, the three-dimensional coordinate components are subtracted item by item, and the difference vector set is collected to generate the left-right symmetric difference vector set.

[0051] Based on the left-right symmetric difference vector set, all difference vectors are arranged in time sequence and labeled with corresponding key point labels to form the instantaneous symmetry deviation vector field.

[0052] Specifically, based on the mirror mapping point coordinate set, the system continues to process the entire action sequence frame by frame. When processing each frame, first, the data is accurately matched according to the frame index, that is, the mirror mapping point coordinate set of the current frame is taken out, which contains three three-dimensional coordinates, respectively representing the positions of the theoretically symmetrical right shoulder, right elbow and right wrist. At the same time, from the original multi-dimensional limb motion trajectory data, the right shoulder coordinates, right elbow coordinates and right wrist coordinates actually captured in the same frame are extracted. Then, the system performs key point pairing and three-dimensional coordinate component subtraction operation. Specifically, the actual right shoulder coordinates are paired with the mirror mapping right shoulder coordinates, and then the X, Y and Z components of the two coordinates are subtracted respectively to obtain a three-dimensional difference vector containing three difference components (Δx, Δy, Δz). This vector represents the direction and distance of the actual right shoulder position relative to its ideal symmetrical position in space. Similarly, the system performs the same pairing and subtraction operation on the right elbow and right wrist to calculate the right elbow difference vector and the right wrist difference vector, respectively. After completing the calculation of the three key points in a single frame, the three newly generated three-dimensional difference vectors are summarized to form a set containing all the symmetry deviation information of the frame. This process is repeated on each frame of the entire action sequence to generate a left-right symmetry difference vector set.

[0053] Based on the left-right symmetry difference vector set, the system integrates and organizes the difference vectors calculated for each frame in time sequence to construct a structured data field. Specifically, the system traverses the entire action sequence from the first frame to the last frame. For each frame in the sequence (for example, the kth frame), the corresponding three difference vectors, i.e. the right shoulder difference vector, the right elbow difference vector and the right wrist difference vector, are retrieved from the left-right symmetry difference vector set. In order to clearly distinguish and reference these vectors in subsequent calculations, the system attaches a clear label to each difference vector, which identifies the key point type it belongs to. For example, text labels such as "RightShoulder", "RightElbow", "RightWrist" or numerical indices such as 1, 2, 3 can be used to represent right shoulder, right elbow and right wrist, respectively. In this way, the data of each frame is represented by a structure containing three labeled difference vectors. By arranging all the data structures of the frames in the time sequence of the original video sequence, a complete time sequence data structure is finally formed, which defines the symmetry deviation of the upper limb key points at any time point (frame), thus forming an instantaneous symmetry deviation vector field.

[0054] The cumulative symmetry deviation value acquisition step is:

[0055] Based on the instantaneous symmetry deviation vector field, the deviation vectors corresponding to the right shoulder, right elbow and right wrist key points in all frames in the action cycle are extracted respectively, the three-dimensional difference vectors of three key points in each frame are recorded, combined according to the frame order and key point type order, and three sets of key point deviation vector time sequence sets are obtained;

[0056] According to the three sets of key point deviation vector time sequence sets, the modulus square of each frame deviation vector is calculated according to the key point type, and the difference vector modulus square of the current frame and the last frame deviation vector is constructed, while the key point type index and frame index are marked, the cumulative symmetry deviation value is calculated, and the calculation formula is:

[0057]

[0058] Among them, D total represents the cumulative symmetry deviation value, K is the total number of frames in the action cycle, j∈{1,2,3} respectively represents the right shoulder, right elbow and right wrist, w j is the action importance weight of the jth key point, is the instantaneous symmetry deviation vector of the jth key point in the kth frame, is the change of the jth key point deviation vector between adjacent frames, and γ is the deviation stability adjustment factor.

[0059] Specifically, based on the instantaneous symmetry deviation vector field, first, the complete single action cycle needs to be defined. Taking the "Double Hands Supporting the Sky to Regulate Sanjiao" in the Eight-Section Broadsword as an example, the action cycle can be determined by analyzing the coordinate trajectory of the double-wrist key points in the vertical direction. The system will traverse the Y-axis coordinates of the left and right wrists of all frames in the entire sequence, find the frame with the local minimum coordinate value as the starting point of the action, and find the next local minimum as the end point of the action, thereby determining the starting frame number and the ending frame number of the action cycle. Within all the frames contained in the action cycle, the system starts to extract the deviation vector frame by frame. For each frame in the cycle, the system accesses the corresponding data structure of the frame in the instantaneous symmetry deviation vector field. This data structure stores the deviation vectors with key point labels. The system accurately retrieves the three-dimensional difference vectors corresponding to the right shoulder, right elbow, and right wrist according to the pre-set labels, i.e., "RightShoulder", "RightElbow", and "RightWrist". After the extraction operation is completed, the system stores these vectors in categories. Specifically, three independent time sequence lists are created to store the deviation vectors of the right shoulder, right elbow, and right wrist, respectively. All extracted right shoulder deviation vectors are added to the first list in the order of their frame sequence in the action cycle. Similarly, the right elbow and right wrist deviation vectors are added to the second and third lists in the order of their frame sequence, respectively. In this way, the original, mixed deviation vector field is reorganized into three independent, time-ordered vector sequences, obtaining the deviation vector time sequence set of the three types of key points.

[0060] Formula: The benefit of the formula is that it comprehensively evaluates the two core dimensions of action symmetry, i.e., the magnitude of spatial deviation and the stability of temporal deviation, through a unified framework. The first term directly quantifies the degree of limb position deviation from the ideal symmetric position at a certain moment, while the second term introduces the consideration of the deviation change rate, punishing those actions with small but drastic changes in deviation, which is particularly important for rehabilitation training, as smooth and controlled actions are the key goal in the rehabilitation process. The introduction of the deviation stability adjustment factor γ allows for flexible adjustment of the sensitivity to action stability according to the needs of different rehabilitation stages. In addition, the setting of the action importance weight w j allows the evaluation model to pay more attention to the accuracy of key joints according to the technical points of different Eight-Section Broadsword movements, thereby achieving more refined and personalized action evaluation;

[0061] K is the total number of frames in the action cycle, which is obtained by analyzing the frame rate of the video sequence and the start and end time points of the specific action. First, read the frame rate from the metadata of the video file, for example, 30 frames per second. Second, identify the start and end of the specific Baguazhang action by algorithm, for example, for "Double-Handed Archery", the system monitors the vertical coordinates of the double-wrist key points. When the double-wrist starts to lift from the lowest point, it is recorded as the starting frame. When the upper arch is completed and falls back to the lowest point, it is recorded as the ending frame. The difference between the two frame numbers is the total frame number K. For example, one action cycle starts from frame 150 and ends at frame 300, so the total frame number K = 300-150+1 = 151. In this example, set the total duration of one action cycle to 5 seconds, and the frame rate to 30 frames per second, then the total frame number K = 5x30 = 150.

[0062] is the instantaneous symmetry deviation vector of the kth frame and the jth key point. This vector is directly obtained from the previously generated deviation vector time series set of the three types of key points. This set contains the three-dimensional deviation vector (unit: meters) of each key point in each frame during the action cycle. For example, when the deviation vector of the right elbow (index j = 2) at frame 10 is needed, the system directly indexes the 10th element from the right elbow's deviation vector time series set. For example, a specific deviation vector can be

[0063] w j is the action importance weight of the jth key point. This weight is based on the biomechanical characteristics of specific Baguazhang movements and expert scores from TCM doctors and rehabilitation therapists. It is quantitatively calculated using the Analytic Hierarchy Process (AHP). Taking "Left and Right Archery" as an example, three experts (one TCM doctor and two rehabilitation therapists) compare the importance of right shoulder (j = 1), right elbow (j = 2), and right wrist (j = 3) in the action. A judgment matrix is constructed. Experts unanimously believe that in this action, the right wrist responsible for drawing the bow and the right elbow stabilizing the bow are equally important, and both are more important than the right shoulder as a support point. The importance is assigned a value of 3. The right elbow and right wrist are equally important, assigned a value of 1. Thus, the judgment matrix is constructed as By calculating the normalized eigenvector corresponding to the largest eigenvalue of this matrix, the weight vector is obtained. The calculation shows that the weights are w1 = 0.14, w2 = 0.43, and w3 = 0.43. This set of weight values reflects that in this action, the symmetry and stability of the elbow and wrist are more critical than the shoulder.

[0064] γ is a bias stability adjustment factor, the setting of which aims to balance the contribution of spatial bias and temporal stability in the total bias, the value of which is determined by comparative analysis of the motion data of the standard group (10 experienced Ba Duan Jin trainers) and the rehabilitation group (10 initial rehabilitation patients), the same action is completed by the two groups and Statistics are performed, and the mean ratio of the two is calculated, the mean ratio of the standard group is about 0.8, and the mean ratio of the rehabilitation group is about 1.5 due to the instability of the action, in order to effectively distinguish the two groups, a grid search is performed in the interval [0.5, 2.0] with a step of 0.1 to find the value of γ that maximizes the difference between the final D total scores of the two groups, and tests show that when γ is 1.2, the cumulative symmetry bias value of the two groups has the highest degree of distinction, so γ is set to 1.2.

[0065] Calculation process:

[0066] Here, the cumulative bias of the right elbow (j = 2) in the next three frames (k = 1, 2, 3, total frame number K = 150) is calculated, where k = 0 is the frame before the start frame, used to calculate the change, and the previously determined parameters are brought in: w2 = 0.43 and γ = 1.2.

[0067] Get the bias vector sequence of the right elbow:

[0068]

[0069] Calculate the root number value of the 1st frame (k = 1):

[0070]

[0071] Calculate the root number value of the 2nd frame (k = 2):

[0072]

[0073] Calculate the root number value of the 3rd frame (k = 3):

[0074]

[0075] Add up the calculation results of all frames in the entire action period (K = 150), here is only an example, for example, the cumulative sum of 150 frames is 3.85, then the total bias of the right elbow part is: 0.43 x 3.85 = 1.6555.

[0076] Similarly, the total bias of the right shoulder (j = 1) and the right wrist (j = 3) in 150 frames is calculated, for example, the weighted results are 0.8520 and 1.7500 respectively, then the final cumulative symmetry bias value is:

[0077] D total= 1.6555 + 0.8520 + 1.7500 = 4.2575.

[0078] The result shows that the cumulative symmetry deviation value of the user in this "left and right open bow like shooting eagle" action is 4.2575, reflecting the degree of spatial symmetry deviation and time stability of the action in the whole cycle. The higher the value, the worse the symmetry or the lower the stability of the action. The value will be a key basis for subsequent evaluation of whether the action is standard. For example, if the preset standard action threshold is 3.0, the result of 4.2575 indicates that the action does not meet the standard requirements.

[0079] The acquisition step of the limb movement energy distribution feature is:

[0080] Based on the left and right open bow action grouping rule, the three-dimensional coordinates of the right shoulder, right elbow, right wrist, left shoulder, left elbow and left wrist key points in the action cycle are read, the speed vector and speed change vector between each key point and each frame adjacent coordinates are calculated in turn, and the virtual mass value of the corresponding key point is recorded;

[0081] According to the speed vector and the speed change vector, the control kinetic energy value is calculated in combination with the virtual mass value, and the calculation formula is:

[0082]

[0083] Among them, E' b,h represents the control kinetic energy value of the b-th frame and the h-th key point, m h is the virtual mass of the h-th key point, is the speed vector of the b-th frame and the h-th key point, is the difference value of the continuous frame speed vector, Δt is the frame time interval, and λ is the smoothness sensitivity coefficient, which is used to reflect the control requirement of acceleration;

[0084] Based on the control kinetic energy value, the control kinetic energy values of the task group and the non-task group key points in the whole action cycle are accumulated respectively, and the control kinetic energy total amount of the two groups is aggregated to construct the limb movement energy distribution feature.

[0085] Specifically, based on the left and right archery action grouping rule, the rule clearly defines that in the "left and right archery like shooting a dove" action, the six key points (left / right shoulder, elbow, wrist) of the upper limbs are divided into a task group and a non-task group according to their functions. For example, in the left archery stage, the three key points of the right arm performing the main archery action are the task group, and the three key points of the left arm playing a stabilizing role are the non-task group. According to this rule, the system first reads the complete three-dimensional coordinate sequence of the six key points from the multi-dimensional limb movement trajectory data within the determined action period. Subsequently, the system processes the time sequence data of each key point frame by frame to calculate the instantaneous kinematics parameters. For the movement state of any key point in the b-th frame, the system reads the three-dimensional coordinates of the b-th frame and the b-1-th frame, and combines the known inter-frame time interval (for example, 1 / 30 seconds) to calculate the velocity vector of the b-th frame by using the backward difference method. Then, by calculating the difference between the velocity vectors of the b-th frame and the b-1-th frame, the velocity change vector reflecting the acceleration is obtained. At the same time, the system records a virtual mass value with the mass dimension (unit: kg) for each key point. This value is calculated by combining the mass distribution of each limb segment in standard anthropometry and the functional importance of each joint in a specific action according to the expert experience evaluation of Baduanjin. These frame-by-frame calculated velocity vectors, velocity change vectors, and corresponding virtual mass values will be used as inputs for the subsequent accurate calculation of the control kinetic energy of each key point.

[0086] Formula: The advantage of the formula is that the "controllability" and "smoothness" of the action are quantitatively integrated into the calculation of the traditional kinetic energy, thereby more comprehensively evaluating the quality of rehabilitation training. The first part of the formula is the standard kinetic energy expression, while the exponential penalty term in the second part is exactly the modulus of acceleration. When the action has rapid and incoherent acceleration and deceleration, this value increases, causing the exponential term to decrease sharply, thereby reducing the final energy value. Conversely, smooth, fluent, and controlled actions will be less penalized due to smaller acceleration, resulting in a score closer to their true kinetic energy. By setting a reasonable smoothness sensitivity coefficient λ, the formula can effectively distinguish between high-quality actions that meet rehabilitation requirements and low-quality actions that pose risks, providing a solid physical foundation for concentration evaluation.

[0087] m h is the virtual mass of the h-th key point, with the unit of kilogram (kg). This value aims to reflect the inertia of different limb segments and their functional importance in a specific action. The steps to obtain it are as follows: First, select a standard human model (for example, a 70 kg adult male) as a reference to determine the percentage of the upper arm, forearm, hand, and other segments in the total mass, and calculate their basic mass m base,hFor example, the hand accounts for 0.6%, and its base mass is 70 kg x 0.6% = 0.42 kg, the forearm with elbow joint accounts for 1.6%, and its base mass is 1.12 kg, the upper arm with shoulder joint accounts for 2.7%, and its base mass is 1.89 kg. Next, five experienced Qigong coaches are invited to score the importance of each key point in the "left and right bow" action on a Likert scale of 1 to 5, where 1 represents the least important and 5 represents the most important, and the average score S is taken. h Finally, the virtual mass m is calculated by the following formula: m h = m base,h x (1 + (S h - 3) x 0.15), which takes the average importance of 3 as the benchmark, and the base mass is adjusted by 15% for each increase or decrease of 1 in the score. For example, if the expert score for the wrist (hand) is 4.5, then the virtual mass is: m wrist = 0.42 kg x (1 + (4.5 - 3) x 0.15) = 0.42 x (1 + 0.225) = 0.5145 kg. In this example, the virtual mass of the right wrist is set to m h =

[0088] 0.5145 kg.

[0089] is the velocity vector of the b-th frame and h-th key point, with units of meters per second (m / s), which is obtained by dividing the displacement of the key point between two adjacent frames by the time interval. The specific calculation is as follows: where P b,h and P b-1,h are the three-dimensional coordinates (units: meters) of the h-th key point in the b-th frame and b-1-th frame obtained from the multi-dimensional limb motion trajectory data, and Δt is the inter-frame time interval (units: seconds). For example, in the b = 50th frame, the coordinates of the right wrist are (0.85, 1.20, 0.30), and in the b-1 = 49th frame, the coordinates are (0.83, 1.19, 0.31), and the inter-frame time interval Δt = 1 / 30 seconds. Then the velocity vector

[0090] Δt is the inter-frame time interval, with units of seconds (s), which is uniquely determined by the frame rate of the video capture device. If the video is recorded at a rate of 30 frames per second (30 FPS), then the inter-frame time interval is Δt = 1 / 30 ≈ 0.0333 s. This value is directly read from the metadata of the video file and serves as the time reference for all kinematic calculations. In this example, Δt is set to 1 / 30 s.

[0091] λ is the smoothness sensitivity coefficient, with units of seconds squared per meter (s 2 / m), which is used to quantify the degree of punishment for acceleration, ensuring that the exponential term is dimensionless, and its value is set to the inverse of a critical acceleration a crit , i.e. λ = 1 / a crit , a crit represents the acceleration threshold that distinguishes between "smooth" and "sudden" movements, and its determination method is as follows: collect the standard movement data of 10 Qigong coaches, calculate the acceleration modulus of all upper limb key points in all movement frames, and count the distribution of these acceleration values, and take the 95th percentile of the distribution as the critical acceleration a crit , which represents the acceleration level rarely exceeded in standard movements, for example, according to statistics, the 95th percentile of the acceleration of standard movements is 4.0 m / s 2 , then set a crit = 4.0 m / s 2 , therefore, the smoothness sensitivity coefficient λ = 1 / 4.0 = 0.25 s 2 / m.

[0092] Calculation process:

[0093] Take the calculation of the control kinetic energy of the right wrist (h = wrist) at the 50th frame (b = 50) as an example, and input the determined parameter values: m wrist = 0.5145 kg, Δt = 1 / 30 s, λ = 0.25 s 2 / m.

[0094] Obtain the velocity vector data:

[0095]

[0096] Calculate the square of the modulus of the velocity vector:

[0097]

[0098] Calculate the modulus of the acceleration vector

[0099]

[0100] Calculate the exponential decay term (i.e. the penalty factor):

[0101]

[0102] Calculate the control kinetic energy value:

[0103]

[0104] E′ 50,wrist = 0.5·0.5145 kg·0.54 (m / s) 2 ·0.2728;

[0105] E′ 50,wrist = 0.1389 J * 0.2728 = 0.0379 Joule (J);

[0106] The result shows that the control kinetic energy value of the right wrist of the practitioner at the 50th frame is 0.0379 Joule, and the acceleration (5.196 m / s 2 ) exceeds the set threshold value (4.0 m / s 2 ), resulting in a small exponential penalty factor (0.2728), so that the calculated control kinetic energy is much lower than its unpenalized instantaneous kinetic energy (0.1389 Joule), which quantitatively reflects the unsmooth and poor control of the movement at this moment. This physical quantity in Joule will be used as a basic unit to form the limb movement energy distribution feature in the subsequent aggregation.

[0107] Based on the control kinetic energy value, the system groups and accumulates the energy in the entire movement cycle to construct the limb movement energy distribution feature. According to the predefined "left and right open bow movement grouping rule", the system classifies the six upper limb key points in a specific stage of the movement into a task group or a non-task group. Then, the system creates two accumulators corresponding to the total energy of the task group and the non-task group, and sets their initial values to zero. The system then starts from the first frame of the movement cycle and traverses to the last frame. In the processing of each frame, the system reads the control kinetic energy value (unit: Joule) calculated for all six key points of the frame, and determines the group to which each key point currently belongs according to the grouping rule. If the key point belongs to the task group, its control kinetic energy value is added to the total energy accumulator of the task group; otherwise, it is added to the total energy accumulator of the non-task group. After completing the traversal of all frames in the movement cycle, the values in the two accumulators, i.e., the "total energy of the task group" and the "total energy of the non-task group", together constitute the quantitative limb movement energy distribution feature. This feature uses energy (Joule) as a unit, which has a clear physical meaning, and directly reflects how energy is distributed between the main executing limb and the auxiliary stabilizing limb during the entire movement process.

[0108] The steps for obtaining the Baduanjin rehabilitation training movement concentration quantitative index are as follows:

[0109] Based on the correspondence between the cumulative symmetry deviation value and the Baduanjin standard movement threshold value, the standard movement threshold value is set as a static reference limit. The cumulative symmetry deviation values of the current movement cycle are traversed in turn, and it is judged whether each value exceeds the standard movement threshold value range to form a set of judgment results that meet the standard movement condition or do not meet the standard movement condition.

[0110] According to the judgment result set, the limb movement energy distribution characteristics corresponding to the current action period are extracted, the total values of the task group energy and the non-task group energy are read respectively, the quantity ratio of the task group energy value and the non-task group energy value is calculated, and an energy concentration ratio index set is formed;

[0111] Based on the energy concentration ratio index set, in combination with each action standard judgment label in the judgment result set, the corresponding energy concentration ratio item is extracted by using label screening, and the quantification index of the Qigong rehabilitation training action concentration is established.

[0112] Specifically, based on the corresponding relationship between the cumulative symmetry deviation value and the Qigong standard action threshold value, first, the system takes the preset Qigong standard action threshold value as a static reference limit, which is determined by analyzing a standard action data set containing 50 experienced Qigong coaches. The specific method is as follows: the cumulative symmetry deviation value of each coach in the data set for completing each posture is calculated to form a deviation value sample set, then the extreme values of the highest 5% and the lowest 5% in the set are removed to reduce the noise caused by individual differences, finally the mean value of the remaining samples is calculated, and the mean value is multiplied by a safety factor 1.2 to serve as the standard action threshold value of the posture. For example, for the “left and right bow” action, the calculated threshold value is 3.5. When evaluating the action of the current user, the system will traverse the cumulative symmetry deviation value of each action period completed by the user in sequence. For example, a user has practiced “left and right bow” for 5 times in succession, and the system will obtain 5 cumulative symmetry deviation values. The system compares each value with the standard action threshold value of 3.5. If the deviation value of a certain action period is less than or equal to 3.5, the system generates a “standard” judgment label for the period. Otherwise, if the deviation value is greater than 3.5, a “non-standard” label is generated. This process provides a clear, binary standard degree judgment for each practice of the user. Finally, all these judgment labels are arranged in the order of practice to form a judgment result set of standard action conditions or non-standard action conditions.

[0113] According to the judgment result set, the system starts processing the limb movement energy distribution characteristics corresponding to each action period. First, the system extracts the limb movement energy distribution characteristics of all exercise periods of the current user from the memory. Each characteristic contains a pair of values, namely "task group energy total" and "non-task group energy total". Both values are in joules, representing the total controllable kinetic energy generated by the main and auxiliary limbs during the action period. The system iterates through each judgment label in the judgment result set and accurately finds and reads the energy distribution characteristics of the corresponding action period based on the action period index. Then, for each read energy distribution characteristic, the system performs a ratio calculation, i.e., dividing the task group energy total by the non-task group energy total. The result is a dimensionless ratio, defined as the energy concentration ratio. For example, in a certain action period, the task group energy total is 15.2 joules, and the non-task group energy total is 3.8 joules. The energy concentration ratio of this period is 15.2 / 3.8 = 4.0. The higher the ratio, the more energy the exerciser effectively concentrates on the limb performing the main task. Conversely, the energy distribution is more dispersed, and there may be redundant or compensatory actions. The system performs this calculation on all exercise periods of the user's energy distribution characteristics and stores all calculated energy concentration ratios in the order of exercise to form the energy concentration ratio index set.

[0114] Based on the energy concentration ratio index set, the system combines each action standard judgment label in the previously generated judgment result set to ultimately extract the core index that can quantify the concentration degree of rehabilitation training actions through a label screening method. The specific execution process is as follows: the system first creates an empty list to store the final concentration degree index. Then, the system iterates through the energy concentration ratio index set and the judgment result set simultaneously. The lengths of these two sets are the same, and the elements correspond one by one. For each pair of data items in the set (i.e., an energy concentration ratio and a judgment label), the system checks the content of the judgment label. The system only processes data items with a "standard" label. For data items with a "non-standard" label, the system directly skips and does not perform any operations. When the system encounters a data item with a "standard" label, it extracts the paired energy concentration ratio and adds this ratio to the previously created concentration degree index list. For example, if a user's 5 exercises are judged as "standard" in the 1st, 3rd, and 5th exercises, the corresponding energy concentration ratios are 4.2, 4.5, and 4.8, respectively. The system will extract these three values to ultimately obtain a list containing only the energy concentration ratios of standard actions, which is defined as the quantification index of the concentration degree of Baduanjin rehabilitation training actions.

[0115] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in other forms. Any skilled person in the art can modify or change the disclosed technical content into equivalent embodiments with equivalent changes, and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical solution content of the present application, still falls within the protection scope of the present application.

Claims

1. A Baduanjin rehabilitation training action recognition method, characterized in that, The method comprises the following steps: Based on the input Ba Duan Jin training action image sequence, the human body skeleton key point coordinates in each frame image are extracted, and multi-dimensional limb movement trajectory data is established; Based on the multi-dimensional limb movement trajectory data, for each frame coordinate point in the left limb key point sequence, a mirror mapping point coordinate set is calculated, and the mirror mapping point coordinate set is operated with the corresponding key point coordinates of the right limb in the multi-dimensional limb movement trajectory data to obtain an instantaneous symmetry deviation vector field; The modulus of all deviation vectors in the instantaneous symmetry deviation vector field is operated within the action period to obtain a cumulative symmetry deviation value, and the task group and non-task group key points are divided for the left and right opening bow action to calculate and obtain the limb movement energy distribution feature; The cumulative symmetry deviation value is compared with the preset Ba Duan Jin standard action threshold value to determine the standard degree of the action, and the ratio of the task group energy to the non-task group energy in the limb movement energy distribution feature is calculated to establish a Ba Duan Jin rehabilitation training action concentration quantitative index.

2. The Ba Duan Jin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the multi-dimensional limb movement trajectory data is: Based on the input Ba Duan Jin training action image sequence, the three-dimensional space coordinate values of all labeled nodes in the whole body skeletal structure are extracted frame by frame, and are arranged and summarized in frame sequence to generate a skeletal key point coordinate set; Based on the skeletal key point coordinate set, the nose tip coordinate, neck coordinate and trunk midpoint coordinate are selected for straight line fitting operation, the three-dimensional trajectories of the left shoulder coordinate, left elbow coordinate, left wrist coordinate and right shoulder coordinate, right elbow coordinate, right wrist coordinate are recorded synchronously within a continuous time window, and a limb key point time sequence coordinate matrix is generated; Based on the limb key point time sequence coordinate matrix, the three-dimensional trajectories of all key points are spliced and dimensionally expanded according to the time sequence, and the central straight line direction vector information is added to form the multi-dimensional limb movement trajectory data.

3. The Baduanjin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the mirror mapping point coordinate set is: Based on the multi-dimensional limb movement trajectory data, the left shoulder coordinate, left elbow coordinate and left wrist coordinate are read frame by frame, the central straight line direction vector in the same frame multi-dimensional limb movement trajectory data is called to perform orthogonal projection operation and reverse translation of the symmetric point position along the central straight line to form the mirror mapping point coordinate set.

4. The Baduanjin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the instantaneous symmetry deviation vector field is: Based on the mirror mapping point coordinate set, the right shoulder coordinate, right elbow coordinate and right wrist coordinate are matched frame by frame, the three-dimensional coordinate components are subtracted item by item, and the difference vectors are summarized to generate a left-right symmetric difference vector set; Based on the left-right symmetric difference vector set, all difference vectors are arranged in time sequence and labeled with corresponding key point labels to form an instantaneous symmetry deviation vector field.

5. The Baduanjin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the cumulative symmetry deviation value is: Based on the instantaneous symmetry deviation vector field, the deviation vectors corresponding to the right shoulder, right elbow and right wrist key points in all frames within the action period are extracted respectively, the three-dimensional difference vectors of the three key points in each frame are recorded, and the three types of key point deviation vector time sequence sets are obtained by combining in frame order and key point type order; According to the deviation vector time series set of the three types of key points, the square of the length of each frame deviation vector is calculated by key point type, and the square of the length of the difference vector between the current frame and the previous frame deviation vector is constructed, while the key point type index and frame index are marked, and the cumulative symmetry deviation value is calculated.

6. The Baduanjin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the limb movement energy distribution feature is: Based on the left and right bow action grouping rule, the three-dimensional coordinates of the right shoulder, right elbow, right wrist, left shoulder, left elbow and left wrist key points in the action period are read, the speed vector and speed change vector between each key point and each frame adjacent coordinates are calculated in turn, and the virtual mass value of the corresponding key point is recorded; According to the speed vector and the speed change vector, the control kinetic energy value is calculated combined with the virtual mass value; Based on the control kinetic energy value, the control kinetic energy values of the task group and the non-task group key points in the whole action period are accumulated respectively, and the control kinetic energy total amount of the two groups is aggregated to form the limb movement energy distribution feature.

7. The Baduanjin rehabilitation training action recognition method according to claim 1, characterized in that, The acquisition step of the Baduanjin rehabilitation training action concentration quantitative index is: Based on the corresponding relationship between the cumulative symmetry deviation value and the Baduanjin standard action threshold, the standard action threshold is set as a static reference limit, the cumulative symmetry deviation value of the current action period is traversed in turn, and whether it exceeds the standard action threshold range is judged item by item to form a judgment result set of whether it meets the standard action condition or not; According to the judgment result set, the limb movement energy distribution feature corresponding to the current action period is extracted, the task group energy and the non-task group energy total value are read respectively, the quantity ratio of the task group energy value and the non-task group energy value is calculated to form an energy concentration ratio index set. 8.The Baduanjin rehabilitation training action recognition method according to claim 7, characterized in that, The acquisition step of the Baduanjin rehabilitation training action concentration quantitative index also includes: based on the energy concentration ratio index set, combined with each action standard judgment label in the judgment result set, the corresponding energy concentration ratio item is extracted by using label screening method to establish the Baduanjin rehabilitation training action concentration quantitative index.

Images