Adaptive training method and system based on taijiquan action recognition

Abstract

This invention belongs to the interdisciplinary field of intelligent sports and computer vision. It proposes an adaptive training method based on Tai Chi movement recognition. The method extracts spatial movement features from the coordinates of key points on the three-dimensional human skeleton; extracts multi-scale motion features of the human body in the temporal domain based on these spatial movement features; performs global attention aggregation based on these multi-scale motion features to obtain human movement recognition results; matches the corresponding Tai Chi movement theory to these results; and, based on the Tai Chi movement theory, overlays multi-dimensional visual cues onto Tai Chi movement videos to intuitively highlight key points of the movements for the user. Based on prior knowledge or preset templates, it calculates movement problems, using graphics and / or text to indicate the location and cause of errors, and provides adjustment suggestions. This provides a more personalized training experience, helps reduce cognitive load, and improves training efficiency, effectiveness, and the fluency of cognitive learning.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of intelligent sports and computer vision, specifically to an adaptive training method and system based on Tai Chi movement recognition. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, with the development of technology and the popularization of equipment, online fitness has received widespread attention from the public. Users can learn and train in various fitness activities such as Tai Chi and yoga anytime and anywhere through mobile phones, computers, and other media. These exercises are relatively flexible and have significant physical and mental therapeutic effects. In particular, Tai Chi Chuan has multiple values, including fitness and health preservation, disease prevention and treatment, and cultural inheritance. Its movements are gentle, continuous, and combine softness and strength, making it extremely popular and listed as a World Intangible Cultural Heritage. However, Tai Chi Chuan movements are complex and delicate, requiring a high degree of precision and coordination. Traditional online platforms mostly use video recordings or live broadcasts, lacking analysis and feedback on individual movements. Learners find it difficult to accurately grasp the key points of the movements and may even suffer sports injuries due to incorrect postures, often failing to achieve the expected learning results.

[0004] To provide intelligent guidance during Tai Chi training, some systems combine feature engineering or template comparison methods for movement recognition. However, these methods rely on manually designed features (such as joint angles and motion trajectories), resulting in limited generalization ability. Recently, deep learning-based movement recognition methods have emerged and made significant progress. However, the Tai Chi movement system is complex, encompassing various routines such as the 8 techniques and 24 forms. Each movement contains intricate changes in movement across multiple stages, making it difficult for general coarse-grained movement recognition methods to adapt to the complex spatiotemporal variations of movements and to distinguish sub-movements in Tai Chi (such as "Peng Shi," "Lu Shi," and "Single Whip"), thus hindering the system from providing targeted guidance. Specifically, these movements often exhibit small inter-class differences and large intra-class divergence. That is, different movements are highly similar in their overall execution patterns, with only subtle differences existing in local spatiotemporal regions. Individuals' cognitive understanding of the movements varies, and the same movement performed by different subjects may show significant differences, posing a significant challenge to movement recognition. Existing recognition methods mostly sample and represent data uniformly from the entire video, weakening the discriminative local features. This results in insufficient recognition capabilities for fine-grained actions, failing to meet the needs of targeted training in online learning scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an adaptive training method and system based on Tai Chi movement recognition. It defines a "part-whole" hierarchical graph structure according to the movement characteristics of Tai Chi and the topological morphology of the human body. Utilizing self-attention, it extracts the intrinsic connections and higher-order relationships of movements, constructs temporal partitioning and aggregation structures to extract temporal features, and locates spatiotemporal sub-regions with significant local differences in movement. This allows for more accurate identification of fine-grained movement categories, supporting the generation of movement feedback. This invention supports learners uploading practice videos, automatically identifies Tai Chi movement categories, provides adaptive guidance, generates personalized feedback reports, and achieves intelligent error correction and progressive training guidance. It generates multimodal feedback including visual comparisons, text prompts, and difficulty adjustment suggestions, providing a more personalized training experience than traditional training methods. This helps reduce cognitive load and improves training efficiency, effectiveness, and the fluency of cognitive learning.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an adaptive training method based on Tai Chi movement recognition.

[0008] An adaptive training method based on Tai Chi movement recognition includes the following process:

[0009] Extract the key skeletal coordinates of the human body from a Tai Chi movement video, and perform posture correction and optimization based on the key skeletal coordinates to obtain the coordinates of key points of the three-dimensional human skeleton.

[0010] Based on the coordinates of the key points of the three-dimensional human skeleton, spatial motion features are extracted;

[0011] Based on the spatial motion characteristics, extract the multi-scale motion characteristics of the human body in the time domain;

[0012] Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results;

[0013] Based on the results of human motion recognition, the corresponding Tai Chi Chuan move theory is matched from the built-in knowledge base or user-uploaded content;

[0014] Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on Tai Chi movement videos to provide users with intuitive guidance on key points of the movements;

[0015] Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, along with adjustment suggestions.

[0016] In one implementation of the first aspect of the present invention, posture correction and optimization are performed based on the key skeleton coordinates to obtain the coordinates of key points of the three-dimensional human skeleton, including:

[0017] The MM Action algorithm is used to locate the coordinates of each key skeleton to obtain the three-dimensional pose data of the movement in the Tai Chi video. Based on the three-dimensional pose data, bidirectional temporal search and linear interpolation are performed to remove outliers. Savitzky-Golay filtering is then applied to smooth the skeletal coordinate sequence to eliminate inter-frame jitter. Combined with an outlier detection mechanism, abnormal data points are removed to obtain the final coordinates of the key points of the three-dimensional human skeleton.

[0018] In one implementation of the first aspect of the present invention, spatial motion features are extracted based on the coordinates of the key points of the three-dimensional human skeleton, including:

[0019] Self-attention between joint nodes is represented as: ;

[0020] The self-attention scores between joint nodes are: ;

[0021] in, Q i It is a query vector. K j It is a key vector. V It is a value vector. Representative based on joint and The transpose of the embedding generated by the shortest path distance between them Representing the The transpose of the enhanced hyperedge representation of a node. The dimension representing the key vector;

[0022] Spatial action features are obtained by using multi-head attention for parallel computation: ;

[0023] in, The self-attention score is controlled by each attention head. For the total number of attention heads.

[0024] In one implementation of the first aspect of the present invention, extracting multi-scale motion features of the human body in the time domain based on the spatial motion features includes:

[0025] Based on the partitioning strategy, sub-blocks are formed, and spatial action features are segmented along the time dimension. A uniform sub-block fragment For each sub-segment, local self-attention computation is performed to obtain the multi-scale motion features: ,in, Q i It is a query vector. K jIt is a key vector. It is a value vector. Dimensions that represent spatial characteristics.

[0026] As a further limitation of the first aspect of the present invention, based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results, including:

[0027] ;

[0028] in, For the first Attention score for each sub-block. For the number of attention heads, This is a global multi-head attention operation.

[0029] In one implementation of the first aspect of the present invention, multi-dimensional visual cues are superimposed onto a Tai Chi exercise video to intuitively guide the user through key movements, including:

[0030] Limb position cues: For postures and movements where the position needs to be emphasized, use colored dots to mark joints and lines to connect limb bones;

[0031] Motion trajectory prompts: For actions that require emphasis on the motion trajectory, the movement path of the limbs is displayed using semi-transparent colored arrows.

[0032] Dynamic mechanics prompts are displayed, with arrows indicating the direction of force and arrow length or color representing the magnitude of the force.

[0033] Metaphorical hints, using preset animations to highlight key points.

[0034] Data and text annotations provide optional overlays, dynamically displaying key text instructions and crucial data for action execution as the action approaches.

[0035] In one implementation of the first aspect of the present invention, the action problem is calculated based on prior knowledge or a preset template, and the error location and cause are indicated using graphics and / or text, along with adjustment suggestions, including:

[0036] The CARL fine-grained motion representation algorithm is used to process Tai Chi videos to generate user motion sequences. The DTW method is used to time-align the user motion sequences with the identified Tai Chi motion sequence templates, providing intuitive comparative feedback that the two actions are at the same speed and at the same moment.

[0037] Calculate the difference between key limb angles and the standard template, quantify joint angle deviations and motion trajectory offsets, and use red dots and lines to highlight incorrect bone connections and joints;

[0038] The teaching content is dynamically adjusted based on user performance and the content library or user-uploaded knowledge, including: feedback intensity and video playback speed.

[0039] Secondly, the present invention provides an adaptive training system based on Tai Chi movement recognition.

[0040] An adaptive training system based on Tai Chi movement recognition includes:

[0041] The human posture extraction module is configured to: extract the key skeletal coordinates of the human body in a Tai Chi movement video, perform posture correction and optimization based on the key skeletal coordinates, and obtain the coordinates of key points of the three-dimensional human skeleton.

[0042] The spatial hypergraph coding module is configured to extract spatial motion features based on the coordinates of the key points of the three-dimensional human skeleton.

[0043] The time-partition coding module is configured to extract multi-scale motion features of the human body in the time domain based on the spatial motion features.

[0044] The global attention aggregation module is configured to perform global attention aggregation based on the multi-scale motion characteristics of the human body in the time domain to obtain human action recognition results.

[0045] The motion knowledge retrieval module is configured to: match the corresponding Tai Chi Chuan move theory from the built-in knowledge base or user-uploaded content based on the human motion recognition results;

[0046] The action key visualization module is configured to: based on the Tai Chi Chuan movement theory, overlay multi-dimensional visual cues onto the Tai Chi Chuan movement video to provide users with intuitive guidance on the key points of the movements;

[0047] The motion analysis and feedback module is configured to: calculate motion problems based on prior knowledge or preset templates, provide graphical and / or textual prompts for error locations and causes, and offer adjustment suggestions.

[0048] Thirdly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium;

[0049] A processor, adapted to execute computer programs;

[0050] A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the adaptive training method based on Tai Chi movement recognition as described in the first aspect of the present invention.

[0051] Fourthly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed as described in the first aspect of the present invention for adaptive training based on Tai Chi movement recognition.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] 1. This invention innovatively proposes an adaptive training method based on Tai Chi movement recognition. According to the movement characteristics of Tai Chi and the topological morphology of the human body, a "part-whole" hierarchical graph structure is defined. Self-attention is used to extract the intrinsic connections and higher-order relationships of movements. Temporal partitioning and aggregation structures are constructed to extract temporal features, locating spatiotemporal sub-regions with significant local differences in the movement. This allows for more accurate identification of fine-grained movement categories, providing support for the generation of movement feedback. Learners can upload practice videos, and the method automatically identifies Tai Chi movement categories and provides adaptive guidance, generating personalized feedback reports. It achieves intelligent error correction and progressive training guidance, generating multimodal feedback including visual comparisons, text prompts, and difficulty adjustment suggestions. Compared to traditional training methods, this provides a more personalized training experience, helping to reduce cognitive load and improve training efficiency, effectiveness, and the fluency of cognitive learning.

[0054] 2. This invention can model the topological relationships and higher-order interactions of the human body based on the characteristics of Tai Chi movements, capture the spatial features of multi-region and joint collaborative work, and construct a refined spatial representation of Tai Chi movements; it can partition time according to the characteristics of Tai Chi movements, extract multi-region time features, and provide multi-scale time representation; it supports users to upload videos of movement execution and identify the corresponding fine-grained Tai Chi movement categories; based on the identification results, it retrieves and provides domain knowledge (such as move names, movement connotations, and movement essentials) corresponding to the current Tai Chi training movements, realizing the integration of theoretical and practical learning; it can overlay real-time visual annotations of essential points (such as key movement trajectories, center of gravity change prompts, breathing rhythm, etc.) to help users intuitively grasp the details of the movements; it can automatically analyze the differences between user movements and standard movements, accurately locate the key joints and movement trajectories of incorrect postures; and it can generate multimodal movement guidance opinions based on domain knowledge, including visual error annotations and text correction suggestions, greatly improving the user experience.

[0055] 3. The solution of this invention is simple, efficient, low-cost, and easy to operate. It adopts a hierarchical spatiotemporal graph attention mechanism, combined with spatial hypergraph coding and dynamic temporal partitioning strategies, which can accurately capture the subtle local differences and global collaborative patterns of Tai Chi movements. Compared with previous coarse-grained recognition methods for general movements, it effectively improves the accuracy of fine-grained recognition of Tai Chi sub-movements. By constructing an intuitive and easy-to-use movement recognition and training system, it can support users to upload videos and view movement recognition results, and obtain relevant domain knowledge and standard demonstration content from the built-in content library to help users efficiently master the movements and key points. Through the movement recognition results, combined with a multimodal real-time feedback mechanism, it provides movement guidance to users using visual annotations and text descriptions to help improve training effects and learning efficiency. It does not require complex settings and expensive equipment, has a low cost, and is suitable for home and personal experience. It is particularly suitable for Tai Chi movement recognition and training, but it can also be easily extended to other types of fitness movement recognition fields, such as Baduanjin, yoga, dance, etc., supporting low-cost and high-efficiency online movement learning, and has high application prospects.

[0056] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0057] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0058] Figure 1 A flowchart illustrating an adaptive training method based on Tai Chi movement recognition, provided as an exemplary embodiment of the present invention;

[0059] Figure 2 An action topology diagram provided as an exemplary embodiment of the present invention;

[0060] Figure 3 A diagram of a fine-grained Tai Chi movement recognition model based on hierarchical spatiotemporal graph attention, provided as an exemplary embodiment of the present invention;

[0061] Figure 4 A schematic diagram of Tai Chi movement recognition results provided as an exemplary embodiment of the present invention;

[0062] Figure 5 A schematic diagram illustrating the Tai Chi movements and key points of the movements, provided as an exemplary embodiment of the present invention;

[0063] Figure 6 A schematic diagram illustrating the accuracy of Tai Chi motion recognition as provided in an exemplary embodiment of the present invention;

[0064] Figure 7 A diagram of a Tai Chi motion recognition interface provided as an exemplary embodiment of the present invention;

[0065] Figure 8 A Tai Chi training demonstration interface diagram provided as an exemplary embodiment of the present invention;

[0066] Figure 9 A simultaneous comparison interface diagram of Tai Chi training provided as an exemplary embodiment of the present invention;

[0067] Figure 10 A schematic diagram of an adaptive training method based on Tai Chi movement recognition provided as an exemplary embodiment of the present invention;

[0068] Figure 11 A schematic diagram of a computer device provided for an exemplary embodiment of the present invention. Detailed Implementation

[0069] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0070] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0071] In online movement training, accurate identification of learners' movement execution status is a prerequisite for locating error sources and achieving effective guidance and feedback, and is crucial for targeted optimization of movements. To achieve the identification of Tai Chi sub-type movements and support online adaptive learning and training, this implementation proposes an adaptive training system based on Tai Chi movement recognition. This system allows users to upload Tai Chi videos according to their personal needs or training goals, accurately identifies fine-grained movement categories, and provides users with training content and adaptive training solutions, including visualized knowledge presentation and personalized movement correction guidance. It can meet the needs of Tai Chi movement content recognition, Tai Chi movement learning, and personalized training. This invention performs motion recognition based on ordinary videos. Leveraging the high degree of freedom and large amplitude of motion, as well as the inherent implicit connections and high-order collaborations within the motion area, it models the topological relationships and high-order interactions of the human body. This captures the spatiotemporal features of multi-region and joint collaborative work, achieving fine-grained recognition of Tai Chi sub-type movements. During training, it provides online Tai Chi training content and key points tailored to the learner's level, with real-time overlay of visual annotations (such as key movement trajectories, center of gravity shift prompts, breathing rhythms, etc.) to help users intuitively master the movements. Furthermore, it automatically analyzes the differences between the user's movements and standard movements to provide personalized feedback, helping to improve learning effectiveness and efficiency.

[0072] In the adaptive training system based on Tai Chi movement recognition proposed in this implementation, movement recognition is performed based on hierarchical spatiotemporal graph attention, and personalized training is achieved on this basis. It includes not only a recognition method with higher accuracy that conforms to the characteristics of Tai Chi movements and an easy-to-operate and use system environment, but also a set of effective Tai Chi movement training content. Users can upload their own Tai Chi movements or select different Tai Chi movements for training according to their needs, meeting diverse adaptive training requirements. The system can utilize mobile phones and computers, and can complete movement capture and recognition based on videos captured by ordinary cameras. During training, it provides movement recognition and multimodal guidance and feedback to offer personalized training. This system does not require expensive and complex movement capture equipment, and users can effectively improve their movement learning efficiency and effectiveness through repeated training.

[0073] Specifically, the Tai Chi Chuan movement recognition and adaptive training system based on hierarchical spatiotemporal graph attention described in this implementation consists of seven main functional modules, including: human posture extraction module, spatial hypergraph encoding module, temporal partitioning encoding module, global attention aggregation module, movement knowledge retrieval module, movement key visualization module, and movement analysis and feedback module.

[0074] The human pose extraction module described in this implementation uses the MM Action algorithm to estimate the three-dimensional human key coordinates of user-uploaded videos, and then performs repair and denoising. This module is combined with the subsequent spatial hypergraph coding module, temporal partition coding module, and global attention aggregation module to identify fine-grained Tai Chi movements in the uploaded videos; it is combined with the movement key visualization module to locate the spatial position of the movement key visualization layout; and it is combined with the movement analysis and feedback module to compare and analyze user movements with standard templates and generate adaptive feedback suggestions.

[0075] The spatial hypergraph coding module described in this implementation is based on the spatial characteristics of Tai Chi, including the natural structure of the human body and the characteristics of Tai Chi limb movements such as coordinated movements of the limbs and symmetrical movements. It defines the topological graph structure of the movements, performs spatial feature mining at multiple levels, extracts the intrinsic connections and higher-order relationships of the movements, and outputs a fine-grained spatial representation that conforms to the characteristics of Tai Chi.

[0076] The time partitioning coding module described in this implementation takes into account the diverse steps of Tai Chi movements, which involve various specific limb movements and changes such as advancing, retreating, forward extension, and backward movement within different local time regions, as well as the collaborative operation of multiple limbs. It introduces a time partitioning coding structure, which explores the movement relationships of limbs in the time domain by dynamically extracting time features of different ranges and levels, in order to capture the relationships between key local areas and global movement patterns. This enables the model to model fine local information and long-term dependencies, thereby obtaining an effective description of local time features.

[0077] The global attention aggregation module described in this implementation uses a multi-head attention structure to globally aggregate features between sub-blocks, integrates the global context information of each sub-block, and improves the model's ability to model long-term dependencies. It learns the collaborative relationships between important local topological regions and time regions of the action in the spatiotemporal dimension through a progressive approach from local to global and from details to the whole. It fully captures and integrates the spatial hierarchical interaction relationships and temporal motion features of the action, constructs a fine spatiotemporal joint feature representation, and maps the spatiotemporal features to the action category space, thereby achieving effective recognition of fine-grained categories of complex Tai Chi actions.

[0078] The action knowledge retrieval module described in this implementation searches for corresponding Tai Chi teaching content in the built-in knowledge base based on the action recognition results, including demonstration movements, key points of the movements, the connotation of the movements, and examples of errors.

[0079] The motion technique visualization module described in this implementation uses augmented reality technology to overlay three-dimensional motion techniques. Based on key point data from the human posture extraction module, it dynamically renders multi-dimensional visual cues in the user's real-time video stream, including joint trajectory arrows, center of gravity movement demonstrations, and breathing rhythm annotations. This intuitive knowledge visualization helps learners master the motion techniques.

[0080] The motion analysis and feedback module described in this implementation calculates the difference between the motion and the template motion based on the key point data of the human posture extraction module, presents the calculation results through visual cues, and generates a video file containing feedback, including comparison results of non-standard motions, as well as key points and modification suggestions and learning suggestions for the error parts generated during training.

[0081] Based on the aforementioned Tai Chi Chuan movement recognition and adaptive training system based on hierarchical spatiotemporal graph attention, this implementation proposes a specific adaptive training method, including the following steps:

[0082] Step (1): The user uploads a Tai Chi movement clip from the internet or recorded by themselves;

[0083] Step (2): Enable the pose extraction module, use the MM Action method to extract the three-dimensional human skeleton coordinates in the uploaded video, each frame contains 17 three-dimensional coordinates; detect and correct abnormal key points based on the spatiotemporal context, fix the problem of missing key points and positioning deviation, obtain more accurate key joint coordinates, and store them in the database.

[0084] Step (3): Activate the spatial hypergraph coding module, and perform spatial feature mining at the joint, limb, and whole body levels according to the spatial characteristics of Tai Chi Chuan movements. Extract the internal connections and high-order relationships of the movements to provide multi-level spatial representations of the movements.

[0085] Step (4): Enable the time partitioning encoding module, perform local time partitioning and modeling based on the time domain characteristics of Tai Chi movements, dynamically extract time features of different ranges and levels, and provide multi-scale time representation of movements;

[0086] Step (5): Enable the global attention aggregation module, combine the extracted spatial hierarchy structure and hierarchical time context to perform dynamic spatiotemporal modeling, realize the accurate identification and classification of the categories of each stage subclass, and show the user key skeletal information and fine-grained recognition results.

[0087] Step (6): Activate the action knowledge retrieval module, retrieve target training content based on the recognition results, and match the corresponding standard action points, common errors, and correction methods. If the user selects a training mode, the system will recommend training content suitable for the current level. If there is no matching content in the knowledge base, the system will prompt the user to upload standard template videos and action points to expand the database;

[0088] Step (7): Enable the motion knowledge visualization module to visualize the retrieved motion knowledge and overlay guidance information such as joint location, motion trajectory, center of gravity shift, and breathing rhythm onto the user-uploaded video stream;

[0089] Step (8): Enable the motion analysis and feedback module, use CARL and DTW algorithms to represent and align motions frame by frame, calculate the spatiotemporal difference between user actions and standard templates, generate multi-dimensional feedback, generate feedback videos containing error comparisons, and dynamically provide content recommendations based on user training status.

[0090] In step (2) of this implementation, the specific methods include:

[0091] (2-1) Based on video stream data, the MM Action algorithm was used to locate 17 key skeleton positions to obtain the three-dimensional pose data of the actions in the uploaded video;

[0092] (2-2) Based on the spatiotemporal context, perform bidirectional temporal search and linear interpolation to detect and correct abnormal key points, and fix the problems of missing key points and positioning deviations;

[0093] (2-3) Temporal filtering is performed using the Savitzky-Golay smoothing algorithm to eliminate inter-frame jitter in the skeletal coordinate sequence, and outlier detection mechanism is combined to remove abnormal data points to ensure the spatiotemporal continuity of the motion trajectory.

[0094] In step (3) of this implementation, the specific methods include:

[0095] (3-1) Define the skeletal topology region, including three partitioning strategies: limb skeletal nodes, limb regions, and contralateral regions;

[0096] (3-2) Calculate the joint connection representation: Define the joint connection structure using vertex sets. and super edge Defined association matrix Representing a topological graph of the human body, the edge relationships in the matrix are defined as follows:

[0097] (1);

[0098] in, These are the joint vertices in the skeletal topology. A hyperedge is composed of joints.

[0099] (3-3) Computation of Region Hyperedge Representation: Define the region hyperedge structure to represent each joint in the skeletal topology graph as follows: dimensional vector The features of each joint are aggregated into the corresponding hyperedge. By weighted summation and normalization of the joint features belonging to the unified hyperedge, a hyperedge representation describing a region composed of multiple joints is obtained:

[0100] (2);

[0101] in, This is the inverse of the hyperedge degree matrix, used to normalize the hyperedge features; The projection matrix parameters are used to transform the hyperedge features to obtain the representation.

[0102] An enhanced hyperedge representation is introduced, in which global information of the hyperedge is fed back and distributed to each internal joint. Higher-order local context information from the hyperedge is incorporated into the feature representation of each joint, constructing an enhanced hyperedge representation, such as:

[0103] (3)

[0104] (3-4) Calculate the self-attention representation that includes pairwise relationships between joints, higher-order hyperedge relationships, and relative position embeddings: Generate embeddings for each pair of joints based on the shortest path distance between joints. This is used to describe the topological structure information of the skeleton, and the self-attention is calculated as follows:

[0105] (4);

[0106] (5);

[0107] Among them, the self-attention structure uses , and Project the input onto the query ,key K j The sum V, the pairwise relationship between joints is determined by Defined, and represented using linear transformations; hyperedge relations are derived from... Definition, taking into account the joint relationships between multiple joints; Calculate the relative position embedding of bones to capture the spatial dependencies between joints; For attention bias, Representing the The first joint node and the first The first joint node Attention score of each attention head Representative based on joint i and j The transpose of the embedding generated by the shortest path distance between them.

[0108] (3-5) Computing spatial representations using multi-head attention:

[0109] (6);

[0110] in, For self-attention computation controlled by each attention head, The number of attention heads is used to calculate the self-attention for each attention head, which includes the sum of the self-attention scores for each key node pair.

[0111] In step (4) of this implementation, the specific methods include:

[0112] (4-1) Define the local partitioning strategy: Set the partitioning factor to P l = That is, the time partitioning factor gradually decreases as the network layer deepens. Sub-blocks are divided according to the partitioning strategy, i.e., given a time step of... of 2D eigenvectors The features are segmented along the time dimension. A uniform sub-block fragment For each sub-segment, perform local self-attention computation to understand the temporal action relationships within the sub-segment;

[0113] (4-2) Calculate local temporal attention: Extract sub-block temporal features using window-limited local self-attention, and limit the region of interest to a specified window. Inside, that is, for the input sequence area in ,use The size of the window used to control the attention working area is defined by the scale. The attention score within each sub-block is calculated as follows:

[0114] (7);

[0115] in, Q i It is a query vector. K j It is a key vector. It is a value vector.

[0116] In step (5) of this implementation, the specific methods include:

[0117] (5-1) A multi-head attention structure is used to globally aggregate features between sub-blocks and establish connections between different sub-blocks. Each head focuses on a different time period, and the global dynamics of the entire action sequence are modeled through the interaction of different time segments. The global temporal attention is expressed as:

[0118] (8);

[0119] in, For intra-block attention, For the number of attention heads, This is a global multi-head attention operation.

[0120] (5-2) Spatiotemporal features are mapped to the action category space through global average pooling and fully connected layers to output fine-grained Tai Chi action categories.

[0121] In step (7) of this implementation, the specific methods include:

[0122] (7-1) The Open AI GPT language model is used to analyze unstructured teaching knowledge and extract the five common entity categories and their corresponding role relationships in the five major action teaching methods: body, action, metaphor, data, and direction.

[0123] (7-2) Based on the extracted entity categories, establish a visualization mapping between semantic content and visualization form, that is, visualize limbs as skeletal lines or joints; visualize metaphors as preset animations; map angles and other data as lines combined with text; and visualize directions as arrows.

[0124] (7-3) Based on the human pose estimation results generated in step (2), the visualization layout is placed in the corresponding spatial location.

[0125] In step (8) of this implementation, the specific methods include:

[0126] (8-1) The CARL fine-grained action representation algorithm generates a fine-grained frame-by-frame representation of the uploaded user action execution sequence.

[0127] (8-2) Use the DTW method to time-align the user's action sequence with the identified Tai Chi action sequence template, providing intuitive comparative feedback that the two actions are at the same speed and at the same moment.

[0128] (8-3) Calculate the difference between the key limb angles and the standard template, quantify the joint angle deviation (such as the knee flexion error) and the movement trajectory offset (such as the hand movement path deviation), and use red dots and lines to highlight the incorrect bone connections and joints.

[0129] As a typical embodiment, such as Figure 1 As shown, this implementation provides an illustrative adaptive training method based on Tai Chi movement recognition, including the following specific steps:

[0130] (1) Start the program;

[0131] (2) Users upload video clips of Tai Chi movements;

[0132] (3) After uploading the video, the system will start the human posture extraction module for the video to extract the three-dimensional skeleton coordinates;

[0133] (4) The system starts the spatial hypergraph coding module to extract multi-level spatial representations;

[0134] (5) The system starts the time partitioning encoding module to extract multi-scale time representation;

[0135] (6) The system starts the global attention aggregation module, generates a spatiotemporal representation, and outputs fine-grained recognition results.

[0136] (7) The system displays key skeletal information and fine-grained recognition results to the user.

[0137] (8) Determine whether to select training mode. If no is selected, jump to (14). If yes is selected, jump to (9).

[0138] (9) The system searches for the corresponding action in the action knowledge retrieval module based on the recognition results;

[0139] (10) Determine if the corresponding action exists in the content library, then jump to (12). If it does not exist, then jump to (11).

[0140] (11) The system prompts the user to upload a standard template video and knowledge of the action points, etc.;

[0141] (12) The system starts the action key visualization module to visualize the retrieved action knowledge and overlay guidance information such as joint positions, action trajectory, center of gravity movement, and breathing rhythm on the video stream uploaded by the user;

[0142] (13) The system starts the motion analysis and feedback module, uses CARL and DTW algorithms to represent and align the motion frame by frame, calculates the spatiotemporal difference between the user's motion and the standard template, generates multi-dimensional feedback, generates feedback videos containing error comparisons, and dynamically provides content recommendations based on the user's training status.

[0143] (14) Determine whether the training has ended. If yes, exit the system; otherwise, jump to (1).

[0144] like Figure 2 As shown, in step (3-2) of this embodiment, the topological map partitioning of Tai Chi movements is defined as joint partitioning; limb partitioning (including five parts: upper body trunk, left upper limb, right upper limb, left lower limb, and right lower limb); and contralateral partitioning (including three parts: left forearm-right forearm, left lower leg-right lower leg, and other paired areas of the upper body). This partitioning definition originates from the high freedom and wide range of movement characteristics of Tai Chi, as well as the movement characteristics of limb coordination and contralateral coordination, which can help the model extract discriminative spatial features.

[0145] like Figure 3 As shown, the model structure for fine-grained movement recognition of Tai Chi Chuan in steps (3), (4), and (5) includes a spatial hypergraph encoding structure, a temporal partitioning encoding structure, and a global attention aggregation structure. Using three-dimensional skeletal coordinates as input, the model first performs spatial hypergraph encoding, including... Figure 2 The process involves hypergraph partitioning and spatial MHSA modeling, followed by temporal partitioning encoding, temporal partition definition, and intra-block attention calculation. Global attention aggregation is then performed, and inter-block attention is calculated through temporal MHSA modeling. After global average pooling and fully connected layers, the fine-grained category of the identified action is finally output.

[0146] like Figure 4 As shown, this example demonstrates the recognition results for some upload actions. It can be seen that this method can effectively distinguish between similar actions and output recognition results for targeted teaching in subsequent training processes.

[0147] like Figure 5 The image shows some examples of Tai Chi movements and a brief description of the key points in this example.

[0148] like Figure 6The image shows the recognition results of the method described in this example on a Tai Chi fine-grained movement analysis dataset. This dataset includes four subcategories of basic Tai Chi movements and 12 atomic movements. The definitions and divisions of each movement component and its corresponding events and stages strictly adhere to Tai Chi standards and reference manuals, and have been certified by national-level professional Tai Chi coaches. It includes 13 densely labeled key events and 12 stages (movement stages between two key events). This dataset was obtained through authorized downloads from video platforms and recordings in collaboration with Tai Chi studios, including 119 complete Tai Chi performance videos (ranging from 602 to 2380 frames) and 2187 stage-structured video clips, with a balanced distribution of data across categories. All videos in the dataset originate from real-world scenes, covering multiple shooting perspectives (such as front, back, left, and right), including various indoor and outdoor scenes such as training venues, homes, parks, and courtyards.

[0149] In this implementation, recognition accuracy is used as the primary evaluation metric to measure the performance of the detection model in classification tasks. This metric is defined as the ratio of the number of samples correctly classified by the model to the total number of samples on a given test dataset, reflecting the model's ability to make correct predictions in action recognition tasks.

[0150] (9);

[0151] Here, "Number of correctly classified samples" represents the number of times the model successfully classifies samples in the dataset into the correct category, while "Total number of samples" represents the total number of samples in the test dataset. It is evident that this method achieves more competitive performance in Tai Chi motion recognition compared to other methods.

[0152] Figure 7 The figure shown is the operation interface diagram of the system for action recognition in this embodiment. The system outputs fine-grained recognition results of video actions and textual instructions. Figure 8 The diagram shown is a system training demonstration interface in this embodiment. The system generates a text-image visualization of the action key points, and users can view explanations of common mistakes, action diagrams, and demonstrations from different angles in the content library. Figure 9 The image shown is a simultaneous comparison diagram in this embodiment. The system calculates the joint angles, marks limbs exceeding preset thresholds or normal characteristic ranges in red, and generates corresponding modification schemes. Training videos can be recorded and uploaded by users using devices such as mobile phone cameras, laptop cameras, and webcams. After the system recognizes the videos, interactive online training can be completed using the built-in knowledge base, tutorials, or user-uploaded content. The training location is flexible, and the equipment is easy to use.

[0153] Figure 10An independent adaptive training method based on Tai Chi movement recognition is presented, comprising the following steps:

[0154] S101: Extract the key skeletal coordinates of the human body from the Tai Chi movement video, perform posture correction and optimization based on the key skeletal coordinates, and obtain the coordinates of key points of the three-dimensional human skeleton.

[0155] S102: Extract spatial motion features based on the coordinates of the key points of the three-dimensional human skeleton;

[0156] S103: Based on the spatial motion characteristics, extract the multi-scale motion characteristics of the human body in the time domain;

[0157] S104: Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain the human action recognition result;

[0158] S105: Based on the human motion recognition results, match the corresponding Tai Chi Chuan move theory from the built-in knowledge base or user-uploaded content;

[0159] S106: Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on the Tai Chi movement video to provide users with intuitive guidance on the key points of the movements;

[0160] S107: For calculation problems based on prior knowledge or preset templates, use graphics and / or text to indicate the location and cause of the error, and provide adjustment suggestions.

[0161] Figure 11 A computer device is shown, which includes a processor 1101, a communication interface 1102, and a computer-readable storage medium 1103. The processor 1101, communication interface 1102, and computer-readable storage medium 1103 can be connected via a bus or other means.

[0162] The communication interface 1102 is used to receive and send data. The computer-readable storage medium 1103 can be stored in the memory of the electronic device. The computer-readable storage medium 1103 is used to store computer programs, which include program instructions. The processor 1101 is used to execute the program instructions stored in the computer-readable storage medium 1103.

[0163] The processor 1101 is the computing and control core of the electronic device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to achieve the corresponding method flow or corresponding function.

[0164] The processor 1101 is configured to perform the following process:

[0165] Extract the key skeletal coordinates of the human body from a Tai Chi movement video, and perform posture correction and optimization based on the key skeletal coordinates to obtain the coordinates of key points of the three-dimensional human skeleton.

[0166] Based on the coordinates of the key points of the three-dimensional human skeleton, spatial motion features are extracted;

[0167] Based on the spatial motion characteristics, extract the multi-scale motion characteristics of the human body in the time domain;

[0168] Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results;

[0169] Based on the results of human motion recognition, the corresponding Tai Chi Chuan move theory is matched from the built-in knowledge base or user-uploaded content;

[0170] Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on Tai Chi movement videos to provide users with intuitive guidance on key points of the movements;

[0171] Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, along with adjustment suggestions.

[0172] This invention also provides a computer-readable storage medium, which is a memory device in an electronic device for storing programs and data. It is understood that the computer-readable storage medium here may include both built-in storage media in the electronic device and extended storage media supported by the electronic device. The computer-readable storage medium provides storage space for storing the processing system of the electronic device.

[0173] Furthermore, this storage space also contains one or more instructions suitable for loading and execution by the processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM memory or unstable memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0174] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to perform the following process:

[0175] Extract the key skeletal coordinates of the human body from a Tai Chi movement video, and perform posture correction and optimization based on the key skeletal coordinates to obtain the coordinates of key points of the three-dimensional human skeleton.

[0176] Based on the coordinates of the key points of the three-dimensional human skeleton, spatial motion features are extracted;

[0177] Based on the spatial motion characteristics, extract the multi-scale motion characteristics of the human body in the time domain;

[0178] Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results;

[0179] Based on the results of human motion recognition, the corresponding Tai Chi Chuan move theory is matched from the built-in knowledge base or user-uploaded content;

[0180] Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on Tai Chi movement videos to provide users with intuitive guidance on key points of the movements;

[0181] Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, along with adjustment suggestions.

[0182] The present invention also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the following process:

[0183] Extract the key skeletal coordinates of the human body from a Tai Chi movement video, and perform posture correction and optimization based on the key skeletal coordinates to obtain the coordinates of key points of the three-dimensional human skeleton.

[0184] Based on the coordinates of the key points of the three-dimensional human skeleton, spatial motion features are extracted;

[0185] Based on the spatial motion characteristics, extract the multi-scale motion characteristics of the human body in the time domain;

[0186] Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results;

[0187] Based on the results of human motion recognition, the corresponding Tai Chi Chuan move theory is matched from the built-in knowledge base or user-uploaded content;

[0188] Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on Tai Chi movement videos to provide users with intuitive guidance on key points of the movements;

[0189] Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, along with adjustment suggestions.

[0190] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0191] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital cable) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0192] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive training method based on Tai Chi movement recognition, characterized in that, Includes the following processes: Extract the key skeletal coordinates of the human body from a Tai Chi movement video, and perform posture correction and optimization based on the key skeletal coordinates to obtain the coordinates of key points of the three-dimensional human skeleton. Based on the coordinates of the key points of the three-dimensional human skeleton, spatial motion features are extracted, including: The self-attention between joint nodes is represented as: ; The self-attention score between joint nodes is: ; in, Q i It is a query vector. K j It is a key vector. V It is a value vector. Representative based on joint i and j The transpose of the embedding generated by the shortest path distance between them Representing the The transpose of the enhanced hyperedge representation of a node. The dimension representing the key vector; Parallel computing is performed using multi-head attention to obtain spatial action features: ; in, The self-attention score is controlled by each attention head. For the total number of attention heads; Based on the spatial motion features, multi-scale motion features of the human body in the time domain are extracted, including: Based on the partitioning strategy, sub-blocks are formed, and spatial action features are segmented along the time dimension. A uniform sub-block fragment For each sub-segment, local self-attention computation is performed to obtain the multi-scale motion features as follows: ; in, Q i It is a query vector. K i It is a key vector. It is a value vector. Dimensions that represent spatial characteristics; Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results; Based on the results of human motion recognition, the corresponding Tai Chi Chuan move theory is matched from the built-in knowledge base or user-uploaded content; Based on the theory of Tai Chi movements, multi-dimensional visual cues are superimposed on Tai Chi movement videos to provide users with intuitive guidance on key points of the movements; Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, along with adjustment suggestions.

2. The adaptive training method based on Tai Chi movement recognition as described in claim 1, characterized in that, Based on the key skeletal coordinates, pose correction and optimization are performed to obtain the coordinates of key points of the three-dimensional human skeleton, including: The MM Action algorithm is used to locate the coordinates of each key bone to obtain the three-dimensional pose data of the movements in the Tai Chi video. Based on the three-dimensional pose data, a bidirectional temporal search and linear interpolation are performed to remove outliers. Savitzky-Golay filtering is then applied to smooth the data and eliminate inter-frame jitter in the skeletal coordinate sequence. An outlier detection mechanism is then used to remove abnormal data points, resulting in the final coordinates of the key points of the three-dimensional human skeleton.

3. The adaptive training method based on Tai Chi movement recognition as described in claim 1, characterized in that, Based on the multi-scale motion characteristics of the human body in the time domain, global attention aggregation is performed to obtain human action recognition results, including: ； in, For the first Attention score for each sub-block. For the number of attention heads, This is a global multi-head attention operation.

4. The adaptive training method based on Tai Chi movement recognition as described in claim 1, characterized in that, Multi-dimensional visual cues are overlaid onto Tai Chi movement videos to intuitively highlight key movements for users, including: Limb position cues: For postures and movements where the position needs to be emphasized, use colored dots to mark joints and lines to connect limb bones; Motion trajectory prompts: For actions that require emphasis on the motion trajectory, the movement path of the limbs is displayed using semi-transparent colored arrows. Dynamic mechanics prompts are displayed, with arrows indicating the direction of force and arrow length or color representing the magnitude of the force. Metaphorical hints, using preset animations to highlight key points; Data and text annotations provide optional overlays, dynamically displaying key text instructions and crucial data for action execution as the action approaches.

5. The adaptive training method based on Tai Chi movement recognition as described in claim 1, characterized in that, Based on prior knowledge or preset templates, the system calculates action problems and uses graphics and / or text to indicate the location and cause of errors, providing adjustment suggestions, including: The CARL fine-grained motion representation algorithm is used to process Tai Chi videos to generate user motion sequences. The DTW method is used to time-align the user motion sequences with the identified Tai Chi motion sequence templates, providing intuitive comparative feedback that the two actions are at the same speed and at the same moment. Calculate the difference between key limb angles and the standard template, quantify joint angle deviations and motion trajectory offsets, and use red dots and lines to highlight incorrect bone connections and joints; The teaching content is dynamically adjusted based on user performance and the content library or user-uploaded knowledge, including: feedback intensity and video playback speed.

6. A self-adaptive training system based on taijiquan action recognition, characterized in that, include: The human posture extraction module is configured to: extract the key skeletal coordinates of the human body in a Tai Chi movement video, perform posture correction and optimization based on the key skeletal coordinates, and obtain the coordinates of key points of the three-dimensional human skeleton. The spatial hypergraph encoding module is configured to extract spatial motion features based on the coordinates of the key points of the three-dimensional human skeleton, including: The self-attention between joint nodes is represented as: ; The self-attention scores between joint nodes are: ; in, Q i It is a query vector. K j It is a key vector. V It is a value vector. Representative based on joint i and j The transpose of the embedding generated by the shortest path distance between them Representing the The transpose of the enhanced hyperedge representation of a node. The dimension representing the key vector; Spatial action features are obtained by using multi-head attention for parallel computation: ; wherein, is a self-attention score controlled by each attention head, is the total number of attention heads; The time-partitioning encoding module is configured to: extract multi-scale motion features of the human body in the time domain based on the spatial motion features; divide the spatial motion features into sub-blocks according to a partitioning strategy; and segment the spatial motion features along the time dimension. A uniform sub-block fragment For each sub-segment, local self-attention computation is performed to obtain the multi-scale motion features as follows: ; in, Q i It is a query vector. K i It is a key vector. It is a value vector. Dimensions that represent spatial characteristics; The global attention aggregation module is configured to perform global attention aggregation based on the multi-scale motion characteristics of the human body in the time domain to obtain human action recognition results. The motion knowledge retrieval module is configured to: match the corresponding Tai Chi Chuan move theory from the built-in knowledge base or user-uploaded content based on the human motion recognition results; The action key visualization module is configured to: based on the Tai Chi Chuan movement theory, overlay multi-dimensional visual cues onto the Tai Chi Chuan movement video to provide users with intuitive guidance on the key points of the movements; The motion analysis and feedback module is configured to: calculate motion problems based on prior knowledge or preset templates, provide graphical and / or textual prompts for error locations and causes, and offer adjustment suggestions.

7. A computer device, comprising: include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the adaptive training method based on Tai Chi movement recognition as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 5, the adaptive training method based on Tai Chi movement recognition.

Images