Method and system for automatically evaluating strength training action posture safety

Abstract

The invention discloses a method and system for automatically evaluating strength training action posture safety, and the method comprises the steps: receiving a video transmitted by a user side, extracting frame image features, and outputting an action category sequence number; outputting a calling model vector according to the frame image features and the action category serial numbers; calling a corresponding body part granularity grade model according to the calling model vector; outputting human body posture key point coordinates according to the body part granularity grade model; calculating a movement speed and a joint angle according to the human body posture key point coordinates and the action category serial numbers; judging whether the movement speed or the joint angle is within a safety threshold range or not; and if the movement speed or the joint angle is recognized not to be within the safety threshold range, giving a prompt alarm. According to the method, subjectivity and inaccuracy in manual observation are reduced, and the safety and effect of exercise training are improved.

Description

Technical Field

[0001] The invention relates to the technical field of fitness, and in particular discloses a method and a system for automatically evaluating the safety of strength training action postures. Background Art

[0002] In today's fitness culture, people are increasingly focusing on shaping their ideal body shape and body through exercise. Many people focus on strength training in indoor gyms, such as using dumbbells, barbells or fixed machines to perform muscle-specific training. Repetitive training of these movements can effectively increase the strength and size of specific muscle groups.

[0003] However, due to the repetitive and fixed nature of the movements, improper posture or lack of proper observation and guidance may increase the risk of sports injuries. For example, when doing a squat, the wrong posture of the knee slightly exceeding the toes may be overlooked. In addition, fatigue or lack of concentration will also affect one's own detection ability, making it impossible to continuously and accurately observe each movement and correct errors in time. In this case, the safety and effectiveness of training are significantly reduced, increasing the risk of sports injuries.

[0004] Therefore, the posture estimation model of the artificial intelligence algorithm can be used to judge the safety of the action. However, there are many types of posture estimation models currently available, which makes people inefficient in model selection. Existing models only focus on identifying specific body parts, such as hands, torso, feet, etc., which requires people to choose different models for different body parts during use, increasing the complexity. In addition, different models have significant differences in accuracy, computational cost, and detail processing capabilities, further increasing the difficulty of selecting a suitable model.

[0005] In addition, the quality of the action video will directly affect the gesture recognition results. In practical applications, due to the limitations of shooting conditions, such as camera angle, distance, lighting, and human body occlusion, the characters in the video are often not fully displayed or the video clarity is insufficient. Under these undesirable detection conditions, even fine-grained estimation methods find it difficult to provide accurate results, and their credibility is reduced.

[0006] Therefore, the prior art mainly has the following defects:

[0007] 1. Human observation and assessment of movement posture are inaccurate, increasing the risk of sports injuries

[0008] In the process of manual observation and evaluation of movement posture, due to differences in human factors and subjective judgment, inaccurate and incomplete evaluation of movement posture may occur, thereby increasing the risk of sports injuries. For example, when doing a squat, you may not notice details such as the knee slightly exceeding the toes, or due to fatigue, lack of concentration, etc., you cannot continuously and accurately monitor the joint angles of the movement, so you cannot find and correct errors in time. Therefore, manual observation and evaluation methods will significantly reduce the safety and effectiveness of training and increase the risk of sports injuries.

[0009] 2. The posture estimation model is complex and diverse, and the model selection efficiency is low

[0010] There are many different pose estimation models available, and each model can usually only recognize specific body parts. For example, some models are specialized for hand pose estimation, while others are used for torso or foot pose recognition. This limitation requires users to select the appropriate model based on their specific needs, which is a complicated and inefficient process. In addition, different models vary in accuracy, computational cost, and granularity, making it more difficult for users to quickly and correctly select the model that best suits the current task, significantly reducing the overall efficiency and application flexibility.

[0011] 3. The video is not clear and the recognition accuracy is low

[0012] In some shooting scene restrictions, people cannot completely cover the subject during shooting, or the video itself is not clear enough, resulting in the detection conditions not meeting the requirements, and the fine-grained pose estimation results are not precise enough and the accuracy is low. In this case, even if a fine-grained pose estimation method is used, the data obtained is difficult to guarantee its accuracy and reliability, and consumes a lot of computing resources.

[0013] In summary, it is particularly important to provide a method to automatically evaluate the safety of strength training movements, which can not only significantly improve the safety and effectiveness of training, but also greatly reduce the risk of sports injuries and provide more scientific training guidance for fitness enthusiasts. Summary of the invention

[0014] The present invention provides a method and system for automatically evaluating the safety of strength training movement postures, aiming to solve at least one of the above-mentioned defects.

[0015] The present invention relates to a method for automatically evaluating the safety of strength training action postures, comprising the following steps:

[0016] Receive a video from the user, extract frame image features, and output action category numbers;

[0017] Output the call model vector according to the frame image features and action category serial number;

[0018] According to the calling model vector, the corresponding body part granularity level model is called;

[0019] Output the coordinates of key points of human posture according to the granularity level model of body parts;

[0020] Calculate the movement speed and joint angle according to the coordinates of the key points of human posture and the sequence number of the action category;

[0021] Determine whether the movement speed or joint angle is within the safety threshold;

[0022] If it is detected that the movement speed or joint angle is not within the safety threshold range, a prompt warning will be given.

[0023] Furthermore, the steps of receiving a video transmitted by a user end, extracting frame image features, and outputting action category serial numbers include:

[0024] Extract a series of continuous frame images from the video at fixed time intervals;

[0025] Preprocess the extracted frame images to ensure the consistency and effectiveness of the input model;

[0026] Use 3D convolutional layers to process fixed frame sequences, and use 3D convolutional kernels to slide in three dimensions: time, width, and height to extract temporal and spatial features simultaneously; flatten the four-dimensional features processed by the 3D convolution module into three-dimensional feature representations; build a backbone network based on a dual-stream Transformer architecture, and use two parallel Transformer encoding modules in each layer to extract temporal and spatial features respectively, and fuse the two features by adding them together to obtain a more comprehensive feature representation; four-dimensional features include frame number, width, height, and feature dimension, and three-dimensional features include temporal, spatial, and feature dimensions;

[0027] The fused three-dimensional feature vector is flattened into a one-dimensional feature vector and input into the fully connected layer. The fully connected layer maps the three-dimensional feature vector to the action category space, and the number of output nodes is equal to the number of action categories. The output is converted into a probability distribution through the softmax activation function. According to the output probability distribution, the category with the highest probability is selected as the prediction result, and the sequence number of the action category with the highest probability is output.

[0028] Furthermore, the step of outputting the call model vector according to the frame image features and the action category sequence number includes:

[0029] The frame image features and the action category serial number output in the previous step are passed as input to the decision model; the action category serial number has a corresponding body part model to be called;

[0030] The decision model calls the body part model of the corresponding granularity level based on the frame image features and the action category number output in the previous step, and the video quality, and outputs the final model calling result, which is represented by a vector.

[0031] Furthermore, before receiving a video from the user, extracting frame image features, and outputting the action category serial number, the data set is prepared, and the corresponding body part model is determined to be called according to the action category serial number; the extracted video feature data is respectively transmitted to the models of each granularity level of the corresponding body part in the model pool, and each model outputs the total number N of key point estimates and the average error ME, and the average error ME is calculated by the following formula:

[0032]

[0033] Among them, ME represents the mean error, and the key point positions output by the model are (x1, y1), (x2, y2)...(x n ,y n ); The key point position of the original image is correctly marked as (x1 ′ , y1 ′ )、(x2 ′ , y2 ′ )……(x n ′ ,y n ′ ), where N represents the total number of key point estimates.

[0034] Furthermore, the decision model calls the body part model of the corresponding granularity level according to the frame image features and the action category number output in the previous step, and outputs the final model calling result according to the video quality. In the step where the result is represented by a vector, if the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are called, the final output vector [3, 3, 0, 0] is obtained.

[0035] Furthermore, according to the body part granularity level model, the step of outputting the coordinates of key points of human posture includes:

[0036] After obtaining the final output vector [3, 3, 0, 0], the output vector [3, 3, 0, 0] is passed into the posture estimation model pool, indicating that the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are to be called to obtain the two-dimensional coordinates of the human body posture key points in each frame of the video.

[0037] Furthermore, the joint angle includes the elbow joint angle. In the step of calculating the motion speed and the joint angle according to the coordinates of the key points of the human body posture and the action category sequence number, the elbow joint angle is calculated by the following formula:

[0038]

[0039] Among them, ∠ABC represents the elbow joint angle, represents the vector from the elbow to the shoulder, represents the vector from elbow to wrist;

[0040]

[0041]

[0042] Among them, the coordinates of the three points ∠ABC are A(x1, y1), B(x2, y2), and C(x3, y3).

[0043] Furthermore, in the step of calculating the motion speed and joint angle according to the coordinates of the key points of the human body posture and the action category number, the motion speed is calculated by the following formula:

[0044]

[0045] Among them, v represents the movement speed, s represents the coordinates (x1, y1) of the key point A in the previous frame to the key point A in the current frame ′ Coordinates(x1 ′ , y1 ′ ), t represents the interval time between each frame of image, and n represents the number of image frames.

[0046] Furthermore, after the step of determining whether the motion speed or the joint angle is within the safety threshold range, the following steps are further included:

[0047] If the movement speed and joint angle are detected to be within the safety threshold range, the process ends.

[0048] The present invention also provides a system for automatically evaluating the safety of strength training movement postures, which is applied to the above-mentioned method for automatically evaluating the safety of strength training movement postures. The system for automatically evaluating the safety of strength training movement postures comprises:

[0049] The first output module is used to receive a video from the user end, extract frame image features, and output action category serial numbers;

[0050] The second output module is used to output the call model vector according to the frame image features and the action category sequence number;

[0051] A calling module, used for calling a corresponding body part granularity level model according to a calling model vector;

[0052] The third output module is used to output the coordinates of key points of human posture according to the granularity level model of body parts;

[0053] A calculation module is used to calculate the movement speed and joint angle according to the coordinates of the key points of the human body posture and the sequence number of the action category;

[0054] A judgment module is used to judge whether the movement speed or joint angle is within a safe threshold range;

[0055] The recognition module is used to give a prompt warning if it recognizes that the movement speed or joint angle is not within the safety threshold range.

[0056] The beneficial effects achieved by the present invention are:

[0057] The present invention provides a method and system for automatically evaluating the safety of strength training action postures, which receives a video transmitted from a user end, extracts frame image features, and outputs an action category serial number; outputs a call model vector based on the frame image features and the action category serial number; calls the corresponding body part granularity level model based on the call model vector; outputs the coordinates of key points of human posture based on the body part granularity level model; calculates the movement speed and joint angle based on the coordinates of key points of human posture and the action category serial number; determines whether the movement speed or joint angle is within a safety threshold range; and gives a prompt warning if it is identified that the movement speed or joint angle is not within the safety threshold range. The method and system for automatically evaluating the safety of strength training action postures provided by the present invention have the following beneficial effects:

[0058] 1. Automatically evaluate the accuracy of movement posture without human observation

[0059] The technology of automatically evaluating the safety of strength training movements can reduce the subjectivity and inaccuracy in human observation, thereby improving the safety and effectiveness of sports training. By capturing movements with a camera and combining it with a posture estimation model, it can automatically detect movement information such as the joint angles and movement speed of the movements in motion, and accurately capture those small but critical posture deviations, such as the position of the knees during squats. This method not only reduces human observation errors caused by fatigue or lack of concentration, but also provides instant feedback to fitness enthusiasts, helping them to adjust their movements in a timely manner and avoid sports injuries.

[0060] 2. Support multi-task calling to improve model selection efficiency

[0061] The decision model can call multiple body part recognition tasks, significantly improving the efficiency of model selection. By integrating posture estimation models of different body parts, taking into account factors such as accuracy, computational cost, and granularity, and designing a flexible task selection mechanism, the decision model can quickly select and call multiple posture estimation tasks or single tasks of different body parts in the posture estimation model pool in parallel. Users can realize posture recognition of multiple body parts without manually selecting a specific model. The task selection mechanism of the decision model not only improves the efficiency of model selection, but also meets the diverse application needs.

[0062] 3. Choose the estimated granularity based on the video quality

[0063] Based on the quality of the input video, the decision model can autonomously select the appropriate estimation granularity. For high-quality videos, the model uses a finer granularity for fine-grained pose estimation to ensure the accuracy and detail richness of the results. For lower-quality videos, the model chooses a coarser granularity to balance the consumption of computing resources and the credibility of the estimation results. This ability to dynamically adjust the granularity enables the model to perform well under a variety of video qualities to provide the best estimation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic flow chart of a method for automatically evaluating the safety of strength training posture movements provided by the present invention;

[0065] Figure 2 The figure is a schematic diagram of calculating the elbow joint angle when the present invention is applied to push-ups. DETAILED DESCRIPTION

[0066] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0067] like Figure 1 and Figure 2 As shown, the first embodiment of the present invention provides a method for automatically evaluating the safety of strength training postures, comprising the following steps:

[0068] Step S100: receiving a video clip transmitted by a user terminal, extracting frame image features, and outputting an action category serial number.

[0069] First, input the video into the action category model. The action category model extracts video features and outputs the action category number. For example, output "1" indicates that the action category is push-ups, output "2" indicates that the action category is plank, and so on.

[0070] Step S200: output a call model vector according to the frame image features and the action category sequence number.

[0071] The frame image features and the action category number output in the previous step are passed as input to the decision model. There are currently 4 key point estimation models for different body parts, namely the hand key point estimation model, the foot key point estimation model, the face key point estimation model, and the torso key point estimation model.

[0072] The video quality will affect the number of key points that can be accurately identified by each body part model. By consulting relevant literature and materials, the recognition granularity of each body part model is divided into several levels according to the size of the granularity level. The lower the level, the coarser the granularity and the fewer the key points that can be accurately identified (level 0 means that the video does not have this body part), as shown in Table 1.

[0073]

[0074]

[0075] Table 1

[0076] As shown in Table 1, each body part model has a data set with different granularity levels, including 3 foot key point estimation models, 4 torso key point estimation models, 5 hand key point estimation models, and 6 face key point estimation models.

[0077] Collect a large number of image datasets with human posture annotations. These datasets contain human body images in different postures and angles. Different body part models correspond to different annotated parts of the dataset.

[0078] Taking the trunk key point estimation model as an example, the four granularity level data sets of the trunk key point estimation model are data sets with 4, 8, 12, and 17 key points annotated on the feet. The most advanced CNN-based top-down human posture estimation algorithm HRNet (High Resolution Network) is used as the backbone network. Each key point estimation model is constructed by replacing different prediction heads to establish a model pool. Taking the trunk key point estimation model with a granularity of 4 as an example, the features extracted by the HRNet backbone network are input into the corresponding prediction head, and the prediction head outputs the corresponding 17 key point coordinates.

[0079] The model is trained using the collected labeled data, and the data sets are divided into training set, validation set, and test set in a ratio of 8:1:1. During the training process, the model parameters are continuously optimized to enable it to accurately predict the location of key points of the human body. The model is evaluated using the validation set to adjust the model parameters and prevent overfitting and underfitting. After training, the model is finally evaluated using the test set to ensure that it performs well on unseen data.

[0080] The same goes for other body part models, and eventually we can get several granularity level models corresponding to each body part model, a total of 18 (3 foot key point estimation models, 4 torso key point estimation models, 5 hand key point estimation models, and 6 face key point estimation models). All the above models are simply spliced ​​and integrated to form a pose estimation model pool.

[0081] The model is vectorized using one-hot encoding, that is, each model is represented by a vector, the length of the vector is equal to the total number of models, and each position in the vector corresponds to a model (the first position represents the foot key point estimation model, the second position represents the torso key point estimation model, the third position represents the hand key point estimation model, and the fourth position represents the face key point estimation model). The number represents the granularity level of the model at the corresponding position, for example:

[0082] (1) Foot key point estimation model - the vector form of granularity level 1 is expressed as [1, 0, 0, 0];

[0083] (2) Trunk key point estimation model - the vector form of granularity level 2 is expressed as [0, 2, 0, 0];

[0084] (3) Hand key point estimation model - the vector form of granularity level 3 is expressed as [0, 0, 3, 0];

[0085] (4) Facial key point estimation model - the vector form of granularity level 4 is expressed as [0, 0, 0, 4];

[0086] (5) The vector form of the hand key point estimation model - granularity level 5 and the face key point estimation model - granularity level 6 is called at the same time and is expressed as [0, 0, 5, 6];

[0087] Follow the above rules and so on.

[0088] It can be seen that each serial number has a corresponding body part key point estimation model to be called. For example, the plank support with serial number "2" needs to call the foot key point estimation model and the torso key point estimation model. After the decision model obtains the above information according to the action category serial number, it determines which level of granularity body part model to call based on the video quality and outputs the final model calling result, which is represented by a vector. For example, if the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are called, the final output vector is [3, 3, 0, 0].

[0089] Step S300: Call the corresponding body part granularity level model according to the called model vector.

[0090] After obtaining the final output vector [3, 3, 0, 0], the vector is passed into the posture estimation model pool, indicating that the foot key point estimation model-granularity level 3 and the torso key point estimation model-granularity level 3 are to be called.

[0091] Step S400: output the coordinates of key points of human posture according to the body part granularity level model.

[0092] According to the granularity level model of body parts, the two-dimensional coordinates of the key points of human posture in each frame of the video can be obtained.

[0093] Step S500: Calculate the motion speed and joint angle according to the coordinates of the key points of the human body posture and the action category sequence number.

[0094] According to the action category number, we can know what action the human body is doing. Each action focuses on different parts. Therefore, to determine whether the action is done accurately, we need to get the joint angle and movement speed of the part of concern. The required joint angle and movement speed can be calculated by the coordinates of the posture key points.

[0095] Step S600: Determine whether the movement speed or joint angle is within a safety threshold range.

[0096] By consulting relevant sports literature and materials, a safety threshold range is set for the joint angle and movement speed that need to be detected for each action. When the joint angle or movement speed is not within the safety threshold range, it is considered that the action will have an injury risk at this time, and a warning will be given.

[0097] Step S700: If it is identified that the movement speed or joint angle is not within the safety threshold range, a prompt warning is given.

[0098] When it is detected that the calculated movement speed or joint angle is not within the pre-judged safety threshold range, a prompt warning is given.

[0099] Further, see Figure 1 and Figure 2 The method for automatically evaluating the safety of strength training postures proposed in this embodiment, step S100 includes:

[0100] Step S110: extract a series of continuous frame images from the video at fixed time intervals.

[0101] Step S120: pre-process the extracted frame image to ensure the consistency and validity of the input model.

[0102] The extracted frame images are preprocessed including resizing, normalization, cropping, etc. to meet the input requirements of the input model.

[0103] Step S130, use the 3D convolution layer to process the fixed frame sequence, use the 3D convolution kernel to slide in the three dimensions of time, width and height, and realize the simultaneous extraction of time and space features; flatten the four-dimensional features processed by the 3D convolution module into a three-dimensional feature representation; build a backbone network based on the dual-stream Transformer architecture, and use two parallel Transformer encoding modules in each layer to extract time and space features respectively, and fuse the two features by adding them to obtain a more comprehensive feature representation; the four-dimensional features include the number of frames, width, height and feature dimension, and the three-dimensional features include time, space and feature dimension.

[0104] The Transformer model architecture is a model proposed by Google in the paper Attention is All you need in 2017. It uses the Self-Attention structure to replace the RNN (Recurrent Neural Network) network structure commonly used in NLP tasks. Compared with the RNN network structure, its biggest advantage is that it can be calculated in parallel.

[0105] Step S140, flatten the fused three-dimensional feature vector into a one-dimensional feature vector, input it into the fully connected layer, the fully connected layer maps the three-dimensional feature vector to the action category space, and the number of output nodes is equal to the number of action categories; through the softmax activation function, the output is converted into a probability distribution; according to the output probability distribution, the category with the highest probability is selected as the prediction result, and the sequence number of the action category with the highest probability is output.

[0106] Preferably, see Figure 1 and Figure 2 The method for automatically evaluating the safety of strength training postures proposed in this embodiment, step S200 includes:

[0107] Step S210: pass the frame image features and the action category serial number output in the previous step as input to the decision model; each action category serial number has a corresponding body part model.

[0108] The frame image features and the action category serial number output in the previous step are passed as input to the decision model. Each serial number has a corresponding body part model to be called. For example, the plank support with serial number "2" requires the foot key point estimation model and the torso key point estimation model to be called.

[0109] Step S220, the decision model calls the body part model of the corresponding granularity level according to the frame image features and the action category number output in the previous step, based on the video quality, and outputs the final model calling result, which is represented by a vector.

[0110] After the decision model obtains the above information according to the action category number, the video quality is used to determine which level of granularity of the body part model to call, and the final model calling result is output. The result is expressed as a vector. For example, if the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are called, the final output vector is [3, 3, 0, 0].

[0111] Further, preferably, see Figure 1 and Figure 2 The method for automatically evaluating the safety of strength training postures proposed in this embodiment includes, before step S100:

[0112] Step S100A: Dataset preparation

[0113] The input data of the decision model is the video and its action category number, and the output is the model pool selection result expressed in vector form. For example, if the hand model-granularity level 3 is called, the output result is [0, 0, 3, 0]; if the hand model-granularity level 5 and the face model-granularity level 6 are called at the same time, the output result is [0, 0, 5, 6]. The input data and the corresponding output vector are used as the data set for subsequent training and verification of the decision model. The data set is prepared as follows.

[0114] According to the action category number, the corresponding body part model is determined. The extracted video feature data is respectively passed to the models of the corresponding body parts at each granularity level in the model pool. Each model outputs the total number of key point estimates N and the average error ME. The average error ME is calculated by the following formula:

[0115]

[0116] In formula (1), ME represents the mean error, and the key point positions output by the model are (x1, y1), (x2, y2)...(x n ,y n ); The key point position of the original image is correctly marked as (x1 ′ , y1 ′ )、(x2 ′ , y2 ′ )……(x n ′ ,y n ′ ), N represents the total number of key point estimates. The unit of mean error ME is mm.

[0117] The total number of key points in posture estimation, N, reflects the granularity of the posture estimation model. In the same body part model, the larger the N, the higher the granularity level, that is, the finer the granularity, but at the same time, the more computing resources are consumed. The mean error, ME, reflects the accuracy of the model's estimation of the position of the posture key points. The lower the ME, the more accurate the position estimation.

[0118] According to the research on the current human posture estimation model, the average error range allowed for models with different granularity levels is also different. After continuous reference, calculation and verification, the average error range allowed for different granularity levels of each body part model is obtained, such as

[0119] As shown in Table 2.

[0120]

[0121] Table 2

[0122] The following is an explanation of Table 2:

[0123] (1) For each body part model, when there are multiple granularity level models that meet the corresponding set average error range, the model with the highest granularity level is selected by default;

[0124] (2) For each body part model, when the granularity level model exceeds the set average error range, the granularity level is defined as 0, indicating that the granularity level model is not selected;

[0125] According to the rules in Table 2, the data set (i.e., input data and its corresponding output vector) is collected. The steps are as follows.

[0126] According to the action category number, each video is input into the body part model to be selected by the model pool, and each body part model outputs a set of N and ME at different granularity levels. By querying Table 2, the corresponding vector form can be output as shown in Table 3).

[0127] For example:

[0128]

[0129] Table 3

[0130] The final result output in Table 3 above is: [3, 1, 4, 0], so the output vector corresponding to the video is [3, 1, 4, 0].

[0131] After the above data preparation steps and operations, we can finally get several videos and their action category numbers, as well as the corresponding output vectors, as the data set of the model. The data set is divided into training set, validation set and test set in a ratio of 8:1:1.

[0132] Step S100B: Building a decision model

[0133] 1. Model network structure

[0134] The model input is the action category number and the video frame sequence, and the output is the model selection vector. First, the image features in the video sequence are extracted, and the network structure of the action category model is referred to, using the 3D-CNN structure and the two-stream Transformer. However, after the frame sequence features are obtained through 3D-CNN processing, the action category number needs to be spliced ​​into the feature vector, and then the spliced ​​feature vector is input into the two-stream feature vector for processing. Finally, the feature is mapped into a 1×4 vector through the fully connected layer and output as the output result of the model.

[0135] 2. Model training

[0136] Before starting to train the model, define the loss function and optimizer. For regression tasks, you can choose mean square error (MSE) as the loss function and use the Adam optimizer to speed up the training process. Input the training set data into the model for training, optimize the model parameters through multiple iterations, and gradually improve the performance of the model.

[0137] 3. Model evaluation and tuning

[0138] During the training process, the validation set is used to evaluate the performance of the model. Based on the feedback from the validation set, hyperparameter tuning is performed, such as adjusting parameters such as learning rate and batch size, to ensure that the model can learn effectively without overfitting. After the model training is completed, the performance of the final model is further evaluated on the test set to ensure that the model can perform well on unseen data and avoid overfitting and underfitting problems.

[0139] 4. Model Deployment

[0140] When the model training and evaluation are completed, the model weights and structure are saved for subsequent use. The trained model is deployed to the production environment so that it can reason or predict new data, thereby efficiently and accurately calling the body part model in the posture estimation model pool.

[0141] Further, see Figure 1 and Figure 2 In the method for automatically evaluating the safety of strength training postures proposed in this embodiment, the joint angle includes the elbow joint angle. In step S500, the elbow joint angle is calculated by the following formula:

[0142]

[0143] In formula (2), ∠ABC represents the elbow joint angle, represents the vector from the elbow to the shoulder, represents the vector from elbow to wrist;

[0144]

[0145] In formulas (3) and (4), the coordinates of the three points ∠ABC are A(x1, y1), B(x2, y2), and C(x3, y3).

[0146] Preferably, see Figure 1 and Figure 2 In the method for automatically evaluating the safety of strength training postures proposed in this embodiment, in step S500, each action focuses on a different part, so the part to be calculated for the movement speed is also different. Figure 2 As shown, taking push-ups as an example, the movement speed focuses on the movement speed of points A, B, and C on the shoulders.

[0147] Taking the calculation of the moving speed of point A as an example, if the video is n frames per second, the interval time t between each frame is The coordinates of the key point A in the previous frame (x1, y1) to the key point A in the current frame ′ Coordinates(x1 ′ , y1 ′ ) is The movement speed is calculated by the following formula:

[0148]

[0149] In formula (5), v represents the motion speed, and s represents the distance from the coordinates (x1, y1) of the key point A in the previous frame to the key point A in the current frame. ′ Coordinates(x1 ′ , y1 ′ ), t represents the interval time between each frame of image, and n represents the number of image frames.

[0150] Further, see Figure 1 and Figure 2 The method for automatically evaluating the safety of strength training postures proposed in this embodiment further includes, after step S600:

[0151] Step S800: If it is identified that the movement speed and the joint angle are within the safety threshold range, then the process ends.

[0152] When it is recognized that the calculated movement speed and joint angle are within the pre-qualified safety threshold range, the entire process ends.

[0153] The present embodiment also provides a system for automatically evaluating the safety of strength training movement postures, which is applied to the above-mentioned method for automatically evaluating the safety of strength training movement postures. The system for automatically evaluating the safety of strength training movement postures includes a first output module, a second output module, a calling module, a third output module, a calculation module, a judgment module and an identification module, wherein the first output module is used to receive a video transmitted by a user end, extract frame image features, and output an action category serial number; the second output module is used to output a calling model vector according to the frame image features and the action category serial number; the calling module is used to call the corresponding body part granularity level model according to the calling model vector; the third output module is used to output the coordinates of the key points of the human body posture according to the body part granularity level model; the calculation module is used to calculate the movement speed and joint angle according to the coordinates of the key points of the human body posture and the action category serial number; the judgment module is used to judge whether the movement speed or the joint angle is within the safety threshold range; the identification module is used to give a prompt warning if it is recognized that the movement speed or the joint angle is not within the safety threshold range.

[0154] The method and system for automatically evaluating the safety of strength training posture movements provided in this embodiment receive a video transmitted by the user end, extract the frame image features, and output the action category number; output the call model vector based on the frame image features and the action category number; call the corresponding body part granularity level model based on the call model vector; output the coordinates of the key points of the human body posture based on the body part granularity level model; calculate the movement speed and joint angle based on the coordinates of the key points of the human body posture and the action category number; determine whether the movement speed or joint angle is within the safety threshold range; if it is identified that the movement speed or joint angle is not within the safety threshold range, a prompt warning is given. The method and system for automatically evaluating the safety of strength training posture movements provided in this embodiment have the following beneficial effects:

[0155] 1. Automatically evaluate the accuracy of movement posture without human observation

[0156] The technology of automatically evaluating the safety of strength training movements can reduce the subjectivity and inaccuracy in human observation, thereby improving the safety and effectiveness of sports training. By capturing movements with a camera and combining it with a posture estimation model, it can automatically detect movement information such as the joint angles and movement speed of the movements in motion, and accurately capture those small but critical posture deviations, such as the position of the knees during squats. This method not only reduces human observation errors caused by fatigue or lack of concentration, but also provides instant feedback to fitness enthusiasts, helping them to adjust their movements in a timely manner and avoid sports injuries.

[0157] 2. Support multi-task calling to improve model selection efficiency

[0158] The decision model can call multiple body part recognition tasks, significantly improving the efficiency of model selection. By integrating posture estimation models of different body parts, taking into account factors such as accuracy, computational cost, and granularity, and designing a flexible task selection mechanism, the decision model can quickly select and call multiple posture estimation tasks or single tasks of different body parts in the posture estimation model pool in parallel. Users can realize posture recognition of multiple body parts without manually selecting a specific model. The task selection mechanism of the decision model not only improves the efficiency of model selection, but also meets the diverse application needs.

[0159] 3. Choose the estimated granularity based on the video quality

[0160] Based on the quality of the input video, the decision model can autonomously select the appropriate estimation granularity. For high-quality videos, the model uses a finer granularity for fine-grained pose estimation to ensure the accuracy and detail richness of the results. For lower-quality videos, the model chooses a coarser granularity to balance the consumption of computing resources and the credibility of the estimation results. This ability to dynamically adjust the granularity enables the model to perform well under a variety of video qualities to provide the best estimation results.

[0161] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatically evaluating the safety of strength training postures, characterized in that: The following steps are involved: Receive a video from the user, extract frame image features, and output action category numbers; Outputting a calling model vector according to the frame image features and the action category sequence number; According to the called model vector, calling the corresponding body part granularity level model; Outputting the coordinates of key points of human posture according to the body part granularity level model; Calculate the movement speed and joint angle according to the coordinates of the key points of the human body posture and the sequence number of the action category; Determining whether the movement speed or the joint angle is within a safety threshold range; If it is identified that the movement speed or the joint angle is not within the safety threshold range, a prompt warning is given.

2. The method for automatically evaluating the safety of strength training postures as claimed in claim 1, characterized in that: The steps of receiving a video transmitted by a user terminal, extracting frame image features, and outputting action category serial numbers include: Extracting a series of continuous frame images from the video at fixed time intervals; Preprocessing the extracted frame image to ensure the consistency and validity of the input model; Use 3D convolution layer to process fixed frame sequence, use 3D convolution kernel to slide in three dimensions of time, width and height, so as to extract time and space features at the same time; flatten the four-dimensional features processed by 3D convolution module into three-dimensional feature representation; build backbone network based on dual-stream Transformer architecture, use two parallel Transformer encoding modules in each layer to extract time series and space features respectively, and fuse the two features by adding them to obtain more comprehensive feature representation; the four-dimensional features include frame number, width, height and feature dimension, and the three-dimensional features include time series, space and feature dimension; The fused three-dimensional feature vector is flattened into a one-dimensional feature vector and input into the fully connected layer. The fully connected layer maps the three-dimensional feature vector to the action category space, and the number of output nodes is equal to the number of action categories. The output is converted into a probability distribution through the softmax activation function. According to the output probability distribution, the category with the highest probability is selected as the prediction result, and the sequence number of the action category with the highest probability is output.

3. The method for automatically evaluating the safety of strength training postures as claimed in claim 2, characterized in that: The step of outputting a call model vector according to the frame image features and the action category sequence number comprises: The frame image features and the action category serial number output in the previous step are both passed as input to the decision model; the action category serial number has a corresponding body part model to be called; The decision model calls the body part model of the corresponding granularity level according to the frame image features and the action category number output in the previous step, based on the video quality, and outputs the final model calling result, which is represented by a vector.

4. The method for automatically evaluating the safety of strength training postures as claimed in claim 3, characterized in that: Before the step of receiving a video from the user end, extracting frame image features, and outputting the action category serial number, a data set is prepared, and the corresponding body part model is determined to be called according to the action category serial number; the extracted video feature data is respectively transmitted to the models of each granularity level of the corresponding body part in the model pool, and each model outputs the total number N of key point estimates and the average error ME, and the average error ME is calculated by the following formula: Among them, ME represents the mean error, and the key point positions output by the model are (x1, y1), (x2, y2)...(x n ,y n ); The key point position of the original image is correctly marked as (x1 ′ ,y1 ′ )、(x2 ′ , y2 ′ )……(x n ′ ,y n ′ ), where N represents the total number of key point estimates.

5. The method for automatically evaluating the safety of strength training postures as claimed in claim 3, characterized in that: The decision model calls the body part model of the corresponding granularity level according to the frame image features and the action category number output in the previous step, and determines the video quality to output the final model calling result. In the step where the result is represented by a vector, if the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are called, the final output vector [3, 3, 0, 0] is obtained.

6. The method for automatically evaluating the safety of strength training postures as claimed in claim 5, characterized in that: The step of outputting the coordinates of key points of human posture according to the body part granularity level model comprises: After obtaining the final output vector [3, 3, 0, 0], the output vector [3, 3, 0, 0] is passed into the posture estimation model pool, indicating that the foot key point estimation model - granularity level 3 and the torso key point estimation model - granularity level 3 are to be called to obtain the two-dimensional coordinates of the human body posture key points in each frame of the video.

7. The method for automatically evaluating the safety of strength training postures as claimed in claim 1, characterized in that: The joint angle includes the elbow joint angle. In the step of calculating the motion speed and the joint angle according to the human body posture key point coordinates and the action category sequence number, the elbow joint angle is calculated by the following formula: Among them, ∠ABC represents the elbow joint angle, represents the vector from the elbow to the shoulder, represents the vector from elbow to wrist; Among them, the coordinates of the three points ∠ABC are A(x1, y1), B(x2, y2), and C(x3, y3).

8. The method for automatically evaluating the safety of strength training postures as claimed in claim 7, characterized in that: In the step of calculating the motion speed and joint angle according to the coordinates of the key points of the human body posture and the action category serial number, the motion speed is calculated by the following formula: Among them, v represents the movement speed, s represents the coordinates (x1, y1) of the key point A in the previous frame to the key point A in the current frame ′ Coordinates(x1 ′ , y1 ′ ), t represents the interval time between each frame of image, and n represents the number of image frames.

9. The method for automatically evaluating the safety of strength training postures as claimed in claim 8, characterized in that: After the step of determining whether the movement speed or the joint angle is within the safety threshold, the following steps are further included: If it is recognized that the movement speed and the joint angle are within the safety threshold range, the process ends.

10. A system for automatically evaluating the safety of strength training posture movements, applied to the method for automatically evaluating the safety of strength training posture movements as claimed in any one of claims 1 to 9, characterized in that: The system for automatically evaluating the safety of strength training postures comprises: The first output module is used to receive a video from the user end, extract frame image features, and output action category serial numbers; A second output module, used for outputting a call model vector according to the frame image feature and the action category sequence number; A calling module, used for calling a corresponding body part granularity level model according to the calling model vector; A third output module, used for outputting the coordinates of key points of human posture according to the body part granularity level model; A calculation module, used for calculating the movement speed and joint angle according to the coordinates of the key points of the human body posture and the sequence number of the action category; A judging module, used for judging whether the movement speed or the joint angle is within a safety threshold range; The recognition module is used to give a prompt warning if it is recognized that the movement speed or the joint angle is not within the safety threshold range.

Images