Mobile terminal video-based hard pull gesture detection method and system

Abstract

The application provides a hard pull gesture detection method and system based on mobile terminal video, which comprises the following steps: a mobile device is used to shoot a hard pull action video of a user, and the hard pull action video is input into a Mediapipe gesture estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video; Gaussian filtering is applied to the joint coordinate sequence in the time dimension for smoothing processing, and biomechanical angle features representing the standardness of the hard pull gesture are calculated according to the joint coordinates after the smoothing processing; frame-by-frame two-stage sequence determination is performed on the biomechanical angle features to evaluate whether the hard pull action is standard, the two-stage sequence determination comprises first-stage determination for the starting gesture and second-stage determination for the pull-up power process, and only when both stages are determined to pass, the hard pull action is a standard action. The application can improve the recognition accuracy of the hard pull action.

Description

Technical Field

[0001] This invention relates to the field of posture detection technology, and in particular to a method and system for detecting deadlift posture based on mobile video. Background Technology

[0002] Deadlifts, a popular fitness exercise, effectively strengthen the back and hips and promote overall muscle development. However, prolonged use of incorrect deadlift form can lead to problems such as herniated discs. Because of the complexity of the deadlift and the high cost of hiring a personal trainer, individuals often rely on subjective judgment to assess their form. Therefore, utilizing computer vision technology to automatically detect deadlift form is crucial for ensuring proper execution, reducing the risk of long-term injuries due to incorrect posture, and improving mastery of the exercise.

[0003] With advancements in information technology and electronic sensors, action recognition technology has developed rapidly, primarily falling into two categories: wearable device-based and computer vision-based methods. However, the former is prone to causing discomfort when worn, affecting data accuracy, and the devices are easily damaged and data is easily lost; the latter relies on large datasets and high-performance equipment, resulting in higher application costs. Furthermore, as a complex, instantaneous movement, the deadlift often suffers from subtle motion differences when relying solely on depth camera sensors, leading to discrepancies between skeletal data and actual movement. Summary of the Invention

[0004] The purpose of this invention is to provide a deadlift posture detection method and system based on mobile video, which aims to reduce the negative impact of insufficient camera resolution, illumination changes, tracking errors and other factors on human skeleton position data, improve the system's recognition accuracy, and achieve more efficient and accurate recognition of non-standard deadlifts.

[0005] In a first aspect, the present invention provides a method for detecting hardp pose based on mobile video, the method comprising:

[0006] The video of a user's deadlift motion is captured by a mobile device and input into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number.

[0007] Gaussian filtering is applied to the joint coordinate sequence in the time dimension for smoothing, and biomechanical angular features representing the standard of deadlift posture are calculated based on the smoothed joint coordinates. The biomechanical angular features include at least the trunk angle, hip joint angle and knee joint angle.

[0008] The biomechanical angular features are subjected to frame-by-frame two-stage sequence determination to evaluate whether the deadlift action is standard. The two-stage sequence determination includes a first stage determination for the starting posture and a second stage determination for the pulling force process. The deadlift action is considered standard only when both stages are determined to be successful.

[0009] In some embodiments, the torso angle is formed by the line connecting the shoulder joint and the hip joint and the gravity vector perpendicular to the ground;

[0010] The hip joint angle is formed by three points: the shoulder joint, the hip joint, and the knee joint.

[0011] The knee joint angle is formed by three points: the hip joint, the knee joint, and the ankle joint.

[0012] In some embodiments, the joint coordinate sequence is smoothed according to the following formula:

[0013] ;

[0014] in, The value of the Gaussian function. denoted as standard deviation, and x as the time offset of the current data point relative to the filter center.

[0015] In some embodiments, the steps of the first stage determination include:

[0016] Extract a preset proportion of frames from the deadlift video sequence as the initial analysis interval;

[0017] Within the analysis interval, frame by frame, it is determined whether the torso angle is within the first standard threshold range and whether the knee joint angle is within the second standard threshold range.

[0018] If the trunk angle is within the first standard threshold range and the knee joint angle is within the second standard threshold range, then the first stage is considered passed.

[0019] In some embodiments, the steps of the second stage determination include:

[0020] Set standard extension range The basic extension characteristics are calculated using the following formula:

[0021] ;

[0022] in, For knee joint angle, The basic stretching feature for frame t;

[0023] The trajectory vertical verification feature is calculated using the following formula:

[0024] ;

[0025] in, The trajectory vertical verification feature for frame t. Let be the horizontal projection distance between the wrist joint coordinate vector and the knee joint coordinate vector in Euclidean space. The maximum deviation threshold;

[0026] The force coordination verification characteristics are calculated using the following formula:

[0027] ;

[0028] ;

[0029] ;

[0030] in, For standard coordinated intervals, For the force coordination verification features of frame t, For motion coordination rate, Let be the instantaneous angular velocity of the hip joint in frame t. Let be the instantaneous angular velocity of the knee joint in frame t. It is a non-zero constant. The sampling interval;

[0031] In the entire deadlift video sequence, all frames where the hip angle is within the third standard threshold range are selected as the key force exertion frame set, and the standard frame ratio of the pull-up phase is calculated according to the following formula:

[0032] ;

[0033] in, The motion quality status of frame t. And operation, This is a set of key force exertion process frames;

[0034] If the proportion of standard frames exceeds the preset second threshold, the second stage is considered to have passed.

[0035] In some embodiments, the method further includes:

[0036] When performing angle threshold judgment at any stage, if the judgment result of the biomechanical angle feature calculated based on the joint point of one side of the human skeleton is not passed, the system will automatically switch to the angle feature calculated based on the corresponding joint point of the other side of the human skeleton, and re-judge according to the same threshold and pass ratio rules.

[0037] Secondly, the present invention provides a deadlift posture detection system based on mobile video, the system comprising:

[0038] The video capture module is used to capture a video of a user's deadlift action via a mobile device and input the deadlift action video into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number.

[0039] The feature extraction module is used to apply Gaussian filtering to smooth the joint coordinate sequence in the time dimension, and calculate biomechanical angle features representing the standard of deadlift posture based on the smoothed joint coordinates. The biomechanical angle features include at least the trunk angle, hip joint angle and knee joint angle.

[0040] The detection execution module is used to perform frame-by-frame two-stage sequence determination on the biomechanical angle features to evaluate whether the deadlift action is standard. The two-stage sequence determination includes a first-stage determination for the starting posture and a second-stage determination for the pulling force process. The deadlift action is considered standard only when both stages are determined to be passed.

[0041] Thirdly, the present invention provides a storage medium that stores one or more programs, which, when executed by a processor, implement the above-described hardlift posture detection method based on mobile video.

[0042] Fourthly, the present invention provides an electronic device, the electronic device comprising a memory and a processor, wherein:

[0043] The memory is used to store computer programs;

[0044] When the processor executes the computer program stored in the memory, it implements the above-described method for detecting deadlift posture based on mobile video.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] This invention first uses Mediapipe to capture the coordinates of the human skeleton joints in deadlift videos in real time, and then uses Gaussian filtering to correct abnormal data caused by hardware errors. Then, guided by prior knowledge, the deadlift behavior is quantified into the torso angle, hip joint angle and knee angle frame by frame. Finally, the quantized features and thresholds are compared frame by frame to determine whether the movement is standard. Attached Figure Description

[0047] Figure 1 This is a flowchart of a hard-pull posture detection method based on mobile video according to an embodiment of the present invention;

[0048] Figure 2 This is a comparison of the angles before and after filtering;

[0049] Figure 3 A schematic diagram of the human skeletal structure;

[0050] Figure 4 A diagram illustrating the angle processing time for each frame;

[0051] Figure 5 Use before-and-after comparison images for filtering;

[0052] Figure 6 A graph showing the data results;

[0053] Figure 7 This is a schematic diagram of the structure of a hardlift posture detection system based on mobile video according to an embodiment of the present invention.

[0054] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but does not exclude other elements or objects.

[0056] like Figure 1 As shown, an embodiment of the present invention proposes a method for detecting hardp pose based on mobile video. The method includes steps S101 to S103, wherein:

[0057] Step S101: Capture a video of the user's deadlift motion using a mobile device, and input the deadlift motion video into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number.

[0058] It should be noted that during the actual filming process, the entire process of the user performing a deadlift is recorded from a fixed position using the mobile device's camera. A frame rate of at least 30fps is recommended to ensure the continuity of the movement. To simulate real-world application scenarios, shooting angles can cover different directions such as front, left front, and right front.

[0059] Furthermore, in some embodiments, the positional information of human joints in each frame is obtained from Mediapipe for the shooting device. Mediapipe is an open-source framework developed by Google for developing machine learning-based computer vision applications, specifically designed for processing real-time data streams. Mediapipe supports a variety of vision tasks, including object detection, pose estimation, face detection, and hand tracking. The reasons for choosing Mediapipe instead of other frameworks (such as OpenPose and PoseNet) in this embodiment are: (1) Mediapipe is specifically designed for real-time data stream processing, meeting the actual needs of frame-by-frame judgment in this study; (2) Mediapipe is mainly used for pose detection under complex poses and various lighting conditions, which is highly compatible with the indoor detection environment of this embodiment; (3) Mediapipe supports multiple platforms and can be used in various environments such as mobile devices. This is highly consistent with the goal of this embodiment to achieve widespread application without adding extra hardware investment and by utilizing existing equipment.

[0060] Step S102: Apply Gaussian filtering to smooth the joint coordinate sequence in the time dimension, and calculate the biomechanical angle features representing the standard of deadlift posture based on the smoothed joint coordinates. The biomechanical angle features include at least the trunk angle, hip joint angle and knee joint angle.

[0061] It should be noted that during the hardening data acquisition process, factors such as insufficient camera resolution, lighting variations, and tracking errors inevitably negatively impact the human skeleton position data. To reduce noise in the base data and improve the accuracy of subsequent angle calculations, Gaussian filtering is used to smooth the collected hardening data. The weights are defined by a two-dimensional Gaussian function:

[0062] ;

[0063] in, The value of the Gaussian function. denoted as standard deviation, and x as the time offset of the current data point relative to the filter center.

[0064] The use of Gaussian filtering is primarily based on two aspects: First, Gaussian filtering achieves a smoothing effect by weighted averaging of each data point and its neighborhood in the time series. The weight of a data point is determined by its distance from the center point; the closer to the center point, the greater the weight, and vice versa. Compared to other filtering methods, Gaussian filtering, while maintaining data smoothness, better preserves the motion trend and temporal structure in the trajectory of key points, reducing distortion of true dynamic features. Second, Gaussian filtering is highly convenient to implement; in a Python environment, it can be implemented with just one line of code. Figure 2As shown: Comparing the curves before and after Gaussian filtering, it is clear that the curve after Gaussian filtering not only retains the original motion trend, but also improves the smoothness of the curve.

[0065] It should also be noted that, in some embodiments, one of the challenges of this embodiment lies in how to quickly, efficiently, and accurately acquire the human joint coordinate information for each frame. Although depth cameras such as Kinect provide convenient channels for acquiring joint coordinates, key positions are prone to drift in close-range situations, not from the perspective of multiple small microcomputer systems, and not facing the camera directly. The mainstream solution in the industry is to use multiple cameras for position correction. However, this approach inevitably requires the addition of expensive monitoring equipment and also places high demands on the position of the monitored person. Mediapipe, on the other hand, can accurately track and extract 33 joint points of the human skeleton in poorly lit indoor environments, such as... Figure 3 As shown, the correspondence between joint numbers and joint names is as follows: 0. Nose, 1. Inner side of right eye, 2. Right eye, 3. Outer side of right eye, 4. Inner side of left eye, 5. Left eye, 6. Outer side of left eye, 7. Right ear, 8. Left side, 9. Right mouth, 10. Left mouth, 11. Right shoulder, 12. Left shoulder, 13. Right elbow, 14. Left elbow, 15. Right wrist, 16. Left wrist, 17. Right little finger joint, 18. Left little finger joint, 19. Right index finger joint, 20. Left index finger joint, 21. Right thumb joint, 22. Left thumb joint, 23. Right hip, 24. Left hip, 25. Right knee, 26. Left knee, 27. Right ankle, 28. Left ankle, 29. Right heel, 30. Left heel, 31. Right index toe, 32. Left index toe. Specifically, the trunk angle is defined as the angle between the line connecting the shoulder joint and the hip joint and the direction perpendicular to gravity. The physical significance of the hip angle lies in assessing back straightness, obtained by calculating the angle between the hip and shoulder and the vertical downward direction. The hip joint angle is defined as the angle formed by the shoulder, hip, and knee joints. Its physical significance lies in assessing hip flexion and force generation patterns, obtained by calculating the angles between the hip and shoulder and between the hip and knee. The knee joint angle is defined as the angle formed by the hip, knee, and ankle joints. Its physical significance lies in assessing the degree of knee flexion, obtained by calculating the angles between the knee and hip and between the knee and ankle.

[0066] Step S103: Perform frame-by-frame dual-stage sequence determination on the biomechanical angle features to evaluate whether the deadlift action is standard. The dual-stage sequence determination includes a first-stage determination for the starting posture and a second-stage determination for the pulling force process. The deadlift action is considered standard only when both stages are determined to be passed.

[0067] It should be noted that the first stage is the starting posture assessment. This stage focuses on the initial preparation posture for the deadlift, ensuring that the movement begins with a standard posture to avoid the risk of injury from the outset. The second stage is the pull-up phase assessment. This stage focuses on the core power generation process of the deadlift, ensuring that the movement does not become deformed under load.

[0068] In some embodiments, the first stage determination process is as follows: extract a preset proportion of frames from the deadlift video sequence as the initial stage analysis interval; within the analysis interval, determine frame by frame whether the torso angle is within a first standard threshold range and whether the knee joint angle is within a second standard threshold range; if the torso angle is within the first standard threshold range and the knee joint angle is within the second standard threshold range, then the frame is determined to meet the first stage, otherwise it does not meet the first stage; if the proportion of frames meeting the first stage exceeds the first threshold, then the first stage is determined to pass, otherwise it does not pass the first stage.

[0069] For example, the initial posture of the deadlift is determined by taking the first 20% of the frames. This is because people who are new to the deadlift often start with the wrong posture. Using the wrong posture to complete the subsequent pull-up will bring a great risk of injury.

[0070] Furthermore, the reason for using the left node for judgment even when the right node is deemed non-standard is that when the camera is filming a deadlift from the left, the joints closer to the camera—namely, the left shoulder, left hip, left knee, left ankle, and lumbar spine joints—are identified better. These joints more closely resemble the joints in a real deadlift. The joints identified from the right side may deviate from the actual human joints, potentially leading to incorrect judgments. The collected video data includes shooting angles from both the left and right sides, thus more closely reflecting real-world usage scenarios. After the first stage of judgment is completed, the second stage of judgment begins.

[0071] The second-stage determination process is as follows: Establish a standard extension range. The basic extension characteristics are calculated using the following formula:

[0072] ;

[0073] in, For knee joint angle, The basic stretching feature for frame t;

[0074] The trajectory vertical verification feature is calculated using the following formula:

[0075] ;

[0076] in, The trajectory vertical verification feature for frame t. This represents the horizontal projection distance between the wrist joint coordinate vector and the knee joint coordinate vector in Euclidean space, for example, between the left wrist joint and the left knee, or between the right wrist joint and the right knee. The maximum deviation threshold;

[0077] The force coordination verification characteristics are calculated using the following formula:

[0078] ;

[0079] ;

[0080] ;

[0081] in, For standard coordinated intervals, For the force coordination verification features of frame t, For motion coordination rate, Let be the instantaneous angular velocity of the hip joint in frame t. Let be the instantaneous angular velocity of the knee joint in frame t. It is a non-zero constant. The sampling interval;

[0082] In the entire deadlift video sequence, all frames where the hip angle is within the third standard threshold range are selected as the key force exertion frame set, and the standard frame ratio of the pull-up phase is calculated according to the following formula:

[0083] ;

[0084] in, The motion quality status of frame t. And operation, This is a set of key force exertion process frames;

[0085] If the proportion of standard frames exceeds a preset second threshold, the second stage is considered passed. If it does not exceed the threshold, the second stage is considered failed. If one side fails, the process switches to the other side for evaluation.

[0086] In summary, traditional deadlift detection relies primarily on subjective observation, failing to quantify the biomechanical characteristics of multi-joint coordinated force exertion and making it difficult to provide accurate early warnings before injuries result from incorrect movements. To address this, this embodiment constructs a three-stage algorithm detection system: First, by defining knee joint angle ranges and state functions, an objective identification standard for the deadlift force exertion phase is established, resolving the benchmark issue for movement phase determination. Second, by calculating the hip-knee angular velocity ratio in real time, the technical requirement of hip-dominant force exertion is transformed into a quantifiable dynamic indicator. This algorithm can keenly identify error patterns such as excessive knee extension or insufficient hip force exertion, issuing warnings before the lumbar spine experiences excessive shear force. Finally, by setting a symmetrical tolerance range, real-time error-tolerant judgment is made on the coordination of bilateral joint movements, promptly detecting asymmetric risks such as center of gravity shift. This transforms the complex assessment of movement standardization into a multi-dimensional parameter system that can be calculated in real time, effectively replacing experience-based judgment. Precise intervention based on data anomalies is achieved at the initial stage of error occurrence, truly realizing a fundamental shift from experience-based reliance to data-driven approaches, and from post-event correction to pre-event prevention, providing reliable algorithmic assurance for sports safety.

[0087] The third standard threshold range is 80-110 degrees. The reason for using frames with a hip joint angle of 80-110 degrees for judgment is that this angle range represents the process of deadlifting. During the lift, excessive weight may cause deformation of the movement quality, increasing the risk of injury. Adding judgment to the lift process will further ensure the standard of the movement and reduce the risk of injury.

[0088] For example, the first 20% of frames of the entire deadlift sequence are taken as the analysis interval for the initial stage. Within this interval, each frame is checked to see if the angles are within their corresponding standard threshold ranges: the torso angle should be within 55.5 ± 9.8, and the knee angle should be within 120.2 ± 9.5. If at least 70% of the frames within the analysis interval simultaneously meet the above angle threshold requirements, then the first stage is considered passed.

[0089] Furthermore, in some embodiments, when performing angle threshold judgment at any stage, if the judgment result of the biomechanical angle feature calculated based on the joint point of one side of the human skeleton is not passed, the system automatically switches to the angle feature calculated based on the corresponding joint point of the other side of the human skeleton, and re-judges according to the same threshold and pass ratio rules.

[0090] Test Example 1

[0091] This test case is based on the following hardware and software: a desktop computer with an AMD Ryzen 7 3700x CPU, an NVIDIA GeForce GTX 1080ti 11GB graphics card, 64GB of RAM, and Windows 10 operating system. The experimental platform is PyCharm 2024, and the development environment is Python 3.10. The camera used for filming was a Redmi K70.

[0092] In this experiment, the test data included five different types of deadlift postures. These included 22 standard deadlift postures and 48 non-standard deadlift postures of four different types. There were 48 positive samples and 22 negative samples. In this experiment, the distance between the phone and the ground was 0.8 meters; the distance between the person and the phone was 1 to 2 meters; the participants included 9 men and 5 women, aged 22 to 26; their height ranged from 1.65 meters to 1.80 meters; and their weight ranged from 50 kg to 75 kg.

[0093] (a) Validation of effectiveness before filtering

[0094] Figure 4 The time consumption of each frame during the deadlift was recorded (a test video was randomly selected from the deadlift videos I collected, with one frame selected every five frames). The experimental results show that this method can quickly and effectively detect non-standard deadlift postures and distinguish between standard and non-standard deadlift postures. The results are shown in Table 1:

[0095] Table 1. Experimental results for 5 different deadlift postures before filtering.

[0096]

[0097] (ii) Validation of effectiveness before filtering

[0098] To demonstrate the effectiveness of Gaussian filtering, the number of correct detections after filtering is shown in Table 2.

[0099] Table 2. Experimental results for five different deadlift postures after filtering.

[0100]

[0101] In the standard posture detection in Table 2, before using Gaussian filtering, the number of correctly identified standard hard pulls out of the first 22 was 16, with an accuracy rate of 72.72%. After using Gaussian filtering in Table 2, the number of correctly identified hard pulls increased by 3, and the accuracy rate improved to 86.36%. Figure 5 As shown.

[0102] Table 2 details the experimental results. According to Table 2, this method is very accurate in identifying non-standard deadlift postures, but there is some error in identifying standard postures. Examining the human deadlift skeleton image identified by Mediapipe reveals that its identification results still have errors compared to the actual human skeleton, causing the calculated angles to be outside the standard range. However, ultimately... Figure 6 The accuracy rate strongly demonstrates Mediapipe's effectiveness and robustness in recognizing human movements.

[0103] In the comparative experiments corresponding to Table 3, the dataset was divided into a training set of 80% (54 samples) and a test set of 20% (16 samples). This method used all samples (70 samples) for testing. Table 3 shows a comparison of the FPS and accuracy of this method with eight classic deep learning techniques. The experimental results show that this method has the highest FPS and recognition accuracy among all compared methods, not only quickly identifying non-standard deadlift postures but also accurately distinguishing between standard and non-standard deadlift postures.

[0104] Table 3 Comparison of various deep learning methods with this method

[0105]

[0106] In summary, based on the aforementioned deadlift posture detection method using mobile video, the coordinates of the human skeleton joints in the deadlift video are captured in real time using Mediapipe, and Gaussian filtering is used to correct abnormal data caused by hardware errors. Then, guided by prior knowledge, the deadlift behavior is quantified into the torso angle, hip joint angle, and knee angle frame by frame. Finally, the quantified features and thresholds are compared frame by frame to determine whether the movement is standard. This reduces the negative impact of insufficient camera resolution, illumination changes, tracking errors, and other factors on the human skeleton position data, improves the system's recognition accuracy, and achieves more efficient and accurate recognition of non-standard deadlifts.

[0107] like Figure 7 As shown, one embodiment of the present invention proposes a deadlift posture detection system based on mobile video, the system comprising:

[0108] The video acquisition module 100 is used to capture a video of a user's deadlift action via a mobile device and input the deadlift action video into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number.

[0109] The feature extraction module 200 is used to apply Gaussian filtering to smooth the joint coordinate sequence in the time dimension, and calculate biomechanical angle features representing the standard of deadlift posture based on the smoothed joint coordinates. The biomechanical angle features include at least the trunk angle, hip joint angle and knee joint angle.

[0110] The detection execution module 300 is used to perform frame-by-frame dual-stage sequence determination on the biomechanical angle features to evaluate whether the deadlift action is standard. The dual-stage sequence determination includes a first-stage determination for the starting posture and a second-stage determination for the pulling force process. The deadlift action is considered standard only when both stages are determined to be passed.

[0111] In another aspect, the present invention also proposes a storage medium on which one or more programs are stored, which, when executed by a processor, implement the above-described method for detecting hardlift postures based on mobile video.

[0112] In another aspect, the present invention also proposes an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to realize the above-mentioned deadlift posture detection method based on mobile video.

[0113] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0114] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0115] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0116] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as set forth in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways.

Claims

1. A method for detecting deadlift posture based on mobile video, characterized in that, The method includes: The video of a user's deadlift motion is captured by a mobile device and input into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number. Gaussian filtering is applied to the joint coordinate sequence in the time dimension for smoothing, and biomechanical angular features representing the standard of deadlift posture are calculated based on the smoothed joint coordinates. The biomechanical angular features include at least the trunk angle, hip joint angle and knee joint angle. The biomechanical angle features are evaluated by a frame-by-frame two-stage sequence determination to assess whether the deadlift is standard. The two-stage sequence determination includes a first-stage determination for the starting posture and a second-stage determination for the pulling and exerting force process. The deadlift is considered a standard movement only when both stages are determined to be passed. Extract a preset proportion of frames from the deadlift action video sequence as the initial analysis interval; Within the analysis interval, frame by frame, it is determined whether the torso angle is within the first standard threshold range and whether the knee joint angle is within the second standard threshold range. If the torso angle is within the first standard threshold range and the knee joint angle is within the second standard threshold range, then the frame is determined to meet the first stage. If the proportion of frames meeting the first stage exceeds the first threshold, then the first stage is considered passed. Set standard extension range The basic extension characteristics are calculated using the following formula: ； in, Let be the knee angle in frame t. The basic stretching feature for frame t; The trajectory vertical verification feature is calculated using the following formula: ； in, The trajectory vertical verification feature for frame t. Let be the horizontal projection distance between the wrist joint coordinate vector and the knee joint coordinate vector in Euclidean space. The maximum deviation threshold; The force coordination verification characteristics are calculated using the following formula: ； ； ； in, For standard coordinated intervals, For the force coordination verification features of frame t, For motion coordination rate, Let be the instantaneous angular velocity of the hip joint in frame t. Let be the instantaneous angular velocity of the knee joint in frame t. It is a non-zero constant. The sampling interval; In the entire deadlift video sequence, all frames where the hip angle is within the third standard threshold range are selected as the key force exertion frame set, and the standard frame ratio of the pull-up phase is calculated according to the following formula: ； in, The motion quality status of frame t. And operation, This is a set of key force exertion process frames; If the proportion of standard frames exceeds the preset second threshold, the second stage is considered to have passed.

2. The method for detecting deadlift posture based on mobile video according to claim 1, characterized in that, The torso angle is formed by the line connecting the shoulder joint and the hip joint and the gravity vector perpendicular to the ground; The hip joint angle is formed by three points: the shoulder joint, the hip joint, and the knee joint. The knee joint angle is formed by three points: the hip joint, the knee joint, and the ankle joint.

3. The method for detecting deadlift posture based on mobile video according to claim 1, characterized in that, The joint coordinate sequence is smoothed according to the following formula: ； in, The value of the Gaussian function. denoted as standard deviation, and x as the time offset of the current data point relative to the filter center.

4. The method for detecting deadlift posture based on mobile video according to claim 3, characterized in that, The method further includes: When performing angle threshold judgment at any stage, if the judgment result of the biomechanical angle feature calculated based on the joint point of one side of the human skeleton is not passed, the system will automatically switch to the angle feature calculated based on the corresponding joint point of the other side of the human skeleton, and re-judge according to the same threshold and pass ratio rules.

5. A deadlift posture detection system based on mobile video, used to implement the deadlift posture detection method based on mobile video as described in any one of claims 1-4, characterized in that, The system includes: The video capture module is used to capture a video of a user's deadlift action via a mobile device and input the deadlift action video into the Mediapipe pose estimation model to extract the joint coordinate sequence of the human skeleton in each frame of the video. The joint coordinate sequence includes multiple joint numbers and the joint coordinates corresponding to each joint number. The feature extraction module is used to apply Gaussian filtering to smooth the joint coordinate sequence in the time dimension, and calculate biomechanical angle features representing the standard of deadlift posture based on the smoothed joint coordinates. The biomechanical angle features include at least the trunk angle, hip joint angle and knee joint angle. The detection execution module is used to perform frame-by-frame two-stage sequence determination on the biomechanical angle features to evaluate whether the deadlift action is standard. The two-stage sequence determination includes a first-stage determination for the starting posture and a second-stage determination for the pulling force process. The deadlift action is considered standard only when both stages are determined to be passed.

6. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the hardlift posture detection method based on mobile video as described in any one of claims 1-4.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the deadlift posture detection method based on mobile video as described in any one of claims 1-4.

Images