Method and system for recognizing bodybuilding actions based on Android mobile terminal

Abstract

This invention provides a fitness motion recognition method and system based on an Android mobile device, belonging to the field of computer vision technology. The method includes: acquiring a video stream of a target performing a fitness motion to be recognized; then calling a MediaPipe pose estimation model deployed on the Android mobile device to extract the coordinates of key human points of the target; when it is determined that the target has completed the initial movement of the fitness motion, performing feature statistics on the coordinates of the key human points within a preset target time period to obtain waveform data of the key human point features; timing the static continuous movements in the fitness motion to be recognized based on the waveform data, and counting the periodic movements in the fitness motion to be recognized to obtain the recognition result of the fitness motion to be recognized. This invention aims to solve the problems of insufficient real-time performance and low recognition accuracy of existing fitness motion recognition models, thereby improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a fitness motion recognition method and system based on an Android mobile device. Background Technology

[0002] Existing fitness motion recognition technologies mostly employ human pose estimation models, such as MoveNet (17 keypoints), OpenPose, or AlphaPose, to count or time single or limited movements. However, these models have significant limitations: OpenPose and AlphaPose are primarily designed for PC or GPU environments. When ported to Android mobile devices, they suffer from low frame rates (typically less than 20 frames per second), making it impossible to achieve high-frame-rate real-time keypoint coordinate output. This results in noticeable latency, making them unsuitable for real-time fitness guidance scenarios on Android mobile devices. Furthermore, they only support offline pre-recorded video frame analysis and cannot provide immediate feedback, leading to insufficient real-time performance in motion recognition. While MoveNet is relatively lightweight, it struggles to recognize complex fitness motion details, resulting in low accuracy and negatively impacting user experience. Summary of the Invention

[0003] In view of this, it is necessary to provide a fitness movement recognition method and system based on Android mobile devices to solve the problems of insufficient real-time performance and low recognition accuracy of existing fitness movement recognition models, and improve user experience.

[0004] To address the aforementioned problems, this invention provides a fitness motion recognition method based on an Android mobile device, applicable to an Android mobile device deployed with a MediaPipe pose estimation model. The method includes: Acquire a video stream of the target performing the fitness movement to be identified, and call the MediaPipe pose estimation model to extract the coordinates of the human key points of the target from the video stream; When it is determined that the target to be identified has completed the starting action of the fitness movement to be identified, the key points of the human body are statistically analyzed within a preset target time period to obtain waveform data of the key point features of the human body. The time interval of the target time period is the duration of the fitness movement to be identified performed by the target to be identified. The static continuous movements in the fitness movement to be identified are timed based on the waveform data, and the periodic movements in the fitness movement to be identified are counted to obtain the identification result of the fitness movement to be identified.

[0005] In one possible implementation, the human body key point features include: the distance between human body key points, the limb angle formed by the human body key points, and the angle between the limb formed by the human body key points and the horizontal ground.

[0006] In one possible implementation, the counting rule for the periodic movements in the fitness exercise to be identified includes: When there is only one type of human body key point feature, the waveform change value of the human body key point feature is determined based on the waveform data; When the waveform change value of the human body key point feature exceeds the preset change threshold, and the feature value of the human body key point feature has periodic changes, the fitness movement to be identified is determined as a periodic movement, and the periodic movement is counted once. When there is more than one type of human body key point feature, the feature value of the human body key point feature is determined based on the waveform data. When the feature value of each human key point feature reaches the preset feature threshold, and the feature value of the human key point feature changes periodically, the fitness movement to be identified is determined as a periodic movement, and the periodic movement is counted once.

[0007] In one possible implementation, the timing rules for the static continuous motion in the fitness movement to be identified include: Determine the waveform change value of at least one key human body feature from the waveform data; When the waveform change value is within a preset threshold range, the fitness movement to be identified is determined as a static continuous movement, and the duration is recorded. When the duration of the waveform change value exceeding the threshold range is greater than the time threshold, the duration of the statistics is stopped. Within the target time period, the cumulative duration is calculated, and the cumulative duration is determined as the timing of the static continuous action.

[0008] In one possible implementation, the starting motion of the fitness movement to be identified is determined in the following way: Get the preset starting state of the starting action; Determine the current action state of the target to be identified. The state value of the current action state includes the height ratio of the target to be identified in the video stream, the standing angle, and the initial action features of the target to be identified. If the difference between the current action state value and the preset starting state value is less than a preset difference threshold, then it is determined that the target to be identified has started to perform an initial action. If the difference between the current action state value and the preset starting state value is not less than a preset difference threshold, a prompt is sent to the target to be identified until the difference value is less than the difference threshold. The prompt is used to instruct the target to adjust the current action state.

[0009] In one possible implementation, the method further includes: Obtain the difficulty coefficient set for the target to be identified; The change threshold, feature threshold, and time threshold for counting periodic actions are updated based on the difficulty coefficient.

[0010] In one possible implementation, the method further includes: Within a preset target time period, the waveform data of the key human body features and the recognition results of the fitness movements to be identified are visualized. The recognition results are evaluated based on preset evaluation rules to obtain an evaluation score for the recognition results, and the evaluation score is visualized.

[0011] The present invention also provides a fitness motion recognition system based on an Android mobile device, comprising: The video acquisition module is used to acquire video streams of the target performing the fitness movements to be identified. The key point feature extraction module is used to call the MediaPipe pose estimation model deployed on the Android mobile terminal to extract the coordinates of human key points of the target to be identified from the video stream; The feature waveform generation module, when it is determined that the target to be identified has completed the starting action of the fitness movement to be identified, performs feature statistics on the coordinates of the human body key points within a preset target time period to obtain waveform data of the human body key point features, wherein the time interval of the target time period is the duration of the fitness movement to be identified performed by the target to be identified. The identification and statistics module is used to time the static continuous movements in the fitness movement to be identified based on the waveform data, and to count the periodic movements in the fitness movement to be identified, so as to obtain the identification result of the fitness movement to be identified.

[0012] The present invention also provides an electronic device, characterized in that it includes a memory and a processor, wherein the memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the above-described fitness motion recognition method based on an Android mobile terminal.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described fitness motion recognition method based on an Android mobile terminal.

[0014] The beneficial effects of the above implementation are as follows: The fitness movement recognition method and system based on Android mobile devices provided by this invention are applied to Android mobile devices with a MediaPipe pose estimation model deployed. By acquiring a video stream of the target performing the fitness movement, and calling the MediaPipe pose estimation model to extract the coordinates of the target's key human points from the video stream, real-time feature analysis of the fitness movement is achieved. Furthermore, after the initial movement is completed, feature statistics are performed on the key human point coordinates within a set target time period, and the changes in key human point features are statistically analyzed using waveform data to complete the recognition of the fitness movement. This achieves instant feedback from video stream analysis to feature point extraction to movement recognition, ensuring the real-time performance of fitness movement recognition on Android mobile devices. Moreover, this recognition process uses specific key human point coordinates as the data foundation and further uses waveform data of the coordinates to finely count and time periodic and static continuous movements separately, improving the accuracy of fitness movement recognition and user experience. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the fitness motion recognition method based on an Android mobile device provided by the present invention. Figure 2 A schematic diagram illustrating the effect of human body key point extraction provided by the present invention; Figure 3 A schematic diagram illustrating the effect of human body key point coordinates provided by the present invention; Figure 4 This is a schematic diagram illustrating the selection of fitness movements to be identified provided by the present invention. Figure 5 A schematic diagram illustrating the effect of the initial action preparation stage provided by the present invention; Figure 6 A schematic diagram of normal waveform data of the limb angle formed by key points of the human body provided by the present invention; Figure 7A schematic diagram of abnormal waveform data of limb angles formed by key points of the human body provided by the present invention; Figure 8 A schematic diagram of the structure of the fitness motion recognition system based on an Android mobile terminal provided by the present invention; Figure 9 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0019] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0020] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] The fitness movement recognition method and system based on Android mobile devices provided by this invention can be specifically applied to fitness exercise scenarios using Android mobile devices. The execution entity is an Android mobile device with a MediaPipe pose estimation model deployed on it. When the target performs the fitness movement to be recognized, a video stream of the target performing the fitness movement is acquired through the video recording mode of the Android mobile device. Then, the fitness movement recognition method based on Android mobile devices provided by this invention is called, and the MediaPipe pose estimation model is used to process the video stream to obtain the recognition result of the fitness movement. Further feedback and evaluation are then provided through the Android mobile device so that the target can know the recognition result and feedback evaluation result of its own fitness exercise.

[0023] The following section details the fitness motion recognition method based on an Android mobile device provided by this invention. This method is applied to an Android mobile device with a MediaPipe pose estimation model deployed on it.

[0024] Figure 1 This is a flowchart illustrating the fitness motion recognition method based on an Android mobile device provided by the present invention. Figure 1 As shown, the fitness motion recognition method based on Android mobile terminal can be implemented through the following steps 101 to 103, which are explained in detail below.

[0025] Step 101: Obtain the video stream of the target performing the fitness movement to be identified, and call the MediaPipe pose estimation model to extract the coordinates of the human key points of the target from the video stream.

[0026] Here, the target to be identified is a fitness user using an Android mobile device. When the target performs the fitness movement to be identified, the video stream of the fitness movement to be identified is recorded by enabling the video recording mode of the Android mobile device, so that the Android mobile device can obtain the video stream of the target performing the fitness movement to be identified.

[0027] The Android mobile application deployment here integrates the MediaPipe library, which includes the MediaPipe pose estimation model. The entire video stream processing is completed on the Android mobile device's local CPU, without requiring network video uploads or server reliance. In practical implementation, it can be deployed and run in an actual Android mobile application. By granting camera permissions to the Android mobile device and enabling the application's video recording mode, the video stream of the fitness movements to be recognized is acquired. Then, the MediaPipe library is called, and the MediaPipe pose estimation model is used to process the video stream.

[0028] The fitness movements to be identified generally include various types of movements, such as push-ups, sit-ups, jumping jacks, planks, and single-leg standing. These can be broadly categorized into two types: periodic movements and static continuous movements. For example, push-ups, sit-ups, and jumping jacks are periodic movements, while planks and single-leg standing are static continuous movements. When identifying fitness movements, it is necessary to time and count both periodic and static continuous movements separately to achieve the identification and evaluation of the fitness movements.

[0029] Before the target device performs a fitness exercise, it can select the corresponding exercise type via an Android mobile app. After the exercise is performed, the Android mobile app starts real-time monitoring and video recording, acquiring a video stream of the target device performing the exercise.

[0030] Furthermore, for each frame of the video stream, a target recognition algorithm is invoked to perform target recognition in real time. First, the target to be recognized in the image is identified, and then multiple human key points of the target to be recognized are extracted.

[0031] In this embodiment of the invention, 33 key points of the human body are extracted, and the specific distribution of the key points is as follows: Figure 2 As shown, the key points of the human body are: nose 0, left inner eye 1, left eye 2, left outer eye 3, right inner eye 4, right eye 5, right outer eye 6, left ear 7, right ear 8, left mouth 9, right mouth 10, left shoulder 11, right shoulder 12, left elbow 13, right elbow 14, left wrist 15, right wrist 16, left finger 17, right finger 18, left index finger 19, right index finger 20, left thumb 21, right thumb 22, left hip 23, right hip 24, left knee 25, right knee 26, left ankle joint 27, right ankle joint 28, left heel 29, right heel 30, left index toe 31, right index toe 32.

[0032] To determine the positional changes of each key point on the human body during exercise movements, it is necessary to determine the positional data of these key points. Specifically, as follows... Figure 3 As shown, a two-dimensional coordinate system with X and Y axes can be established based on the distribution range of human feature points. Then, the coordinates of 33 human key points can be determined one by one in the two-dimensional coordinate system to characterize the positional features of human key points.

[0033] Step 102: When it is determined that the target to be identified has completed the starting movement of the fitness movement to be identified, the key point coordinates of the human body are statistically analyzed within the preset target time period to obtain the waveform data of the key point features of the human body.

[0034] During the preparation phase, the fitness movement that the target to be identified begins to perform is the starting movement. The starting movement needs to be judged and synchronized to ensure that the current movement state of the target to be identified is aligned with the preset starting state of the starting movement.

[0035] In one possible implementation, the starting motion of the fitness movement to be identified is determined in the following manner, as explained in detail below.

[0036] First, obtain the preset starting state of the starting action.

[0037] Once the type of fitness movement is determined for the target to be identified, the preset starting state of the corresponding starting movement can be determined. The specific preset starting state's display effect on an Android mobile device can be seen as follows: Figure 4 As shown, the preset initial state values ​​include the height ratio of the target to be identified in the video stream, the standing angle, and the initial action features of the target. Since the video stream is captured by an Android mobile device, the height ratio of the target to be identified in the video stream is the same as the height ratio of the target to be identified on the Android mobile device's screen. For example, a height ratio of 4:5 ensures that the current height of the target to be identified is 4:5 of the height of the Android mobile device's screen. The standing angle is generally a right angle between the target's body and the horizontal ground, ensuring that the target is standing upright without any left or right leaning. The initial action features are the coordinates of the human body's key points at the start of the action, meaning that the coordinates of each key point of the target's body must coincide with the coordinates of the key points at the start of the action.

[0038] Furthermore, the current action state of the target to be identified is determined. Here, the state value of the current action state can be determined based on the acquired video frames. The state value of the current action state includes the height ratio of the target to be identified in the video stream, the standing angle, and the initial action features of the target to be identified.

[0039] If the difference between the current action state value and the preset starting state value is less than the preset difference threshold, then the target to be identified is determined to start making an initial action.

[0040] This step compares the current action state of the target to be identified with the preset initial state of the starting action, and calculates the difference between the two state values, that is, the difference values ​​of height ratio, standing angle, and starting action features. If the calculated difference value is less than the preset difference threshold, it means that the current action state has basically met the preset initial state of the starting action, and at this time it can be determined that the target to be identified has begun to perform the starting action.

[0041] If the difference between the current action state value and the preset starting state value is not less than the preset difference threshold, a prompt is sent to the target to be identified until the difference value is less than the difference threshold.

[0042] If the difference between the current action state value and the preset starting state value is not less than a preset difference threshold, it indicates that the current action state does not conform to the preset starting state for the initial action, and it can be determined that the target to be identified has not yet started performing the initial action. In this case, a prompt is sent to the target, such as through vibration or voice prompts on an Android mobile device. This prompt instructs the target to adjust its current action state, thus informing it to adjust its current action and standardize the initial action. After the target adjusts its current action, the current action state is recalculated, and the difference between the current action state value and the preset starting state is recalculated until the difference value is less than the difference threshold, at which point it is determined that the target has begun performing the initial action.

[0043] In this embodiment of the invention, during the preparation stage of the fitness movement to be identified, the starting movement of the fitness movement to be identified is determined, and the timestamp of the actual human fitness movement is aligned with the timestamp of the data processing and movement recognition of the fitness movement to be identified. This ensures the effectiveness of the subsequent data processing and movement recognition of the fitness movement to be identified and avoids time misalignment in the subsequent timing and counting process, which could lead to loss of real-time performance.

[0044] When the target to be identified completes the starting movement of the fitness movement to be identified, the key points of the human body are statistically analyzed within the preset target time period to obtain waveform data of the key points of the human body.

[0045] Next, by aligning the initial state, the system determines when the target has completed the initial movement of the fitness exercise to be recognized, at which point the formal fitness exercise recognition process begins. The fitness exercise recognition process is generally completed within a set target time period, which is the duration of the target's fitness exercise, for example, one minute. Specifically, the timing of the target time period can begin a short time after the target completes the initial movement of the fitness exercise, for example, within the last 5 seconds.

[0046] During this target time period, the target to be identified continuously performs fitness movements of the corresponding movement type. At this time, the human body key point coordinates are extracted from the video stream by monitoring, and feature statistics are performed on the human body key point coordinates to obtain waveform data of human body key point features.

[0047] In one possible implementation, the human body keypoint features include: the distance between human body keypoints, the limb angle formed by the human body keypoints, and the angle between the limb formed by the human body keypoints and the horizontal ground.

[0048] Regarding the distance between human body key points, the Euclidean distance between any two human body key point coordinates out of 33 key points can be calculated. The limb angle formed by these key points is based on the line connecting them; this line represents the limb direction of the target being identified, and its direction can be specifically described as follows: Figure 3 As shown, Figure 3 The diagram shows the connections established between 33 key points on the human body. For example, a line is formed between the right knee 26 and the right ankle 28, and a line is formed between the right hip 24 and the right knee 26. The angle formed by these two lines is the limb angle.

[0049] Of course, there are various angles formed by key points on the human body, such as... Figure 5 As shown, when the target (fitness person) performs jumping jacks, angles are formed between the lines connecting key points on the left, right, right, and left limbs and the lines connecting key points on other body parts. Furthermore, since the human body has left and right sides, when calculating the lines connecting key points, the midpoints of these two sides can be selected, and the angles between the lines connecting these midpoints can be determined; these are also called limb angles. For example, the hip midpoint is the midpoint of the line connecting the left and right hips, and the shoulder midpoint is the midpoint of the line connecting the left and right shoulders. The angle formed by the line connecting these two midpoints and other connecting lines is also called a limb angle. These angle data are all statistically analyzed.

[0050] by Figure 5 Taking the lower limb angle ∠DCE of the jumping jack as an example, A and B are the key points of the right hip and left hip, respectively. C is the midpoint of the line connecting the key points of the left hip and right hip, i.e., the midpoint of the hip. D and E are the key points of the right ankle joint and left ankle joint, respectively. The calculation steps of the angle ∠DCE are explained below.

[0051] First, calculate the coordinates of the midpoint C of the hip. The formula for calculating the coordinates is as follows: (1) (2) in, and The x and y coordinates are the x and y coordinates of the key point A on the right hip, respectively. and The x and y coordinates are the key point B on the left hip, respectively. and Here, x and y are the x and y coordinates of the midpoint C of the hip, respectively.

[0052] Next, we will use the Pythagorean theorem to calculate the lengths of the three sides. Taking D and E as the key points of the right and left ankle joints, we will calculate the Euclidean distances of line segments DC, CE, and DE, respectively, that is, calculate the lengths of the three sides DC, CE, and DE respectively: (3) (4) (5) in, and Let x and y be the x and y coordinates of the midpoint C of the hip, respectively. and Let x and y be the x and y coordinates of the right ankle joint D, respectively. and The x-coordinate and y-coordinate of the left ankle joint E are respectively.

[0053] Finally, the included angle ∠DCE is calculated using the law of cosines. Based on the lengths of the three sides DC, CE, and DE, the included angle ∠DCE can be calculated using the law of cosines, as shown in the following formula: (6) in, This represents the inverse cosine function.

[0054] Therefore, when it is necessary to calculate the angle between the lines connecting key points of the human body or the angle with the horizontal ground, the above calculation method can be used as a reference. The length of the connecting line can be calculated based on the coordinates of each key point of the human body, and then the specific corresponding angle value can be calculated.

[0055] The angle between the limb formed by the key points of the human body and the horizontal ground is the angle between the line connecting the key points of the human body and the horizontal ground. For example... Figure 3 As shown, the direction of the horizontal ground can be considered as the direction of the X-axis. The limbs formed by the key points of the human body are the lines connecting these key points, and the angle between the limb and the horizontal ground is the angle between the connecting lines and the horizontal ground.

[0056] For example, in Figure 3 In the diagram, the line connecting the right ankle (28°) and the right heel (30°) forms the angle between the line and the X-axis. This angle represents the angle between the limb and the horizontal ground, formed by the lines connecting key points of other limbs such as the left hand, right hand, and left foot. Similarly, since the human body has left and right sides, when calculating the lines connecting key points, the midpoints of these two sides can be selected. The angle between the line connecting these midpoints and the X-axis can then be determined. For example, the hip midpoint is the midpoint of the line connecting the left and right hips, and the shoulder midpoint is the midpoint of the line connecting the left and right shoulders. The angle between the line connecting these two midpoints and the horizontal plane also needs to be calculated.

[0057] Therefore, based on 33 human body key points, the distance between human body key points, the limb angle formed by human body key points, and the angle between the limb formed by human body key points and the horizontal ground are calculated as human body key point features, which are used as the basis for subsequent action recognition.

[0058] In this embodiment of the invention, by performing feature statistics on the coordinates of key points on the human body to obtain the features of key points on the human body, the feature representation of the fitness movements to be identified can be accurately expressed.

[0059] Furthermore, in this embodiment of the invention, the extracted human body key point features are statistically analyzed, and corresponding waveform data is generated based on the feature values ​​of the key point features. Because the human body key points change in real time as the target performs fitness movements within the target time period, the statistically analyzed human body key point features also fluctuate in real time. Therefore, the human body key point feature values ​​within the target time period can be recorded in chronological order to obtain the corresponding waveform data.

[0060] like Figure 6 As shown, Figure 6 It shows the limb angle formed by key points of the human body. Waveform data varying with time t. The Y-axis represents the limb angle formed by key points on the human body. The X-axis represents time t. When the target to be identified performs a fitness movement, the limb angle at each time point t can be calculated. These data coordinates form waveform data. Based on this waveform data, the timing and counting of the fitness movements to be recognized can be achieved.

[0061] Step 103: Based on the waveform data, time the static continuous movements in the fitness movement to be identified, and count the periodic movements in the fitness movement to be identified, to obtain the identification result of the fitness movement to be identified.

[0062] In this embodiment of the invention, the recognition process of the fitness movement to be identified mainly involves timing the static continuous movements and counting the periodic movements. The final timing and counting results within a preset target time period are the recognition results of the fitness movement to be identified. The process of identifying the fitness movement to be identified based on waveform data is described below.

[0063] In one possible implementation, the counting rules for periodic movements in the fitness exercise to be identified can be implemented in two ways, which are described below.

[0064] The first method is to count based on the waveform changes of key human body features.

[0065] When there is only one type of human body key point feature, the waveform change value of the human body key point feature is determined based on the waveform data; when the waveform change value of the human body key point feature exceeds the preset change threshold and the feature value of the human body key point feature shows periodic changes, the fitness movement to be identified is determined as a periodic movement and one periodic movement is counted.

[0066] Specifically, key human body features include three categories: the distance between key human body points, the limb angle formed by key human body points, and the angle between the limb formed by key human body points and the horizontal ground. In practical calculations, for simpler fitness movements such as jumping jacks, one type of key human body feature can be selected to count periodic movements; generally, the limb angle formed by key human body points is chosen. The waveform change value is the change value of the feature. For example, if the limb angle formed by key human body points changes between the first and second seconds, the difference between these two seconds is the waveform change value. Furthermore, it is necessary to count whether the limb angle formed by key human body points undergoes repeated changes. For example, if the limb angle formed by key human body points returns to the same level as in the first second in the third second, it indicates that the limb angle formed by key human body points undergoes periodic changes within these three seconds.

[0067] Further, the judgment mainly involves determining whether the waveform change value exceeds a preset change threshold, such as whether the change in the limb angle formed by the human body's key points exceeds 180°. Simultaneously, if the limb angle formed by the human body's key points exhibits periodic changes, then the fitness movement being identified can be determined to be a periodic movement, and this periodic movement is counted once. Thus, using the same method described above, periodic movements are identified and counted in real time within a preset target time period (within 1 minute).

[0068] For example, a fitness enthusiast might be performing jumping jacks. The key features of the human body are the angles formed by these key points. During counting, the angles between the left ankle (27) and the midpoint of the hip (the midpoint of the line connecting the left and right hips) and the right ankle (28), the angle between the left wrist (15) and the midpoint of the shoulder (the midpoint of the line connecting the left and right shoulders) and the right wrist (16), and whether the angles between the left wrist (15), the left elbow (13), and the left shoulder (11) are nearly straight (approaching 180°) are monitored. Only when these angles synchronously complete the "opening and closing" reciprocating motion and successively meet the corresponding preset thresholds is it counted as one valid jumping jack.

[0069] When there is more than one type of human body key point feature, the feature value of the human body key point feature is determined based on the waveform data; when the feature value of each human body key point feature reaches the preset feature threshold and the feature value of the human body key point feature shows periodic changes, the fitness movement to be identified is determined as a periodic movement and one periodic movement is counted.

[0070] Specifically, when counting complex exercises such as sit-ups and push-ups, multiple key human features can be selected to count periodic movements. The counting process involves directly extracting the feature values ​​of these key human features from the waveform data. The process for judging periodic movements is similar to that of judging a single feature, and will not be elaborated here.

[0071] In the specific judgment, the main focus is on whether the feature values ​​have all reached the preset feature thresholds, such as whether the limb angle formed by the human body key points reaches 180°, whether the limb formed by the human body key points and the horizontal ground reaches 45°, and whether the distance between the human body key points reaches 0.5 meters, etc.

[0072] Simultaneously, by determining that the limb angles formed by key points on the human body exhibit periodic changes, it can be confirmed that the fitness movement being identified is a periodic movement, and thus, this periodic movement is counted once. Therefore, using the same method described above, periodic movements are identified and counted in real time within a preset target time period (within 1 minute).

[0073] For example, if a fitness enthusiast is doing push-ups, the key features of the human body are judged by combining all three types of key features. During counting, the distance between the left wrist 15-left elbow (12)-left shoulder 11 (reaching 0.3 meters) and the angle between them (reaching 180°), the angle between the left shoulder 11-left hip 23-left heel 29 (reaching 180°, close to a straight line), and the angle between the torso (i.e., the line connecting the midpoint of the left and right shoulders to the midpoint of the left and right hips) and the horizontal plane (reaching 0°) are monitored in real time. Only when these angles synchronously complete the "opening and closing" reciprocating change, and the angles and distances meet the corresponding feature thresholds one by one, is it counted as a valid push-up.

[0074] In this embodiment of the invention, when counting periodic movements in fitness exercises to be identified, different numbers of human key point features are flexibly selected to deal with different types of fitness exercises to be identified. This flexible feature selection method can ensure that fitness exercises of different types can be counted with high accuracy.

[0075] The timing rules for static continuous movements in the fitness exercises to be identified are explained in detail below.

[0076] First, determine the waveform change value of at least one key human body feature from the waveform data.

[0077] The timing of static continuous motion mainly involves counting the duration of a motion that remains static. The duration of a motion remaining static depends on the waveform changes of key human features. Therefore, we first determine the feature value of at least one key human feature from the waveform data, and then calculate the waveform change value corresponding to the feature value.

[0078] When the waveform change value is within the preset threshold range, the fitness movement to be identified is determined as a static continuous movement, and the duration is counted.

[0079] Next, a threshold range is set to determine whether the exercise movement to be identified is static. If the exercise movement remains static, the waveform change value of its human key point features must fall within this threshold range. Therefore, if the waveform change value is within the preset threshold range, the exercise movement to be identified can be determined as a static continuous movement, and the duration of this state can be counted.

[0080] For example, if a fitness enthusiast is performing a plank exercise, the key features of the human body can be the angles formed by the limbs and the horizontal ground, such as the angle between the spine (the line connecting the left and right shoulders) and the horizontal ground, or the angle between the torso (the line connecting the midpoints of the left and right shoulders to the midpoints of the left and right hips) and the horizontal ground. When the waveform changes of these angles are all within the threshold range of 0° to 5°, it can be determined that the fitness enthusiast is maintaining a plank exercise, and the duration of the plank hold can then be counted.

[0081] When the time for which the waveform change value exceeds the threshold range is greater than the time threshold, the duration of the statistics is stopped.

[0082] Because the target object, while maintaining a static continuous motion, may become unable to continue due to exhaustion or external environmental factors, the waveform change value may exceed this threshold range. Specifically, as shown below... Figure 7 As shown, Figure 7 It shows the limb angle formed by key points of the human body. Waveform data varying with time t. The Y-axis represents the limb angle formed by key points on the human body. The X-axis represents time t. When the target to be identified performs a fitness movement, the limb angle at each time point t can be calculated. These coordinate points form waveform data. Based on this waveform data, the limb angle can be determined. The waveform change value is within the range of 0° to 45°, but at a certain point in time, the limb angle... It dropped directly to 0°, exceeding the range of 0° to 45°, and no longer fluctuated. Furthermore, this process took longer than a time threshold, for example, more than 2 seconds. So, at the angle between the limbs... After the temperature drops directly to 0°, the duration of the exercise movement is recorded and not counted. During this process, the target can be prompted to adjust via voice or other means. If this process takes less than 2 seconds, there is no need to stop recording the duration. After recording the duration, the target readjusts and resumes the exercise movement; the limb angle in the waveform data will be recorded. The waveform change value returned to the range of 0° to 45°. At this point in the recovery period, the duration of the fitness movements was continuously recorded until the target time period ended.

[0083] Therefore, by continuously monitoring the waveform changes of key human body features within a preset target time period using the above method, the statistical duration for which the waveform changes are within a preset threshold range can be calculated.

[0084] Finally, the cumulative duration within the target time period is calculated, and this cumulative duration is determined as the timing of the static continuous action.

[0085] Within the target time period (within 1 minute), the duration of static continuous actions may be counted multiple times using the above-mentioned method. After the target time period ends, these counted durations can be accumulated. These accumulated durations are the timing of the static continuous actions, which are used to characterize the total duration of the static continuous actions performed by the target to be identified within the target time period.

[0086] In this embodiment of the invention, a dual constraint of threshold range and time threshold is set in the timing rules for static continuous actions, and the timing of static continuous actions is realized by accumulating the duration, thereby reducing statistical errors and ensuring the effectiveness of static continuous action recognition.

[0087] Therefore, by using the counting rules for periodic movements and the timing rules for static continuous movements in the fitness exercises to be identified, it is possible to simultaneously time the static continuous movements and count the periodic movements within the target time period. The final statistical results of timing and counting are the identification results of the fitness exercises to be identified.

[0088] In one possible implementation, in order to more strictly regulate periodic movements and static continuous movements, embodiments of the present invention can also adjust the recognition process in real time. On Android mobile devices, the difficulty level of the fitness movements to be recognized can be set in advance.

[0089] At this point, the difficulty coefficient set for the target to be identified is obtained, and then the change threshold, feature threshold, and time threshold used to count periodic actions are updated based on the difficulty coefficient.

[0090] When counting periodic movements, it is necessary to determine whether the feature values ​​of human key points all reach preset feature thresholds and whether the waveform changes of human key points all exceed preset change thresholds. The feature thresholds and change thresholds can be adjusted using a difficulty coefficient. For example, a higher difficulty coefficient results in more stringent counting of periodic movements, higher feature and change thresholds, higher required feature values ​​and waveform changes for human key points, and greater constraints on periodic movements. Conversely, a lower difficulty coefficient results in more relaxed counting of periodic movements, lower feature and change thresholds, lower required feature values ​​and waveform changes for human key points, and less constraints on periodic movements.

[0091] When counting static continuous movements, the timing of stopping the counting depends on whether the waveform changes of key human body features exceed a threshold for a period of time. This time threshold can be adjusted using a difficulty level. For example, a higher difficulty level results in stricter timing of static continuous movements, a lower time threshold, a shorter requirement for waveform changes to exceed the preset threshold, and greater constraints on the static continuous movements. Conversely, a lower difficulty level allows for more relaxed timing, a higher time threshold, a longer requirement for waveform changes to exceed the preset threshold, and less constraints on the static continuous movements.

[0092] In this embodiment of the invention, by setting a difficulty coefficient, the recognition difficulty of the fitness movement to be recognized can be freely controlled, thereby enabling real-time control of the movement requirements of periodic movements and static continuous movements, and meeting different fitness movement recognition needs.

[0093] To further demonstrate the recognition effect of fitness movements, this embodiment of the invention also implements a result visualization process, which is described in detail below.

[0094] First, within a preset target time period, the waveform data of key human body features and the recognition results of the fitness movements to be identified are visualized.

[0095] Here, waveform data of key human body features are calculated in real time within a preset target time period, which can be obtained through methods such as... Figure 6 and Figure 7 The waveform data is visualized using a coordinate graph. The recognition results for the fitness movements to be identified include timing and counting results. For example, within one minute, the count might show 10 jumping jacks, 10 sit-ups, or a 20-second single-leg stand. These recognition results can be visualized on an Android mobile device's display screen, presenting them to the target (the fitness participant) so they can clearly understand their performance on the fitness movements.

[0096] Furthermore, the recognition results are evaluated based on preset evaluation rules to obtain an evaluation score for the recognition results, and the evaluation score is visualized.

[0097] Here, the recognition results can be evaluated, for example, by scoring them according to preset evaluation rules. For instance, the evaluation rules might stipulate that 30 repetitive movements within one minute earn 50 points, and a static, sustained movement lasting 20 seconds earns another 50 points, for a maximum score of 100. Therefore, based on these rules, the current recognition results are assessed. 10 jumping jacks and 10 sit-ups do not reach 30 repetitive movements, so they are scored as 50 × 2 / 3, approximately 34 points. However, a single-leg stand lasting 20 seconds earns 50 points, resulting in a final evaluation score of 84 points.

[0098] Finally, the evaluation score can still be visualized on the Android mobile device's display screen, presenting the target (fitness person) to be identified. This allows the target to clearly understand the shortcomings of their fitness movements through the evaluation score. For example, the score indicates a deficiency in performing periodic movements, which can be improved in subsequent fitness training.

[0099] In this embodiment of the invention, by visualizing waveform data, recognition results, and evaluation scores, effective information transmission with the target to be identified (fitness person) can be achieved, enabling the target to perceive the effect of fitness movements in real time, providing a basis for the improvement of subsequent fitness movements, and enhancing the experience of fitness breakage.

[0100] In summary, this embodiment of the invention acquires a video stream of a target performing a fitness movement on an Android mobile device, and uses a MediaPipe pose estimation model deployed on the Android mobile device to extract the coordinates of key human points from the video stream. This enables real-time feature analysis of the fitness movement. Furthermore, after the initial movement is completed, feature statistics are performed on the key human point coordinates within a set target time period, and the changes in key human point features are analyzed using waveform data to complete the recognition of the fitness movement. This achieves instant feedback from video stream analysis to feature point extraction to movement recognition, ensuring real-time performance of fitness movement recognition on the Android mobile device. Moreover, this recognition process uses specific key human point coordinates as its data foundation and further utilizes waveform data of the coordinates to finely count and time periodic and static continuous movements, improving the accuracy of fitness movement recognition and user experience.

[0101] The following describes in detail the fitness motion recognition system based on an Android mobile terminal provided by an embodiment of the present invention. For example... Figure 8As shown, the fitness motion recognition system based on the Android mobile terminal specifically includes: a video acquisition module 801, a key point feature extraction module 802, a feature waveform generation module 803, and a recognition statistics module 804.

[0102] Specifically, the video acquisition module 801 is used to acquire a video stream of the target performing the fitness movement to be identified; the key point feature extraction module 802 is used to call the MediaPipe pose estimation model deployed on the Android mobile terminal to extract the human key point coordinates of the target from the video stream; the feature waveform generation module 803, when it is determined that the target has completed the starting action of the fitness movement to be identified, performs feature statistics on the human key point coordinates within a preset target time period to obtain waveform data of human key point features, wherein the time interval of the target time period is the duration of the fitness movement to be identified performed by the target; and the recognition statistics module 804 is used to time the static continuous movements in the fitness movement to be identified based on the waveform data, and count the periodic movements in the fitness movement to be identified to obtain the recognition result of the fitness movement to be identified.

[0103] In one possible implementation, such as Figure 8 As shown, the fitness motion recognition system based on the Android mobile terminal also includes a difficulty adjustment module 805.

[0104] The difficulty adjustment module 805 is used to obtain the difficulty coefficient set for the target to be identified; and to update the change threshold, feature threshold and time threshold used to count periodic actions and to time static continuous actions according to the difficulty coefficient.

[0105] In one possible implementation, such as Figure 8 As shown, the fitness motion recognition system based on the Android mobile terminal also includes a result visualization module 806.

[0106] The result visualization module 806 is used to visualize the waveform data of the human body key point features and the recognition results of the fitness movements to be recognized within a preset target time period; to evaluate the recognition results based on preset evaluation rules, to obtain the evaluation score of the recognition results, and to visualize the evaluation score.

[0107] The fitness motion recognition system based on Android mobile terminal provided in the above embodiments can realize the technical solutions described in the above embodiments of the fitness motion recognition method based on Android mobile terminal. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the fitness motion recognition method based on Android mobile terminal, and their technical effects can also be referred to each other, which will not be repeated here.

[0108] like Figure 9 As shown, the present invention also provides an electronic device 900. The electronic device 900 includes a processor 901, a memory 902, and a display 903. Figure 9 Only some components of the electronic device 900 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0109] In some embodiments, memory 902 may be an internal storage unit of electronic device 900, such as a hard disk or memory of electronic device 900. In other embodiments, memory 902 may also be an external storage device of electronic device 900, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 900.

[0110] Furthermore, the memory 902 may include both internal storage units of the electronic device 900 and external storage devices. The memory 902 is used to store application software and various types of data installed on the electronic device 900.

[0111] In some embodiments, processor 901 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 902 or process data, such as the fitness motion recognition method based on Android mobile terminal in this invention.

[0112] In some embodiments, display 903 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 903 is used to display information from electronic device 900 and to display a visual user interface. Components 901-903 of electronic device 900 communicate with each other via a system bus.

[0113] In some embodiments of the present invention, when the processor 901 executes the fitness movement recognition program in the memory 902, the following steps can be implemented: acquiring a video stream of the target performing the fitness movement to be recognized, and calling the MediaPipe pose estimation model to extract the human body key point coordinates of the target from the video stream; when it is determined that the target has completed the initial movement of the fitness movement to be recognized, performing feature statistics on the human body key point coordinates within a preset target time period to obtain waveform data of human body key point features, wherein the time interval of the target time period is the duration of the fitness movement to be recognized performed by the target; timing the static continuous movements in the fitness movement to be recognized based on the waveform data, and counting the periodic movements in the fitness movement to be recognized to obtain the recognition result of the fitness movement to be recognized.

[0114] It should be understood that when the processor 901 executes the fitness motion recognition program in the memory 902, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0115] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 900 mentioned. Electronic device 900 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, electronic device 900 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0116] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the fitness motion recognition method based on an Android mobile terminal provided by the methods described above. The method includes: acquiring a video stream of a target performing a fitness motion to be recognized, and calling a MediaPipe pose estimation model to extract the coordinates of human key points of the target from the video stream; when it is determined that the target has completed the initial movement of the fitness motion to be recognized, performing feature statistics on the coordinates of the human key points within a preset target time period to obtain waveform data of human key point features, wherein the time interval of the target time period is the duration of the fitness motion performed by the target; timing the static continuous movements in the fitness motion to be recognized based on the waveform data, and counting the periodic movements in the fitness motion to be recognized to obtain the recognition result of the fitness motion to be recognized.

[0117] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0118] The above provides a detailed description of the fitness motion recognition method and system based on Android mobile devices provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for recognizing a fitness action based on an Android mobile terminal, characterized in that, The method, applied to Android mobile devices with a MediaPipe pose estimation model deployed, includes: Acquire a video stream of the target performing the fitness movement to be identified, and call the MediaPipe pose estimation model to extract the coordinates of the human key points of the target from the video stream; When it is determined that the target to be identified has completed the starting action of the fitness movement to be identified, the key points of the human body are statistically analyzed within a preset target time period to obtain waveform data of the key point features of the human body. The time interval of the target time period is the duration of the fitness movement to be identified performed by the target to be identified. The static continuous movements in the fitness movement to be identified are timed based on the waveform data, and the periodic movements in the fitness movement to be identified are counted to obtain the identification result of the fitness movement to be identified.

2. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The key features of the human body include: the distance between key points of the human body, the limb angle formed by key points of the human body, and the angle between the limb formed by key points of the human body and the horizontal ground.

3. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The counting rules for the periodic movements in the fitness movements to be identified include: When there is only one type of human body key point feature, the waveform change value of the human body key point feature is determined based on the waveform data; When the waveform change value of the human body key point feature exceeds the preset change threshold, and the feature value of the human body key point feature has periodic changes, the fitness movement to be identified is determined as a periodic movement, and the periodic movement is counted once. When there is more than one type of human body key point feature, the feature value of the human body key point feature is determined based on the waveform data. When the feature value of each human key point feature reaches the preset feature threshold, and the feature value of the human key point feature changes periodically, the fitness movement to be identified is determined as a periodic movement, and the periodic movement is counted once.

4. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The timing rules for the static continuous movements in the fitness movements to be identified include: Determine the waveform change value of at least one key human body feature from the waveform data; When the waveform change value is within a preset threshold range, the fitness movement to be identified is determined as a static continuous movement, and the duration is recorded. When the duration of the waveform change value exceeding the threshold range is greater than the time threshold, the duration of the statistics is stopped. Within the target time period, the cumulative duration is calculated, and the cumulative duration is determined as the timing of the static continuous action.

5. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The starting motion of the fitness movement to be identified is determined in the following way: Get the preset starting state of the starting action; Determine the current action state of the target to be identified. The state value of the current action state includes the height ratio of the target to be identified in the video stream, the standing angle, and the initial action features of the target to be identified. If the difference between the current action state value and the preset starting state value is less than a preset difference threshold, then it is determined that the target to be identified has started to perform an initial action. If the difference between the current action state value and the preset starting state value is not less than a preset difference threshold, a prompt is sent to the target to be identified until the difference value is less than the difference threshold. The prompt is used to instruct the target to adjust the current action state.

6. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The method further includes: Obtain the difficulty coefficient set for the target to be identified; The change threshold, feature threshold, and time threshold for counting periodic actions are updated based on the difficulty coefficient.

7. The fitness motion recognition method based on an Android mobile terminal according to claim 1, characterized in that, The method further includes: Within a preset target time period, the waveform data of the key human body features and the recognition results of the fitness movements to be identified are visualized. The recognition results are evaluated based on preset evaluation rules to obtain an evaluation score for the recognition results, and the evaluation score is visualized.

8. A fitness motion recognition system based on an Android mobile terminal, characterized in that, include: The video acquisition module is used to acquire video streams of the target performing the fitness movements to be identified. The key point feature extraction module is used to call the MediaPipe pose estimation model deployed on the Android mobile terminal to extract the coordinates of human key points of the target to be identified from the video stream; The feature waveform generation module, when it is determined that the target to be identified has completed the starting action of the fitness movement to be identified, performs feature statistics on the coordinates of the human body key points within a preset target time period to obtain waveform data of the human body key point features, wherein the time interval of the target time period is the duration of the fitness movement to be identified performed by the target to be identified. The identification and statistics module is used to time the static continuous movements in the fitness movement to be identified based on the waveform data, and to count the periodic movements in the fitness movement to be identified, so as to obtain the identification result of the fitness movement to be identified.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the fitness motion recognition method based on an Android mobile terminal as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the fitness motion recognition method based on an Android mobile terminal as described in any one of claims 1 to 7.

Images