Method and system for attitude detection and real-time feedback

Abstract

The invention discloses a posture detection and real-time feedback method and system, and the method comprises the steps: a user executes an exercise action, and an application program captures a current video frame image in real time, and transmits the current video frame image to a rear end; estimating a user posture coordinate according to the video frame image by using a posture estimation model; calculating a similarity score of the current action posture according to the standard posture coordinates and the user posture coordinates by using a posture similarity algorithm; feeding back according to the similarity score, and displaying correct and wrong parts of the current action posture; the application program continues to capture the next frame of image until the user finishes the exercise action, and posture counting is recorded and displayed. A posture estimation model and a posture similarity algorithm are utilized to more accurately estimate and evaluate the posture of the user; the correct and wrong parts of the posture are dynamically fed back in real time, the user is guided to adjust the action in time, and the limitation of hysteresis feedback in a traditional method is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence image recognition, and particularly to a method and system for pose detection and real-time feedback. Background Art

[0002] With the improvement of health awareness and the progress of technology, pose detection technology has gradually become an important means to promote personal health, optimize sports performance and prevent sports injuries. Most traditional pose detection technologies are based on two-dimensional image analysis. By capturing the user's image through a camera and comparing it with a preset standard pose, the accuracy of the user's pose is evaluated. However, this two-dimensional analysis method faces many limitations in practical applications and is difficult to meet the growing needs of health management and sports training.

[0003] Currently, for pose detection, the following technical problems exist:

[0004] (1) Due to relying only on two-dimensional image information, it is difficult for the existing technology to comprehensively and accurately capture the user's three-dimensional pose information, especially the pose evaluation at different angles. This leads to limited accuracy of pose detection, is prone to misjudgment due to perspective deviation, and thus increases the risk of injury;

[0005] (2) In sports training that requires repeated counting, traditional methods often rely on manual counting or simple image processing techniques. This method is not only inefficient but also prone to errors, affecting the training effect and the reliability of data;

[0006] (3) Most existing pose detection technologies can only provide static or post-analysis results, lacking the ability to adjust the user's actions in real time and dynamically from multiple angles during the movement. This lagging feedback cannot correct the user's wrong pose in time, limiting its application effect in improving sports efficiency and preventing injuries. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for pose detection and real-time feedback to solve the problems in the existing pose detection technology mentioned in the above background art, such as the difficulty in comprehensively capturing three-dimensional pose information and correcting the user's wrong pose.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions:

[0009] According to one aspect of the present invention, a method for pose detection and real-time feedback is provided, and the method includes:

[0010] The user performs an exercise action, and the application program captures the current video frame image in real time and sends it to the backend;

[0011] Using a pose estimation model, estimate the user's pose coordinates according to the video frame image;

[0012] Use the pose similarity algorithm to calculate the similarity scores of each limb and the overall of the current action pose based on the standard pose coordinates and the user pose coordinates;

[0013] Provide feedback based on the similarity scores to display the correct and incorrect parts of the current action pose;

[0014] The application continues to capture the next frame of image until the user ends the exercise action, records and displays the pose count.

[0015] Based on the foregoing solution, the user pose coordinates are three-dimensional key point coordinates; the pose estimation model receives the current video frame image as input, extracts the two-dimensional feature map of the video frame image using a neural network, trains the model to predict the three-dimensional key point coordinates based on the two-dimensional feature map, and fits on a three-dimensional pose dataset.

[0016] Based on the foregoing solution, find the standard pose of each exercise action according to the timestamp from the database or other data sources, and convert the standard pose into the three-dimensional standard pose coordinates.

[0017] Based on the foregoing solution, the standard pose also includes being collected from the real-time streaming media for the user to obtain immediate feedback.

[0018] Based on the foregoing solution, align the user pose coordinates with the standard pose coordinates so that the user pose coordinates and the standard pose coordinates are in the same reference coordinate system.

[0019] Based on the foregoing solution, the pose similarity algorithm uses the limb-to-limb similarity scores to evaluate the similarity between the user pose coordinates and the standard pose coordinates, including calculating the position vectors of the key points of each limb of the user pose and the standard pose respectively, and calculating the dot product of the position vectors to obtain the similarity scores of each limb, and weighting and averaging the similarity scores of each limb to obtain the overall similarity score.

[0020] Based on the foregoing solution, the providing feedback according to the overall similarity score includes comparing the overall similarity score with a set threshold to determine whether the current pose is correct or close to correct; displaying the user's pose in real time on the user interface and marking the correct and incorrect parts with different colors or marks.

[0021] Based on the foregoing solution, if the overall similarity score reaches the set threshold, it is considered that the current pose is correct or close to correct, and positive feedback is provided to the user; if it does not reach the set threshold, it is considered that the current pose needs to be corrected, and corrective feedback is provided to the user.

[0022] Based on the foregoing solution, before the user finishes the exercise action, the application continuously captures and processes video frames, determines whether the exercise action is completed through user input or automatic detection, and displays the result of the pose count on the user interface.

[0023] According to another aspect of the present invention, there is provided a system for pose detection and real-time feedback, which includes a video capture module, a pose estimation module, a similarity evaluation module, a feedback display module, a counting and recording module, and an interaction control module;

[0024] The video capture module is used to capture video frame images of the user performing exercise actions in real time;

[0025] The pose estimation module receives the video frame images and estimates the user's pose coordinates using a pose estimation model;

[0026] The similarity evaluation module calculates the similarity scores of each limb and the whole of the current action pose using a pose similarity algorithm based on the user's pose coordinates and standard pose coordinates;

[0027] The feedback display module shows the correct and incorrect parts of the current action pose to the user according to the similarity scores;

[0028] The counting and recording module records the number of times the user correctly completes the exercise action each time and displays the total count when the user finishes the exercise;

[0029] The interaction control module processes the user's interaction instructions, and the interaction instructions include but are not limited to starting, pausing, and ending the exercise.

[0030] It can be seen from the above technical solutions that the present invention has at least the following advantages and positive effects compared with the prior art:

[0031] (1) By using a pose estimation model and a pose similarity algorithm, the present invention can more accurately estimate and evaluate the user's pose, especially the accuracy is significantly improved in different angles and complex motion scenarios, reducing misjudgment caused by perspective deviation;

[0032] (2) The automated and intelligent pose counting function enables the user to get rid of the cumbersome and inefficient manual counting, while reducing the possibility of human errors, improving the accuracy and reliability of the data, thereby enhancing the efficiency of exercise and training;

[0033] (3) Real-time and dynamic feedback on the correct and incorrect parts of the pose helps the user adjust the action in time, ensure the correct form, avoid the limitations of lagging feedback in traditional methods, and improve the exercise effect and safety.

[0034] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and should not limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments in accordance with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts. In the drawings:

[0036] Figure 1 is a schematic diagram of a method for attitude detection and real-time feedback of the present invention;

[0037] Figure 2 is a schematic diagram of the process of the present invention for the attitude estimation model to output the user's attitude coordinates;

[0038] Figure 3 is a schematic diagram of the process of the present invention for the attitude similarity algorithm and providing feedback;

[0039] Figure 4 is a schematic diagram of a system for attitude detection and real-time feedback of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] In order to more clearly elaborate the objectives, technical solutions and advantages of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. The exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art.

[0041] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of the present invention. However, those skilled in the art will realize that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring aspects of the present invention.

[0042] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the drawings are only illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0044] The present invention will be described in detail below with reference to specific embodiments:

[0045] Embodiment 1

[0046] As Figure 1 shown, this embodiment provides a method for attitude detection and real-time feedback, and the specific steps of this method are as follows:

[0047] S1: The user performs an exercise action, and the application captures the current video frame image in real time and sends it to the backend.

[0048] In this embodiment, the user uses the application on the mobile device for sports training. When the user starts to perform an exercise action, the application captures the real-time video stream and extracts the video frame image;

[0049] Specifically, when the application starts, it requests the user to authorize access to the device's camera to ensure that the application has obtained the camera access permission of the device; uses the camera API of the mobile device to capture the video stream in real time and start the real-time video preview; for video frame capture, a timer can be set or the callback mechanism of the video stream can be utilized to capture the image of the current frame from the video stream at regular intervals (such as 30 frames per second). When extracting the video frame, the video frame can be extracted from the video stream at a fixed frame rate (such as 30 FPS); necessary processing is performed on the captured image, such as resizing, grayscale conversion, etc., to adapt to the input requirements of the pose estimation model.

[0050] Furthermore, the resolution and frame rate of the video frame are set according to the actual usage requirements to ensure that the resolution and frame rate of the video frame are sufficient to support accurate pose estimation; considering the influence of lighting conditions on the image quality, automatic exposure adjustment or image enhancement algorithms may need to be added; ensure the efficiency of image capture and processing to avoid affecting the smoothness of the video.

[0051] S2: Use the pose estimation model to estimate the user's pose coordinates according to the video frame image.

[0052] In this embodiment, the user posture coordinates are the three-dimensional key point coordinates of the body; the posture estimation model receives the current video frame image as input, extracts the two-dimensional feature map of the video frame image using a neural network, trains the model to predict the three-dimensional key point coordinates based on the two-dimensional feature map, and fits on a three-dimensional posture dataset.

[0053] Specifically, when the application starts or is used for the first time, download the pre-trained posture estimation model from the server or load it from local storage; preprocess the extracted video frame image and then input it into the posture estimation model, and the model outputs the two-dimensional coordinates of the body key points (such as head, shoulders, knees, etc.).

[0054] Further, as Figure 2 shown, the posture estimation model receives the video frame image as input, uses CNN to extract the two-dimensional feature map of each frame image, and this feature map contains rich spatial information for subsequent three-dimensional posture estimation; train a neural network model on a dataset containing a large number of pictures, two-dimensional key point annotations (as features), and corresponding three-dimensional key point coordinates to reconstruct and predict the X, Y, Z three-dimensional coordinates of each key point from the two-dimensional key point coordinates. Optionally, fit a statistical model on the three-dimensional posture dataset to map from two-dimensional coordinates to three-dimensional coordinates to further enhance the prediction ability of the three-dimensional key point coordinates. Optionally, after obtaining the user posture coordinates, techniques such as non-maximum suppression (NMS) can be applied to optimize the key point detection results, and time smoothing and geometric constraints are used to further improve the stability and accuracy of the posture estimation.

[0055] Further, the posture estimation model can be trained end-to-end to directly output three-dimensional postures from the input image, avoiding multi-stage processing in traditional methods.

[0056] S3: Use the posture similarity algorithm to calculate the similarity scores of each limb and the whole of the current action posture according to the standard posture coordinates and the user posture coordinates.

[0057] In this embodiment, collect the standard postures of each exercise action from the database or other data sources, and convert the standard postures into the three-dimensional standard posture coordinates. The standard postures also include those collected from real-time streaming media for users to obtain instant feedback. Or, a set of key point coordinates of standard postures can be predefined for each exercise action, and according to the requirements of the exercise action, a set of standard three-dimensional posture coordinates can be defined, which can be a set of positions of multiple key points.

[0058] Further, align the user pose coordinates with the standard pose coordinates so that the user pose coordinates and the standard pose coordinates are in the same reference coordinate system. Optionally, methods such as affine transformation or similarity transformation are used to align the user pose coordinates and the standard pose coordinates to the same reference frame to minimize the error between the two.

[0059] Optionally, the method of maximizing overlap is adopted to adjust the position of the user's body parts in the virtual space to align them with the standard pose to the greatest extent; the user's pose is translated and rotated to minimize the spatial difference between the user's pose and the coach's pose; image processing or deep learning techniques can be used to identify and track the user's key body parts, that is, key points, and then by calculating the offset between the corresponding points in the user pose coordinates and the standard pose coordinates, the user is guided to adjust the pose.

[0060] Optionally, the method of transforming the basis vectors is used to align the user's pose with the standard pose. In three-dimensional space, the basis vectors usually refer to the direction vectors of the coordinate axes (such as the x, y, and z axes); by transforming these basis vectors, the rotation and translation of the entire pose framework can be achieved, so that the user's pose matches the coach's pose; by calculating the appropriate transformation matrices (such as rotation matrix and translation matrix), it can be ensured that every detail of the user's pose is precisely aligned with the coach's pose.

[0061] Optionally, the method of matching the forward vectors is adopted to ensure that the user maintains the same direction as the standard pose during the practice. In three-dimensional space, the forward vector usually refers to the direction that the user or the coach is facing; by matching these two forward vectors, it can be ensured that the user not only has the correct pose but also the same direction when imitating the action; the angle between the user's forward vector and the coach's forward vector can be calculated and the corresponding rotation can be applied to eliminate this angle difference.

[0062] Further, as Figure 3 shown, the pose similarity algorithm uses the limb-to-limb similarity scores to evaluate the similarity between the user pose coordinates and the standard pose coordinates, including calculating the position vectors of each limb key point of the user pose and the standard pose respectively, and calculating the dot product of the position vectors to obtain the similarity score of each limb, and weighting and averaging the similarity scores of each limb to obtain the overall similarity score.

[0063] Specifically, the limb-to-limb similarity scores are usually used to evaluate the similarity between two poses (such as a standard pose and a user pose), and this algorithm usually involves comparing the positions and directions of the corresponding limbs in the poses.

[0064] In this embodiment, first, the key point coordinates of each limb in the user posture and the standard posture are defined. Each limb is represented by two key points (similar to the two endpoints of a line segment), forming a position vector. The calculation method of the position vector of each limb is as follows:

[0065] Let the three-dimensional coordinates of the two key points of limb i be K i,1 =(x1, y1, z1) and K i,2 =(x2, y2, z2), then the position vector of this limb is: P i =K i,2 -K i,1 =(x2 - x1, y2 - y1, z2 - z1).

[0066] Furthermore, according to the dot product of the user's limb and the standard limb two vectors to measure the similarity between the user posture and the standard posture, where the value range of the dot product is [-1, 1], where 1 represents complete similarity (coincidence), and -1 represents that the limb movements are completely opposite. Furthermore, considering the angle and the degree of coincidence, the dot product can be appropriately transformed or combined with other metrics (such as the vector length ratio, the cosine value of the angle, etc.) to more accurately reflect the similarity, and normalized to obtain the similarity score of each limb; the similarity scores of all limbs are weighted and averaged to obtain the overall similarity score between the user posture and the standard posture to evaluate the similarity between the two. S4: Provide feedback according to the overall similarity score, and display the correct and incorrect parts of the current action posture;

[0067] In this embodiment, compare the overall similarity score with a set threshold to determine whether the current posture is correct or close to correct; display the user's posture in real time on the user interface, and mark the correct and incorrect parts with different colors or marks.

[0068] Furthermore, if the overall similarity score reaches the set threshold, it is considered that the current posture is correct or close to correct, and positive feedback is provided to the user; if it does not reach the set threshold, it is considered that the current posture needs to be corrected, and corrective feedback is provided to the user. Furthermore, feedback can also be provided to the user according to the similarity score of the limb. Optionally, when it is judged to be correct or close to correct, the interface displays positive feedback, such as "Well done!", "Keep it up!", etc.; when it is judged to need correction, the incorrect part of the user's posture is marked with highlighting or different colors, and corrective feedback is displayed, such as "Please adjust the position of your arm", etc. Optionally, according to the overall similarity score or the similarity scores of each limb, positive or corrective feedback is given to the user through a voice assistant or text prompts on the screen.

[0069] Further, if the overall similarity score reaches the set threshold, it is considered that the current posture is correct or close to correct, and positive feedback is provided to the user; if it does not reach the set threshold, it is considered that the current posture needs to be corrected, and corrective feedback is provided to the user. Further, feedback can also be provided to the user according to the similarity score of the limb. Optionally, when it is judged to be correct or close to correct, the interface displays positive feedback, such as "Well done!", "Keep it up!", etc.; when it is judged to need correction, the incorrect part of the user's posture is marked with highlighting or different colors, and corrective feedback is displayed, such as "Please adjust the position of your arm", etc. Optionally, according to the overall similarity score or the similarity scores of each limb, positive or corrective feedback is given to the user through a voice assistant or text prompts on the screen.

[0070] Furthermore, provide Render Feedback on Frame for all video frames. Regardless of whether the similarity score is sufficient, feedback information will be rendered on each video frame so that users can intuitively see whether their postures are correct and make continuous adjustments based on the continuous feedback. Optionally, overlay the user's live image with a standard posture model and highlight the different parts by means of colors, lines or shadows; create a virtual "mirror" that shows the comparison between the user's actions and the standard posture, making the user feel like interacting with their own mirror image; animations or arrows can be added to the feedback to indicate how the user should adjust their posture or actions.

[0071] S5: The application continues to capture the next frame image until the user ends the exercise action, records and displays the posture count;

[0072] In this embodiment, before the user ends the exercise action, the operations of steps S1 to S4 are repeated to continuously capture and process video frames. Whenever it is detected that the user successfully completes a standard-compliant posture (i.e., the similarity score reaches the set threshold), the posture counter is incremented. Determine whether the exercise action has ended through user input (such as clicking a button) or automatic detection (such as detecting no movement for a long time), and display the result of the posture count on the user interface for the user to refer to.

[0073] Embodiment 2

[0074] As Figure 4 shown, this embodiment exemplarily presents a posture detection and real-time feedback system, including a video capture module, a posture estimation module, a similarity evaluation module, a feedback display module, a counting and recording module, and an interaction control module;

[0075] The video capture module is used to capture video frame images of the user performing an exercise action in real time; the posture estimation module receives the video frame images and estimates the user's posture coordinates using a posture estimation model; the similarity evaluation module aligns the user's posture coordinates and the standard posture coordinates, and then uses a posture similarity algorithm to calculate the similarity scores of each limb and the whole of the current action posture; the feedback display module shows the correct and incorrect parts of the current action posture to the user according to the similarity score; the counting and recording module records the number of times the user correctly completes the exercise action each time and displays the total count when the user ends the exercise; the interaction control module processes the user's interaction instructions, including but not limited to starting, pausing, and ending the exercise.

[0076] Specifically, the video capture module captures video frame images of the user performing exercise actions in real time, uses the device's camera API to capture the video stream, and extracts the image data of each frame; implements an interface to provide the image frame data for subsequent module processing.

[0077] The pose estimation module receives the video frame images, uses the pose estimation model to estimate the user's pose coordinates, loads the pre-trained pose estimation model, processes the input image frames, and outputs the user's key point coordinates; implements an interface to receive the image frames and output the user's pose coordinate data.

[0078] The similarity evaluation module calculates the similarity score of the current pose using the pose similarity algorithm based on the aligned user pose coordinates and the standard pose coordinates; implements an interface to receive the user pose coordinates and the standard pose coordinates and output the similarity scores for each limb and the whole body.

[0079] The feedback display module shows the correct and incorrect parts of the user's pose to the user according to the overall similarity score, and uses the graphical interface to overlay and highlight on the video frame; implements an interface to receive the similarity score and the user's pose coordinates and output visual feedback to the user interface. Further, if the similarity score reaches the set threshold, it is considered that the current pose is correct or nearly correct, and positive feedback is provided to the user; if it does not reach the set threshold, it is considered that the current pose needs to be corrected, and corrective feedback is provided to the user.

[0080] The counting and recording module processes the user's interaction instructions, such as starting, pausing, and ending the exercise, listens for user input, and calls the functions of the corresponding modules; implements an interface to receive the user instructions, call the functions of other modules, and display the status update to the user.

[0081] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the claims. It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for attitude detection and real-time feedback, characterized in that, The method includes: The user performs an exercise action, and the application captures the current video frame image in real time and sends it to the backend. Using a pose estimation model, estimate the user's pose coordinates based on the video frame image. Using a pose similarity algorithm, calculate the similarity scores of each limb and the overall of the current action pose according to the standard pose coordinates and the user's pose coordinates. Provide feedback based on the similarity scores, and display the correct and incorrect parts of the current action pose. The application continues to capture the next frame image until the user ends the exercise action, records and displays the pose count.

2. The method for attitude detection and real-time feedback according to claim 1, characterized in that, The user's pose coordinates are three-dimensional key point coordinates; the pose estimation model receives the current video frame image as input, extracts the two-dimensional feature map of the video frame image using a neural network, trains the model to predict the three-dimensional key point coordinates based on the two-dimensional feature map, and fits on a three-dimensional pose dataset.

3. A method for attitude detection and real-time feedback according to claim 1, characterized in that, Collect the standard poses of each exercise action from a database or other data sources, and convert the standard poses into the three-dimensional standard pose coordinates.

4. A method for attitude detection and real-time feedback according to claim 3, characterized in that, The standard pose also includes being collected from a real-time streaming media for the user to obtain instant feedback.

5. A method for attitude detection and real-time feedback according to claim 1, characterized in that Align the user's pose coordinates with the standard pose coordinates so that the user's pose coordinates and the standard pose coordinates are in the same reference coordinate system.

6. A method for attitude detection and real-time feedback according to claim 1, characterized in that, The pose similarity algorithm uses the limb-to-limb similarity scores to evaluate the similarity between the user's pose coordinates and the standard pose coordinates, including calculating the position vectors of each limb key point of the user's pose and the standard pose respectively, and calculating the dot product of the position vectors to obtain the similarity score of each limb, and weighted averaging the similarity scores of each limb to obtain the overall similarity score.

7. A method for attitude detection and real-time feedback according to claim 1, characterized in that, The providing feedback based on the similarity scores includes comparing the similarity scores with a set threshold to determine whether the current pose is correct or close to correct; displaying the user's pose on the user interface in real time, and marking the correct and incorrect parts with different colors or marks.

8. A method for attitude detection and real-time feedback according to claim 7, characterized in that, If the similarity score reaches the set threshold, it is considered that the current pose is correct or close to correct, and positive feedback is provided to the user; if it does not reach the set threshold, it is considered that the current pose needs to be corrected, and corrective feedback is provided to the user.

9. A method for attitude detection and real-time feedback according to claim 1, characterized in that Before the user ends the exercise action, the application continuously captures and processes video frames, determines whether the exercise action ends through user input or automatic detection, and displays the result of the pose count on the user interface.

10. A system for attitude detection and real-time feedback, characterized in that, It includes a video capture module, a pose estimation module, a similarity evaluation module, a feedback display module, a counting and recording module, and an interaction control module. The video capture module is used to capture the video frame image of the user performing the exercise action in real time. The pose estimation module receives the video frame image and estimates the user's pose coordinates using a pose estimation model. The similarity evaluation module calculates the similarity scores of each limb and the overall of the current action pose according to the user's pose coordinates and the standard pose coordinates using a pose similarity algorithm. The feedback display module shows the correct and incorrect parts of the current action pose to the user according to the similarity scores. The counting and recording module records the number of times the user correctly completes the exercise action each time, and displays the total count when the user ends the exercise; The interaction control module processes the user's interaction instructions, and the interaction instructions include but are not limited to starting, pausing, and ending the exercise.

Images