Fitness training method, system, device and medium based on fitness device

Abstract

The present invention discloses a fitness training method, system, device and medium based on a fitness device, which relates to the field of fitness. The first user posture is obtained according to a first fitness video to determine whether the user is performing fitness training; the second user posture is obtained according to a second fitness video, the second user posture is scored, and the user is judged according to the scoring result to determine whether the user follows the second fitness video to perform fitness training; the first user posture is obtained according to the first fitness video, the first user posture is scored, and the scoring result is fed back to the user. Through three stages, the user is respectively demonstrated fitness training movements, slow motion teaching and normal following are performed, the user's movements are identified, and it is determined whether the user has movements in the first stage, whether the user has followed in the second stage, and whether the user's movements meet the standards in the third stage, which effectively guarantees the user's fitness effect.

Description

Technical Field

[0001] The present invention relates to the field of fitness, and in particular to a fitness training method, system, device and medium based on a fitness device. Background Art

[0002] In recent years, the health literacy of the people has been continuously improved, and the demand for fitness exercises has been continuously increasing. The market of the fitness industry is huge. Various intelligent fitness equipment has developed rapidly. The existing mirror fitness equipment has built-in various devices in the body and displayed and / or mirrored by the front screen. Users can perform fitness training according to the displayed fitness content. In this way, when using it, the fitness coach can conduct fitness teaching to multiple users through live video or pre-recorded video. Among them, when conducting fitness teaching, the fitness coach usually conducts unified fitness teaching based on the approximate fitness level of the multiple users, such as the fitness level of flow yoga level 2. However, there are many problems with such fitness training teaching methods, such as the lack of timely feedback on the user's follow-up situation. If the user does not follow or does not follow the standard, the fitness effect is poor, which is not convenient for long-term use. Summary of the invention

[0003] The purpose of the present invention is to better obtain the training situation of the user and improve the fitness effect when the user is given fitness training through fitness videos.

[0004] To achieve the above-mentioned object of the invention, the present invention provides a fitness training method based on a fitness device, comprising:

[0005] Acquiring a first user's posture according to a first fitness video, and determining whether the user is performing fitness training;

[0006] Acquire a second user posture according to the second fitness video, score the second user posture, and determine whether the user follows the second fitness video for fitness training according to the scoring result;

[0007] A first user posture is acquired according to the first fitness video, the first user posture is scored, and the scoring result is fed back to the user.

[0008] For the present invention, the fitness video is played three times. The first time is mainly for demonstration. By playing the fitness video, the user can understand the training content. In this stage, it is mainly to see whether the user has performed simple follow-up. However, in this process, it is only judged whether the user has performed actions, but the actions that have occurred are not scored, and it is not judged whether the user has performed the same actions. When playing for the second time, it is a slow motion play of the first fitness video. The second fitness video can be a slow motion play of the fitness video played for the first time, or it can be a decomposition of the actions of the first fitness video. When playing, it monitors whether the user follows the second fitness video, and scores the user's actions to judge whether the user has performed the same actions as the fitness video, and then judge whether the user has followed. When playing for the third time, the video played for the third time is the same as the video played for the first time, but when playing the video for the third time, the user's actions will be identified, and after identification, it will be judged whether the user's actions meet the standards through scoring, thereby improving the user's fitness effect. Corresponding to the present invention, the video played for the first time, the video played for the second time, and the video played for the third time are all videos with the same content, except that the video played for the second time is a slow motion or action decomposition of the video played for the first time.

[0009] The step of obtaining the first user's posture according to the first fitness video and determining whether the user is performing fitness training specifically includes:

[0010] Obtaining a first fitness video; when obtaining the first fitness video, the first fitness video is played directly in a fitness device, which may be a smart fitness mirror or other fitness devices capable of playing videos;

[0011] When playing the first fitness video, the fitness device identifies the target fitness area of ​​the first fitness video. At this time, the user is exercising in the target fitness area. Therefore, when extracting features of the target fitness area, the first user posture can be obtained.

[0012] According to the first user posture, whether the user is performing fitness training is determined. In this process, whether the user has any action is determined. If there is any action, it means that the user is performing fitness training. If there is no action, the user is not performing fitness training. If there is no action, the effect of playing the video for the first time is poor, and it cannot be guaranteed that the user can quickly master the action when playing the video for the second or third time.

[0013] After playing the video for the first time, the user has a preliminary familiarity with the movements in the fitness video. After becoming familiar with the movements, the second user posture is obtained according to the second fitness video, and the second user posture is scored. According to the scoring result, it is determined whether the user follows the second fitness video for fitness training, which specifically includes:

[0014] Presetting a second standard posture according to a second fitness video;

[0015] identifying a target fitness area of ​​the second fitness video, performing feature extraction on the target fitness area, and obtaining a second user posture;

[0016] The second standard posture and the second user posture are compared to obtain a familiarity score of the second user posture based on the second standard posture, and whether the user follows the second fitness video for fitness training is determined according to the score result.

[0017] The step of identifying a target fitness area of ​​the second fitness video, extracting features of the target fitness area, and obtaining a second user posture specifically includes:

[0018] Obtain a first time period in which a second standard posture appears in a second fitness video;

[0019] Obtaining a video segment corresponding to the second fitness video within the first time period, performing frame processing on the video segment, and obtaining frame images at a plurality of consecutive moments corresponding to the video segment;

[0020] identifying a target fitness area of ​​a second fitness video within a first time period, extracting features of the target fitness area, and obtaining a plurality of second user postures corresponding to the frame images one by one;

[0021] Comparing the corresponding plurality of frame images with the plurality of second user gestures to obtain a familiarity score of each second user gesture based on the corresponding frame image;

[0022] The second user posture with the highest familiarity score is obtained as the scoring result.

[0023] When playing the fitness video for the second time, the user's actions need to be scored in this stage to determine whether the user is following the second fitness video. In this process, the present invention does not score and judge each frame in the second fitness video, but presets a second standard posture. By comparing the first time period in which the second standard posture appears in the second fitness video, the second user posture corresponding to each frame image in the first time period is obtained, and the corresponding several frame images and several second user postures are compared to obtain the familiarity score of each second user posture based on the corresponding frame image. When judging whether the user is following, the second user posture of the user in the corresponding time period is obtained through the preset second standard posture, and each frame image is compared with the corresponding second user posture, so as to better judge whether the user is following.

[0024] For the present invention, the specific scoring process includes:

[0025] Train the twin neural network model to obtain a trained standard posture recognition model;

[0026] Inputting the second user posture and the second standard posture into the standard posture recognition model to obtain a familiarity score;

[0027] If the scoring result is greater than or equal to the scoring threshold, the second standard posture and the second user posture are of the same category, and the user performs fitness training;

[0028] If the scoring result is less than the scoring threshold, the second standard posture and the second user posture are not of the same category, and the user has not performed fitness training.

[0029] When playing the first fitness video for the third time, specifically including:

[0030] Acquiring a first fitness video, that is, playing the first fitness video on the fitness device, and presetting a first standard posture according to the first fitness video;

[0031] identifying a target fitness area of ​​a first fitness video, extracting features of the target fitness area, and obtaining a first user posture;

[0032] The first standard posture and the first user posture are compared to obtain a familiarity score of the first user posture based on the first standard posture, and the score result is fed back to the user.

[0033] The step of identifying a target fitness area of ​​the first fitness video, extracting features of the target fitness area, and obtaining the first user posture specifically includes:

[0034] Acquire a second time period in which the first standard posture appears in the first fitness video;

[0035] identifying a target fitness area of ​​the first fitness video within a second time period, extracting features of the target fitness area, and obtaining a plurality of first user postures;

[0036] Comparing the first standard posture with the plurality of first user postures to obtain a familiarity score of each first user posture based on the first standard posture;

[0037] The first user posture with the highest familiarity score is obtained as the scoring result.

[0038] When playing the video for the second time and the third time, when playing the video for the second time, each frame image in the first time period is compared with the corresponding second user posture, and when playing the video for the third time, the first user posture in the second time period is compared with the first standard posture. In this way, when playing the video for the third time, it is possible to better confirm whether the user's actions are up to standard, thereby improving the efficiency of fitness teaching and training.

[0039] Corresponding to the method of the present invention, the present invention also provides a fitness training system based on a fitness device, comprising:

[0040] An acquisition module is used to acquire and process fitness videos;

[0041] A recognition module, used to identify the target fitness area of ​​the fitness video, extract features of the target fitness area, and obtain the user posture;

[0042] A comparison module, used to compare the user's posture with a preset standard posture to obtain a comparison result;

[0043] The judgment module is used to judge whether the user is performing fitness training or whether the action meets the standard according to the comparison result.

[0044] Corresponding to the method in the present invention, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned fitness training method based on the fitness device when executing the computer program.

[0045] Corresponding to the method in the present invention, the present invention further provides a storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the fitness training method based on the fitness device are implemented.

[0046] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0047] When conducting fitness teaching and training, the present invention respectively demonstrates fitness training movements to users, teaches in slow motion, and follows normally through three stages. During this process, the user's movements are identified to determine whether the user has made movements in the first stage, whether the user has followed in the second stage, and whether the user's movements meet the standards in the third stage, and the scoring results are fed back to the user.

[0048] Furthermore, when the present invention identifies and compares user actions, the identification and comparison are performed according to the first time period or the second time period corresponding to the preset first standard posture or the second standard posture. There is no need to identify and compare the entire video, so the comparison effect is better and the result is obtained faster, which effectively guarantees the user's fitness effect and can obtain the user's fitness status at each stage, thereby improving the efficiency of fitness use and being more convenient for long-term use. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0050] Figure 1 A flowchart of a fitness training method based on a fitness device;

[0051] Figure 2 A schematic diagram of the composition of a fitness training system based on a fitness device;

[0052] Figure 3 is the ROC curve. DETAILED DESCRIPTION

[0053] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0055] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0056] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0057] Please refer to Figure 1 , Figure 1 The present invention provides a fitness training method based on a fitness device, and the method includes:

[0058] Get Fitness First workout videos;

[0059] identifying a target fitness area of ​​a first fitness video, extracting features of the target fitness area, and obtaining a first user posture;

[0060] determining, according to the first user posture, whether the user is performing fitness training;

[0061] Presetting a second standard posture according to a second fitness video;

[0062] identifying a target fitness area of ​​the second fitness video, performing feature extraction on the target fitness area, and obtaining a second user posture;

[0063] Comparing the second standard posture with the second user posture, obtaining a familiarity score of the second user posture based on the second standard posture, and judging whether the user follows the second fitness video for fitness training according to the score result;

[0064] Obtain a first fitness video, and preset a first standard posture according to the first fitness video;

[0065] identifying a target fitness area of ​​a first fitness video, extracting features of the target fitness area, and obtaining a first user posture;

[0066] The first standard posture and the first user posture are compared to obtain a familiarity score of the first user posture based on the first standard posture, and the score result is fed back to the user.

[0067] The step of identifying a target fitness area of ​​the second fitness video, extracting features of the target fitness area, and obtaining a second user posture specifically includes:

[0068] Obtain a first time period in which a second standard posture appears in a second fitness video;

[0069] Obtaining a video segment corresponding to the second fitness video within the first time period, performing frame processing on the video segment, and obtaining frame images at a plurality of consecutive moments corresponding to the video segment;

[0070] identifying a target fitness area of ​​a second fitness video within a first time period, extracting features of the target fitness area, and obtaining a plurality of second user postures corresponding to the frame images one by one;

[0071] Comparing the corresponding plurality of frame images with the plurality of second user gestures to obtain a familiarity score of each second user gesture based on the corresponding frame image;

[0072] The posture of the second user with the highest familiarity score is obtained as the scoring result. In which, the twin neural network model is trained to obtain a trained standard posture recognition model;

[0073] Inputting the second user posture and the second standard posture into the standard posture recognition model to obtain a familiarity score;

[0074] If the scoring result is greater than or equal to the scoring threshold, the second standard posture and the second user posture are of the same category, and the user performs fitness training;

[0075] If the scoring result is less than the scoring threshold, the second standard posture and the second user posture are not of the same category, and the user has not performed fitness training.

[0076] The step of identifying a target fitness area of ​​the first fitness video, extracting features of the target fitness area, and obtaining the first user posture specifically includes:

[0077] Acquire a second time period in which the first standard posture appears in the first fitness video;

[0078] identifying a target fitness area of ​​the first fitness video within a second time period, extracting features of the target fitness area, and obtaining a plurality of first user postures;

[0079] Comparing the first standard posture with the plurality of first user postures to obtain a familiarity score of each first user posture based on the first standard posture;

[0080] Obtain the first user posture with the highest familiarity score as a scoring result;

[0081] Train the twin neural network model to obtain a trained standard posture recognition model;

[0082] Inputting the first user posture and the first standard posture into a standard posture recognition model to obtain a familiarity score;

[0083] If the scoring result is greater than or equal to the scoring threshold, the first standard posture and the first user posture are of the same category, and the user action meets the standard;

[0084] If the scoring result is less than the scoring threshold, the second standard posture and the second user posture are not of the same type, and the user action does not meet the standard.

[0085] The following is an introduction to the fitness training method based on the fitness device in the present invention with reference to specific examples:

[0086] Step 1: Obtain a first fitness video. In this embodiment, the fitness device is a fitness mirror, and the first fitness video is played on the mirror surface of the fitness mirror.

[0087] Step 2: obtaining the first user's posture according to the first fitness video, and determining whether the user is performing fitness training;

[0088] Step 2.1 The user follows the first fitness video in the target fitness area of ​​the fitness mirror;

[0089] Step 2.2: Preset a first standard posture in the first fitness video;

[0090] Step 2.3 obtains a second time period in which the first standard posture appears in the first fitness video;

[0091] Step 2.4: In the second time period, the fitness mirror recognizes the user's action in the target fitness area, extracts features of the target fitness area, and obtains the first user posture;

[0092] Step 2.5: If the first user gesture acquired in the second time period has different actions, it indicates that the user is following, that is, the user is doing fitness training; if the first user gesture is not acquired in the second time period, or the first user gesture has no actions, the user is not following, that is, the user is not doing fitness training. In step 2, in the second time period, the user's actions are recognized, but no scoring is performed in this process.

[0093] Step 3: acquiring the posture of the second user according to the second fitness video, scoring the posture of the second user, and judging whether the user follows the second fitness video for fitness training according to the scoring result;

[0094] Step 3.1: Play the second fitness video on the fitness mirror;

[0095] Step 3.2 presets a second standard posture according to a second fitness video;

[0096] Step 3.3 obtains a first time period in which the second standard posture appears in the second fitness video;

[0097] Step 3.4: within the first time period, obtain a video segment corresponding to the second fitness video within the first time period, perform frame processing on the video segment, and obtain frame images of several consecutive moments corresponding to the video segment;

[0098] Step 3.5: In the first time period, the fitness mirror recognizes the user's action in the target fitness area, extracts features of the target fitness area, and obtains a plurality of second user gestures corresponding to the frame images;

[0099] Step 3.6: Compare the corresponding frame images and the second user postures to obtain a familiarity score of each second user posture based on the corresponding frame image;

[0100] Step 3.61 trains the twin neural network model to obtain a trained standard posture recognition model;

[0101] Step 3.62: input the second user posture and the second standard posture into the standard posture recognition model to obtain a familiarity score;

[0102] Step 3.7 obtains the second user posture with the highest familiarity score as the scoring result; if the scoring result is greater than or equal to the scoring threshold, the second standard posture and the second user posture are of the same category, and the user performs fitness training; if the scoring result is less than the scoring threshold, the second standard posture and the second user posture are not of the same category, and the user does not perform fitness training.

[0103] Step 4: obtaining the first user's posture according to the first fitness video, scoring the first user's posture, and feeding back the scoring result to the user;

[0104] Step 4.1 The user follows the first fitness video in the target fitness area of ​​the fitness mirror;

[0105] Step 4.2: presetting a first standard posture in the first fitness video;

[0106] Step 4.3 obtains a second time period in which the first standard posture appears in the first fitness video;

[0107] Step 4.4: In the second time period, the fitness mirror recognizes the user's action in the target fitness area, extracts features of the target fitness area, and obtains the first user posture;

[0108] Step 4.5: comparing the first standard posture with the plurality of first user postures to obtain a familiarity score of each first user posture based on the first standard posture;

[0109] Step 4.51 inputs the first user posture and the first standard posture into the standard posture recognition model to obtain a familiarity score;

[0110] Step 4.52 obtains the first user posture with the highest familiarity score as the scoring result;

[0111] If the scoring result is greater than or equal to the scoring threshold, the first standard posture and the first user posture are of the same category, and the user action meets the standard; if the scoring result is less than the scoring threshold, the first standard posture and the first user posture are not of the same category, and the user action does not meet the standard.

[0112] In this embodiment, if the first standard posture appears at 10000ms in the fitness video. Now the fitness video is played to 10000ms, because the user is following the video, his movements are behind or ahead of the course, so we will set an interval near 10000ms, such as the first 800ms and the last 200ms, that is, the time interval [10000-800, 10000+200]. In this interval with a total duration of 1 second, the similarity between the first standard posture and the first user posture is calculated for each frame, and then the score with the highest similarity in this interval is output as the final score.

[0113] In this embodiment, if the second standard posture appears at 10000ms in the fitness video. Now the fitness video is played to 10000ms, because the user is following the video, his movements are behind or ahead of the course, so we will set an interval near 10000ms, such as the first 800ms and the last 200ms, that is, the time interval [10000-800, 10000+200]. In this interval with a total duration of 1 second, calculate the similarity between each frame and the first user's posture, and then output the highest score in this interval as the final score.

[0114] In the specific calculation, the first standard posture and the second standard posture are both static postures, not a continuous action. The specific method of comparison is to train a twin network structure model based on a convolutional neural network, which accepts two postures and maps the two postures to a point in a high-dimensional space respectively.

[0115] In this embodiment, the specific method of inputting the user posture and the standard posture into the standard posture recognition model to obtain the familiarity score is:

[0116] Obtain the skeletal key points of the standard posture and the user posture and the position coordinates corresponding to each skeletal key point, wherein the standard posture and the user posture both include 16 skeletal key points, and each of the 16 skeletal key points corresponds to a two-dimensional position coordinate; the 16 skeletal key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella;

[0117] Input the position coordinates corresponding to each skeletal key point of the standard posture and the user posture into the trained standard posture recognition model to obtain the output vector V1 of the standard posture and the output vector V2 of the user posture respectively;

[0118] Calculate the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture;

[0119] Among them, in the present invention, a human posture has a total of 16 two-dimensional coordinate skeleton points, each skeleton point has x and y coordinate components, so a human posture can be abstracted into a 32-dimensional skeleton point vector, that is, [x1, y1, x2, y2, x3, y3, ..., x16, y16]. After passing through the trained posture recognition model, this 32-dimensional skeleton point vector will be mapped into a higher-dimensional vector. In the present invention, the output vector is 100-dimensional, that is, the output vector V1 of the standard posture and the output vector V2 of the user posture are both 100-dimensional, that is, [a1, a2, a3, ..., a100]. When performing posture comparison, the standard posture and the user posture will each be mapped into a 100-dimensional vector, that is, V1 and V2, through the trained model, and then the Euclidean distance between V1 and V2 is calculated.

[0120] The present invention uses a deep neural network, which accepts a 32-dimensional vector, i.e., a human posture in the present invention, and then passes through a series of intermediate layer operations, such as nonlinear correction, full connection, etc., and finally outputs a 100-dimensional vector. This 100-dimensional vector is a highly abstract feature; ultimately, if the two postures are very similar, the Euclidean distance of the two 100-dimensional vectors output by the network is very small, otherwise, the Euclidean distance is very large.

[0121] Our neural network has 4 layers in total. The number of nodes in each layer from input to output is 32->64->128->100, that is, a 32-dimensional vector is input and a 100-dimensional vector is output. The Euclidean distance calculation formula in n-dimensional space is mapped to 100 dimensions in the present invention, that is, n=100:

[0122]

[0123] A Euclidean distance threshold T is obtained based on the standard posture recognition model, and the threshold T is used to determine whether the user posture and the standard posture are of the same type; if the Euclidean distance of the scoring result output by the standard posture recognition model is greater than or equal to the threshold T, the user posture and the standard posture are of the same type; if the Euclidean distance of the scoring result output by the standard posture recognition model is less than or equal to the threshold T, the user posture and the standard posture are of different types;

[0124] The Euclidean distance is converted into a familiarity score between the user posture and the standard posture, and the user posture with the highest familiarity score is obtained as the scoring result. Specifically, if the Euclidean distance between two postures exceeds the threshold T, they are considered to be of different types; otherwise, they are considered to be of the same model. For each threshold T, an ROC curve can be drawn, such as Figure 3As shown, the area under the ROC curve, called AUC, is a value of 0-1. The larger the AUC, the better the model performance. Find an optimal threshold T-best so that the AUC is maximized on the test set. If the AUC is maximized, the model will judge two postures that originally belong to the same category as the same category as much as possible, and at the same time, it will try to misjudge two postures that do not belong to the same category as the same category as little as possible. After obtaining the optimal distance threshold T-best, we set a critical score according to actual business needs, such as 40 points, indicating that at this time, the model believes that the two postures are just at the critical point of similarity and dissimilarity. Then the mapping relationship is as follows: when the actual distance t is in the interval [0, T-best], the similarity score s is [100, 40]; when the actual distance t is (T-best, infinity), the similarity score s is (40, 0). In this embodiment, the threshold T is 40.

[0125] Both the standard posture and the user posture include 16 skeletal key points, each of which corresponds to a two-dimensional position coordinate. The 16 skeletal key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella.

[0126] Embodiment 3

[0127] Step 1 is based on Example 1 and further applied to the somatosensory game. For parkour games, we set a little person or little animal simulating the user. For the game, some obstacles that the little person or little animal simulating the user needs to avoid will be set on the road. The little person or little animal needs to jump or tilt the body to the left / right to avoid them. Correspondingly, we correspond the user's standing to the little person or little animal's automatic forward walking, the user's left / right waist twisting to the little person or little animal's left / right body tilting, and the user's in-situ jumping to the little person or little animal's jumping. In addition, some other actions can be set, such as corresponding the user's high leg raising to the accelerated running of the little person or little animal. The user follows the first fitness video in the target fitness area of ​​the fitness mirror.

[0128] The first fitness video uses standing, twisting the waist left and right, jumping on the spot and raising the legs high as the first standard posture;

[0129] Obtain a second time period in which each first standard posture appears in the first fitness video;

[0130] In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, extracts features of the target fitness area, and obtains the first user postures corresponding to standing, twisting the waist left and right, jumping on the spot, and raising the legs high; if the first user posture obtained in the second time period has different actions, it means that the user is following, that is, the user is doing fitness training; if the first user posture is not obtained in the second time period, or the first user posture has no action, the user is not following, that is, the user is not doing fitness training. In the second time period, the user's actions are recognized, but no scoring is performed in this process.

[0131] Step 2 In the second fitness video, standing, twisting the waist left and right, jumping on the spot and raising the legs high are used as the second standard posture;

[0132] Obtain a first time period in which each second standard posture appears in the second fitness video;

[0133] During the first time period, the fitness mirror recognizes the user's movements in the target fitness area, extracts features of the target fitness area, and obtains second user postures corresponding to standing, twisting the waist left and right, jumping on the spot, and raising the legs high. Based on Example 1, each second user posture is scored, and whether the user follows the second fitness video for fitness training is determined based on the scoring results.

[0134] Step 3 In the first fitness video, standing, twisting the waist left and right, jumping on the spot and raising the legs high are used as the first standard posture;

[0135] Obtain a second time period in which each first standard posture appears in the first fitness video;

[0136] During the second time period, the fitness mirror recognizes the user's movements in the target fitness area, extracts features of the target fitness area, and obtains the first user postures corresponding to standing, twisting the waist left and right, jumping on the spot, and raising the legs high; based on Example 1, each first user standard posture is scored, and the scoring results are fed back to the user.

[0137] In this embodiment, since the fitness video is a game video, the movement of the little people or animals in the game video can be controlled through different standard postures. Therefore, when the present invention recognizes the user's actions, the fitness training method of the present invention can not only be used to recognize the actions and evaluate and score the actions, but also based on the standard posture, the user posture can also control the movement of the little people or animals in the fitness video, among which standing is to control the animal to walk forward, twisting the waist left and right is to control the little people or animals to tilt their bodies to the left or right, jumping on the spot is to control the animal to jump, and raising the legs high is to control the little people or animals to run faster.

[0138] Embodiment 3

[0139] Please refer to Figure 2 , Figure 2 Schematic diagram of the composition of a fitness training system based on a fitness device. Embodiment 2 of the present invention provides a fitness training system based on a fitness device. Based on Embodiment 2, the system includes:

[0140] An acquisition module is used to acquire and process fitness videos;

[0141] A recognition module, used to identify the target fitness area of ​​the fitness video, extract features of the target fitness area, and obtain the user posture;

[0142] A comparison module, used to compare the user's posture with a preset standard posture to obtain a comparison result;

[0143] The judgment module is used to judge whether the user is performing fitness training or whether the action meets the standard according to the comparison result.

[0144] Embodiment 4

[0145] A fourth embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the fitness training method based on the fitness device when executing the computer program.

[0146] The processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0147] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the fitness training device based on the fitness device in the invention by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0148] Embodiment 5

[0149] A fifth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the fitness training method based on the fitness device are implemented.

[0150] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0151] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0152] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A fitness training method based on a fitness device, characterized in that: include: Acquiring a first user's posture according to a first fitness video, and determining whether the user is performing fitness training; Acquire a second user posture according to the second fitness video, score the second user posture, and determine whether the user follows the second fitness video for fitness training according to the scoring result; Acquire a first user's posture according to a first fitness video, score the first user's posture, and feed back the scoring result to the user; Acquiring the first user's posture according to the first fitness video and determining whether the user is performing fitness training specifically includes: Get the first fitness video; identifying a target fitness area of ​​a first fitness video, extracting features of the target fitness area, and obtaining a first user posture; determining, according to the first user posture, whether the user is performing fitness training; The method further comprises: obtaining a second user posture according to the second fitness video, scoring the second user posture, and judging whether the user follows the second fitness video for fitness training according to the scoring result. Specifically, the method comprises: Presetting a second standard posture according to a second fitness video; identifying a target fitness area of ​​the second fitness video, performing feature extraction on the target fitness area, and obtaining a second user posture; Comparing the second standard posture with the second user posture, obtaining a familiarity score of the second user posture based on the second standard posture, and judging whether the user follows the second fitness video for fitness training according to the score result; Identifying a target fitness area of ​​the second fitness video, extracting features of the target fitness area, and obtaining a second user posture, specifically includes: Obtain a first time period in which a second standard posture appears in a second fitness video; Obtaining a video segment corresponding to the second fitness video within the first time period, performing frame processing on the video segment, and obtaining frame images at a plurality of consecutive moments corresponding to the video segment; identifying a target fitness area of ​​a second fitness video within a first time period, extracting features of the target fitness area, and obtaining a plurality of second user postures corresponding to the frame images one by one; Comparing the corresponding plurality of frame images with the plurality of second user gestures to obtain a familiarity score of each second user gesture based on the corresponding frame image; The second user posture with the highest familiarity score is obtained as the scoring result.

2. The fitness training method based on the fitness device according to claim 1, characterized in that: Acquiring a first user posture according to a first fitness video and scoring the first user posture specifically includes: Obtain a first fitness video, and preset a first standard posture according to the first fitness video; identifying a target fitness area of ​​a first fitness video, extracting features of the target fitness area, and obtaining a first user posture; The first standard posture and the first user posture are compared to obtain a familiarity score of the first user posture based on the first standard posture, and the score result is fed back to the user.

3. The fitness training method based on the fitness device according to claim 2, characterized in that: Identifying a target fitness area of ​​the first fitness video, extracting features of the target fitness area, and obtaining a first user posture, specifically includes: Acquire a second time period in which the first standard posture appears in the first fitness video; identifying a target fitness area of ​​the first fitness video within a second time period, extracting features of the target fitness area, and obtaining a plurality of first user postures; Comparing the first standard posture with the plurality of first user postures to obtain a familiarity score of each first user posture based on the first standard posture; The first user posture with the highest familiarity score is obtained as the scoring result.

4. The fitness training method based on the fitness device according to claim 2 or 3, characterized in that: Comparing the first standard posture with the first user posture specifically includes: Train the twin neural network model to obtain a trained standard posture recognition model; Inputting the first user posture and the first standard posture into a standard posture recognition model to obtain a familiarity score; If the scoring result is greater than or equal to the scoring threshold, the first standard posture and the first user posture are of the same category, and the user action meets the standard; If the scoring result is less than the scoring threshold, the first standard posture and the first user posture are not of the same category, and the user action does not meet the standard.

5. A fitness training system based on a fitness device, characterized in that: The system is used to execute the fitness training method based on the fitness device according to any one of claims 1 to 4, and the system comprises: An acquisition module is used to acquire and process fitness videos; A recognition module, used to identify the target fitness area of ​​the fitness video, extract features of the target fitness area, and obtain the user posture; A comparison module, used to compare the user's posture with a preset standard posture to obtain a comparison result; The judgment module is used to judge whether the user is performing fitness training or whether the action meets the standard according to the comparison result.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the fitness training method based on the fitness device as claimed in any one of claims 1 to 4 are implemented.

7. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the fitness training method based on a fitness device as claimed in any one of claims 1 to 4 are implemented.

Images