A method and system for evaluating exercise effect based on APP screening function

Abstract

The present invention belongs to the field of computer vision technology, and specifically relates to a method and system for evaluating sports effects based on an APP screening function. The method includes: playing and displaying a standard fitness video recorded by an APP, and collecting video data of users exercising following the standard fitness video in multiple dimensions; performing frame processing on the collected multi-dimensional video data, performing image recognition processing after obtaining video image frames of all dimensions, determining the start and end video image frames of each fitness action and the key feature points in each video image frame; determining the motion feature parameters of the fitness action in each video image frame based on the key feature points, and inputting the motion feature parameters into the APP to compare with the standard motion parameters synchronized in the standard fitness video to determine the user's sports effect data; scoring the user's sports effect after completing a stage according to the sports effect data, and displaying the scoring data to evaluate the user's offline sports actions.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and specifically relates to a method and system for evaluating sports effects based on an APP screening function. Background Art

[0002] With economic development and the improvement of living standards, people are paying more and more attention to physical exercise. When doing fitness exercises, in addition to the guidance of professional coaches, people are more likely to follow fitness software to exercise. Because various fitness apps can usually only provide users with some professional sports courses for guidance, including fitness training, technical guidance, exercise frequency and exercise volume guidance.

[0003] However, there is no relevant evaluation and inspection for the actual exercise effect of users offline. In the process of following the sports course for fitness training, although it is possible to play and train according to the standard fitness video recorded by the APP, with the control of voice and time rhythm, it has a good guiding effect. However, when guiding and encouraging users to follow the standard fitness video for fitness training, if the user's fitness training movements are not standard or standardized, the expected exercise effect cannot be achieved. Instead, the lack of standardization of the movements will cause unnecessary damage to the user's body. For this reason, it is necessary to provide a method and system for evaluating the exercise effect based on the APP screening function to ensure that users follow the standard fitness. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for evaluating exercise effects based on an APP screening function, so as to at least solve the technical problem in the related art that fitness training only follows the demonstration of a standard video to guide users to train, but cannot evaluate the user's offline exercise movements to ensure that the user's exercise can achieve the desired exercise effect.

[0005] The present invention is implemented by the following technical solutions:

[0006] A method for evaluating exercise effects based on an APP screening function, the method comprising:

[0007] Play and display the standard fitness videos recorded by the APP, and collect video data of users exercising following the standard fitness videos in multiple dimensions;

[0008] Perform frame processing on the collected multi-dimensional video data, obtain video image frames of all dimensions, perform image recognition processing, determine the start and end video image frames of each fitness action and the key feature points in each video image frame;

[0009] Determine the motion characteristic parameters of the fitness action in each video image frame based on the key characteristic points, and input the motion characteristic parameters into the APP for comparison with the standard motion parameters synchronized in the standard fitness video to determine the user's exercise effect data;

[0010] The exercise effect of the user after completing a stage of a standard fitness video is scored according to the exercise effect data, and the scoring data is displayed.

[0011] Optionally, the multi-dimensionally collected video data of the user exercising following the standard fitness video includes at least one set of video data from the same perspective as the standard fitness video being played, and the video data is shot by a collection device bound to the APP login device, wherein the collection device includes a digital camera, a video camera or a webcam.

[0012] Optionally, determining the start and end video image frames of each fitness action includes:

[0013] The video data of the user exercising following the standard fitness video collected in multiple dimensions are framed according to the dimensions to obtain video image frames arranged in time sequence; the key point data of the user's body and bones are identified using human skeleton key point detection technology; the key point data of the video image frame after each dimension is framed is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP; the video image frames belonging to the same type of standard action are divided according to the start and end video image frames, and the start and end video image frames of each fitness action are determined.

[0014] Optionally, human skeleton key point detection technology is used to identify key point data of the user's body and skeleton, including: performing 3D target detection from video image frames based on the MobilePose lightweight network, and identifying human key points in video image frames; generating a human skeleton of a single person posture based on the identified human key points, and exporting the human skeleton data to generate key point data of the user's body and skeleton.

[0015] Optionally, the key point data of the video image frame after each dimension is divided into frames is input into a K-nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of the standard fitness video in the APP, including:

[0016] Based on the key point data of the user's body and skeleton, different characteristic parts of the human body are abstracted into a set number of characteristic points, and the characteristic points form a human skeleton to reflect the user's human posture at the current moment, and obtain the joint angle values ​​of the human skeleton in the corresponding video image frame as the motion characteristic parameters of the fitness action in the video image frame;

[0017] The standard motion parameters synchronized with the standard fitness videos in the APP are used as comparison templates, and compared with the motion feature parameters of the fitness movements in the input user video image frames, to judge the user's training movements, the corresponding standard movements, and the difference in angle values ​​with the standard motion parameters, and analyze whether the trainee's movements are standard to determine the user's exercise effect data.

[0018] Optionally, when the human skeleton composed of the feature points reflects the human posture of the user at the current moment and the joint angle value of the human skeleton in the corresponding video image frame is obtained, the length between two feature points in the human skeleton is obtained by calculating the Euclidean distance, and the angle value of the angle between the joints of the human action is calculated using the cosine theorem;

[0019] When calculating the length between two feature points, the one-dimensional distance data from the human skeleton to the image frame in the video image frame is extracted, and the minimum value, the second minimum value, the maximum value, and the second maximum value are calculated as the positions of different parts of the limbs. The spacing between the two feature points is calculated by the Euclidean distance of the two feature points.

[0020] The height of the human skeleton, the angle between nodes and the inclination angle relative to the bottom edge of the image are calculated according to the distribution of the feature points, and the angle value of the angle between joints of the human body movement is calculated according to the cosine theorem.

[0021] Optionally, the key point data of the video image frame after each dimension is divided into frames is input into a K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of the standard fitness video in the APP, which also includes:

[0022] Based on the key point data of the video image frame, the user's motion behavior is recognized, and the key point data is compared with the database loaded in the APP to classify the user's motion behavior. The database stores a data set that has been pre-segmented and classified. Each data set corresponds to a type of motion and contains standard human skeleton information and motion feature parameters.

[0023] Behavior detection is performed based on the key point data of each video image frame, and the target action is matched from the database loaded in the APP based on the key point data.

[0024] Optionally, the exercise effect of the user after completing a stage of the standard fitness video is scored according to the exercise effect data, and before the scoring data is displayed, it also includes: mapping the standard actions corresponding to the standard motion parameters in the APP synchronized with the standard fitness video to the video image frames, and highlighting the areas that do not overlap with the standard actions, and marking the differences in motion feature parameters in the video image frames; combining the processed video image frames to generate a processed exercise effect evaluation video, saving and displaying it.

[0025] The present invention also includes a sports effect evaluation system based on the APP screening function, including: a video acquisition unit, which is used to play and display a standard fitness video recorded based on the APP, and to collect video data of users exercising following the standard fitness video in multiple dimensions; a key feature point extraction unit, which is used to perform frame processing on the collected multi-dimensional video data, and perform image recognition processing after obtaining video image frames of all dimensions, and determine the start and end video image frames of each fitness action and the key feature points in each video image frame; a parameter comparison unit, which is used to determine the motion feature parameters of the fitness action in each video image frame based on the key feature points, and input the motion feature parameters into the APP for comparison with the standard motion parameters synchronized with the standard fitness video to determine the user's motion effect data; an effect scoring unit, which is used to score the user's motion effect of completing a stage of the standard fitness video according to the motion effect data, and display the scoring data.

[0026] Optionally, it also includes an action mapping unit, which is used to map the standard actions corresponding to the standard motion parameters synchronized with the standard fitness video in the APP to the video image frames, highlight the areas that do not overlap with the standard actions, and mark the differences in motion feature parameters in the video image frames.

[0027] The present invention also includes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the exercise effect evaluation method based on the APP screening function when loading and executing the computer-readable instructions.

[0028] The present invention also includes a storage medium storing computer-readable instructions, which, when loaded and executed by a processor, implement the steps of a method for evaluating exercise effects based on an APP screening function.

[0029] The technical solution provided by the present invention has the following beneficial effects:

[0030] The exercise effect evaluation method and system based on the APP screening function provided by the present invention are based on the playback and display of standard fitness videos recorded by the APP, and multi-dimensionally collect video data of users exercising following the standard fitness videos; the collected multi-dimensional video data are frame-processed, and image recognition processing is performed after obtaining video image frames of all dimensions to determine the start and end video image frames of each fitness action and the key feature points in each video image frame; based on the key feature points, the motion feature parameters of the fitness action in each video image frame are determined, and the motion feature parameters are input into the APP for comparison with the standard motion parameters synchronized with the standard fitness video to determine the user's exercise effect data; according to the motion effect data, the user's exercise effect after completing a stage of the standard fitness video is scored, and the scoring data is displayed. By identifying the user's fitness movements, comparing them with standard fitness training videos, and determining the user's fitness exercise effects, the purpose of providing real-time feedback on the user's offline exercise effects and providing feedback on the final exercise scores is achieved, thereby realizing interaction with the user's fitness training movements, and increasing the user's interaction frequency and interaction effect based on the display of exercise effects and scoring data, thereby increasing user interest and improving the technical effect of fitness training results. The present invention solves the technical problem in the related art that fitness training only follows the demonstration of standard videos to guide users to train, but cannot evaluate the user's offline exercise movements to ensure that the user's exercise can achieve the desired exercise effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 The present invention is a flowchart of a method for evaluating exercise effects based on an APP screening function according to an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of single-person target detection using a MobilePose lightweight network in a motion effect evaluation method based on an APP screening function according to an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of multi-person target detection using a MobilePose lightweight network in a motion effect evaluation method based on an APP screening function according to an embodiment of the present invention.

[0035] Figure 4 The system block diagram of the exercise effect evaluation system based on the APP screening function in one embodiment of the present invention.

[0036] Figure 5It is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] It should be noted that all technical and scientific terms used in this document have the same meanings as those generally understood by technicians in the technical field of this application; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0039] According to an embodiment of the present invention, a method embodiment of an exercise effect evaluation method based on an APP screening function is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0041] Since there is no relevant evaluation and inspection on the actual exercise effects of users offline, in the process of following sports courses for fitness training, although it is possible to train while playing the standard fitness videos recorded by the APP, combined with voice and time rhythm control, it has a good guiding effect. However, when guiding and encouraging users to follow the standard fitness videos for fitness training, if the user's fitness training movements are not standard or standardized, the expected exercise effect cannot be achieved, but it will have the opposite fitness effect, causing unnecessary damage to the user's body.

[0042] In view of this, an embodiment of the present invention provides an exercise effect evaluation method and system based on an APP screening function, which determines the user's exercise effect by identifying the user's fitness movements and comparing them with standard fitness training videos, thereby achieving the purpose of providing real-time feedback on the user's offline exercise effects and providing feedback on the final exercise scores, thereby realizing interaction with the user's fitness training movements, and improving the user's interaction frequency and interaction effect based on the display of exercise effects and scoring data, thereby improving user interest and improving the technical effect of fitness training results, so as to at least solve the technical problem that fitness training in related technologies only follows the demonstration of standard videos to guide users to train, and cannot evaluate the user's offline exercise movements to ensure that the user's exercise can achieve the desired exercise effect.

[0043] The embodiment of the present application provides a method for evaluating the sports effect based on the APP screening function, which can be applied to a mobile terminal or server with an APP installed, and can be specifically applied to an APP installed on a terminal device, or a system framework composed of a server and a terminal device, wherein the APP and the terminal device, and the server and the terminal device are connected via a wired or wireless network. Various APP client applications can also be installed on the terminal device, such as image processing applications, beauty applications, instant messaging software, etc.

[0044] Specifically, the embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0045] See also Figure 1 As shown, Figure 1 A flowchart showing a process of an embodiment of a method for evaluating exercise effects based on an APP screening function according to an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown. In an embodiment of the present invention, this embodiment provides a method for evaluating exercise effects based on an APP screening function, comprising the following steps:

[0046] S10, playing and displaying a standard fitness video recorded based on the APP, and collecting video data of the user exercising following the standard fitness video in multiple dimensions;

[0047] In this embodiment, the multi-dimensionally collected video data of the user exercising following the standard fitness video includes at least one group of video data from the same perspective as the standard fitness video being played and displayed, and the video data is shot by a collection device bound to the APP login device, wherein the collection device includes a digital camera, a video camera or a camera.

[0048] During the acquisition, in addition to acquiring video data from the same perspective as the standard fitness video being played and displayed, at least one group of video data from other dimensional perspectives may also be included. Based on the comparison of the video data from the same perspective as the standard fitness video being played and displayed with the standard fitness training video, video data from other dimensional perspectives may also be used for verification.

[0049] S20, performing frame processing on the collected multi-dimensional video data, performing image recognition processing after obtaining video image frames of all dimensions, and determining the start and end video image frames of each fitness action and key feature points in each video image frame.

[0050] In this embodiment, the multi-dimensional video data is subjected to frame processing, and the collected multi-dimensional video data can be subjected to video frame processing at any time period according to different dimensions to obtain video image frames.

[0051] In one embodiment of the present invention, when multi-dimensional video data is processed by frame division, the frame rate of the acquired video data is converted to obtain frame-processed video frame data, video image frames are acquired in a time period, a video image frame data set is obtained, and the video image frames are processed one by one according to the frame division time sequence. For example, video image frames are acquired at a frame rate of 1 frame per second, and the video image frames are sorted in time sequence as a video image frame data set.

[0052] S30. Determine motion characteristic parameters of the fitness action in each video image frame based on the key feature points, and input the motion characteristic parameters into the APP for comparison with standard motion parameters synchronized with the standard fitness video to determine the user's exercise effect data.

[0053] S40, scoring the exercise effect of the user after completing a stage of the standard fitness video according to the exercise effect data, and displaying the scoring data.

[0054] The sports effect evaluation method based on the APP screening function of the embodiment of the present invention determines the user's sports effect data through the above operation; the sports effect of the user in completing a stage of the standard fitness video is scored according to the sports effect data, and the scoring data is displayed. By identifying the user's fitness movements and comparing them with the standard fitness training videos, the sports effect of the user's fitness is determined, and the purpose of real-time feedback of the user's offline sports effect and feedback of the final sports score is achieved, thereby realizing interaction with the user's fitness training movements, and improving the user's interaction frequency and interaction effect according to the display of the sports effect and scoring data, thereby improving the user's fun and improving the technical effect of the fitness training results. The present invention solves the technical problem that the fitness training in the related art only follows the demonstration of the standard video to exercise and guide the user to train, and cannot evaluate the user's offline sports movements to ensure that the user's exercise can achieve the desired sports effect.

[0055] In some embodiments of the present invention, in step S20, determining the start and end video image frames of each fitness action includes:

[0056] The video data of the user exercising following the standard fitness video collected in multiple dimensions are framed according to the dimensions to obtain video image frames arranged in time sequence; the key point data of the user's body and bones are identified using human skeleton key point detection technology; the key point data of the video image frame after each dimension is framed is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP; the video image frames belonging to the same type of standard action are divided according to the start and end video image frames, and the start and end video image frames of each fitness action are determined.

[0057] In this embodiment, see Figure 1 and Figure 2 As shown, human skeleton key point detection technology is used to identify the key point data of the user's body and skeleton, including: performing 3D target detection from video image frames based on the MobilePose lightweight network, identifying the key points of the human body in the video image frames; generating a human skeleton of a single person posture according to the identified human key points, and exporting the human skeleton data to generate the key point data of the user's body and skeleton.

[0058] In this embodiment, when recognizing the user's offline behavioral actions, the MobilePose lightweight network is used to estimate the key points of the human body. For the user's running, jumping, squatting, walking, jumping and various stretching exercises during offline exercise, a lightweight neural network is used to identify the user's behavior. The video data can be divided into single-frame images or continuous frame images for recognition.

[0059] Based on the MobilePose lightweight neural network, the human skeleton information of a single person's posture is recognized. When processing video image frames, the interference of complex factors such as different user body shapes, clothing, and background can be effectively eliminated. The lightweight network can be used to identify the key points of the human body. Secondly, see Figure 1 and Figure 3 As shown, you can also detect the position of people by adding yoloV5, and then recognize the skeleton to achieve multi-person posture recognition.

[0060] When applying, when skeleton extraction is performed based on the MobilePose lightweight network, it can be installed through the APP application or deployed on the mobile terminal to implement a lightweight version of the application, which can realize single-person behavior recognition, action matching and other functions on the mobile terminal.

[0061] Before application, you can also obtain the data of the human skeleton, or the coordinate points or vectors of the human skeleton for training, using machine learning methods such as SVM, or a fully connected network. After the human skeleton is extracted, make a separate mask map and do a simple convolutional network training. The source code of the MobilePose lightweight network includes ResNet18, MobileNetV2, ShuffleNetV2, and SqueezeNet1.1 lightweight networks.

[0062] In some embodiments of the present invention, AlphaPose can be used to first detect the human body and then obtain key points and skeletons, or PoseC3D, a skeletal behavior recognition framework based on 3D-CNN, can be used to use a 2D human skeleton heat map stack as input to achieve better recognition results.

[0063] The key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP, including:

[0064] Based on the key point data of the user's body and skeleton, different characteristic parts of the human body are abstracted into a set number of characteristic points, and the characteristic points form a human skeleton to reflect the user's human posture at the current moment, and obtain the joint angle values ​​of the human skeleton in the corresponding video image frame as the motion characteristic parameters of the fitness action in the video image frame;

[0065] The standard motion parameters synchronized with the standard fitness videos in the APP are used as comparison templates, and compared with the motion feature parameters of the fitness movements in the input user video image frames, to judge the user's training movements, the corresponding standard movements, and the difference in angle values ​​with the standard motion parameters, and analyze whether the trainee's movements are standard to determine the user's exercise effect data.

[0066] When comparing movements, the key point data of the user's body and skeleton are abstracted into N feature points according to the different characteristic parts of the human body. The human skeleton composed of these feature points can reflect the posture of the human body at the moment, and the angle values ​​between specific joints can be used to judge the accuracy of a person's movements. Among them, the standard movement picture is processed to obtain the joint angle value of the human skeleton in the picture as a comparison template. The action picture of the trainee to be judged is input to obtain the joint angle value. The angle values ​​of the two are analyzed to determine whether the trainee's movement is standard, thereby achieving the effect of movement comparison.

[0067] In some embodiments of the present invention, when the human skeleton composed of the feature points reflects the human posture of the user at the current moment and the joint angle value of the human skeleton in the corresponding video image frame is obtained, the length between two feature points in the human skeleton is obtained by calculating the Euclidean distance, and the angle value of the angle between the joints of the human action is calculated using the cosine theorem;

[0068] When calculating the length between two feature points, the one-dimensional distance data from the human skeleton to the image frame in the video image frame is extracted, and the minimum value, the second minimum value, the maximum value, and the second maximum value are calculated as the positions of different parts of the limbs. The spacing between the two feature points is calculated by the Euclidean distance of the two feature points.

[0069] The height of the human skeleton, the angle between nodes and the inclination angle relative to the bottom edge of the image are calculated according to the distribution of the feature points, and the angle value of the angle between joints of the human body movement is calculated according to the cosine theorem.

[0070] In this embodiment, the key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of the standard fitness video in the APP, which also includes:

[0071] Based on the key point data of the video image frame, the user's motion behavior is recognized, and the key point data is compared with the database loaded in the APP to classify the user's motion behavior. The database stores a data set that has been pre-segmented and classified. Each data set corresponds to a type of motion and contains standard human skeleton information and motion feature parameters.

[0072] Behavior detection is performed based on the key point data of each video image frame, and the target action is matched from the database loaded in the APP based on the key point data.

[0073] In an embodiment of the present invention, the exercise effect of the user after completing a stage of a standard fitness video is scored according to the exercise effect data, and before the scoring data is displayed, it also includes: mapping the standard actions corresponding to the standard motion parameters in the APP synchronized with the standard fitness video to the video image frames, and highlighting the areas that do not overlap with the standard actions, and marking the differences in motion feature parameters in the video image frames; combining the processed video image frames to generate a processed motion effect evaluation video, saving and displaying it.

[0074] like Figure 4 As shown, Figure 4 It is a structural block diagram of a motion effect evaluation system based on an APP screening function provided in an embodiment of the present application. The motion effect evaluation system based on the APP screening function can be applied to the user's offline motion for screening and motion effect evaluation, and can execute the method of correction guidance based on user motion feature extraction in any of the above method embodiments. Specifically, in one embodiment of the present invention, a motion effect evaluation system based on an APP screening function is provided, including a video acquisition unit 100, a key feature point extraction unit 200, a parameter comparison unit 300, an effect scoring unit 400, and an action mapping unit 500.

[0075] The video acquisition unit 100 is used to play and display a standard fitness video recorded based on the APP, and to acquire video data of a user exercising following the standard fitness video in multiple dimensions.

[0076] The key feature point extraction unit 200 is used to perform frame processing on the collected multi-dimensional video data, perform image recognition processing after obtaining video image frames of all dimensions, and determine the start and end video image frames of each fitness action and the key feature points in each video image frame.

[0077] The parameter comparison unit 300 is used to determine the motion characteristic parameters of the fitness action in each video image frame based on the key feature points, and input the motion characteristic parameters into the APP to compare with the standard motion parameters synchronized in the standard fitness video to determine the user's exercise effect data.

[0078] The effect scoring unit 400 is used to score the exercise effect of the user after completing a stage of the standard fitness video according to the exercise effect data, and display the scoring data.

[0079] The action mapping unit 500 is used to map the standard action corresponding to the standard motion parameters synchronized with the standard fitness video in the APP to the video image frame, highlight the area that does not overlap with the standard action, and mark the difference in motion feature parameters in the video image frame.

[0080] In this embodiment, the exercise effect evaluation system based on the APP screening function may also adopt the steps of the aforementioned exercise effect evaluation method based on the APP screening function during execution.

[0081] The present invention can identify the user's fitness movements, compare them with standard fitness training videos, determine the user's fitness exercise effect, achieve the purpose of real-time feedback of the user's offline exercise effect, and feedback the final exercise score, thereby realizing interaction with the user's fitness training movement, and improving the user's interaction frequency and interaction effect according to the display of the exercise effect and scoring data, thereby improving the user's fun, and improving the technical effect of the fitness training result. The present invention solves the technical problem that fitness training in the related art only follows the demonstration of the standard video to exercise, guides the user to train, and cannot evaluate the user's offline exercise to ensure that the user's exercise can achieve the desired exercise effect. Among them, the exercise effect evaluation system based on the APP screening function adopts the steps of a method for evaluating the exercise effect based on the APP screening function as described above during execution. Therefore, the operation process of the exercise effect evaluation system based on the APP screening function in this embodiment is no longer introduced in detail.

[0082] See also Figure 5 As shown, in one embodiment of the present invention, a computer device 1000 is further provided, including a memory 1001 and a processor 1002. The memory 1001 stores computer-readable instructions. When the processor 1002 loads and executes the computer-readable instructions, the steps in the embodiment of the exercise effect evaluation method based on the APP screening function are implemented, including:

[0083] Play and display the standard fitness videos recorded by the APP, and collect video data of users exercising following the standard fitness videos in multiple dimensions;

[0084] Perform frame processing on the collected multi-dimensional video data, obtain video image frames of all dimensions, perform image recognition processing, determine the start and end video image frames of each fitness action and the key feature points in each video image frame;

[0085] Determine the motion characteristic parameters of the fitness action in each video image frame based on the key characteristic points, and input the motion characteristic parameters into the APP for comparison with the standard motion parameters synchronized in the standard fitness video to determine the user's exercise effect data;

[0086] The exercise effect of the user after completing a stage of a standard fitness video is scored according to the exercise effect data, and the scoring data is displayed.

[0087] Among them, those skilled in the art can understand that the computer device 1000 here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, embedded devices, etc. The computer device 1000 can be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. The computer device 1000 can interact with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0088] In addition, some embodiments of the present invention also include a storage medium having a program for executing the method described in this specification on a computer, on which computer-readable instructions are stored. When the computer-readable instructions are loaded and executed by the processor 1002, the steps in the above-mentioned method embodiments are implemented. Examples of the computer-readable recording medium include hardware devices specially constructed to store and execute program commands: magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, ROMs, RAMs, flash memories, etc. Examples of program commands may include: machine language codes written by a compiler and high-level language codes executed by a computer using an interpreter, etc.

[0089] The processor 1002 may be a central processing unit, a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. In this embodiment, the processor 1002 is used to run computer-readable instructions or process data stored in the memory 1001, such as computer-readable instructions for running the image-based health status recognition method.

[0090] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory 1001, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0091] In summary, the exercise effect evaluation method and system based on the APP screening function provided by the present invention are based on the playback and display of standard fitness videos recorded by the APP, and multi-dimensionally collect video data of users exercising following the standard fitness videos; the collected multi-dimensional video data is frame-processed, and image recognition processing is performed after obtaining video image frames of all dimensions to determine the start and end video image frames of each fitness action and the key feature points in each video image frame; based on the key feature points, the motion feature parameters of the fitness action in each video image frame are determined, and the motion feature parameters are input into the APP for comparison with the standard motion parameters synchronized with the standard fitness video to determine the user's exercise effect data; according to the motion effect data, the user's exercise effect after completing a stage of the standard fitness video is scored, and the scoring data is displayed. By identifying the user's fitness movements, comparing them with standard fitness training videos, and determining the user's fitness exercise effects, the purpose of providing real-time feedback on the user's offline exercise effects and providing feedback on the final exercise scores is achieved, thereby realizing interaction with the user's fitness training movements, and increasing the user's interaction frequency and interaction effect based on the display of exercise effects and scoring data, thereby increasing user interest and improving the technical effect of fitness training results. The present invention solves the technical problem in the related art that fitness training only follows the demonstration of standard videos to guide users to train, but cannot evaluate the user's offline exercise movements to ensure that the user's exercise can achieve the desired exercise effect.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the effect of exercise based on the APP screening function, characterized in that: include: Play and display standard fitness videos recorded by the APP, and collect video data of users exercising following the standard fitness videos in multiple dimensions; Perform frame processing on the collected multi-dimensional video data, obtain video image frames of all dimensions, perform image recognition processing, determine the start and end video image frames of each fitness action and the key feature points in each video image frame; Determine the motion characteristic parameters of the fitness action in each video image frame based on the key characteristic points, and input the motion characteristic parameters into the APP for comparison with the standard motion parameters synchronized in the standard fitness video to determine the user's exercise effect data; Scoring the exercise effect of the user after completing a stage of the standard fitness video according to the exercise effect data, and displaying the scoring data; Wherein, determining the start and end video image frames of each fitness action includes: The video data of the user exercising following the standard fitness video collected in multiple dimensions is frame-processed according to the dimensions to obtain video image frames arranged in time sequence; Use human skeleton key point detection technology to identify the key point data of the user's body and skeleton; Input the key point data of the video image frame after each dimension is divided into frames into the K nearest neighbor algorithm, and find the standard action and standard motion parameter closest to each video image frame in the standard motion parameter database of standard fitness videos in the APP; Divide the video image frames belonging to the same type of standard movements according to the start and end video image frames, and determine the start and end video image frames of each fitness movement; Among them, the human skeleton key point detection technology is used to identify the key point data of the user's body and skeleton, including: Based on the MobilePose lightweight network, 3D object detection is performed from video image frames to identify key points of the human body in video image frames; Generate a human skeleton of a single person's posture based on the recognized human key points, and generate key point data of the user's body and bones after exporting the human skeleton data; The key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP, including: Based on the key point data of the user's body and skeleton, different characteristic parts of the human body are abstracted into a set number of characteristic points, and the characteristic points form a human skeleton to reflect the user's human posture at the current moment, and obtain the joint angle values ​​of the human skeleton in the corresponding video image frame as the motion characteristic parameters of the fitness action in the video image frame; The standard motion parameters synchronized with the standard fitness video in the APP are used as comparison templates, and compared with the motion characteristic parameters of the fitness movements in the input user video image frames, to judge the user's training movements, the corresponding standard movements, and the difference in angle values ​​with the standard motion parameters, and analyze whether the trainer's movements are standard, and determine the user's exercise effect data; Wherein, when the human skeleton composed of the feature points reflects the human posture of the user at the current moment and the joint angle value of the human skeleton in the corresponding video image frame is obtained, the length between two feature points in the human skeleton is obtained by calculating the Euclidean distance, and the angle value of the angle between the joints of the human action is calculated using the cosine theorem; When calculating the length between two feature points, the one-dimensional distance data from the human skeleton to the image frame in the video image frame is extracted, and the minimum value, the second minimum value, the maximum value, and the second maximum value are calculated as the positions of different parts of the limbs. The spacing between the two feature points is calculated by the Euclidean distance of the two feature points. The height of the human skeleton, the angle between the nodes and the inclination angle relative to the bottom edge of the image are calculated according to the distribution of the feature points, and the angle value of the angle between the joints of the human body movement is calculated according to the cosine theorem; Among them, the key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP, which also includes: Based on the key point data of the video image frame, the user's motion behavior is recognized, and the key point data is compared with the database loaded in the APP to classify the user's motion behavior. The database stores a data set that has been pre-segmented and classified. Each data set corresponds to a type of motion and contains standard human skeleton information and motion feature parameters. Behavior detection is performed based on the key point data of each video image frame, and the target action is matched from the database loaded in the APP based on the key point data.

2. The method for evaluating the exercise effect based on the APP screening function according to claim 1, characterized in that: The multi-dimensionally collected video data of the user exercising following the standard fitness video includes at least one set of video data from the same perspective as the standard fitness video being played and displayed, and the video data is shot by a collection device bound to the APP login device, wherein the collection device includes a digital camera, a video camera or a webcam.

3. The method for evaluating the exercise effect based on the APP screening function according to claim 1, characterized in that: Scoring the exercise effect of the user after completing a stage of the standard fitness video according to the exercise effect data, and before displaying the scoring data, the method further includes: Map the standard movements corresponding to the standard motion parameters synchronized with the standard fitness videos in the APP to the video image frames, highlight the areas that do not overlap with the standard movements, and mark the differences in the motion feature parameters in the video image frames; The processed video image frames are combined to generate a processed motion effect evaluation video, which is saved and displayed.

4. A sports effect evaluation system based on APP screening function, characterized in that: include: A video acquisition unit, used to play and display standard fitness videos recorded by the APP, and to collect video data of users exercising following the standard fitness videos in multiple dimensions; A key feature point extraction unit is used to perform frame processing on the collected multi-dimensional video data, obtain video image frames of all dimensions, perform image recognition processing, and determine the start and end video image frames of each fitness action and the key feature points in each video image frame; A parameter comparison unit, used to determine the motion characteristic parameters of the fitness action in each video image frame based on the key characteristic points, and input the motion characteristic parameters into the APP to compare with the standard motion parameters synchronized in the standard fitness video to determine the user's exercise effect data; An effect scoring unit, used to score the exercise effect of the user after completing a stage of the standard fitness video according to the exercise effect data, and display the scoring data; Wherein, determining the start and end video image frames of each fitness action includes: The video data of the user exercising following the standard fitness video collected in multiple dimensions is frame-processed according to the dimensions to obtain video image frames arranged in time sequence; Use human skeleton key point detection technology to identify the key point data of the user's body and skeleton; Input the key point data of the video image frame after each dimension is divided into frames into the K nearest neighbor algorithm, and find the standard action and standard motion parameter closest to each video image frame in the standard motion parameter database of standard fitness videos in the APP; Divide the video image frames belonging to the same type of standard movements according to the start and end video image frames, and determine the start and end video image frames of each fitness movement; Among them, the human skeleton key point detection technology is used to identify the key point data of the user's body and skeleton, including: Based on the MobilePose lightweight network, 3D object detection is performed from video image frames to identify key points of the human body in video image frames; Generate a human skeleton of a single person's posture based on the recognized human key points, and generate key point data of the user's body and bones after exporting the human skeleton data; The key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP, including: Based on the key point data of the user's body and skeleton, different characteristic parts of the human body are abstracted into a set number of characteristic points, and the characteristic points form a human skeleton to reflect the user's human posture at the current moment, and obtain the joint angle values ​​of the human skeleton in the corresponding video image frame as the motion characteristic parameters of the fitness action in the video image frame; The standard motion parameters synchronized with the standard fitness video in the APP are used as comparison templates, and compared with the motion characteristic parameters of the fitness movements in the input user video image frames, to judge the user's training movements, the corresponding standard movements, and the difference in angle values ​​with the standard motion parameters, and analyze whether the trainer's movements are standard, and determine the user's exercise effect data; Wherein, when the human skeleton composed of the feature points reflects the human posture of the user at the current moment and the joint angle value of the human skeleton in the corresponding video image frame is obtained, the length between two feature points in the human skeleton is obtained by calculating the Euclidean distance, and the angle value of the angle between the joints of the human action is calculated using the cosine theorem; When calculating the length between two feature points, the one-dimensional distance data from the human skeleton to the image frame in the video image frame is extracted, and the minimum value, the second minimum value, the maximum value, and the second maximum value are calculated as the positions of different parts of the limbs. The spacing between the two feature points is calculated by the Euclidean distance of the two feature points. The height of the human skeleton, the angle between the nodes and the inclination angle relative to the bottom edge of the image are calculated according to the distribution of the feature points, and the angle value of the angle between the joints of the human body movement is calculated according to the cosine theorem; Among them, the key point data of the video image frame after each dimension is divided into frames is input into the K nearest neighbor algorithm, and the standard action and standard motion parameter closest to each video image frame are found in the standard motion parameter database of standard fitness videos in the APP, which also includes: Based on the key point data of the video image frame, the user's motion behavior is recognized, and the key point data is compared with the database loaded in the APP to classify the user's motion behavior. The database stores a data set that has been pre-segmented and classified. Each data set corresponds to a type of motion and contains standard human skeleton information and motion feature parameters. Behavior detection is performed based on the key point data of each video image frame, and the target action is matched from the database loaded in the APP based on the key point data.

5. The sports effect evaluation system based on APP screening function according to claim 4 is characterized in that: Also includes: The action mapping unit is used to map the standard action corresponding to the standard motion parameters synchronized with the standard fitness video in the APP to the video image frame, highlight the area that does not overlap with the standard action, and mark the difference in motion feature parameters in the video image frame.

Images