A user motion evaluation method and system for motion image recognition

Abstract

The application provides a user motion evaluation method and system for motion image recognition, which comprises the following steps: collecting a motion image sequence of a user, obtaining a height information sequence and a plurality of motion image groups; obtaining a plurality of change degree sequences according to the plurality of motion image groups, calculating a plurality of first standard scores, and obtaining a plurality of key frame image sets; calculating a plurality of total change degrees according to the plurality of change degree sequences, configuring a plurality of motion standard recognition coefficients, performing motion standard recognition on the plurality of key frame image sets, and obtaining a plurality of second standard scores; and calculating a user motion score as a motion evaluation result according to the plurality of first standard scores, the plurality of second standard scores and a motion amount score. The application solves the technical problem that it is difficult to accurately evaluate the action standard degree and the motion amount of the user in the process of repeated motion in the height direction in the prior art, and achieves the technical effect of accurately evaluating the action standard degree and the motion amount of the user in the process of repeated motion in the height direction.

Description

Technical Field

[0001] This invention relates to the field of image analysis, and more particularly to a user motion assessment method and system for motion image recognition. Background Technology

[0002] Currently, for sports involving repetitive movements in the vertical direction, such as pull-ups, the assessment of exercise quality mainly relies on the subjective judgment of professional coaches or simple sensor technology to record the number of repetitions. A coach's subjective judgment lacks objective quantitative standards and is easily limited by personal experience and observational perspectives; while sensor technology can accurately record the number of repetitions, it cannot assess the standard and quality of the movement. Therefore, existing technologies suffer from the problem of accurately assessing the standard of movement and the amount of exercise performed by users. Summary of the Invention

[0003] This invention addresses the technical problem in the prior art of accurately assessing the standard of a user's movements and the amount of exercise during repetitive movements in the vertical direction by providing a user motion assessment method and system based on motion image recognition.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0005] In a first aspect, the present invention provides a user motion assessment method based on motion image recognition, comprising: during user motion, acquiring a sequence of motion images of the user; performing user height recognition on each motion image to obtain a height information sequence; grouping the motion images to obtain multiple motion image groups; performing variability analysis on the multiple motion image groups to obtain multiple variability sequences; calculating multiple first standard scores; and filtering based on the multiple variability sequences to obtain multiple keyframe image sets; calculating multiple total variability based on the multiple variability sequences; configuring multiple motion standard recognition coefficients; performing motion standard recognition on the multiple keyframe image sets to obtain multiple second standard scores; classifying and obtaining motion quantity scores based on the number of motion images within the multiple motion image groups; calculating multiple standard scores based on the multiple first standard scores and the multiple second standard scores; and combining the motion quantity scores to calculate the user's motion score as the motion assessment result.

[0006] Secondly, the present invention provides a user motion assessment system based on motion image recognition, comprising: an image processing module, configured to acquire a sequence of motion images of the user during the user's motion, perform user height recognition on each motion image to obtain a height information sequence, and group the motion images to obtain multiple motion image groups; a keyframe determination module, configured to perform variability analysis on the multiple motion image groups to obtain multiple variability sequences, calculate multiple first standard scores, and filter multiple keyframe image sets based on the multiple variability sequences; a standard recognition module, configured to calculate multiple total variability based on the multiple variability sequences, configure multiple motion standard recognition coefficients, perform motion standard recognition on the multiple keyframe image sets, and obtain multiple second standard scores; and a score calculation module, configured to classify and obtain motion intensity scores based on the number of motion images in the multiple motion image groups, calculate multiple standard scores based on the multiple first standard scores and the multiple second standard scores, and combine the motion intensity scores to calculate the user's motion score as the motion assessment result.

[0007] The beneficial effects of this invention are:

[0008] During user movement, motion image sequences are collected. For each motion image, user height is identified to obtain a height information sequence. The motion images are then grouped to obtain multiple motion image groups, providing a basis for subsequent analysis of the user's movement standardization at different height stages. Based on multiple motion image groups, variability analysis is performed to obtain multiple variability sequences. Multiple primary standard scores are calculated, and multiple keyframe image sets are obtained by filtering based on these variability sequences. By analyzing the variability between adjacent motion images within the same height stage, the stability and consistency of the user's movements at that height stage are evaluated. The smaller the variability, the more uniform the user's movements at that height stage, and the higher the primary standard score. Simultaneously, the keyframe image with the highest variability is selected to provide representative samples for subsequent motion standard recognition. Based on multiple variability sequences, multiple total variability are calculated, multiple motion standard recognition coefficients are configured, and motion standard recognition is performed on multiple keyframe image sets to obtain multiple secondary standard scores. By performing motion standard recognition on the keyframe image sets, the user's movements at different height stages are evaluated to determine whether they conform to standard postures, such as the flexion and extension angles of the arm joints, thus obtaining secondary standard scores reflecting the standardization of the movements. Based on the number of motion images within multiple motion image groups, a motion intensity score is obtained by classification. Based on multiple primary standard scores and multiple secondary standard scores, multiple standard scores are calculated. Combined with the motion intensity score, the user's motion score is calculated as the motion assessment result, which accurately reflects the user's motion quality.

[0009] The above technical solution enables an objective and accurate assessment of the standard of a user's movements and the amount of exercise during repetitive movements in the vertical direction. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a user motion assessment method for motion image recognition provided by the present invention;

[0011] Figure 2 This is a schematic diagram of the structure of a user motion assessment system for motion image recognition provided by the present invention.

[0012] In the attached diagram, the components represented by each number are as follows:

[0013] Image processing module 11, key frame determination module 12, standard recognition module 13, and scoring calculation module 14. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0016] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0017] Example 1, as Figure 1 As shown, this embodiment of the invention provides a user motion assessment method for motion image recognition, including:

[0018] S100: During the user's movement, the system collects a sequence of motion images of the user, identifies the user's height in each motion image to obtain a height information sequence, and groups the motion images to obtain multiple motion image groups.

[0019] Specifically, during the user's movement, an image acquisition device continuously captures a sequence of motion images of the user. This sequence contains the complete process of the user's movement from start to finish, and the capture frequency can be 15-30 frames per second to ensure the continuity of the action.

[0020] For each captured motion image, a user height recognizer is used. This recognizer, built on a convolutional neural network, can accurately identify the user's height position relative to the ground or a fixed reference point in each frame. By recognizing the height of each frame, a complete sequence of height information is obtained, recording the user's height change trajectory throughout the entire motion process.

[0021] Based on the obtained height information sequence, and according to a preset height interval range (e.g., every 10 centimeters), motion images with similar heights are grouped into the same motion image group. Within the same height interval, users should exhibit similar standard movement postures. For example, in pull-ups, when users are at the same height, their arm joint flexion and extension angles, body posture, etc., should remain consistent to demonstrate the standardization and stability of the movement.

[0022] By employing a height-based grouping strategy, multiple motion image groups are ultimately obtained, each group representing a set of user motion states within a specific height range, providing a classification basis for subsequent motion standardization evaluation.

[0023] S200: Based on the multiple motion image groups, perform variability analysis to obtain multiple variability sequences, calculate multiple first standard scores, and filter multiple keyframe image sets based on the multiple variability sequences.

[0024] Specifically, by comparing images within a set of motion images, the degree of variability between adjacent images is analyzed. This variability reflects the fluctuation of the user's movements within the same height range. A smaller variability indicates more stable and consistent user movements, and a higher level of standardization. The variability of images within each set of motion images is analyzed sequentially, resulting in multiple variability sequences. Then, the average variability of each sequence is calculated and compared to a preset variability standard value to obtain multiple primary standard scores. These primary standard scores quantify the consistency level of the user's movements across different height ranges. Next, the K motion images with the highest variability are selected from each variability sequence to form a keyframe image set. These keyframe images represent the moments when the user's movements change most significantly across different height ranges, possessing high analytical representativeness and effectively reflecting potential non-standardization in the user's movements, providing crucial data samples for subsequent refined evaluation.

[0025] Through the above-described variation analysis process, we can quantitatively assess the consistency of user actions across different height ranges and select the keyframe images with the greatest analytical value, laying the foundation for subsequent motion accuracy assessment.

[0026] S300: Based on the multiple degree of change sequences, calculate multiple total degree of change, configure multiple motion standard recognition coefficients, perform motion standard recognition on the multiple key frame image sets, and obtain multiple second standard scores.

[0027] Specifically, the mean of each variability sequence is first calculated to obtain multiple total variability values. These total variability values ​​reflect the overall changes in user movement within each height range. Then, the maximum value among the total variability values ​​is selected, and the ratio of each total variability value to the maximum total variability value is calculated to obtain multiple motion standard recognition coefficients. These motion standard recognition coefficients are used to configure the number of branches required for motion standard recognition. Height ranges with greater variability values ​​correspond to higher motion standard recognition coefficients, and more recognition branch resources will be allocated for evaluation to ensure accuracy. Through this adaptive configuration mechanism, computational resources can be rationally allocated, enabling more detailed standard evaluation of height ranges with large movement fluctuations.

[0028] For each set of keyframe images, a number of branches are randomly selected from the pre-trained motion standard recognition branches according to the configured number of branches. Motion standard recognition is then performed on each keyframe image set to obtain multiple sets of motion standard scores. These motion standard scores quantify the user's standardization during key actions at various height ranges. Subsequently, the mean of each set of motion standard scores is calculated to obtain multiple second standard scores. These second standard scores reflect the user's overall standardization during key actions at various height ranges, providing a reference for the final motion evaluation.

[0029] S400: Based on the number of motion images in the multiple motion image groups, classify and obtain motion volume scores; calculate multiple standard scores based on the multiple first standard scores and multiple second standard scores; combine the motion volume scores to calculate the user's motion score as the motion assessment result.

[0030] Specifically, firstly, the number of motion images within multiple motion image groups is counted to obtain the total number of motion images, and their average is calculated to obtain the average number of motion images. This value objectively reflects the user's activity level; a higher number of motion images indicates more exercise sessions completed by the user. Subsequently, the calculated average number of motion images is input into a pre-established activity level rating classification table, and the corresponding activity level rating is obtained through index matching. This activity level rating classification table, built based on a large amount of sample data, can map the average number of motion images in different ranges to corresponding activity level rating intervals, achieving a quantitative evaluation of activity level.

[0031] Next, based on the multiple primary and secondary standard scores obtained, a comprehensive standard score is calculated to reflect the user's movement standardization across different height ranges, encompassing evaluation indicators of both movement consistency and key movement standardization. Then, the multiple standard scores are weighted and calculated with the exercise volume score to obtain a final exercise score, serving as a comprehensive assessment result of the user's exercise. This comprehensive assessment considers both the quantity and quality of the user's exercise, providing an objective and accurate evaluation of exercise quality, which helps users to specifically improve their training methods and enhance exercise results.

[0032] Furthermore, during the user's movement, a sequence of motion images of the user is acquired. The user's height is identified in each motion image to obtain a height information sequence. The motion images are then grouped to obtain multiple motion image groups, including:

[0033] S110: During the user's movement, acquire the user's motion image sequence, wherein the movement is a repetitive motion in the height direction;

[0034] S120: Identify the user's height in each motion image within the motion image sequence to obtain a height information sequence;

[0035] S130: Based on the height information sequence, the motion images corresponding to the height information falling within the same height range are classified into a motion image group, thereby obtaining multiple motion image groups.

[0036] In one optional implementation, firstly, an image acquisition device is used to capture the user's movement process in real time, continuously acquiring a sequence of motion images. It is worth noting that this application is applicable to sports involving repetitive movements in the vertical direction, such as pull-ups, squats, and push-ups. These sports share the common characteristic of significant positional changes in the user's body in the vertical direction and the repetitive nature of the movements. The acquired motion image sequence completely records the entire process from the start to the end of the exercise, providing a raw data foundation for subsequent analysis.

[0037] Then, the acquired motion image sequence is processed to identify the user's height information in each motion image. For example, a pre-trained user height recognizer accurately detects the user's height position relative to a reference point in each frame. This height information is arranged in chronological order of the images to form a height information sequence, which completely records the user's height change trajectory throughout the entire motion process, providing a basis for subsequent action grouping and evaluation.

[0038] Subsequently, based on the obtained height information sequence, motion images whose height information falls within the same preset height range are grouped into the same motion image group. Within the same preset height range, such as 1.95m-2.00m, users should exhibit similar standard action postures. Through height grouping, multiple motion image groups are ultimately obtained, each group representing a set of user actions within a specific height range, laying the foundation for subsequent evaluation of the standardization of actions within each height range.

[0039] Furthermore, identifying the user's height within each motion image in the motion image sequence includes:

[0040] S121: Preprocess each motion image in the motion image sequence;

[0041] S122: Input multiple preprocessed motion images into the user height recognizer, and recognize and output multiple height information. The user height recognizer is constructed based on a convolutional neural network and is trained under supervision until convergence using sample motion images and labeled sample height information.

[0042] S123: Arrange multiple height information sequences according to the order of multiple motion images to obtain a height information sequence.

[0043] In a preferred embodiment, the acquired motion image sequence is first preprocessed. This preprocessing includes, but is not limited to, image cropping, resizing, noise removal, and illumination equalization, aiming to eliminate interference factors that may be introduced during image acquisition and improve the accuracy of subsequent height recognition. Through preprocessing, standardized motion images can be obtained, laying the foundation for accurate height recognition by the user.

[0044] Subsequently, multiple preprocessed motion images are input into a pre-trained user height recognizer. This user height recognizer, built on a convolutional neural network, can automatically extract features related to user height from the images and output corresponding height values, thus obtaining multiple height information. The user height recognizer is trained using a supervised learning method, utilizing a large number of sample motion images labeled with precise sample height information until the model converges. Through deep learning technology, this user height recognizer can adapt to the height recognition needs of users with different body types and has strong generalization ability.

[0045] Subsequently, the multiple height information points obtained from the recognition were arranged and organized according to the chronological order of the motion images to form a complete height information sequence. This sequence maintains a one-to-one correspondence with the original motion image sequence, fully recording the user's height change trajectory throughout the entire movement process, providing a temporal basis for subsequent action grouping and evaluation.

[0046] Furthermore, based on the multiple motion image groups, variability analysis is performed to obtain multiple variability sequences, multiple first standard scores are calculated, and multiple keyframe image sets are obtained by filtering based on the multiple variability sequences, including:

[0047] S210: Within the first group of motion images, select the first frame motion image and the second frame motion image, and use the first frame motion image as a reference to identify the degree of change in the second frame motion image.

[0048] S220: Continue to identify and obtain the first degree of change sequence of the first motion image group, and identify and obtain multiple degree of change sequences of the multiple motion image groups;

[0049] S230: Calculate multiple average degrees of change based on the multiple degree of change sequences;

[0050] S240: Calculate the ratio of the preset degree of change to the multiple average degrees of change to obtain multiple first standard scores;

[0051] S250: Based on the multiple change degree sequences, select the K motion images corresponding to the largest change degree in each of the multiple motion image groups as key frame images to obtain multiple key frame image sets.

[0052] In a preferred embodiment, the first motion image group is any one of multiple motion image groups. First, two adjacent motion image frames are selected within the first motion image group, namely the first motion image and the second motion image. Using the first motion image as a reference frame, the degree of change of the second motion image relative to the reference frame is calculated. This degree of change quantifies the difference between adjacent user actions within the same height range, reflecting the stability and consistency of the user's actions. The smaller the degree of change, the more stable and consistent the user's actions are within that height range. Then, following a similar method, the degree of change between all adjacent images within the first motion image group is analyzed sequentially to obtain a complete first degree of change sequence. Similarly, the same degree of change analysis process is performed on other motion image groups, ultimately obtaining multiple degree of change sequences, each corresponding to the changes in user actions within a specific height range.

[0053] Next, the average value of each variation sequence is calculated to obtain multiple average variation values. Each average variation value represents the overall fluctuation of the user's movements within a specific height range, serving as an indicator for evaluating the consistency of the user's movements within that range. Subsequently, a pre-set standard variation threshold is compared with each average variation value to obtain multiple primary standard scores. The larger the ratio, the higher the consistency and standardization of the user's movements within the corresponding height range. In this way, the standardization of the user's movements in each height range can be quantitatively evaluated. Then, based on the obtained variation sequences, the K motion images with the highest variation values ​​are selected from each motion image group as the keyframe image set for that height range. These keyframe images represent the moments when the user's movements change most significantly within each height range, effectively reflecting potential non-standard aspects of the user's movements and providing data samples for subsequent refined evaluation.

[0054] Furthermore, within the first group of moving images, a first frame and a second frame are selected. Using the first frame as a reference, the degree of change in the second frame is identified, including:

[0055] S211: Within the first group of moving images, select the first frame moving image and the second frame moving image;

[0056] S212: Input the first frame motion image and the second frame motion image into the change recognizer, and recognize and output the degree of change. The change recognizer is constructed based on a convolutional neural network and is trained under supervision until convergence using a combination of sample motion images and labeled sample change. Each combination of sample motion images includes two sample motion images.

[0057] In a preferred embodiment, two motion images are selected from the first motion image group, namely a first motion image and a second motion image. These two images are typically temporally adjacent, reflecting the user's action state at two consecutive moments within the same height range. By analyzing the differences between these two images, the consistency of the user's actions within that height range can be assessed.

[0058] Then, the selected first and second motion images are fed into a pre-trained change recognizer for processing. This change recognizer, built on a convolutional neural network, automatically extracts key features from the two images and calculates the degree of change between them. The change recognizer is trained using supervised learning, employing a large number of sample combinations containing two motion images and their corresponding labeled degrees of change, until the model converges. For example, the degree of change might be the pre-labeled magnitude of the user's body pose change, such as 5%. Through deep learning technology, this change recognizer accurately captures subtle differences between images, quantifies the degree of change in user actions, and provides precise data support for subsequent action accuracy evaluation.

[0059] Furthermore, based on the multiple variability sequences, multiple total variability are calculated, multiple motion standard recognition coefficients are configured, and motion standard recognition is performed on the multiple keyframe image sets to obtain multiple second standard scores, including:

[0060] S310: Calculate the mean of the multiple change degree sequences to obtain multiple total change degrees;

[0061] S320: Filter the maximum value among the multiple total variability values ​​to obtain the maximum total variability value, calculate the ratio of each total variability value to the maximum total variability value, and obtain multiple motion standard recognition coefficients;

[0062] S330: The plurality of motion standard recognition coefficients are each multiplied by the number Q of the plurality of pre-trained motion standard recognition branches and rounded to obtain the number of the plurality of motion standard recognition branches.

[0063] S340: Randomly select motion standard recognition branches according to the number of multiple motion standard recognition branches, perform motion standard recognition on the multiple keyframe image sets, obtain multiple motion standard score sets, calculate the mean of each set, and obtain multiple second standard scores.

[0064] In a preferred embodiment, firstly, the mean of each variation sequence is calculated to obtain multiple total variation values. Each total variation value represents the overall change in the user's actions within a specific height range and serves as the basis for configuring subsequent motion standard recognition branches. A larger total variation value indicates more significant fluctuations in the user's actions within that height range, requiring more computational resources for evaluation. Then, the maximum value is selected from the multiple total variation values ​​as the maximum total variation value. Subsequently, the ratio of each total variation value to the maximum total variation value is calculated to obtain multiple motion standard recognition coefficients. These coefficients reflect the relative degree of action change within each height range, providing a quantitative basis for subsequent adaptive configuration of recognition resources.

[0065] Next, the obtained motion standard recognition coefficients are multiplied by the pre-set total number of motion standard recognition branches Q, and rounded up to obtain the required number of motion standard recognition branches for each height interval, for example, Q = 10. This adaptive configuration mechanism can rationally allocate computing resources according to the complexity of motion changes in each height interval, allocating more recognition branches to height intervals with larger motion fluctuations, thereby improving the accuracy of the evaluation. Then, based on the configured number of motion standard recognition branches, a corresponding number of branches are randomly selected from the pre-trained Q motion standard recognition branches to perform motion standard recognition on each keyframe image set, obtaining multiple motion standard score sets. By calculating the average of each score set, multiple secondary standard scores are finally obtained. These secondary standard scores quantify the standardization of the user's key actions at each height interval, providing an important basis for the final motion evaluation.

[0066] Furthermore, the pre-training steps for the multiple motion standard recognition branches include:

[0067] S331: Based on historical user motion data, collect a set of sample keyframe images, and label each sample keyframe image with a motion standard score to obtain a set of sample motion standard scores.

[0068] S332: There are Q randomly selected motion standard recognition training data with replacement within the set of sample keyframe images and the set of sample motion standard scores;

[0069] S333: Using the Q sets of motion standard recognition training data, Q motion standard recognition branches are trained respectively, wherein each motion standard recognition branch is constructed and trained based on machine learning.

[0070] In a preferred embodiment, firstly, a representative set of sample keyframe images is collected based on a large amount of historical user motion data. These sample keyframe images cover various action postures that different users may exhibit at different heights. Professional sports coaches or assessment experts annotate each sample keyframe image with motion standard scores, forming a sample motion standard score set. This annotated data reflects the professional's judgment criteria for the degree of motion standardization, providing training targets for subsequent machine learning models. Then, from the obtained set of sample keyframe images and the corresponding sample motion standard score set, Q sets of motion standard recognition training data are selected using random sampling with replacement. This sampling method increases the diversity of training data, improves the generalization ability of the model, and enables the trained recognition branches to adapt to the motion assessment needs of different users. Subsequently, Q motion standard recognition branches are trained using the Q sets of motion standard recognition training data. Each recognition branch is constructed based on machine learning methods, such as support vector machines, random forests, neural networks, etc. Through this ensemble learning strategy, the evaluation results of multiple motion standard recognition branches can be integrated, reducing the evaluation bias that may be caused by a single model and improving the accuracy and stability of motion standard recognition.

[0071] Furthermore, based on the number of motion images within the multiple motion image groups, a motion intensity score is obtained by classification. Based on the multiple variability sequences, multiple scores combining the multiple standard scores are calculated to obtain the user's motion score, which serves as the motion assessment result, including:

[0072] S410: Count the number of motion images within the multiple motion image groups to obtain the number of multiple motion images, and calculate the average to obtain the average number of motion images;

[0073] S420: Input the average number of motion images into the motion score classification table, and obtain the motion score by index classification. The motion score classification table is constructed based on the index relationship between the sample average number of motion images set and the sample motion score set.

[0074] S430: Calculate multiple standard scores based on the multiple first standard scores and multiple second standard scores;

[0075] S440: Based on the multiple standard scores and exercise volume scores, a weighted exercise score is calculated as the exercise assessment result.

[0076] In a preferred embodiment, firstly, the number of motion images within each motion image group is counted to obtain multiple motion image counts. Then, the average of these multiple motion image counts is calculated to obtain the average number of motion images. The average number of motion images objectively reflects the amount of exercise completed by the user; a higher average number of motion images indicates that the user spent more time in each height range and completed more exercises. Next, the obtained average number of motion images is input into a pre-established exercise volume rating classification table, and the corresponding exercise volume rating is obtained through index matching. This exercise volume rating classification table is constructed based on a large amount of sample data and establishes a mapping relationship between the average number of motion images and the exercise volume rating. In this way, quantitative image quantity information can be transformed into qualitative exercise volume evaluation, achieving an objective assessment of the user's exercise volume.

[0077] Next, based on multiple primary standard scores (reflecting movement consistency) and multiple secondary standard scores (reflecting the standardization of key movements), a total comprehensive standard score is calculated using a weighted average or other fusion algorithm. These standard scores comprehensively reflect the user's movement standardization across various height ranges. Then, based on the multiple standard scores and the exercise volume score, a weighted calculation is performed using preset weighting coefficients to finally obtain the user's comprehensive exercise score, which serves as the exercise assessment result. This exercise assessment result comprehensively considers both the quality (standardization) and quantity (exercise volume) dimensions of the user's exercise, providing accurate exercise assessment results and helping users to specifically improve their training methods and enhance exercise effectiveness.

[0078] Example 2, as Figure 2 As shown, based on the same inventive concept as the user motion assessment method for motion image recognition provided in Embodiment 1, this embodiment of the invention also provides a user motion assessment system for motion image recognition, comprising:

[0079] Image processing module 11 is used to acquire motion image sequences of the user during the user's movement, perform user height recognition on each motion image to obtain height information sequence, and group the motion images to obtain multiple motion image groups;

[0080] The keyframe determination module 12 is used to perform variability analysis to obtain multiple variability sequences based on the multiple motion image groups, calculate multiple first standard scores, and filter multiple keyframe image sets based on the multiple variability sequences.

[0081] The standard recognition module 13 is used to calculate multiple total changes based on the multiple change sequences, configure multiple motion standard recognition coefficients, perform motion standard recognition on the multiple key frame image sets, and obtain multiple second standard scores.

[0082] The scoring calculation module 14 is used to classify and obtain a motion volume score based on the number of motion images in the multiple motion image groups, calculate multiple standard scores based on the multiple first standard scores and multiple second standard scores, and combine the motion volume scores to calculate the user's motion score as the motion assessment result.

[0083] Furthermore, the image processing module 11 includes the following execution steps:

[0084] During the user's movement, a sequence of motion images of the user is collected, wherein the movement is a repetitive motion in the vertical direction;

[0085] Identify the user's height within each motion image in the motion image sequence to obtain a height information sequence;

[0086] Based on the height information sequence, motion images corresponding to height information falling within the same height range are classified into a motion image group, thus obtaining multiple motion image groups.

[0087] Furthermore, the image processing module 11 also includes the following execution steps:

[0088] Preprocess each motion image in the motion image sequence;

[0089] Multiple preprocessed motion images are input into a user height recognizer, and multiple height information are obtained from the recognition output. The user height recognizer is constructed based on a convolutional neural network and is trained under supervision until convergence using sample motion images and labeled sample height information.

[0090] By arranging multiple height information data in the order of multiple motion images, a height information sequence is obtained.

[0091] Furthermore, the keyframe determination module 12 includes the following execution steps:

[0092] Within the first group of moving images, select the first frame and the second frame of moving images, and use the first frame of moving images as a reference to identify the degree of change in the second frame of moving images.

[0093] Continue to identify and obtain the first degree of change sequence of the first motion image group, and identify and obtain multiple degree of change sequences of the multiple motion image groups;

[0094] Calculate multiple average degrees of change based on the multiple degree of change sequences;

[0095] Calculate the ratio of the preset degree of change to the multiple average degrees of change to obtain multiple first standard scores;

[0096] Based on the multiple variability sequences, the motion images corresponding to the K largest variability values ​​are selected from multiple motion image groups as keyframe images, thereby obtaining multiple keyframe image sets.

[0097] Furthermore, the keyframe determination module 12 also includes the following execution steps:

[0098] Within the first group of moving images, select the first and second moving images.

[0099] The first and second motion images are input into the change recognizer, which identifies and outputs the degree of change. The change recognizer is built based on a convolutional neural network and is trained under supervision until convergence using a combination of sample motion images and labeled sample change. Each combination of sample motion images includes two sample motion images.

[0100] Furthermore, the standard identification module 13 includes the following execution steps:

[0101] Calculate the mean of the multiple change degree sequences to obtain multiple total change degrees;

[0102] The maximum value among the multiple total variability is selected to obtain the maximum total variability. The ratio of each total variability to the maximum total variability is calculated to obtain multiple motion standard recognition coefficients.

[0103] The number of motion standard recognition branches is obtained by multiplying the multiple motion standard recognition coefficients by the number Q of the multiple pre-trained motion standard recognition branches and rounding down.

[0104] According to the number of multiple motion standard recognition branches, the motion standard recognition branches are randomly selected, and motion standard recognition is performed on the multiple keyframe image sets to obtain multiple motion standard score sets. The mean of each set is calculated to obtain multiple second standard scores.

[0105] Furthermore, the pre-training steps for the multiple motion standard recognition branches include:

[0106] Based on historical user motion data, a set of sample keyframe images is collected, and motion standard scoring is labeled for each sample keyframe image to obtain a set of sample motion standard scores.

[0107] Within the set of sample keyframe images and the set of sample motion standard scores, there are Q randomly selected motion standard recognition training data with replacement;

[0108] Using the Q sets of motion standard recognition training data, Q motion standard recognition branches are trained to obtain each branch, wherein each motion standard recognition branch is built and trained based on machine learning.

[0109] Furthermore, the scoring calculation module 14 includes the following execution steps:

[0110] The number of motion images within the multiple motion image groups is counted to obtain the total number of motion images, and the average number of motion images is calculated.

[0111] The average number of motion images is input into the motion volume score classification table, and the motion volume score is obtained by index classification. The motion volume score classification table is constructed based on the index relationship between the sample average number of motion images set and the sample motion volume score set.

[0112] Based on the multiple first standard scores and multiple second standard scores, multiple standard scores are calculated to obtain multiple standard scores;

[0113] An exercise score is calculated by weighting the multiple standard scores and exercise volume scores, which serves as the exercise assessment result.

[0114] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0115] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0120] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A user motion evaluation method of motion image recognition, characterized by, The method comprises: During the user's movement, a sequence of movement images of the user is collected, the height of the user in each movement image is recognized to obtain a sequence of height information, the movement images are grouped to obtain a plurality of movement image groups, comprising: During the user's movement, a sequence of movement images of the user is collected, wherein the movement item is a movement item with repeated actions in the height direction; The height of the user in each movement image in the sequence of movement images is recognized to obtain a sequence of height information; According to the sequence of height information, the movement images corresponding to the height information falling into the same height interval are classified into a movement image group to obtain a plurality of movement image groups; According to the plurality of movement image groups, a plurality of change degree sequences are obtained by change degree analysis, a plurality of first standard scores are calculated, and a plurality of key frame image sets are obtained by screening according to the plurality of change degree sequences, comprising: In the first movement image group, a first frame of movement image and a second frame of movement image are selected, and the change degree of the second frame of movement image is recognized based on the first frame of movement image; The first change degree sequence of the first movement image group is obtained by continuous recognition, and a plurality of change degree sequences of the plurality of movement image groups are obtained by recognition; According to the plurality of change degree sequences, a plurality of average change degrees are calculated; The ratio of the preset change degree and the plurality of average change degrees is calculated respectively to obtain a plurality of first standard scores; According to the plurality of change degree sequences, the movement images corresponding to the maximum K change degrees are selected as key frame images in a plurality of movement image groups respectively, and a plurality of key frame image sets are obtained; According to the plurality of change degree sequences, a plurality of total change degrees are calculated, a plurality of movement standard recognition coefficients are configured, and a plurality of second standard scores are obtained by movement standard recognition on the plurality of key frame image sets, comprising: The mean of the plurality of change degree sequences is calculated to obtain a plurality of total change degrees; The maximum value in the plurality of total change degrees is screened to obtain a maximum total change degree, and the ratio of each total change degree to the maximum total change degree is calculated to obtain a plurality of movement standard recognition coefficients; The plurality of movement standard recognition coefficients are multiplied by the number Q of pre-trained movement standard recognition branches and rounded to obtain a plurality of movement standard recognition branch numbers; According to the plurality of movement standard recognition branch numbers, movement standard recognition branches are randomly selected to perform movement standard recognition on the plurality of key frame image sets to obtain a plurality of movement standard score sets, and the mean is calculated respectively to obtain a plurality of second standard scores; According to the number of movement images in the plurality of movement image groups, a movement amount score is classified, a plurality of standard scores are calculated according to the plurality of first standard scores and the plurality of second standard scores, and a user's movement score is calculated in combination with the movement amount score as a movement evaluation result.

2. The user motion evaluation method of motion image recognition according to claim 1, characterized by, The height of the user in each movement image in the sequence of movement images is recognized, comprising: The preprocessing of each movement image in the sequence of movement images is performed; The pre-processed multiple motion images are input into a user height identifier to identify and output multiple height information, wherein the user height identifier is constructed based on a convolutional neural network and supervised training is performed on sample motion images and labeled sample height information until convergence; The multiple height information is arranged in the order of the multiple motion images to obtain a height information sequence.

3. The user motion evaluation method of motion image recognition according to claim 1, characterized by, In the first motion image group, a first frame motion image and a second frame motion image are selected, and the change degree of the second frame motion image is identified based on the first frame motion image, including: In the first motion image group, a first frame motion image and a second frame motion image are selected; The first frame motion image and the second frame motion image are input into a change identifier to identify and output a change degree, wherein the change identifier is constructed based on a convolutional neural network and supervised training is performed on sample motion image combinations and labeled sample change degrees until convergence, and each sample motion image combination includes two sample motion images.

4. The user motion evaluation method of motion image recognition according to claim 1, characterized by, The pre-training step of the multiple motion standard identification branches includes: According to historical user motion data, a sample key frame image set is collected, and each sample key frame image is labeled with a motion standard score to obtain a sample motion standard score set; Q portions of motion standard identification training data are randomly selected with replacement from the sample key frame image set and the sample motion standard score set; The Q motion standard identification branches are trained respectively using the Q portions of motion standard identification training data, wherein each motion standard identification branch is constructed and trained based on machine learning.

5. The user motion evaluation method of motion image recognition according to claim 1, characterized by, According to the number of motion images in the multiple motion image groups, a motion amount score is classified, a user motion score is calculated based on the multiple change degree sequences and the multiple standard scores, and the user motion score is calculated as a motion evaluation result, including: The number of motion images in the multiple motion image groups is counted to obtain multiple motion image quantities, and an average is calculated to obtain an average motion image quantity; The average motion image quantity is input into a motion amount score classification table to index and classify a motion amount score, wherein the motion amount score classification table is constructed based on the index relationship between a sample average motion image quantity set and a sample motion amount score set; The multiple standard scores are calculated based on the multiple first standard scores and the multiple second standard scores; The motion score is weighted and calculated based on the multiple standard scores and the motion amount score as the motion evaluation result.

6. A user motion evaluation system for motion image recognition, characterized by, A user motion evaluation method for implementing the motion image identification of any one of claims 1-5, the system comprising: An image processing module for collecting a motion image sequence of a user during user motion, identifying the user height of each motion image to obtain a height information sequence, and grouping the motion images to obtain multiple motion image groups, including: During user motion, a motion image sequence of a user is collected, wherein the sports item is a sports item with repeated actions in the height direction; The user height in each motion image in the motion image sequence is identified to obtain a height information sequence; According to the height information sequence, motion images corresponding to height information falling into the same height interval are classified into a motion image group, and a plurality of motion image groups are obtained; A key frame determination module is configured to perform change degree analysis on the plurality of motion image groups to obtain a plurality of change degree sequences, calculate a plurality of first standard scores, and filter a plurality of key frame image sets according to the plurality of change degree sequences, including: In the first motion image group, a first frame of motion image and a second frame of motion image are selected, and the change degree of the second frame of motion image is identified based on the first frame of motion image; The first change degree sequence of the first motion image group is continuously identified, and a plurality of change degree sequences of the plurality of motion image groups are identified; According to the plurality of change degree sequences, a plurality of average change degrees are calculated; The ratio of a preset change degree and the plurality of average change degrees is calculated respectively to obtain a plurality of first standard scores; According to the plurality of change degree sequences, the motion images corresponding to the maximum K change degrees in the plurality of motion image groups are filtered as key frame images to obtain a plurality of key frame image sets; A standard identification module is configured to calculate a plurality of total change degrees according to the plurality of change degree sequences, configure a plurality of motion standard identification coefficients, perform motion standard identification on the plurality of key frame image sets to obtain a plurality of second standard scores, including: The mean of the plurality of change degree sequences is calculated to obtain a plurality of total change degrees; The maximum value in the plurality of total change degrees is filtered to obtain a maximum total change degree, and the ratio of each total change degree to the maximum total change degree is calculated to obtain a plurality of motion standard identification coefficients; The plurality of motion standard identification coefficients are multiplied by the number Q of motion standard identification branches pre-trained and rounded to obtain a plurality of motion standard identification branch numbers; According to the plurality of motion standard identification branch numbers, motion standard identification branches are randomly selected to perform motion standard identification on the plurality of key frame image sets to obtain a plurality of motion standard score sets, and the mean is calculated respectively to obtain a plurality of second standard scores; A score calculation module is configured to classify a motion amount score according to the number of motion images in the plurality of motion image groups, calculate a plurality of standard scores according to the plurality of first standard scores and the plurality of second standard scores, combine the motion amount score, calculate a user's motion score as a motion evaluation result.

Images