Motion quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses an action quality evaluation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring video data of motion of a to-be-assessed person; determining first information by using the video data, the first information representing body key point data of the to-be-evaluated person in each video frame; using the first information and second information to extract at least one video key frame from the video data, the second information being associated with motion characteristics of motions performed by the person to be assessed, and the motions including symmetric motions; and evaluating the motion of the motion performed by the person to be evaluated by using the extracted at least one video key frame. According to the scheme provided by the embodiment of the invention, automatic evaluation for symmetric movement actions is realized, and no electronic equipment needs to be worn while the labor cost is saved.

Description

Technical Field

[0001] The present application relates to the field of machine perspective technology, and in particular to a motion quality assessment method, apparatus, related equipment and storage medium. Background Art

[0002] With the development of society and economy, people pay more and more attention to physical exercise. Some symmetrical exercises of whole-body exercise, such as jumping jacks, burpees, squat jumps, etc., can exercise the muscles of the arms and legs, quickly burn fat, improve coordination, enhance cardiopulmonary function, improve muscle endurance, and reduce osteoporosis. However, if the exercise posture is incorrect, it will cause great damage to the knee and calf muscles. At present, the action quality evaluation methods are mainly divided into two categories: based on manual and hardware sensor equipment.

[0003] The manual motion evaluation method monitors each person to be evaluated in real time, checks the person's movements, and evaluates the movements. The motion evaluation method based on hardware sensor equipment requires a set of hardware equipment to be deployed in the sports venue, and the tester wears the corresponding hardware to detect the movement of the person to be evaluated in real time during the exercise. However, in the actual evaluation process, the manual motion quality evaluation method is very labor-intensive, requires real-time supervision, and has low evaluation efficiency. The motion evaluation method based on hardware sensor equipment has high equipment cost and complex deployment. It requires users to wear certain hardware equipment, which affects the exercise effect. Summary of the invention

[0004] To solve the related technical problems, the embodiments of the present application provide a motion quality assessment method, apparatus, related equipment and storage medium.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for evaluating motion quality, including:

[0007] Acquire video data of the movement of the person to be evaluated;

[0008] Determine first information using the video data, where the first information represents key point data of the body of the person to be evaluated in each video frame;

[0009] Extracting at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with a motion feature of a motion performed by the person to be evaluated, the motion comprising a symmetric motion;

[0010] The motion performed by the person to be evaluated is evaluated using the extracted at least one video key frame.

[0011] In the above scheme, the body key point data of the person to be evaluated in each video frame includes the coordinates of the body key points of the person to be evaluated, the second information includes the features of at least one key posture of the motion, and the extracting at least one video key frame from the video data using the first information and the second information includes:

[0012] For each video frame, using the body key point data of the person to be evaluated, analyzing the positional relationship between the body key points of the person to be evaluated to obtain an analysis result;

[0013] Using the analysis result and the feature of at least one key posture of the motion action, it is determined whether the video frame is a video key frame.

[0014] In the above solution, the step of determining whether a video frame is a video key frame by using the analysis result and the feature of at least one key posture of the motion action includes:

[0015] In the case where the analysis result matches the feature of at least one key gesture of the motion action, it is determined that the video frame is a video key frame.

[0016] In the above solution, the key postures of the motion action include at least two, and the method further includes:

[0017] The at least one video key frame forms a key gesture sequence;

[0018] When it is determined that a first video key frame included in the at least one video key frame does not match video key frames of all other gestures except the key gesture of the first video key frame in the at least two key gestures in the key gesture sequence, the first video frame is deleted from the extracted at least one video key frame.

[0019] In the above scheme, the step of evaluating the motion performed by the person to be evaluated by using the extracted at least one video key frame includes:

[0020] For each video key frame, the movement of the person to be evaluated is evaluated using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame, wherein the first parameter represents a symmetry degree benchmark of the movement, and the second parameter represents a completion degree benchmark of the movement.

[0021] In the above scheme, the step of evaluating the motion of the person to be evaluated by using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the key frame of the video includes:

[0022] Using the body key point data and the first parameter of the person to be evaluated in the video key frame, evaluating the degree of symmetry of the action of the person to be evaluated;

[0023] and / or,

[0024] The body key point data and the second parameter of the person to be evaluated in the video key frame are used to evaluate the degree of completion of the action of the person to be evaluated.

[0025] In the above solution, the determining the first information by using the video data includes:

[0026] The first information is determined using the video data and a first model.

[0027] In the above scheme, the method further includes:

[0028] Using the video data and the second model, identifying a body frame and a corresponding body frame confidence of at least one candidate, wherein the at least one candidate includes the person to be evaluated;

[0029] When the confidence level of the body frame is less than a preset confidence level, third information is generated, and the third information is used for warning.

[0030] In the above scheme, the method further includes:

[0031] The evaluation result of the person to be evaluated is outputted.

[0032] The present application also provides a method for evaluating motion quality, including:

[0033] Acquire video data of a person to be evaluated performing a movement, wherein the movement includes a symmetrical movement;

[0034] evaluating the motion using key frame information of the video data;

[0035] An evaluation result is output, wherein the evaluation result includes a symmetry degree of the action and / or a completion degree of the action.

[0036] The present application also provides a motion quality assessment device, including:

[0037] An acquisition unit, used for acquiring video data of the movement of the person to be evaluated;

[0038] An evaluation unit is used to determine first information using the video data, wherein the first information represents body key point data of the person to be evaluated in each video frame; extract at least one video key frame from the video data using the first information and second information, wherein the second information is associated with action features of a movement performed by the person to be evaluated, wherein the movement includes symmetrical movements; and evaluate the movement performed by the person to be evaluated using the extracted at least one video key frame.

[0039] The present application also provides a motion quality assessment device, including:

[0040] An acquisition unit, used for acquiring video data of a person to be evaluated performing a movement, wherein the movement includes a symmetrical movement;

[0041] An evaluation unit, configured to evaluate the motion using key frame information of the video data;

[0042] The output unit is used to output the evaluation result, wherein the evaluation result includes the symmetry degree of the action and / or the completion degree of the action.

[0043] The present application also provides an electronic device, including: a processor and a memory for storing a computer program that can be run on the processor.

[0044] Wherein, the processor is used to execute the steps of any of the above methods when running the computer program.

[0045] An embodiment of the present application further provides a storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0046] The motion quality assessment method, device, electronic device and storage medium provided in the embodiment of the present application obtain video data of the motion of the person to be assessed; using the video data, determine the first information, the first information represents the body key point data of the person to be assessed in each video frame; using the first information and the second information, extract at least one video key frame from the video data, the second information is associated with the motion characteristics of the motion performed by the person to be assessed, and the motion includes symmetrical motion; using at least one extracted video key frame, the motion performed by the person to be assessed is evaluated. The scheme provided in the embodiment of the present application extracts key frames from the video data of the motion of the person to be assessed according to the characteristics of symmetrical motion, and evaluates the motion of the person to be assessed based on the key frames, so that the automatic evaluation of the motion can be realized, saving labor costs; at the same time, when implementing the scheme of the present application, the person to be assessed does not need to wear any electronic device, so it is easy to implement (it can also be understood as easy to deploy, that is, the evaluation process is very simple for the person to be assessed), and the person to be assessed can start the motion at any position, which can be widely used in sports scenes such as campus, home fitness, and gymnasium. At the same time, the scheme provided by the embodiment of the present application obtains video data of the person to be evaluated performing motion, wherein the motion includes symmetrical motion; the motion is evaluated using the key frame information of the video data; and the evaluation result is output, wherein the evaluation result includes the degree of symmetry of the motion and / or the degree of completion of the motion. The scheme provided by the embodiment of the present application uses key frame information to evaluate the motion of the person to be evaluated from the dimensions of the degree of symmetry of the motion and / or the degree of completion of the motion and output the evaluation result, which can realize the automatic evaluation of the motion, save labor costs, and reach a unified standard to realize the comprehensive evaluation of symmetrical motion, and enable the evaluator and the person to be evaluated to quickly obtain the quantitative evaluation of the symmetrical motion performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the action quality assessment method according to an embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of the body detection model architecture of an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of the body posture estimation model architecture of an embodiment of the present application;

[0050] Figure 4 This is a schematic diagram of the opening and closing jumping action of the embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of key frame extraction of a jumping jack video according to an embodiment of the present application;

[0052] Figure 6This is a schematic diagram of a kicking and jumping action according to an embodiment of the present application;

[0053] Figure 7 This is a flowchart of a method for evaluating the quality of a jumping jack based on automatic key frame extraction as an application example of this application;

[0054] Figure 8 This is a schematic diagram of the structure of the motion quality assessment device according to an embodiment of the present application;

[0055] Fig. 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The present application will be further described in detail below through the accompanying drawings and specific embodiments.

[0057] In the related art, when evaluating the quality of some symmetrical movements of whole-body exercises, such as jumping jacks, burpees, squat jumps, etc., the manual movement quality evaluation method has different subjective standards for each person, is highly dependent on the coach's visual experience, lacks a data-based evaluation method, is easily affected by subjective factors, and is difficult to accurately and effectively evaluate the standardization of its movements and give reasonable improvement suggestions. Especially when a large number of people to be evaluated exercise together, it is very labor-intensive, requires real-time supervision, and has low evaluation efficiency. The movement evaluation method based on hardware sensing equipment is often a set of equipment for one person, which is very bulky, has high equipment cost, and is complex to deploy. It requires users to wear certain hardware equipment, which affects the exercise effect. In addition, solutions such as infrared sensor measurement have different physiological conditions such as height and fatness for different people to be evaluated, which are easily interfered with.

[0058] In summary, in the related art, in the scenario of symmetrical movements, there is a lack of an action quality assessment solution that can realize automatic assessment and simple deployment.

[0059] Based on this, in various embodiments of the present application, according to the characteristics of symmetrical motion, key frames are extracted from the video data of the motion of the person to be evaluated, and the motion of the person to be evaluated is evaluated based on the key frames. In this way, the automatic evaluation of the motion can be realized, saving labor costs; at the same time, when implementing the scheme of the present application, the person to be evaluated does not need to wear any electronic device, so it is easy to implement (it can also be understood as easy to deploy, that is, the evaluation process is very simple for the person to be evaluated), and the person to be evaluated can start the movement at any position, which can be widely used in sports scenes such as campus, family fitness, and gymnasium. At the same time, the scheme provided by the embodiment of the present application uses key frame information to evaluate the motion of the person to be evaluated from the dimensions of the degree of symmetry of the action and / or the degree of completion of the action and output the evaluation result, which can realize the automatic evaluation of the motion, save labor costs, and can also reach a unified standard to realize the comprehensive evaluation of symmetrical motion, and can enable the evaluator and the person to be evaluated to quickly obtain the quantitative evaluation of the symmetrical motion performed.

[0060] The symmetrical movement refers to: during the movement, the force applied and received on the left and right sides of the athlete's body are mirror-symmetrical. When the movement is completely standard and in compliance with the specification, the key points of the athlete's body are bilaterally symmetrical with the axis of the athlete's body as the axis of symmetry in each frame of the video data. For example, the symmetrical movement can be jumping jacks, burpees, kick jumps, spider jumps, mountain climbers, test jumps, tumbling jumps, etc., while movements such as single-leg jumps and rotation jumps are asymmetrical movements.

[0061] The present application embodiment provides a method for evaluating action quality. Figure 1 As shown, the method includes:

[0062] Step 101: Obtaining video data of the movement of the person to be evaluated;

[0063] Step 102: using the video data, determining first information, wherein the first information represents key point data of the body of the person to be evaluated in each video frame;

[0064] Step 103: extracting at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with a motion feature of a motion performed by the person to be evaluated, and the motion includes a symmetrical motion;

[0065] Step 104: using the extracted at least one video key frame, evaluating the motion performed by the person to be evaluated.

[0066] Among them, in actual application, the method of the embodiment of the present application is applied to electronic devices, such as smart phones, tablet computers, large-screen devices, etc., as long as the device can implement the solution of the embodiment of the present application, the embodiment of the present application is not limited to this.

[0067] In step 101, the electronic device may obtain the video data of the movement of the person to be evaluated by means of a camera. The embodiment of the present application does not limit the method of obtaining the video data of the movement of the person to be evaluated.

[0068] Here, it should be noted that before exercising, the person to be evaluated needs to perform warm-up movements. Specifically, the person to be evaluated faces the camera, with his hands hanging on both sides of the body, and stands upright without tilting his head within the range where the camera can collect video data. At this time, the line connecting the two shoulders of the person to be evaluated is perpendicular to the line connecting the camera and the person to be evaluated. The person to be evaluated faces the camera directly, so that in the video data collected by the camera, the body orientation of the person to be evaluated is frontal, and the left and right sides of the body are symmetrical. In this way, when performing action evaluation later, interference caused by different body orientations of the person to be evaluated can be avoided.

[0069] Among them, in a multi-person sports scene, multiple persons to be evaluated need to maintain a sufficient shooting distance from the camera to ensure that all persons to be evaluated are within the collection range, and at the same time, sufficient spacing is maintained between the persons to be evaluated to avoid mutual occlusion in the collected video data, so that the body information of all persons to be evaluated can be fully collected. All persons to be evaluated stand upright without tilting their heads, with their hands hanging on both sides of their bodies, and facing the camera. For example, when all persons to be evaluated maintain the same distance from the camera, they can stand in an arc or fan shape within the range where the camera can collect video data. At this time, in the collected video data, the body orientation of all persons to be evaluated is frontal, and the left and right sides of the body are symmetrical. Here, the shooting distance can be set as needed, such as according to the performance parameters of the video acquisition equipment, or the size of the venue for exercise, etc., and the embodiment of the present application does not limit this; at the same time, the spacing can be set as needed, such as 1 meter, and the embodiment of the present application does not limit this.

[0070] In actual application, in the process of collecting video data of the movement of the person to be evaluated, in order to prevent the collection of video data of people who do not participate in the evaluation from interfering with the subsequent evaluation, the people who do not participate in the evaluation can be identified first and eliminated; of course, if they are people who participate in the evaluation, they can be prompted to enter the collection area so that they can make an accurate assessment of the quality of the movement later.

[0071] Based on this, in one embodiment, before step 101, the method may further include:

[0072] Using the video data and the second model, identifying a body frame and a corresponding body frame confidence of at least one candidate, wherein the at least one candidate includes the person to be evaluated;

[0073] When the confidence level of the body frame is less than a preset confidence level, third information is generated, and the third information is used for warning.

[0074] Here, the electronic device can identify the body frame and the corresponding body frame confidence of the area where each candidate in the video data is located through a body detection model (i.e., the second model), and when the body frame confidence of a candidate is less than a preset confidence, an interference warning (i.e., the third information) is given, thereby prompting the candidate to leave the current video acquisition area or move to a new area by adjusting the distance and orientation from the video acquisition device; wherein, when the candidate wants to participate in the evaluation, he can move to a new area by adjusting the distance and orientation from the video acquisition device so that the body frame confidence of the candidate is greater than or equal to the preset confidence. In this way, it is possible to eliminate the interference of people who do not participate in the evaluation in the collected video data with the subsequent evaluation.

[0075] Exemplarily, in a multi-person exercise preparation scenario, at least one candidate stands facing a video capture device with his feet together and his hands vertically placed on both sides of his body, ready to start exercising. After the video capture device captures the video data of the candidate, the electronic device processes the captured video data through the second model to determine the body parts in the video data. At the same time, in order to distinguish each candidate, all body parts in the video data are divided according to the candidates, and the body frame corresponding to each candidate is marked (the range defined by the body frame at least includes all body parts of the corresponding candidate in the video data), and the body frame coordinates and body frame confidence corresponding to the candidate are calculated. For a body frame whose body frame confidence is less than a preset confidence, the candidate corresponding to the body frame is considered to be a person who does not participate in the evaluation, and the body frame is deleted from the output of the second model and an interference warning is given.

[0076] Here, the value of the preset confidence can be set as needed, such as 0.5, which is not limited in the embodiment of the present application; at the same time, the shape of the body frame can be set as needed, such as a rectangle, which is not limited in the embodiment of the present application.

[0077] In the above process, in actual application, for a body frame whose body frame confidence is greater than or equal to a preset confidence, the candidate framed by the body frame is determined to be the person to be evaluated, and the area of ​​the body frame corresponding to the person to be evaluated can be enlarged as the corresponding body motion area, and the body motion area is numbered 1, 2, 3..., N (N is the total number of body frames whose confidence is greater than or equal to the preset confidence, that is, the total number of persons to be evaluated), to form a motion area list, and through subsequent steps, the movements of all persons to be evaluated in the motion area list are evaluated to achieve synchronous evaluation of multiple people's movements. Here, the enlargement multiple of the area of ​​the body frame of the person to be evaluated can be set as needed, such as 1.5 times, and this embodiment of the present application is not limited to this.

[0078] Exemplarily, the second model may include a body detection model based on a convolutional neural network, such as Figure 2 As shown, the second model may include a feature extraction component, a feature fusion component and an output module, wherein the feature extraction component is used to extract image features, and specifically may include two stages. In the first stage, multiple convolutional neural networks (e.g. Figure 2 The components formed by the 3) and activation function shown in ) and the activation function extract image features, and the activation function can be a ReLU activation function; in the second stage, different degrees of deep semantic features are obtained by downsampling with different numbers of convolutions, such as Figure 2 As shown in the figure, two semantic features of different depths are obtained by using one and two convolution downsampling respectively, and the convolution downsampling can be a 3x3 convolution kernel; the feature fusion component is used to fuse features of different depths, specifically by performing deconvolution upsampling on the deep semantic features to recover the shallow position features from the feature map, and then fusing the deep semantic features with the shallow position features; finally, in the output module, three parallel neural networks are used to output the target category in the image (indicating the target monitored by the model, that is, the target category is body), body frame, and body frame confidence.

[0079] It should be noted that the specific implementation of the second model can be determined as needed, and the embodiments of the present application do not limit this.

[0080] In step 102, the determination of the first information can be understood as estimating the body posture of the person to be evaluated. Since the first information is determined using the video data, the obtained first information actually contains a sequence of changes in the positions of key points of the body used to evaluate the movements of the person to be evaluated (a time series).

[0081] In actual application, in order to ensure the accuracy of the position change sequence of the key points of the body (ie, the first information), the electronic device can process the video data through a model.

[0082] Based on this, in one embodiment, the specific implementation of step 102 may include:

[0083] The first information is determined using the video data and a first model.

[0084] Here, in actual application, the electronic device may analyze each frame of the video data, and obtain the body key point data in each frame of the movement of the person to be evaluated through the first model as the first information.

[0085] In actual application, in a multi-person motion preparation scenario, it may include, before performing body posture estimation, first using the video data and the second model to identify the body frame and the corresponding body frame confidence of at least one candidate, and based on the body frame and the corresponding body frame confidence, obtaining the motion area list, the motion area list including the motion area coordinates of each of the persons to be evaluated; when performing body posture estimation, it may include performing single-person posture estimation for each of the persons to be evaluated in each frame of the video data, specifically, for one person to be evaluated, determining the motion area coordinates corresponding to the person to be evaluated through the motion area list, and for the image area defined by the motion area coordinates, obtaining the body key point data in each frame of the person to be evaluated's motion through the first model as the first information.

[0086] Exemplarily, the first model may include a body posture estimation model based on a convolutional neural network, such as Figure 3 As shown in the figure, the model can extract image features in the form of three sub-networks in parallel: high resolution, medium resolution, and low resolution, and can obtain rich feature representations, thereby improving the accuracy of identifying body key point coordinates (i.e., body key point data) at different resolution scales.

[0087] Specifically, Figure 3As shown, the electronic device uses the first model: firstly, the image is feature extracted through the full high-resolution subnet to obtain the image feature connection at high resolution, and at the same time, a medium-resolution image is obtained by downsampling, and the medium-resolution image is feature extracted through the medium-resolution subnet to obtain the image feature connection at medium resolution, and the image information in the high-resolution and medium-resolution subnets is feature fused, and then a low-resolution image is obtained by downsampling again, and the low-resolution image is feature extracted through the low-resolution subnet to obtain the feature connection at low resolution, and after exchanging the information in the three parallel resolution corresponding subnets for many times, the feature maps finally obtained by the medium-resolution and low-resolution subnets are upsampled respectively, and the feature fusion is performed with the feature map finally obtained by the high-resolution subnet, and the coordinates of the position of the key points of the body are output. Wherein, the downsampling can specifically be a convolution operation on the image, and different convolution kernels and corresponding step sizes are used in the convolution operation to obtain images of different resolutions. For example, a 3x3 convolution kernel can be used to obtain a medium-resolution image, and a 5x5 convolution kernel can be used to obtain a low-resolution image. The size of the convolution kernel can be determined as needed, and the embodiment of the present application is not limited to this.

[0088] It should be noted that the specific implementation of the first model can be determined as needed, and the embodiments of the present application do not limit this.

[0089] In practical application, before executing step 103, the electronic device can also use a filter to filter out abnormal key points (the causes of the abnormal key points include unstable output of the body posture estimation model or abnormal fluctuations of the key points caused by camera frame drops), and obtain a smooth key point sequence change graph, so that when extracting the video key frame in step 103, the video key frame can be more accurately determined, reducing the interference caused by the abnormal fluctuations of the key points, and thus improving the accuracy of the motion quality assessment. Here, the filter can be a Savitzky-Golay filter, and can also be set as needed, which is not limited in the embodiment of the present application.

[0090] In practical applications, in order to evaluate the quality of the movement performed by the person to be evaluated, the key postures of the movement can be defined according to the characteristics of the movement, and the movement process of the person to be evaluated can also be represented by a key posture sequence composed of a series of key postures. For example, assuming that the movement performed by the person to be evaluated is jumping jacks, Figure 4 As shown, the movement characteristics of jumping jacks can be used to define that there are two key postures of jumping jacks, namely closing and taking off. When the person to be evaluated performs two jumping jacks in total in the video data, the complete movement process of the person to be evaluated can be characterized by a key posture sequence consisting of closing posture-taking off posture-closing posture-taking off posture.

[0091] Based on this, the evaluation of the movement of the person to be evaluated can be specifically implemented by evaluating each key posture in the key posture sequence. Specifically, the video data can be divided according to the characteristics of the key posture, so that each section of video data corresponds to a key posture in the key posture sequence. However, each section of video data contains multiple video frames. If each frame in the multiple video frames is evaluated, it takes a lot of computing power and time. In order to be able to efficiently evaluate each key posture, a video key frame can be extracted from the multiple video frames corresponding to each key posture. The video key frame can be understood as the video frame that best meets the characteristics of the corresponding key posture. In other words, the evaluation of the movement of the person to be evaluated can be simplified to the evaluation of a video key frame sequence composed of a series of extracted video key frames.

[0092] In actual application, in order to realize automatic and uniform standard extraction of the video key frames, the positional relationship between the key points of the body of the person to be evaluated in each video frame can be analyzed, and the analysis result can be matched with the characteristics of at least one key posture of the movement, and the matched video frame (that is, the one that meets the characteristics of at least one key posture) can be used as the video key frame.

[0093] Based on this, in one embodiment, the specific implementation of step 103 may include:

[0094] For each video frame, using the body key point data of the person to be evaluated, analyzing the positional relationship between the body key points of the person to be evaluated to obtain an analysis result;

[0095] Using the analysis result and the feature of at least one key posture of the motion action, it is determined whether the video frame is a video key frame.

[0096] Specifically, in one embodiment, the determining whether a video frame is a video key frame by using the analysis result and the feature of at least one key gesture of the motion action includes:

[0097] In the case where the analysis result matches the feature of at least one key gesture of the motion action, it is determined that the video frame is a video key frame.

[0098] Of course, when the analysis result does not match the feature of at least one key gesture of the motion action, it is determined that the video frame is not a video key frame.

[0099] For example, assuming that the exercise performed by the person to be evaluated is jumping jacks, after obtaining the key point information of the body, a Savitzky-Golay filter is used to obtain a smooth key point sequence change graph, and two key postures of take-off and closing are defined according to the movement characteristics of jumping jacks, and the position information sequence of the wrist, shoulder, hip, and ankle is selected to extract the video key frame. Specifically, Figure 4 As shown, for each video frame image, the sum of the angles of the left wrist-shoulder-hip angle 401, the right wrist-shoulder-hip angle 402, the left ankle-hip-hip vertical ground point angle 403 and the right ankle-hip-hip vertical ground point angle 404 is used for analysis, as shown in FIG. Figure 5 As shown, a relationship diagram of the sum of the angles changing with the frame number (the sequence number of each video frame arranged in chronological order) is obtained, and the angle thresholds corresponding to the take-off posture and the closing posture can be set respectively. When the sum of the angles exceeds the angle threshold, it is determined that the key posture of the subject to be evaluated has changed, and the frames corresponding to the maximum point and the minimum point are respectively used as the take-off key frame and the closing key frame. All the determined take-off key frames and closing key frames are merged and sorted to obtain the opening and closing key posture sequence of the entire video.

[0100] Here, it should be noted that the angle threshold can be set as needed, for example, the difference between the angle threshold corresponding to the take-off posture and the maximum value of the sum of the angles is set to 1 / 10 of the difference between the maximum value and the minimum value of the sum of the angles. By using the angle threshold to judge the change of the key posture of the person to be evaluated, it can be ensured that the key frame can be accurately extracted even if the action of the person to be evaluated is not standardized or the key points of the body are shaking.

[0101] For example, it is assumed that the exercise performed by the person to be evaluated is kicking and jumping, such as Figure 6 As shown, two key postures of squatting and rising are defined according to the kicking and jumping motion characteristics, and the position information sequence of the hips, knees, and ankles is selected to extract the key frames of the video. Specifically, the change of the sum of the height coordinate values ​​of the hips and toes of the left and right body is used for analysis, and the distance threshold of the squatting and rising postures is set. When the sum of the height coordinate values ​​exceeds the distance threshold, it is determined that the key posture of the person to be evaluated has changed, and the frame numbers corresponding to the maximum and minimum points are respectively used as the rising and squatting key frames. All the determined squatting key frames and rising key frames are merged and sorted to obtain the kicking and jumping key posture sequence of the entire video.

[0102] In actual application, when the first action has at least two key postures, isolated video key frames may be extracted in the process of extracting video key frames from the first video. When performing action evaluation, isolated video key frames cannot represent a complete action, so they will cause interference and should be eliminated before evaluation. Exemplarily, when the person to be evaluated performs jumping jacks, the person to be evaluated ends the movement in a closed posture after performing two jumping jacks, then the extracted key frame sequence may be a closing key frame-a take-off key frame-a closing key frame-a take-off key frame-a closing key frame. Here, since the person to be evaluated ends in a closed posture, the last closing key frame is isolated, that is, there is no matching take-off key frame, and it does not constitute a complete jumping jack action. When evaluating the jumping jack action, the last closing key frame should not be considered.

[0103] Here, it should be noted that the reason for the isolation of the video key frame may be that the body posture of the person to be evaluated at the end of the movement has no subsequent matching key posture, or the output of the body posture model is unstable or the video acquisition device has frame drops, etc.

[0104] Based on this, in one embodiment, the key postures of the motion action include at least two, and the method may further include:

[0105] The at least one video key frame forms a key gesture sequence;

[0106] When it is determined that a first video key frame included in the at least one video key frame does not match video key frames of all other gestures except the key gesture of the first video key frame in the at least two key gestures in the key gesture sequence, the first video frame is deleted from the extracted at least one video key frame.

[0107] Exemplarily, when the movement is a jumping jack, a take-off posture will match a closing posture to form a jumping jack process. If only a single closing posture or a take-off posture is recognized, the video key frame is deleted from the key posture sequence; when the movement is a kicking jump, a squat posture will match a rising posture to form a kicking jump process. If only a single squat posture or a rising posture is recognized, the video key frame is deleted from the key posture sequence.

[0108] In actual application, in one embodiment, the electronic device can determine whether the person to be evaluated has left the moving body area. If so, the extraction of the video key frames is terminated, and the action evaluation is performed using the obtained video key frames.

[0109] Exemplarily, in a multi-person motion scene, when any person to be evaluated leaves the corresponding motion body motion area, the extraction of the video key frames of the person to be evaluated ends, and the video key frames of the person to be evaluated are evaluated; for the remaining persons to be evaluated who are still in the corresponding motion body area, the extraction of video key frames continues.

[0110] In actual application, when the movement performed by the person to be evaluated is a symmetrical movement, in order to achieve a quantitative evaluation of the movement and reach a unified standard, the degree of standardization of the movement can be evaluated from the dimensions of symmetry and / or completion to achieve a comprehensive quality evaluation of the movement.

[0111] Based on this, in one embodiment, the specific implementation of step 104 may include:

[0112] For each video key frame, the movement of the person to be evaluated is evaluated using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame, wherein the first parameter represents a symmetry degree benchmark of the movement, and the second parameter represents a completion degree benchmark of the movement.

[0113] Specifically, in one embodiment, the step of evaluating the motion performed by the person to be evaluated by using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame includes:

[0114] Using the body key point data and the first parameter of the person to be evaluated in the video key frame, evaluating the degree of symmetry of the action of the person to be evaluated;

[0115] and / or,

[0116] The body key point data and the second parameter of the person to be evaluated in the video key frame are used to evaluate the degree of completion of the action of the person to be evaluated.

[0117] Here, in actual application, the first parameter and the second parameter may be related to the relative positions between key points of the body of the person to be evaluated.

[0118] Specifically, in order to achieve a quantitative evaluation of the degree of symmetry, the evaluation may include determining whether the relative positions between the key points of the right part of the body of the person to be evaluated are the same as the relative positions between the corresponding key points of the left part of the body. Based on this, the first parameter may be a standard deviation between the left relative position value and the right relative position value. In other words, the electronic device uses the key point data of the body to determine the difference between the relative positions between the key points of the right part of the body of the person to be evaluated and the relative positions between the key points of the left part of the body of the person to be evaluated, and uses the difference and the first parameter to evaluate the degree of symmetry of the action of the person to be evaluated.

[0119] In order to achieve a quantitative evaluation of the degree of completion, the evaluation may include first determining the relative position between the key points of the body of the person to be evaluated whose relative position does not change as a reference value, such as the distance between the two eyes, the actual length of the arm when standing, etc., and then determining at least one evaluation value based on the positional relationship between the key points of the body related to the key posture of the first action, and then setting at least one evaluation index based on the standard action, each of which corresponds to an evaluation value and a reference value, and judging whether the ratio between the evaluation value and the reference value exceeds the set threshold. Based on this, the second parameter may include a threshold corresponding to at least one evaluation index, and the more the number of evaluation index items whose ratio between the evaluation value and the reference value exceeds the threshold, the higher the degree of completion of the action of the person to be evaluated, and the higher the degree of completion score. That is, the electronic device uses the body key point data to determine the relative position between the key points of the body related to the movement, and uses the relative position and the second parameter to evaluate the degree of completion of the action of the person to be evaluated.

[0120] In actual application, the first parameter, the second parameter and the related thresholds can be defined according to the sports characteristics of the sport, or determined from relevant sports normative books and standards, or defined according to actual needs, and the embodiments of the present application are not limited to this.

[0121] Exemplarily, when the movement is a jumping jack, the calculation of symmetry is defined by the difference between the left wrist-shoulder-hip angle and the right wrist-shoulder-hip angle in each video key frame and the difference between the left ankle-hip-hip vertical ground point angle and the right ankle-hip-hip vertical ground point angle. The degree of completion is defined by four indicators: the distance between the two feet, the distance between the left and right wrists, and the distance from the left and right wrists to the shoulders in each take-off posture key frame. When the person to be evaluated finishes the jumping jack, the symmetry and completion of the action of the person to be evaluated are calculated.

[0122] Exemplarily, when the motion is a kick, Figure 6As shown, the calculation of symmetry is defined by the difference between the left toe-knee-hip angle 602 and the right toe-knee-hip angle in each video key frame. The degree of completion is defined by the distance 601 from the left hip to the toe in each video key frame. When the person to be evaluated finishes the kick, the symmetry and completion of the action of the person to be evaluated are calculated.

[0123] In one embodiment, after calculating the symmetry and / or completion of the subject to be evaluated, the method may further include:

[0124] The evaluation result of the person to be evaluated is outputted.

[0125] Specifically, the evaluation result includes the symmetry and / or completion of the action of the person to be evaluated.

[0126] Here, outputting the evaluation result may be by displaying the evaluation result on a front-end interface for human-computer interaction, or by outputting data as needed, which is not limited in this embodiment of the present application.

[0127] Outputting the evaluation results can enable the evaluator and the person to be evaluated to quickly obtain a quantitative evaluation of the person's movements.

[0128] It can be seen from the above description that the embodiment of the present application provides a method for evaluating action quality, which specifically includes the following steps:

[0129] Acquire video data of a person to be evaluated performing a movement, wherein the movement includes a symmetrical movement;

[0130] evaluating the motion using key frame information of the video data;

[0131] An evaluation result is output, wherein the evaluation result includes a symmetry degree of the action and / or a completion degree of the action.

[0132] In one embodiment, the specific implementation of evaluating the motion by using the key frame information of the video data may include:

[0133] Determine first information using the video data, where the first information represents key point data of the body of the person to be evaluated in each video frame;

[0134] Extracting at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with a motion feature of a motion performed by the person to be evaluated, the motion comprising a symmetric motion;

[0135] The motion performed by the person to be evaluated is evaluated using the extracted at least one video key frame.

[0136] It should be noted here that the specific implementation of the above steps has been described in detail above and will not be repeated here.

[0137] The motion quality evaluation method provided by the embodiment of the present application, according to the characteristics of symmetrical motion, extracts key frames from the video data of the motion of the person to be evaluated, and evaluates the motion of the person to be evaluated based on the key frames, so that the automatic evaluation of the motion can be realized, saving labor costs; at the same time, when implementing the scheme of the present application, the person to be evaluated does not need to wear any electronic device, so it is easy to implement (it can also be understood as easy to deploy, that is, the evaluation process is very simple for the person to be evaluated), and the person to be evaluated can start the motion at any position, which can be widely used in sports scenes such as campus, family fitness, and gymnasium. At the same time, the scheme provided by the embodiment of the present application, using key frame information, from the dimension of the degree of symmetry of the action and / or the degree of completion of the action, the motion of the person to be evaluated is evaluated and the evaluation result is output, which can realize the automatic evaluation of the motion, save labor costs, and can also reach a unified standard, realize the comprehensive evaluation of symmetrical motion, and enable the evaluator and the person to be evaluated to quickly obtain the quantitative evaluation of the symmetrical motion performed.

[0138] The present application is further described in detail below in conjunction with application examples.

[0139] Application Example 1

[0140] In this application example, the action quality assessment of jumping jacks is taken as an example. This application example proposes a method for evaluating the action quality of jumping jacks based on automatic key frame extraction. Figure 7 As shown, the method includes:

[0141] Step 700: Collect video data of the evaluation site, and then complete the evaluation of the action quality through the following four steps;

[0142] Step 701: body detection;

[0143] Specifically, the method starts with a scene where multiple people are preparing to do jumping jacks. The person to be evaluated stands still, puts his feet together, and places his hands vertically on both sides of his body, preparing to do jumping jacks. The body frame coordinates and confidence are output according to the body detection model. For body frames with a confidence greater than or equal to 0.5, the area is expanded by 1.5 times as the body jumping jack area. The area is numbered 1, 2, 3, ..., N (N is the total number of body frames with a confidence greater than or equal to 0.5), forming a list of jumping jack areas for subsequent body posture estimation. For body frames with a confidence less than 0.5, the body frame is deleted and an interference warning is given.

[0144] In the specific implementation process, the body detection model includes: a network model based on a convolutional neural network, which may include a feature extraction component, a feature fusion component and an output module, wherein the feature extraction component is used to extract image features, and may specifically include two stages. In the first stage, multiple convolutional neural networks (e.g. Figure 2 The components formed by the 3) and activation function shown in ) and the activation function extract image features, and the activation function can be a ReLU activation function; in the second stage, different degrees of deep semantic features are obtained by downsampling with different numbers of convolutions, such as Figure 2 As shown in the figure, two semantic features of different depths are obtained by using one and two convolution downsampling respectively, and the convolution downsampling can be a 3x3 convolution kernel; the feature fusion component is used to fuse features of different depths, specifically by performing deconvolution upsampling on the deep semantic features to recover the shallow position features from the feature map, and then fusing the deep semantic features with the shallow position features; finally, in the output module, three parallel neural networks are used to output the target category in the image (indicating the target monitored by the model, that is, the target category is body), body frame, and body frame confidence.

[0145] Step 702: body posture estimation;

[0146] Specifically, after obtaining the list of jumping jack regions, the person to be evaluated starts to perform jumping jacks. The person to be evaluated is subjected to body posture estimation, and the position changes of the key points of the person to be evaluated in each jumping jack region are recorded. The specific implementation process includes: performing single-person posture estimation on each body region in each frame of the image during the jumping jack process, obtaining the body frame coordinates, using the body posture estimation model based on the convolutional neural network to extract features from the image framed by the body frame coordinates, firstly extracting features from the image through the full high-resolution subnet to obtain image feature connections at high resolution, and at the same time obtaining a medium-resolution image through downsampling, and extracting features from the medium-resolution image through the medium-resolution subnet to obtain image feature connections at medium resolution, after feature fusion of the image information in the high-resolution and medium-resolution subnets, obtaining a low-resolution image through downsampling again, and extracting features from the low-resolution image through the low-resolution subnet to obtain feature connections at low resolution, after exchanging the information in the three parallel resolution corresponding subnets for multiple times, upsampling the feature maps finally obtained from the medium-resolution and low-resolution subnets respectively, and performing feature fusion with the feature maps finally obtained from the high-resolution subnet, and outputting the position coordinates of the key points of the body. By extracting image features in parallel using three subnetworks, high resolution, medium resolution, and low resolution, we can obtain rich feature representations, thereby improving the accuracy of body key point position data recognition at different resolution scales.

[0147] Step 703: Automatically extract key frames;

[0148] Specifically, after obtaining the position data of the key points of the body, a Savitzky-Golay filter is used to obtain a smooth position change diagram of the key points of the body sequence, and then the key points of the body are selected according to the characteristics of the jumping jack movement, specifically the wrist, shoulder, hip, and ankle, and the two key postures of take-off and closing are extracted through the position information sequence of the key points of the body. The specific implementation process includes: using the angle sum change between the coordinates of the left and right wrists, shoulders and hips and the angle between the ankles, hips and the vertical ground points of the hips to analyze, taking the maximum and minimum points to form a sequence, and analyzing the maximum and minimum sequences. Setting the angle threshold of the take-off and closing postures, when exceeding this angle threshold, it is determined that its state has changed, and saving the corresponding indexes of the corresponding maximum and minimum points as the take-off key frame and the closing key frame respectively. The take-off key frame and the closing key frame are merged and sorted, and the key posture of the jumping jack of the entire video is output. The threshold is set to 1 / 10 of the difference between the maximum angle and the minimum angle.

[0149] Step 704: Jumping jack action evaluation.

[0150] Specifically, it is determined whether the person to be evaluated has left the jumping jack body area. If so, the key frame extraction is terminated, the jumping jack action is evaluated for the obtained key frame, and the symmetry and completion scores of the person to be evaluated are output. The symmetry is calculated by calculating the angle between the wrist-shoulder-hip and the angle between the ankle-hip-hip perpendicular to the ground of the left and right body of each key frame. If the difference between the two angles of the left and right body is less than 10 degrees, Figure 4 As shown, if the difference between angle 401 and angle 402 and the difference between angle 403 and angle 404 are both less than 10 degrees, the symmetry of this key frame is recorded as 1, otherwise it is recorded as 0. Finally, the sum of the total key frame symmetry is divided by the total number of key frames for normalization to obtain the symmetry score. Among the four indicators of completion: if the distance between the two feet is greater than 1.5 shoulder distances, the distance between the left and right wrists is less than the distance between the left and right eyes, and the ratio of the distance from the left and right wrists to the shoulders to the actual length of the arm of the person to be evaluated is greater than 0.9, then the completion of the take-off key frame is 1, otherwise, 0.25 points will be deducted for each indicator that fails to meet the standard. Finally, the sum of the total take-off posture key frame completion is divided by the total number of opening and closing jumps for normalization to obtain the completion score.

[0151] From the above description, it can be seen that this application example provides a jumping jack action evaluation scheme based on automatic key frame extraction. It uses a human body detection model to divide the motion area of ​​the picture, and can realize multi-person simultaneous evaluation and anti-interference without dividing the fixed motion space in advance. At the same time, the key frame automatic extraction model is first used to define the key frames of the motion process, and a new method for automatic extraction of key postures of jumping jacks is realized by using motion feature parameters. After obtaining the motion key frames, the motion quality evaluation model is used to calculate the symmetry and completion of the key posture sequence to realize a new motion quality evaluation scheme. Compared with the method based on manual evaluation, the solution provided by this application example unifies the calculation standards, saves labor costs, and realizes automatic evaluation of motion. Compared with the method based on hardware sensing device evaluation, the solution provided by this application embodiment does not require wearing any electronic equipment, the evaluation is very simple, and the person to be evaluated can start the movement at any position.

[0152] In order to implement the action quality evaluation method of the embodiment of the present application, the embodiment of the present application also provides an action quality evaluation device, which is arranged on an electronic device, such as Figure 8 As shown, the device comprises:

[0153] An acquisition unit 801 is used to acquire video data of the movement of the person to be evaluated;

[0154] The evaluation unit 802 is used to determine first information using the video data, wherein the first information represents the body key point data of the person to be evaluated in each video frame; extract at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with the action characteristics of the movement performed by the person to be evaluated, and the movement includes symmetric movement; and evaluate the movement performed by the person to be evaluated using the extracted at least one video key frame. In other words, the evaluation unit 802 is used to evaluate the movement using the key frame information of the video data.

[0155] In one embodiment, the evaluation unit 802 is used to:

[0156] Using the video data and the second model, a body frame and a corresponding body frame confidence of at least one candidate are identified, wherein the at least one candidate includes the person to be evaluated; if the body frame confidence is less than a preset confidence, third information is generated, and the third information is used for warning.

[0157] In one embodiment, the evaluation unit 802 is specifically configured to:

[0158] The first information is determined using the video data and a first model.

[0159] In one embodiment, the body key point data of the person to be evaluated in each video frame includes the coordinates of the body key points of the person to be evaluated, the second information includes the features of at least one key posture of the motion, and the evaluation unit 802 is specifically configured to:

[0160] For each video frame, the positional relationship between the body key points of the person to be evaluated is analyzed using the body key point data of the person to be evaluated to obtain an analysis result; and whether the video frame is a video key frame is determined using the analysis result and the feature of at least one key posture of the motion action.

[0161] In one embodiment, the key postures of the motion action include at least two, and the evaluation unit 802 is further used for:

[0162] The at least one video key frame forms a key gesture sequence; when it is determined that a first video key frame included in the at least one video key frame does not match the video key frames of all other gestures in the at least two key gestures except the key gesture of the first video key frame in the key gesture sequence, the first video frame is deleted from the extracted at least one video key frame.

[0163] In one embodiment, the evaluation unit 802 is used to:

[0164] For each video key frame, the movement of the person to be evaluated is evaluated using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame, wherein the first parameter represents a symmetry degree benchmark of the movement, and the second parameter represents a completion degree benchmark of the movement.

[0165] In one embodiment, the evaluation unit 802 is specifically configured to:

[0166] The symmetry of the action of the person to be evaluated is evaluated by using the body key point data and the first parameter of the person to be evaluated in the video key frame; and / or the completion degree of the action of the person to be evaluated is evaluated by using the body key point data and the second parameter of the person to be evaluated in the video key frame.

[0167] In one embodiment, the device may further include:

[0168] The output unit is used to output the evaluation result of the person to be evaluated.

[0169] In actual application, the acquisition unit 801 can be implemented by a communication interface in the motion quality assessment device; the assessment unit 802 can be implemented by a processor in the motion quality assessment device.

[0170] It should be noted that: the action quality assessment device provided in the above embodiment only uses the division of the above program units as an example when performing action quality assessment. In practical applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device is divided into different program units to complete all or part of the processing described above. In addition, the action quality assessment device provided in the above embodiment and the action quality assessment method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0171] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, such as Fig. 9 As shown, the electronic device 900 includes:

[0172] Communication interface 901, capable of interacting with other devices;

[0173] A processor 902, connected to the communication interface 901 to interact with other devices, and configured to execute the method provided by one or more of the above technical solutions when running a computer program;

[0174] A memory 903 , on which the computer program is stored.

[0175] Specifically, the communication interface 901 is used to obtain video data of the movement of the person to be evaluated;

[0176] The processor 902 is configured to:

[0177] Determine first information using the video data, where the first information represents key point data of the body of the person to be evaluated in each video frame;

[0178] Extracting at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with a motion feature of a motion performed by the person to be evaluated, the motion comprising a symmetric motion;

[0179] The motion performed by the person to be evaluated is evaluated using the extracted at least one video key frame.

[0180] In one embodiment, the processor 902 is configured to:

[0181] For each video frame, using the body key point data of the person to be evaluated, analyzing the positional relationship between the body key points of the person to be evaluated to obtain an analysis result;

[0182] Using the analysis result and the feature of at least one key posture of the motion action, it is determined whether the video frame is a video key frame.

[0183] In one embodiment, the processor 902 is configured to:

[0184] In the case where the analysis result matches the feature of at least one key gesture of the motion action, it is determined that the video frame is a video key frame.

[0185] In one embodiment, the processor 902 is further configured to:

[0186] The at least one video key frame forms a key gesture sequence;

[0187] When it is determined that a first video key frame included in the at least one video key frame does not match video key frames of all other gestures except the key gesture of the first video key frame in the at least two key gestures in the key gesture sequence, the first video frame is deleted from the extracted at least one video key frame.

[0188] In one embodiment, the processor 902 is configured to:

[0189] For each video key frame, the movement of the person to be evaluated is evaluated using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame, wherein the first parameter represents a symmetry degree benchmark of the movement, and the second parameter represents a completion degree benchmark of the movement.

[0190] In one embodiment, the processor 902 is configured to:

[0191] Using the body key point data and the first parameter of the person to be evaluated in the video key frame, evaluating the degree of symmetry of the action of the person to be evaluated;

[0192] and / or,

[0193] The body key point data and the second parameter of the person to be evaluated in the video key frame are used to evaluate the degree of completion of the action of the person to be evaluated.

[0194] In one embodiment, the processor 902 is configured to:

[0195] The first information is determined using the video data and a first model.

[0196] In one embodiment, the processor 902 is further configured to:

[0197] Using the video data and the second model, identifying a body frame and a corresponding body frame confidence of at least one candidate, wherein the at least one candidate includes the person to be evaluated;

[0198] When the confidence level of the body frame is less than a preset confidence level, third information is generated, and the third information is used for warning.

[0199] In other words, the processor 902 is configured to evaluate the motion using key frame information of the video data.

[0200] In one embodiment, the communication interface 901 is further used to output the evaluation result of the person to be evaluated.

[0201] It should be noted that the specific processing process of the communication interface 901 and the processor 902 can be understood by referring to the above method.

[0202] Of course, in actual application, the various components in the electronic device 900 are coupled together through the bus system 904. It can be understood that the bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 Various buses are labeled as bus system 904.

[0203] The memory 903 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 900. Examples of such data include: any computer program used to operate on the electronic device 900.

[0204] The method disclosed in the above embodiment of the present application can be applied to the processor 902, or implemented by the processor 902. The processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 902 or an instruction in the form of software. The above-mentioned processor 902 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 902 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 903, and the processor 902 reads the information in the memory 903 and completes the steps of the above method in combination with its hardware.

[0205] It can be understood that the memory (memory 903) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0206] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0207] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 903 storing a computer program, and the computer program can be executed by a processor 902 of an electronic device 900 to complete the steps of the aforementioned model training method, or to complete the steps of the aforementioned path determination method. The computer-readable storage medium can be a ROM, a PROM, an EPROM, an EEPROM, a FRAM, a Flash Memory, a magnetic surface storage, an optical disk, or a CD-ROM; the magnetic surface storage can be a disk storage or a tape storage.

[0208] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0209] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0210] The above is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for evaluating motion quality, characterized in that: include: Acquire video data of the movement of the person to be evaluated; Determine first information using the video data, where the first information represents key point data of the body of the person to be evaluated in each video frame; Extracting at least one video key frame from the video data using the first information and the second information, wherein the second information is associated with a motion feature of a motion performed by the person to be evaluated, the motion comprising a symmetric motion; The motion performed by the person to be evaluated is evaluated using the extracted at least one video key frame.

2. The method according to claim 1, characterized in that The method further comprises: The evaluation result of the person to be evaluated is outputted.

3. The method according to claim 1, characterized in that The body key point data of the person to be evaluated in each video frame includes the coordinates of the body key points of the person to be evaluated, the second information includes the features of at least one key posture of the motion, and the extracting at least one video key frame from the video data using the first information and the second information includes: For each video frame, using the body key point data of the person to be evaluated, analyzing the positional relationship between the body key points of the person to be evaluated to obtain an analysis result; Using the analysis result and the feature of at least one key posture of the motion action, it is determined whether the video frame is a video key frame.

4. The method according to claim 3, characterized in that The determining whether a video frame is a video key frame by using the analysis result and the feature of at least one key posture of the motion action comprises: In the case where the analysis result matches the feature of at least one key gesture of the motion action, it is determined that the video frame is a video key frame.

5. The method according to claim 4, characterized in that The key postures of the motion action include at least two, and the method further includes: The at least one video key frame forms a key gesture sequence; When it is determined that a first video key frame included in the at least one video key frame does not match video key frames of all other gestures except the key gesture of the first video key frame in the at least two key gestures in the key gesture sequence, the first video frame is deleted from the extracted at least one video key frame.

6. The method according to claim 1, characterized in that The step of evaluating the motion performed by the person to be evaluated by using the extracted at least one video key frame includes: For each video key frame, the movement of the person to be evaluated is evaluated using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame, wherein the first parameter represents a symmetry degree benchmark of the movement, and the second parameter represents a completion degree benchmark of the movement.

7. The method according to claim 6, characterized in that The step of evaluating the motion of the person to be evaluated by using the body key point data, the first parameter and / or the second parameter of the person to be evaluated in the video key frame includes: Using the body key point data and the first parameter of the person to be evaluated in the video key frame, evaluating the degree of symmetry of the action of the person to be evaluated; and / or, The body key point data and the second parameter of the person to be evaluated in the video key frame are used to evaluate the degree of completion of the action of the person to be evaluated.

8. The method according to any one of claims 1 to 7, characterized in that: The determining the first information by using the video data includes: The first information is determined using the video data and a first model.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Using the video data and the second model, identifying a body frame and a corresponding body frame confidence of at least one candidate, wherein the at least one candidate includes the person to be evaluated; When the confidence level of the body frame is less than a preset confidence level, third information is generated, and the third information is used for warning.

10. A method for evaluating motion quality, characterized in that: include: Acquire video data of a person to be evaluated performing a movement, wherein the movement includes a symmetrical movement; evaluating the motion using key frame information of the video data; An evaluation result is output, wherein the evaluation result includes a symmetry degree of the action and / or a completion degree of the action.

11. An action quality assessment device, characterized in that: include: An acquisition unit, used for acquiring video data of the movement of the person to be evaluated; An evaluation unit is used to determine first information using the video data, wherein the first information represents body key point data of the person to be evaluated in each video frame; extract at least one video key frame from the video data using the first information and second information, wherein the second information is associated with action features of a movement performed by the person to be evaluated, wherein the movement includes symmetrical movements; and evaluate the movement performed by the person to be evaluated using the extracted at least one video key frame.

12. An action quality assessment device, characterized in that: include: An acquisition unit, used for acquiring video data of a person to be evaluated performing a movement, wherein the movement includes a symmetrical movement; An evaluation unit, configured to evaluate the motion using key frame information of the video data; The output unit is used to output the evaluation result, wherein the evaluation result includes the symmetry degree of the action and / or the completion degree of the action.

13. An electronic device, characterized in that: include: a processor and a first memory for storing a computer program executable on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 10.

14. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images