A posture evaluation method, apparatus, device, storage medium, and computer program product.
By acquiring benchmark videos and videos to be evaluated from different evaluation dimensions, and dividing and processing the video information of each dimension, the problem of low accuracy in dance evaluation is solved, and a multi-dimensional comprehensive evaluation is achieved.
Patent Information
- Application Number
- CN202410704227.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-31
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-05-31
AI Technical Summary
Existing dance evaluation technologies have low accuracy and cannot effectively consider multiple evaluation dimensions, resulting in inaccurate evaluation results.
By acquiring benchmark videos of different evaluation dimensions, the videos to be evaluated for the target objects are divided, each evaluation dimension is processed to determine the evaluation information, and a comprehensive evaluation is performed based on multiple evaluation information.
It improves the accuracy of dance evaluation, taking into account multiple evaluation dimensions of movement and posture, and provides more accurate evaluation results.
Smart Images

Figure CN118736666B_ABST
Abstract
Description
Technical Field
[0001] This application relates to attitude evaluation technology in the field of computers, and more particularly to an attitude evaluation method, apparatus, device, storage medium, and computer program product. Background Technology
[0002] With the advancement of education evaluation system reform, the digital transformation of the examination field is accelerating. As a crucial component of college art program admissions, dance examinations are seeing increasingly intelligent evaluation methods for candidates. Currently, related technologies compare the similarity of a candidate's movement posture in a specific dimension with the standard posture of a standard object at the same time to obtain an evaluation result for the candidate's movement posture, and then score the candidate based on this single-dimensional evaluation result. However, these methods suffer from low accuracy in posture evaluation. Summary of the Invention
[0003] To address the aforementioned technical problems, embodiments of this application provide a posture evaluation method, apparatus, device, storage medium, and computer program product, which solves the problem of low accuracy in posture evaluation during dance evaluation in related technologies.
[0004] The technical solution of this application is implemented as follows: A posture evaluation method, the method comprising: Obtain benchmark videos with different evaluation dimensions; wherein, the different evaluation dimensions represent different evaluation standards for the action; The video of the target object's pose to be evaluated is obtained, and the video to be evaluated is divided into multiple sub-videos to be evaluated based on the different evaluation dimensions; wherein, the actions of the objects in the benchmark video and the sub-videos to be evaluated are the same for the same evaluation dimension. For each evaluation dimension, each benchmark video and each sub-video to be evaluated are processed to determine the evaluation information for each evaluation dimension; Based on multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
[0005] In the above scheme, the step of processing each benchmark video and each sub-video to be evaluated for each evaluation dimension to determine the evaluation information for each evaluation dimension includes: For each sub-video to be evaluated, based on the position of the key point of the part corresponding to the first pose in each first image of the sub-video to be evaluated in the first image, the first vector of the part corresponding to the first pose is determined. For each reference video, based on the position of the key points of the part corresponding to the second pose in each second image of the reference video in the second image, a second vector of the part corresponding to the second pose is determined; The evaluation information for each evaluation dimension is determined by processing multiple first vectors and multiple second vectors.
[0006] In the above scheme, processing multiple first vectors and multiple second vectors to determine the evaluation information for each evaluation dimension includes: Obtain sample videos for different evaluation dimensions; wherein, the actions of the objects in the sample videos and the benchmark videos corresponding to the same evaluation dimension are the same; For each of the sample videos, based on the position of the key points of the part corresponding to the sample pose in each sample image of the sample video, the sample vector of the part corresponding to the sample pose is determined. For each evaluation dimension, multiple benchmark parameters are determined based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors; Based on multiple first vectors and multiple benchmark parameters, evaluation information is determined for each evaluation dimension.
[0007] In the above scheme, for each evaluation dimension, determining multiple benchmark parameters based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors includes: The third vector is determined from the second vector for each part corresponding to the second pose in the second image of the benchmark video for the first evaluation dimension; For the same part, the third vector is adjusted based on the adjustment parameters of the first evaluation dimension and the sample vector of the part corresponding to the sample pose in the sample image of the sample video of the first evaluation dimension to obtain the reference vector; wherein, the reference parameters include the reference vector.
[0008] In the above scheme, determining the evaluation information for each evaluation dimension based on multiple first vectors and multiple benchmark parameters includes: For the first evaluation dimension, the first vector and the benchmark vector of the same part are matched, and the evaluation information of the first evaluation dimension is determined based on multiple matching results.
[0009] In the above scheme, determining multiple benchmark parameters for each evaluation dimension based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors further includes: Based on multiple sample vectors of each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension, the first parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension is determined. Based on multiple first parameters, the second parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension is determined. Based on the second vectors of each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension, a third parameter between each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension is determined. For the same adjacent part, the third parameter is adjusted based on the adjustment parameter of the second evaluation dimension and the second parameter to obtain the adjusted third parameter; wherein, the benchmark parameter includes the adjusted third parameter.
[0010] In the above scheme, determining the evaluation information for each evaluation dimension based on multiple first vectors and multiple benchmark parameters further includes: Based on multiple first vectors, the target parameters between each adjacent part in the part corresponding to the first pose in the first image of the sub-video to be evaluated in the second evaluation dimension are determined. For the second evaluation dimension, the target parameters of the same adjacent parts are matched with the adjusted third parameters, and the evaluation information of the second evaluation dimension is determined based on multiple matching results.
[0011] In the above scheme, determining the target evaluation information of the posture to be evaluated of the target object based on multiple evaluation information includes: Determine the weight corresponding to each evaluation dimension; Based on multiple weights and multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
[0012] An attitude evaluation device, the device comprising: An acquisition unit is used to acquire benchmark videos with different evaluation dimensions; the different evaluation dimensions represent different evaluation criteria for the action. The processing unit is used to acquire the video of the pose of the target object to be evaluated, and divide the video of the object to be evaluated into multiple sub-videos to be evaluated based on the different evaluation dimensions; wherein, the objects in the benchmark video and the sub-videos to be evaluated have the same action corresponding to the same evaluation dimension. The first determining unit is used to process each benchmark video and each sub-video to be evaluated for each evaluation dimension to determine the evaluation information for each evaluation dimension. The second determining unit is used to determine the target evaluation information of the posture to be evaluated of the target object based on multiple evaluation information.
[0013] An attitude evaluation device, the device comprising: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the attitude evaluation program in the memory to implement the steps of the attitude evaluation method described above.
[0014] A computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the attitude evaluation method described above.
[0015] A computer program product comprising a computer program that, when executed by a processor, implements the aforementioned attitude evaluation method.
[0016] The posture evaluation method, apparatus, device, storage medium, and computer program product provided in the embodiments of this application can acquire reference videos representing different evaluation dimensions of different evaluation standards for actions, and divide the acquired video of the posture to be evaluated of the target object into multiple sub-videos to be evaluated based on different evaluation dimensions. Then, for each evaluation dimension, each reference video and each sub-video to be evaluated are processed to determine the evaluation information of each evaluation dimension. Finally, based on multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined. In this way, the video of the posture to be evaluated of the target object can be divided into multiple sub-videos to be evaluated according to different evaluation dimensions, and the reference video and sub-video to be evaluated of each evaluation dimension can be processed to obtain the evaluation information of the posture of each evaluation dimension. Then, the evaluation information of each evaluation dimension is combined to comprehensively evaluate the posture of the target object, instead of evaluating only a single dimension of posture as in related technologies. This solves the problem of low accuracy of posture evaluation in dance evaluation in related technologies. Attached Figure Description
[0017] Figure 1 A flowchart illustrating an attitude evaluation method provided in an embodiment of this application; Figure 2 A flowchart illustrating another attitude evaluation method provided in this application embodiment; Figure 3A schematic diagram showing the distribution of key points of human body parts in a posture evaluation method provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of an attitude evaluation device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an attitude evaluation device provided in an embodiment of this application. Detailed Implementation
[0018] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0019] It should be understood that the phrases "embodiments of this application" or "foreign embodiments" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "embodiments of this application" or "in the foreign embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.
[0020] Unless otherwise specified, any step in the embodiments of this application performed by the electronic device may be executed by the processor of the electronic device. It is also worth noting that the embodiments of this application do not limit the order in which the electronic device performs the following steps. Furthermore, the methods used to process data in different embodiments may be the same or different methods. It should also be noted that any step in the embodiments of this application can be executed independently by the electronic device; that is, when the electronic device performs any step in the following embodiments, it may not depend on the execution of other steps.
[0021] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.
[0022] This application provides an attitude evaluation method, which can be applied to attitude evaluation equipment. (Refer to...) Figure 1 As shown, the method may include the following steps: Step 101: Obtain benchmark videos for different evaluation dimensions.
[0023] Different evaluation dimensions represent different evaluation criteria for actions.
[0024] In the embodiments of this application, the reference video may refer to a recorded dance video of a reference object; in one possible implementation, the reference object may refer to a dancer with solid dance skills.
[0025] In this application embodiment, if this application is applied to the unified art examination scenario, the benchmark video may refer to a recorded dance video of a professional dancer; the evaluation dimension may be determined according to the evaluation standards of the unified art examination. In this case, the evaluation dimension may include a specific skill dimension, a basic skill dimension, and a basic form dimension; wherein, the specific skill dimension may characterize the stability of the candidate's limbs; the basic skill dimension may characterize the flexibility of the candidate's limbs; and the basic form dimension may characterize the proportional information of the candidate's limbs.
[0026] Step 102: Obtain the video of the pose to be evaluated of the target object, and divide the video to be evaluated into multiple sub-videos based on different evaluation dimensions.
[0027] In this evaluation, the actions of the objects in the benchmark video and the sub-video to be evaluated are identical for the same evaluation dimension. In other words, the dance movements performed by the objects in the benchmark video and the sub-video to be evaluated are the same. In one feasible approach, the dance movements may include leg lifts and fourth-position turns.
[0028] In this embodiment of the application, the video to be evaluated may refer to a recorded dance video of the target object, and the target object performs corresponding dance movements by controlling its own body parts, and the body parts may be composed of key points of the body parts.
[0029] In this embodiment, after obtaining the video of the target object's pose to be evaluated, the video can be segmented according to different evaluation dimensions to obtain sub-videos for each evaluation dimension. Specifically, a deep learning algorithm can be used to identify key point information of human body parts in the video to be evaluated and the benchmark video. Then, all images in the video to be evaluated are traversed sequentially to extract the sum of the variances of the offsets of all human body parts key points between adjacent images to form a variance sequence. At the same time, multiple variance sequences of human body parts key points corresponding to the benchmark action are constructed based on the key point information of human body parts in the benchmark videos of different evaluation dimensions. After processing the variance sequences of the video to be evaluated and the multiple variance sequences of the benchmark videos using a clustering algorithm, the variance sequence of the video to be evaluated is matched with the variance sequence of the benchmark videos for each evaluation dimension to determine the start and end images of the video to be evaluated for each evaluation dimension. Then, the video to be evaluated is segmented to obtain sub-videos for each evaluation dimension.
[0030] It should be noted that deep learning algorithms can refer to Convolutional Neural Networks (CNNs) algorithms or top-down human keypoint detection algorithms.
[0031] Step 103: For each evaluation dimension, process each benchmark video and each sub-video to be evaluated to determine the evaluation information for each evaluation dimension.
[0032] In this embodiment of the application, for each evaluation dimension, the pose corresponding to each image of the benchmark video and the pose corresponding to each image of the sub-video to be evaluated can be matched to determine the evaluation information of the target object for each evaluation dimension.
[0033] In one feasible approach, if this application is applied to the unified art examination scenario, the postures in each frame of the benchmark video and the sub-video to be evaluated for a specific skill dimension, the postures in each frame of the benchmark video and the sub-video to be evaluated for a basic skill dimension, and the postures in each frame of the benchmark video and the sub-video to be evaluated for a basic form dimension can be matched to obtain evaluation information of the action postures for the specific skill dimension, the basic skill dimension, and the basic form dimension.
[0034] Step 104: Based on multiple evaluation information, determine the target evaluation information of the posture to be evaluated of the target object.
[0035] In this embodiment, after determining the evaluation information of the posture to be evaluated for each evaluation dimension, the weight of each evaluation dimension can be determined. Then, based on the weight and evaluation information of each evaluation dimension among multiple evaluation dimensions, the target evaluation information of the posture to be evaluated of the target object is determined. In one feasible approach, the candidate's dance posture can be comprehensively evaluated based on the weight and evaluation information of specific skill dimensions, the weight and evaluation information of basic skills dimensions, and the weight and evaluation information of basic form dimensions.
[0036] The posture evaluation method provided in the embodiments of this application can divide the video of the posture to be evaluated of the target object into multiple sub-videos according to different evaluation dimensions, and process the benchmark video and the sub-video to be evaluated for each evaluation dimension to obtain the posture evaluation information for each evaluation dimension. Then, the evaluation information of each evaluation dimension is combined to comprehensively evaluate the posture of the target object, instead of evaluating only a single dimension of posture as in related technologies. This solves the problem of low accuracy of posture evaluation in dance evaluation in related technologies.
[0037] Based on the foregoing embodiments, embodiments of this application provide a posture evaluation method, referring to... Figure 2 As shown, the method may include the following steps: Step 201: The posture evaluation device acquires benchmark videos for different evaluation dimensions.
[0038] Different evaluation dimensions represent different evaluation criteria for actions.
[0039] In one feasible approach, if this application is applied to a standardized arts entrance examination, the benchmark videos can refer to three videos recorded by professional dancers corresponding to the three evaluation dimensions. Specifically, the benchmark videos can include benchmark videos for specific skill dimensions, benchmark videos for basic skills dimensions, and benchmark videos for basic form dimensions. The benchmark videos for specific skill dimensions can include three movements: a flat turn, a fourth position turn, and a mid-air jump; the benchmark videos for basic skills dimensions can include three movements: leg lifts, leg controls, and backbends; and the benchmark videos for basic form dimensions can include upright movements.
[0040] Step 202: The posture evaluation device acquires the video of the posture to be evaluated of the target object, and divides the video of the posture to be evaluated into multiple sub-videos based on different evaluation dimensions.
[0041] In this evaluation, the actions of the objects in the benchmark video and the sub-video to be evaluated are identical for the same evaluation dimension. In other words, the dance moves performed by the objects in the benchmark video and the sub-video to be evaluated for the same evaluation dimension are identical. In one feasible approach, the dance moves performed by the objects in both the benchmark video and the sub-video to be evaluated for the same evaluation dimension can be either aerial leaps or leg controls.
[0042] It should be noted that the target audience performs corresponding dance movements by controlling their own body parts, and these body parts can be composed of key points of the body. In one possible implementation, body parts can include the left shoulder, right shoulder, left upper arm, right upper arm, left forearm, right forearm, left upper body, right upper body, left and right hips, left thigh, right thigh, left calf, and right calf, while key points of the body can include the neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.
[0043] In one feasible approach, the distribution of key points in body parts is as follows: Figure 3As shown, at this point, a deep learning algorithm can be used to identify the key point information of 13 human body parts in the video to be evaluated and the benchmark video, that is, the coordinate information of the 13 key points of human body parts in each frame of the image. After determining the coordinates of each key point of human body parts in each frame of the video to be evaluated, the variance of the x-coordinate and the variance of the y-coordinate of a key point in the first frame and a key point in the second frame are calculated. The sum of the variances of the x-coordinate and the y-coordinate is determined as the positional variance of a key point between the first and second frames. Based on this, the positional variances of the 13 key points of human body parts between the first and second frames are calculated respectively. Finally, the positional variances of the 13 key points of human body parts are calculated to obtain the sum of the offset variances of all key points of human body parts between the first and second frames. By analogy, the variance sequence of the video to be evaluated can be obtained. ,and It can be .in, It can represent the sum of the offset variances of all key points of human body parts between the second and third frame images; This can be represented as the sum of the offset variances of all human body keypoints between the (N-1)th frame and the Nth frame. It should be noted that at the beginning of video recording, the subject is standing, and the variance of the human body keypoints between adjacent frames is small, indicating relatively smooth movement. From the start to the end of the dance, the positions of the subject's keypoints change continuously, resulting in a larger variance between adjacent frames and greater movement fluctuations. From the end of the dance to the end of video recording, the subject is standing again, the variance between adjacent frames decreases again, and the movement becomes smooth again. The coordinates of the 13 human body keypoints in each frame are as follows: neck... left shoulder right shoulder , left elbow , right elbow , left wrist , right wrist Left hip Right hip left knee , right knee left ankle right ankle It should be noted that i represents the frame number of the image.
[0044] Furthermore, after determining the variance sequence of the video to be evaluated, a clustering algorithm is used to divide the variance sequence into large value categories. and small value category Among them, the large value category This can include variances showing significant variations in key points of human body parts between adjacent images. Small value category This can include the variance of small variations in key points of human body parts between adjacent images. And so on, calculating the variance sequence of the benchmark video for each evaluation dimension. , and .in, include and , include and , include and .
[0045] For each evaluation dimension, the benchmark video is used as the class with the largest value in its variance sequence as the matching kernel, and the data is then compared with the variance sequence of the video to be evaluated. Large value category in the column Starting from the first position in the algorithm, a sliding motion with a step size of 1 is performed, and the relationship between the sliding motion and the algorithm is calculated during the sliding process. The similarity is calculated, and the sliding stops when the character reaches the last position, and the characters that were sliding during the process are compared with the character's appearance. The image with the highest similarity variance is used as the starting frame image to match the benchmark video for that evaluation dimension. Then, the same method is used to determine the ending frame image to match the benchmark video for that evaluation dimension, and the video between the starting frame image and the ending frame image is extracted as the sub-video to be evaluated for that evaluation dimension. By analogy, sub-videos to be evaluated for different evaluation dimensions can be obtained.
[0046] It should be noted that for the benchmark video and the sub-video to be evaluated for the same evaluation dimension, the Dynamic Time Warping algorithm needs to be used to align the benchmark video and the sub-video to be evaluated in order to enable subsequent video comparison.
[0047] Step 203: For each sub-video to be evaluated, the posture evaluation device determines the first vector of the part corresponding to the first posture in the first image based on the position of the key points of the part corresponding to the first posture in each first image of the sub-video to be evaluated.
[0048] In this embodiment, the position of the key point corresponding to the first pose in the first image can refer to the coordinates of the key point corresponding to the first pose in the first image. In this case, the pose evaluation device can subtract the coordinates of adjacent key points of human body parts in the first image to obtain the first vector of the human body part formed by adjacent key points. For example, the coordinates of the left elbow and left shoulder in the first image can be subtracted to obtain the vector of the left upper arm, and so on, calculating the first vector of all human body parts formed by all adjacent key points in each first image. It should be noted that each part in each image of the sub-video to be evaluated corresponds to one first vector.
[0049] In one feasible approach, if the coordinates of the key points of the human body parts corresponding to the pose of the target object in each first image of the sub-video to be evaluated are respectively neck... left shoulder right shoulder , left elbow , right elbow , left wrist , right wrist left hip right hip left knee , right knee left ankle right ankle Then the vector of the human body part can be obtained as the left shoulder. right shoulder left upper arm right upper arm left forearm right forearm upper left body , upper right body left and right hips left thigh right thigh left calf right calf .
[0050] Step 204: For each reference video, the attitude evaluation device determines the second vector of the part corresponding to the second attitude in the second image based on the position of the key points of the part corresponding to the second attitude in each second image of the reference video.
[0051] In this embodiment, the position of the key points of the human body parts corresponding to the second pose in the second image can refer to the specific coordinates of the key points of the human body parts corresponding to the second pose in the second image. In this case, the coordinates of adjacent key points of human body parts in the second image can be subtracted to obtain the second vector of the human body parts formed by adjacent key points of human body parts in the second image. For example, the coordinates of the right elbow and the right shoulder in the image can be subtracted to obtain the vector of the right upper arm, and so on, calculating the second vector of all human body parts formed by all adjacent key points of human body parts in each second image. It should be noted that each part in each image of the reference video corresponds to one second vector.
[0052] In one feasible approach, if the coordinates of the key points of the human body parts corresponding to the second pose of the object in each second image of the reference video are respectively neck left shoulder right shoulder , left elbow , right elbow , left wrist , right wrist left hip right hip left knee , right knee left ankle right ankle Based on this, the second vector of the human body part can be obtained as the left shoulder. right shoulder left upper arm right upper arm left forearm right forearm upper left body , upper right body left and right hips left thigh right thigh left calf right calf .
[0053] Step 205: The attitude evaluation device processes multiple first vectors and multiple second vectors to determine the evaluation information for each evaluation dimension.
[0054] In the embodiments of this application, the evaluation information for each evaluation dimension may refer to the evaluation score for each evaluation dimension.
[0055] In one feasible approach, for each evaluation dimension, the first and second vectors corresponding to the poses of each part in the same frame of the sub-video to be evaluated and the benchmark video can be directly matched for similarity to obtain the pose similarity matching result in each frame. Then, based on the pose similarity matching result in each frame, the similarity matching result between the sub-video to be evaluated and the benchmark video is calculated. Finally, the similarity matching result between the sub-video to be evaluated and the benchmark video for each evaluation dimension is calculated to obtain the evaluation score (i.e., evaluation information) for each evaluation dimension.
[0056] In the embodiments of this application, step 205 can be implemented by steps 205a to 205d.
[0057] Step 205a: The posture evaluation device acquires sample videos of different evaluation dimensions.
[0058] In this context, the actions of the objects in the sample video and the benchmark video corresponding to the same evaluation dimension are identical; that is, the dance moves performed by the objects in the benchmark video and the sample video corresponding to the same evaluation dimension are the same. In one feasible approach, the dance moves performed by the objects in the benchmark video and the sample video corresponding to the same evaluation dimension can both be aerial leaps or both be horizontal turns.
[0059] In this embodiment, the sample video can refer to a recorded dance video of a sample object across different evaluation dimensions. If this application is applied to an arts entrance examination scenario, the sample video can refer to a dance video of a historically outstanding candidate or a dance video of a current year's outstanding candidate. It should be noted that each evaluation dimension can correspond to more than one sample video; that is, each evaluation dimension can have multiple sample videos. Furthermore, for the same evaluation dimension, the actions of the object in each sample video are identical; that is, the dance movements performed by the object in each sample video are the same.
[0060] Step 205b: For each sample video, the posture evaluation device determines the sample vector of the part corresponding to the sample posture in the sample image based on the position of the key points of the part corresponding to the sample posture in each sample image of the sample video.
[0061] In this embodiment, the position of the key points of the human body parts corresponding to the sample pose in the sample image can refer to the coordinates of the key points of the human body parts corresponding to the sample pose in the sample image. In this case, the pose evaluation device can subtract the coordinates of adjacent key points of human body parts in the sample image to obtain the sample vector of the human body parts composed of adjacent key points of human body parts in the sample image. For example, the coordinates of the left knee and the left ankle in the sample image can be subtracted to obtain the vector of the left calf, and so on, to calculate the sample vector of all human body parts composed of all adjacent key points of human body parts in each sample image. It should be noted that each part in each image of the sample video corresponds to a sample vector.
[0062] In one feasible approach, if the coordinates of the key points of the human body parts corresponding to the sample pose in each sample image of the sample video are respectively neck left shoulder right shoulder , left elbow , right elbow , left wrist , right wrist left hip right hip left knee , right knee left ankle right ankle Based on this, the sample vector for the human body part can be obtained as the left shoulder. right shoulder left upper arm right upper arm left forearm right forearm upper left body , upper right body left and right hips left thigh right thigh left calf right calf .
[0063] Step 205c: For each evaluation dimension, the attitude evaluation device determines multiple baseline parameters based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors.
[0064] In this embodiment, the adjustment parameters can be used to correct the second pose in the second image of the reference video. The adjustment parameters are values determined based on the specific application scenario and historical data. It should be noted that the adjustment parameters for each evaluation dimension can be the same or different; this is not limited here.
[0065] It should be noted that the methods for determining the benchmark parameters can differ for different evaluation dimensions. In one feasible approach, for a specific evaluation dimension, the second vector of each human body part corresponding to the second posture can be directly modified based on the adjustment parameters and the sample vector of each human body part corresponding to the sample posture, thus obtaining the benchmark parameters for each human body part. Similarly, for other evaluation dimensions, the first parameter can be obtained by first calculating the multiple sample vectors of the human body part corresponding to the sample posture, then the second parameter can be obtained by calculating the multiple second vectors of the human body part corresponding to the second posture using the same method. Finally, the second parameter is modified based on the adjustment parameters and the first parameter to obtain the benchmark parameters.
[0066] It should be noted that the adjustment parameter ranges from [0,1]. When the adjustment parameter is 0, there is no need to adjust the second vector of the part corresponding to the second pose in the second image of the reference video. When the adjustment parameter is 1, the second vector of the part corresponding to the second pose in the second image of the reference video is the sample vector of the part corresponding to the sample pose in the sample image of the sample video. If this application is for the unified art examination scenario, if the adjustment parameter is 0, it means that the posture evaluation is based entirely on the reference video recorded by a professional dancer; if the adjustment parameter is 1, it means that the posture evaluation is based on the sample video recorded by the sample object, that is, the dance video of a historical outstanding candidate or the dance video of an outstanding candidate is used as the reference video.
[0067] In this embodiment, by introducing adjustment parameters for different evaluation dimensions, the scoring criteria are corrected, and the attribute characteristics of the samples themselves can be included in the standard reference range, making the evaluation method of this application more capable of handling different data samples. At the same time, data from some outstanding candidates in this year's competition can be used as benchmark videos, or outstanding historical dance works from this province can be used as benchmark videos. This solves the problem in related technologies where traditional posture matching algorithms ignore the influence of the overall physical fitness of the candidates and the mainstream teaching style on movement guidance.
[0068] In this embodiment of the application, step 205c can be implemented by steps 205c1~205c2 or steps 205c3~205c6.
[0069] Step 205c1: The posture evaluation device determines the third vector of each part corresponding to the second posture in the second image of the reference video of the first evaluation dimension from the second vector.
[0070] In this embodiment of the application, after determining the second vector of the part corresponding to the first pose in the first image of the sub-video to be evaluated for each evaluation dimension, the second vector of the part corresponding to the second pose in the second image of the benchmark video for the first evaluation dimension can be selected as the third vector.
[0071] In this application embodiment, if this application is aimed at the unified art examination scenario, the first evaluation dimension may refer to a specific skill dimension.
[0072] Step 205c2: For the same part, the posture evaluation device adjusts the third vector based on the adjustment parameters of the first evaluation dimension and the sample vector of the part corresponding to the sample posture in the sample image of the sample video of the first evaluation dimension, so as to obtain the reference vector.
[0073] The reference parameters include the reference vector.
[0074] In this embodiment, the first evaluation dimension can correspond to multiple sample videos. It should be noted that the actions of the objects in each sample video are identical; that is, the dance movements performed by the objects in each sample video are the same. In this case, for the same part corresponding to the sample posture in each sample video of the first evaluation dimension, the sample vectors of that part corresponding to the sample posture in all sample videos can be summed to obtain the sum of the sample vectors for each part. Then, the sum of the sample vectors for each part is divided by the number of sample videos to obtain the average sample vector for each part. Afterward, for the same part, based on the adjustment parameters of the first evaluation dimension and the average sample vector for each part, the vectors of each part corresponding to the second posture in each second image of the reference video are corrected to obtain the reference vector for each part. In one feasible approach, for each part of the left shoulder, right shoulder, left upper arm, right upper arm, left lower arm, right lower arm, left upper body, right upper body, left and right hips, left thigh, right thigh, left calf, and right calf, the third vector of each part can be corrected according to the adjustment parameters of the first evaluation dimension and the mean of the sample vector of each part, so as to obtain the baseline vector of each part.
[0075] In this embodiment of the application, if the application targets a specific skill dimension in the unified art examination scenario and the sample video is a dance video of all candidates in this year, then for the same part, the following formula (1) is used to calculate the part vector corresponding to the posture of all candidates in this year in each sample image of the sample video, so as to obtain the mean value of the sample vector of each part: Formula (1) in, Let represent the mean of the sample vectors for the i-th part. Let represent the vector of the i-th part of the first candidate. This represents the vector of the i-th part of the second candidate. The vector represents the i-th part of the nth candidate, where n represents the number of candidates.
[0076] After determining the mean value of the sample vector for each part, the third vector corresponding to the second pose in the second image of the reference video with the same number of frames is adjusted according to formula (2): Formula (2) in, Let S represent the adjusted baseline vector for the i-th part, and let S represent the adjustment parameter for the first evaluation dimension. Let represent the mean of the sample vectors for the i-th part. Let S represent the third vector of the i-th part. In one possible implementation, if S is 0, then the reference vector of each part is the third vector of that part; if S is 1, then the reference vector of each part is the mean of the sample vectors of that part; if S is... Then, the third vector corresponding to the second pose in the second image of the reference video can be adjusted according to formula (3): Formula (3) Step 205c3: The posture evaluation device determines the first parameter between each adjacent part of the part corresponding to the sample posture in the sample image of the sample video of the second evaluation dimension based on multiple sample vectors of each adjacent part.
[0077] In this embodiment, the first parameter can refer to the angle between adjacent parts corresponding to the sample pose, or it can refer to the proportional information between adjacent parts corresponding to the sample pose. If this application is for an art exam scenario, and if the second evaluation dimension is a basic skills evaluation dimension, then the first parameter can refer to the angle between adjacent parts; if the second evaluation dimension is a basic form evaluation dimension, then the first parameter can refer to the proportional information between adjacent parts. It should be noted that each adjacent part corresponds to one first parameter, and for the same adjacent part, multiple sample videos correspond to multiple first parameters.
[0078] In one feasible approach, for the second evaluation dimension, the magnitude of the sample vector of each adjacent part can be determined based on the sample vectors of the two adjacent parts included in the part corresponding to the sample posture. Then, the sample vectors of the two adjacent parts are multiplied to obtain the product of the sample vectors of the adjacent parts. Then, the product of the sample vectors of the adjacent parts and the magnitude of the sample vector of each part are calculated to obtain the angle between the adjacent parts (i.e., the first parameter). In another feasible approach, for the second evaluation dimension, the magnitude of the sample vector of each adjacent part can also be calculated based on the sample vector of each part in the adjacent parts corresponding to the sample posture. Then, the magnitudes of the sample vectors of each part are divided to obtain the ratio between the two sample vectors in the adjacent parts (i.e., the first parameter).
[0079] In one feasible approach, if the second evaluation dimension is the fundamental skills dimension and the corresponding action is a leg lift, then adjacent parts can refer to the left and right thighs. In this case, the angle between the left and right thighs can be obtained based on the sample vectors of the left and right thighs in the parts corresponding to the sample posture. Specifically, if the sample vector of the left thigh in the parts corresponding to the sample posture... = The sample vector of the right thigh At this point, the angle between the left and right thighs can be obtained by calculating the sample vectors of the left and right thighs according to formula (4). : Formula (4) In this embodiment of the application, if the second evaluation dimension is the basic shape dimension and the basic shape dimension corresponds to the body-to-leg ratio, then adjacent parts can refer to the upper left body and the left leg. In this case, the sample vector of the left thigh in the part corresponding to the sample posture can be evaluated first. and the sample vector of the left calf Calculations were performed to obtain the sample vector of the left leg. Then, for the sample vector of the upper left body and the sample vector of the left leg The calculations were performed to obtain the proportion information of the upper left body and the left leg.
[0080] Step 205c4: The posture evaluation device determines the second parameters between each adjacent part of the part corresponding to the posture in the sample image of the sample video of the second evaluation dimension based on multiple first parameters.
[0081] In this embodiment of the application, for the same adjacent part, multiple first parameters corresponding to multiple sample videos can be summed to obtain the sum of the first parameters for the adjacent part in the part corresponding to the sample pose in all sample videos. Then, the sum of the first parameters is divided by the number of first parameters to obtain the second parameter between the adjacent parts. Similarly, the second parameter between each adjacent part in the part corresponding to the sample pose in the sample video of the second evaluation dimension is calculated using the same method.
[0082] In this embodiment of the application, if the second evaluation dimension is the basic skills dimension and the action corresponding to the basic skills dimension is leg lifting, and the sample video is the dance video of all candidates in this year, then for the same adjacent part, the angle between the left thigh and the right thigh corresponding to the posture of all candidates in this year is calculated according to the following formula (5) to obtain the mean angle between the left thigh and the right thigh. : Formula (5) in, This indicates the angle between the left and right thighs. This represents the angle between the left and right thighs of the first examinee. This represents the angle between the left and right thighs of the second examinee. Let represent the angle between the left and right thighs of the nth candidate, where n represents the number of candidates. Similarly, the angle between each adjacent part corresponding to the posture of all candidates in this session can be calculated using formula (6) to obtain the average angle between each adjacent part, which is the second parameter.
[0083] In this embodiment of the application, if the second evaluation dimension is the basic form dimension and the ratio corresponding to the basic form dimension is the body-to-leg ratio, and the sample video is the dance video of all candidates in this year, then for the same adjacent part, the average ratio between the left upper body and the left leg of all candidates in this year is calculated, which is the second parameter.
[0084] Step 205c5: The posture evaluation device determines the third parameter between each adjacent part of the part corresponding to the second posture in the second image of the reference video of the second evaluation dimension based on multiple second vectors of each adjacent part.
[0085] In this embodiment, the third parameter can refer to the angle between adjacent parts corresponding to the second posture, or it can refer to the ratio between adjacent parts corresponding to the second posture. If this application is for an art exam scenario and the second evaluation dimension is a basic skills evaluation dimension, then the third parameter can refer to the angle between adjacent parts corresponding to the second posture; if the second evaluation dimension is a basic form evaluation dimension, then the third parameter can refer to the ratio between adjacent parts corresponding to the second posture. Specifically, multiple second vectors of each adjacent part in the part corresponding to the second posture can be calculated to obtain the angle or ratio information between each adjacent part. Then, multiple angles or multiple ratio information are calculated to obtain the angle or ratio between each adjacent part.
[0086] In one feasible approach, for the second evaluation dimension, the magnitude of the second vector of each adjacent part can be determined based on the second vectors of two adjacent parts in the part corresponding to the second posture. Then, the second vectors of the two adjacent parts are multiplied to obtain the product of the second vectors of the adjacent parts. The product of the second vectors of the adjacent parts and the magnitude of the second vector of each part are then calculated to obtain the angle between the adjacent parts corresponding to the second posture (i.e., the third parameter). In another feasible approach, for the second evaluation dimension, the magnitude of the second vector of each adjacent part in the part corresponding to the second posture can also be calculated based on the second vector of each part in the part corresponding to the second posture. Then, the magnitudes of the second vectors of each part are divided to obtain the ratio between the second vectors of the adjacent parts corresponding to the second posture (i.e., the third parameter).
[0087] In one feasible approach, if the second evaluation dimension is a fundamental skills dimension and the corresponding action is a leg lift, then adjacent parts can refer to the left and right thighs. In this case, the angle between the left and right thighs can be obtained based on the second vectors of the left and right thighs in the parts corresponding to the second posture. In another feasible approach, if the second vector of the left thigh in the parts corresponding to the second posture... = The second vector of the right thigh At this point, the angle between the left and right thighs can be calculated using formula (6). That is, the third parameter: Formula (6) In this embodiment, if the second evaluation dimension is the basic shape dimension and the basic shape dimension corresponds to the body-to-leg ratio, then adjacent parts can refer to the upper left body and the left leg. In this case, the second vector of the left thigh in the part corresponding to the second posture can be evaluated first. and the second vector of the left calf Calculations were performed to obtain the second vector of the left leg. Then, the second vector on the upper left side. and the second vector of the left leg The calculation is performed to obtain the proportion information of the upper left body and the left leg, which is the third parameter.
[0088] Step 205c6: For the same adjacent part, the attitude evaluation device adjusts the third parameter based on the adjustment parameters of the second evaluation dimension and the second parameter to obtain the adjusted third parameter.
[0089] The baseline parameters include the adjusted third parameter.
[0090] In this embodiment, after determining the second parameter between each adjacent part corresponding to the sample pose and the third parameter between each adjacent part corresponding to the second pose, for the second evaluation dimension, the third parameter between each adjacent part corresponding to the second pose can be corrected based on the adjustment parameter and the second parameter between each adjacent part corresponding to the sample pose to obtain the adjusted third parameter. Specifically, the adjustment parameter and the second parameter can be multiplied, and the absolute value of the difference between the adjustment parameter and 1 can be multiplied by the third parameter. Then, the first product of the adjustment parameter and the second parameter, and the second product of the absolute value of the difference between the adjustment parameter and 1 and the third parameter can be calculated. Finally, the first product and the second product are added to obtain the adjusted third parameter.
[0091] In this embodiment, if the second evaluation dimension is the basic skills dimension, then the third parameter can refer to the angle between adjacent parts in the part corresponding to the second posture. In this case, the third parameter corresponding to the second posture in the second image of the reference video with the same number of frames can be adjusted according to formula (7): Formula (7) in, This represents the angle between the i-th adjacent parts after adjustment (i.e., the third parameter after adjustment). This indicates the adjustment parameters for the second evaluation dimension. This represents the average angle between the i-th adjacent parts (i.e., the second parameter). This represents the angle between the i-th adjacent parts (i.e., the third parameter). If the second evaluation dimension is the basic morphology evaluation dimension, then the third parameter can refer to the ratio between adjacent parts corresponding to the second pose. In this case, the third parameter corresponding to the second pose in the second image of the reference video with the same number of frames can be adjusted according to formula (8): Formula (8) in, This represents the proportional information between the i-th adjacent parts after adjustment. This indicates the adjustment parameters for the second evaluation dimension. This represents the average ratio between the i-th adjacent parts (i.e., the second parameter). This represents the ratio between the i-th adjacent parts (i.e., the third parameter).
[0092] Step 205d: The attitude evaluation device determines the evaluation information for each evaluation dimension based on multiple first vectors and multiple reference parameters.
[0093] In this embodiment, for each evaluation dimension, multiple first vectors and multiple reference parameters of the part corresponding to the first pose in each first image of the sub-video to be evaluated are matched to obtain evaluation information for each evaluation dimension. Specifically, for the first evaluation dimension, multiple first vectors and multiple reference vectors can be matched to obtain evaluation information for the first evaluation dimension; for the second evaluation dimension, multiple first vectors can be calculated first to obtain target parameters, and then the target parameters and adjusted third parameters can be matched to obtain evaluation information for the second evaluation dimension.
[0094] It should be noted that if step 205c is implemented through steps 205c1 to 205c2, then step 205d is implemented through step 205d1; if step 205c is implemented through steps 205c3 to 205c6, then step 205d is implemented through steps 205d2 to 205d3.
[0095] Step 205d1: For the first evaluation dimension, the posture evaluation device matches the first vector and the reference vector of the same part, and determines the evaluation information of the first evaluation dimension based on multiple matching results.
[0096] In this embodiment, for the first evaluation dimension, similarity matching can be performed on the first vector and reference vector of the same part corresponding to the pose in the same frame image of the sub-video to be evaluated and the reference video to obtain the similarity matching result of one part in the same frame image. Similarly, similarity matching is performed on the first vector and reference vector of each part in the same frame image of the sub-video to be evaluated and the reference video to obtain the similarity matching result of all parts in a frame image. Then, the similarity matching results of all parts in a frame image are added to obtain the matching result sum value. Then, the similarity matching result sum value of all parts is divided by the number of parts corresponding to the pose to obtain the similarity matching result of a frame image. Similarly, the similarity matching result of each frame image in the sub-video to be evaluated and the reference video is calculated. Then, the similarity matching result of each frame image is summed to obtain the similarity matching result of all frames image in the sub-video to be evaluated and the reference video. Finally, the similarity matching result of all frames image is divided by the number of image frames to obtain the similarity matching result between the sub-video to be evaluated and the reference video. Specifically, according to formula (9), the first vector and the reference vector of the same part corresponding to the pose in the same frame image of the sub-video to be evaluated and the reference video are matched for similarity to obtain the similarity matching result of each human body part.
[0097] Formula (9) in, This represents a metric indicating the similarity between the i-th human body part corresponding to the first pose in the sub-video to be evaluated and the second pose in the benchmark video. Let the first vector represent the i-th body part. Let represent the reference vector of the i-th human body part.
[0098] After determining the similarity metric for each part, the similarity metric between each frame of the sub-video to be evaluated and each frame of the reference video is calculated according to the following formula (10): Formula (10) in, This represents a measure of the similarity between the i-th frame of the sub-video to be evaluated and the i-th frame of the benchmark video. This represents the similarity metric value of the first human body part in the i-th frame image. This represents the similarity metric value of the second human body part in the i-th frame image. This represents the similarity metric for the 13th human body part in the i-th frame image.
[0099] Furthermore, the similarity metric between each frame of the sub-video to be evaluated and the benchmark video in the first evaluation dimension is calculated according to the following formula (11) to obtain the similarity matching value between the sub-video to be evaluated and the benchmark video in the first evaluation dimension, which is the evaluation information of the first evaluation dimension: Formula (11) Where M represents the similarity match value between the sub-video to be evaluated and the benchmark video in the first evaluation dimension. This is a measure of the similarity between the first frame of the sub-video being evaluated and the baseline video. This is a metric representing the similarity between the second frame of the sub-video being evaluated and the baseline video. This is a measure of the similarity between the m-th frame of the sub-video to be evaluated and the benchmark video, where m represents the number of frames in the sub-video to be evaluated in the first evaluation dimension.
[0100] Step 205d2: The posture evaluation device determines the target parameters between each adjacent part of the part corresponding to the first posture in the first image of the sub-video to be evaluated in the second evaluation dimension, based on multiple first vectors.
[0101] In this embodiment, the target parameter can refer to the angle between adjacent parts corresponding to the first posture, or it can refer to the ratio between adjacent parts corresponding to the first posture. If this application is for an art exam scenario and the second evaluation dimension is a basic skills evaluation dimension, then the target parameter can refer to the angle between adjacent parts corresponding to the first posture; if the second evaluation dimension is a basic form evaluation dimension, then the target parameter can refer to the ratio between adjacent parts corresponding to the first posture. Specifically, multiple first vectors of each adjacent part in the part corresponding to the first posture can be calculated to obtain the angle or ratio between each adjacent part. Then, multiple angles or multiple ratios can be calculated to obtain the angle or ratio between each adjacent part.
[0102] In one feasible approach, for the second evaluation dimension, the product of the first vectors of adjacent parts and the magnitude of the first vector of each adjacent part can be determined based on the first vector of the adjacent parts corresponding to the first posture. Then, the product of the first vectors of adjacent parts and the magnitude of the first vector of each part are calculated to obtain the angle between adjacent parts in the first posture (i.e., the target parameter). In another feasible approach, for the second evaluation dimension, the magnitude of the first vector of each adjacent part can also be calculated. Then, the magnitudes of the first vectors of each part are divided to obtain the ratio of the first vectors of adjacent parts corresponding to the first posture (i.e., the target parameter).
[0103] In this embodiment of the application, if the second evaluation dimension is the basic skills dimension and the action corresponding to the basic skills dimension is leg lifting, then the adjacent parts can refer to the left thigh and the right thigh. At this time, the angle between the left thigh and the right thigh, i.e. the target parameter, can be obtained based on the first vector of the left thigh and the right thigh in the part corresponding to the first posture.
[0104] In this embodiment of the application, if the second evaluation dimension is the basic shape dimension and the basic shape dimension corresponds to the body-to-leg ratio, then the adjacent parts can refer to the upper left body and the left leg. In this case, the first vector of the left thigh and the first vector of the left calf in the part corresponding to the first posture can be calculated first to obtain the first vector of the left leg. Then, the first vector of the upper left body and the first vector of the left leg can be calculated to obtain the ratio information between the upper left body and the left leg, i.e., the target parameter.
[0105] Step 205d3: For the second evaluation dimension, the attitude evaluation device matches the target parameters of the same adjacent parts with the adjusted third parameters, and determines the evaluation information of the second evaluation dimension based on multiple matching results.
[0106] In this embodiment of the application, for the second evaluation dimension, the target parameters and adjusted third parameters of the same adjacent parts corresponding to the pose in the same frame image of the sub-video to be evaluated and the reference video can be matched for similarity to obtain the similarity matching result of the adjacent parts. Similarly, the target parameters and third parameters of the same adjacent parts corresponding to the pose in each frame image can be matched for similarity to obtain the similarity matching result of each frame image. Then, the similarity matching results of each frame image are added together to obtain the similarity matching result of all frame images. Finally, the similarity matching result of all frame images is divided by the number of frames of the image to obtain the similarity matching result between the sub-video to be evaluated and the reference video in the second evaluation dimension, that is, the evaluation information of the second evaluation dimension.
[0107] In this application embodiment, if this application is for the unified art examination scenario and the second evaluation dimension is the basic skills dimension, then the similarity matching can be performed between the angles of each adjacent part corresponding to the posture in the same frame image of the sub-video to be evaluated and the benchmark video to obtain the posture similarity matching result in the same frame image. Similarly, the posture similarity matching result in each frame image of the sub-video to be evaluated and the benchmark video is calculated. Then, the posture similarity matching result in each frame image is calculated to obtain the evaluation information of the second evaluation dimension.
[0108] In this embodiment, if the application targets the unified art examination scenario and the second evaluation dimension is the basic morphological dimension, the similarity matching of the proportions between each adjacent part corresponding to the posture in the same frame image of the sub-video to be evaluated and the benchmark video can be performed to obtain the similarity matching result of the posture in the same frame image. Then, the similarity matching result of the posture in each frame image is calculated to obtain the evaluation information of the second evaluation dimension. In one feasible approach, a quantitative evaluation model for calculating the proportions between parts can be constructed based on the ideal proportions between various parts of the human body known in advance. Then, the length, width, or angle of different parts of each candidate can be obtained, and the size indicators are input into the quantitative evaluation model for calculation to obtain the candidate's basic morphological evaluation score.
[0109] Step 206: The attitude evaluation device determines the weight corresponding to each evaluation dimension.
[0110] In this embodiment, the weights represent the importance of each evaluation dimension. Specifically, the weight of each evaluation dimension can be manually determined based on the specific application scenario and historical data. It should be noted that the sum of the weights for each evaluation dimension is 1.
[0111] In this application embodiment, if this application is aimed at the unified art examination scenario, the weight of the specific skill dimension can be set to 0.5, the weight of the basic skills dimension can be set to 0.3, and the weight of the basic form dimension can be set to 0.2.
[0112] Step 207: The attitude evaluation device determines the target evaluation information of the attitude of the target object to be evaluated based on multiple weights and multiple evaluation information.
[0113] In this embodiment of the application, the weight and evaluation information corresponding to each evaluation dimension can be multiplied to obtain the evaluation score of each evaluation dimension. Then, the evaluation scores of each evaluation dimension are added together to obtain the evaluation score of the target object's posture, that is, the target evaluation information of the target object's posture to be evaluated.
[0114] In this embodiment of the application, if the application is aimed at the unified art examination scenario, then the target evaluation information = the evaluation score of a specific skill dimension. 0.5+ in the basic skills dimension evaluation score 0.3+ Evaluation score for basic morphology dimension 0.2.
[0115] Furthermore, targeting the unified art examination scenario, this application selected 10 college students to conduct a simulated dance examination experiment. These 10 students were selected from different skill levels. The proposed scoring method was used for scoring, and five expert judges provided manual scoring. The average score from the five experts was taken as the judges' score. The final scoring results are shown in Table 1 below:
[0116] Table 1 As can be seen from the above scoring comparison, the posture evaluation method of this application uses different methods to compare similarity for three different evaluation dimensions: specific skills, basic skills, and basic form. This achieves intelligent evaluation of the dance examination process, and the scoring results are basically the same as the average score of the five judges, which means that it can effectively achieve intelligent evaluation of candidates. Furthermore, the evaluation process incorporates the similarity of body part movements, the similarity of angles between body parts, and the similarity of body part proportions for comprehensive evaluation, improving the objectivity of posture evaluation. At the same time, it incorporates the candidate's overall data as a benchmark factor, making the evaluation results more realistic and credible.
[0117] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.
[0118] The posture evaluation method provided in the embodiments of this application can divide the video of the posture to be evaluated of the target object into multiple sub-videos according to different evaluation dimensions, and process the benchmark video and the sub-video to be evaluated for each evaluation dimension to obtain the posture evaluation information for each evaluation dimension. Then, the evaluation information of each evaluation dimension is combined to comprehensively evaluate the posture of the target object, instead of evaluating only a single dimension of posture as in related technologies. This solves the problem of low accuracy of posture evaluation in dance evaluation in related technologies.
[0119] Based on the foregoing embodiments, this application provides an attitude evaluation device that can be applied to... Figures 1-2 In the attitude evaluation method provided in the corresponding embodiment, refer to Figure 4 As shown, the attitude evaluation device 3 may include: an acquisition unit 31, a processing unit 32, a first determination unit 33, and a second determination unit 34, wherein: Acquisition unit 31 is used to acquire benchmark videos with different evaluation dimensions; different evaluation dimensions represent different evaluation standards for actions; Processing unit 32 is used to acquire the video of the pose to be evaluated of the target object, and divide the video of the object to be evaluated into multiple sub-videos to be evaluated based on different evaluation dimensions; wherein, the actions of the objects in the benchmark video and the sub-video to be evaluated are the same for the same evaluation dimension. The first determining unit 33 is used to process each benchmark video and each sub-video to be evaluated for each evaluation dimension to determine the evaluation information for each evaluation dimension. The second determining unit 34 is used to determine the target evaluation information of the posture to be evaluated of the target object based on multiple evaluation information.
[0120] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: For each sub-video to be evaluated, based on the position of the key points of the part corresponding to the first pose in the first image of each first image of the sub-video to be evaluated, the first vector of the part corresponding to the first pose is determined. For each reference video, based on the position of the key points of the part corresponding to the second pose in each second image of the reference video in the second image, the second vector of the part corresponding to the second pose is determined. Multiple first vectors and multiple second vectors are processed to determine the evaluation information for each evaluation dimension.
[0121] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: Obtain sample videos for different evaluation dimensions; where the actions of objects in the sample videos and the benchmark videos corresponding to the same evaluation dimension are the same. For each sample video, the sample vector of the part corresponding to the sample pose is determined based on the position of the key point of the part corresponding to the sample pose in each sample image of the sample video. For each evaluation dimension, multiple benchmark parameters are determined based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors. Based on multiple first vectors and multiple benchmark parameters, evaluation information is determined for each evaluation dimension.
[0122] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: The third vector is determined from the second vector for each part of the second image of the benchmark video corresponding to the second pose in the first evaluation dimension; For the same part, the third vector is adjusted based on the adjustment parameters of the first evaluation dimension and the sample vector of the part corresponding to the sample pose in the sample image of the sample video of the first evaluation dimension to obtain the reference vector; wherein, the reference parameters include the reference vector.
[0123] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: For the first evaluation dimension, the first vector and the benchmark vector of the same part are matched, and the evaluation information of the first evaluation dimension is determined based on multiple matching results.
[0124] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: Based on multiple sample vectors of each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension, the first parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension is determined. Based on multiple first parameters, the second parameters between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension are determined. The third parameter between each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension is determined by multiple second vectors of the second part of the part of the second pose in the second image of the benchmark video of the second evaluation dimension. For the same adjacent parts, the third parameter is adjusted based on the adjustment parameters of the second evaluation dimension and the second parameter to obtain the adjusted third parameter; wherein, the benchmark parameter includes the adjusted third parameter.
[0125] In other embodiments of this application, the first determining unit 33 is further configured to perform the following steps: Based on multiple first vectors, the target parameters between each adjacent part in the part corresponding to the first pose in the first image of the sub-video to be evaluated in the second evaluation dimension are determined. For the second evaluation dimension, the target parameters of the same adjacent parts are matched with the adjusted third parameters, and the evaluation information of the second evaluation dimension is determined based on multiple matching results.
[0126] In other embodiments of this application, the second determining unit 34 is further configured to perform the following steps: Determine the weight corresponding to each evaluation dimension; Based on multiple weights and multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
[0127] It should be noted that a detailed explanation of the steps performed by each unit can be found in [reference needed]. Figures 1-2 The attitude evaluation method provided in the corresponding embodiments will not be described in detail here.
[0128] The posture evaluation device provided in the embodiments of this application can divide the video of the posture to be evaluated of the target object into multiple sub-videos according to different evaluation dimensions, and process the reference video and the sub-video to be evaluated for each evaluation dimension to obtain the evaluation information of the posture for each evaluation dimension. Then, the evaluation information of each evaluation dimension is combined to comprehensively evaluate the posture of the target object, instead of evaluating only a single dimension of posture as in related technologies. This solves the problem of low accuracy of posture evaluation in dance evaluation in related technologies.
[0129] Based on the foregoing embodiments, embodiments of this application provide an attitude evaluation device that can be applied to... Figures 1-2 In the attitude evaluation method provided in the corresponding embodiment, refer to Figure 5 As shown, the attitude evaluation device 4 may include: a processor 41, a memory 42, and a communication bus 43, wherein: Communication bus 43 is used to realize the communication connection between processor 41 and memory 42; The processor 41 is used to execute the attitude evaluation program in the memory 42 to perform the following steps: Obtain benchmark videos with different evaluation dimensions; where different evaluation dimensions represent different evaluation standards for the action; The video of the target object's pose to be evaluated is obtained, and the video is divided into multiple sub-videos to be evaluated based on different evaluation dimensions; among them, the objects in the baseline video and the sub-video to be evaluated have the same action for the same evaluation dimension. For each evaluation dimension, each benchmark video and each sub-video to be evaluated are processed to determine the evaluation information for each evaluation dimension. Based on multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
[0130] In other embodiments of this application, the processor 41 is used to execute the attitude evaluation program in the memory 42 to process each reference video and each sub-video to be evaluated for each evaluation dimension, and determine the evaluation information for each evaluation dimension to achieve the following steps: For each sub-video to be evaluated, based on the position of the key points of the part corresponding to the first pose in the first image of each first image of the sub-video to be evaluated, the first vector of the part corresponding to the first pose is determined. For each reference video, based on the position of the key points of the part corresponding to the second pose in each second image of the reference video in the second image, the second vector of the part corresponding to the second pose is determined. Multiple first vectors and multiple second vectors are processed to determine the evaluation information for each evaluation dimension.
[0131] In other embodiments of this application, the processor 41 is configured to process multiple first vectors and multiple second vectors stored in the memory 42 to determine evaluation information for each evaluation dimension, thereby implementing the following steps: Obtain sample videos for different evaluation dimensions; where the actions of objects in the sample videos and the benchmark videos corresponding to the same evaluation dimension are the same. For each sample video, the sample vector of the part corresponding to the sample pose is determined based on the position of the key point of the part corresponding to the sample pose in each sample image of the sample video. For each evaluation dimension, multiple benchmark parameters are determined based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors. Based on multiple first vectors and multiple benchmark parameters, evaluation information is determined for each evaluation dimension.
[0132] In other embodiments of this application, the processor 41 is further configured to execute the attitude evaluation program in the memory 42 to determine multiple reference parameters for each evaluation dimension based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors, in order to achieve the following steps: The third vector is determined from the second vector for each part of the second image of the benchmark video corresponding to the second pose in the first evaluation dimension; For the same part, the third vector is adjusted based on the adjustment parameters of the first evaluation dimension and the sample vector of the part corresponding to the sample pose in the sample image of the sample video of the first evaluation dimension to obtain the reference vector; wherein, the reference parameters include the reference vector.
[0133] In other embodiments of this application, the processor 41 is further configured to execute the attitude evaluation program in the memory 42 to determine evaluation information for each evaluation dimension based on a plurality of first vectors and a plurality of reference parameters, in order to implement the following steps: For the first evaluation dimension, the first vector and the benchmark vector of the same part are matched, and the evaluation information of the first evaluation dimension is determined based on multiple matching results.
[0134] In other embodiments of this application, the processor 41 is further configured to execute the attitude evaluation program in the memory 42 to determine multiple reference parameters for each evaluation dimension based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors, in order to achieve the following steps: Based on multiple sample vectors of each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension, the first parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension is determined. Based on multiple first parameters, the second parameters between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension are determined. The third parameter between each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension is determined by multiple second vectors of the second part of the part of the second pose in the second image of the benchmark video of the second evaluation dimension. For the same adjacent parts, the third parameter is adjusted based on the adjustment parameters of the second evaluation dimension and the second parameter to obtain the adjusted third parameter; wherein, the benchmark parameter includes the adjusted third parameter.
[0135] In other embodiments of this application, the processor 41 is further configured to execute the attitude evaluation program in the memory 42 to determine evaluation information for each evaluation dimension based on a plurality of first vectors and a plurality of reference parameters, in order to implement the following steps: Based on multiple first vectors, the target parameters between each adjacent part in the part corresponding to the first pose in the first image of the sub-video to be evaluated in the second evaluation dimension are determined. For the second evaluation dimension, the target parameters of the same adjacent parts are matched with the adjusted third parameters, and the evaluation information of the second evaluation dimension is determined based on multiple matching results.
[0136] In other embodiments of this application, the processor 41 is further configured to execute the attitude evaluation program in the memory 42 to determine the target evaluation information of the attitude of the target object to be evaluated based on multiple evaluation information, so as to implement the following steps: Determine the weight corresponding to each evaluation dimension; Based on multiple weights and multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
[0137] It should be noted that a detailed description of the steps performed by the processor can be found in [reference needed]. Figures 1-2 The attitude evaluation method provided in the corresponding embodiments will not be described in detail here.
[0138] The posture evaluation device provided in the embodiments of this application can divide the video of the posture to be evaluated of the target object into multiple sub-videos according to different evaluation dimensions, and process the benchmark video and the sub-video to be evaluated for each evaluation dimension to obtain the evaluation information of the posture for each evaluation dimension. Then, the evaluation information of each evaluation dimension is combined to comprehensively evaluate the posture of the target object, instead of evaluating only a single dimension of posture as in related technologies. This solves the problem of low accuracy of posture evaluation in dance evaluation in related technologies.
[0139] Based on the foregoing embodiments, embodiments of this application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to achieve... Figures 1-2 The corresponding embodiments provide the steps of the attitude evaluation method.
[0140] Based on the foregoing embodiments, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements... Figures 1-2 The corresponding embodiments provide the steps of the attitude evaluation method.
[0141] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable attitude evaluation device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable attitude evaluation device, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable attitude evaluation device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0144] These computer program instructions can also be loaded onto a computer or other programmable attitude evaluation device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A posture evaluation method, characterized in that, The method includes: Obtain benchmark videos with different evaluation dimensions; wherein, the different evaluation dimensions represent different evaluation standards for the action; The video of the target object's pose to be evaluated is obtained, and the video to be evaluated is divided into multiple sub-videos to be evaluated based on the different evaluation dimensions; wherein, the actions of the objects in the benchmark video and the sub-videos to be evaluated are the same for the same evaluation dimension. For each sub-video to be evaluated, a first vector is determined for the part corresponding to the first pose based on the position of the key point of the part corresponding to the first pose in each first image of the sub-video to be evaluated in the first image; for each reference video, a second vector is determined for the part corresponding to the second pose based on the position of the key point of the part corresponding to the second pose in each second image of the reference video in the second image. Obtain sample videos for different evaluation dimensions; wherein, the actions of objects in the sample videos and the benchmark videos corresponding to the same evaluation dimension are the same; for each sample video, based on the position of the key points of the part corresponding to the sample posture in each sample image of the sample video, determine the sample vector of the part corresponding to the sample posture; for each evaluation dimension, based on the adjustment parameters of each evaluation dimension, multiple sample vectors and multiple second vectors, determine multiple benchmark parameters; based on the multiple first vectors and the multiple benchmark parameters, determine the evaluation information for each evaluation dimension; Based on multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
2. The method according to claim 1, characterized in that, For each evaluation dimension, based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors, multiple benchmark parameters are determined, including: The third vector is determined from the second vector for each part corresponding to the second pose in the second image of the benchmark video for the first evaluation dimension; For the same part, the third vector is adjusted based on the adjustment parameters of the first evaluation dimension and the sample vector of the part corresponding to the sample pose in the sample image of the sample video of the first evaluation dimension to obtain the reference vector; wherein, the reference parameters include the reference vector.
3. The method according to claim 2, characterized in that, The step of determining evaluation information for each evaluation dimension based on multiple first vectors and multiple benchmark parameters includes: For the first evaluation dimension, the first vector and the benchmark vector of the same part are matched, and the evaluation information of the first evaluation dimension is determined based on multiple matching results.
4. The method according to claim 1, characterized in that, For each evaluation dimension, based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors, multiple benchmark parameters are determined, including: Based on multiple sample vectors of each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension, the first parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension is determined. Based on multiple first parameters, determine the second parameter between each adjacent part in the part corresponding to the sample pose in the sample image of the sample video of the second evaluation dimension. Based on the multiple second vectors of each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension, a third parameter between each adjacent part of the part corresponding to the second pose in the second image of the benchmark video of the second evaluation dimension is determined. For the same adjacent part, the third parameter is adjusted based on the adjustment parameter of the second evaluation dimension and the second parameter to obtain the adjusted third parameter; wherein, the benchmark parameter includes the adjusted third parameter.
5. The method according to claim 4, characterized in that, The step of determining evaluation information for each evaluation dimension based on multiple first vectors and multiple benchmark parameters includes: Based on multiple first vectors, the target parameters between each adjacent part in the part corresponding to the first pose in the first image of the sub-video to be evaluated in the second evaluation dimension are determined. For the second evaluation dimension, the target parameters of the same adjacent parts are matched with the adjusted third parameter, and the evaluation information of the second evaluation dimension is determined based on multiple matching results.
6. The method according to claim 1, characterized in that, The step of determining the target evaluation information of the posture to be evaluated of the target object based on multiple evaluation information includes: Determine the weight corresponding to each evaluation dimension; Based on the multiple weights and multiple evaluation information, the target evaluation information of the posture to be evaluated of the target object is determined.
7. An attitude evaluation device, characterized in that, The device includes: An acquisition unit is used to acquire benchmark videos with different evaluation dimensions; the different evaluation dimensions represent different evaluation criteria for the action. The processing unit is used to acquire the video of the pose to be evaluated of the target object, and divide the video to be evaluated into multiple sub-videos to be evaluated based on the different evaluation dimensions; wherein, the actions of the objects in the benchmark video and the sub-video to be evaluated are the same for the same evaluation dimension. The first determining unit is configured to: for each sub-video to be evaluated, determine a first vector for the part corresponding to the first pose based on the position of the key point of the part corresponding to the first pose in each first image of the sub-video to be evaluated; for each reference video, determine a second vector for the part corresponding to the second pose based on the position of the key point of the part corresponding to the second pose in each second image of the reference video; acquire sample videos for different evaluation dimensions; wherein the actions of the objects in the sample videos and the reference videos corresponding to the same evaluation dimension are the same; for each sample video, determine a sample vector for the part corresponding to the sample pose based on the position of the key point of the part corresponding to the sample pose in each sample image of the sample video; for each evaluation dimension, determine multiple benchmark parameters based on the adjustment parameters of each evaluation dimension, multiple sample vectors, and multiple second vectors; and determine evaluation information for each evaluation dimension based on the multiple first vectors and the multiple benchmark parameters. The second determining unit is used to determine the target evaluation information of the posture to be evaluated of the target object based on multiple evaluation information.
8. An attitude evaluation device, characterized in that, The device includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the attitude evaluation program in the memory to implement the steps of the attitude evaluation method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the attitude evaluation method as described in any one of claims 1 to 6.
10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the attitude evaluation method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Audio and video quality evaluation method and device, readable storage medium and electronic equipment
CN112261437A
Rehabilitation action recognition method and system based on multi-modal information fusion
CN117137435A
