Professional dance evaluation method for realizing human body posture detection based on deep transfer learning

A technology of transfer learning and human body posture, applied in the direction of neural learning methods, instruments, biological neural network models, etc., can solve problems such as inability to achieve accurate scoring, wrong recognition, and missed recognition, and achieve improved recognition accuracy, low cost, and improved efficiency effect

Active Publication Date: 2021-03-26
TONGJI UNIV
6 Cites 7 Cited by

AI-Extracted Technical Summary

Problems solved by technology

However, professional dance postures often contain some exaggerated body movements and a large number of overlapping and occluding movements of the limbs, which may easily lead to missed recognition and wrong recognition; in addition, the model applied...
View more

Method used

The present invention utilizes depth migration learning principle to train human body posture detection model, has improved the efficiency and the accuracy of training; Compared with other real-time human body posture detection ...
View more

Abstract

The invention relates to a professional dance evaluation method for realizing human body posture detection based on deep transfer learning, which comprises the following steps: S1, establishing a special human body posture detection model by utilizing a deep transfer learning principle and combining posture characteristics of professional dance training; s2, collecting a video for demonstrating dancing actions, and inputting the video into the human body posture detection model to obtain a human body key point data flow which changes along with time as a reference standard for evaluation; andS3, obtaining the human body key point information of the dancing action of the testee in the same way, and taking the similarity between the human body key point information and the reference standard as the evaluation of the dancing posture standard degree. According to the invention, the efficiency and accuracy of model training are improved by using deep migration learning.

Application Domain

Character and pattern recognitionNeural architectures +1

Technology Topic

Computer visionOphthalmology +3

Image

  • Professional dance evaluation method for realizing human body posture detection based on deep transfer learning
  • Professional dance evaluation method for realizing human body posture detection based on deep transfer learning

Examples

  • Experimental program(1)

Example Embodiment

[0037] The present invention will be described in detail below with reference to the disclosure of which. The present embodiment is implemented in terms of the technical solution of the present invention, and a detailed embodiment and a specific operation process are given, but the scope of the invention is not limited to the following examples.
[0038] Such as figure 1 Down:
[0039] Professional dance evaluation method for realizing human attitude detection based on deep migration learning, as follows:
[0040] Step S1: Using the principle of depth migration learning, combined with professional dance training attitude detection model;
[0041] Step S11: Using the pre-training convolutional neural network model and source field training set, used to extract the characteristics of the image hierarchy, and realize human joint point identification;
[0042] The data set used in the present invention is a video data set, and the number of video key frames is required to detect, shear, and alignment pre-processing before performing model training. Specifically, the present invention selects the characteristics of each frame using the IMAGENET pre-trained VGG-16, and determines a key frame to obtain a video, and the shear is completed, and the present invention is based on the feature matching to achieve a feature point alignment.
[0043] The present invention selects the RESNET improvement model as the characteristic of the image hierarchy in the video of the video, and realizes the human joint point identification.
[0044]The RESNET-18 model has a total of 4 rolls, each consisting of 4 to 5 convolution layers, and each two convolution layers are connected by a SHORTCUT residual structure, and the model is finally a full connection layer. Where the convolution layer is 3 * 3 in addition to the first convolution layer 7 * 7, the first convolution layer of each block is 2, and the remaining layers are 1, ie The model is reduced by each block of the first convolution layer; the pool layer window size is 3 * 3, and the step size is 2. Improved RESNET-18 model pick frame is: 1 64-channel convolution layer → Pihua layer → 4 64 channels of convolution layer → 4 128 channels of convolution layer → 4 256 channel convolution layers → 4 512 channels of convolutional layer → global pilot layer → output to n-dimensional full connecting layer (n is the number of human attitude detection points).
[0045] Using the source domain training set, fed the improved RESNET-18 convolutional neural network model, the characteristics of the extraction of the image hierarchy were performed and the human key recognition training; at the same time, the posture detection point of the training convolutional neural network model training Pixel point continental distance with standard points as a loss function, for:
[0046] f = || x-g (x) ||
[0047] Where f represents the loss function function, X is the joint I standard attitude point position information, G (x) is the artistic attitude point location information obtained by the pre-training convolutional neural network model. The human body attitude point information is a series of multi-frame multi-dimensional coordinate information formed based on a plurality of individual parts (head, waist, shoulder, elbow, buttock, and ankle, etc.), each of the key point coordinate information is human The central point is the reference origin and the relative pole coordinate representation of the connection relationship of the human body is established. In the head, waist, shoulder, elbow, buttocks, and ankle joints (as an example of elbow joint position), the present invention takes a relative coordinate reference to other joints thereof (such as shoulder joint as an example). The original point, the plane formed by the elbow joint and the shoulder joint as an example of the coordinate surface, calculating the displacement distance, direction and angle of the elbow joint relative to the shoulder joint, indicating that the following:
[0048] x i+1 = X i (ρ, α)
[0049] Where X i Indicates the coordinate point of the joint I, ρ is the relative displacement value of the joint I + 1 relative to the joint I, and the relative angle value of the joint I + 1 relative to the joint I.
[0050] Step S12: According to the requirements of professional dance evaluation, we will perform deep migration and training, design a special humanistic posture detection model;
[0051] A deep migration learning model training will be used in combination with a large number of labeling source field training sets and a small amount of tag or professional assessment of a target domain sample, and design a special human gesture detection network model.
[0052] The process is as follows:
[0053] First, use the RESNET-18 for the initial network model training for a large number of label source training sets to obtain specific weights and offset parameters of the network;
[0054] Second, a small amount of target domain dataset is input into the pre-trained network, and the newly established convolutional neural network model is updated by network weight and offset parameters;
[0055] Finally, model learning training is performed on the obtained label target domain data set.
[0056] Step S2: The video of the demonstration dance action is collected, enters the aforementioned human body attitude detection model, obtain a reference standard for evaluation of human key data streams over time; obtaining a group of two-dimensional body that is changed over time Key coordinate information, the coordinate is a relatively coordinate of reference in the human center point.
[0057] Step S21: Cut the demonstration dance video, align the pretreatment;
[0058] The present invention will show a video key frame number detection, shear, and alignment of the video key frame. Specifically, the present invention selects the characteristics of each frame using the IMAGENET pre-trained VGG-16, and determines a key frame to obtain a video, and the shear is completed, and the present invention is based on the feature matching to achieve a feature point alignment.
[0059] Step S22: Enter the exemplary dance image into the human body attitude detection model to get the human key data.
[0060] The exemplary dance image obtained by the pre-processed is input to the human gesture detection model for testing, and the homogeneous key point is obtained.
[0061] Step S3: The homogeneous information of the dance video action to be evaluated by step S2, the similarity between the reference standard is evaluated as the degree of dance standard.
[0062] The present invention will have a depth migration learning model (i.e., a human posture detection model) and the pixel point container distance of the reference standard point as a similarity function, and calculate the evaluation score of the testist dance action by unified weighted scoring rules. .
[0063] f = || x-g (x) ||
[0064] Where F represents the similarity function, X is the joint I standard attitude point position information, G (x) is the attitude point position information obtained by the joint I by the depth migration learning model.
[0065] The step S3 is specifically:
[0066] Step S31: Waiting for the dance video to be evaluated, aligning pretreatment;
[0067] The present invention will dance with dance sounds as a time axis, which will be compared by the corresponding relationship between the treated dance and the demonstration dance of the exemplary dance. At a certain moment, the music and dance movements should have a fixed correspondence, so the audio data of the video is aligned.
[0068] Step S32: Will evaluate the dance video to enter the human body attitude detection model to get the human key data;
[0069] Step S33: The human body key data to be evaluated to be evaluated to evaluate with the human body attitude detection point data of the demonstration dance is compared by the corresponding relationship, and the degree of similarity is measured and evaluated.
[0070] The present invention will be converted to the human key data to be evaluated and demonstrative dance under the same standard, and each corresponding posture detection point is similar to the pixel point European distance of reference standard points; the total similarity in the entire dance process will be As an evaluation of the dance. Similarity calculations have reasonable degree of fault tolerance rate and can remove data points that have obvious errors.
[0071] During the specific implementation, the dance video to be evaluated by the video device (the content to be evaluated should be exactly the same appearance in the demonstration dance video), and the video outputs the human key information by training a good humanistic posture detection model. Will be evaluated to be evaluated with the key information of the human body key information of the corresponding demonstration dance, and the human key information of the evaluation of the dance is made to make standardization to make it with standard information in the same coordinate system. The audio information of the dance soundtrack is the timeline. After alignment of the key information flow of the two, the key point coordinate deviation is converted to the similar degree. Finally, the total similarity output of the entire dance process was final as an evaluation.
[0072] The present invention uses a depth migration learning principle to train human body posture detection models, improves the efficiency and accuracy of training; compared to other real-time human attitude detection models, the model in the present invention has focused on professional dance assessment evaluation. Increased the identification accuracy of all kinds of dance attitude.
[0073] The present invention has good development prospects, and its applications can expand to other related fields. If the dance evaluation system is developed to the personal side, assisting the dance professional candidates or learners according to the published dance demonstration; can also be applied to the health field, assisting patients with action barriers to rehabilitate or assist sports enthusiasts in sports exercises Adjust your body attitude; in addition, it is also applicable to the learning of human videos in human video and hard-hardware development in other related fields.

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.

Similar technology patents

Method for detecting insulator string breakage defect based on sparse representation

ActiveCN106778734AHigh robustness and practicalityImprove recognition accuracy
Owner:保定市斯德尔电气设备制造有限公司

Classification and recommendation of technical efficacy words

  • Improve recognition accuracy
  • Improve training efficiency

A short-term load prediction method considering somatosensory temperature and radiation intensity

ActiveCN109886567AReduce the dimensionality of the neural networkImprove training efficiency
Owner:NARI TECH CO LTD +4

Publication PDF layout analysis and recognition method based on mixing of multiple neural networks

InactiveCN110866388AImprove training efficiencyMitigate model overfitting
Owner:重庆华龙网海数科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products