A two-dimensional pose estimation method for trampoline motion based on contrastive learning

Abstract

The present invention relates to a two-dimensional posture estimation method for trampoline motion based on contrastive learning, comprising: performing frame slicing processing on a collected trampoline motion video to obtain a trampoline motion dataset, which is divided into a training set and a test set; utilizing a contrastive learning method, through a twin network model, using images of an ImageNet public dataset to perform model training to obtain a pre-trained model; using the trampoline motion dataset, performing transfer learning to obtain a SimplebaseLine model, obtaining human joint coordinate information, calculating the joint angle and the joint similarity OKS. The present invention uses multi-scale feature vectors to predict different views, thereby improving the detection capability of small-scale targets such as ankle joints, and migrating the pre-trained model to a semi-supervised human posture estimation framework ESCP. After migration, not only can the accuracy of joint detection be improved, but the generalization effect of model parameters can also be better, the demand for annotated samples can be reduced, and the problem of large annotation errors in trampoline scenes can be solved, thereby improving detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of two-dimensional human body posture estimation, in particular to a two-dimensional posture estimation method for trampoline motion based on contrast learning. Background Art

[0002] Trampoline exercise consumes a lot of energy, improves blood circulation, and is a great way to improve coordination. Home trampolines are compact and affordable, making them popular among people of all ages. Existing intelligent trampoline systems typically consist of motion data acquisition, processing, and wireless data transmission modules. They capture human motion using devices like infrared sensors or depth cameras, but these devices are expensive and difficult to scale up.

[0003] Deep learning has recently made significant progress. Among these advances, posture estimation technology can detect joints in a moving human body. Using only a standard camera, it can acquire the spatial position of these joints in real time. Compared to infrared sensors or depth cameras, this technology offers advantages such as lower deployment costs and faster detection speeds. Using posture estimation to detect joint positions can also determine the angles and inclinations between limbs, enabling correction of the user's movement posture and preventing injuries caused by improper movement.

[0004] However, obtaining a dataset for trampoline exercise is difficult. Because the human body rises and falls rapidly in mid-air, afterimages appear in many samples, leading to large errors in joint annotation, which can affect the performance of the trained model. Furthermore, when a user bounces, the point where their feet contact the bed significantly sinks, obscuring their ankles, further complicating the acquisition and annotation of the dataset. Summary of the Invention

[0005] In order to solve the problems that trampoline motion dataset samples are difficult to obtain and the sample annotation errors are large, the purpose of the present invention is to provide a trampoline motion two-dimensional posture estimation method based on contrastive learning, which improves the accuracy of joint point detection, reduces the demand for annotated samples, and reduces the trampoline scene annotation errors.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for estimating two-dimensional posture of trampoline motion based on contrastive learning, the method comprising the following steps in sequence:

[0007] (1) Data acquisition and processing: The collected trampoline motion videos are frame-cut to obtain a trampoline motion dataset, which is then divided into a training set and a test set.

[0008] (2) Pre-training: Using the contrastive learning method, the Siamese network model is used to train the model using images from the ImageNet public dataset to obtain a pre-trained model;

[0009] (3) Perform semi-supervised training: Using the trampoline motion dataset, transfer learning is performed by fine-tuning the pre-trained model to obtain the SimplebaseLine model. The SimplebaseLine model is used to obtain the coordinate information of the human joint points, and the joint point angles and the similarity OKS with the joint points are calculated.

[0010] The step (1) specifically includes the following steps:

[0011] (1a) Slice the collected trampoline motion video at a frame rate of 5 frames per second and extract motion pictures frame by frame;

[0012] (1b) removing ghosting, blurring, and occlusion of limbs from the extracted motion pictures to obtain a first data set;

[0013] (1c) selecting images with clear ankle joints from the first data set to obtain a second data set;

[0014] (1d) selecting pictures of standard movements from the second data set, building a standard movement comparison library, and storing standard movement coordinates in the standard movement comparison library;

[0015] (1e) Randomly select 10% of the image samples in the second dataset for joint point annotation to obtain a trampoline motion dataset, and divide the trampoline motion dataset into a training set and a test set.

[0016] The step (2) specifically includes the following steps:

[0017] (2a) Construct a Siamese network model with two branches, one upper and one lower. Randomly select an image X from the ImageNet public dataset, use different data augmentation methods, and obtain different data augmented views X1′ and X2′, which are respectively sent to the upper and lower branches of the Siamese network model for processing. The upper and lower branches share weights.

[0018] (2b) The backbone networks of the upper and lower branches of the twin network model are both composed of ResNet50 networks. The fully connected layer at the end of the ResNet50 network is removed, and the feature vectors f1, f2, f3, and f4 of Layer2, Layer3, Layer4, and Layer5 of the ResNet50 network are forwarded in parallel to the feature pyramid module FPN for feature fusion, and the fused feature vectors are obtained. The dimensions of f1, f2, f3, and f4 are 256, 512, 1024, and 2048 respectively:

[0019] For the feature vector f1, the feature fusion formula is:

[0020]

[0021] For feature vectors f2, f3, f4, the feature fusion formula is:

[0022]

[0023] Among them, Pool represents the maximum pooling operation, Conv1 and Conv2 represent convolution operations with kernel sizes of 1 and 3 respectively, Up represents a 2x upsampling operation, and the feature dimensions of the feature vectors f1, f2, f3, and f4 after being forwarded to the feature pyramid are all 512-dimensional;

[0024] (2c) The fused feature vector Forward to the projection layer for encoding to obtain the encoded feature vector

[0025]

[0026] Where i∈{1,2,3,4}, Proj(·) represents the mapping code, FC-BN-ReLU represents the multi-layer perceptron mapping operation, including fully connected mapping, batch normalization, and activation function, ×2 means that the multi-layer perceptron mapping is repeated twice, the input and output fully connected layers in the mapping layer are 512-dimensional, and the hidden fully connected layer is 512-dimensional;

[0027] (2d) The feature vector F1 of the upper branch is forwarded to the prediction layer for secondary encoding to obtain the encoded feature vector

[0028]

[0029] Where i∈{1,2,3,4}, Pred(·) represents predictive coding, the input and output dimensions of the prediction layer are 512-dimensional, the hidden fully connected layer is 128-dimensional, each multilayer perceptron layer in the projection layer is connected to a batch normalization layer, and only the first multilayer perceptron in the prediction layer is connected to a batch normalization layer;

[0030] The lower branch only requires identity mapping;

[0031] (2e) Use Denote the upper branch eigenvector, and use represents the lower branch eigenvector, and Perform similarity comparison:

[0032] The eigenvector The negative cosine similarity is used as the loss function, and the feature vector of the upper branch is used To predict the feature vector of the next branch The backbone network is enabled to learn the ability to extract multi-scale features. The calculation formula of normalized negative cosine similarity is:

[0033]

[0034] Among them, the eigenvector The dimension is 512, ||||2 is the L2 norm equivalent to the mean square error of the L2 normalized vector, Represents the feature vector The normalized negative cosine similarity of , whose minimum value is -1, indicates the similarity of the feature vectors;

[0035] According to the symmetric structure of the twin network, the order in which views X1′ and X2′ are sent to the upper and lower branches is swapped, and the formula for the normalized negative cosine similarity generated after re-encoding is:

[0036]

[0037] Using two negative cosine similarities The sum of is used as the loss function The following formula expresses it:

[0038]

[0039] Among them, stopgrad(·) means stopping the gradient operation, Is a constant and does not participate in the process of network back propagation gradient;

[0040] (2f) is the loss function Set weights, total loss function The calculation formula is as follows:

[0041]

[0042] Among them, i represents the i-th level feature extracted by the backbone network, is the loss value of the i-th pair of features, λ i for The balance weight of

[0043] (2g) Pre-training is performed. After the pre-training is completed, the weights of the backbone network in the twin network model are saved to obtain the pre-training model.

[0044] The step (3) specifically includes the following steps:

[0045] (3a) The pre-trained model is transferred to the semi-supervised human pose estimation framework ESCP by fine-tuning, and the SimpleBaseline method is used to estimate the heat map to obtain the SimpleBaseline model;

[0046] (3b) During transfer learning, a total of about 36K iterations were trained on the trampoline dataset using the Adam optimizer with an initial learning rate of 1e-3. The learning rate was reduced to 1e-4 and 1e-5 at 24k and 30k iterations, respectively.

[0047] (3c) After training, the weight parameters of the SimpleBaseline model are extracted, and the performance of the SimpleBaseline model is verified on the test set of the trampoline dataset;

[0048] (3d) Based on the human joint point results obtained by the SimpleBaseline model, key joints and bones are selected and the angles of the joint points are calculated:

[0049] Select the spatial angles of the knee and elbow joints, and the joint points E, F, and G are the hip, knee, and ankle joints, respectively;

[0050] Let the coordinates of the hip joint E be (x e ,y e ), the coordinates of the knee joint F are (x f ,y f ), the coordinate of the knee joint G is (x g ,y g ), bone vector FE=(x f -x e ,y f -y e ), FG=(x f -x g ,y f -y g );

[0051] Substitute the human joint coordinate information output by the SimpleBaseline model into the spatial angle formula: Calculate the degree of the spatial angle ∠EFG of the knee joint F;

[0052] (3e) Based on the human joint coordinate information obtained by the SimpleBaseline model and the standard action coordinates, the joint similarity OKS is calculated as an indicator to measure the standardization of the action:

[0053] Assume (x i ,y i ) represents the current joint point detection result, (x i ′,y i ′) represents the joint point detection result of the standard action, and d i The Euclidean distance between the joint point with sequence number i in the currently detected set of joint points and the joint point with sequence number i in the standard action is calculated as follows:

[0054]

[0055] v i =1 means the visibility of this joint point is 1, that is, the joint point is not blocked and marked on the standard action, v i =2 means that the joint point is marked even if it is blocked;

[0056] S represents the scale factor of the current human body, and its value is the square root of the area of ​​the pedestrian detection box: w and h are the width and height of the detection box respectively;

[0057] σ i The normalization factor of the joint point with the serial number i is represented. The calculation formula of the joint point similarity OKS is as follows:

[0058]

[0059] It can be seen from the above technical solution that the beneficial effects of the present invention are: First, multi-scale feature vectors are used to predict different views, which improves the detection ability of small-scale targets such as ankle joints, and migrates the pre-trained model to the semi-supervised human posture estimation framework ESCP. After migration, not only the accuracy of joint detection can be improved, but also the generalization effect of model parameters can be better, the demand for annotation samples is reduced, and the problem of large annotation errors in trampoline scenes is solved. After testing, compared with the existing methods, the detection accuracy is improved; second, a multi-scale comparative learning method is designed, and multi-scale features are introduced on the basis of the twin network model to better focus on fine-grained information, which is conducive to inferring joints that are more difficult to identify; third, a feature pyramid module is added before the projection layer to avoid the problem of network performance degradation due to the small receptive field of shallow features and the easy focus on redundant background information; fourth, the comparative learning method is innovatively applied in the field of posture estimation, which effectively alleviates the problems of long time and large errors in annotating joints. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of the method of the present invention;

[0061] Figure 2 This is a structural diagram of the twin network model in the present invention;

[0062] Figure 3 It is a structural diagram of the feature pyramid in the present invention;

[0063] Figure 4 It is a structural diagram of the projection layer in the present invention;

[0064] Figure 5 This is a structural diagram of the prediction layer in the present invention;

[0065] Figure 6A visualization of human joint angle calculations. DETAILED DESCRIPTION

[0066] like Figure 1 As shown, a method for estimating two-dimensional posture of trampoline motion based on contrastive learning comprises the following steps in sequence:

[0067] (1) Data acquisition and processing: The collected trampoline motion videos are frame-cut to obtain a trampoline motion dataset, which is then divided into a training set and a test set.

[0068] (2) Pre-training: Using the contrastive learning method, the Siamese network model is used to train the model using images from the ImageNet public dataset to obtain a pre-trained model;

[0069] (3) Perform semi-supervised training: Using the trampoline motion dataset, transfer learning is performed by fine-tuning the pre-trained model to obtain the SimplebaseLine model. The SimplebaseLine model is used to obtain the coordinate information of the human joint points, and the joint point angles and the similarity OKS with the joint points are calculated.

[0070] The step (1) specifically includes the following steps:

[0071] (1a) Slice the collected trampoline motion video at a frame rate of 5 frames per second and extract motion pictures frame by frame;

[0072] (1b) removing ghosting, blurring, and occlusion of limbs from the extracted motion pictures to obtain a first data set;

[0073] (1c) selecting images with clear ankle joints from the first data set to obtain a second data set;

[0074] (1d) selecting pictures of standard movements from the second data set, building a standard movement comparison library, and storing standard movement coordinates in the standard movement comparison library;

[0075] (1e) Randomly select 10% of the image samples in the second dataset for joint point annotation to obtain a trampoline motion dataset, and divide the trampoline motion dataset into a training set and a test set.

[0076] By processing collected trampoline videos and applying contrastive learning to semi-supervised human pose estimation, we significantly reduce the need for annotated samples and alleviate the difficulty of manually annotating samples in trampoline scenarios. In this method, only 10% of the images in the dataset are selected for annotation as labeled samples, while the remaining images are used as unlabeled samples.

[0077] like Figure 2As shown, the step (2) specifically includes the following steps:

[0078] (2a) Construct a Siamese network model with two branches, one upper and one lower. Randomly select an image X from the ImageNet public dataset, use different data augmentation methods, and obtain different data augmented views X1′ and X2′, which are respectively sent to the upper and lower branches of the Siamese network model for processing. The upper and lower branches share weights.

[0079] The pre-training dataset for contrastive learning in this paper is constructed using images from the public ImageNet dataset. The ImageNet dataset is currently the most popular dataset for self-supervised pre-training tasks. It contains 1.28 million images across 1,000 classes, with a well-balanced class distribution. Images often include iconic views of objects. During pre-training, data augmentation consists of random 224×224 pixel resizing and cropping, random color jittering, random grayscale conversion, Gaussian blurring, and random horizontal flipping.

[0080] (2b) Figure 3 As shown in the figure, the backbone networks of the upper and lower branches of the twin network model are composed of ResNet50 networks. The fully connected layer at the end of the ResNet50 network is removed, and the feature vectors f1, f2, f3, and f4 of Layer2, Layer3, Layer4, and Layer5 of the ResNet50 network are forwarded in parallel to the feature pyramid module FPN for feature fusion to obtain the fused feature vector The dimensions of f1, f2, f3, and f4 are 256, 512, 1024, and 2048 respectively:

[0081] For the feature vector f1, the feature fusion formula is:

[0082]

[0083] For feature vectors f2, f3, f4, the feature fusion formula is:

[0084]

[0085] Among them, Pool represents the maximum pooling operation, Conv1 and Conv2 represent convolution operations with kernel sizes of 1 and 3 respectively, Up represents a 2x upsampling operation, and the feature dimensions of the feature vectors f1, f2, f3, and f4 after being forwarded to the feature pyramid are all 512-dimensional;

[0086] (2c) Figure 4 As shown, the fused feature vector Forward to the projection layer for encoding to obtain the encoded feature vector

[0087]

[0088] Where i∈{1,2,3,4}, Proj(·) represents the mapping code, FC-BN-ReLU represents the multi-layer perceptron mapping operation, including fully connected mapping, batch normalization, and activation function, ×2 means that the multi-layer perceptron mapping is repeated twice, the input and output fully connected layers in the mapping layer are 512-dimensional, and the hidden fully connected layer is 512-dimensional;

[0089] (2d) Figure 5 As shown, the eigenvector of the upper branch is Forwarded to the prediction layer for secondary encoding to obtain the encoded feature vector

[0090]

[0091] Where i∈{1,2,3,4}, Pred(·) represents predictive coding, the input and output dimensions of the prediction layer are 512-dimensional, the hidden fully connected layer is 128-dimensional, each multilayer perceptron layer in the projection layer is connected to a batch normalization layer, and only the first multilayer perceptron in the prediction layer is connected to a batch normalization layer;

[0092] The lower branch only requires identity mapping;

[0093] (2e) Use Denote the upper branch eigenvector, and use represents the lower branch eigenvector, and Perform similarity comparison:

[0094] The eigenvector The negative cosine similarity is used as the loss function, and the feature vector of the upper branch is used To predict the feature vector of the next branch The backbone network is enabled to learn the ability to extract multi-scale features. The calculation formula of normalized negative cosine similarity is:

[0095]

[0096] Among them, the eigenvector The dimension is 512, ||||2 is the L2 norm equivalent to the mean square error of the L2 normalized vector, Represents the feature vector The normalized negative cosine similarity of , whose minimum value is -1, indicates the similarity of the feature vectors;

[0097] According to the symmetric structure of the twin network, the order in which views X1′ and X2′ are sent to the upper and lower branches is swapped, and the formula for the normalized negative cosine similarity generated after re-encoding is:

[0098]

[0099] Using two negative cosine similarities The sum of is used as the loss function The following formula expresses it:

[0100]

[0101] Among them, stopgrad(·) means stopping the gradient operation, Is a constant and does not participate in the process of network back propagation gradient;

[0102] (2f) is the loss function Set weights, total loss function The calculation formula is as follows:

[0103]

[0104] Among them, i represents the i-th level feature extracted by the backbone network, is the loss value of the i-th pair of features, λ i for The balance weight of

[0105] Model pre-training was performed on Linux using Python 3.8, CUDA 11.3, and PyTorch 1.11, using MMSelfSup 0.9 as the underlying framework. The model was trained on four Nvidia GeForce RTX 3090 GPUs, with a batch size of 64 per GPU and a total batch size of 256. Training was performed for 100 epochs, or 1 million iterations, on the public ImageNet dataset, with a full training period of three days. SGD was used as the optimizer, with weight decay and momentum set to 0.0001 and 0.9, respectively. The initial learning rate was set to 0.05 with cosine learning rate decay.

[0106] (2g) Pre-training is performed. After the pre-training is completed, the weights of the backbone network in the twin network model are saved to obtain the pre-training model.

[0107] This paper designs a method for pairwise contrastive learning of multi-scale features. It extracts multi-scale features from different views of the same image, fuses them into a feature pyramid, and predicts feature vectors for the corresponding views. Using the negative cosine similarity between feature vectors as a loss function, the backbone network is trained to acquire visual representations. This method achieves superior performance with a small number of annotated samples, effectively alleviating the time-consuming and inconsistent standards associated with manual annotation of joint points.

[0108] The present invention selects multi-scale feature vectors for comparison. This is because the expressive power of features at different levels in the feature map is different. Shallow features mainly reflect details such as light and dark, edges, etc., while deep features reflect a richer overall structure. Using shallow features alone cannot contain global structural information and may weaken the expressive power of features. The deep features themselves are further constructed by shallow features, so they naturally contain information about shallow features. The feature pyramid is to upsample the shallow features to matching dimensions, and then fuse the shallow features into the deep features, taking into account both details and the overall picture. The fused features will have richer expressive power and help the network learn more spatial context information.

[0109] The step (3) specifically includes the following steps:

[0110] (3a) The pre-trained model is transferred to the semi-supervised human pose estimation framework ESCP by fine-tuning, and the SimpleBaseline method is used to estimate the heat map to obtain the SimpleBaseline model;

[0111] (3b) During transfer learning, a total of about 36K iterations were trained on the trampoline dataset using the Adam optimizer with an initial learning rate of 1e-3. The learning rate was reduced to 1e-4 and 1e-5 at 24k and 30k iterations, respectively.

[0112] (3c) After training, the weight parameters of the SimpleBaseline model are extracted, and the performance of the SimpleBaseline model is verified on the test set of the trampoline dataset;

[0113] (3d) Based on the human joint point results obtained by the SimpleBaseline model, key joints and bones are selected and the angles of the joint points are calculated:

[0114] like Figure 6 As shown, the spatial angles of the knee joint and elbow joint are taken as an example, and the joint points E, F, and G are hip joint, knee joint, and ankle joint respectively;

[0115] Let the coordinates of the hip joint E be (x e ,y e ), the coordinates of the knee joint F are (x f ,y f ), the coordinate of the knee joint G is (x g ,y g ), bone vector FE=(x f -x e ,y f -y e ), FG=(x f -xg ,y f -y g );

[0116] Substitute the human joint coordinate information output by the SimpleBaseline model into the spatial angle formula: Calculate the degree of the spatial angle ∠EFG of the knee joint F;

[0117] (3e) Based on the human joint coordinate information obtained by the SimpleBaseline model and the standard action coordinates, the joint similarity OKS is calculated as an indicator to measure the standardization of the action:

[0118] Assume (x i ,y i ) represents the current joint point detection result, (x i ′,y i ′) represents the joint point detection result of the standard action, and d i The Euclidean distance between the joint point with sequence number i in the currently detected set of joint points and the joint point with sequence number i in the standard action is calculated as follows:

[0119]

[0120] v i =1 means the visibility of this joint point is 1, that is, the joint point is not blocked and marked on the standard action, v i =2 means that the joint point is marked even if it is blocked;

[0121] S represents the scale factor of the current human body, and its value is the square root of the area of ​​the pedestrian detection box: w and h are the width and height of the detection box respectively;

[0122] σ i The normalization factor of the joint point with the serial number i is represented. The calculation formula of the joint point similarity OKS is as follows:

[0123]

[0124] Through the above results, we can obtain the specific degrees of the trampoline athlete's knee, elbow and other key joint angles, and obtain the similarity score between the user's current posture and the standard movement, which helps to standardize sports movements and better use the trampoline for exercise.

[0125] After pre-training, the parameter weights of the pre-trained model are used as the initial weights for semi-supervised training. Pre-trained models generated using contrastive learning can significantly improve the model's generalization capabilities. After training on a large dataset, the model can be well adapted to tasks with less sample data, such as trampoline.

[0126] Semi-supervised pose estimation, on the other hand, requires only a small number of annotated samples to train a highly accurate SimpleBaseline model. This paper uses the ESCP semi-supervised framework to more accurately predict the pose information of difficult images by setting difficult-easy sample pairs. This prevents highly responsive samples from being classified as background and prevents network crashes. This semi-supervised approach effectively solves a series of problems associated with trampoline motion sample annotation.

[0127] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A two-dimensional posture estimation method for trampoline motion based on contrastive learning, characterized by: The method comprises the following steps in sequence: (1) Data acquisition and processing: The collected trampoline motion videos are frame-cut to obtain a trampoline motion dataset, which is then divided into a training set and a test set. (2) Pre-training: Using the contrastive learning method, the Siamese network model is used to train the model using images from the ImageNet public dataset to obtain a pre-trained model; (3) Perform semi-supervised training: Using the trampoline motion dataset, transfer learning is performed by fine-tuning the pre-trained model to obtain the SimplebaseLine model. The SimplebaseLine model is used to obtain the coordinate information of the human joints, and the joint angles and the similarity OKS with the joints are calculated; The step (2) specifically includes the following steps: (2a) Construct a Siamese network model with two branches, one upper and one lower. Randomly select an image X from the ImageNet public dataset, use different data augmentation methods, and obtain different data augmented views X1′ and X2′, which are respectively sent to the upper and lower branches of the Siamese network model for processing. The upper and lower branches share weights. (2b) The backbone networks of the upper and lower branches of the twin network model are both composed of ResNet50 networks. The fully connected layer at the end of the ResNet50 network is removed, and the feature vectors f1, f2, f3, and f4 of Layer2, Layer3, Layer4, and Layer5 of the ResNet50 network are forwarded in parallel to the feature pyramid module FPN for feature fusion, and the fused feature vectors are obtained. The dimensions of f1, f2, f3, and f4 are 256, 512, 1024, and 2048 respectively: For the feature vector f1, the feature fusion formula is: For feature vectors f2, f3, f4, the feature fusion formula is: Among them, Pool represents the maximum pooling operation, Conv1 and Conv2 represent convolution operations with kernel sizes of 1 and 3 respectively, Up represents a 2x upsampling operation, and the feature dimensions of the feature vectors f1, f2, f3, and f4 after being forwarded to the feature pyramid are all 512-dimensional; (2c) The fused feature vector Forward to the projection layer for encoding to obtain the encoded feature vector Where i∈{1,2,3,4}, Proj(·) represents the mapping code, FC-BN-ReLU represents the multi-layer perceptron mapping operation, including fully connected mapping, batch normalization, and activation function, ×2 means that the multi-layer perceptron mapping is repeated twice, the input and output fully connected layers in the mapping layer are 512-dimensional, and the hidden fully connected layer is 512-dimensional; (2d) The eigenvector of the upper branch Forwarded to the prediction layer for secondary encoding to obtain the encoded feature vector Where i∈{1,2,3,4}, Pred(·) represents predictive coding, the input and output dimensions of the prediction layer are 512-dimensional, the hidden fully connected layer is 128-dimensional, each multilayer perceptron layer in the projection layer is connected to a batch normalization layer, and only the first multilayer perceptron in the prediction layer is connected to a batch normalization layer; The lower branch only requires identity mapping; (2e) Use Denote the upper branch eigenvector, and use represents the lower branch eigenvector, and Perform similarity comparison: The eigenvector The negative cosine similarity is used as the loss function, and the feature vector of the upper branch is used To predict the feature vector of the next branch The backbone network is enabled to learn the ability to extract multi-scale features. The calculation formula of normalized negative cosine similarity is: Among them, the eigenvector The dimension is 512, ||||2 is the L2 norm equivalent to the mean square error of the L2 normalized vector, Represents the feature vector The normalized negative cosine similarity of , whose minimum value is -1, indicates the similarity of the feature vectors; According to the symmetric structure of the twin network, the order in which views X1′ and X2′ are sent to the upper and lower branches is swapped, and the formula for the normalized negative cosine similarity generated after re-encoding is: Using two negative cosine similarities The sum of is used as the loss function The following formula expresses it: Among them, stopgrad(·) means stopping the gradient operation, Is a constant and does not participate in the process of network back propagation gradient; (2f) is the loss function Set weights, total loss function The calculation formula is as follows: Among them, i represents the i-th level feature extracted by the backbone network, is the loss value of the i-th pair of features, λ i for The balance weight of (2g) Pre-training is performed. After the pre-training is completed, the weights of the backbone network in the twin network model are saved to obtain the pre-training model.

2. The method for two-dimensional posture estimation of trampoline motion based on contrastive learning according to claim 1, characterized in that: The step (1) specifically includes the following steps: (1a) Slice the collected trampoline motion video at a frame rate of 5 frames per second and extract motion pictures frame by frame; (1b) removing ghosting, blurring, and occlusion of limbs from the extracted motion pictures to obtain a first data set; (1c) selecting images with clear ankle joints from the first data set to obtain a second data set; (1d) selecting pictures of standard movements from the second data set, building a standard movement comparison library, and storing standard movement coordinates in the standard movement comparison library; (1e) Randomly select 10% of the image samples in the second dataset for joint point annotation to obtain a trampoline motion dataset, and divide the trampoline motion dataset into a training set and a test set.

3. The method for two-dimensional posture estimation of trampoline motion based on contrastive learning according to claim 1, characterized in that: The step (3) specifically includes the following steps: (3a) The pre-trained model is transferred to the semi-supervised human pose estimation framework ESCP by fine-tuning, and the SimpleBaseline method is used to estimate the heat map to obtain the SimpleBaseline model; (3b) During transfer learning, a total of about 36K iterations were trained on the trampoline dataset using the Adam optimizer with an initial learning rate of 1e-3. The learning rate was reduced to 1e-4 and 1e-5 at 24k and 30k iterations, respectively. (3c) After training, the weight parameters of the SimpleBaseline model are extracted, and the performance of the SimpleBaseline model is verified on the test set of the trampoline dataset; (3d) Based on the human joint point results obtained by the SimpleBaseline model, key joints and bones are selected and the angles of the joint points are calculated: Select the spatial angles of the knee and elbow joints, and the joint points E, F, and G are the hip, knee, and ankle joints, respectively; Let the coordinates of the hip joint E be (x e ,y e ), the coordinates of the knee joint F are (x f ,y f ), the coordinate of the knee joint G is (x g ,y g ), bone vector FE=(x f -x e ,y f -y e ), FG=(x f -x g ,y f -y g ); Substitute the human joint coordinate information output by the SimpleBaseline model into the spatial angle formula: Calculate the degree of the spatial angle ∠EFG of the knee joint F; (3e) Based on the human joint coordinate information obtained by the SimpleBaseline model and the standard action coordinates, the joint similarity OKS is calculated as an indicator to measure the standardization of the action: Assume (x i ,y i ) represents the current joint point detection result, (x i ′,y i ′) represents the joint point detection result of the standard action, and d i The Euclidean distance between the joint point with sequence number i in the currently detected set of joint points and the joint point with sequence number i in the standard action is calculated as follows: v i =1 means the visibility of this joint point is 1, that is, the joint point is not blocked and marked on the standard action, v i =2 means that the joint point is marked even if it is blocked; S represents the scale factor of the current human body, and its value is the square root of the area of ​​the pedestrian detection box: w and h are the width and height of the detection box respectively; σ i The normalization factor of the joint point with the serial number i is represented. The calculation formula of the joint point similarity OKS is as follows:

Images