A skeleton graph structure constraint-based cross-domain pose estimation method and application

Abstract

The application provides a cross-domain pose estimation method based on skeleton graph structure constraint and application, constructs skeleton graph positive and negative contrast samples based on source domain data set labels; constructs a graph contrast learning network, trains the constructed skeleton contrast sample set; constructs a pose estimation model based on heat map regression, and trains based on the source domain data set; obtains the pseudo label of the target domain data, uses the pose estimation model to infer in the target domain data, and uses the graph contrast model to distinguish the correct predicted pseudo label; constructs a domain adaptation training model, and trains using the source domain data and the target domain data with the pseudo label. The application proposes to use the discriminability of key point skeletons in the graph feature space to obtain high-precision pseudo labels, and constructs a domain migration training model that can learn target features specifically, aiming at the cross-domain pose estimation problem. The application effectively improves the prediction accuracy of cross-domain pose estimation and eliminates the influence of domain differences between data sets on model performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and computer vision, and particularly relates to a cross-domain pose estimation method based on skeleton graph constraint and application. BACKGROUND

[0002] Pose estimation is an important direction in the field of artificial intelligence and computer vision, and plays a crucial role in automatic driving, brain-computer interface and animal protection.

[0003] The rapid development of deep learning this year has made significant progress in the task of pose estimation, but the current mainstream pose estimation research needs a large amount of labeled data. In animal experiments and other practical applications, most of the scenarios we face are unlabeled information. How to make the trained pose estimation model also perform well in these scenarios is an important but limited research problem.

[0004] Unsupervised domain adaptation is the main method to solve the problem of domain difference between data sets, which aims to transfer the model trained on the source domain data set to the unlabeled target domain data. The mainstream domain adaptation related research is for classification task, which is mainly divided into three categories according to the learning method: based on adversarial training, based on difference and based on reconstruction.

[0005] Compared with the classification task, the unsupervised domain adaptation research in pose estimation is limited, and the mainstream model is based on pseudo label generation. These methods are based on the consistency of the prediction results to filter the pseudo labels of the target domain data, and then fine-tune the model or other types of transfer training based on the obtained pseudo labels, so as to reduce the influence of domain difference between data sets on the performance of the model.

[0006] The current pose estimation model based on unsupervised domain adaptation basically adopts the key point position consistency criterion when obtaining the pseudo label of the target domain data. Due to the sparsity and feature deficiency of the key points in the Euclidean space, the pseudo label obtained by using the criterion has the characteristics of low precision and high noise, which has a great influence on the transfer training of the model. SUMMARY

[0007] In view of the above problems existing in the existing cross-domain pose estimation model, the application provides a cross-domain pose estimation method based on skeleton graph structure constraint.

[0008] The cross-domain pose estimation method based on skeleton graph structure constraint provided by the application comprises the following steps:

[0009] S1: Constructing skeleton graph positive and negative contrast samples based on source domain data set labels;

[0010] S2: Construct a graph contrast learning network, and train the constructed skeleton contrast sample set;

[0011] S3: Construct a pose estimation model based on a heat map regression, and train the pose estimation model based on a source domain data set;

[0012] S4: Obtain pseudo labels of target domain data, including inferring the pose estimation model trained on the source domain data on the target domain data, and using the graph contrast model to determine the pseudo labels of correct prediction;

[0013] S5: Construct a domain adaptation training model, and train the model using the source domain data and the target domain data.

[0014] The step S1 specifically comprises:

[0015] S11: Generating a rotation augmented sample for the key point label of each source domain sample;

[0016] S12: Randomly selecting a number from the set [0, 1] as a generated sample pair label;

[0017] S13: According to the label type of the sample pair, adding a corresponding noise transformation type to the key point label of the source domain sample and the augmented sample to construct a key point simulation sample pair;

[0018] S14: Generating a node feature matrix M of the skeleton graph structure in the sample pair f ;

[0019] S15: Returning a sample set containing the key point simulation sample pair, the edge connection matrix of the skeleton graph structure, and the sample pair label.

[0020] The step S13 specifically comprises: if the randomly selected label is 1, the noise added to the key point label of the source domain sample and the augmented sample is all positive sample noise N pos ; if the randomly selected label is 0, at least one of the noise added to the key point label of the source domain sample and the augmented sample is negative sample noise N neg .

[0021] Wherein, the positive sample noise and the negative sample noise are defined according to the percentage of the noise range to the length of the target body, specifically: the sampling range of the positive sample label N pos is N pos ∈{n|0<n<0.2*l}, and the sampling range of the negative sample label N neg is N neg ∈{n|0.2*l<n<l}, wherein l is the length of the target trunk.

[0022] The step S14 specifically comprises generating the node feature matrix M of the skeleton graph structure in the sample pair f Specifically comprising: calculating each key point P i The included angle A between the adjacent two key points i Then, the node feature matrix M is composed of the Euclidean space coordinates of the key points according to the following formula f :

[0023]

[0024] The step S2 specifically comprises:

[0025] S21: constructing a contrast training model based on a graph neural network, wherein the graph contrast training model is composed of three GINconv graph convolution layers, a full connection layer and a Sigmoid activation layer;

[0026] S22: the input of the graph neural network model is the node feature matrix M of the positive and negative sample pairs of the skeleton graph f The concatenation matrix in the x dimension;

[0027] S23: training the graph contrast model by using the input constructed by the positive and negative sample pairs of the skeleton graph and the sample pair labels.

[0028] The step S3 specifically comprises:

[0029] S31: generating a heat map by using the key point coordinate labels of the source domain dataset as the training label of the pose estimation model;

[0030] S32: constructing a pose estimation model based on key point heat map regression;

[0031] S33: training the pose estimation model by using the training images from the dataset and the corresponding heat map label set, and the loss function used is the mean square error loss.

[0032] The step S32 of constructing a pose estimation model based on key point heat map regression specifically comprises: a feature extraction module, a feature fusion module based on a transformer structure and a feature decoding module.

[0033] The feature extraction module is a pre-trained Resnet50, the input of which is a three-dimensional image with a resolution of 3×w×h, and the output is a tensor feature with a size of .

[0034] The feature fusion module based on the transformer architecture consists of three parallel attention mechanism modules, including two self-attention modules and one cross-attention module. The input to the two self-attention modules is image features from the feature extraction module, while the query vector, key vector, and query vector of the cross-attention module are derived from the two self-attention modules, respectively. The concatenated features of the two attention modules along the channel dimension constitute the output of the transformer-based feature fusion module.

[0035] The feature decoding module consists of three upsampling modules and a convolutional output layer with a kernel size of 1×1.

[0036] Furthermore, the upsampling module consists of an upsampling layer with a stride of 2, a two-dimensional convolutional layer, a batch regularization layer, and a ReLU activation layer in sequence.

[0037] Step S4 specifically includes:

[0038] S41: Obtain augmented samples of the image in the target domain through rotation transformation;

[0039] S42: The target domain image and its augmented sample are inferred through the pose estimation model trained on the source domain to obtain the predicted key point heatmap, and the key point coordinates are obtained through non-maximum suppression.

[0040] S43: Based on step S14, the key point prediction structure of the target domain image and its augmented samples is processed to obtain its skeleton map sample pair.

[0041] S44: Use the graph comparison model trained in step S2 to predict the skeleton graph samples of the target domain data. Reasoning is performed to obtain the discrimination result S. T ;

[0042] S45: Based on the discrimination result S T The value divides the target domain dataset into pseudo-labeled subsets. and subsets without pseudo-labels

[0043] S45 describes the judgment result S T The criterion for dividing the target domain dataset into two subsets is: for samples from the target domain dataset have

[0044]

[0045] Step S5 specifically includes:

[0046] S51: constructing a domain adaptation training model;

[0047] S52: training by using source domain data and target domain data;

[0048] S53: inference of the pose estimation model after domain adaptation training.

[0049] The domain adaptation training model comprises a double-flow feature extraction module, a target feature alignment module, and a double-flow feature decoding module.

[0050] Further, the double-flow feature extraction module is composed of two Resnet50 networks sharing weights, and the pre-training weights are inherited from the feature extraction module of the pose estimation model trained based on the source domain data in step S3.

[0051] The target feature alignment module is of a transformer structure, and the structure and weights are from the feature fusion module of the pose estimation model trained based on the source domain data in step S3. The two self-attention modules in the module input two output features from the double-flow feature extraction module.

[0052] The double-flow feature decoding module is composed of two groups of feature decoding modules with the same structure, and the structure and weights of each group of feature decoding modules are from the feature decoding module of the pose estimation model trained based on the source domain data in step S3.

[0053] The training by using source domain data and target domain data specifically comprises: randomly selecting one image from the source domain data set and the target domain data set with pseudo labels as a sample pair; then generating two groups of heat maps as training targets according to the key point labels; the two images in the sample pair are respectively used as the two inputs of the double-flow feature extraction module; and the two groups of heat map labels are respectively used to calculate the mean square error loss with the inputs from the double-flow feature decoder.

[0054] The inference of the pose estimation model after domain adaptation training specifically comprises: assigning weights to each module of the pose estimation model and target domain model inference.

[0055] Further, the weight assignment to each module of the pose estimation model is as follows: the pose estimation model structure is the same as the pose estimation model constructed in step S3; the weights of the feature extraction module, the feature fusion module, and the feature decoding module are respectively inherited from the target domain data branch in the double-flow feature extraction module, the target feature alignment module, and the target domain data branch in the double-flow feature decoding module of the model trained in step S52.

[0056] ​The target domain model inference specifically comprises: for each test sample in a target domain data set, firstly, a size sampling transformation is performed on the test sample to 3xwxh, then a pose estimation model obtained in step S32 is used for inference to obtain a predicted key point heat map set, and finally, the heat map set is subjected to a non-maximum suppression operation to obtain a final predicted key point set.

[0057] Another object of the present application is to provide application of the cross-domain pose estimation method in model migration training, especially in cross-scene and cross-species animal pose estimation tasks, to achieve application in model migration training independent of data labeling.

[0058] The present application has the following advantages:

[0059] 1. The present application proposes a cross-domain pose estimation method based on graph structure constraint to solve the problem of model performance decline caused by domain difference between data sets in the pose estimation task. The domain adaptation training proposed by the present application does not require labeling of target domain data, and belongs to an unsupervised domain adaptation method. In the case of no additional labeling cost, the pose estimation performance of the model on unlabeled data is effectively improved.

[0060] 2. The present application proposes a pseudo-label generation method based on skeleton graph structure consistency to solve the problem of high noise and low precision of the pseudo-label obtained in the unsupervised domain adaptation based pose estimation task. The method compares and learns the simulated prediction results of the key points in the graph feature space of the key point connection, so as to improve the discrimination ability of the model for the key point prediction results, and obtain high-precision target domain data pseudo-labels.

[0061] 3. The present application proposes a target feature alignment module based on the Transformer mechanism to solve the problem of inaccurate model response to target features caused by domain difference between data in domain migration training. The module implicitly enables the model to learn to focus on and align the target-related features in the source domain and the target domain through two parallel self-attention modules and a cross-attention module, reduces the influence of irrelevant features such as background on the model, and effectively improves the cross-domain migration ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 The figure is a whole flowchart of the present application.

[0063] Figure 2 The figure is a graph contrast learning module schematic diagram of the present application.

[0064] Figure 3 The figure is a target feature alignment module schematic diagram based on the Transformer of the present application.

[0065] Figure 4 The figure is a model domain migration training schematic diagram of the present application. DETAILED DESCRIPTION

[0066] The application will be described in detail below with reference to the accompanying drawings and embodiments.

[0067] Embodiment 1

[0068] As shown in the figure, the application provides a cross-domain pose estimation method based on graph structure constraint, which includes the following steps: Figure 1

[0069] Step S1: Constructing skeleton graph positive-negative contrast samples based on source domain dataset labels;

[0070] Step S2: As shown in the figure, constructing a graph contrast learning network, and training with the constructed skeleton contrast sample set; Figure 2

[0071] Step S3: Constructing a pose estimation model based on heat map regression, and training based on the source domain dataset;

[0072] Step S4: Obtaining pseudo labels of target domain data, including using the pose estimation model trained on the source domain data to infer on the target domain data, and using the graph contrast model to determine the correct predicted pseudo labels;

[0073] Step S5: Constructing a domain adaptation training model, and training using the source domain data and the target domain data.

[0074] Further, the step S1 specifically includes:

[0075] Step S11: Generating rotation augmented samples for the key point labels of each source domain sample;

[0076] Step S12: Randomly selecting a number from the set [0, 1] as the label of the generated sample pair;

[0077] Step S13: According to the label type of the sample pair, adding corresponding noise transformation types to the key point labels of the source domain sample and the augmented sample, and constructing a key point simulation sample pair;

[0078] Step S14: Generating the node feature matrix M f of the skeleton graph structure in the sample pair;

[0079] Step S15: Returning a sample set containing the key point simulation sample pair, the edge connection matrix of the skeleton graph structure, and the sample pair label.

[0080] Preferably, the construction of the key point simulation sample pair specifically includes: if the randomly selected label is 1, the noise added to the key point labels of the source domain sample and the augmented sample is all positive sample noise N pos ​​If the randomly selected label is 0, then at least one of the key point labels of the source domain sample and the noise added by the augmented sample is negative sample noise N neg .

[0081] Preferably, the positive and negative sample noise is defined according to the percentage of the noise range to the target body length, specifically: the sampling range of the positive sample label N pos is: N pos ∈{n|0<n<0.2*l}, and the sampling range of the negative sample label N neg is: N neg ∈{n|0.2*l<n<l}, where l is the length of the target torso.

[0082] Preferably, the node feature matrix M f of the skeleton graph structure in the generated sample pair specifically includes: calculating the included angle A i between each key point P i and the adjacent two key points, and then jointly forming the node feature matrix M f with the key point Euclidean space coordinates according to the following formula:

[0083]

[0084] Further, the step S2 specifically includes:

[0085] Step S21: constructing a graph neural network-based contrast training model, wherein the graph contrast training model is composed of three GINconv graph convolution layers, a fully connected layer, and a Sigmoid activation layer;

[0086] Step S22: the input of the graph neural network model is the node feature matrix M f of the skeleton graph positive and negative samples in the x-dimension splicing matrix;

[0087] Step S23: training the graph contrast model using the input constructed by the skeleton graph positive and negative samples and the sample pair label, and the loss function used is cross-entropy loss.

[0088] Further, the step S3 specifically includes:

[0089] Step S31: generating a heat map using the key point coordinate label of the source domain dataset as the training label of the pose estimation model;

[0090] Step S32: constructing a pose estimation model based on key point heat map regression;

[0091] Step S33: training the pose estimation model using the training images in the source domain dataset and the corresponding heat map label set, and the loss function used is mean square error loss.

[0092] Preferably, the construction of the pose estimation model based on key point heatmap regression specifically includes: a feature extraction module, a feature fusion module based on a transformer structure, and a feature decoding module.

[0093] Preferably, the feature extraction module is a pre-trained ResNet50, whose input is a 3D image with a resolution of 3×w×h, and whose output is a 3D image with a size of 3×w×h. Tensor characteristics.

[0094] Preferably, such as Figure 3 As shown, the feature fusion module based on the transformer structure consists of three parallel attention mechanism modules, including two self-attention modules and one cross-attention module. The input to the two self-attention modules is image features from the feature extraction module, while the query vector, key vector, and query vector of the cross-attention module are derived from the two self-attention modules, respectively. The concatenated features of the two attention modules along the channel dimension constitute the output of the transformer-based feature fusion module.

[0095] Preferably, the feature decoding module consists of three upsampling modules and a convolutional output layer with a kernel size of 1×1.

[0096] Preferably, the upsampling module consists of an upsampling layer with a stride of 2, a two-dimensional convolutional layer, a batch regularization layer, and a ReLU activation layer in sequence.

[0097] Furthermore, step S4 specifically includes:

[0098] Step S41: Obtain augmented samples of the image in the target domain through rotation transformation;

[0099] Step S42: The target domain image and its augmented sample are inferred by the pose estimation model trained on the source domain to obtain the predicted key point heatmap, and the key point coordinates are obtained by non-maximum suppression.

[0100] Step S43: Based on the key point prediction structure of the target domain image and its augmented samples in step S14, obtain its skeleton map sample pair.

[0101] Step S44: Use the graph comparison model trained in step S2 to predict the skeleton graph samples of the target domain data. Reasoning is performed to obtain the discrimination result S. T ;

[0102] Step S45: Based on the discrimination result S T The value divides the target domain dataset into pseudo-labeled subsets. and a subset of no pseudo labels

[0103] Preferably, the step S4 includes: T The standard for dividing the target domain data set into two subsets according to the value of the discriminant result S is that for the samples from the target domain data set There are

[0104]

[0105] Further, the step S5 specifically includes:

[0106] Step S51: constructing a domain adaptation training model;

[0107] Step S52: training using source domain data and target domain data;

[0108] Step S53: inference of the posture estimation model after domain adaptation training.

[0109] Preferably, the step of constructing a domain adaptation training model specifically includes a double-flow feature extraction module, a target feature alignment module, and a double-flow feature decoding module.

[0110] Preferably, the double-flow feature extraction module is composed of two Resnet50 networks sharing weights, and the pre-training weights are inherited from the feature extraction module of the posture estimation model trained based on the source domain data in step S3.

[0111] Preferably, the target feature alignment module is from the feature fusion module based on the transformer structure of the posture estimation model trained based on the source domain data in step S3. The two self-attention modules in this module input respectively from the two output features of the double-flow feature extraction module.

[0112] Preferably, the double-flow feature decoding module is composed of two groups of feature decoding modules with the same structure, and the structure and weights of each group of feature decoding modules are from the feature decoding module of the posture estimation model trained based on the source domain data in step S3.

[0113] The step of training using source domain data and target domain data specifically includes: randomly selecting one image from each of the source domain data set and the target domain data set with pseudo labels as a sample pair; then generating two groups of heat maps as training targets according to the key point labels respectively; the two images in the sample pair are respectively used as the two inputs in the double-flow feature extraction module; and the two groups of heat map labels are respectively calculated with the input from the double-flow feature decoder to calculate the mean square error loss.

[0114] ​Preferably, the domain adaptation training of the trained pose estimation model specifically comprises: pose estimation model module weight assignment and target domain model inference.

[0115] Preferably, the pose estimation model module weight assignment specifically comprises: the used pose estimation model structure is the pose estimation model constructed in step S3; the weights of the feature extraction module, the feature fusion module and the feature decoding module are respectively inherited from the target domain data branch in the double-flow feature extraction module, the target feature alignment module and the target domain data branch in the double-flow feature decoding module of the model trained in step S52.

[0116] Preferably, the target domain model inference specifically comprises: for each test sample in the target domain data set, first, the size of the sample is transformed to 3×w×h, then the trained pose estimation model obtained in step S32 is used for inference to obtain a predicted key point heat map set, and finally, the heat map set is subjected to a non-maximum suppression operation to obtain a final predicted key point set.

Claims

1. A method for cross-domain pose estimation based on skeleton graph structure constraint, characterized in that, The method comprises the following steps: S1: constructing skeleton graph positive and negative contrast samples based on source domain dataset labels; S2: constructing a graph contrast learning model and training the constructed skeleton graph structure positive and negative contrast sample set; S3: constructing a pose estimation model based on heat map regression and training the model based on the source domain dataset; S4: obtaining pseudo labels of target domain data, comprising inferring the target domain data using the pose estimation model trained on the source domain data, and using the graph contrast learning model to determine the pseudo labels of correct prediction; S5: constructing a domain adaptation training model and training the model using the source domain data and the target domain data; The step S1 specifically comprises: S11: generating rotation augmented samples for the key point labels of each source domain sample; S12: randomly selecting a number from the set [0, 1] as the label of the generated sample pair; S13: According to the label type of the sample pair, add the corresponding noise transformation type to the key point label of the source domain sample and its augmented sample to construct the key point simulation sample pair: if the randomly selected label is 1, the noise added to the key point label of the source domain sample and its augmented sample is all positive sample noise N pos ; if the randomly selected label is 0, at least one of the noise added to the key point label of the source domain sample and its augmented sample is negative sample noise N neg , the sampling range of the positive sample label N pos is: N pos ∈{n|0<n<0.2l}, and the sampling range of the negative sample label N neg is: N neg ∈{n|0.2*l<n<l}, wherein l is the length of the target torso; S14: Generate the node feature matrix M of the skeleton graph structure in the sample pair. f Calculate each keypoint P i Angle A between the two adjacent key points i Then, together with the Euclidean space coordinates of the key points, they form the node feature matrix M according to the following formula. f : S15: returning a sample set containing key point simulation sample pairs, node feature matrices of skeleton graph structure, and sample pair labels. 2.The method of claim 1, wherein, The step S2 specifically comprises: S21: constructing a graph contrast learning model, which comprises three GINconv graph convolution layers, a fully connected layer, and a Sigmoid activation layer; S22: The input of the graph contrast learning model is the node feature matrix M of the skeleton graph positive and negative samples f Concatenation matrix in x dimension; S23: training the graph contrast learning model using the input constructed by the skeleton graph positive and negative samples and the sample pair labels, and using a cross-entropy loss function. 3.The method of claim 1, wherein, The step S3 specifically comprises: S31: generating heat maps using the key point coordinate labels of the source domain dataset as the training labels of the pose estimation model; S32: constructing a pose estimation model based on key point heat map regression, specifically comprising a Resnet-based feature extraction module, a transformer structure-based feature fusion module, and a feature decoding module, wherein the feature decoding module comprises three up-sampling modules and a convolution kernel size of 1×1 convolution output layer, and the up-sampling module comprises an up-sampling layer with a step of 2, a two-dimensional convolution layer, a batch normalization layer, and a Relu activation layer in sequence; S33: training the pose estimation model using the training images in the source domain dataset and the corresponding heat map label set, and using a mean square error loss function.

4. The method of claim 3, wherein, The transformer structure-based feature fusion module in S32 comprises three parallel attention mechanism modules, including two self-attention modules and one cross-attention module, wherein the input of the two self-attention modules is the image feature from the feature extraction module, and the query vector, key vector and query vector of the cross-attention module come from the two self-attention modules, respectively. The channel dimension of the output features of the two attention modules is concatenated to obtain the output of the transformer structure-based feature fusion module.

5. The method of claim 1, wherein the method is based on skeleton graph structure constraint. The step S4 specifically comprises: S41: obtaining augmented samples of the target domain images through rotation transformation; S42: respectively inferring the target domain images and their augmented samples using the pose estimation model trained on the source domain to obtain predicted key point heat maps, and obtaining key point coordinates through non-maximum suppression; S43: process the key point prediction structure of the target domain image and its augmented sample to obtain a skeleton graph sample pair according to step S14 S44: predicting a skeleton graph sample pair of target domain data by using the graph contrast learning model trained in step S2 reasoning to obtain a discrimination result S T ; S45: According to the discrimination result S T The value divides the target domain data set into a subset with pseudo labels And a subset without pseudo labels Specifically: for samples from the target domain data set There are 6. The method of claim 1, wherein the method is based on skeleton graph structure constraints. The step S5 specifically comprises: S51: Constructing a domain adaptation training model, specifically including: a double-flow feature extraction module, a target feature alignment module, and a double-flow feature decoding module, wherein the double-flow feature extraction module is composed of two Resnet50 networks sharing weights, the pre-training weights of which are inherited from the feature extraction module of the pose estimation model trained based on the source domain data in step S3, the structure and weights of the target feature alignment module are from the feature fusion module based on the transformer structure of the pose estimation model trained based on the source domain data in step S3, the two self-attention modules in this module input respectively from the two output features of the double-flow feature extraction module, the double-flow feature decoding module is composed of two groups of feature decoding modules with the same structure, and the structure and weights of each group of feature decoding modules are from the feature decoding module of the pose estimation model trained based on the source domain data in step S3; S52: Training using source domain data and target domain data; S53: Inference of the pose estimation model after domain adaptation training, specifically including: pose estimation model module weight assignment and target domain model inference.

7. The method of claim 6, wherein, The training with the source domain data and the target domain data in S52 specifically includes: randomly selecting source domain data and target domain data with pseudo labels from a source domain data set and a target domain data set with pseudo labels Each of the pairs of images is selected as a sample pair; then two groups of heat maps are generated as training targets according to key point labels of the sample pair respectively; the two images in the sample pair are respectively used as two inputs in the double-flow feature extraction module; and the two groups of heat map labels are respectively used to calculate mean square error loss with inputs from the double-flow feature decoder.

8. The method of claim 6, wherein, The pose estimation model module weight assignment in S53 is specifically: the pose estimation model structure used is the pose estimation model constructed in step S3; The weights of the feature extraction module, the feature fusion module, and the feature decoding module are respectively inherited from the target domain data branch in the double-flow feature extraction module, the target feature alignment module, and the target domain data branch in the double-flow feature decoding module of the model trained in step S52; The target domain model inference in S53 is specifically: for each test sample in the target domain data set, first transform its size to 3×w×h, then use the pose estimation model obtained in step S32 to infer to get the predicted key point heat map set, and finally get the final predicted key point set through the non-maximum suppression operation on the heat map set. 9.The method of claim 1, wherein the method is applied in model migration training. In the animal pose estimation task across scenes and across species, the model migration training independent of data annotation is realized.

Images