Physique recognition method based on label-free facial image

Abstract

In order to make full use of a large amount of label-free face data to cope with the actual problems of slow labeling progress, label shortage and the like in the initial stage of face data set construction, the invention provides a body constitution recognition method based on a label-free face image. Taking the facial constitution data as a target domain for feature extraction to obtain source domain features and target domain features; according to the category center feature vector of the source domain feature sample, respectively determining optimal transmission matching streams of the source domain feature sample and the category center feature vector as well as the target domain feature sample and the category center feature vector; and determining the classification probability of the source domain feature sample according to the optimal transmission matching flow, and identifying the physique according to the classification probability.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more particularly to a physical constitution recognition method based on unlabeled facial images. Background Art

[0002] In traditional Chinese medicine theory, physical constitution refers to the physiological and psychological characteristics formed by an individual under the combined action of genetic and environmental factors. According to the classification of traditional Chinese medicine physical constitutions, humans can be divided into nine main physical constitution types, including balanced constitution, qi-deficiency constitution, yang-deficiency constitution, yin-deficiency constitution, phlegm-dampness constitution, damp-heat constitution, blood stasis constitution, qi stagnation constitution, and special endowment constitution. These physical constitution types are closely related to an individual's health status and disease tendency.

[0003] The traditional method is to judge an individual's physical constitution type by observing the individual's facial features such as expression, complexion, and macroscopic appearance. This method relies heavily on the experience and subjective judgment of traditional Chinese medicine practitioners.

[0004] In recent years, the development of artificial intelligence, especially deep learning technology, has made it possible to automatically and objectively identify physical constitution types. Deep learning models can automatically extract features related to physical constitution types and classify them by learning a large amount of facial image data.

[0005] However, using deep learning for facial physical constitution recognition faces an important challenge: training a deep neural network usually requires a large number of labeled facial images as training samples, and obtaining these labeled samples is not only costly but also time-consuming and laborious.

[0006] Currently, although there are a large number of unlabeled facial image data, due to the lack of corresponding physical constitution type labels, these data are difficult to be directly used for supervised learning. At the same time, the complexity of the labeling work and the requirements for professional knowledge make it more difficult to collect a sufficient number of labeled samples.

[0007] Therefore, how to effectively use unlabeled facial image data for pre-training and thus perform physical constitution recognition based on unlabeled facial images has become an urgent problem to be solved in the current technology.

[0008] The above background art part provides a technical background for this patent, explains the limitations of traditional methods, elaborates on the challenges of the existing technology, and outlines the main purpose and advantages of the solution proposed by the present invention. When writing a patent document, it should be ensured that the description is accurate, clear, and highlights the innovation points. Summary of the Invention

[0009] In view of this, the present invention provides a physical constitution recognition method based on unlabeled facial images, aiming to design an algorithm architecture that meets the cross-domain reconstruction consistency and hierarchical optimal transport constraints, make full use of rich unlabeled data resources for model pre-training, so as to achieve accurate facial physical constitution recognition, thereby reducing the dependence on labeled data and improving the generalization ability and recognition accuracy of the model.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A physical constitution recognition method based on unlabeled facial images, which performs physical constitution recognition by acquiring unlabeled facial images and combining facial physical constitution data. The recognition steps include:

[0012] S1. Using the unlabeled facial image as the source domain and the facial physical constitution data as the target domain to extract features, obtaining source domain features and target domain features;

[0013] S2. According to the class center feature vectors of the source domain feature samples, respectively determine the optimal transport matching flows of the source domain feature samples and the target domain feature samples with the class center feature vectors;

[0014] S3. Determine the classification probability of the source domain feature samples according to the optimal transport matching flow, and recognize the physical constitution according to the classification probability.

[0015] Preferably, in step S2, the optimal transport matching flow of the feature sample and the class center feature vector is determined according to the following formula;

[0016]

[0017] In the formula, Z represents the set of feature samples, B represents the set of class center feature vectors, Distance(Z,B) represents the optimal transport distance between the two set distributions, γ represents the optimal transport matching flow, represents the geodesic cost, Π(μ Z ,μ B ) represents the probability coupling set between the empirical distribution μ Z of the set of feature samples and the empirical distribution μ B of the set of class center feature vectors.

[0018] Preferably, the probability coupling set is obtained by the following formula,

[0019] Π(μ Z ,μ B )={γ∈(R + ) I×J |γ×1 J =μ Z ,γ T ×1 I =μ B}

[0020] In the formula, R + represents a positive real number, I represents an I-dimensional category unit vector, J represents a J-dimensional sample unit feature vector, 1 J represents the number of samples, 1 I represents the number of cluster centers;

[0021] Among them,

[0022]

[0023] The function and represent the probability distributions of the feature sample x i and the category center feature vector c k respectively.

[0024] Preferably, the geodesic cost calculates the distance from the sample vector x i to each prototype vector c k by cosine distance, and the formula is expressed as:

[0025]

[0026] Preferably, cross-domain feature reconstruction is performed on the source domain features and the target domain features respectively to obtain the source domain reconstructed features and the target domain reconstructed features;

[0027] The optimal transport matching flow between the source domain feature samples and the category center feature vectors is constrained by using the source domain reconstructed features, and the optimal transport matching flow between the target domain feature samples and the category center feature vectors is constrained by using the target domain reconstructed features.

[0028] Preferably, the steps of the consistency constraint include:

[0029] Based on the category center feature vector of the source domain feature samples, obtain the classification probability of the reconstructed features; the acquisition formula is:

[0030]

[0031] In the formula, x new represents the reconstructed features, τ represents the temperature coefficient in the Softmax process, c k represents the k-th category center feature vector, c m represents the m-th category center feature vector, which is used for denominator traversal summation.

[0032] Using the classification probability of the reconstructed features, the optimal transport matching flow between the corresponding feature samples and the category center feature vectors is constrained by KL divergence; the constraint formula is as follows:

[0033]

[0034] In the formula, γ represents the optimal transport matching flow, and p(x) represents the probability of reconstructing the feature sample x.

[0035] Preferably, the steps of cross-domain feature reconstruction for source domain features and target domain features include:

[0036] Determine the query, key, and value for the reconstruction of source domain features and target domain features respectively according to the learnable linear transformation matrix;

[0037] Swap the keys and values of the source domain features with the keys and values of the target domain features correspondingly, and calculate their respective self-attention;

[0038] Pass the self-attention of the source domain features and the self-attention of the target domain features through residual connection, layer normalization, and feed-forward calculation in sequence to obtain the source domain reconstruction features and the target domain reconstruction features.

[0039] Preferably, based on the hierarchical optimal transport distance, constrain the source domain reconstruction features and the target domain reconstruction features; the constraint loss is:

[0040]

[0041] In the formula, Z s represents the source domain reconstruction features, Z t represents the target domain reconstruction features, represents the hierarchical optimal transport loss between the source domain reconstruction features and the target domain reconstruction features, γ represents the optimal transport matching flow, Π(μ s , μ t ) represents the set of probability couplings between the empirical distribution μ s of the source domain reconstruction features and the empirical distribution μ t of the target domain reconstruction features, represents the sum of the optimal transport distance and the Euclidean distance between two samples.

[0042] Preferably, The calculation formula of

[0043]

[0044] In the formula, α and β respectively represent weight parameters, M represents the number of projections, θ m represents the m-th projection head that maps the high-dimensional embedding to 1 d , is the transpose of θ m , are respectively the feature maps after extracting features of two samples .

[0045] Preferably, the sum of the hierarchical optimal transport loss and the consistency loss is used as the objective function of the physical constitution recognition method, and the expression is:

[0046]

[0047] In the formula, λ1, λ2, and λ3 respectively represent the weight coefficients of each part of the loss, and KL(P s ||γ s ) represents the consistency constraint loss between the source domain reconstructed feature classification probability and the optimal transport matching flow of the source domain features, and KL(P t ||γ t ) represents the consistency constraint loss between the target domain reconstructed feature classification probability and the optimal transport matching flow of the target domain features.

[0048] As can be seen from the above technical solutions, the present invention discloses a physical constitution recognition method based on unlabeled facial images, which is used to make full use of a large amount of unlabeled facial data to address practical problems such as slow annotation progress and lack of labels in the initial stage of facial dataset construction. When more labeled data is gradually accumulated, the model can further improve the effect and performance through fine-tuning, and has good practicability; compared with the prior art, the present application has the following effects:

[0049] 1) By means of cross-domain feature reconstruction technology and hierarchical optimal transport distance constraint, the network can simultaneously focus on the local and global features of facial images, improving the performance.

[0050] 2) Through the classification consistency constraint of the original features and the reconstructed features, the generalization performance of the encoder in the target domain can be strengthened, enabling the model to effectively utilize the prior knowledge of the source domain to complete the image classification of the target domain in the absence of target domain labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0052] Figure 1 It is a flowchart of the physical constitution recognition method based on unlabeled facial images of the present invention;

[0053] Figure 2 It is a schematic diagram of the physical constitution recognition process based on unlabeled facial images of the present invention;

[0054] Figure 3 It is a schematic diagram of the cross-domain feature cross-reconstruction process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0056] To make full use of a large amount of unlabeled data in the source domain and the target domain, an embodiment of the present invention discloses a physical constitution recognition method based on unlabeled facial images, aiming to use the data in the two domains to achieve cross-domain few-shot learning, and then fine-tune with a small number of labels on the target domain to achieve high-performance facial physical constitution recognition.

[0057] Specifically, the implementation steps of this application include:

[0058] Obtain unlabeled facial images, and perform physical constitution recognition in combination with facial physical constitution data, such as Figure 1 , and the recognition steps include:

[0059] S1. Use the unlabeled facial image as the source domain and the facial physical constitution data as the target domain for feature extraction to obtain source domain features and target domain features; in this application, there may be a small amount of labeled data in the facial physical constitution data;

[0060] S2. According to the class center feature vectors of the source domain feature samples, respectively determine the optimal transport matching flows of the source domain feature samples and the target domain feature samples with the class center feature vectors;

[0061] S3. Determine the classification probability of the source domain feature samples according to the optimal transport matching flow, and identify the physical constitution according to the classification probability. That is, the greater the distance from the target domain sample to each class center feature vector determined by the optimal transport matching flow, the smaller the probability of classifying it into the class represented by the class center feature vector, and the class corresponding to the maximum probability is the physical constitution class of the sample, thus completing the physical constitution recognition.

[0062] Among them, referring to Figure 2 ,

[0063] In step S1, features in the unlabeled facial image and the facial physical constitution data are extracted through an encoder;

[0064] In step S2, after extracting the feature vectors of all samples in each class in the source domain, the average of all feature vectors is obtained to obtain the class center feature vector of the source domain feature samples, and then all class center vectors are stored in a prototype library. In this application, the prototype library is embodied as a fully connected layer structure in algorithm implementation, and its weights are stored as an updatable parameter matrix to store the class centers. At this time, the parameter matrix of the prototype library can be expressed as B = [c1,..., ck ,…,c n ,c k ∈R H×W×C 。

[0065] Furthermore, the category to which the sample belongs is calculated by computing the distance from the sample point to the center of each category. In this embodiment, the optimal transport distance is used as the cost function, but the optimal transport distance does not participate in the model's backpropagation as a loss. Instead, the optimal transport matching flow between the sample and the category center feature is used as the category probability of the sample.

[0066] Suppose μ Z and μ B are the probability distributions of the samples x i within a batch Z and the prototype feature vectors c k in the prototype library B respectively. Then μ Z and μ B can be represented by the formula:

[0067]

[0068] where the Dirac functions and represent the probability distributions of the feature samples x i and the category center feature vectors c k ; μ Z and μ B are defined as uniform distributions. The optimal transport distance between these two set distributions can be defined as:

[0069]

[0070] In the formula, Z represents the set of feature samples, B represents the set of category center feature vectors, Distance(Z,B) represents the optimal transport distance between the two set distributions, γ represents the optimal transport matching flow, represents the geodesic cost, <·,·> F is the Frobenius inner product, Π(μ Z ,μ B ) represents the set of probability couplings between the empirical distribution μ Z of the feature sample set and the empirical distribution μ B of the category center feature vector set, and is expressed by the formula:

[0071] Π(μ Z ,μ B ) = {γ ∈ (R + ) I×K |γ1 K = μ Z ,γ T 1 I = μB}

[0072] Wherein, R + , I, 1 J , 1 I respectively represent positive real numbers, the number of samples, the number of cluster centers, the J-dimensional sample unit feature vector, and the I-dimensional class unit vector.

[0073] In this embodiment, the geodesic cost matrix is non-negative, representing the distance between the sample and the prototype vector; for the geodesic cost the cosine distance is used to calculate the distance from the sample x i to each prototype vector, and the calculation formula is

[0074]

[0075] In this application, by minimizing the optimal transport distance between two distributions, the optimal matching flow γ s , γ t from the sample nodes of the two domains to the class center vector is calculated. After normalization, the obtained matching flow can reflect the classification probability of the original sample for each prototype vector in the prototype library.

[0076] In a preferred embodiment, cross-domain feature reconstruction is performed on the source domain features and the target domain features respectively to achieve data augmentation through feature reconstruction;

[0077] Then, the optimal transport matching flow between the source domain feature samples and the class center feature vector is constrained by using the source domain reconstructed features, and the optimal transport matching flow between the target domain feature samples and the class center feature vector is constrained by using the target domain reconstructed features.

[0078] In one embodiment, the steps of performing cross-domain feature reconstruction on the source domain features and the target domain features include:

[0079] According to the learnable linear transformation matrix, the queries, keys, and values for the source domain feature reconstruction and the target domain feature reconstruction are determined respectively;

[0080] The keys and values of the source domain features are swapped with the keys and values of the target domain features correspondingly, and their respective self-attention is calculated;

[0081] The self-attention of the source domain features and the self-attention of the target domain features are respectively passed through residual connection, layer normalization, and feed-forward calculation to obtain the source domain reconstructed features and the target domain reconstructed features.

[0082] It should be specifically noted that in this embodiment, by introducing the self-attention-based cross-domain feature recombination SCF module, the diversity of the target domain samples is enhanced while the key features for in-domain classification are retained.

[0083] As shown Figure 3 in the figure, the SCF module calculates self-attention using samples within a batch, and generates new features by weighting according to the self-attention matrix. These new features incorporate most of the key features from the source domain and the style information from another domain, enriching the samples of the target domain with a large number of source domain samples to address the few-shot problem.

[0084] The cross-domain feature cross-reconstruction method reconstructs samples of one domain using features of the other domain by cross-utilizing the feature projections of different views within two domains. Taking the reconstruction of source domain features as an example, as Figure 2 , the self-attention calculation formula at this time is as follows:

[0085]

[0086] Among them, is the source domain batch sample feature set, represents the target domain batch sample feature set. Connect Z s , Z t into a matrix Among them

[0087] Given a learnable linear transformation matrix calculate the query (Q), key (K), and value (V) according to formula (2);

[0088] Q = ZW q , K = ZW k , V = ZW v

[0089] After calculating the weighted self-attention matrix representing the source domain sample features using the above formula, through residual connection, layer normalization, and feed-forward calculation, the reconstructed source domain sample features are obtained; the corresponding formula is expressed as:

[0090]

[0091] Among them, LN represents layer normalization, and FC represents the fully connected layer of feed-forward propagation. The reconstructed sample as a data augmentation version of the source domain sample, is input into the reconstruction consistency constraint module for subsequent training together with the original sample Z s .

[0092] Similarly, the calculation formula for the reconstructed target domain sample features is as follows:

[0093]

[0094] Similarly, the reconstructed sample with the original sample Z t will continue to participate in the subsequent training of the online clustering module.

[0095] In this embodiment, the self-attention calculation process is implemented based on the encoding layer in the Transformer model. Different from the traditional self-attention solution, this application uses cross self-attention to complete feature reconstruction, providing a solution for the data augmentation method of unlabeled images.

[0096] In addition, since the reconstructed image has the same label as the original image, the present invention does not need to obtain the image label, and the downstream module can form supervision through the consistency constraint of the features of the original image and the reconstructed image. This is similar to the idea of self-supervised training, without sampling positive and negative samples, simplifying the training process and reducing the cost of label annotation.

[0097] In one embodiment, the batch feature sets Z s , Z t from the source domain and the target domain, after feature reconstruction, form a new batch of enhanced features Since the inputs are all unlabeled samples, to make full use of the structured information contained in the unlabeled samples, this application constrains the consistency of the classification probabilities of the original features and the reconstructed features to achieve the purpose of "supervision".

[0098] Specifically, for the original image and the reconstructed version of the same picture, this application hopes that their classification probabilities are consistent. Therefore, the classification results of the original features in the two domains and the classification results of the reconstructed features of their respective samples are subjected to a consistency constraint. While training the prototype vectors in the prototype library to complete online clustering, the ability of the encoder to adapt to the target domain tasks is enhanced. Since the reconstructed sample features are all composed of the information embedding of another domain, this consistency constraint can enhance the generalization performance of the encoder in the two domains to narrow the style difference between the two domains.

[0099] First, obtain the classification results of the reconstructed features of their respective samples;

[0100] Since there are differences between the two domains, it is unreasonable to directly pass the two types of features through the same classifier violently, because this strong constraint will cause the encoder to ignore the local key discriminative information. Without the guidance of label information, all pictures will ultimately be classified into the same category. Therefore, this application sets up a prototype library to buffer the strong constraint caused by using the same classifier structure.

[0101] The prototype library is implemented as a fully connected layer structure in the algorithm, and its weights are stored as an updatable parameter matrix for class centers. In this case, the parameter matrix of the prototype library can be expressed as B = [c1,..., c k ,..., c n , ck ∈R H ×W×C 。

[0102] For the input batch reconstruction feature set To calculate the similarity between the reconstruction feature and each prototype vector, this application is implemented by cross-multiplying the two vectors and through the feed-forward classification process of the fully connected layer; the formula is expressed as:

[0103]

[0104] In the formula, x new represents the reconstruction feature, τ represents the temperature coefficient in the Softmax process, c k represents the k-th category center feature vector, c m represents the m-th category center feature vector, which is used for the denominator traversal summation.

[0105] Secondly, using the classification probability of the reconstruction feature, the optimal transport matching flow between the corresponding feature sample and the category center feature vector is constrained for consistency through KL divergence; the constraint formula is as follows:

[0106]

[0107] In the formula, γ represents the optimal transport matching flow, and p(x) represents the probability of the reconstruction feature sample.

[0108] In an exemplary embodiment, based on the prototype library, the model obtains the basic classification probability through Softmax regression, and the calculation formula is as follows:

[0109]

[0110] Among them, P s , P t are the classification probabilities of the reconstruction features in the two domains, τ is the temperature coefficient in the Softmax process, is the feature map of a reconstruction sample in the two domains.

[0111] Subsequently, this method constrains the consistency with the above-mentioned reconstruction feature classification probability according to the category probability distribution of the original sample (i.e., the optimal transport matching flow γ s , γ t ) through KL divergence. The calculation formula is:

[0112]

[0113]

[0114] To further optimize the above technical solution, the present application constrains the source domain reconstruction features and the target domain reconstruction features based on the hierarchical optimal transport distance; the constraint loss is:

[0115]

[0116] In the formula, Z s represents the source domain reconstruction features, and Z t represents the target domain reconstruction features. represents the hierarchical optimal transport loss between the source domain reconstruction features and the target domain reconstruction features, γ represents the optimal transport matching flow, and Π(μ s , μ t ) represents the set of probability couplings between the empirical distribution μ s of the source domain reconstruction features and the empirical distribution μ t of the target domain reconstruction features. represents the sum of the optimal transport distance and the Euclidean distance between two samples.

[0117] In the present invention, the calculation of the optimal transport distance focuses on local features of the image such as the morphology of facial features. However, overall features of the face such as mental state and facial color are also very important. Therefore, when calculating the optimal transport distance between samples , the Euclidean distance is added to focus on global features; the corresponding calculation formula is:

[0118]

[0119] In the formula, α and β respectively represent weight parameters, M is the number of projections, and θ m is the m-th projection head that maps the high-dimensional embedding to 1 d , is the transpose of θ m , and are the feature maps after extracting features of two samples respectively.

[0120] According to the above formula, the hierarchical transport distance between the in-batch general face dataset and the traditional Chinese medicine facial constitution dataset can be calculated. Using this distance as a loss for backpropagation helps to weaken the influence brought by domain style information such as expression and shooting angle, and at the same time enriches the sample composition of the traditional Chinese medicine facial constitution dataset, which helps to enhance the accuracy of the classification task.

[0121] In a preferred embodiment, in the constitution recognition method disclosed by the present invention, the objective function is composed of a reconstruction consistency loss and an inter-domain proximity loss. Among them, the hierarchical optimal transport distance is directly used as the inter-domain proximity loss to participate in backpropagation, and the formula is as follows:

[0122]

[0123] wherein, λ1, λ2, and λ3 respectively represent the weight coefficients of each part of the loss, and KL(P s ||γ s ) represents the consistency constraint loss between the source domain reconstructed feature classification probability and the optimal transport matching flow of the source domain features, and KL(P t ||γ t ) represents the consistency constraint loss between the target domain reconstructed feature classification probability and the optimal transport matching flow of the target domain features.

[0124] Furthermore, since there is partial labeled data in the target domain, after training, the network is further fine-tuned using the labeled data in the target domain. At this time, a simple and effective cross-entropy function can be used to calculate the classification loss during the fine-tuning process. Assume that the target domain data is Then the cross-entropy loss is calculated as follows:

[0125]

[0126] where φ represents the model parameters, and f φ (x i ) represents the model prediction result.

[0127] In a preferred embodiment, the present application also provides a physical constitution recognition network / device, as shown, which respectively includes an encoder, a cross-domain feature cross-reconstruction module, a reconstruction consistency module based on a prototype library, and a hierarchical optimal transport module;

[0128] wherein, the input of the network is a batch of unlabeled general face datasets from the source domain and facial physical constitution datasets from the target domain;

[0129] The features are extracted by the encoder and input into the cross-domain feature cross-reconstruction module, and data augmentation is achieved through feature reconstruction, and the hierarchical optimal transport constraint is satisfied;

[0130] Finally, the reconstructed sample features and the features extracted by the encoder are input into the reconstruction consistency module in the prototype library to constrain the online clustering process. The center vector of each category is obtained through clustering, and then, with the optimal transport distance as the metric, the probability distribution of the sample belonging to each category is obtained through the prototype classifier, and the consistency loss between the probability distributions of the original sample and the augmented sample is calculated. In addition, the augmented sample is also input into the hierarchical optimal transport network to calculate the inter-domain optimal transport loss.

[0131] Based on the fully trained model, it is further fine-tuned using the labeled facial physical constitution data.

[0132] The advantages of the present invention are that a large amount of unlabeled facial data can be fully utilized, which is easily obtainable, to address practical problems such as slow annotation progress and lack of labels in the initial stage of facial dataset construction. Secondly, as the amount of labeled facial constitution data increases, the model can further improve its performance through fine-tuning, and has good practicality.

[0133] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0134] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A physical constitution recognition method based on unlabeled facial images, characterized in that an unlabeled facial image is obtained, and physical constitution recognition is performed in combination with facial physical constitution data. The recognition steps include: S1. Using the unlabeled facial image as the source domain and the facial physical constitution data as the target domain for feature extraction to obtain source domain features and target domain features; S2. According to the class center feature vectors of the source domain feature samples, respectively determine the optimal transport matching flows between the source domain feature samples and the target domain feature samples and the class center feature vectors; S3. Determine the classification probability of the source domain feature samples according to the optimal transport matching flow, and recognize the physical constitution according to the classification probability.

2. The physique recognition method based on label-free facial images according to claim 1, characterized in that In step S2, the optimal transport matching flow between the feature sample and the class center feature vector is determined according to the following formula; Wherein, Z represents the set of characteristic samples, B represents the set of category center characteristic vectors, Distance(Z, B) represents the optimal transport distance between the distributions of the two sets, γ represents the optimal transport matching flow, represents the geodesic cost, Π(μ Z , μ B ) represents the empirical distribution μ Z of the set of characteristic samples and the empirical distribution μ B between the probability coupling sets of the set of category center characteristic vectors.

3. The physique recognition method based on the tagless facial image according to claim 2, wherein The probability coupling set is obtained by the following formula, Π(μ Z ,μ B ) = {γ ∈ (R + ) I×J | γ × 1 J = μ Z , γ T × 1 I = μ B} where R + represents a positive real number, I represents an I - dimensional category unit vector, J represents a J - dimensional sample unit feature vector, 1 J represents the number of samples, 1 I represents the number of clustering centers; where Function and represent the probability distributions of the feature sample x i and the class center feature vector c k .

4. The physique recognition method based on unlabeled facial images according to claim 2, characterized in that, The geodesic cost calculates the distance between the sample vector x i and the feature vector c of the k-th class center k using the cosine distance, and is expressed by the formula:

5. The physique recognition method based on label-free facial images according to claim 1, characterized in that Cross-domain feature reconstruction is respectively performed on the source domain features and the target domain features to obtain source domain reconstruction features and target domain reconstruction features; The optimal transport matching flow between the source domain feature samples and the class center feature vectors is constrained by using the source domain reconstruction features, and the optimal transport matching flow between the target domain feature samples and the class center feature vectors is constrained by using the target domain reconstruction features.

6. The physique recognition method based on the tagless facial image according to claim 5, wherein The steps of consistency constraint include: Based on the class center feature vector of the source domain feature sample, obtain the classification probability of the reconstruction feature; the acquisition formula is: where x new represents the reconstruction feature, τ represents the temperature coefficient in the Softmax process, c k represents the k-th class center feature vector, c m represents the m-th class center feature vector. Using the classification probability of the reconstruction feature, perform consistency constraint on the optimal transport matching flow between the corresponding feature sample and the class center feature vector through KL divergence; the constraint formula is as follows: In the formula, γ represents the optimal transport matching flow, and p(x) represents the probability of the reconstruction feature sample x.

7. The physique recognition method based on label-free facial images according to claim 5, wherein, The steps of cross-domain feature reconstruction for the source domain features and the target domain features include: According to the learnable linear transformation matrix, respectively determine the query, key, and value for the reconstruction of the source domain features and the target domain features; Exchange the keys and values of the source domain features with the keys and values of the target domain features correspondingly, and calculate their respective self-attention; Respectively pass the self-attention of the source domain features and the self-attention of the target domain features through residual connection, layer normalization, and feed-forward calculation to obtain source domain reconstruction features and target domain reconstruction features.

8. The constitution recognition method based on the unlabeled facial image according to claim 5, characterized in that, Based on the hierarchical optimal transport distance, constrain the source domain reconstruction features and the target domain reconstruction features; the constraint loss is: Wherein, Z s represents the source domain reconstruction feature, and Z t represents the target domain reconstruction feature. represents the hierarchical optimal transport loss between the source domain reconstruction feature and the target domain reconstruction feature. γ represents the optimal transport matching flow, and Π(μ s , μ t ) represents the set of probability couplings between the empirical distribution μ s of the source domain reconstruction feature and the empirical distribution μ t of the target domain reconstruction feature. represents the sum of the optimal transport distance and the Euclidean distance between two samples.

9. The physique recognition method based on label-free facial images according to claim 8, characterized in that, The calculation formula is as follows: where α and β represent weight parameters respectively, M represents the number of projections, and θ m represents the m-th projection head that maps the high-dimensional embedding to 1 d , and is the transpose of θ m . and are the feature maps after extracting features of two samples respectively.

10. The constitution recognition method based on label-free facial images according to claim 8, characterized in that, Taking the sum of the hierarchical optimal transport loss and the consistency loss as the objective function of the physical constitution recognition method, the expression is: where λ1, λ2, and λ3 respectively represent the weight coefficients of each part of the loss, and KL(P s ||γ s ) represents the consistency constraint loss between the source domain reconstructed feature classification probability and the optimal transport matching flow of the source domain features, and KL(P t ||γ t ) represents the consistency constraint loss between the target domain reconstructed feature classification probability and the optimal transport matching flow of the target domain features.

Images