Method for improving environment generalization ability of multi-person gait identity recognition model

Abstract

The invention provides a method for improving the environment generalization ability of a multi-person gait identity recognition model, and the method comprises the steps: carrying out the data collection, obtaining a pre-training feature extractor, carrying out the feature extraction of source domain data and target domain data on a Doppler branch and a time domain autocorrelation (TAC) branch through the pre-training feature extractor, carrying out the feature extraction of the target domain data, carrying out the feature extraction of the target domain data, and carrying out the feature extraction of the target domain data, clustering is carried out according to feature distribution, category prototypes of the two domains are constructed, intra-domain prototype self-supervision loss is calculated, a system carries out inter-domain sample-prototype matching in a single branch of a network by using the clustering prototypes, cross-domain matching loss is calculated, feature fusion of the two branches is carried out in a source domain and a target domain, and the self-supervision loss of the intra-domain prototypes is calculated. The method comprises the following steps of: firstly, performing sample confidence estimation, realizing adaptive updating of network classifier parameters by using a high-confidence sample, and finally, performing further fine tuning on the model through classification task loss and mutual information loss optimization to improve the classification performance of the model on a target domain, thereby improving the generalization ability of the identity recognition model.

Description

Technical Field

[0001] The present invention relates to the field of communications and computer technology, and in particular to a method for improving the generalization capability of a recognition model environment. Background Art

[0002] Currently, multi-person gait identification models based on machine learning or deep learning can achieve high recognition accuracy in the training environment. However, when the task is switched to a new environment, their recognition accuracy often drops significantly. This phenomenon is mainly due to the high sensitivity of Wi-Fi signals to environmental changes. Even if the same target performs the same action in different scenarios, the impact of signal multipath effects can vary significantly. As a result, the channel state information (CSI) collected in different environments exhibits completely different feature distributions and data patterns. This difference in feature distribution between the source and target domains hinders cross-domain transfer of the model and reduces its generalization ability.

[0003] To achieve good model performance in the target domain, traditional solutions require re-collecting a large amount of annotated data in the new environment for model training, a process that often requires significant human, material, and time investment. For example, patent publication number CN116343261A discloses a gesture recognition method based on multimodal feature fusion and small-sample learning. The method obtains source and target domain data for gesture recognition. First and second feature extractors are trained using the source domain data from different links to obtain first and second features, respectively. These first and second features are then fed into a multimodal feature fusion process to generate multimodal fusion features. Training features are obtained from the target domain data based on the multimodal feature fusion process, and a classifier is trained based on these training features. Gesture recognition is then performed based on the trained classifier. This method only involves fine-tuning the parameters of the top-level classifier, without performing related fine-tuning on other modules of the model. Furthermore, it still requires some supervised target domain data as support. Due to privacy protection and high labeling costs, labeling each target domain data to obtain sufficient labeled data samples is impractical. In order to promote the wider promotion and deployment of multi-person gait recognition systems in new environments, it is urgent to develop effective cross-domain adaptation methods so that the model can effectively adapt between different domains, apply the learned knowledge and experience to related tasks in another domain, and improve the model's generalization ability. Summary of the Invention

[0004] In response to the technical problem that existing identity recognition models have poor generalization ability in new environments, the present invention proposes a method for improving the environmental generalization ability of a multi-person gait identity recognition model. Through self-supervised learning of prototypes within the domain and inter-domain sample-prototype matching, dual-branch feature fusion and adaptive network fine-tuning, the generalization ability of the identity recognition model is improved.

[0005] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] A method for improving the environmental generalization ability of a multi-person gait identity recognition model, comprising the following steps:

[0007] S1: Collect and preprocess source domain data and target domain small sample data, extract the corrected Doppler spectrum and TAC information from the preprocessed data, pre-train the identity recognition model based on the source domain data and obtain the pre-trained feature extractor;

[0008] S2: Use the pre-trained feature extractor to extract features from the corrected Doppler spectrum and TAC information respectively, and then construct a dual-branch source-target domain memory library and a dual-branch source-target domain prototype. Based on the dual-branch source-target domain prototype, perform intra-domain prototype self-supervision learning and inter-domain sample-prototype matching to obtain the intra-domain prototype self-supervision loss and inter-domain sample-prototype matching loss respectively;

[0009] S3: Based on the extracted corrected Doppler spectrum features and TAC information features, dual-branch intra-domain feature fusion is performed. The fused dual-branch intra-domain features are used as input, and the classifier is used to obtain classification prediction results. The prediction results of the dual-domain samples with high confidence are selected to calculate the mutual information loss. The total loss is calculated by combining the intra-domain prototype self-supervision loss and the inter-domain sample-prototype matching loss. The weights of the feature extractor and the weights of the first fully connected layer of the classifier are updated.

[0010] S4: Based on high-confidence dual-domain samples, dual-branch domain prototype fusion is performed, and the weights of the second fully connected layer of the classifier are adaptively updated based on the updated weights of the first fully connected layer of the classifier. Steps S2 to S4 are repeated until the optimal multi-person gait identity recognition model is obtained.

[0011] Furthermore, the method for constructing a dual-branch source domain and target domain memory library is as follows:

[0012] In the Doppler branch and TAC branch, the pre-trained feature extractor is used to extract the source domain features F * (* s ) and target domain features F * (* t ), according to the source domain feature F * (* s ) and target domain features F * (* t ) respectively build source domain memory and target domain memory in, represents the feature vector of the stored source domain samples, Represents the feature vector of the stored target domain sample, n sand n t are the number of source domain samples and the number of target domain samples respectively; among them, From the source domain feature F * (* s ) Flattened source domain feature vector The memory momentum β is used to update each batch of training: By the target domain feature F * (* t ) The flattened target domain feature vector The memory momentum β is used to update each batch of training: Where * represents the Doppler branch D or TAC branch T.

[0013] Furthermore, the method for constructing a dual-branch source domain and target domain prototype is as follows:

[0014] In the Doppler branch and TAC branch, the source domain memory and target domain memory Perform N K-Means clustering, and obtain the source domain cluster in each K-Means clustering and the target domain cluster represents the jth cluster in the source domain cluster, K is the total number of clusters, represents the jth cluster in the target domain cluster;

[0015] Based on the source domain cluster and the target domain cluster Get the source domain prototypes respectively and target domain prototype The acquisition method is:

[0016]

[0017] in, represents the prototype of the jth cluster in the source domain cluster or the target domain cluster, represents the mean of all feature vectors belonging to the jth cluster in the source or target domain, ||·|| represents the norm operation, Represents the feature vector within the j-th source domain cluster or target domain cluster The number of

[0018] Furthermore, the method for performing self-supervised learning of prototypes in the domain is as follows: calculating the source domain feature vector in the Doppler branch and the TAC branch and each source domain prototype Similarity distribution between Using the target domain feature vector and each source domain prototype Similarity distribution between The cross entropy loss is used to calculate the in-domain prototype self-supervision loss L of the Doppler branch respectively In-doppler and the in-domain prototype self-supervision loss of the TAC branch and L In-TAC , the calculation method is:

[0019]

[0020] Furthermore, the method for performing inter-domain sample-prototype matching is as follows: in the Doppler branch and the TAC branch, for the source domain sample feature vector and target domain prototype Calculate the similarity distribution:

[0021]

[0022] in, θ is the temperature hyperparameter that controls the smoothness of the distribution;

[0023] According to the target domain sample feature vector and source domain prototype Calculate the similarity distribution:

[0024]

[0025] in,

[0026] Calculate the entropy of the similarity distribution:

[0027]

[0028] The inter-domain sample-prototype matching loss L of the Doppler branch is calculated based on the entropy of the similarity distribution. Cross-domain-doppler and the inter-domain sample-prototype matching loss L of the TAC branch Cross-domain-TAC , the calculation method is:

[0029] Furthermore, the method for performing dual-branch feature fusion in step S3 is: fusing the features extracted from the Doppler branch and the TAC branch in the source domain and the target domain respectively based on the attention mechanism:

[0030]

[0031] in, is the source domain fusion feature, is the target domain fusion feature, is the channel attention weight of the Doppler branch source domain feature, is the channel attention weight of the source domain feature of the TAC branch, is the channel attention weight of the target domain feature of the Doppler branch, is the channel attention weight of the target domain feature of the TAC branch.

[0032] Furthermore, the method of using the classifier to obtain the classification prediction results and selecting the high confidence dual domain sample prediction results to calculate the mutual information loss is: fusion feature of the source domain Fusion features with the target domain Input the classifier and obtain the predicted probability distribution of source domain labeled samples and target domain unlabeled samples; for category k, based on the predicted probability distribution of target domain unlabeled samples, t p Establish a target domain sample set with high confidence for the confidence threshold, including a Doppler branch high confidence target domain sample set and TAC branch high confidence target domain sample set For category k, select high-confidence samples with sample labels corresponding to category k from the predicted probability distribution of labeled samples in the source domain and establish a source domain sample set with high confidence, including a Doppler branch high-confidence source domain sample set and TAC branch high confidence source domain sample set

[0033] The mutual information loss is calculated based on the predicted probability distribution of high-confidence target domain samples and the predicted probability distribution of high-confidence source domain samples:

[0034]

[0035] in, is a dual-domain high confidence sample x i The average predicted probability distribution, K is the number of categories, j represents the category index, y j is the true category label of the jth category of the source domain data, is the number of high confidence samples in the dual domains.

[0036] Furthermore, the total loss is:

[0037] L FUDA =L cls +λ In L In-domain +λ Cross L Cross-domain +λ MIM L MIM

[0038] Among them, L cls is the cross entropy loss, L In-domain =L In-doppler +L In-TAC is the total loss of prototype self-supervision in the domain, L Cross-domain =L Cross-domain-doppler +L Cross-domain-TACis the total loss of inter-domain sample-prototype matching, λ In ,λ Cross and λ MIM They are all hyperparameters that control and adjust the proportion of each loss.

[0039] Furthermore, the method for performing dual-branch domain prototype fusion based on high-confidence dual-domain samples is as follows: establishing a target domain high-confidence prototype based on the target domain memory library and the target domain high-confidence samples, and establishing a source domain high-confidence prototype based on the source domain memory library and the source domain high-confidence samples; using the channel attention weight to fuse the Doppler branch high-confidence prototype and the TAC branch high-confidence prototype of the target domain and the source domain respectively; adaptively updating the weights of the second fully connected layer of the classifier based on the target domain fusion prototype and the source domain fusion prototype and the updated weights of the first fully connected layer of the classifier.

[0040] Furthermore, the method for obtaining the high-confidence prototype of the target domain is:

[0041] The high confidence prototype of the Doppler branch target domain is:

[0042]

[0043] in, The feature vector of the target domain samples stored in the target domain memory bank of the Doppler branch;

[0044] The high-confidence prototype of the TAC branch target domain is:

[0045]

[0046] in, The feature vector of the target domain sample stored in the target domain memory bank of the TAC branch;

[0047] The method for obtaining the source domain high confidence prototype is:

[0048] The high confidence prototype of the Doppler branch source domain is:

[0049]

[0050] in, The feature vector of the source domain samples stored in the source domain memory bank of the Doppler branch;

[0051] The high-confidence prototype of the TAC branch source domain is:

[0052]

[0053] in, The feature vector of the source domain sample stored in the source domain memory bank of the TAC branch;

[0054] The method for adaptively updating the weights of the second fully connected layer of the classifier is:

[0055]

[0056] Among them, t n is the threshold of the number of high confidence target domain samples, is the weight of the first fully connected layer, LReLU(·) is the Leak ReLU activation function, norm(·) represents the normalization operation, w k is the weight of the second fully connected layer of the updated classifier.

[0057] The beneficial effects of the present invention are:

[0058] The present invention uses labeled data from the source domain to pre-train the identity recognition model to obtain a feature extractor with discriminative capabilities, aiming to establish an initial feature representation, provide reliable parameter initialization for subsequent unsupervised domain adaptation, and avoid the model from falling into local optimality when training in the target domain. Secondly, in response to the problem of lack of cross-domain semantic similarity matching in traditional domain adaptation methods, the present invention proposes a joint optimization strategy based on prototype learning and self-supervised learning, performs intra-domain prototype self-supervised learning and inter-domain sample-prototype matching, and applies it to a dual-branch feature fusion network. This method implicitly encodes the semantic information contained in the data into the embedding space, and encodes and aligns the semantic structures of the source domain and the target domain in the shared embedding space, thereby avoiding the sensitivity of direct sample matching and more robustly realizing the transfer of the system from the source domain to the target domain. Finally, in the target domain adaptation stage, the design uses high-confidence sample prototypes to adaptively update the network classifier parameters, and further updates the model through classification task loss and mutual information loss optimization, significantly improving the classification performance of the model in the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 The figure is an overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0062] A method to improve the environmental generalization ability of multi-person gait identity recognition model, such as Figure 1 As shown, the steps are:

[0063] S1: Collect and preprocess source domain data and target domain small sample data, perform Doppler spectrum correction extraction and TAC information extraction on the preprocessed data, pre-train the identity recognition model based on the source domain data and obtain the pre-trained feature extractor.

[0064] Data Collection: In this example, four mini PCs equipped with Intel 5300 network cards were used as transceivers to collect CSI data from three environments. Data from environment 1 was used as the source domain data, and data from environments 2 and 3 was used as small sample data for the target domain. The source domain data was labeled, while the target domain data labels were not included in the system during training and were only used to calculate classification accuracy during performance analysis during the testing phase.

[0065] Preprocessing: For the data in the source and target domains, a multi-person signal separation algorithm based on the inverse solution of Multiple Hypothesis Tracking (MHT) and Multiple Signal Classification (MUSIC) is used to separate the walking signal of each target, and low-pass filtering and denoising operations are performed on the amplitude and phase of the signal respectively.

[0066] Corrected Doppler spectrum extraction and TAC information extraction: For the pre-processed source domain data and target domain small sample data, extract their corrected Doppler spectrum (source domain is represented by D s , the target domain is represented as D t ) and TAC information (the source domain is represented by T s , the target domain is represented as T t ) as a complementary gait fingerprint in the time-frequency domain for gait feature extraction in subsequent modules. Because the input data dimension of the subsequent dual-branch feature fusion identity recognition model based on the attention mechanism must be fixed, the gait fingerprint also needs to be interpolated into data samples of the same dimension as the input of the subsequent identity recognition model.

[0067] Obtain a pre-trained feature extractor: The network trained in environment 1 is used as the pre-trained model. Its feature extractor parameters are used in the subsequent generalization capability improvement method based on small-sample unsupervised domain adaptation (FUDA), and its parameters are fine-tuned during the domain adaptation process. In this embodiment, the feature extractor is composed of a residual convolutional network, a spatial attention network, and a channel attention network, and the classifier is composed of a two-layer fully connected network.

[0068] S2: Use the pre-trained feature extractor to extract features of the corrected Doppler spectrum and TAC information respectively, and then construct a dual-branch source domain target domain memory library and a dual-branch source domain target domain prototype. Based on the dual-branch source domain target domain prototype, intra-domain prototype self-supervised learning and inter-domain sample-prototype matching are performed to obtain the intra-domain prototype self-supervised loss and inter-domain sample-prototype matching loss respectively.

[0069] S2.1: Construct a dual-branch source domain and target domain memory library:

[0070] In the Doppler branch, the pre-trained feature extractor is used to extract the source domain features F D (D s ) and target domain features F D (D t ), according to the source domain feature F D (D s ) and target domain features F D (D t ) respectively build source domain memory and target domain memory in, represents the feature vector of the stored source domain samples, Represents the feature vector of the stored target domain sample; where, From the source domain feature F D (D s ) Flattened source domain feature vector The memory momentum β is used to update each batch of training: Similarly, By the target domain feature F D (D t ) The flattened target domain feature vector The memory momentum β is used to update each batch of training:

[0071] In the TAC branch, the method is the same as above, but it complies with D s Change to T s , in line with D t Change to T t , which will not be elaborated here. The final TAC branch source domain memory library is represented as The target domain memory of the TAC branch is represented as

[0072] S2.2: Construct a dual-branch source domain and target domain prototype:

[0073] In the Doppler branch, the source domain memory and target domain memory Perform K-Means clustering to obtain source domain clusters and the target domain cluster represents the jth cluster in the source domain cluster, K is the total number of clusters, represents the jth cluster in the target domain cluster, and K is the total number of clusters;

[0074] Based on the source domain cluster The group obtains the source domain prototype:

[0075]

[0076] in, represents the prototype of the jth cluster in the source domain cluster, represents the mean of all feature vectors belonging to the jth cluster in the source domain, ||·|| represents the norm operation, Represents the feature vector within the jth cluster The number of

[0077] Similarly, according to the target domain cluster The group obtains the prototype of the target domain

[0078] Similarly, in the TAC branch, the source domain prototype is The target domain prototype is

[0079] S2.3: Perform in-domain prototype self-supervision learning to obtain in-domain prototype self-supervision loss:

[0080] In the Doppler branch, the source domain eigenvector is calculated and each source domain prototype Similarity distribution between Using the target domain feature vector and each source domain prototype Similarity distribution between The in-domain prototype self-supervision loss of the Doppler branch is calculated using cross entropy loss:

[0081]

[0082] in, and are the clustering indices of source and target domain samples, respectively, and CELoss is the cross entropy loss:

[0083]

[0084] * indicates D s or D t , is an indicator function if and only if j = c * (i) It is equal to 1, otherwise it is equal to 0.

[0085] Furthermore, in order to avoid the random influence of clustering caused by outliers, N K-Means clustering is performed, and the domain prototype self-supervision loss can be further rewritten as:

[0086]

[0087] In the TAC branch, the same calculation method is used above. The domain prototype self-supervision loss on the TAC branch is:

[0088]

[0089] Furthermore, the total in-domain prototype self-supervision loss is expressed as:

[0090] L In-domain =L In-doppler +L In-TAC .

[0091] S2.4: Perform inter-domain sample-prototype matching on each branch to obtain the inter-domain sample-prototype matching loss:

[0092] In the Doppler branch, for the source domain sample feature vector and target domain prototype Calculate the similarity distribution:

[0093]

[0094] in, θ is a temperature hyperparameter that controls the smoothness of the distribution. and source domain prototype Calculate the similarity distribution:

[0095]

[0096] in, Furthermore, the entropy of the similarity distribution is calculated:

[0097]

[0098] Furthermore, the inter-domain sample-prototype matching loss of the Doppler branch is:

[0099]

[0100] Similarly, in the TAC branch, the inter-domain sample-prototype matching loss of the TAC branch is:

[0101]

[0102] The final total inter-domain sample-prototype matching loss is:

[0103] L Cross-domain =L Cross-domain-doppler +L Cross-domain-TAC .

[0104] S3: Based on the extracted corrected Doppler spectrum features and TAC information features, dual-branch intra-domain feature fusion is performed. The fused dual-branch intra-domain features are used as input, and the classifier is used for classification to obtain the classification prediction results. The high-confidence dual-domain sample prediction results are selected to calculate the mutual information loss. The total loss is calculated by combining the intra-domain prototype self-supervision loss and the inter-domain sample-prototype matching loss, and the feature extractor weights and the weights of the first fully connected layer of the classifier are updated.

[0105] A dual-branch structure is adopted, and the features extracted by the Doppler branch and the TAC branch are fused in the source domain and target domain respectively based on the attention mechanism:

[0106]

[0107] in, is the source domain fusion feature, is the target domain fusion feature, is the channel attention weight of the Doppler branch source domain feature, is the channel attention weight of the source domain feature of the TAC branch, is the channel attention weight of the target domain feature of the Doppler branch, is the channel attention weight of the target domain feature of the TAC branch.

[0108] Fusion of source domain features Fusion features with the target domain Input the classifier to obtain the predicted probability distribution of labeled samples in the source domain and the predicted probability distribution of unlabeled samples in the target domain. Select the prediction results of high-confidence dual-domain samples to calculate the mutual information loss:

[0109] For category k, a target domain sample set with high confidence is established based on the predicted probability distribution of unlabeled samples in the target domain:

[0110] The set of high confidence target domain samples of the Doppler branch is:

[0111]

[0112] The set of high confidence target domain samples of the TAC branch is:

[0113]

[0114] in, Represents the high confidence samples of the Doppler branch target area, T i t is a high confidence sample of the target domain of the TAC branch, is the target domain sample prediction probability distribution of the Doppler branch, is the target domain sample prediction probability distribution of the TAC branch, High confidence samples Confidence level, t p is the confidence threshold.

[0115] For category k, select high-confidence samples from the predicted probability distribution of labeled samples in the source domain and establish a set of source domain samples with high confidence:

[0116] The set of high confidence source domain samples of the Doppler branch is:

[0117]

[0118] The set of high-confidence source domain samples of the TAC branch is:

[0119]

[0120] in, is a high confidence sample in the Doppler branch source domain, T i s is a high confidence sample of the TAC branch source domain, Predict the probability distribution of labeled samples in the source domain of the Doppler branch, is the predicted probability distribution of labeled samples in the source domain of the TAC branch, y i is the source domain sample label.

[0121] To improve the model's class discrimination on unsupervised target domain data, the loss function aims to maximize the mutual information between samples and class predictions, thereby reducing the classifier's prediction uncertainty. This ensures that the model's class distribution in the target domain does not become uniform. Information inherently relates to the correlation between the "global distribution" and the "sample-conditional distribution." By maximizing the mutual information loss, the model learns more discriminative features / prediction patterns that better align with global patterns, ensuring a better fit between the source and target domains in terms of feature distribution and class association.

[0122] The method for calculating mutual information loss is:

[0123]

[0124] in, is a dual-domain high confidence sample x i The average predicted probability distribution, K is the number of categories, j represents the category index, y j is the true category label of the jth category of the source domain data, and the number of high confidence samples in the source domain and the target domain is

[0125] Furthermore, the network’s feature extractor and classifier are trained using the standard cross-entropy loss using source domain labeled data:

[0126]

[0127] in, Is the indicator function, judging whether the condition is met, y i is the label of the i-th source domain sample, n s is the number of source domain samples, c i is the i-th source domain sample.

[0128] The total loss is:

[0129] L FUDA =L cls +λ In L In-domain +λ Cross L Cross-domain +λ MIM L MIM

[0130] Among them, λ In ,λ Cross and λ MIM To control the hyperparameters of the loss ratio. By optimizing the above loss function, the feature extractor can extract domain-independent features, the classifier is more adapted to the target domain data, and the model is more adapted to the feature distribution of the target domain, thus improving the classification performance in small-sample unsupervised cross-domain learning tasks.

[0131] S4: Based on high-confidence dual-domain samples, dual-branch domain prototype fusion is performed, and the weights of the second-layer fully connected layer of the classifier are adaptively updated based on the updated weights of the first-layer fully connected layer of the classifier. Steps S2 to S4 are repeated until the preset training round is reached to obtain the optimal multi-person gait identity recognition model.

[0132] Perform dual-branch domain prototype fusion based on high-confidence dual-domain samples:

[0133] A target domain high-confidence prototype is established based on the target domain memory library and the target domain high-confidence samples, and a source domain high-confidence prototype is established based on the source domain memory library and the source domain high-confidence samples.

[0134] The high confidence prototype of the Doppler branch target domain is:

[0135]

[0136] in, It is the feature vector of the target domain samples stored in the Doppler branch target domain memory bank.

[0137] The high-confidence prototype of the TAC branch target domain is:

[0138]

[0139] in, It is the feature vector of the target domain samples stored in the target domain memory of the TAC branch.

[0140] The high confidence prototype of the Doppler branch source domain is:

[0141]

[0142] in, It is the feature vector of the source domain samples stored in the Doppler branch source domain memory bank.

[0143] The high-confidence prototype of the TAC branch source domain is:

[0144]

[0145] in, It is the feature vector of the source domain samples stored in the source domain memory bank of the TAC branch.

[0146] Furthermore, the channel attention weight is used to fuse the high-confidence prototypes of the Doppler branch and the high-confidence prototypes of the TAC branch of the target domain and the source domain respectively:

[0147]

[0148] in, is the source domain fusion prototype, is the target domain fusion prototype, is the channel attention weight of the high confidence prototype in the Doppler branch source domain, is the channel attention weight of the high confidence prototype in the source domain of the TAC branch, is the channel attention weight of the high confidence prototype in the target domain of the Doppler branch, is the channel attention weight of the high confidence prototype in the target domain of the TAC branch.

[0149] Based on the updated weights of the first fully connected layer of the classifier, the weights of the second fully connected layer of the classifier are adaptively updated: Since the source domain data labels are known, the source domain fusion prototype is More stable, using source domain fusion prototype Perform adaptive update of weights of the second fully connected layer of the classifier. In the later stage of training, after the system performs domain adaptation, the prototype is fused using the target domain. To update the weights of the second fully connected layer of the classifier, the system is more adapted to the target domain task.

[0150]

[0151] Among them, t n is the threshold of the number of high confidence target domain samples, is the weight of the first fully connected layer, LReLU(·) is the Leak ReLU activation function, norm(·) represents the normalization operation, w k is the weight of the second fully connected layer of the updated classifier.

[0152] The technical solution of the embodiment of the present invention proposes a mechanism for improving the environmental generalization ability of a multi-person gait identity recognition model based on small-sample unsupervised domain adaptation. It mainly includes three parts: data acquisition and preprocessing, intra-domain prototype self-supervised learning and inter-domain sample-prototype matching, and dual-branch feature fusion and adaptive network fine-tuning. First, in the data acquisition and preprocessing stage, walking data in the new environment is collected as target domain data, and then the source domain labeled data is used to pre-train the aforementioned research identity recognition model, and the obtained feature extractor parameters are used for subsequent unsupervised domain adaptation. In the intra-domain prototype self-supervised learning and inter-domain sample-prototype matching stages, the pre-trained model is used to extract features of the source domain data and the target domain data on the Doppler branch and the TAC branch respectively, and the semantic information of the data is implicitly encoded into the embedding space. Clustering is then performed according to the feature distribution, and the category prototypes of the two domains are constructed, and the intra-domain prototype self-supervision loss is calculated. Secondly, the system uses clustered prototypes in a single branch of the network to perform inter-domain sample-prototype matching and calculates a cross-domain matching loss. This encodes and aligns semantic structures in a shared embedding space, avoiding the sensitivity of direct sample matching and enabling information transfer from the source domain to the target domain. Finally, in the dual-branch feature fusion and adaptive network fine-tuning stage, features from the two branches are fused in the source and target domains, respectively. Sample confidence is then estimated, and high-confidence samples are used to adaptively update the network classifier parameters. Finally, the model is further fine-tuned using classification task loss and mutual information loss optimization to improve classification performance in the target domain.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for improving the environmental generalization ability of a multi-person gait identity recognition model, characterized in that: The steps are: S1: Collect and preprocess source domain data and target domain small sample data, extract the corrected Doppler spectrum and TAC information from the preprocessed data, pre-train the identity recognition model based on the source domain data and obtain the pre-trained feature extractor; S2: Use the pre-trained feature extractor to extract features from the corrected Doppler spectrum and TAC information respectively, and then construct a dual-branch source-target domain memory library and a dual-branch source-target domain prototype. Based on the dual-branch source-target domain prototype, perform intra-domain prototype self-supervision learning and inter-domain sample-prototype matching to obtain the intra-domain prototype self-supervision loss and inter-domain sample-prototype matching loss respectively; S3: Based on the extracted corrected Doppler spectrum features and TAC information features, dual-branch intra-domain feature fusion is performed. The fused dual-branch intra-domain features are used as input, and the classifier is used to obtain classification prediction results. The prediction results of the dual-domain samples with high confidence are selected to calculate the mutual information loss. The total loss is calculated by combining the intra-domain prototype self-supervision loss and the inter-domain sample-prototype matching loss. The weights of the feature extractor and the weights of the first fully connected layer of the classifier are updated. S4: Based on high-confidence dual-domain samples, dual-branch domain prototype fusion is performed, and the weights of the second fully connected layer of the classifier are adaptively updated based on the updated weights of the first fully connected layer of the classifier. Steps S2 to S4 are repeated until the optimal multi-person gait identity recognition model is obtained.

2. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 1, characterized in that: The method for constructing a dual-branch source domain and target domain memory library is as follows: In the Doppler branch and TAC branch, the pre-trained feature extractor is used to extract the source domain features F * (* s ) and target domain features F * (* t ), according to the source domain feature F * (* s ) and target domain features F * (* t ) respectively build source domain memory and target domain memory in, represents the feature vector of the stored source domain samples, Represents the feature vector of the stored target domain sample, n s and n t are the number of source domain samples and the number of target domain samples respectively; among them, From the source domain feature F * (* s ) Flattened source domain feature vector The memory momentum β is used to update each batch of training: By the target domain feature F * (* t ) The flattened target domain feature vector The memory momentum β is used to update each batch of training: Where * represents the Doppler branch D or TAC branch T.

3. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 2, characterized in that: The method for constructing a dual-branch source domain and target domain prototype is as follows: In the Doppler branch and TAC branch, the source domain memory and target domain memory Perform N K-Means clustering, and obtain the source domain cluster in each K-Means clustering and the target domain cluster represents the jth cluster in the source domain cluster, K is the total number of clusters, represents the jth cluster in the target domain cluster; according to the source domain cluster and the target domain cluster Get the source domain prototypes respectively and target domain prototype The acquisition method is: in, represents the prototype of the jth cluster in the source domain cluster or the target domain cluster, represents the mean of all feature vectors belonging to the jth cluster in the source or target domain, ||·|| represents the norm operation, Represents the feature vector within the j-th source domain cluster or target domain cluster The number of 4. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 3, characterized in that: The method for performing self-supervised learning of prototypes in the domain is as follows: the source domain feature vector is calculated in the Doppler branch and the TAC branch. and each source domain prototype Similarity distribution between Using the target domain feature vector and each source domain prototype Similarity distribution between The cross entropy loss is used to calculate the in-domain prototype self-supervision loss L of the Doppler branch respectively In-doppler and the in-domain prototype self-supervision loss of the TAC branch and L In-TAC , the calculation method is:

5. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 3 or 4, characterized in that: The method for performing inter-domain sample-prototype matching is as follows: in the Doppler branch and the TAC branch, for the source domain sample feature vector and target domain prototype Calculate the similarity distribution: in, θ is the temperature hyperparameter that controls the smoothness of the distribution; According to the target domain sample feature vector and source domain prototype Calculate the similarity distribution: in, Calculate the entropy of the similarity distribution: The inter-domain sample-prototype matching loss L of the Doppler branch is calculated based on the entropy of the similarity distribution. Cross-domain-doppler and the inter-domain sample-prototype matching loss L of the TAC branch Cross-domain-TAC , the calculation method is:

6. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to any one of claims 2 to 4, characterized in that: The method for performing dual-branch feature fusion in step S3 is: fusing the features extracted from the Doppler branch and the TAC branch in the source domain and the target domain respectively based on the attention mechanism: in, is the source domain fusion feature, is the target domain fusion feature, is the channel attention weight of the Doppler branch source domain feature, is the channel attention weight of the source domain feature of the TAC branch, is the channel attention weight of the target domain feature of the Doppler branch, is the channel attention weight of the target domain feature of the TAC branch.

7. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 6, characterized in that: The method of using the classifier to obtain the classification prediction result and selecting the high confidence dual domain sample prediction result to calculate the mutual information loss is: fusion feature of the source domain Fusion features with the target domain Input the classifier and obtain the predicted probability distribution of source domain labeled samples and target domain unlabeled samples; for category k, based on the predicted probability distribution of target domain unlabeled samples, t p Establish a target domain sample set with high confidence for the confidence threshold, including a Doppler branch high confidence target domain sample set and TAC branch high confidence target domain sample set For category k, select high-confidence samples with sample labels corresponding to category k from the predicted probability distribution of labeled samples in the source domain and establish a source domain sample set with high confidence, including a Doppler branch high-confidence source domain sample set and TAC branch high confidence source domain sample set The mutual information loss is calculated based on the predicted probability distribution of high-confidence target domain samples and the predicted probability distribution of high-confidence source domain samples: in, is a dual-domain high confidence sample x i The average predicted probability distribution, K is the number of categories, j represents the category index, y j is the true category label of the jth category of the source domain data, is the number of high confidence samples in the dual domains.

8. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 5 or 7, characterized in that: The total loss is: L FUDA =L cls +λ In L In-domain +λ Cross L Cross-domain +λ MIM L MIM Among them, L cls is the cross entropy loss, L In-domain =L In-doppler +L In-TAC is the total loss of prototype self-supervision in the domain, L Cross-domain =L Cross-domain-doppler +L Cross-domain-TAC is the total loss of inter-domain sample-prototype matching, λ In ,λ Cross and λ MIM They are all hyperparameters that control and adjust the proportion of each loss.

9. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 7, characterized in that: The method for performing dual-branch domain prototype fusion based on high-confidence dual-domain samples is as follows: establishing a target domain high-confidence prototype based on a target domain memory library and target domain high-confidence samples, and establishing a source domain high-confidence prototype based on a source domain memory library and source domain high-confidence samples; using channel attention weights to fuse the Doppler branch high-confidence prototype and TAC branch high-confidence prototype of the target domain and source domain respectively; and adaptively updating the weights of the second fully connected layer of the classifier based on the target domain fusion prototype and the source domain fusion prototype and the updated weights of the first fully connected layer of the classifier.

10. The method for improving the environmental generalization capability of a multi-person gait identity recognition model according to claim 9, characterized in that: The method for obtaining the high-confidence prototype of the target domain is: The high confidence prototype of the Doppler branch target domain is: in, The feature vector of the target domain samples stored in the target domain memory bank of the Doppler branch; The high-confidence prototype of the TAC branch target domain is: in, The feature vector of the target domain sample stored in the target domain memory bank of the TAC branch; The method for obtaining the source domain high confidence prototype is: The high confidence prototype of the Doppler branch source domain is: in, The feature vector of the source domain samples stored in the source domain memory bank of the Doppler branch; The high-confidence prototype of the TAC branch source domain is: in, The feature vector of the source domain sample stored in the source domain memory bank of the TAC branch; The method for adaptively updating the weights of the second fully connected layer of the classifier is: Among them, t n is the threshold of the number of high confidence target domain samples, is the weight of the first fully connected layer, LReLU() is the Leak ReLU activation function, norm() represents the normalization operation, w k is the weight of the second fully connected layer of the updated classifier.

Images