A Cross-Domain Human Action Recognition Method and System Based on Feature Distribution Similarity

Abstract

This invention discloses a cross-domain human action recognition method and system based on feature distribution similarity. The method includes the following steps: S01. Inputting the signal to be recognized and performing data preprocessing to calculate the corresponding micro-Doppler spectrum; S02. Using the micro-Doppler spectrum as input data for unsupervised contrastive learning to extract invariant features. During unsupervised contrastive learning, the input data is augmented twice and then assigned to two feature encoders with the same structure; S03. Inputting the micro-Doppler spectrum into a teacher feature encoder, a student feature encoder, and two classifiers for cross-domain transfer training. The teacher feature encoder learns from step S02, and during the transfer training process, the teacher feature encoder transmits invariant features to the student feature encoder. This invention can achieve high accuracy and purity in cross-domain human action recognition.

Description

Technical Field

[0001] This invention relates to the field of human motion recognition technology based on radio frequency signals, and in particular to a cross-domain human motion recognition method and system based on feature distribution similarity. Background Technology

[0002] Human motion recognition involves identifying small or large limb movements of the human body. Current technologies typically employ computer vision, which involves acquiring human images, processing them, and then identifying the type of human movement. However, this type of computer vision processing method has drawbacks such as susceptibility to environmental interference, poor environmental adaptability, and inability to recognize movements when occluded.

[0003] Benefiting from its contactless and privacy-preserving characteristics, human activity recognition based on wireless reflection signals has received widespread attention in recent years. While using wireless reflection signals for human action recognition can solve the problems mentioned above in computer vision methods, existing methods are typically implemented in fixed environments, meaning they can only classify actions within specific conditions. In actual deployment, the sensing subject or environment may be dynamically changing, leading to a decline in the model's recognition performance. Summary of the Invention

[0004] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a cross-domain human action recognition method and system based on feature distribution similarity that is simple to implement, has high accuracy and purity, and strong cross-domain recognition capability.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A cross-domain human action recognition method based on feature distribution similarity, comprising the following steps:

[0007] Step S01. Data preprocessing: Input the signal to be identified and perform data preprocessing to calculate the corresponding micro-Doppler spectrum.

[0008] Step S02. Invariant Feature Extraction: The micro-Doppler spectrogram is used as input data for unsupervised contrastive learning to extract invariant features. During unsupervised contrastive learning, the input data is augmented twice and then assigned to two feature encoders F with the same structure. q and F l For the feature encoder F q Optimization is performed using the Info-NCE loss function as the objective, and another feature encoder F... l The weights are updated using momentum.

[0009] Step S03. Cross-domain transfer based on distribution similarity: The micro-Doppler spectrograms are input into a teacher feature encoder Fe. T And a student feature encoder Fe S And cross-domain transfer training is performed between two classifiers, with each feature encoder followed by one of the classifiers, the teacher feature encoder Fe T The feature encoder Fe, learned from step S02, is the teacher feature encoder used during the transfer training process. T To the student feature encoder Fe S The invariant features of the target domain are passed on, and the final recognition result is obtained after training.

[0010] Furthermore, step S01 includes:

[0011] S101. Obtain the fast-slow time matrix of the input signal to be identified, and subtract the mean of the corresponding slow time dimension from each item of the matrix to eliminate background clutter;

[0012] S102. Perform a Fast Fourier Transform on the filtered data from step S101, and then filter it along the slow time axis using a Butterworth filter.

[0013] S103. Perform a short-time Fourier transform on the filtered data from step S102 to extract the corresponding micro-Doppler spectrum.

[0014] Furthermore, step S02 includes:

[0015] S201. A series of positive sample pairs are obtained by processing the original input data with one or more of the following methods: random cropping, random flipping, and adding Gaussian noise.

[0016] S202. Input the positive sample pairs obtained in step S201 into the feature encoder F respectively. q And momentum feature encoder F l among;

[0017] S203. After each positive sample pair passes through the corresponding feature encoder, the two sets of features from the same batch are treated as positive samples by the automatic feature encoder F. l The features of different batches are used as negative samples, and the Info-NCE loss is calculated.

[0018] S204. Based on the calculated Info-NCE loss, construct the optimization objective for the invariant feature extraction module, namely:

[0019] L q For the Info-NCE loss;

[0020] S205. After pre-training the backbone network, a feature encoder F with preliminary classification capabilities is obtained. q .

[0021] Furthermore, the feature encoder F q The momentum feature encoder F is updated using gradient backpropagation. l Momentum is updated according to the following formula:

[0022] θ l ←mθ l +(1-m)θ q ,

[0023] Where, θ q Represents the feature encoder F q The parameter, θ l F represents the momentum feature encoder l The parameter is m, which is the momentum coefficient hyperparameter.

[0024] Furthermore, in step S203, the Info-NCE loss is calculated according to the following formula:

[0025]

[0026] Among them, z q F represents q The feature encoding given by the encoder, z l This represents F obtained from queue Q. l The encoder provides the feature encoding, and the queue Q maintains a negative sample queue according to the first-in-first-out principle. τ is the temperature hyperparameter.

[0027] Furthermore, step S03 includes:

[0028] S301. Semantic Information Recognition: For source domain data with existing labels, the entire network is trained in a supervised manner using labels to enable the model to recognize the semantic information of different features; for target domain data without labels, an unsupervised contrastive learning method is used to pre-train the teacher feature encoder Fe. T As a teacher model and student feature encoder, Fe S As a student model; then using the teacher feature encoder Fe T The process of guiding the consistency of decision results in the target domain is carried out by using the teacher model's ability to extract invariant features and implicitly transmitting the distribution information of target domain features to the student model through knowledge distillation.

[0029] S302. Maximize Decision Difference: Maximize the decision difference between the two classifiers to determine the decision boundary of the classifiers;

[0030] S303. Decision Consistency: By optimizing the teacher feature encoder Fe T This ensures that the decisions of the two classifiers are consistent, enabling clustering of features from different domains.

[0031] S304. Move towards the center point: Pull the target domain features closer to the center of the source domain.

[0032] Furthermore, in step S301, the similarity matrix is ​​first calculated using the target domain feature vector z. Then the Softmax function is used along the similarity matrix. For each row, calculate the similarity probability of the two features. The similarity matrix of the target domain features calculated by the teacher model is used as the distribution information of the target domain features. By comparing the distance between the distributions of the feature similarity matrices of the teacher model and the student model, the distribution distance loss L2 is calculated, thereby guiding the student model to implicitly learn the distribution features of the target domain.

[0033] Furthermore, maximizing the classifier decision difference in step S302 includes: maximizing the L1 norm of the decision vectors output by the two classifiers, and setting constraints to construct the current final optimization objective as follows: Where, Θ C1 Let C1, Θ be the training classifier. C2 Let C2 represent the training classifier, L2 represent the distribution distance loss, L1 represent the extracted semantic information, and L3 represent the result of maximizing the classifier decision difference.

[0034] Furthermore, step S304 includes:

[0035] Let k represent the labels for different actions. This represents a typical action from a source domain labeled k. For unlabeled target domain features, the label of the nearest cluster center is used as its label estimate.

[0036]

[0037] After obtaining the label estimates for the target domain features, the target domain features are made to converge towards the source domain cluster centers with the same labels by minimizing the distance between the target domain features and the corresponding cluster centers:

[0038]

[0039] The objective constraint is:

[0040]

[0041] A cross-domain human action recognition system based on feature distribution similarity, the cross-domain human action recognition system comprising:

[0042] The data preprocessing module is used to input the signal to be identified, perform data preprocessing, and calculate the corresponding micro-Doppler spectrum.

[0043] The invariant feature extraction module is used to extract invariant features by performing unsupervised contrastive learning on the micro-Doppler spectrogram as input data. During unsupervised contrastive learning, the input data is augmented twice and then assigned to two feature encoders F with identical structures. q and F l For the feature encoder F q Optimization is performed using the Info-NCE loss function as the objective, and another feature encoder F... l The weights are updated using momentum.

[0044] A cross-domain transfer module based on distribution similarity is used to input the micro-Doppler spectrograms into a teacher feature encoder Fe. T And a student feature encoder Fe S And cross-domain transfer training is performed between two classifiers, with each feature encoder followed by one of the classifiers, the teacher feature encoder Fe T The feature encoder Fe, learned from step S02, is the teacher feature encoder used during the transfer training process. T To the student feature encoder Fe S The invariant features of the target domain are passed on, and the final recognition result is obtained after training.

[0045] Alternatively, the cross-domain human motion recognition system may include a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method described above.

[0046] Compared with existing technologies, the advantages of this invention are as follows: This invention first extracts invariant features from the micro-Doppler spectrogram through unsupervised contrastive learning, and then inputs the micro-Doppler spectrogram into two feature encoders and two classifiers for cross-domain transfer training. During the transfer training process, the teacher feature encoder transmits the invariant features of the target domain to the student feature encoder. By using knowledge distillation, the distribution information of the invariant features of the target domain can be implicitly passed to the clustering process, effectively reducing the occurrence of incorrect clustering and improving cross-domain recognition ability. It can also utilize labeled or unlabeled data generated during deployment to train in an unsupervised, semi-supervised, or supervised manner, thereby continuously improving the model's adaptability to new environments. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the principle and flow of the cross-domain human action recognition method based on feature distribution similarity in this embodiment.

[0048] Figure 2 This is a schematic diagram of the unsupervised learning distribution results obtained in a specific application embodiment.

[0049] Figure 3 This is a schematic diagram of the process for implementing unsupervised feature learning in this embodiment.

[0050] Figure 4 This is a schematic diagram of the GeCro cross-domain learning process in this embodiment.

[0051] Figure 5 This is a schematic diagram illustrating the recognition effect under different subjects obtained in a specific application embodiment of the present invention.

[0052] Figure 6 This is a schematic diagram illustrating the recognition effect under different environments obtained in specific application embodiments of the present invention.

[0053] Figure 7 A schematic diagram illustrating the recognition effect under domain-free information obtained in a specific application embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0055] like Figure 1 As shown, the steps of the cross-domain human action recognition method based on feature distribution similarity in this embodiment include:

[0056] Step S01. Data preprocessing: Input the signal to be identified and perform data preprocessing to calculate the corresponding micro-Doppler spectrum.

[0057] First, for the original fast-time-slow-time matrix, subtract the mean of its slow-time dimension from each element to eliminate background clutter. Then, perform a Fast Fourier Transform on the data and filter it using a Butterworth filter along the slow-time axis. Next, perform a Short-Time Fourier Transform to extract the micro-Doppler spectrum. Finally, use the normalized micro-Doppler spectrum as input to the model. The detailed steps are as follows:

[0058] S101. Obtain the fast-slow time matrix of the input signal to be identified, and subtract the mean of the corresponding slow time dimension from each item of the matrix to eliminate background clutter;

[0059] S102. Perform a Fast Fourier Transform on the filtered data from step S101, and then filter it along the slow time axis using a Butterworth filter.

[0060] S103. Perform a short-time Fourier transform on the filtered data from step S102 to extract the corresponding micro-Doppler spectrum.

[0061] Step S02. Invariant Feature Extraction: Using the micro-Doppler spectrogram as input data, unsupervised contrastive learning is performed to extract invariant features. During unsupervised contrastive learning, the input data is augmented twice and then assigned to two feature encoders F with the same structure. q and F l For the feature encoder F q Optimization is performed using the Info-NCE loss function as the objective, and another feature encoder F... l The weights are updated in terms of momentum.

[0062] Compared to supervised learning, unsupervised learning can automatically learn relevant features in the Doppler spectrogram, enabling similar actions to extract similar features. Figure 2 This refers to the feature distribution map extracted using unsupervised learning, where similar features are distributed relatively closely, while dissimilar features differ significantly. Based on this characteristic, this embodiment uses unsupervised learning to pre-train a feature encoder, enabling it to automatically learn invariant features of the input data. This reduces the distance between features extracted from similar actions and increases the distance between features extracted from dissimilar actions.

[0063] like Figure 3 As shown, the steps for extracting invariant features using unsupervised contrastive learning in this embodiment include:

[0064] S201. After processing the original input data with random cropping, random flipping, and adding Gaussian noise, a series of positive sample pairs are obtained.

[0065] S202. Input the positive sample pairs obtained in step S201 into the feature encoder F respectively. q And momentum feature encoder F l Among them, the feature encoder F a The momentum feature encoder F is updated using gradient backpropagation. l Momentum is updated according to the following formula:

[0066] θ l ←mθ l +(1-m)θ q (1)

[0067] Where, θ q Represents the feature encoder F q The parameter, θ l F represents the momentum feature encoder lThe parameter is m, which is the momentum coefficient hyperparameter.

[0068] S203. After each positive sample pair passes through the corresponding feature encoder, the two sets of features from the same batch are treated as positive samples, and the features from different batches of the automatic feature encoder F1 are treated as negative samples. The Info-NCE loss is calculated, and the expression for the Info-NCE loss is as follows:

[0069]

[0070] Among them, z q F represents q The feature encoding given by the encoder, z l This represents F obtained from queue Q. l The encoder provides the feature encoding, and the queue Q maintains a negative sample queue according to the first-in-first-out principle. τ is the temperature hyperparameter.

[0071] S204. Based on the calculated Info-NCE loss, construct the optimization objective for the invariant feature extraction module, namely:

[0072]

[0073] Among them, L q This is for Info-NCE loss.

[0074] S205. After pre-training the backbone network, a feature encoder F with preliminary classification capabilities is obtained. q The feature encoder F q That is, invariant features can be extracted.

[0075] Step S03. Cross-domain transfer based on distribution similarity: Input the micro-Doppler spectrograms into a teacher feature encoder Fe. T And a student feature encoder Fe S Cross-domain transfer training is performed using two classifiers, C1 and C2, with each feature encoder followed by a classifier (i.e., the teacher feature encoder Fe). T Followed by classifier C1, student feature encoder Fe S Following classifier C2), teacher feature encoder Fe T The teacher feature encoder Fe is learned from step S02. T It possesses target domain invariance features, and during the transfer training process, it is controlled by the teacher's feature encoder Fe. T To the student feature encoder Fe S The invariant features of the target domain are passed on, and the final recognition result is obtained after training.

[0076] In this embodiment, the entire transfer learning process can be divided into four steps: 1) identifying semantic information; 2) maximizing decision differences; 3) decision consistency; and 4) moving towards the cluster center. Step 1) guides the model to learn the differences between different actions in the source and target domains, determining the cluster center. Steps 2) and 3) are two opposing steps. Through this opposing minimization mechanism, similar features located near the cluster center are moved into the decision boundary, thus achieving unsupervised feature clustering. Finally, to improve the model's generalization ability, step 4) moves the target domain features to the cluster center. The specific processes of each step are as follows:

[0077] S301. Identifying Semantic Information: To perform cross-domain transfer, it is first necessary to identify the differences between different actions. For source domain data with existing labels, the entire network is trained in a supervised manner using labels to enable the model to identify the semantic information of different features. For labeled source domain data, semantic information can be extracted using formula (4):

[0078]

[0079] Where, x s y represents Doppler spectral data from the source domain. s For the corresponding label, K is the number of label categories.

[0080] For unlabeled target domain data, unsupervised contrastive learning is used to pre-train the teacher feature encoder Fe. T As a teacher model and student feature encoder, Fe S As a student model; then using the teacher feature encoder Fe T The process of guiding the consistency of decision results in the target domain is achieved by leveraging the teacher model's ability to extract invariant features and implicitly transmitting the distribution information of target domain features to the student model through knowledge distillation.

[0081] Traditional clustering models typically rely on the assumption that similar features are located close to each other. However, due to the unique nature of cross-domain problems, features of the same semantic actions from different domains are not necessarily similar. Furthermore, feature unification processes often only consider how to bring features closer to the cluster centers, without considering whether the features brought closer to the decision boundary are consistent. This can lead to erroneous unification of features across different actions during clustering, resulting in decreased model recognition accuracy. To address this issue, this embodiment uses unsupervised knowledge distillation to reduce erroneous clustering of actions, thereby improving the model's cross-domain transferability.

[0082] Unsupervised contrastive learning can effectively extract invariant features. This embodiment uses unsupervised contrastive learning to pre-train a feature encoder Fe. T As a Teacher model; subsequently, Fe T The process guides the consistency of decision results in the target domain, so that the target domain features that are mistakenly pulled into the decision boundary have an external force to move towards similar features in other target domains, and ultimately allow these multiple similar features to enter the correct decision boundary as a whole.

[0083] Specifically, the similarity matrix is ​​first calculated using the target domain feature vector z. Then the Softmax function is used along the similarity matrix. For each row, calculate the similarity probability of the two features. The similarity matrix of the target domain features calculated by the teacher model is used as the distribution information of the target domain features. By comparing the distance between the distributions of the feature similarity matrices of the teacher model and the student model, the distribution distance loss L2 is calculated, thereby guiding the student model to implicitly learn the distribution features of the target domain. The calculation formula is as follows:

[0084]

[0085]

[0086]

[0087] set up Represents the training feature encoder Fe S Θ C1 Let C1, Θ be the training classifier. C2 Let C2 be the classifier being trained. Then, for the first stage, the overall optimization objective can be expressed as formula (8):

[0088]

[0089] S302. Maximize Decision Difference: Maximize the decision difference between the two classifiers to determine the decision boundary of the classifier.

[0090] To determine the decision boundaries of the classifiers, this embodiment specifically ensures that the decision boundaries of the two classifiers tightly surround the source domain. This requires maximizing the decision difference between the two classifiers, which can be achieved by maximizing the L1 norm of the decision vectors output by the two classifiers. That is:

[0091]

[0092] To prevent the decision boundary from changing uncontrollably, constraints need to be added to maximize the decision difference between the two classifiers while ensuring accuracy. In this embodiment, the final optimization objective for step S302 can be expressed as:

[0093]

[0094] S303. Decision Consistency: By optimizing the feature encoder Fe S This ensures that the decisions of the two classifiers are consistent, enabling clustering of features from different domains.

[0095] Contrary to the optimization objective of step S302, step S303 optimizes the feature encoder Fe. S This ensures that the decisions of the two classifiers are consistent, thereby bringing target domain features located outside the classifiers into the classifier decision boundary, achieving clustering of features from different domains. The optimization objective of step S303 in this embodiment can be expressed as minimizing the L3 loss function, i.e.:

[0096]

[0097] S304. Move towards the center point: Pull the target domain features closer to the center of the source domain.

[0098] Although the methods in steps S301-S303 can pull the target domain features into the source domain, the differences between the source and target domains mean that the pulled target domain features may remain near the decision boundary. Features in the boundary region may then exceed the decision boundary range due to random noise, leading to incorrect classification of these features. This embodiment further pulls the target domain features closer to the center of the source domain, reducing misclassification caused by random noise and further improving classification accuracy.

[0099] Let k represent the labels for different actions. This represents a typical action from the source domain labeled k. For unlabeled target domain features, the label k of their nearest cluster center is used as their label estimate.

[0100]

[0101] After obtaining the label estimates for the target domain features, the target domain features are made to converge towards the source domain cluster centers with the same labels by minimizing the distance between the target domain features and the corresponding cluster centers:

[0102]

[0103] In summary, the constraint objective for step S304 can be obtained as follows:

[0104]

[0105] By continuously repeating steps S301-S304, the target domain action will gradually move closer to the cluster center of the source domain, and eventually make the target domain action located near the cluster center of the corresponding source domain action.

[0106] like Figure 1 As shown, this embodiment constructs a general cross-domain learning model, GeCro, which can improve the accuracy of cross-scene human activity recognition using labeled or unlabeled data. The GeCro cross-domain learning model specifically includes three main modules: a data preprocessing module, an invariant feature extraction module, and a cross-domain transfer based on distribution similarity, corresponding to steps S01 to S03 above. Each module is as follows:

[0107] (1) Data Preprocessing Module. In the data preprocessing module, the micro-Doppler spectrum is calculated as the input to the model. To obtain the micro-Doppler spectrum, firstly, for the original fast-time-slow-time matrix, the mean of its slow-time dimension is subtracted from each element of the matrix to eliminate background clutter. Subsequently, a Fast Fourier Transform is performed on the data, and a Butterworth filter is used along the slow-time axis for filtering. Then, a Short-Time Fourier Transform is performed on the data to extract the micro-Doppler spectrum. Finally, the normalized micro-Doppler spectrum is used as the input to the model.

[0108] (2) Invariant Feature Extraction Module. An invariant feature extraction module based on contrastive learning is designed. First, the model's input data is augmented twice to obtain two sets of input data, which are then assigned to two feature encoders with identical structures. Subsequently, one feature encoder is optimized using the Info-NCE loss function, while the weights of the other feature encoder are updated using momentum. This results in a single feature encoder capable of extracting invariant features from the input data. Simultaneously, the distance between features of similar actions is reduced, while the distance between features of dissimilar actions is increased.

[0109] (3) Cross-domain transfer module based on distribution similarity. In this module, the transfer module consists of two feature encoders Fe s and Fe T It consists of two classifiers, C1 and C2, where Fe T The teacher feature encoder learned from step S02 will be transferred to the student feature encoder Fe during the transfer training process. S It transmits invariant feature information of the target domain. During training, in the feature encoder Fe... SA Min-Max game and multiple adversarial mechanisms were designed between the feature encoder and the classifier. Through these adversarial mechanisms, I can improve the feature encoder's performance. S Learn the common features of the same action in different domains, and make the decision boundary of the classifier closely surround these features.

[0110] This embodiment utilizes the aforementioned general cross-domain learning framework GeCro to train the model in unsupervised, semi-supervised, or supervised manner using labeled or unlabeled data generated during deployment, thereby continuously improving the model's adaptability to new environments. Unlike traditional nearest-neighbor clustering methods, this embodiment uses knowledge distillation to implicitly pass the invariant feature distribution information of the target domain to the clustering process, effectively reducing erroneous clustering and improving cross-domain recognition capabilities. GeCro can also fully utilize both unlabeled and labeled data to continuously enhance the model's cross-domain recognition ability.

[0111] To verify the effectiveness of this invention, experiments were conducted on its method. Specifically, a Novdale X4M05 commercial UWB radar was used to collect the necessary data. The radar operates at a frequency of 7.3 or 8.7 GHz, with a bandwidth of 1.4 GHz and a frame rate of 48 frames per second. It has a pair of transmit-receive antennas with a FoV of 65° in both azimuth and elevation. Experimental data was collected from 16,000 actions across 5 different subjects in 7 different environments, including 11 human-environment pairings. The 5 subjects exhibited significant differences, ranging in height from 1.55 meters to 1.8 meters and weight from 45 kg to 75 kg. Furthermore, there were differences in the subjects' movement habits (e.g., some people habitually drink water with their left hand, while others drink with their right; these differences significantly impacted the reflected signal). The experimental environments included a living room, study, bedroom, laboratory, dormitory, and two locations within a classroom. During the experiment, the duration of the motion detection segment was fixed at 5 seconds. The detected motion types were divided into two categories: small-amplitude limb movements and large-amplitude limb movements. Specifically, it included eight common daily actions: writing, using a mobile phone, typing, turning pages, drinking water, raising a hand, leaning on a table, and standing / sitting. The detection range for human motion was approximately 1 to 1.5 meters around the radar.

[0112] Recognition performance under different subjects, such as Figure 5 As shown, where Figure 5Figures (a) and (b) respectively reflect the changes in accuracy and purity of each model with the sample size. Overall, as the sample size in the target domain increases, the GeCro model is able to continuously learn more class difference information from the target domain samples, improving the accuracy of cross-domain recognition. Secondly, on average, the GeCro-U model has an overall average accuracy of 84.38%, only lower than the GeCro-SS model (using semi-supervised learning) and the GeCro-S model (using supervised learning). Furthermore, the GeCro-U model achieves an average purity of 83.71%, with a difference of only 0.67% from the average accuracy, indicating that the clustering effect of the GeCro model in this invention is good.

[0113] Besides changes in the target individual affecting the wireless reflection signal, environmental changes also influence the received wireless reflection signal. The main reasons for decreased recognition performance fall into two categories: 1) Changes in the position of the wireless receiving / transmitting equipment. When the relative position between the equipment and the individual changes, the detection angle may change, leading to differences in the wireless signal ultimately received by the equipment. 2) Changes in the static environment surrounding the individual. Besides being directly reflected to the receiving equipment, the wireless signal reflected by a human body may also collide with a series of objects along the reflection path and be reflected again before being received by the receiving equipment; this phenomenon is known as multipath propagation. In indoor human motion recognition, the multipath propagation effect is exacerbated by the confined space and cluttered objects. Therefore, to verify the model's cross-domain learning ability under environmental changes, each model was trained across three different target domains, and the average performance of the three cases under different sample sizes was compared as follows: Figure 6 As shown. Experimental results show that environmental changes have little impact on the recognition performance of the GeCro model of this invention. Even with only 4 unlabeled samples per class, the GeCro-U model can still achieve an average accuracy of 95.21% and an average purity of 94.17%. Moreover, as the number of samples per class increases to 64, the average accuracy and average purity can further increase to 97.71% and 97.08%, respectively.

[0114] Domain-free information refers to situations where the individual performing the action in the target domain and the environmental state are different from those in the source domain. In this case, the model is completely unaware of the information in the target domain. Cross-domain learning is most challenging in this scenario. To verify the model's cross-domain learning ability in the absence of domain information, the model was trained across three different target domains. The average accuracy results are as follows: Figure 7 As shown in (a), the average purity results are as follows: Figure 7As shown in (b). Experiments show that even under conditions of no domain information, as the sample size of the target domain increases, the GeCro model of this invention can still continuously learn the differential information of different categories, reasonably cluster different actions, and improve the accuracy of human activity recognition.

[0115] In summary, under unsupervised cross-domain conditions, the GeCro model of this invention improves accuracy and purity by 7.46% and 10.99% respectively compared to existing models. Under supervised cross-domain conditions, the unsupervised GeCro model improves accuracy and purity by 2.28% and 6.49% respectively compared to state-of-the-art models, while the supervised GeCro model improves accuracy and purity by 5.01% and 8.96% respectively compared to state-of-the-art models. This invention effectively improves recognition accuracy and purity under both unsupervised and supervised cross-domain conditions.

[0116] This embodiment describes a cross-domain human motion recognition system based on feature distribution similarity. The cross-domain human motion recognition system includes:

[0117] The data preprocessing module is used to input the signal to be identified, perform data preprocessing, and calculate the corresponding micro-Doppler spectrum.

[0118] The invariant feature extraction module is used to extract invariant features by performing unsupervised contrastive learning on the micro-Doppler spectrogram as input data. During unsupervised contrastive learning, the input data is augmented twice and then assigned to two feature encoders F with identical structures. q and F l For the feature encoder F q Optimization is performed using the Info-NCE loss function as the objective, and another feature encoder F... l The weights are updated using momentum.

[0119] A cross-domain transfer module based on distribution similarity is used to input the micro-Doppler spectrograms into a teacher feature encoder Fe. T And a student feature encoder Fe S And cross-domain transfer training is performed between two classifiers, with each feature encoder followed by one of the classifiers, the teacher feature encoder Fe T The feature encoder Fe, learned from step S02, is the teacher feature encoder used during the transfer training process. T To the student feature encoder Fe S The invariant features of the target domain are passed on, and the final recognition result is obtained after training.

[0120] The cross-domain human motion recognition system based on feature distribution similarity in this embodiment corresponds one-to-one with the cross-domain human motion recognition method based on feature distribution similarity described above, and will not be described in detail here.

[0121] In another embodiment, the cross-domain human motion recognition system based on feature distribution similarity may further include a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the cross-domain human motion recognition method as described above.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A cross-domain human action recognition method based on feature distribution similarity, characterized in that the steps of Comprise: Step S01. Data preprocessing: input the signal to be identified for data preprocessing, and calculate the corresponding micro-Doppler spectrum diagram; Step S02. Invariance feature extraction: unsupervised contrastive learning is performed on the micro-Doppler spectrogram as input data to extract invariance features, wherein the unsupervised contrastive learning is performed by assigning the input data to two feature encoders with the same structure after using twice data augmentation on the input data and , wherein the feature encoders are optimized by taking the Info-NCE loss function as the objective, and the weights of the other feature encoders are updated in a momentum manner; Step S03. Cross-domain migration based on distribution similarity: inputting the micro-Doppler spectrum into a teacher feature encoder and a student feature encoder respectively for cross-domain migration training and two classifiers, each feature encoder being connected with one of the classifiers, the teacher feature encoder being learned from step S02, transferring the invariance features of the target domain to the student feature encoder during the migration training process, and obtaining the final recognition result after the training is completed ​​ The step S03 comprises: S301. Recognize semantic information: For the source domain data with labels, train the whole network supervised by the labels to let the model recognize the semantic information of different features; for the target domain data without labels, pre-train the teacher feature encoder using unsupervised contrastive learning as a teacher model, a student feature encoder as a student model; then use the teacher feature encoder guide the decision result unification process of the target domain, so as to implicitly pass the distribution information of the target domain features to the student model through the way of knowledge distillation by means of the ability of the teacher model for extracting invariant features; S302. Maximize decision difference: maximize the decision difference of two classifiers to determine the decision boundary of the classifier; S303. Decision Consistency: By optimizing the teacher feature encoder to make the decisions of the two classifiers consistent, enabling clustering of features from different domains; S304. Move to the center point: pull the target domain feature into the vicinity of the source domain center. 2.The cross-domain human action recognition method based on feature distribution similarity according to claim 1, wherein, The step S01 comprises: S101. Obtain the fast-time-slow-time matrix of the input signal to be identified, and subtract the mean value of the corresponding slow-time dimension from each item of the matrix to eliminate background clutter; S102. Perform fast Fourier transform on the filtered data of step S101, and filter along the slow-time axis using a Butterworth filter; S103. Perform short-time Fourier transform on the filtered data of step S102, and extract the corresponding micro-Doppler spectrum diagram. 3.The cross-domain human action recognition method based on feature distribution similarity according to claim 1, wherein, Step S02 comprises: S201. A series of positive sample pairs are obtained by using any one or more of random cropping, random flipping, and adding Gaussian noise on the original input data; S202. input the positive sample pair obtained in step S201 into the feature encoder and the momentum feature encoder among them; S203. After passing through the corresponding feature encoders, each positive sample pair takes two groups of features from the same batch as positive samples from the different batches of features as negative samples, and calculates the Info-NCE loss. S203. After passing through the corresponding feature encoders, each positive sample pair takes two groups of features from the same batch as positive samples from the different batches of features as negative samples, and calculates the Info-NCE loss. S204. According to the calculated Info-NCE loss, the optimization objective of the invariance feature extraction module is constructed, that is: , is the Info-NCE loss; S205. After pre-training the backbone network, a feature encoder with preliminary classification ability is obtained . 4.The cross-domain human action recognition method based on feature distribution similarity according to claim 3, characterized in that, The feature encoder updating using a gradient backpropagation approach, the momentum feature encoder updating the momentum according to the following equation: , wherein, denote parameters of the feature encoder , denote parameters of the momentum feature encoder , is a momentum coefficient hyperparameter. 5.The cross-domain human action recognition method based on feature distribution similarity according to claim 3, wherein, The Info-NCE loss is calculated according to the following formula in step S203: , wherein, denotes a feature encoding given by an encoder, denotes a feature encoding given by an encoder, acquired from a queue denotes a feature encoding given by an encoder, queue maintains a queue of negative samples according to a first-in first-out principle, is a temperature hyperparameter. 6.The cross-domain human action recognition method based on feature distribution similarity according to claim 1, wherein, In the step S301, the target domain feature vector is first used The similarity matrix is calculated Then, the Softmax function is used to calculate the similarity probability of two features along each row of the similarity matrix The similarity matrix of the target domain features calculated by the teacher model is used as the distribution information of the target domain features, and the distribution distance loss is calculated by comparing the distance between the distribution of the feature similarity matrix of the teacher model and the student model , and then the student model implicitly learns the distribution characteristics of the target domain.​ 7.The cross-domain human action recognition method based on feature distribution similarity according to claim 1, wherein, The step S302 of maximizing the classifier decision difference comprises: maximizing the L1 norm of the decision vector output by two classifiers, setting a constraint condition, and constructing a current final optimization target as: wherein, represents training a classifier , represents training a classifier , is a distribution distance loss, represents extracted semantic information, is a result of maximizing the classifier decision difference. 8.The cross-domain human action recognition method based on feature distribution similarity according to claim 1, wherein, The step S304 comprises: Set Labels representing different actions, representing a typical action from the source domain labeled for target domain features without labels, use the label of the nearest cluster center as its label estimate : , After obtaining the label estimate of the target domain feature, the target domain feature is pulled towards the source domain cluster center with the same label by minimizing the distance between the target domain feature and the corresponding cluster center: , The constraint target is obtained as: 。 9. A cross-domain human action recognition system based on similarity of feature distributions, characterized in that, The cross-domain human action recognition system comprises: A data preprocessing module for inputting the signal to be identified for data preprocessing and calculating the corresponding micro-Doppler spectrum diagram; An invariance feature extraction module is configured to perform unsupervised contrastive learning on the micro-Doppler spectrogram as input data to extract invariance features, wherein the unsupervised contrastive learning is performed by assigning the input data to two feature encoders with the same structure after two times of data augmentation and , wherein the feature encoders are optimized with an Info-NCE loss function, and the weights of the other feature encoders are updated in a momentum manner. A cross-domain migration module based on distribution similarity is used for inputting the micro-Doppler spectrum into a teacher feature encoder and a student feature encoder respectively Two classifiers are used for cross-domain migration training, and one of the classifiers is connected after each feature encoder The teacher feature encoder learns from step S02, and the teacher feature encoder passes the invariance features of the target domain to the student feature encoder during the migration training process, and the final recognition result is obtained after the training is completed The cross-domain transfer module based on distribution similarity comprises: A semantic information unit is used to train the entire network using label-supervised training to enable the model to identify the semantic information of different features for source domain data with existing labels, and to pre-train the teacher feature encoder using unsupervised contrastive learning for target domain data without labels as a teacher model, a student feature encoder as a student model; and subsequently using the teacher feature encoder guiding the decision result unification process of the target domain, so as to implicitly transfer the distribution information of the target domain features to the student model through knowledge distillation by means of the ability of the teacher model to extract invariant features; A maximum decision difference unit for maximizing the decision difference of two classifiers to determine the decision boundary of the classifier; a decision consistency unit configured to achieve clustering of features from different domains by optimizing a teacher feature encoder to make decisions of two classifiers consistent with each other; S304. Move to the center point: pull the target domain feature into the vicinity of the source domain center. Or the cross-domain human action recognition system comprises a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to execute the method as claimed in any one of claims 1-8.

Images