A cross-domain action recognition method with small sample learning based on CSI and triplet Siamese network

Abstract

The present invention relates to a small sample learning cross-domain action recognition method based on CSI and triplet twin network, comprising: collecting CSI data; performing data preprocessing; converting the preprocessed CSI data from a one-dimensional signal into a three-dimensional CSI matrix; constructing a data set; using a pre-trained model as a feature extractor of the triplet twin network, inputting the CSI matrix of the meta-training data set into the triplet twin network in the form of triplets for meta-training, and obtaining a meta-trained triplet twin network feature extractor; performing fine-tuning; using the meta-trained triplet twin network feature extractor and the fine-tuned Softmax classifier to classify the CSI matrix of the query set, and obtain a classification result of the query set. The present invention improves the accuracy of the CSI-based action recognition system in cross-domain scenarios, reduces the number of samples required for training the model, and solves the problem that the system recognition accuracy decreases in cross-domain scenarios and a large number of target domain samples need to be retrained.

Description

Technical Field

[0001] The present invention relates to the field of wireless sensing technology, and in particular to a small sample learning cross-domain action recognition method based on CSI and triplet twin networks. Background Art

[0002] CSI-based wireless sensing technology analyzes changes in Wi-Fi signals during propagation to obtain channel state information that reflects the spatial characteristics of signal propagation. This enables wireless sensing, overcoming the challenges of traditional sensors, which require specialized wearable equipment, as well as image-based methods that are susceptible to lighting effects, have blind spots, and pose privacy risks. Furthermore, due to the widespread deployment and use of Wi-Fi devices such as wireless network cards, smartphones, and routers, Wi-Fi signals are ubiquitous in daily life. Wireless sensing based on Wi-Fi channel state information avoids the high costs of sensing hardware such as radar.

[0003] Existing CSI-based action recognition systems typically use deep learning methods to classify CSI signals and identify actions. Examples include WiSDAR, a human action recognition system based on Wi-Fi CSI and deep learning, and LCED, a deep learning-based human action recognition system for diverse individuals developed by Dalian University of Technology. These methods utilize deep learning to achieve CSI-based human perception and action recognition. Training the perception network with CSI requires a large amount of training data. This large amount of collected CSI data is typically preprocessed using digital signal processing algorithms and then labeled with the type of human information it reflects before it can be used to train the human perception network model. This undoubtedly increases the implementation cost of these methods. Furthermore, because Wi-Fi channel state information is fine-grained and easily affected by the environment, a trained perception model may produce different results in different test scenarios.

[0004] Existing CSI-based cross-domain action recognition methods mainly fall into two categories: transfer learning and generative adversarial networks. These two methods enhance the robustness of the system in cross-scenario problems. However, transfer learning methods still require a large number of samples during the model transfer process, and generative adversarial networks also require the preparation of data from multiple scenarios in advance for adversarial training. Therefore, how to develop CSI-based action recognition technology while solving the problem of reduced system recognition accuracy in cross-domain scenarios, which requires retraining with a large number of target domain samples, is of great significance to improving the accuracy of CSI-based cross-domain action recognition systems, reducing system training costs, and enhancing the usability of CSI-based action recognition systems. Summary of the Invention

[0005] In order to solve the problem that existing wireless sensing technology is difficult to perform high-precision and low-training cost action recognition in cross-domain scenarios, the purpose of the present invention is to provide a small-sample learning cross-domain action recognition method based on CSI and triplet twin networks, which can ensure the accuracy of model action recognition in cross-domain scenarios while reducing the number of target domain samples required for training the model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a small sample learning cross-domain action recognition method based on CSI and triplet twin network, the method comprising the following steps in sequence:

[0007] (1) Collect CSI data: Place wireless signal transceivers supporting IEEE 802.11a / b / g / n wireless communication protocols in the source and target domain perception environments, and collect CSI data when different people perform different actions in different perception environments;

[0008] (2) Data preprocessing: The collected CSI data is filtered, de-noised, and normalized to obtain preprocessed CSI data;

[0009] (3) Perform SVD+channel replication: convert the pre-processed CSI data from a one-dimensional signal into a three-dimensional CSI matrix;

[0010] (4) Dataset construction: According to the domain where the CSI matrix is located, all CSI matrices are constructed into a meta-training dataset of the source domain and a meta-test dataset of the target domain, and the meta-test dataset is divided into a support set and a query set;

[0011] (5) Perform meta-training: Use the pre-trained model as the feature extractor of the triplet twin network, input the CSI matrix of the meta-training dataset into the triplet twin network in the form of triplets for meta-training, and obtain the triplet twin network feature extractor after meta-training;

[0012] (6) Fine-tuning: Use the support set to fine-tune the parameters of the Softmax classifier to obtain a fine-tuned Softmax classifier;

[0013] (7) Obtain the classification results of the query set: Use the meta-trained triplet twin network feature extractor and the fine-tuned Softmax classifier to classify the CSI matrix of the query set and obtain the classification results of the query set.

[0014] The step (3) specifically includes the following steps:

[0015] (3a) Take a piece of preprocessed CSI data. Its real matrix form is a two-dimensional matrix with dimensions W × S, denoted as H, where W is the number of packets intercepted in the time domain for this piece of CSI data, and S is the number of subcarriers in each piece of CSI data.

[0016] (3b) Decompose H into U, ∑, V by SVD formula T The product between:

[0017] H T =U∑V T (1)

[0018] Where ∑ is a diagonal matrix of dimension S×W, the main diagonal of ∑ contains the singular values of H, U and V are the left singular matrix and the right singular matrix, respectively. The left singular matrix has dimension S×S, and the right singular matrix has dimension W×W. T represents the transpose of the matrix.

[0019] (3c) Map the time domain measurement value on each subcarrier to the subcarrier space, retaining the characteristic components of the subcarrier, and obtain a matrix H' of dimension S×K1:

[0020] H'=H T ×V2 (2)

[0021] Where V2 is a W×K1 matrix consisting of K1 W-dimensional column vectors in the right singular matrix V, where K1 is the number of the largest singular values in the diagonal matrix Σ;

[0022] (3d) Calculate H' T The Pearson correlation coefficient between each column is used to obtain the correlation matrix P that can reflect the linear correlation between subcarriers:

[0023]

[0024]

[0025] Among them, h i Indicates H' T The i-th column vector, h j Indicates H' T The jth column vector of cov(h i ,h j ) represents h i and h j The covariance between i and σ j Respectively represent h i and h j The standard deviation of j,j It is h i and h jThe Pearson correlation coefficient between them, the element in row i and column j in P is h i and h j The Pearson correlation coefficient ρ between i,j ;

[0026] (3e) The correlation matrix P is subjected to weighted average channel replication in a channel-first manner to obtain a three-dimensional CSI matrix similar to an image with a constant size.

[0027] The step (4) specifically includes the following steps:

[0028] (4a) Based on the domains in which the CSI matrices are located, all CSI matrices are constructed into a meta-training dataset and a meta-testing dataset, where the meta-training dataset corresponds to the CSI data collected from the source domain, and the meta-testing dataset corresponds to the CSI data collected from the target domain;

[0029] (4b) Randomly extract N-way and K-shot CSI matrix samples from the meta-test dataset to form the support set for fine-tuning, and the remaining CSI matrix samples constitute the query set for testing the classification results of the system, where N-way means that the CSI matrix in the meta-test dataset has N categories, and K-shot means that there are K CSI matrix samples under each category in the meta-test dataset.

[0030] The step (5) specifically includes the following steps:

[0031] (5a) In the meta-training dataset, a CSI matrix sample is randomly selected as the anchor point x 0 , the anchor point x 0 The action category is recorded as category A, and another CSI matrix sample is randomly selected from category A as the positive sample x + , and then randomly select a CSI matrix sample from the data other than category A as the negative sample x - , negative sample x - The action category is recorded as category B, that is, an anchor point x 0 , a positive sample x + and a negative sample x - Form a triple;

[0032] (5b) The pre-trained model is the ResNet18 pre-trained model. The ResNet18 pre-trained model is selected as the feature extractor of the triplet twin network, and the anchor point x in the triplet is 0 , positive sample x + , negative sample x - Input them into the ResNet18 pre-training model respectively to obtain the feature vector f(x 0 )、f(x+ ) and f(x - );

[0033] (5c) Calculate the eigenvector f(x 0 ) and f(x + )、f(x 0 ) and f(x - ) between the cosine similarity:

[0034] Cosine_Similarity(α,β)=α T *β / (|α|*|β|) (5)

[0035] Where α and β represent the two column vectors to be calculated for cosine similarity, Cosine_Similarity(α,β) represents the cosine similarity between α and β; T represents the transposition of the vector;

[0036] (5d) In each meta-training iteration, triples are constructed according to the batch size. The number of constructed triplets is recorded as batch_size, and their feature vectors f(x 0 ) and f(x + )、f(x 0 ) and f(x - ), and obtain n = 2 * batch_size cosine similarities, recorded as Cos_Sim, and construct the similarity label Label according to the number of cosine similarities. Calculate the mean square error between all cosine similarities and labels as the loss function Loss of the triplet twin network:

[0037]

[0038] Where MSE(Cos_Sim, Label) represents the mean square error between Cos_Sim and Label;

[0039] (5e) Use the back-propagation algorithm to calculate the gradient, and use the gradient descent method to iteratively update the parameters of the ResNet18 pre-trained model in the triplet twin network to minimize the loss function Loss.

[0040] The step (6) specifically includes the following steps:

[0041] (6a) For an N-wayK-shot action classification task in the target domain, the meta-trained triplet Siamese network feature extractor is used to extract features of N×K CSI matrix samples in the support set;

[0042] (6b) Calculate the average of the K feature vectors for each action category and normalize them to obtain N normalized average feature vectors:

[0043] M=m1,m2,…m N (7)

[0044] Where mN represents the normalized average eigenvector of K eigenvectors in the Nth action category; M represents the matrix composed of N normalized average eigenvectors;

[0045] (6c) Combine the Softmax classifier with the cosine similarity and set the optimal initial parameters in the first iteration:

[0046]

[0047] The optimal initial parameter of parameter D is set to M, and each element d1, d2, ... d in D is N The optimal initial parameters are set to m1, m2, ...m N , the optimal initial parameters of parameter C are set to all elements c1,c2,…c N A vector of all 0s, q represents the feature vector to be classified, Sim(D,q) represents the calculation of the cosine similarity between each element in D and q, Sim(d N ,q) represents d N The cosine similarity between and q, the Softmax classifier outputs the predicted distribution p predict is a probability distribution, p predict Each element in represents the confidence that q belongs to each category;

[0048] (6d) Using the meta-trained triplet twin network feature extractor, the feature vectors of all CSI matrix samples in the support set are extracted and normalized. These feature vectors are used as the feature vectors to be classified q and are input into the Softmax classifier represented by formula (8) to obtain the probability distribution of N × K feature vectors to be classified belonging to different action categories;

[0049] (6e) Calculate the cross entropy between N×K Softmax predictions and the true label value:

[0050]

[0051] Among them, CrossEntropy(p actual ,p predict ) represents the probability distribution P actual and p predict The cross entropy, P actual Represents the target true distribution obtained according to the CSI matrix sample label;

[0052] (6f) Calculate the sum of N×K cross entropy losses as the objective function, add an entropy regularization term to prevent overfitting, calculate the gradient of the objective function with respect to the Softmax classifier parameters D and C, and iteratively update the classifier parameters using the gradient descent method to minimize the loss function so that the output of the Softmax classifier is close to the probability distribution of the true label of the CSI matrix sample.

[0053] The step (7) specifically includes the following steps:

[0054] (7a) For a CSI matrix sample in the query set, the features of the CSI matrix are extracted using the meta-trained triplet twin network feature extractor and normalized to obtain the feature vector q query ;

[0055] (7b) Using the fine-tuned Softmax classifier, the feature vector q query Perform classification and obtain the probability distribution p of the Softmax classifier output query ;

[0056] (7c) Probability distribution p query Each element in represents the confidence that the CSI matrix belongs to each action category, p query The action category corresponding to the largest element in is the classification result of the CSI matrix;

[0057] (7d) Classify all CSI matrix samples in the query set according to the classification steps of a single CSI matrix sample described in steps (7a) to (7c) to obtain a classification result of the query set.

[0058] It can be seen from the above technical solution that the beneficial effects of the present invention are: First, the present invention ensures the accuracy of model action recognition in cross-domain scenarios while reducing the number of target domain samples required for training the model through small sample learning based on the triplet twin network; Second, by using the ResNet18 pre-trained model as the feature extractor of the triplet twin network, it avoids the need to use a large amount of CSI data to train the feature extractor in the pre-training stage, so that the model converges faster and can make full use of more advanced and mature research results in the field of deep learning and image recognition, and realize effective feature extraction of CSI data by the pre-training model; Third, the pre-processed The CSI data is converted from a one-dimensional signal into a three-dimensional CSI matrix similar to an image with a constant size, so that the matrix can be effectively input into the pre-trained model and the triplet twin network, while reducing the size of the data to reduce the computational burden of the system; Fourth, by using the support set to fine-tune the parameters of the Softmax classifier, and by setting the optimal initial parameters, cross entropy regularization, combining cosine similarity and other fine-tuning techniques, the classification effect of Softmax is effectively improved, further improving the accuracy of action recognition when the target domain sample size is limited; Fifth, it has the advantages of low deployment cost, small computational burden, fast response speed, wide adaptability to scenarios, and avoidance of privacy risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of the method of the present invention;

[0060] Figure 2 Schematic diagram of the weighted average channel replication step in a channel priority manner according to the present invention;

[0061] Figure 3 This is a comparison chart of the classification accuracy of the present invention when the number of support set samples K takes different values. DETAILED DESCRIPTION

[0062] like Figure 1 As shown in FIG, a small sample learning cross-domain action recognition method based on CSI and triplet Siamese network is proposed, which includes the following steps in sequence:

[0063] (1) Collect CSI data: Place wireless signal transceivers supporting IEEE 802.11a / b / g / n wireless communication protocols in the source and target domain perception environments, and collect CSI data when different people perform different actions in different perception environments;

[0064] (2) Data preprocessing: The collected CSI data is filtered, de-noised, and normalized to obtain preprocessed CSI data;

[0065] (3) Perform SVD+channel replication: convert the pre-processed CSI data from a one-dimensional signal into a three-dimensional CSI matrix;

[0066] (4) Dataset construction: According to the domain where the CSI matrix is located, all CSI matrices are constructed into a meta-training dataset of the source domain and a meta-test dataset of the target domain, and the meta-test dataset is divided into a support set and a query set;

[0067] (5) Perform meta-training: Use the pre-trained model as the feature extractor of the triplet twin network, input the CSI matrix of the meta-training dataset into the triplet twin network in the form of triplets for meta-training, and obtain the triplet twin network feature extractor after meta-training;

[0068] (6) Fine-tuning: Use the support set to fine-tune the parameters of the Softmax classifier to obtain a fine-tuned Softmax classifier; further improve the classification effect of Softmax by selecting the best initial parameters, cross entropy regularization, and combining cosine similarity and other fine-tuning techniques;

[0069] (7) Obtain the classification results of the query set: Use the meta-trained triplet twin network feature extractor and the fine-tuned Softmax classifier to classify the CSI matrix of the query set and obtain the classification results of the query set.

[0070] The present invention uses the SVD+channel replication method to convert the pre-processed CSI data from a one-dimensional signal into a three-dimensional CSI matrix of constant size similar to an image; these conversions are not simple dimensional transformations, but contain profound physical meanings, that is, the differences in different actions, which were originally reflected in the changes in certain subcarriers of the CSI data in the time domain, are transformed into the similarities between all subcarriers of the CSI data after computational processing. Moreover, the converted three-dimensional CSI matrix can be directly input into the pre-training model in the form of an image, so that the pre-training model can perform effective feature extraction on the CSI, and at the same time can reduce the size of the data to reduce the computational burden of the system in subsequent training. The step (3) specifically includes the following steps:

[0071] (3a) Take a piece of preprocessed CSI data. Its real matrix form is a two-dimensional matrix with dimensions W × S, denoted as H, where W is the number of packets intercepted in the time domain for this piece of CSI data, and S is the number of subcarriers in each piece of CSI data.

[0072] (3b) Decompose H into U, Σ, V by SVD formula T The product between:

[0073] H T =U∑V T(1)

[0074] Where ∑ is a diagonal matrix of dimension S × W, the main diagonal of ∑ contains the singular values of H, U and V are the left singular matrix and the right singular matrix, respectively. The left singular matrix has dimension S × S, and the right singular matrix has dimension W × W. t represents the transpose of the matrix.

[0075] (3c) Map the time domain measurement value on each subcarrier to the subcarrier space, retaining the characteristic components of the subcarrier, and obtain a matrix H' of dimension S×K1:

[0076] H'=H T ×V2 (2)

[0077] Where V2 is a matrix of dimension W×K1 consisting of K1 W-dimensional column vectors in the right singular matrix V, where K1 is the number of the largest singular values in the diagonal matrix ∑;

[0078] (3d) Calculate H' T The Pearson correlation coefficient between each column is used to obtain the correlation matrix P that can reflect the linear correlation between subcarriers:

[0079]

[0080]

[0081] Among them, h i Indicates H' T The i-th column vector, h j Indicates H' T The jth column vector of cov(h i ,h j ) represents h i and h j The covariance between i and σ j Respectively represent h i and h j The standard deviation of i,j It is h i and h j The Pearson correlation coefficient between them, the element in row i and column j in P is h i and h j The Pearson correlation coefficient ρ between i,j ;

[0082] (3e) Figure 2 As shown, the correlation matrix P is subjected to weighted average channel replication in a channel-first manner to obtain a three-dimensional CSI matrix similar to an image with a constant size.

[0083] The present invention ensures the accuracy of model action recognition in cross-domain scenarios while reducing the number of target domain samples required for training the model through small sample learning based on the triplet twin network; by using the ResNet18 pre-trained model as the feature extractor of the triplet twin network, it avoids the need to use a large amount of CSI data to train the feature extractor in the pre-training stage, so that the model converges faster and can make full use of more advanced and mature research results in the fields of deep learning and image recognition to achieve effective feature extraction of CSI data by the pre-trained model; it solves the problem of the system recognition accuracy decreasing in cross-domain scenarios and the need for retraining with a large number of target domain samples, which is of great significance to improving the accuracy of the CSI-based cross-domain action recognition system, reducing the system training cost, and enhancing the availability of the CSI-based action recognition system.

[0084] The step (4) specifically includes the following steps:

[0085] (4a) Based on the domains in which the CSI matrices are located, all CSI matrices are constructed into a meta-training dataset and a meta-testing dataset, where the meta-training dataset corresponds to the CSI data collected from the source domain, and the meta-testing dataset corresponds to the CSI data collected from the target domain;

[0086] (4b) Randomly extract N-way and K-shot CSI matrix samples from the meta-test dataset to form the support set for fine-tuning, and the remaining CSI matrix samples constitute the query set for testing the classification results of the system, where N-way means that the CSI matrix in the meta-test dataset has N categories, and K-shot means that there are K CSI matrix samples under each category in the meta-test dataset.

[0087] The step (5) specifically includes the following steps:

[0088] (5a) In the meta-training dataset, a CSI matrix sample is randomly selected as the anchor point x 0 , the anchor point x 0 The action category is recorded as category A, and another CSI matrix sample is randomly selected from category A as the positive sample x + , and then randomly select a CSI matrix sample from the data other than category A as the negative sample x - , negative sample x - The action category is recorded as category B, that is, an anchor point x 0 , a positive sample x + and a negative sample x - Form a triple;

[0089] (5b) The pre-trained model is the ResNet18 pre-trained model. The ResNet18 pre-trained model is selected as the feature extractor of the triplet twin network, and the anchor point x in the triplet is 0 , positive sample x + , negative sample x - Input them into the ResNet18 pre-training model respectively to obtain the feature vector f(x 0 )、f(x + ) and f(x - );

[0090] (5c) Calculate the eigenvector f(x 0 ) and f(x + )、f(x 0 ) and f(x - ) between the cosine similarity:

[0091] Cosine_Similarity(α,β)=α T *β / (|α|*|β|) (5)

[0092] Where α and β represent the two column vectors to be calculated for cosine similarity, Cosine_Similarity(α,β) represents the cosine similarity between α and β; T represents the transposition of the vector;

[0093] (5d) In each meta-training iteration, triples are constructed according to the batch size. The number of constructed triplets is recorded as batch_size, and their feature vectors f(x 0 ) and f(x + )、f(x 0 ) and f(x - ), and obtain n = 2 * batch_size cosine similarities, recorded as Cos_Sim, and construct the similarity label Label according to the number of cosine similarities. Calculate the mean square error between all cosine similarities and labels as the loss function Loss of the triplet twin network:

[0094]

[0095] Where MSE(Cos_Sim, Label) represents the mean square error between Cos_Sim and Label;

[0096] (5e) Use the back-propagation algorithm to calculate the gradient, and use the gradient descent method to iteratively update the parameters of the ResNet18 pre-trained model in the triplet twin network to minimize the loss function Loss.

[0097] The step (6) specifically includes the following steps:

[0098] (6a) For an N-wayK-shot action classification task in the target domain, the meta-trained triplet Siamese network feature extractor is used to extract features of N×K CSI matrix samples in the support set;

[0099] (6b) Calculate the average of the K feature vectors for each action category and normalize them to obtain N normalized average feature vectors:

[0100] M=m1,m2,…m N (7)

[0101] Among them, m N represents the normalized average eigenvector of K eigenvectors in the Nth action category; M represents the matrix composed of N normalized average eigenvectors;

[0102] (6c) Combine the Softmax classifier with the cosine similarity and set the optimal initial parameters in the first iteration:

[0103]

[0104] The optimal initial parameter of parameter D is set to M, and each element d1, d2, ... d in D is N The optimal initial parameters are set to m1, m2, ...m N , the optimal initial parameters of parameter C are set to all elements c1,c2,…c N A vector of all 0s, q represents the feature vector to be classified, Sim(D,q) represents the calculation of the cosine similarity between each element in D and q, Sim(d N ,q) represents d N The cosine similarity between and q, the Softmax classifier outputs the predicted distribution p predict is a probability distribution, p predict Each element in represents the confidence that q belongs to each category;

[0105] (6d) Using the meta-trained triplet twin network feature extractor, the feature vectors of all CSI matrix samples in the support set are extracted and normalized. These feature vectors are used as the feature vectors to be classified q and are input into the Softmax classifier represented by formula (8) to obtain the probability distribution of N × K feature vectors to be classified belonging to different action categories;

[0106] (6e) Calculate the cross entropy between N×K Softmax predictions and the true label value:

[0107]

[0108] Among them, CrossEntropy(p actual ,p predict ) represents the probability distribution p actual and p predict The cross entropy of p actual Represents the target true distribution obtained according to the CSI matrix sample label;

[0109] (6f) Calculate the sum of N×K cross entropy losses as the objective function, add an entropy regularization term to prevent overfitting, calculate the gradient of the objective function with respect to the Softmax classifier parameters D and C, and iteratively update the classifier parameters using the gradient descent method to minimize the loss function so that the output of the Softmax classifier is close to the probability distribution of the true label of the CSI matrix sample.

[0110] The step (7) specifically includes the following steps:

[0111] (7a) For a CSI matrix sample in the query set, the features of the CSI matrix are extracted using the meta-trained triplet twin network feature extractor and normalized to obtain the feature vector q query ;

[0112] (7b) Using the fine-tuned Softmax classifier, the feature vector q query Perform classification and obtain the probability distribution p of the Softmax classifier output query ;

[0113] (7c) Probability distribution p query Each element in represents the confidence that the CSI matrix belongs to each action category, p query The action category corresponding to the largest element in is the classification result of the CSI matrix;

[0114] (7d) Classify all CSI matrix samples in the query set according to the classification steps of a single CSI matrix sample described in steps (7a) to (7c) to obtain a classification result of the query set.

[0115] Example 1

[0116] First, CSI data was collected in three different sensory environments: an office, a conference room, and a classroom. CSI data was collected from different people performing four different actions: standing still, walking, jumping, and standing up. The office was used as the source domain, and the conference room and classroom were used as the target domains. The people involved were two subjects of different heights and weights, and the wireless signal coverage area covered the entire office, conference room, and classroom. The experimental platform used a four-antenna ASUS RT-AC86U Wi-Fi router. The Nexmon CSI extraction tool was used to collect CSI data consisting of 242 active subcarriers on an 80 MHz channel.

[0117] In three sensory environments, 50 CSI data points were collected for each action performed by each participant. Each collected CSI data point was filtered, denoised, and normalized to obtain preprocessed CSI data. Using SVD and channel replication, the preprocessed CSI data was converted from a one-dimensional signal into a three-dimensional CSI matrix of constant size, similar to an image. The CSI matrices processed from the office area CSI data constituted a meta-training dataset, denoted as E1, while the CSI matrices processed from the conference room and classroom CSI data constituted two meta-test datasets, denoted as E2 and E3, respectively. Four-way, K-shot CSI matrix samples were randomly sampled from meta-test datasets E2 and E3, respectively, to form two support sets for fine-tuning, denoted as S2 and S3. The remaining CSI samples formed two query sets, denoted as Q2 and Q3, for testing the system's classification results. "4-way" indicates that the CSI matrices in the dataset cover four action categories, and "K-shot" indicates that there are K CSI matrix samples for each category in the dataset.

[0118] The pre-trained ResNet18 model was used as the feature extractor for the triplet twin network. The CSI matrix of the meta-training dataset E1 was input as triplets into the triplet twin network for meta-training. After meta-training, the support sets S2 and S3 were used, respectively, and the parameters of the Softmax classifier were fine-tuned by selecting the optimal initial parameters, using cross-entropy regularization, and incorporating fine-tuning techniques such as cosine similarity. This resulted in two sets of Softmax classifiers with different parameters, denoted as Softmax2 and Softmax3.

[0119] Use the meta-trained triplet twin network feature extractor and Softmax2 to classify the CSI matrix in the query set Q2; use the meta-trained triplet twin network feature extractor and Softmax3 to classify the CSI matrix in the query set Q3; obtain the classification results of query sets Q2 and Q3.

[0120] Table 1 shows the classification accuracy of this method for query sets Q2 and Q3 in different target domains. It can be seen from Table 1 that this method can still maintain a high accuracy when generalized from the source domain to different target domains. Moreover, as the number of samples K in the support sets S2 and S3 increases, the classification accuracy of this method for query sets Q2 and Q3 will also improve.

[0121] Table 1 Classification accuracy of different target domain query sets Q2 and Q3

[0122]

[0123] In order to further evaluate the impact of the number of samples K in the support set on the classification accuracy of the present invention, Figure 3 The classification accuracy of the present invention for the query set Q2 is shown when the value of the number of samples K in the support set S2 increases from 1 to 10. It can be seen that as the value of K increases, the classification accuracy of the present invention for the query set continues to improve. When K = 10, the average classification accuracy of the present invention reaches a maximum of 96%. In addition, when K = 1, the average classification accuracy of the present invention is 79.37%, and when K = 5, the average classification accuracy of the present invention is improved to 94.08%. This shows that there are obvious differences between the target domain and the source domain, and the present invention only needs a small number of target domain support set samples for fine-tuning to generalize the present invention from the source domain to the target domain, proving the effectiveness of fine-tuning in the present invention in improving classification accuracy. In addition, when the value of K increases from 1 to 4, the average classification accuracy of the system increases rapidly, but after that, as the number of samples continues to increase, the rate of improvement of its accuracy slows down significantly; therefore, in order to achieve a balance between classification accuracy and fine-tuning training time, in actual testing and application, the 5-shot support set is mainly used to fine-tune the Softmax classifier.

[0124] To evaluate the accuracy of the present invention in classifying different actions, Table 2 shows the confusion matrix generated based on the classification results of the present invention for four different actions in the query set Q2, namely, stationary, walking, jumping, and standing, using the support set S2 as a 4-way, 5-shot system. It can be seen that the system achieves the highest classification accuracy for stationary actions, reaching 96.67%. The CSI data for jumping and walking actions exhibit similar motion patterns in terms of body movements during actual acquisition, and analysis of the images generated by these two CSI matrices reveals that these two types of actions do share a high degree of similarity. Despite this, the present invention still achieves high classification accuracy for these two types of actions, reaching 91.35% and 93.19%, respectively, further demonstrating the effectiveness of the present invention.

[0125] Table 2 Confusion matrix of query set Q2 classification results

[0126]

[0127] In summary, the present invention improves the accuracy of the CSI-based action recognition system in cross-domain scenarios while reducing the number of samples required for training the model, solving the problem of the system's reduced recognition accuracy in cross-domain scenarios requiring retraining with a large number of target domain samples. This is of great significance for improving the accuracy of the CSI-based cross-domain action recognition system, reducing the system training cost, and enhancing the usability of the CSI-based action recognition system. It also has the advantages of low deployment cost, small computational burden, fast response speed, wide adaptability to scenarios, and avoidance of privacy risks.

[0128] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A small-sample learning cross-domain action recognition method based on CSI and triplet Siamese network, characterized by: The method comprises the following steps in sequence: (1) Collect CSI data: Place wireless signal transceivers supporting IEEE 802.11a / b / g / n wireless communication protocols in the source and target domain perception environments, and collect CSI data when different people perform different actions in different perception environments; (2) Data preprocessing: The collected CSI data is filtered, de-noised, and normalized to obtain preprocessed CSI data; (3) Perform SVD+channel replication: convert the pre-processed CSI data from a one-dimensional signal into a three-dimensional CSI matrix; (4) Dataset construction: According to the domain where the CSI matrix is located, all CSI matrices are constructed into a meta-training dataset of the source domain and a meta-test dataset of the target domain, and the meta-test dataset is divided into a support set and a query set; (5) Perform meta-training: Use the pre-trained model as the feature extractor of the triplet twin network, input the CSI matrix of the meta-training dataset into the triplet twin network in the form of triplets for meta-training, and obtain the triplet twin network feature extractor after meta-training; (6) Fine-tuning: Use the support set to fine-tune the parameters of the Softmax classifier to obtain a fine-tuned Softmax classifier; (7) Obtain the classification results of the query set: Use the meta-trained triplet twin network feature extractor and the fine-tuned Softmax classifier to classify the CSI matrix of the query set and obtain the classification results of the query set.

2. The cross-domain action recognition method based on small sample learning using CSI and triplet twin networks according to claim 1 is characterized by: The step (3) specifically includes the following steps: (3a) Take a piece of preprocessed CSI data. Its real matrix form is a two-dimensional matrix with dimensions W × S, denoted as H, where W is the number of packets intercepted in the time domain for this piece of CSI data, and S is the number of subcarriers in each piece of CSI data. (3b) Decompose H into U, ∑, V by SVD formula T The product between: H T =U∑V T (1) Where ∑ is a diagonal matrix of dimension S×W, the main diagonal of ∑ contains the singular values of H, U and V are the left singular matrix and the right singular matrix, respectively. The left singular matrix has dimension S×S, and the right singular matrix has dimension W×W. T represents the transpose of the matrix. (3c) Map the time domain measurement value on each subcarrier to the subcarrier space, retaining the characteristic components of the subcarrier, and obtain a matrix H′ with a dimension of S×K1: H′=H T ×V2 (2) Where V2 is a matrix of dimension W×K1 consisting of K1 W-dimensional column vectors in the right singular matrix V, where K1 is the number of the largest singular values in the diagonal matrix ∑; (3d) Calculate H′ T The Pearson correlation coefficient between each column is used to obtain the correlation matrix P that can reflect the linear correlation between subcarriers: Among them, h i Represents H′ T The i-th column vector, h j Represents H′ T The jth column vector of cov(h i , h j ) represents h i and h j The covariance between i and σ j Respectively represent h i and h j The standard deviation of i，j It is h i and h j The Pearson correlation coefficient between them, the element in row i and column j in P is h i and h j The Pearson correlation coefficient ρ between i，j ; (3e) The correlation matrix P is subjected to weighted average channel replication in a channel-first manner to obtain a three-dimensional CSI matrix similar to an image with a constant size.

3. The cross-domain action recognition method based on small sample learning using CSI and triplet twin networks according to claim 1 is characterized in that: The step (4) specifically includes the following steps: (4a) Based on the domains in which the CSI matrices are located, all CSI matrices are constructed into a meta-training dataset and a meta-testing dataset, where the meta-training dataset corresponds to the CSI data collected from the source domain, and the meta-testing dataset corresponds to the CSI data collected from the target domain; (4b) Randomly extract N-way and K-shot CSI matrix samples from the meta-test dataset to form the support set for fine-tuning, and the remaining CSI matrix samples constitute the query set for testing the classification results of the system, where N-way means that the CSI matrix in the meta-test dataset has N categories, and K-shot means that there are K CSI matrix samples under each category in the meta-test dataset.

4. The cross-domain action recognition method based on small sample learning using CSI and triplet twin networks according to claim 1 is characterized in that: The step (5) specifically includes the following steps: (5a) In the meta-training dataset, a CSI matrix sample is randomly selected as the anchor point x 0 , the anchor point x 0 The action category is recorded as category A, and another CSI matrix sample is randomly selected from category A as the positive sample x + , and then randomly select a CSI matrix sample from the data other than category A as the negative sample x - , negative sample x - The action category is recorded as category B, that is, an anchor point x 0 , a positive sample x + and a negative sample x - Form a triple; (5b) The pre-trained model is the ResNet18 pre-trained model. The ResNet18 pre-trained model is selected as the feature extractor of the triplet twin network, and the anchor point x in the triplet is 0 , positive sample x + , negative sample x - Input them into the ResNet18 pre-training model respectively to obtain the feature vector f(x 0 )、f(x + ) and f(x - ); (5c) Calculate the eigenvector f(x 0 ) and f(x + )、f(x 0 ) and f(x - ) between the cosine similarity: Cosine_Similarity(a,b)=a T *β / (|α|*|β|) (5) Where α and β represent the two column vectors whose cosine similarity is to be calculated, Cosine_Similarity(α, β) represents the cosine similarity between α and β; T represents the transposition of the vector; (5d) In each meta-training iteration, triples are constructed according to the batch size. The number of constructed triplets is recorded as batch_size, and their feature vectors f(x 0 ) and f(x + )、f(x 0 ) and f(x - ), and obtain n = 2 * batch_size cosine similarities, recorded as Cos_Sim, and construct the similarity label Label according to the number of cosine similarities. Calculate the mean square error between all cosine similarities and labels as the loss function Loss of the triplet twin network: Where MSE(Cos_Sim, Label) represents the mean square error between Cos_Sim and Label; (5e) Use the back-propagation algorithm to calculate the gradient, and use the gradient descent method to iteratively update the parameters of the ResNet18 pre-trained model in the triplet twin network to minimize the loss function Loss.

5. The cross-domain action recognition method based on small sample learning using CSI and triplet twin networks according to claim 1 is characterized in that: The step (6) specifically includes the following steps: (6a) For an N-way K-shot action classification task in the target domain, the meta-trained triplet Siamese network feature extractor is used to extract features of N × K CSI matrix samples in the support set; (6b) Calculate the average of the K feature vectors for each action category and normalize them to obtain N normalized average feature vectors: M=m1,m2,...m N (7) Among them, m N represents the normalized average eigenvector of K eigenvectors in the Nth action category; M represents the matrix composed of N normalized average eigenvectors; (6c) Combine the Softmax classifier with the cosine similarity and set the optimal initial parameters in the first iteration: The optimal initial parameter of parameter D is set to M, and each element d1, d2, ...d in D is N The optimal initial parameters are set to m1, m2, ...m N , the optimal initial parameters of parameter C are set to all elements c1, c2, ...c N A vector of all 0s, q represents the feature vector to be classified, Sim(D,q) represents the cosine similarity between each element in D and q, Sim(d N ,q) represents d N The cosine similarity between and q, the Softmax classifier outputs the predicted distribution p predict is a probability distribution, p predict Each element in represents the confidence that q belongs to each category; (6d) Using the meta-trained triplet twin network feature extractor, the feature vectors of all CSI matrix samples in the support set are extracted and normalized. These feature vectors are used as the feature vectors to be classified q and are input into the Softmax classifier represented by formula (8) to obtain the probability distribution of N × K feature vectors to be classified belonging to different action categories; (6e) Calculate the cross entropy between N×K Softmax predictions and the true label value: Among them, CrossEntropy(p actual , p predict ) represents the probability distribution p actual and p predict The cross entropy of p actual Represents the target true distribution obtained according to the CSI matrix sample label; (6f) Calculate the sum of N×K cross entropy losses as the objective function, add an entropy regularization term to prevent overfitting, calculate the gradient of the objective function with respect to the Softmax classifier parameters D and C, and iteratively update the classifier parameters using the gradient descent method to minimize the loss function so that the output of the Softmax classifier is close to the probability distribution of the true label of the CSI matrix sample.

6. The cross-domain action recognition method based on small sample learning using CSI and triplet twin networks according to claim 1, characterized in that: The step (7) specifically includes the following steps: (7a) For a CSI matrix sample in the query set, the features of the CSI matrix are extracted using the meta-trained triplet twin network feature extractor and normalized to obtain the feature vector q query ; (7b) Using the fine-tuned Softmax classifier, the feature vector q query Perform classification and obtain the probability distribution p of the Softmax classifier output query ; (7c) Probability distribution p query Each element in represents the confidence that the CSI matrix belongs to each action category, p query The action category corresponding to the largest element in is the classification result of the CSI matrix; (7d) Classify all CSI matrix samples in the query set according to the classification steps of a single CSI matrix sample described in steps (7a) to (7c) to obtain a classification result of the query set.

Images