Activity identification method, system and equipment based on dual-band signal contrast learning

Abstract

The invention discloses an activity identification method, system and equipment based on dual-band signal comparative learning. The method comprises the following steps: synchronously acquiring channel state information (CSI) signal data X24 and X5 of dual bands through multi-antenna Wi-Fi equipment; processing acquired dual-band CSI signal data X24 and X5 through a residual neural network model based on comparative learning and multi-task joint training to generate dual-band vectors f24 and f5, and performing dynamic attention weighted feature fusion to generate a fusion feature; and inputting the fusion feature into a classifier of the residual neural network model for activity identification and classification, and then outputting an identification result. The method supports high-robustness multi-person activity recognition, and is suitable for an intelligent sensing scene in a dynamic environment.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and particularly relates to a method, system, and device for activity recognition based on dual-band signal contrast learning. Background Art

[0002] With the development of smart home and Internet of Things technologies, human activity recognition technology (HAR) has gradually become a research hotspot. As a low-cost and non-invasive sensing means, Wi-Fi channel state information (CSI) has been widely used in human activity recognition. Existing technologies mostly use single-band Wi-Fi signals for activity recognition, and use deep learning models such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), etc. to extract signal features and classify them.

[0003] In current Wi-Fi CSI-based activity recognition methods, some studies have attempted to combine CNNs with LSTMs or GRUs (gated recurrent units) to enhance the processing ability of time series data. For example, a continuous human activity recognition method based on CSI (Patent CN 116304888A) combines a one-dimensional CNN and a BiGRU model to extract spatial and temporal dimension features of actions.

[0004] This method segments CSI signal data through a sliding window technique to eliminate the interference of inactive data, and uses a network combining CNN and GRU for feature extraction. However, most of these methods rely on single-band CSI signals and are often affected by factors such as signal noise and occlusion in complex dynamic environments, resulting in a decrease in recognition accuracy.

[0005] In addition, some other studies have adopted a combination of contrast learning and few-shot learning (Patent CN 119377770A) to solve the dependence on a large amount of labeled data during the training process. Especially in the few-shot scenario, it can effectively learn through a small amount of labeled data.

[0006] Although these methods have good performance in some specific scenarios, they usually rely on a single frequency band (such as 2.4 GHz or 5 GHz) for activity recognition, and the multi-modal processing of signals (such as simultaneously using 2.4 GHz and 5 GHz frequency bands) has not been fully explored.

[0007] Deficiencies of the Existing Technology:

[0008] Single frequency band: Most of the existing technologies are based on CSI signals of a single frequency band (such as 2.4 GHz or 5 GHz), ignoring the complementary characteristics of signals in different frequency bands. Signals of a single frequency band are easily affected by multipath effects, reflections, and interference, resulting in limited recognition accuracy.

[0009] Lack of multimodal fusion: Most of the existing technologies focus on the processing of single-frequency band signals and fail to effectively fuse the information of multi-frequency band signals (such as 2.4 GHz and 5 GHz), unable to make full use of the complementary characteristics of signals in different frequency bands. Summary of the Invention

[0010] In order to solve the above technical problems, the present invention proposes an activity recognition method, system, and device based on contrastive learning of dual-frequency band signals. By combining Wi-Fi CSI signals of 2.4 GHz and 5 GHz dual-frequency bands and using contrastive learning and deep learning models for multi-frequency band signal fusion, the problems of single frequency band and lack of multimodal fusion in the existing technologies are solved, and the accuracy and robustness of multi-human activity recognition are improved.

[0011] In order to achieve the above object, the present invention provides the following solutions:

[0012] In the first aspect, the present invention proposes an activity recognition method based on contrastive learning of dual-frequency band signals, characterized in that the method includes:

[0013] Synchronously collect channel state information CSI signal data X 24 and X5 of dual-frequency bands through a multi-antenna Wi-Fi device;

[0014] Pass the collected CSI signal data X 24 and X5 of dual-frequency bands through a residual neural network model based on contrastive learning and multi-task joint training to generate dual-frequency band vectors f 24 and f5, and generate a fusion feature f fusion through dynamic attention weighted feature fusion;

[0015] Input the fusion feature f fusion into the classifier of the residual neural network model for activity recognition and classification, and output the recognition result;

[0016] wherein, the dual-frequency band includes 2.4 GHz and 5 GHz frequency bands.

[0017] As a further improvement, the step of synchronously collecting channel state information CSI signal data X 24 and X5 of dual-frequency bands through a multi-antenna Wi-Fi device includes: simultaneously collecting CSI data of 2.4 GHz and 5 GHz frequency bands through a Wi-Fi network card supporting dual-frequency bands to generate initial tensor data and and generate dual - band input tensor data X after standardization 24 and X5, where and N is the number of samples, B is the batch size, P is the number of antennas, T is the time step, and Q is the number of sub - carriers.

[0018] As a further improvement, the steps of the residual neural network model based on contrast learning and multi - task joint training include:[[]]

[0019] Give the input samples with a batch of N to the initial residual neural network model Extract dual - band high - dimensional features and generate low - dimensional features z after non - linear dimensionality reduction based on projection 24 and z5, and obtain unit vectors after L2 normalization and

[0020] Construct a cross - band similarity matrix where the element represents the cosine similarity between the 2.4GHz feature of the i - th sample and the 5GHz feature of the j - th sample;

[0021] Based on the temperature parameter τ, the mask matrix M, and the cosine similarity, construct a contrast loss function:[[]]

[0022]

[0023] Based on the classification probability output of the lightweight multi - layer perceptron MLP classifier, use the binary cross - entropy loss function with a class balance factor for supervision:[[]]

[0024]

[0025] Generate a total loss function according to the contrast loss function and the binary cross - entropy loss function:[[]]

[0026]

[0027] where, y (i) represents the sample label, α is the class weight, σ is the Sigmoid function, is the output classification probability, logs ij / τ represents the logarithmic similarity, and λ is a hyperparameter.

[0028] As a further improvement, the steps of extracting dual - band high - dimensional features and generating low - dimensional features z after linear dimensionality reduction based on projection include:[[]] 24 and z5 include:[[]]

[0029] Take the input samples Map the 2048 - dimensional input to a 1024 - dimensional hidden space and apply the ReLU activation function; and further compress it to a 128 - dimensional low - dimensional embedding space. The projection process is defined as:

[0030]

[0031] where is a learnable weight matrix, is a bias term, and m ∈ {24, 5}.

[0032] As a further improvement, the steps of generating the dual - band vectors f 24 and f5 from the collected dual - band CSI signal data X 24 through a residual neural network model based on contrast learning and multi - task joint training include:

[0033] Input the collected dual - band CSI signal data X 24 and X5 into the trained residual neural network model. In the residual neural network model, extract shallow spatio - temporal features through a 7×7 convolutional layer and a max - pooling layer, and then successively pass through 4 residual modules. The 4 residual modules respectively contain 3 Bottleneck units, 4 Bottleneck units, 6 Bottleneck units, and 3 Bottleneck units. Each Bottleneck unit adopts a "1×1→3×3→1×1" convolutional stacking structure. Finally, generate the dual - band vectors f 24 and f5 through a global average pooling layer, where and is expressed by the following expression:

[0034] f 24 = Flatten(ResNet50(X 24 )), f5 = Flatten(ResNet50(X5))

[0035] where B represents the batch size.

[0036] As a further improvement, the steps of generating the fused feature f 24 from the dual - band vectors f fusion through dynamic attention - weighted feature fusion specifically include:

[0037] Perform L2 normalization on the dual - band vectors f 24 and f5:

[0038]

[0039] Calculate the attention scores:

[0040]

[0041] Generate attention weights based on the Softmax function:

[0042] α 24 = Softmax(s 24 ), α5 = Softmax(s5)

[0043] Through element-wise scaling, obtain the weighted features:

[0044]

[0045] Directly add the weighted dual-band features to form the fused feature:

[0046]

[0047] Among them, represents the attention parameter matrix, represents the overall importance score of each sample feature, and satisfies ⊙ represents element-wise multiplication, and the scalar weight is extended to the feature dimension through the broadcasting mechanism.

[0048] As a further improvement, the steps of inputting the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification and then outputting the recognition result include:

[0049] Input the fused feature f fusion into the classifier of the residual neural network model, map the 2048-dimensional feature to a 1024-dimensional hidden space in the first linear layer and apply the ReLU activation function:

[0050]

[0051] Apply the Dropout operation with a probability p of 0.1:

[0052] h drop = Dropout(h, p = 0.1)

[0053] Second linear layer: Compress the 1024-dimensional feature to the target category dimension out:

[0054]

[0055] Normalize the output p final by Sigmoid and generate the output result based on the binary prediction label through the preset threshold θ:

[0056]

[0057] Among them, σ is the Sigmoid function, is the indicator function, and the output is a class label matrix of 0 or 1.

[0058] As a further improvement, the preset threshold θ is 0.5.

[0059] In a second aspect, the present invention also proposes an identification system for implementing the identification method in the first aspect. The system includes:

[0060] An acquisition device for synchronously acquiring channel state information CSI signal data X 24 and X5 of dual bands through a multi-antenna Wi-Fi device;

[0061] A processing device for generating dual-band vectors f 24 and f5 after the acquired CSI signal data X 24 and X5 of dual bands pass through a residual neural network model based on contrast learning and multi-task joint training, and generating a fused feature f fusion after dynamic attention weighted feature fusion;

[0062] An output device for inputting the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification and then outputting an identification result;

[0063] Among them, the dual bands include 2.4 GHz and 5 GHz bands.

[0064] In a third aspect, the present aspect also proposes an electronic device, including a memory and a processor, and the memory is coupled to the processor; wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the activity recognition method based on dual-band signal contrast learning described in the first aspect.

[0065] Since Wi-Fi signals are blocked by obstacles such as walls and furniture during propagation, and phenomena such as attenuation, reflection, and refraction often occur when the signals reach the receiving end, this multipath effect of the signals will introduce greater interference, resulting in inaccurate feature extraction. Especially in complex environments, such as scenes with high-density furniture arrangements or multi-person interactions, the influence of signal interference and noise will be more significant, thus affecting the accuracy of activity recognition.

[0066] The present invention reduces the influence of signal interference in the following ways:

[0067] Complementary nature of multi-band signals: Wi-Fi signals in the 2.4GHz and 5GHz bands have different environmental adaptabilities. The 5GHz band signal has a shorter propagation distance and less interference, but is inferior to the 2.4GHz in terms of penetration. The 2.4GHz band signal has stronger penetration in complex environments but is more susceptible to interference. By fusing these two signals, their respective deficiencies can be compensated for, thereby improving the robustness of the signal and reducing the impact of interference on feature extraction.

[0068] Multi-modal feature learning: A contrastive learning mechanism is adopted to optimize the feature extraction process. Contrastive learning can enhance the model's ability to identify valid signals by maximizing the differences between positive and negative samples and minimizing the differences between similar samples, thereby reducing the impact of noise interference on feature extraction. Description of the Drawings

[0069] Figure 1 This is a flowchart of the activity recognition method based on contrastive learning of dual-band signals proposed by the present invention.

[0070] Figure 2 This is a schematic diagram of the overall architecture of the activity recognition method based on contrastive learning of dual-band signals proposed by the present invention.

[0071] Figure 3 This is a schematic diagram of the ResNet50 feature extractor in the activity recognition method based on contrastive learning of dual-band signals proposed by the present invention.

[0072] Figure 4 This is a framework diagram of the activity recognition method based on contrastive learning of dual-band signals proposed by the present invention.

[0073] Figure 5 This is a schematic diagram of the electronic device proposed by the present invention. Detailed Implementation Manner

[0074] The following further describes the present application in detail with reference to the drawings. It is necessary to point out here that the following detailed implementation manner is only used to further illustrate the present application and cannot be construed as a limitation on the protection scope of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0075] As Figure 1-2As shown in the figure, the present invention proposes an activity recognition method based on dual - band signal contrast learning. By synchronously collecting channel state information (CSI) signals in the 2.4GHz and 5GHz frequency bands, amplitude features are extracted and a spatio - temporal dimension tensor is constructed as the input. The pre - trained ResNet50 model is used to extract high - dimensional features of the dual - band respectively, and a learnable projection module (Projector) is combined to achieve feature dimensionality reduction and discriminability enhancement; an adaptive attention mechanism is introduced to dynamically optimize the cross - band feature fusion weights, solving the generalization defect of traditional fixed - weight fusion. In the training stage, through the joint optimization strategy of contrast loss (InfoNCE) and multi - task classification loss (BCEWithLogitsLoss), the collaborative improvement of unsupervised feature alignment and supervised classification is realized, strengthening the model's ability to distinguish complex activities. Finally, the fused features are directly output as activity prediction results through a lightweight multi - layer perceptron (MLP) classifier, supporting highly robust multi - person activity recognition and applicable to intelligent perception scenarios in dynamic environments.

[0076] Specifically, it includes the following steps:

[0077] Step S101. Synchronously collect the channel state information CSI signal data X 24 and X5 of the dual - band through a multi - antenna Wi - Fi device;

[0078] Step S102. After passing the collected CSI signal data X 24 and X5 of the dual - band through a residual neural network model based on contrast learning and multi - task joint training, generate dual - band vectors f 24 and f5, and generate fused feature f fusion through dynamic attention - weighted feature fusion;

[0079] Step S103. Input the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification, and then output the recognition result;

[0080] Among them, the dual - band includes the 2.4GHz and 5GHz frequency bands.

[0081] In this embodiment, through a Wi - Fi network card that supports dual - band (such as Intel 5300), the CSI data of the 2.4GHz and 5GHz frequency bands are collected simultaneously. Specifically, it includes: generating initial tensor data and by simultaneously collecting the CSI data of the 2.4GHz and 5GHz frequency bands through a Wi - Fi network card that supports dual - band, and generating dual - band input tensor data X 24 and X5 after normalization processing, where is the same as N is the number of samples, B is the batch size, P is the number of antennas, T is the time step, and Q is the number of subcarriers.

[0082] In this embodiment, the number of subcarriers Q is set to 30, the number of antennas P is set to 3, and the time step T is set to 3000. The complex channel responses are recorded respectively. The modulus value of the complex CSI of each subcarrier is taken to extract the amplitude feature. Thus, the initial tensor is obtained. and For the possible packet loss problem, a zero-padding strategy is adopted to complement the missing time-series data, ensuring that the time step of all samples is uniformly 3000. Before inputting into the model, the initial tensor data and is adjusted to the standard format [B, C, H, W] of the ResNet50 backbone network, where C = 3 corresponds to the antenna channels, H = 3000 is the time-series length, and W = 30 is the subcarrier spatial dimension.

[0083] After standardization processing, the dual-band input tensor data X 24 and X5 are generated as the model input, ensuring that the subsequent feature extraction module can efficiently process spatio-temporal signals to support the end-to-end activity recognition task.

[0084] The present invention adopts a deep architecture based on the Residual Neural Network (ResNet50), combined with a learnable projection module, to achieve efficient spatio-temporal feature representation and compression of dual-band CSI signals. ResNet50 serves as the core backbone network, and its design incorporates the residual learning idea, effectively alleviating the problem of gradient vanishing in deep networks through cross-layer skip connections. This network contains 49 convolutional layers and 1 fully connected layer, originally designed for image classification tasks. To adapt to the requirements of wireless signal feature extraction, the present invention makes key modifications to it: removing the end fully connected layer and retaining the convolutional layers as a general feature extractor, with a final output dimension of 2048-dimensional high-order abstract features.

[0085] As Figure 3 shown, for the input 2.4GHz and 5GHz band CSI signals and (where B is the batch size), the model first extracts shallow spatio-temporal features through a 7×7 convolutional layer and a max pooling layer, and then successively passes through 4 residual modules (each containing 3, 4, 6, and 3 Bottleneck units). Each Bottleneck unit adopts a convolutional stacking structure of "1×1→3×3→1×1", reducing the number of parameters through the strategy of dimensionality reduction - feature transformation - dimensionality increase, and at the same time introducing the ReLU activation function and batch normalization (BatchNorm) to accelerate convergence. Finally, the global average pooling layer at the end of the network compresses the feature map into a vector obtained through a flattening operation The 5GHz signal is obtained in the same way

[0086] The mathematical representation of this process is:

[0087] f 24 = Flatten(ResNet50(X 24 ))), f5 = Flatten(ResNet50(X5))

[0088] To adapt to the characteristics of wireless signals, the convolutional kernel parameters of ResNet50 are fine-tuned based on the pre-trained weights (ImageNet dataset) so that it can capture the local correlations (such as spatio-temporal fluctuations caused by multipath effects) and global patterns (such as periodic changes caused by human movement) of CSI signals. By introducing the pre-trained model, the data requirements for training from scratch are significantly reduced, and at the same time, model degradation caused by task differences is avoided.

[0089] Subsequently, the high-dimensional features are non-linearly reduced in dimension through a two-branch projection module (Projector). Each Projector consists of two fully connected layers: the first layer maps the 2048-dimensional input to a 1024-dimensional hidden space and applies the ReLU activation function to enhance the non-linear expression ability; the second layer further compresses it to a 128-dimensional low-dimensional embedding space to reduce redundant information. The projection process is defined as:

[0090]

[0091] where is a learnable weight matrix, is the bias term (m ∈ {24, 5}). The weight initialization adopts the Xavier normal distribution. According to the formula, it ensures stable gradients in the initial stage of training and avoids optimization oscillations caused by differences in parameter scales.

[0092] The final output of this module is As a low-dimensional embedded feature, its dimension compression ratio reaches 16:1, significantly reducing the subsequent computational overhead while retaining discriminative information, and providing a highly discriminative feature basis for cross-band contrast learning and dynamic fusion.

[0093] The present invention introduces an adaptive attention mechanism to independently evaluate the importance of features in the 2.4GHz and 5GHz bands, and realizes the dynamic fusion of cross-band features through weighted summation. The specific implementation is as follows:

[0094] For the high-dimensional features of 2.4GHz and 5GHz extracted by ResNet50 and (B is the batch size), first perform L2 normalization on both of them respectively:

[0095]

[0096] To eliminate the influence of feature amplitude differences on attention calculation. Subsequently, calculate the attention weights of each band feature through a dual-channel attention module respectively:

[0097] (1) Generation of attention scores:

[0098] Each band feature independently performs a dot product operation with the learnable attention parameter matrix to generate the original attention scores:

[0099]

[0100] where represents the overall importance score of each sample feature.

[0101] (2) Weight normalization and feature weighting:

[0102] Apply the Softmax function to the attention scores to convert them into attention weights in the form of a probability distribution:

[0103] α 24 = Softmax(s 24 ), α5 = Softmax(s5)

[0104] where and satisfies Finally, scale the original features element-wise by the scalar weights to obtain the weighted features:

[0105]

[0106] where ⊙ represents element-wise multiplication, and the scalar weights are extended to the feature dimension through the broadcast mechanism.

[0107] (3) Cross - band feature fusion:

[0108] Directly add the weighted dual - band features to form the fused features:

[0109]

[0110] This operation enables the model to dynamically adjust the contribution ratio of the two - band features according to the input samples. For example, when the 5GHz signal is affected by multipath interference, its weight is automatically reduced, while the weight of the 2.4GHz signal weakens in a low - SNR scenario.

[0111] In this embodiment, the steps of the residual neural network model based on contrastive learning and multi - task joint training in step 102 specifically include:

[0112] In the training stage, the present invention combines contrastive learning and multi - task joint optimization to achieve unsupervised alignment of cross - band features and collaborative improvement of classification performance. Specifically, given a batch of N input samples The model first extracts dual - band high - dimensional features through ResNet50 and and maps them to a low - dimensional embedding space through the Projector module On this basis, the contrastive learning module achieves feature alignment through the following steps:

[0113] Feature normalization and similarity calculation:

[0114] Perform L2 normalization on the low - dimensional features z 24 and z5 to obtain unit vectors and Construct a cross - band similarity matrix where the element represents the cosine similarity between the 2.4GHz feature of the i - th sample and the 5GHz feature of the j - th sample. Introduce a temperature parameter τ, where τ can be taken as 0.07, which is used to adjust the distribution sharpness, and define the logarithmic similarity as logs ij / τ.

[0115] Positive and negative sample dynamic screening:

[0116] Dynamically construct positive and negative sample pairs based on label consistency: If the labels y (i) of samples i and j are equal to y (j), then (i, j) is a positive sample pair; otherwise it is a negative sample pair. To enhance robustness, a confidence mask is further introduced: when the classification prediction results of the 2.4GHz or 5GHz modality exactly match the true labels (i.e., or ), mark this sample as a high-confidence positive sample, and filter out noise samples through the mask matrix M ∈ {0, 1} N×N . Finally, the contrast loss function is defined as:

[0117]

[0118] This loss forces the cross-band features of similar samples to closely cluster in the embedding space, and the features of dissimilar samples to repel each other.

[0119] Joint optimization of multi-modal classification:

[0120] Based on contrastive learning, synchronously optimize the classification performance of 2.4GHz, 5GHz, and fused features. Let p 24 = MLP(f 24 ), p5 = MLP(f 24 ), and p fusion = MLP(f fusion ) be the classification probability outputs of each modality and the fused feature respectively, and use binary cross-entropy loss with class balance factor (BCEWithLogitsLoss) for supervision:

[0121]

[0122] where α is the class weight, and α can be taken as 6, and σ is the Sigmoid function. Finally, the total loss function is the weighted sum of the contrast loss and the classification loss:

[0123]

[0124] By adjusting the hyperparameter λ, where λ can be taken as 0.1, balance the trade-off between feature alignment and classification accuracy.

[0125] In this embodiment, step S103 specifically includes:

[0126] In the end-to-end activity recognition and real-time classification stage, the present invention uses a lightweight multi-layer perceptron (MLP) classifier to efficiently infer the fused features and directly output the activity category prediction results.

[0127] The specific process is as follows:

[0128] Classifier architecture and input processing:

[0129] The fused feature Input classifier, whose structure includes two fully connected layers and a Dropout module:

[0130] The first linear layer: maps 2048-dimensional features to a 1024-dimensional hidden space and applies the ReLU activation function:

[0131]

[0132] Dropout: During the training phase, a Dropout operation with a probability p of 0.1 is applied to the output h of the hidden layer to prevent overfitting:

[0133] h drop = Dropout(h, p = 0.1)

[0134] The second linear layer: compresses 1024-dimensional features to the target class dimension out:

[0135]

[0136] Prediction result generation:

[0137] After the fused feature f fusion is input into the classifier of the residual neural network model, directly perform Sigmoid normalization on the classifier output and generate a binary prediction label through a preset threshold θ (such as θ = 0.5):

[0138]

[0139] where σ is the Sigmoid function, is the indicator function, and the output is a class label matrix of 0 or 1.

[0140] In this embodiment, by fusing Wi-Fi signals in the 2.4GHz and 5GHz frequency bands and making full use of their complementarity, the recognition accuracy and robustness are significantly improved. Although the 2.4GHz signal has strong penetration ability, it is easily interfered; while the 5GHz signal has strong anti-interference ability but a small coverage range. By weighted fusion of these two frequency band signals and combined with dynamic weight adjustment, the technical solution of this embodiment effectively reduces signal interference and improves the overall recognition accuracy.

[0141] The WiMANS dataset is a multi-person human activity recognition dataset that contains data from multiple environmental scenarios, such as classrooms, meeting rooms, and empty rooms. Through this dataset, the present invention has demonstrated significant effects in experiments comparing different signal frequency bands and methods. In the existing 2.4 / 5 GHz methods, the 2.4 GHz and 5 GHz data are separately input for training and testing, that is, the data input each time only comes from a single frequency band of 2.4 GHz or 5 GHz, and the data between frequency bands are processed separately. Therefore, this method does not fully utilize the complementary advantages of dual-frequency signals.

[0142] In contrast, in this embodiment, the 2.4 + 5 GHz method inputs the signal data of 2.4 GHz and 5 GHz each time, and the data of the two frequency bands are simultaneously used as inputs for training and testing, so as to fully utilize the complementary information of the two frequency bands for joint learning. This joint input method can achieve more accurate feature fusion and improve the recognition accuracy and robustness in complex environments.

[0143] The experimental results on the WiMANS dataset show that after fusing the 2.4 GHz and 5 GHz signals, the method of the present invention has increased the average recognition accuracy to 65.4% in scenarios such as classrooms, meeting rooms, and empty rooms, which is about 6 - 15 percentage points higher than the traditional 2.4 / 5 GHz method.

[0144] Table 1: Recognition accuracies of different methods in different environments:

[0145]

[0146] This result proves that the dual-frequency signal fusion and contrast learning mechanism can effectively improve the accuracy and robustness of human activity recognition. The present invention has strong environmental adaptability and can be widely applied to fields such as smart homes, security monitoring, and health monitoring, especially suitable for multi-person human activity recognition tasks.

[0147] This embodiment also proposes a recognition system 400 for an activity recognition method based on dual-band signal contrast learning. The recognition system 400 includes:

[0148] A collection device 401 for synchronously collecting the channel state information CSI signal data X of the dual frequency bands through a multi-antenna Wi-Fi device 24and X5. In this embodiment, the acquisition device can be a multi-antenna Wi-Fi device equipped with an Intel 5300 network as the signal receiving end. Deployed in a target environment (such as a 10m×8m meeting room), the CSI signals in the 2.4GHz and 5GHz frequency bands are synchronously acquired. The sampling rate is set to 1kHz, and the continuous capture duration is 3 seconds (corresponding to 3000 time points). Each frequency band contains 30 subcarriers. The complex CSI data is recorded by the receiving end and stored as a tensor of dimension N×3×3000×30 (N is the total number of samples), where 3 represents the number of antennas. For the packet loss problem, a zero-padding strategy is adopted to complement the missing time points to ensure that the time series length of all samples is uniformly 3000.

[0149] The processing device 402 is used to process the CSI signal data X of the dual-frequency bands 24 and X5 through a residual neural network model based on contrast learning and multi-task joint training to generate dual-frequency vectors f 24 and f5, and generate a fused feature f after dynamic attention weighted feature fusion fusion .

[0150] In this embodiment, the original CSI data is converted into amplitude features, and the dual-frequency tensor format input to the model is (B is the batch size), encapsulated as a batch data stream through the DataLoader of PyTorch. When training the model, the model is built using the PyTorch framework, and the training server is configured with an NVIDIA A100 GPU. The model hyperparameters are set as follows: learning rate 3×10 -4 (Adam optimizer), batch size B = 32, temperature parameter τ = 0.07, contrast loss weight λ = 0.1, classification loss positive sample weight α = 6. The ResNet50 backbone network loads the ImageNet pre-trained weights, freezes the parameters of the first 3 residual modules, and only fine-tunes the parameters of the last 1 residual module and the Projector module. The structure of the Projector module is fully connected layer 2048→1024→128, and the weights are initialized using the Xavier normal distribution. The learnable parameters of the dynamic attention module are initialized as zero vectors, and the training period is 200 rounds.

[0151] The output device 403 is used to input the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification, and then output the recognition result;

[0152] Among them, the dual-frequency bands include the 2.4GHz and 5GHz frequency bands.

[0153] Figure 5 Illustrates a schematic diagram of the physical structure of an electronic device, such asFigure 5 As shown in the figure, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 complete their mutual communication through the communication bus 540. The processor 510 may call the logical instructions in the memory 530 to execute the above-mentioned activity recognition method based on dual-band signal contrast learning.

[0154] In addition, when the logical instructions in the above-mentioned memory 530 are implemented in the form of software functional units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0155] The purpose of the present invention is to solve the following technical problems existing in the prior art by fusing 2.4GHz and 5GHz dual-band Wi-Fi CSI signals and combining a contrast learning mechanism:

[0156] 1. Improve the accuracy and robustness of human activity recognition in a multi-person environment

[0157] In a multi-person environment, due to the interaction and occlusion effects between multiple human activities, traditional Wi-Fi CSI signal processing methods based on a single frequency band are difficult to effectively distinguish the activities of different individuals, resulting in a significant decrease in recognition accuracy. Especially in complex environments, factors such as the attenuation, multipath effect, and occlusion of Wi-Fi signals make the reliability and stability of single-frequency band signals insufficient, thus affecting the effect of human activity recognition. The present invention fuses the Wi-Fi CSI signals of the 2.4GHz and 5GHz frequency bands and utilizes their complementarity to significantly improve the recognition accuracy and robustness in a multi-person environment.

[0158] Specifically, the present invention adopts the following technical means to improve the recognition accuracy:

[0159] Signal fusion: Wi-Fi CSI signals in the 2.4GHz and 5GHz bands have different propagation characteristics and coverage. Although the 5GHz band has strong anti-interference capabilities, its coverage is small, while the 2.4GHz band has strong penetration and is suitable for a larger spatial range. By fusing the signals of these two frequency bands, the advantages of each frequency band can be maximized, improving the adaptability to complex activity scenarios.

[0160] Contrastive learning: Deep neural networks are used to extract features from 2.4GHz and 5GHz signals respectively, and the performance of the feature extractor is improved by introducing a contrastive learning mechanism. This contrastive learning method can significantly improve the robustness of the model, especially in multi-person activity recognition.

[0161] 2. Reduce the impact of signal interference on feature extraction

[0162] Since Wi-Fi signals are blocked by obstacles such as walls and furniture during propagation, signals often attenuate, reflect, and refract when they reach the receiving end. This multipath effect of the signal will introduce significant interference, leading to inaccurate feature extraction. Especially in complex environments, such as high-density furniture arrangements or multi-person interactions, the impact of signal interference and noise will be more significant, thus affecting the accuracy of activity recognition.

[0163] The present invention reduces the impact of signal interference by:

[0164] Complementarity of multi-band signals: Wi-Fi signals in the 2.4GHz and 5GHz bands have different adaptability to the environment. The propagation distance of 5GHz band signals is shorter and less subject to interference, but they are inferior to 2.4GHz in terms of penetration. 2.4GHz band signals have strong penetration in complex environments, but are more subject to interference. By fusing the two signals, their respective shortcomings can be compensated, thereby improving the robustness of the signal and reducing the impact of interference on feature extraction.

[0165] Multimodal feature learning: Contrastive learning mechanism is used to optimize the feature extraction process. Contrastive learning can maximize the difference between positive and negative samples and minimize the difference between similar samples, thereby improving the model's ability to identify valid signals and reducing the impact of noise interference on feature extraction.

[0166] In the present invention, unless otherwise clearly defined and limited, terms such as "installed", "connected", "linked", "fixed", etc. shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral one; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0167] In the present invention, unless otherwise clearly defined and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "underneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0168] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. An activity recognition method based on contrastive learning of dual-band signals, characterized in that, The method includes: synchronously collecting channel state information CSI signal data X of dual bands through a multi-antenna Wi-Fi device 24 and X5; The collected CSI signal data X of dual bands 24 and X5 are input into a residual neural network model based on contrastive learning and multi-task joint training to generate dual-band vectors f 24 and f5, and then through dynamic attention weighted feature fusion, a fused feature f is generated fusion ; Input the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification, and then output the recognition result; Among them, the dual band includes 2.4GHz and 5GHz bands.

2. The activity recognition method based on dual-band signal contrast learning according to claim 1, wherein Synchronously collect the channel state information CSI signal data X of the dual band through a multi-antenna Wi-Fi device 24 The steps of generating the initial tensor data X and X5 include: simultaneously collecting CSI data in the 2.4 GHz and 5 GHz bands through a Wi-Fi network card supporting dual bands to generate initial tensor data and and generating the dual-band input tensor data X after normalization processing 24 and X5, where and N is the number of samples, B is the batch size, P is the number of antennas, T is the time step, and Q is the number of subcarriers.

3. The activity recognition method based on dual-band signal contrast learning according to claim 2, wherein The steps of the residual neural network model based on contrastive learning and multi-task joint training include: Given a batch of N input samples to the initial residual neural network model Extract dual-band high-dimensional features and generate low-dimensional features z after non-linear dimensionality reduction based on projection 24 and z5, and obtain unit vectors after L2 normalization and Construct a cross-band similarity matrix where the element represents the cosine similarity between the 2.4GHz feature of the i-th sample and the 5GHz feature of the j-th sample; Based on the temperature parameter τ, the mask matrix M, and the cosine similarity, a contrastive loss function is constructed: Based on the classification probability output of the lightweight multi-layer perceptron MLP classifier, a binary cross-entropy loss function with a class balance factor is used for supervision: A total loss function is generated according to the contrastive loss function and the binary cross-entropy loss function: Among them, y (i) represents the sample label, α is the class weight, σ is the Sigmoid function, is the output classification probability, logs ij / τ represents the log similarity, and λ is the hyperparameter.

4. The activity recognition method based on dual-band signal contrast learning according to claim 3, wherein Extract dual-band high-dimensional features and generate low-dimensional feature z after linear dimensionality reduction based on projection 24 The steps for [specific content related to z5] include: The input sample is mapped from a 2048-dimensional input to a 1024-dimensional hidden space, and the ReLU activation function is applied; and it is further compressed to a 128-dimensional low-dimensional embedding space. The projection process is defined as: wherein is a learnable weight matrix, is a bias term, and m ∈ {24, 5}.

5. The activity recognition method based on dual-band signal contrast learning according to claim 1, wherein The collected CSI signal data X of dual bands 24 and X5 are processed through a residual neural network model based on contrastive learning and multi-task joint training to generate dual-band vectors f 24 and f5, and the steps include: The collected CSI signal data X of dual bands 24 and X5 are input into the trained residual neural network model. In the residual neural network model, shallow spatio-temporal features are extracted through a 7×7 convolutional layer and a max pooling layer, and then sequentially pass through 4 residual modules. The 4 residual modules respectively contain 3 Bottleneck units, 4 Bottleneck units, 6 Bottleneck units and 3 Bottleneck units. Each Bottleneck unit adopts a convolutional stacking structure of "1×1→3×3→1×1". Finally, a dual-band vector f 24 and f5 are generated, where and are expressed by the following expression: f 24 = Flatten(ResNet50(X 24 ))), f5 = Flatten(ResNet50(X5)) Among them, B represents the batch size.

6. The activity recognition method based on dual-band signal contrast learning according to claim 5, characterized in that The dual - band vector f 24 and f5 generate the fused feature f fusion after dynamic attention - weighted feature fusion. The specific steps are as follows: Perform L2 normalization on the dual-band vectors f 24 and f5: Calculate the attention score: Generate attention weights based on the Softmax function: α 24 = Softmax(s 24 ), α5 = Softmax(s5) Through element-wise scaling, the weighted features are obtained: The weighted dual-band features are directly added together to form a fused feature: Among them, represents the attention parameter matrix, represents the overall importance score of each sample feature, and satisfies ⊙ represents element-wise multiplication, and the scalar weight is extended to the feature dimension through the broadcasting mechanism.

7. The activity recognition method based on dual-band signal contrast learning according to claim 6, characterized in that Input the fusion feature f fusion The steps of inputting into the classifier of the residual neural network model for activity recognition and classification and then outputting the recognition result include: Input the fused feature f fusion into the classifier of the residual neural network model, map the 2048-dimensional feature to a 1024-dimensional hidden space in the first linear layer and apply the ReLU activation function: Apply a Dropout operation with a probability p of 0.1: h drop = Dropout(h, p = 0.1) The second linear layer: Compress the 1024-dimensional features to the target class dimension out: The output p final is subjected to Sigmoid normalization and an output result based on a binarized prediction label is generated through a preset threshold θ: where σ is the Sigmoid function, is the indicator function, and the output is a categorical label matrix of 0 or 1.

8. The activity recognition method based on dual-band signal contrast learning according to claim 7, characterized in that The preset threshold θ is 0.

5.

9. The recognition system of the activity recognition method based on dual-band signal contrast learning according to any one of claims 1-8, characterized in that, The system includes: A collection device for synchronously collecting channel state information CSI signal data X of dual bands through a multi-antenna Wi-Fi device 24 and X5; A processing device for processing the collected CSI signal data X of dual bands 24 and X5, which are input into a residual neural network model based on contrastive learning and multi-task joint training to generate dual-band vectors f 24 and f5, and then generate a fused feature f through dynamic attention weighted feature fusion fusion ; An output device for inputting the fused feature f fusion into the classifier of the residual neural network model for activity recognition and classification, and then outputting a recognition result; Among them, the dual band includes 2.4GHz and 5GHz bands.

10. An electronic device, characterized in that, It includes a memory and a processor, characterized in that the memory is coupled to the processor; wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the activity recognition method based on dual-band signal contrastive learning according to any one of claims 1-8 above.

Images