Through-the-wall radar human body behavior recognition method and recognition system based on spatial-temporal characteristics

Abstract

The invention provides a through-the-wall radar human body behavior recognition method and recognition system based on spatial-temporal characteristics, and the method comprises the steps: obtaining and processing through-the-wall radar human body behavior echo sampling signals, and obtaining a corresponding time-Doppler spectrogram as a sample set; and training a constructed TWRMama network by using the sample set, in which the network adopts a CA-Mama block as a core feature extraction module, can efficiently model and input a time sequence dynamic feature and a spatial dependency relationship of a time Doppler spectrogram, and effectively enhance the feature expression ability in a shielding scene, thereby realizing accurate recognition of human behaviors of the through-the-wall radar. Besides, information interaction between patches is introduced into a patch embedding module of the TWRMama network, so that the expression capability of the network can be further improved, and the calculation complexity of the network is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of radar target recognition, and particularly relates to a through-wall radar human behavior recognition method and system based on spatiotemporal features. BACKGROUND

[0002] Human behavior recognition plays an important role in many fields such as traffic safety, medical care, autonomous driving, urban warfare and public security. Traditional human behavior recognition methods usually rely on wearable sensors, optical cameras and Wi-Fi-based sensing technology. However, these methods often face problems such as privacy leakage, strong environmental sensitivity and poor user experience. In contrast, radar-based human behavior recognition is increasingly valued due to its unique advantages such as non-contact operation, insensitivity to lighting conditions and privacy protection. Through the use of advanced signal processing and deep learning technologies, radar-based human behavior recognition methods have been widely applied. In recent years, ultra-wideband (UWB) radar has been widely used in through-wall radar (TWR) human behavior recognition due to its high range resolution, strong anti-interference capability and excellent penetration performance.

[0003] In the application of through-wall human behavior recognition based on radar sensors, the target human body is often obstructed by walls or other obstacles. Electromagnetic waves will be significantly attenuated when penetrating obstacles, resulting in severe distortion of radar echo signals. The recognition performance of existing methods based on traditional machine learning is significantly reduced. With the gradual application of deep learning technology in the field of radar human behavior recognition, researchers have attempted to use convolutional neural networks (CNN), recurrent neural networks (RNN) and Transformer models to automatically extract discriminative spatiotemporal features from distorted radar signals to improve recognition accuracy and robustness. However, the recognition accuracy of these methods still cannot meet the actual application requirements, and the overall recognition performance needs to be further improved. In addition, since the above methods usually rely on deep and complex network structures, the computational resource consumption is large, the number of model parameters is significantly increased, and in actual deployment, the system complexity and inference delay problems are high.

[0004] Therefore, how to effectively improve the recognition accuracy and robustness of the through-wall radar human behavior recognition method, and reduce the computational resource consumption and the number of model parameters, has become a problem to be solved. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a through-wall radar human behavior recognition method and system based on spatiotemporal features to solve the problems of insufficient recognition accuracy and high network complexity of through-wall radar human behavior recognition in occlusion scenarios.

[0006] This invention provides a method for human behavior recognition using through-wall radar based on spatiotemporal features, comprising:

[0007] S1: Build a UWB radar data acquisition platform and use the platform to measure the echo of various human behaviors behind the wall to obtain echo sampling signals. Then, stack the echo sampling signals into an echo matrix by rows and process the echo matrix to obtain the corresponding time-Doppler spectrum.

[0008] S2: Divide the time-Doppler spectrum into training set, validation set and test set according to the proportions;

[0009] S3: Construct a TWRMamba network for human behavior recognition using through-wall radar and train, validate, and test the TWRMamba network using the training, validation, and test sets to obtain a trained TWRMamba network. The TWRMamba network includes a Patch Embedding module, a TWRMamba encoder, and a prediction head arranged sequentially. The Patch Embedding module contains a two-dimensional deep convolutional layer, a two-dimensional convolutional layer, and a positional encoding embedding operation. The TWRMamba encoder is composed of multiple stacked CA-Mamba blocks. Each CA-Mamba block contains a spatial branch, a temporal branch, and a residual connection. The spatial branch introduces a coordinate attention mechanism, the temporal branch uses a bidirectional Mamba network, and the residual connection adds the outputs of the spatial and temporal branches to the input of the CA-Mamba block. The prediction head contains a global average pooling layer and a fully connected layer. The fully connected layer contains a linear layer and a softmax layer.

[0010] S4: Use a trained TWRMamba network for through-wall radar human behavior recognition.

[0011] Preferably, in S1, stacking the echo sampled signals row by row into an echo matrix includes:

[0012] Acquire the through-wall radar human behavior echo sampling signal, where the first The pulse, the first The echo sampling signal corresponding to each sampling point is represented as follows: , , ,in, This indicates the number of slow-time samples, i.e., the number of pulses transmitted. Indicates the sampling interval of the pulse. This indicates the number of fast-time samples, i.e., the number of sampling points within each pulse;

[0013] Will formed after stacking in rows of the echo matrix .

[0014] Further preferably, in S1, processing the echo matrix to obtain a corresponding time-Doppler spectrogram comprises:

[0015] S11: preprocessing the echo matrix to weaken the interference of wall reflection, static clutter and background noise to obtain a matrix ;

[0016] S12: performing Hilbert transform on each row of the matrix to convert the matrix into a complex-valued matrix , wherein represents the Hilbert transform result of each row, represents the imaginary unit;

[0017] S13: performing clipping on the fast time dimension of the complex-valued matrix to obtain a region of interest of human behavior, and then performing coherent superposition on the slow time signal corresponding to the region of interest to obtain a superposed slow time signal , wherein is the distance unit index range of the region of interest;

[0018] S14: performing short-time Fourier transform on the slow time signal to obtain a time-Doppler matrix , wherein and are the indexes of frequency and time, respectively, and then mapping the time-Doppler matrix to a three-channel RGB image to obtain a time-Doppler spectrogram and unify the size.

[0019] Further preferably, in S1, the preprocessing is mean subtraction filtering processing along the slow time dimension of the echo matrix.

[0020] Further preferably, in S3, training the TWRMamba network comprises the following steps:

[0021] S31: inputting the time-Doppler spectrogram into the Patch Embedding module, first, down-sampling through a two-dimensional deep convolutional layer, then dividing into multiple patches through a two-dimensional convolutional layer and mapping the feature dimension to a set dimension to obtain a feature map, and then flattening the feature map to add position encoding to obtain an encoding sequence ;

[0022] ​S32: The TWRMamba encoder contains multiple consecutive CA-Mamba block pairs in the encoded sequence. The processing involves extracting the encoded sequence via spatial and temporal branches within the CA-Mamba block. Spatial and temporal features, with residual connections in the CA-Mamba block used to integrate the spatial and temporal features with the encoded sequence. Adding them together yields a feature vector representing the target information. ;

[0023] The spatial branch introduces a coordinate attention mechanism, and the spatial branch uses the encoded sequence generated by the Patch Embedding module. As input, we focus on the spatial feature information and output a weighted encoded sequence; in the time branch, a bidirectional Mamba network is used to extract the temporal features of the input.

[0024] S33: Use the prediction head to extract the feature vector The results are obtained by converting the data into a category probability distribution.

[0025] Further optimization, in S32, the encoded sequence is extracted. The steps for spatial features are as follows:

[0026] For the input encoded sequence First, perform layer normalization, then transform its size to... ,get Then, global average pooling is performed along the horizontal and vertical directions respectively to obtain the result after global average pooling. and ;

[0027] Then and Stacked along the channel dimension, the input is fed into a convolutional layer, where channel compression is performed and positional information in two directions is encoded to obtain intermediate features. ,in, , Indicates the channel attenuation rate;

[0028] Next, The system is divided into two parts along both the horizontal and vertical directions. Each part is processed by a convolutional layer to adjust the channels and restore them to the channel dimension. The weights in the horizontal and vertical directions are obtained. and ;

[0029] Finally, the weights and and Perform point-by-point multiplication and transform the result into a one-dimensional sequence to obtain the output of the spatial branch. .

[0030] Further optimization, in S32, the encoded sequence is extracted. The steps for defining the time characteristics are as follows:

[0031] First, the input encoded sequence Perform layer normalization and use a linear layer to obtain two projection vectors. and ;

[0032] Subsequently, the vector is processed through two parallel branches. The forward output is obtained by processing the forward and reverse versions of the forward output. and reverse output Each branch contains a one-dimensional convolutional layer and an SSM;

[0033] Finally, the output of the time branch is calculated using the following formula. :

[0034] ;

[0035] In the formula, SiLU represents the SiLU activation function. This indicates element-wise multiplication. This represents a linear layer with both input and output neurons having the same number of neurons. .

[0036] Further optimization involves training the TWRMamba network with a batch size of 32, using cross-entropy loss as the loss function, AdamW as the optimizer, a weight decay coefficient of 0.05, an initial learning rate of 0.0001, and halving the learning rate every 20 training epochs.

[0037] The present invention also provides a through-wall radar human behavior recognition system based on spatiotemporal features, for performing the above-described through-wall radar human behavior recognition method based on spatiotemporal features.

[0038] This invention provides a method and system for human behavior recognition using through-wall radar based on spatiotemporal features. It constructs a TWRMamba network model based on CA-Mamba, combining a Mamba network designed based on the State Space Model (SSM) principle with a Coordinate Attention (CA) mechanism. The TWRMamba network model can efficiently model the temporal dynamic features and spatial dependencies of the input time-Doppler spectrum, effectively enhancing feature representation capabilities in occluded scenarios, thereby achieving accurate recognition of human behavior using through-wall radar. Furthermore, information interaction between patches is introduced in the Patch Embedding module, which not only further improves the network's expressive power but also effectively reduces its computational complexity. Compared with existing through-wall radar human behavior recognition methods, this method not only exhibits higher recognition accuracy and robustness under occlusion conditions but also significantly reduces computational resource consumption and the number of model parameters, making it more suitable for resource-constrained deployment environments.

[0039] This invention provides a method and system for human behavior recognition using through-wall radar based on spatiotemporal features. It integrates a spatiotemporal feature extraction mechanism, employing CA-Mamba as the core feature extraction module. Its classification accuracy and computational complexity outperform traditional CNN, RNN, and Transformer models. TWRMamba effectively combines the spatial location sensitivity of the CA mechanism with the global modeling capability of Mamba, enabling efficient capture of spatiotemporal features in the temporal spectral map. This method solves the problems of low accuracy and high computational resource consumption in through-wall radar human behavior recognition, significantly improving the recognition performance of through-wall radar human behavior under low resource consumption conditions. Attached Figure Description

[0040] Figure 1 A flowchart of the through-wall radar human behavior recognition method based on spatiotemporal features provided by the present invention;

[0041] Figure 2 This is a diagram of the overall structure of a TWRMamba network.

[0042] Figure 3 This is a network structure diagram of a CA-Mamba block;

[0043] Figure 4 This is a network structure diagram of the CA mechanism;

[0044] Figure 5 The test set confusion matrix diagram provided for the embodiment;

[0045] Figure 6 A bar chart showing the accuracy of the test set and related evaluation metrics provided in the example. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] To effectively address the issue of insufficient accuracy in human behavior recognition using through-wall radar in occluded scenarios, while significantly reducing network complexity, such as... Figure 1 As shown, this invention provides a method for human behavior recognition using through-wall radar based on spatiotemporal features, comprising the following steps:

[0048] S1: Build a UWB radar data acquisition platform and use the platform to measure the echo of various human behaviors behind the wall to obtain echo sampling signals. Then, stack the echo sampling signals into an echo matrix by rows and process the echo matrix to obtain the corresponding time-Doppler spectrum.

[0049] During the echo measurement process, each participant performs various actions behind a wall of a certain thickness at a certain distance from the radar. The echo data is transmitted to a computer for storage. The radar transmits the data on the [number]th ... The pulse, the first The echo sampling signal corresponding to each sampling point is represented as follows: , , ,in, This indicates the number of slow-time samples, i.e., the number of pulses transmitted. Indicates the sampling interval of the pulse. This indicates the number of fast-time samples, i.e., the number of sampling points within each pulse. After being stacked row by row, it forms echo matrix ;

[0050] Among them, for the echo matrix The processing to obtain the corresponding time-Doppler spectrum includes the following steps:

[0051] S11: For the echo matrix Preprocessing is performed to reduce interference from wall-reflected waves, static clutter, and background noise. Specifically, the echo matrix is ​​preprocessed. A mean-subtraction filter is performed along the slow time dimension to obtain the matrix. ,in, The calculation formula is as follows:

[0052] ;

[0053] S12: For the matrix Perform a Hilbert transformation on each row of the matrix to transform it. Transform into a complex-valued matrix ,in, express The Hilbert transform result for each row, Represents the imaginary unit;

[0054] S13: For the complex-valued matrix The fast time dimension is cropped to obtain the region of interest (ROI) of human behavior. Then, the slow time signals corresponding to the ROI are coherently superimposed to obtain the superimposed slow time signal. ,in, The distance cell index range for the region of interest;

[0055] S14: For the slow-time signal Perform a short-time Fourier transform to obtain the time-Doppler matrix. ,in, and These are the indices for frequency and time, respectively. Then, the time-Doppler matrix is... The image is mapped to a three-channel RGB image to obtain a time-Doppler spectrum, and its size is uniformly transformed to 224×224×3.

[0056] S2: Divide the time-Doppler spectrum into a training set, a validation set, and a test set in a 3:1:1 ratio;

[0057] S3: Construct a TWRMamba network for human behavior recognition using through-wall radar and train, validate, and test it using the training, validation, and test sets to obtain a trained TWRMamba network. The overall network structure of the TWRMamba network is as follows: Figure 2 As shown, it includes a Patch Embedding module, a TWRMamba encoder, and a prediction head arranged sequentially. The design of the Patch Embedding module is as follows: Figure 2As shown, the system includes a two-dimensional deep convolutional (DWConv2d) layer, a two-dimensional convolutional (Conv2d) layer, and a positional encoding embedding operation. The DWConv2d layer comprises a DWConv2d layer, a batch normalization (BN) layer, and a ReLU activation function, with 3 kernels, a size of 7×7, and a stride of 2. The Conv2d layer consists of only one Conv2d layer with 128 kernels, a size of 7×7, and a stride of 7. The positional encoding embedding operation adds the positional encoding to the sequence after flattening the feature map output from the Conv2d layer. The TWRMamba encoder is mainly composed of 8 stacked CA-Mamba blocks, designed as follows: Figure 3 As shown, it includes a spatial branch, a temporal branch, and a residual connection. The spatial branch introduces a coordinate attention (CA) mechanism to extract enhanced spatial features and improve the model's representational ability. Preferably, its channel attenuation rate is... Let's set it to 4. The network structure diagram of the CA mechanism is as follows: Figure 4 As shown, the time branch employs a bidirectional Mamba network to enhance the Mamba network's ability to model two-dimensional time-Doppler spectrogram data. Preferably, its hidden state dimension... Set to 128, the residual connection is used to add the outputs of the spatial and temporal branches to the input of the CA-Mamba block; the prediction head is designed as follows... Figure 2 As shown, it includes a global average pooling layer and a fully connected layer, wherein the fully connected layer contains a linear layer and a softmax layer;

[0058] Training the TWRMamba network includes the following steps:

[0059] S31: The 224×224×3 time-Doppler spectrum is input into the Patch Embedding module. First, it is downsampled to 112×112×3 by the DWConv2d layer. Then, it is divided into several 7×7 patches by the Conv2d layer, and the feature dimension is mapped to 128 dimensions to obtain a feature map of size 16×16×128. Next, the feature map is flattened into a 256×128 sequence, and a learnable absolute position code of size 256×1 is generated by a linear layer. This code is then added to the image sequence using a broadcast mechanism to obtain a 256×128 encoded sequence. The specific formula is as follows:

[0060] ;

[0061] In the formula, This indicates the DWConv2d operation. and These represent the ReLU activation function and the batch normalization operation, respectively. This represents the convolution operation used for block segmentation. Represents a learnable absolute positional encoding;

[0062] Traditional Patch Embedding divides an image into several non-overlapping patches, neglecting inter-patch information interaction during feature learning and ignoring some feature information. Therefore, this invention improves the Patch Embedding module by introducing a DWConv2d layer, enabling the network to integrate information from neighboring patches during feature learning. This allows each patch to contain features from surrounding patches, facilitating inter-patch information interaction. Furthermore, since the Mamba network ignores the positional information of elements in a sequence when processing sequences, and the patches in the time-Doppler spectrogram are arranged in an ordered manner, this invention adds positional encoding to each element in the sequence. This facilitates learning the spatial relationships between elements in the input, making feature extraction easier for the Mamba network.

[0063] S32: The encoding sequence consists of 8 consecutive CA-Mamba blocks in the TWRMamba encoder. The processing involves extracting the encoded sequence via spatial and temporal branches within the CA-Mamba block. The spatial and temporal features are obtained by adding the residual connections in the CA-Mamba block to the encoded sequence X, resulting in a feature vector representing the target information. ;

[0064] The spatial branch uses the encoded sequence generated by the Patch Embedding module. As input, the spatial feature information is considered, and a weighted encoded sequence is output. When converting a two-dimensional data matrix into a one-dimensional sequence representation for feature learning, even with the addition of positional encoding, the loss of spatial positional information is unavoidable. To alleviate this problem, this invention introduces a CA (Carry-Agent) mechanism in the spatial branch to extract and enhance spatial features, thereby improving the model's representational ability. This mechanism effectively preserves spatial positional information while capturing global dependencies and enabling cross-channel feature interaction.

[0065] like Figure 3 , Figure 4 As shown, the encoded sequence is extracted. The steps for spatial features are as follows:

[0066] For the input encoded sequence , , , First, perform layer normalization, then transform its size to... Then, global average pooling is performed along the horizontal and vertical directions, respectively, with the following formula:

[0067] ;

[0068] In the formula, and These represent the results after global average pooling along the horizontal and vertical directions, respectively. Presentation layer normalization operation, This indicates a size transformation operation. , represents the channel index, This step represents the channel dimension; it is to perceive the long-distance dependencies of feature maps from two different spatial directions, thus preserving the positional information of the two different directions.

[0069] Then and Stacked along the channel dimension, the input is a convolutional kernel of size 1×1 with a stride of 1 and a number of kernels. In the Conv2d layer, a BN layer, and a convolutional layer with a Hard-Swish activation function, channel compression is performed to encode positional information in two directions, as shown in the following formula:

[0070] ;

[0071] In the formula, Indicates intermediate features, This represents a two-dimensional convolution operation. Indicating channel attenuation rate, in this invention, Set to 4, This represents the Hard-Swish activation function. This indicates a batch normalization operation;

[0072] Next, Divided into and The system consists of two parts, each processed by a convolutional kernel of size 1×1 with a stride of 1 and a number of kernels. Channel adjustment is performed on the Conv2d layer and the convolutional layer with activation function to restore the channel dimension to [value missing]. The weights in the horizontal and vertical directions are obtained. and The formula is as follows:

[0073] ;

[0074] In the formula, This represents the Sigmoid activation function. and These represent convolution operations in two different directions;

[0075] Finally, the weights and and Perform point-by-point multiplication and transform the result into a one-dimensional sequence to obtain the output of the spatial branch. The formula is as follows:

[0076] ;

[0077] like Figure 3 As shown, the encoded sequence is extracted. The steps for defining the time characteristics are as follows:

[0078] First, the input encoded sequence Perform layer normalization and use a linear layer to obtain two projection vectors. and The number of input and output neurons in the linear layer are respectively and Projection vector and The first dimension of the output tensor of this linear layer is the first dimension of the last dimension of the output tensor. Each component and the last One component;

[0079] Subsequently, vector Its flipped version is processed through two parallel branches, each containing a one-dimensional convolutional layer and an SSM, which respectively produce the forward output. and reverse output The formula is as follows:

[0080] ;

[0081] In the formula, This indicates a convolution kernel size of 4, a stride of 1, and a number of kernels. One-dimensional convolution operation, This represents the operation of reversing the order of elements in a sequence. The representational space state model is the core of the Mamba network, given an input sequence. The formula is as follows:

[0082] ;

[0083] In the formula, and For continuous parameters, and For time-scale parameters Discretized discrete parameters, It is the identity matrix. For the mapping matrix, Represents a structured convolution kernel. This represents the convolution operation. and They represent lengths of The input sequence and the output sequence;

[0084] Finally, the output of the time branch is calculated using the following formula. :

[0085] ;

[0086] In the formula, SiLU represents the SiLU activation function. This indicates element-wise multiplication. This represents a linear layer with the number of input and output neurons being equal to 1. .

[0087] The use of a bidirectional Mamba network in the time branch to extract the temporal features of the input can improve the network's ability to model two-dimensional data. Mamba introduces an innovative selection mechanism that can dynamically adjust the parameters of the State Space Model (SSM) based on the input data. As one of the SSM architectures designed specifically for deep learning, Mamba efficiently captures long-range dependencies with linear computational complexity.

[0088] After obtaining the outputs of the temporal and spatial branches, they are connected to the encoded sequence via residual concatenation. Adding them together yields a feature vector representing the target information. The formula for the final output of a CA-Mamba block is as follows:

[0089] ;

[0090] S33: Use the prediction head to extract the feature vector The results are obtained by converting the data into a category probability distribution.

[0091] The prediction head consists of a global average pooling layer, a linear layer, and a softmax layer in sequence, and finally outputs the recognition result. For an N-class task, the linear layer is set with N output neurons, and the output dimension is consistent with the number of classes. After softmax normalization, the class with the highest probability is the final recognition result.

[0092] Preferably, the batch size during training is set to 32, the loss function is cross-entropy loss, the optimizer is AdamW, the weight decay coefficient is set to 0.05, the initial learning rate is set to 0.0001, and the learning rate is halved every 20 training epochs. Early stopping is used during training: if the recognition accuracy on the validation set does not improve after 20 consecutive epochs, or if the maximum number of training epochs (120 epochs) is reached, training is stopped.

[0093] S4: Use a trained TWRMamba network for through-wall radar human behavior recognition.

[0094] A trained TWRMamba network can be obtained through S1-S3. Then, the trained TWRMamba network can be used to perform human behavior recognition through-wall radar.

[0095] The present invention also provides a through-wall radar human behavior recognition system based on spatiotemporal features, for performing the above-described through-wall radar human behavior recognition method based on spatiotemporal features.

[0096] The following detailed description of the through-wall radar human behavior recognition method based on spatiotemporal features provided by the present invention, with reference to specific embodiments, is as follows:

[0097] The antenna of the UWB radar system was placed on a table 0.9 meters high, close to a 12-centimeter-thick brick wall. Six behavioral actions were measured in an indoor environment from four volunteers: (a) walking, (b) stepping, (c) jogging, (d) bowing, (e) punching, and (f) waving. The UWB radar system used a carrier-free pulse system with a center frequency of 4.3 GHz, a bandwidth of 1.7 GHz, a fast-time sampling frequency of 16.387 GHz, 768 fast-time sampling points, and a slow-time sampling frequency of 200 Hz, with 4000 slow-time sampling points per measurement. During the data collection, each participant performed the six actions within a range of 1–5.26 m from the radar, corresponding to a radar range element index range of [100, 768]. The same action was performed continuously for 20 seconds, with a repetition every 2 seconds. Subsequently, the fixed 20-second radar echo data was divided into ten non-overlapping 2-second intervals, forming 10 samples for each measurement group. The entire dataset contains 2400 samples (10 samples × 10 measurements × 6 behaviors × 4 participants). Finally, for each participant and each behavior, all corresponding samples are divided into training, validation, and test sets in a 3:1:1 ratio, resulting in 1440 training time-Doppler spectra, 480 validation time-Doppler spectra, and 480 test time-Doppler spectra for model training and testing. Each time-Doppler spectra is fixed at 224 × 224 × 3 pixels. All training was performed using PyTorch on a desktop computer equipped with an Intel(R) Xeon(R) Gold 6132 CPU and an NVIDIA GeForce RTX 3090 GPU. The network was trained using the AdamW optimizer with a cross-entropy loss function. The training batch size was set to 32, the weight decay coefficient to 0.05, and the initial learning rate to 0.0001, which was halved every 20 training epochs. The training process lasts a maximum of 120 epochs. If the accuracy on the validation set does not improve for 20 consecutive epochs, the training will be terminated early. The model with the best performance on the validation set will be used for evaluation.

[0098] Figure 5 The confusion matrices for five action test samples are shown, where the horizontal axis represents the predicted label and the vertical axis represents the true label. Although some misidentifications occur due to the similarity of features between different actions, the proposed network model can still correctly identify most samples, maintaining excellent overall recognition performance. Figure 6 The model's accuracy, precision, recall, and F1 score on the test set were further demonstrated, with an accuracy of 94.38%. All metrics indicate that the model has high reliability and stability.

[0099] This invention provides a method and system for human behavior recognition using through-wall radar based on spatiotemporal features. Employing a TWRMamba network model and utilizing a CA-Mamba structure, it effectively models the spatiotemporal features of the time-Doppler spectrum, enabling high-precision human behavior recognition even in occluded environments. Furthermore, this method introduces an information exchange mechanism between image patches in the Patch Embedding module, which not only enhances the network's feature representation capabilities but also significantly reduces computational complexity. This method effectively improves the overall recognition performance and computational efficiency of the system while maintaining high recognition accuracy.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A through-the-wall radar human behavior recognition method based on space-time features, characterized in that, Comprise: S1: build a UWB radar data acquisition platform and use the platform to measure the echo of various human behaviors behind the wall, obtain the echo sample signal, then stack the echo sample signal into an echo matrix according to rows, and process the echo matrix to obtain the corresponding time-Doppler spectrum; S2: divide the time-Doppler spectrum into a training set, a validation set and a test set according to a proportion; S3: build a TWRMamba network for through-wall radar human behavior recognition and use the training set, the validation set and the test set to train, validate and test the TWRMamba network, obtain the trained TWRMamba network, wherein the TWRMamba network comprises a Patch Embedding module, a TWRMamba encoder and a prediction head arranged in sequence, wherein the Patch Embedding module includes a two-dimensional deep convolutional layer, a two-dimensional convolutional layer and a position encoding embedding operation; the TWRMamba encoder is stacked by a plurality of CA-Mamba blocks, the CA-Mamba block includes a spatial branch, a time branch and a residual connection, the spatial branch introduces a coordinate attention mechanism, the time branch adopts a bidirectional Mamba network, and the residual connection is used to add the outputs of the spatial branch and the time branch to the input of the CA-Mamba block; the prediction head includes a global average pooling layer and a fully connected layer, the fully connected layer includes a linear layer and a Softmax layer; S4: use the trained TWRMamba network to perform through-wall radar human behavior recognition.

2. The method according to claim 1, wherein the method is characterized in that, In S1, stacking the echo sample signal into an echo matrix according to rows comprises: Obtaining a through-wall radar human behavior echo sampling signal, wherein the echo sampling signal corresponding to the first pulse and the first sampling point is expressed as , , , , , , , , wherein M represents the number of slow time sampling, that is, the number of transmitted pulses, represents the sampling interval of the pulse, and represents the number of fast time sampling, that is, the number of sampling points in each pulse. The After stacking by row, an echo matrix of is formed.

3. The method of claim 2, wherein the method of human behavior recognition based on spatio-temporal features through wall radar, further comprises: In S1, processing the echo matrix to obtain the corresponding time-Doppler spectrum comprises: S11: preprocessing the echo matrix to weaken the interference of wall reflection, static clutter and background noise, obtaining matrix ; S12: perform a Hilbert transform on each row of the matrix and convert the matrix to a complex-valued matrix where denotes the Hilbert transform result of each row, denotes the imaginary unit;​ S13: cropping the fast time dimension of the complex matrix to obtain a region of interest of human behavior, and then coherently superimposing the slow time signals corresponding to the region of interest to obtain a superimposed slow time signal wherein, is a distance unit index range of the region of interest. S14: performing a short-time Fourier transform on the slow-time signal to obtain a time-Doppler matrix wherein, and are the indices of frequency and time, respectively, and then mapping the time-Doppler matrix to a three-channel RGB image to obtain a time-Doppler spectrogram and unifying the size.

4. The method of claim 3, wherein the method of human behavior recognition based on the spatio-temporal features is characterized by, In S1, the preprocessing is mean subtraction filtering processing along the slow time dimension of the echo matrix.

5. The method of claim 1, wherein the method of human behavior recognition based on spatio-temporal features through wall radar, further comprises: In S3, training the TWRMamba network comprises the following steps: S31: input the time-Doppler spectrogram to the Patch Embedding module, first, downsample through a two-dimensional deep convolutional layer, then, divide into multiple patches through a two-dimensional convolutional layer, and map the feature dimension to a set dimension to obtain a feature map, then, flatten the feature map and add position encoding to obtain an encoded sequence ; S32: a plurality of successive CA-Mamba blocks in the TWRMamba encoder process the encoding sequence , wherein the spatial branch and the temporal branch in the CA-Mamba block are respectively used to extract the spatial feature and the temporal feature of the encoding sequence , and the residual connection in the CA-Mamba block is used to add the spatial feature and the temporal feature to the encoding sequence , so as to obtain a feature vector representing target information ; Among them, the spatial branch introduces a coordinate attention mechanism, and the spatial branch uses the encoding sequence generated by the Patch Embedding module As input, pay attention to the spatial feature information therein, and output the weighted encoding sequence; the time branch uses a bidirectional Mamba network to extract the time features of the input; S33: convert the feature vector into a class probability distribution using the prediction head to obtain a recognition result. convert the feature vector into a class probability distribution using the prediction head to obtain a recognition result.

6. The method of claim 5, wherein the human behavior recognition method based on the spatio-temporal features is characterized by, In S32, the step of extracting the spatial features of the coding sequence is as follows: the spatial features of the coding sequence For the input encoded sequence , first, a layer normalization operation is performed, and then its size is transformed to , to obtain , then, global average pooling is performed along the horizontal direction and the vertical direction respectively, to obtain the global average pooled results and ; Subsequently, the input is stacked along the channel dimension, input into a convolutional layer, compressed in the channel dimension, and encoded with position information in two directions to obtain intermediate features and Stacked along the channel dimension, input into a convolutional layer, compressed in the channel dimension, and encoded with position information in two directions to obtain intermediate features wherein, , represents the channel attenuation rate; Next, the following is obtained is divided into two parts along the horizontal direction and the vertical direction, each part is adjusted in channel by a convolutional layer, and restored to the channel dimension , to obtain the horizontal direction and the vertical direction weight and ; Finally, the weights and are pointwise multiplied with and the result is transformed into a one-dimensional sequence, resulting in the output of the spatial branch .

7. The method of claim 5, wherein the human behavior recognition method based on the spatio-temporal features is characterized by, In S32, the step of extracting the time characteristics of the coding sequence is as follows: the time characteristics of the coding sequence First, the input encoded sequence is layer normalized and two projection vectors and are obtained using linear layers; Subsequently, the vector and its flipped version are processed through two parallel branches, resulting in forward output and backward output respectively, where each branch contains a one-dimensional convolutional layer and an SSM. Finally, the output of the time branch is calculated using the following equation : ； In the formula, SiLU represents a SiLU activation function, represents element-wise multiplication, represents a linear layer and the input and output neuron numbers of the linear layer are both .

8. The method of claim 1, wherein the method of human behavior recognition based on spatio-temporal features through a wall radar, further comprises: When training the TWRMamba network, the batch size is set to 32, the loss function adopts cross-entropy loss, the optimizer is AdamW, the weight decay coefficient is set to 0.05, the initial learning rate is set to 0.0001, and the learning rate is halved every 20 training rounds.

9. A through-the-wall radar human activity recognition system based on spatio-temporal features, characterized in that, A through-wall radar human behavior recognition method based on spatio-temporal features is provided.

Images