An adaptive micro-doppler corner point feature extraction method and device

Abstract

The application discloses a kind of self-adapting micro-doppler corner point feature extraction method and device, comprising: it is proposed to use micro-doppler corner point feature to realize through-wall radar human behavior recognition, and gives a kind of self-adapting corner point feature extraction method based on Gauss difference convolution filter and deformable convolution network;Micro-doppler corner point feature is defined as the point that gray scale sharply changes in different directions in radar square distance-time image and square doppler-time image, reflects the inflection point of human body limb node's motion trajectory curve, stationary point, curve intersection and boundary;The corner point feature extraction method proposed uses Gauss difference convolution filter to extract micro-doppler corner point supervision label on simulation data, then trains μD-CornerDet model using these labels;Inference stage, only μD-CornerDet model is used to obtain corner point feature map for measured data;The application verifies the effectiveness and robustness of the proposed method through numerical simulation and measurement.

Description

Technical Field

[0001] This invention relates to the field of radar signal processing, specifically to an adaptive micro-Doppler corner feature extraction method and apparatus. Background Technology

[0002] Ultra-wideband (UWB) through-the-wall radar (TWR) utilizes the penetrating power of low-frequency electromagnetic waves to detect human targets behind walls, and is widely used in disaster relief and other fields. Human behavior recognition is one of the most popular research topics in these fields. However, limitations imposed by walls on electromagnetic wave propagation, such as attenuation, refraction, and multipath effects, lead to significant distortion of echo signals and difficulties in feature extraction. Therefore, directly transplanting existing classification models based on behavior recognition results in a significant decrease in recognition accuracy.

[0003] For the UWB TWR human activity recognition task, current research in academia has gone through two development stages: classical methods and intelligent methods. Classical methods use function fitting or matrix factorization algorithms to convert images into feature vectors, and then employ statistical decision methods for classification. These methods define relatively simple experimental scenarios but have strong interpretability. Intelligent methods, on the other hand, are mainly based on deep learning techniques, including Autoencoder Networks (AEN), Residual Convolutional Neural Networks (Res-CNN), Long Short-Term Memory Networks (LSTM), Multi-Layer Perceptrons (MLP), and Probability Graph Models (PGM).

[0004] Previous work investigated various network models based on feature representation learning, achieving either more refined micro-Doppler feature extraction or faster inference speed for human behavior recognition. Due to their powerful parameter estimation capabilities, neural network-based algorithms are often the best choice for human behavior recognition tasks. However, existing models directly use range-time maps (RTMs) or doppler-time maps (DTMs) for training and validation, resulting in poor interpretability of feature extraction and consequently reduced accuracy, generalization ability, and robustness.

[0005] Based on the above research, existing methods mainly rely on the amplitude and timing information of the received signal, but these methods are not accurate or complete in extracting target behavior in complex scenes. To address this issue, we propose an innovative feature extraction method that incorporates micro-Doppler corner features. Our method is based on the weak Doppler frequency shift information in the signal received by through-wall radar and utilizes techniques such as ultra-wideband radar waveform transmission and reception, signal preprocessing, data preprocessing, imaging reconstruction, and image interpretation to extract the corner features of the target. By analyzing the micro-Doppler corner features of the target's human behavior, we can achieve more accurate and reliable behavior recognition. Summary of the Invention

[0006] In view of this, the present invention provides an adaptive micro-Doppler corner feature extraction method and device, which can solve the adverse effects of walls on electromagnetic signal propagation, such as attenuation, refraction and multipath effects, significant distortion of ultra-wideband through-wall radar echo signals, a significant decrease in the feature extraction accuracy of human behavior recognition, poor robustness against low signal-to-noise ratio image input, and very challenging technical problems in system deployment.

[0007] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0008] A micro-Doppler corner feature extraction method includes:

[0009] Step S1: Acquire the raw data of human behavior echoes from through-wall radar, and preprocess the raw data; the preprocessing includes clutter and noise suppression, image contrast enhancement, and image coordinate axis transformation to obtain the processed radar squared range-time image R. 2 TM and squared Doppler one-time image D 2 TM;

[0010] Step S2: Use the difference of Gaussian convolutional filter to extract the micro-Doppler corner point supervision labels on the simulated radar squared distance-time image and squared Doppler-time image. Then use the supervision labels to train the μD-CornerDet model, which is jointly constructed based on a deformable convolutional network, a domain-adaptive deformable convolutional network, a feature pyramid network, and a region-learnable global attention module.

[0011] Step S3: Inference stage, the μD-CornerDet model is used to predict the radar squared distance-time image and squared Doppler-time image of the measured data to obtain corner feature maps.

[0012] Preferably, in step S1, after mixing and low-pass filtering, the baseband time-domain echo signal of human motion from the through-wall radar is obtained. The received echo signal is converted into a one-dimensional range profile by performing a discrete inverse Fourier transform on the echo signal. By stitching the range profile along the slow-time dimension, channel correction, clutter suppression, and noise suppression are performed respectively to obtain the range-time matrix φ. r (t s (n); based on the distance-time matrix φ r (t s From n), we obtain the radar range-time map (RTM) and the Doppler-time map (DTM), and then obtain the radar squared range-time image (R). 2 TM and squared Doppler one-time image D 2 TM.

[0013] Preferably, in step S1, the radar squared range-time image R is obtained. 2 TM's methods include: by adjusting matrix φ r (t s Taking the absolute value of (n, ..., n) and converting it to a pseudo-color image yields the RTM. An adaptive histogram equalization algorithm is used to enhance the contrast of the RTM. Then, interpolation is used to stretch the distance dimension of the RTM from the linear axis to the square axis, resulting in R... 2 TM; where, during the interpolation process, an interval filling algorithm is used to fill empty distance cells on the vertical coordinate with gray values ​​in sequence that contain pixel information.

[0014] Preferably, in step S1, the squared Doppler-time image D is obtained. 2 TM's methods include: along the fast time dimension, the distance-time matrix φ r (t s The summation of n) is performed, and a short-time Fourier transform is executed on the slow-time dimension to generate a DTM. Then, contrast enhancement, pseudo-color mapping, and vertical axis stretching are applied to the DTM to generate a D 2 TM, where, during the stretching process, the image is cropped into two half-matrices along the zero Doppler axis and then filled with gaps separately.

[0015] Preferably, the method for extracting micro-Doppler corner supervision labels from simulated radar squared range-time images and squared Doppler-time images using a Gaussian difference convolution filter includes:

[0016] Assume the input R 2 TM and D 2 TM is uniformly represented by the variable I(x, y), where x and y represent the horizontal and vertical image spatial scales, respectively; the Gaussian kernel is defined as follows:

[0017]

[0018] Where σ is the standard deviation of the Gaussian kernel; accordingly, the difference-of-gaussian convolution kernel is expressed as:

[0019]

[0020] Where k = 1, 2, 3, ..., K, This represents the number of image scaling pyramid layers for a given scale set. Represents the Laplace operator;

[0021] Gaussian difference convolution is performed on I(x,y), and nonmaximum suppression is applied across multiple scale sets:

[0022]

[0023] Here, * represents the image convolution operator. For each image within a certain scale set, the center point is selected as the coordinates of the corner point within the rectangular window corresponding to the maximum value. By traversing various scale sets, a series of corner point coordinates are obtained under different image scale sets, which serve as training supervision labels for the μD-CornerDet module.

[0024] Better, in R 2 TM and D 2 In TM, the obtained micro-Doppler corner features exclude corner points within the following coordinate ranges: (1) R 2 TM[0:0.09m] 2 ,:],(2)D 2 TM[-4.5: 4.5Hz] 2 According to the above criteria, select and store the corner features in valid two-dimensional coordinate form from the Gaussian difference convolution results.

[0025] Preferably, the method for training the μD-CornerDet model using supervised labels includes:

[0026] The μD-CornerDet module uses ResNet-101 as the backbone network, downsampled feature pyramid networks and upsampled feature pyramid networks as the connection parts, and fully connected layers as the heads for classification and regression. The specific scheme is as follows:

[0027] The measured R input to the backbone network 2 TM and D 2 The TM image is transformed into a 3-channel image using pseudo-color mapping and then uniformly scaled to 224×224. The input image is then processed through a 7×7 convolutional layer with 64 channels to obtain the input feature map.

[0028] The feature maps are processed through convolutional layers of sizes 1×1, 3×3, and 1×1, with 64, 64, and 256 channels respectively. The processed result is added to the input feature map to obtain the output feature map, which serves as a first-level processing module in ResNet-101. This first-level processing module is repeated three times to obtain a second-level processing unit in ResNet-101. Next, the second-level processing units are stacked sequentially, with the number of convolutional channels in each layer of the processing unit doubling the number of convolutional channels in the previous stack, and this is repeated four times. Finally, average pooling layers are sequentially connected to obtain the backbone network.

[0029] In the upsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order. Each layer is obtained by concatenating the input feature maps of the corresponding secondary processing unit along the channel dimension. Similarly, in the downsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order. Each layer is obtained by concatenating the input feature maps of the corresponding secondary processing unit in reverse order along the channel dimension. In the upsampling feature pyramid network, the image of each layer is processed by a deformable convolutional network (DCN), and the processing result of each layer is directly summed with the input of the next layer. In the downsampling feature pyramid network, the image of each layer is also processed by a deformable convolutional network (DCN), and the processing result of each layer is directly summed with the input of the previous layer.

[0030] Next, the outputs of each layer of the downsampled feature pyramid network and the upsampled feature pyramid network are processed by the learnable global attention module LRGA, and then the processing results of the corresponding two layers are concatenated through inter-layer channel dimensions to obtain a layer of features, thus forming a concatenated feature pyramid. Inter-layer connections are made through the domain adaptive deformable convolutional network module TDCN. The images from the upsampled feature pyramid network and the downsampled feature pyramid network in each layer are processed by TDCN and summed layer by layer, thereby mapping the concatenated feature pyramid to the head module for classification and two-dimensional coordinate regression.

[0031] Preferably, the classification and 2D coordinate regression of the head module are implemented by fully connected layers; in the regression branch, the number of output nodes corresponds to twice the number of micro-Doppler corner points required, and its output contains two covariates: the horizontal and vertical coordinates (x, y) of the corner point; the output of the classification branch is a binary value, where 1 indicates that it is a desired corner point, and 0 indicates that it is an unwanted corner point filtered out by the difference-of-gaussian convolution module; during training, the parameters of the fully connected layers of both branches are optimized simultaneously;

[0032] The classification branch is trained using the cross-entropy loss function; the regression branch is trained using the minimum mean squared error loss function.

[0033] Ideally, during the inference process, the network is iterated layer by layer sequentially, while simultaneously providing R. 2 TM and D 2 The coordinates and types of corner points on the TM are used to retain corner points with a category branch output of 1 and remove corner points with a category branch output of 0.

[0034] An apparatus for micro-Doppler corner feature extraction, characterized in that it comprises: an image processing module, a feature extraction module, and a prediction module;

[0035] The image processing module is used to implement the method in step 1;

[0036] The feature extraction module is used to implement the method in step 2;

[0037] The prediction module is used to implement the method in step 3.

[0038] The present invention has the following beneficial effects:

[0039] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which can achieve a high accuracy rate in micro-Doppler corner feature extraction. Furthermore, because its sub-module construction method is based on supervised-unsupervised models, multi-scale networks, and other technologies, the overall training and prediction of the algorithm are robust to low signal-to-noise ratio through-wall radar data, providing possibilities for system deployment and human behavior feature extraction in complex scenarios.

[0040] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which differs from traditional through-wall radar human motion feature extraction and behavior recognition technologies. This technology does not require direct contact with the human body; it can recognize human behavior through through-wall radar signals transmitted via a wall. Compared to traditional methods such as cameras that require direct observation of the human body, this non-contact characteristic offers advantages in terms of privacy protection and security.

[0041] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which differs from traditional through-wall radar human motion feature extraction and behavior recognition technologies. The through-wall radar signal used in this technology has strong penetrating power, capable of passing through thick non-metallic obstacles such as double-layered hollow brick walls, enabling the monitoring and recognition of human behavior behind walls. This allows the technology to work effectively in complex urban building environments, unaffected by wall obstructions.

[0042] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which differs from traditional through-wall radar human motion feature extraction and behavior recognition technologies. This technology, based on micro-Doppler corner feature-based through-wall radar human behavior recognition, can monitor and identify human behavior in real time. The ultra-wideband characteristics of through-wall radar signals enable it to capture rapidly changing human movements with high accuracy and stability, unaffected by environmental factors such as lighting and weather.

[0043] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which differs from traditional through-wall radar human motion feature extraction and behavior recognition technologies. Because through-wall radar human behavior recognition technology based on micro-Doppler corner features mainly relies on the feature extraction and analysis of radar signals, it is less affected by factors such as light, shadow, and color compared to sensor technologies such as image processing, thus exhibiting higher accuracy and robustness.

[0044] This invention discloses an adaptive micro-Doppler corner feature extraction method and apparatus, which differs from traditional through-wall radar human motion feature extraction and behavior recognition technologies. Compared to visual sensors such as cameras, this technology does not involve image and facial features in human behavior recognition, focusing instead on the detection and analysis of human motion. This characteristic can, to some extent, reduce the infringement of personal privacy and protect users' privacy rights. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the adaptive micro-Doppler corner feature extraction method provided by the present invention;

[0046] Figure 2 This is a schematic diagram of the framework of an adaptive micro-Doppler corner feature extraction method provided by the present invention;

[0047] Figure 3 This is a schematic diagram of the Gaussian difference convolution module provided by the present invention;

[0048] Figure 4 This is a schematic diagram of the μD-CornerDet module provided by the present invention;

[0049] Figure 5 This is a schematic diagram of an adaptive micro-Doppler corner feature extraction device provided by the present invention;

[0050] Figure 6 This is a schematic diagram of the through-wall radar data collected by the present invention and the feature extraction effect;

[0051] Figure 7 The robustness test curve of the adaptive micro-Doppler corner feature extraction method provided by the present invention is shown. Detailed Implementation

[0052] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0053] Inspired by research findings in optical image processing, this invention proposes a novel approach for human behavior recognition using micro-Doppler corner features in through-wall radar. The sparse corner representation of micro-Doppler features is of significant value for human behavior recognition, and utilizing this feature enhances model interpretability. Micro-Doppler corner features reflect the inflection points, stationary points, curve corners, and boundaries of the trajectory curves of human limb nodes. In the field of optical image processing, there are some research results on corner detection, including the Harris detector, Moravec detector, Forstner detector, and ECFRNet. However, the difference in gray-level gradient distribution between optical and radar images leads to low accuracy in corner detection when these methods are directly applied to radar images. Based on the above analysis, this invention proposes an adaptive micro-Doppler corner extraction method based on the Difference of Gaussian (DoG) filter and Deformable Convolutional Network (DCN). By using the sparse corner representation of micro-Doppler features, feature selection and dimensionality reduction are simultaneously achieved. Furthermore, the present invention verifies the effectiveness and robustness of the proposed method through numerical simulation and field tests, and provides an adaptive micro-Doppler corner feature extraction system deployment method and apparatus.

[0054] An adaptive micro-Doppler corner feature extraction method includes:

[0055] Step S1: Obtain the raw data of human behavior echo from through-wall radar, and preprocess the raw data; the preprocessing includes clutter and noise suppression, image contrast enhancement, and image coordinate axis transformation to obtain the processed radar squared range-time image and squared Doppler-time image;

[0056] Step S2: Use the difference of Gaussian convolutional filter to extract micro-Doppler corner supervision labels on the simulated radar squared range-time image and squared Doppler-time image, and then use these labels to train the μD-CornerDet model, which is jointly constructed based on a deformable convolutional network, a domain-adaptive deformable convolutional network, a feature pyramid network, and a region-learnable global attention module.

[0057] Step S3: Inference stage, only the μD-CornerDet model is used to predict the radar squared range-time image and squared Doppler-time image of the measured data to obtain the corner feature map.

[0058] Step S1 specifically includes the following steps:

[0059] S11. After mixing and low-pass filtering, the baseband time-domain echo signal of human motion in the through-wall radar is obtained. To achieve signal transmission and reception, the proposed ultra-wideband through-wall radar system employs Step Frequency Continuous Waveform (SFCW) technology. By mixing the echo signal with the local transmitted signal and performing low-pass filtering, the baseband echo signal is obtained.

[0060]

[0061] Where k = 0, 1, 2, ..., K-1 represents the number of frequency points, f0 is the carrier frequency, Δf is the frequency step, and T is the signal period. f τ is the sampling period, τ is the two-way delay, and t is the sampling period. f It is a fast-time variable, and rect[] is a rectangular window function with zero mean and unit length.

[0062] S12, regarding S r (t f Performing an Inverse Discrete Fourier Transform (IDFT) converts the received echo signal into a one-dimensional range profile.

[0063]

[0064] Where n is the number of distance units.

[0065] S13, via the slow time dimension t s By stitching together the range profile and performing channel correction, clutter suppression, and noise suppression respectively, the range-time matrix φ was obtained. r (t s (n).

[0066] S14, Based on the distance-time matrix φ r (t s From n), radar range-time maps (RTM) and Doppler-time maps (DTM) can be obtained, which are suitable for corner feature extraction. Specific methods for generating RTM and DTM include:

[0067] By analyzing matrix φ r (t sTaking the absolute value of (n, ..., n) and converting it to a pseudo-color image yields the RTM. An adaptive histogram equalization algorithm is used to enhance the contrast of the RTM. Then, interpolation is used to stretch the distance dimension of the RTM from the linear axis to the square axis, resulting in R... 2 TM. During interpolation, an interval filling algorithm is used to fill empty distance cells on the vertical coordinate with gray values ​​in sequence that contain pixel information.

[0068] DTM is obtained by analyzing the distance-time matrix φ along the fast time dimension. r (t s The expression is generated by summing (n, ..., n) and performing a Short-Time Fourier Transform (STFT) on the slow-time dimension. This is then compared with the generated R... 2 The algorithm for TM is similar, D 2 TM is also generated through contrast enhancement, pseudo-color mapping, and vertical axis stretching. However, unlike the above operations, during the stretching process, the image needs to be cropped into two half-matrices along the zero Doppler axis and then filled with gaps separately.

[0069] R was obtained using the method described above. 2 TM and D 2 TM, which will serve as the expected input images for both the supervisory and supervised ends of the proposed corner feature extraction method.

[0070] In step S2, the input to the corner feature extraction method is R. 2 TM and D 2 The TM module outputs corner feature maps. During training, a differential Gaussian convolutional filtering module is used to infer corner labels on the simulated dataset. These labels are then used to train the μD-CornerDet module. During the prediction phase, only μD-CornerDet is used to output corner feature maps from the measured data.

[0071] Preferably, this invention uses differential Gaussian convolution combined with non-maximum suppression (NMS) to generate supervision labels. To simplify the problem, assume the input R... 2 TM and D 2 TM is uniformly represented by the variable I(x, y), where x and y represent the horizontal and vertical image spatial scales, respectively. The Gaussian kernel is defined as follows:

[0072]

[0073] Where σ is the standard deviation of the Gaussian kernel. Therefore, the difference-of-gaussian convolution kernel can be expressed as:

[0074]

[0075] Where k = 1, 2, 3, ..., K, This represents the number of image scaling pyramid layers for a given scale set. This represents the Laplace operator.

[0076] This invention performs Gaussian difference convolution on I(x, y) and nonmaximum suppression on multiple scale sets:

[0077]

[0078] Here, * represents the image convolution operator. For each image within a certain scale set, the center point is selected as the coordinates of the corner points within the rectangular window corresponding to the maximum value. By traversing various scale sets, a series of corner point coordinates are obtained under different image scale sets. However, in R... 2 TM and D 2 In TM, the obtained micro-Doppler corner features need to exclude the following coordinate ranges: (1) R 2 TM[0:0.09m] 2 ,:],(2)D 2 TM[-4.5: 4.5Hz] 2 According to the above criteria, effective two-dimensional coordinate features are selected and stored from the Gaussian difference convolution results. These two-dimensional coordinate features will serve as training supervision labels for the μD-CornerDet module.

[0079] Preferably, the μD-CornerDet module proposed in this invention uses ResNet-101 as the backbone network, a feature pyramid network as the connection part, and a fully connected layer as the head for classification and regression. The specific scheme is as follows:

[0080] This invention uses ResNet-101 as the backbone network from R 2 TM and D 2 Features are extracted from the TM. The backbone network is input with the measured R... 2 TM and D 2 The TM image is transformed into a 3-channel image using pseudo-color mapping and uniformly scaled to 224×224. The input image is then processed through a 7×7 convolutional layer with 64 channels to obtain the input feature map.

[0081] The feature maps are processed through convolutional layers of sizes 1×1, 3×3, and 1×1, with 64, 64, and 256 channels respectively. The processed result is added to the input feature map to obtain the output feature map, which serves as a first-level processing module in ResNet-101. This first-level processing module is repeated three times to obtain a second-level processing unit in ResNet-101. Next, the second-level processing units are stacked sequentially, with the number of convolutional channels in each layer of the processing unit doubling the number of convolutional channels in the corresponding layer of the previous stack, and this is repeated four times. Finally, average pooling layers are sequentially connected to obtain the backbone network.

[0082] To describe micro-Doppler features at different scales, this invention also designs downsampling and upsampling feature pyramid networks, and connects them to the intermediate layers in the backbone network using a 3×3 deformable convolutional network module. Specifically: in the upsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order, and each layer is obtained by concatenating the input feature maps of the corresponding secondary processing units along the channel dimension. Similarly, in the downsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order, and each layer is obtained by concatenating the input feature maps of the corresponding secondary processing units in reverse order along the channel dimension. In the upsampling feature pyramid network, the image of each layer is processed by a deformable convolutional module (DCN), and the processing result of each layer is directly summed with the input of the next layer. In the downsampling feature pyramid network, the image of each layer is also processed by a deformable convolutional module (DCN), and the processing result of each layer is directly summed with the input of the previous layer. Therefore, in upsampled feature pyramid networks, image information is passed towards higher semantic layers, while in downsampled feature pyramid networks, image information is passed towards lower semantic layers.

[0083] Next, this invention utilizes a Region Learnable Global Attention (LRGA) module to achieve effective focusing of point cloud features in dense small target detection scenarios by interpolating layer by layer between feature pyramid networks. Specifically, the outputs of each layer of the downsampled feature pyramid network and the upsampled feature pyramid network are processed by the LRGA module, and then the processing results of the corresponding two layers are concatenated through inter-layer channel dimensions to obtain a feature layer, thus forming a concatenated feature pyramid. Inter-layer connections are made through a Domain Adaptive Deformable Convolutional Network (TDCN) module. In each layer, the images from the upsampled feature pyramid network and the downsampled feature pyramid network are processed by the TDCN and summed layer by layer. This enables the concatenated feature pyramid to be mapped to the head of another feature pyramid that can be used for both classification and regression tasks.

[0084] Preferably, the proposed μD-CornerDet module header contains two branches for classification and 2D coordinate regression, respectively, both branches consisting of fully connected layers. In the regression branch, the number of output nodes corresponds to twice the number of desired micro-Doppler corner points, and its output includes two covariates: the horizontal and vertical coordinates (x, y) of the corner point. The output of the classification branch is a binary value, where 1 indicates a desired corner point, and 0 indicates an unwanted corner point filtered out by the difference-of-Gaussian convolution module. During training, the parameters of the fully connected layers in both branches are optimized simultaneously. During inference, the network iterates layer by layer, simultaneously providing the required R0. 2 TM and D 2 The coordinates and types of corner points on the TM are used to retain corner points with a category branch output of 1 and remove corner points with a category branch output of 0.

[0085] Preferably, when training the μD-CornerDet module, the training of the classification branch is implemented using the cross-entropy loss function. Let the output of μD-CornerDet be... The label is y i If ∈{0,1}, then:

[0086]

[0087] Where B is the batch size (BS) for network training.

[0088] Preferably, the regression branch is trained using the minimum mean square error (MSE) loss function, i.e.:

[0089]

[0090] Where, p ij These are the actual location coordinates. R represents the predicted location coordinates using μD-CornerDet. B is the batch size during training, and T is the number of corner points in each sample. During the prediction process, R... 2 TM and D 2 The TM module directly inputs the pre-trained μD-CornerDet module to generate a corner feature map. The resulting corner feature map is a sparse two-dimensional point cloud image that contains only the required micro-Doppler corner features, while the data at other non-corner feature locations are all 0.

[0091] The adaptive micro-Doppler corner feature extraction method provided by this invention includes:

[0092] Image processing module: configured to acquire the raw image of human behavior echo from through-wall radar, and preprocess the raw image; the preprocessing includes clutter and noise suppression, contrast enhancement, and coordinate axis stretching to obtain the processed image;

[0093] Feature extraction module: Configured to extract micro-Doppler corner supervision labels from simulation data using a difference-of-Gaussian convolutional filter, and then use these labels to train a μD-CornerDet model. This model is constructed based on a deformable convolutional network, a domain-adaptive deformable convolutional network, a feature pyramid network, and a region-learnable global attention module. During the inference phase, only the μD-CornerDet model is used to acquire corner feature maps from the measurement data.

[0094] Prediction module: Configures corner features to be extracted from simulated radar squared distance-time images and Doppler-time images through a supervised-supervised model framework, and uses them for direct inference prediction of measured data. It also integrates multi-scale image information to generate a corner feature dataset.

[0095] The through-wall radar proposed in this invention is a device that uses radar technology to detect objects behind walls. Through-wall radar can detect the movement and behavioral characteristics of the human body through non-metallic obstacles such as walls and buildings, possessing significant research value and application potential. Through-wall radar can acquire the movement and behavioral characteristics of the detected human body without contact. Compared to traditional methods such as surveillance cameras, through-wall radar offers better privacy protection, avoiding infringement on personal privacy. Through-wall radar can also penetrate walls and other obstacles to observe the human body behind them. This is crucial for certain special environments or tasks, such as fire rescue, as it can provide real-time information on the human body's location and behavior, helping decision-makers make accurate judgments. Furthermore, through-wall radar can acquire real-time information on the human body's position, speed, and posture, which is highly valuable for tasks requiring real-time monitoring and tracking, such as personnel positioning and health monitoring. To conduct experiments on human behavioral feature extraction using through-wall radar, the following conditions are required: First, select a suitable through-wall radar device, such as a radar system based on ultra-wideband (UWB) or millimeter-wave technology. These devices can penetrate walls and acquire the human body's movement characteristics. Secondly, a relatively isolated experimental site needs to be selected to simulate the application scenario of through-wall radar in a real environment. Obstacles such as walls and furniture need to be set up in the site, and moving human targets should be introduced for detection by the through-wall radar. Finally, a suitable data acquisition system needs to be built, including the connection between the radar equipment and the computer, and software for data storage and processing. This system can be used to receive and store the data acquired by the through-wall radar and perform subsequent signal processing and behavioral feature extraction. Based on this, we conducted numerical simulations and experiments to verify the effectiveness and robustness of the proposed method. Corner extraction methods used for comparison include the Harris-Moravec detector, the Forstner detector, and ECFRNet.

[0096] like Figure 5 As shown, the prototype ultra-wideband through-wall radar developed in this invention includes a transmitter and a receiver. The distance between the transmitter and receiver is 0.15m, and the height above the ground is 1.5m. The transmitter signal is swept within the range of 0.5-2.5GHz. The sampling window is approximately 4s, containing 1024 echoes. The wall thickness is 0.12m, and the range of human movement is 1-4m from the radar. The simulation parameters are consistent with the measured parameters. In the simulation and measured experiments, the training dataset contains 3200 samples, and the validation dataset contains 800 samples.

[0097] like Figure 6 As shown, taking walking activities as an example, the first and second rows display R 2Images of corner features extracted by the Difference-of-Gaussian convolution module and the corner features extracted by the μD-CornerDet module in TM. The corner features extracted by μD-CornerDet can effectively represent R. 2 The distance trajectory of human movement in TM. In contrast, when the human movement speed is high, the corner features extracted by the difference of Gaussian convolution module have the problem of missing information. Compared with μD-CornerDet, this module shows weaker detection performance. Figure 6 The third and fourth lines in the text show D 2 Corner feature images extracted by the Difference of Gaussian Convolution module and the corner feature images extracted by the μD-CornerDet module in TM. The corner features extracted by μD-CornerDet can effectively represent D 2 The velocity trajectory of human motion in TM. In contrast, corner features extracted by the difference-of-Gaussian convolution module tend to be located at the intersection of the velocity trajectory and the zero Doppler axis, resulting in a higher false alarm rate. The inference results using only the difference-of-Gaussian convolution module show that this detector cannot well characterize the details of human motion. This problem is somewhat improved in the inference results of μD-CornerDet. This is because the network learns from a large amount of data to achieve joint estimation of multiple adjacent echo information of interference.

[0098] Preferably, such as Figure 7 As shown, this invention utilizes corner detection accuracy to evaluate the performance of the proposed method. Corner detection accuracy can be calculated in the following way:

[0099]

[0100] Here, Total represents the total number of corner points detected in the image. Tp is defined as the number of corner points falling within the upper and lower envelope constraints of the image. On simulation and experimental datasets, it is clear that the network-based method can extract corner points more accurately. For the simulation dataset, the proposed method performs well in R... 2 TM and D 2 The accuracy on TM is 96.7%, outperforming other methods. For the Harris-Moravec and ECFRNet detectors, their accuracy on R... 2 The detection performance on TM is better than that on D. 2 TM. For the experimental dataset, the proposed method performs well in R. 2 The accuracy on TM is 100.0%, achieving optimal detection accuracy. However, on D... 2 The detection accuracy on the TM is only 86.7%, the same as the Forstner detector.

[0101] like Figure 7As shown, for R 2 TM and D 2 TM incorporates Gaussian noise with varying signal-to-noise ratios (SNR). The accuracy shown in the figure is R. 2 TM and D 2 The average detection accuracy on TM. On both simulation and experimental datasets, the network-based method exhibits better noise robustness than the classical method. The proposed method shows the smallest decrease in detection accuracy as the signal-to-noise ratio decreases, thus demonstrating higher robustness compared to other methods.

[0102] The verification process of this invention encompasses the entire signal path from the prediction module feedback to the signal and data processing module. This step improves the interpretability of the invention during training and ensures its convergent solvability.

[0103] Compared with existing technologies, this invention has significant innovations in the following aspects:

[0104] The invention introduces micro-Doppler corner features: by utilizing the weak Doppler frequency shift information received by the radar and combining it with the corner feature extraction method, the proposed method can more accurately capture subtle changes in the target human behavior. This method is a novel approach for human behavior recognition applied to through-wall radar, which is the first of its kind in the industry.

[0105] Improved recognition accuracy and robustness: Compared with traditional methods, the feature extraction method proposed in this invention can provide more comprehensive and accurate human behavior information, thereby improving the accuracy and robustness of behavior recognition and thus improving the reliability of its system deployment.

[0106] Adaptable to complex scenarios: This invention can work in complex wall-penetrating environments, including when different types of wall materials and multiple targets exist simultaneously, and can still effectively extract the characteristics of human behavior of the target, making it highly practical in scenarios such as disaster relief.

[0107] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A method for extracting micro-Doppler corner features, characterized in that, include: Step S1: Obtain the raw data of human behavior echo from through-wall radar and preprocess the raw data; The preprocessing includes clutter and noise suppression, image contrast enhancement, and image coordinate axis transformation, resulting in a processed radar squared range-time image. and square Doppler-time images ; Step S2: Use the difference of Gaussian convolutional filter to extract micro-Doppler corner supervision labels on the simulated radar squared range-time image and squared Doppler-time image. Then use the supervision labels to train the μD-CornerDet model, which is jointly constructed based on a deformable convolutional network, a domain-adaptive deformable convolutional network, a feature pyramid network, and a region-learnable global attention module. Step S3: Inference stage, the μD-CornerDet model is used to predict the radar squared range-time image and squared Doppler-time image of the measured data to obtain corner feature maps; The method for training the μD-CornerDet model using supervised labels includes: The μD-CornerDet module uses ResNet-101 as the backbone network, downsampled feature pyramid networks and upsampled feature pyramid networks as the connection parts, and fully connected layers as the heads for classification and regression. The specific scheme is as follows: Actual measured input to the backbone network and The image is transformed using a pseudo-color map mapping method. Channels, and uniformly scaled to Scale; The input image is scaled to a size of The number of channels is The input feature map is obtained through convolutional layer processing; The feature maps are respectively processed by a size of , and The number of channels is The convolutional layers are processed, and the processing result is added to the input feature map to obtain the output feature map, which serves as a first-level processing module in ResNet-101. This first-level processing module is repeated three times to obtain a second-level processing unit in ResNet-101. Next, the second-level processing units are stacked sequentially, with the number of convolutional channels in each layer of the processing unit being doubled in each stack, and this is repeated four times. Finally, the average pooling layers are connected sequentially to obtain the backbone network. In the upsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order. Each layer is obtained by concatenating the input feature maps of the corresponding secondary processing unit along the channel dimension. Similarly, in the downsampling feature pyramid network, each layer with a spatial scale from largest to smallest corresponds one-to-one with the four secondary processing units in the ResNet-101 backbone in their respective order. Each layer is obtained by concatenating the input feature maps of the corresponding secondary processing unit in reverse order along the channel dimension. In the upsampling feature pyramid network, the image of each layer is processed by a deformable convolutional module (DCN), and the processing result of each layer is directly summed with the input of the next layer. In the downsampling feature pyramid network, the image of each layer is also processed by a deformable convolutional module (DCN), and the processing result of each layer is directly summed with the input of the previous layer. Next, the outputs of each layer of the downsampled feature pyramid network and the upsampled feature pyramid network are processed by the learnable global attention module LRGA, and then the processing results of the corresponding two layers are concatenated through inter-layer channel dimensions to obtain a layer of features, thus forming a concatenated feature pyramid. Inter-layer connections are made through the domain adaptive deformable convolutional network module TDCN. The images from the upsampled feature pyramid network and the downsampled feature pyramid network in each layer are processed by TDCN and summed layer by layer, thereby mapping the concatenated feature pyramid to the head module for classification and two-dimensional coordinate regression.

2. The micro-Doppler corner feature extraction method as described in claim 1, characterized in that, In step S1, after mixing and low-pass filtering, the baseband time-domain echo signal of human motion from the through-wall radar is obtained. The received echo signal is converted into a one-dimensional range profile by performing discrete inverse Fourier transform on the echo signal. By stitching the range profile along the slow time dimension, channel correction, clutter suppression, and noise suppression are performed respectively to obtain the range-time matrix. Based on distance-time matrix The radar range-time map (RTM) and Doppler-time map (DTM) are obtained, and then the radar squared range-time image is obtained. and square Doppler-time images .

3. The micro-Doppler corner feature extraction method as described in claim 2, characterized in that, In step S1, the radar squared range-time image is obtained. The methods include: by analyzing the matrix Taking the absolute value and converting it to a pseudo-color image yields the RTM (Real-Time Model). An adaptive histogram equalization algorithm is used to enhance the contrast of the RTM. Then, interpolation is used to stretch the distance dimension of the RTM from the linear axis to the square axis, resulting in... During the interpolation process, an interval filling algorithm is used to fill empty distance cells on the vertical coordinate with gray values ​​in sequence that contain pixel information.

4. The micro-Doppler corner feature extraction method as described in claim 2, characterized in that, In step S1, the squared Doppler-time image is obtained. The methods include: aligning the distance-time matrix along the fast time dimension. Summation is performed, and a short-time Fourier transform is executed in the slow-time dimension to generate a DTM. Then, contrast enhancement, pseudo-color mapping, and vertical axis stretching are applied to the DTM to generate... During the stretching process, the image is cropped into two half matrices along the zero Doppler axis and then filled with gaps at the respective intervals.

5. The micro-Doppler corner feature extraction method as described in claim 1, characterized in that, The method for extracting micro-Doppler corner point supervision labels from simulated radar squared range-time images and squared Doppler-time images using a Gaussian difference convolution filter includes: Assuming the input and All are unified as variables ,in and Representing the horizontal and vertical image spatial scales; the Gaussian kernel is defined as follows: （3） in, It is the standard deviation of the Gaussian kernel; therefore, the difference-of-gaussian convolution kernel is expressed as: （4） in, , This represents the number of image scaling pyramid layers for a given scale set. Represents the Laplace operator; right Perform Gaussian difference convolution and nonmaximum suppression on multiple scale sets: （5） in, Represents the image convolution operator; for each image within a certain scale set, the center point is selected as the coordinates of the corner point within the rectangular window corresponding to the maximum value; by traversing each scale set, a series of corner point coordinates are obtained under different image scale sets, which serve as training supervision labels for the μD-CornerDet module.

6. The micro-Doppler corner feature extraction method as described in claim 5, characterized in that, exist and In the micro-Doppler corner feature obtained, the corner points in the following coordinate ranges are excluded: (1) (2) Based on the above criteria, select and store valid two-dimensional coordinate features from the Gaussian difference convolution results.

7. The micro-Doppler corner feature extraction method as described in claim 1, characterized in that, The classification and 2D coordinate regression of the head module are implemented by fully connected layers; in the regression branch, the number of output nodes corresponds to twice the number of micro-Doppler corner points required, and its output includes the horizontal and vertical coordinates of the corner points. The two covariates; the output of the classification branch is a binary value, where This indicates that it is a desired corner point, and This indicates that it is an unwanted corner point filtered out by the difference of Gaussian convolution module; during training, the parameters of the fully connected layers in both branches are optimized simultaneously; The classification branch is trained using the cross-entropy loss function; the regression branch is trained using the minimum mean squared error loss function.

8. The micro-Doppler corner feature extraction method as described in claim 7, characterized in that, During the reasoning process, the network iterates sequentially layer by layer, while simultaneously providing... and The coordinates and types of the corner points are listed, and the output retains the classification branch. The corner points, after removing the classification branches, output as The corner point.

9. An apparatus for implementing the micro-Doppler corner feature extraction method according to any one of claims 1-8, characterized in that, include: Image processing module, feature extraction module, and prediction module; The image processing module is used to implement the method of step S1; The feature extraction module is used to implement the method of step S2; The prediction module is used to implement the method of step S3.

Images