Millimeter wave radar non-line-of-sight moving target behavior identification method based on learning preprocessing

Abstract

The invention discloses a millimeter wave radar non-line-of-sight moving target behavior identification method based on learnable preprocessing, and the method comprises the steps: constructing a learnable preprocessing module through a plurality of linear layers, extracting the features of radar echoes in the distance, Doppler and angle dimensions, carrying out the incoherent accumulation of Doppler-angle information, and obtaining a learnable preprocessing module; target absolute position correlation is weakened, and feature expression of motion robustness is enhanced; and inputting the preprocessed features into a CNN-LSTM network, and realizing accurate identification of moving target behaviors in a non-line-of-sight environment by combining the spatial feature extraction capability of the convolutional neural network and the time sequence modeling capability of a long short-term memory network. The method can effectively overcome the feature difference caused by the target position change, improves the behavior recognition accuracy, and is suitable for the practical application scenes of intelligent driving, security monitoring and the like.

Description

Technical Field

[0001] This invention belongs to the field of radar signal processing technology, and particularly relates to the behavior recognition of non-line-of-sight moving targets. Background Technology

[0002] Non-Line-of-Sight (NLOS) moving target behavior recognition, as a masked target detection technology, has significant application value in scenarios such as intelligent driving and urban warfare. Its core principle is to utilize the reflection or diffraction propagation path of electromagnetic waves in non-line-of-sight environments to obtain target echoes and extract target behavioral features, thereby achieving behavior recognition of non-line-of-sight targets. During the movement of a non-line-of-sight target, its distance, speed, and angle relative to the radar constantly change, leading to differences in characteristics even for the same behavior. These differences reduce the accuracy of moving target behavior recognition.

[0003] Conventional behavior recognition methods can obtain the range-time spectrum (RT map) by performing an FFT on the range dimension or obtain the velocity-time spectrum through a Short-Time Fourier Transform (STFT). These two spectra are then input into a classification network to achieve behavior recognition. However, the RT map lacks Doppler analysis and cannot reflect changes in target velocity or periodic motion information. For moving target recognition, the information in the RT map is too simplistic, making it difficult to extract velocity or frequency-related features. The STFT cannot extract abrupt or continuous changes in the echo signal, and in non-line-of-sight environments, target signals are weak, features are sparse, and easily masked by noise. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a non-line-of-sight moving target behavior recognition method based on learnable preprocessing for millimeter-wave radar. This invention addresses the issue that changes in the position of moving targets lead to feature differences in the same behavior, resulting in decreased recognition accuracy. First, a learnable neural network preprocessing module consisting of three sets of complex linear layers is constructed to extract features that are relevant to the target's motion and robust to position changes, while weakening features strongly correlated with absolute position. Then, a CNN-LSTM network is constructed to extract features from the output of the learnable preprocessing module, achieving non-line-of-sight moving target behavior recognition.

[0005] Figure 1 This is a signal processing flowchart of an embodiment of the present invention. To achieve the purpose of the present invention, the technical solution adopted is: a method for non-line-of-sight moving target behavior recognition based on learnable preprocessing millimeter-wave radar, comprising the following steps: Step 1: Non-line-of-sight target echo data and preprocessing The target behavior echo data at different locations is obtained by millimeter-wave radar. The transmitted signal is mixed with the echo signal received by the receiving antenna to obtain the intermediate frequency signal, which is then saved as... A four-dimensional matrix, where The number of frames in the echo data. The number of chirp pulses in each frame of echo data. The number of receiving antennas. The number of sampling points for each chirp signal.

[0006] Step 2: Construct a learnable preprocessing module Learnable preprocessing modules such as Figure 2 As shown, this module consists of three complex linear transformation layers stacked sequentially, each layer corresponding to the feature extraction of radar echo data in the range, Doppler, and azimuth dimensions. At the end of the preprocessing module, the Doppler-angle information from different range units is incoherently accumulated, retaining features that are related to the target's motion and robust to changes in target position, while weakening features strongly correlated with the target's absolute position.

[0007] For including One transmitting antenna and An array of receiving antennas, each generating a chirp pulse signal. One echo signal. Extract the original echo signal. , and The composition of dimensional data: 3D tensor ,in For the index of the sampling points in the distance dimension, For the index of the Dopplerweis chirp pulse sequence, This is the index for the receiving array element.

[0008] Distance dimension feature extraction , in Represents the distance dimension of the complex linear layer. The weights of each frequency point are learned through iterative training, and the weight parameters that best represent the target distance feature are obtained.

[0009] Doppler feature extraction , in Describing the Doppler complex linear layer 1 The weights of each Doppler frequency point.

[0010] Angular dimension feature extraction , in For the angular dimension complex linear layer of the first The weight of each angle frequency point.

[0011] The learnable preprocessing module generates three-dimensional complex output. Take the amplitude of the output. and in the distance dimension Incoherent superposition is performed to aggregate the distance dimension, resulting in... .

[0012] Step 3: Feature extraction and behavior recognition based on CNN-LSTM network CNN-LSTM networks, such as Figure 3 As shown, the front end of the CNN network consists of several convolutional and pooling layers to extract spatial features of different behaviors. Batch normalization is added in the middle to improve generalization ability. Finally, several fully connected layers are connected and the ReLU activation function is used to introduce non-linearity, outputting the probability of each behavior category. The feature vector sequence extracted by the CNN network is input into the LSTM network to extract the temporal features of different behaviors over time.

[0013] This completes a method for non-line-of-sight moving target behavior recognition based on learnable preprocessing in millimeter-wave radar.

[0014] Beneficial effects: This invention is applied to non-line-of-sight moving target behavior recognition. By designing a learnable preprocessing module, it weakens the differences in identical behavioral features caused by changes in target position, while retaining features that are related to the target's motion itself and robust to position changes. This method can achieve accurate non-line-of-sight moving target behavior recognition and is an effective and practical non-line-of-sight target behavior recognition method, specifically including: This invention achieves accurate behavior recognition even when the target position changes continuously; This invention maintains the characteristic of fast solution; This invention is applicable to MIMO millimeter-wave radar. Attached Figure Description

[0015] Figure 1 , is a signal processing flowchart of an embodiment of the present invention; Figure 2 This is a structural diagram of the learnable preprocessing module proposed in this invention; Figure 3 The diagram shows the CNN-LSTM network structure of this invention. Figure 4 These are actual test scenario diagrams of the present invention; Figure 5The results are discrete human behavior recognition results using the distance dimension FFT and STFT preprocessing methods: (a) STFT preprocessing - training position, (b) STFT preprocessing - untrained position, (c) distance dimension FFT preprocessing - training position, (d) distance dimension FFT preprocessing - untrained position. Figure 6 The results are continuous human behavior recognition results using distance-dimensional FFT and STFT preprocessing methods; (a) STFT preprocessing, (b) distance-dimensional FFT preprocessing; Figure 7 , are the discrete human behavior recognition results of the method proposed in this invention; (a) training position, (b) untrained position; Figure 8 The results of continuous human behavior recognition are those obtained by the method proposed in this invention. Detailed Implementation

[0016] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] Step 1: Non-line-of-sight target echo data and preprocessing The target behavior echo data at different locations is obtained by millimeter-wave radar. The transmitted signal is mixed with the echo signal received by the receiving antenna to obtain the intermediate frequency signal, which is then saved as... A four-dimensional matrix, where The number of frames in the echo data. The number of chirp pulses in each frame of echo data. The number of receiving antennas. The number of sampling points for each chirp signal.

[0018] The target behavior echo data at different locations is obtained by millimeter-wave radar. The transmitted signal is mixed with the echo signal received by the receiving antenna to obtain the intermediate frequency signal, which is then saved as... The four-dimensional matrix has 20 echo data frames, 128 chirp pulse signals in each echo data frame, 8 equivalent receiving antennas, and 256 sampling points for each chirp signal.

[0019] Step 2: Construct a learnable preprocessing module Learnable preprocessing modules such as Figure 2 As shown, this module consists of three complex linear transformation layers stacked sequentially, each layer corresponding to the feature extraction of radar echo data in the range, Doppler, and azimuth dimensions. At the end of the preprocessing module, the Doppler-angle information from different range units is incoherently accumulated, retaining features that are related to the target's motion and robust to changes in target position, while weakening features strongly correlated with the target's absolute position.

[0020] For including One transmitting antenna and An array of receiving antennas, each generating a chirp pulse signal. One echo signal. Extract the original echo signal. , and The composition of dimensional data: 3D tensor ,in For the index of the sampling points in the distance dimension, For the index of the Dopplerweis chirp pulse sequence, This is the index for the receiving array element.

[0021] Distance dimension feature extraction , in Represents the distance dimension of the complex linear layer. The weights of each frequency point are learned through iterative training, and the weight parameters that best represent the target distance feature are obtained.

[0022] Doppler feature extraction , in Describing the Doppler complex linear layer 1 The weights of each Doppler frequency point.

[0023] Angular dimension feature extraction , in For the angular dimension complex linear layer of the first The weight of each angle frequency point.

[0024] The learnable preprocessing module generates three-dimensional complex output. Take the amplitude of the output. and in the distance dimension Incoherent superposition is performed to aggregate the distance dimension, resulting in... .

[0025] The radar equipment has 2 transmitting antennas and 4 receiving antennas, with 256 range frequency points, 128 Doppler frequency points, and 128 angular frequency points.

[0026] Step 3: Feature extraction and behavior recognition based on CNN-LSTM network CNN-LSTM networks, such as Figure 3As shown, the front end of the CNN network consists of several convolutional and pooling layers to extract spatial features of different behaviors. Batch normalization is added in the middle to improve generalization ability. Finally, several fully connected layers are connected and the ReLU activation function is used to introduce non-linearity, outputting the probability of each behavior category. The feature vector sequence extracted by the CNN network is input into the LSTM network to extract the temporal features of different behaviors over time.

[0027] The CNN network consists of three convolutional layers, each followed by a batch normalization layer, a ReLU non-linear activation function, and a two-dimensional max pooling layer. The first convolutional layer uses four convolutional kernels, each with a size of [size missing]. With a stride of 1, a convolutional layer is followed by a... The first layer uses max pooling with a pooling window stride of 2. The second and third convolutional-pooling layers have similar structures to the first layer, with 8 and 16 kernels respectively. The LSTM network uses a single-layer structure with a hidden state dimension of 512. The learnable preprocessing module and the classification network are jointly trained using the cross-entropy loss function and the Adam optimizer.

[0028] Example To verify the proposed method for non-line-of-sight moving target behavior recognition based on learnable preprocessing in millimeter-wave radar, a field experiment was designed and analyzed. The experimental parameters are shown in Table 1.

[0029] Table 1. Experimental parameter settings

[0030] exist Figure 4 The non-line-of-sight (Line-of-Sight) area was tested for moving target behavior recognition. The radar equipment was a TIAWR1642BOOST millimeter-wave radar, which remained stationary throughout the experiment. In the experimental scenario, the radar's location was used as the origin, with the radar height above the ground at 0.98m and the corner coordinates at (-0.6m, 0.4m). Eight different locations were selected, and different behavioral actions were repeated at each location. Additionally, behavioral data was collected from six locations; this data was not used for network training but was used to verify the recognition accuracy of the trained network when the target was located at different positions in the non-line-of-sight area. The target was continuously moved in the non-line-of-sight scene, and 100 frames of data were collected. Test data was extracted from the 100 frames of continuous movement using a sliding window of 20 frames, and this test data was input into the trained network for behavior recognition.

[0031] Preprocessing was performed using two methods: distance-dimensional FFT and STFT. The resulting RT map and velocity-time map were then input into a CNN-LSTM network. The discrete action recognition results and continuous action recognition results are shown below. Figure 5 , Figure 6As shown; the behavior recognition results using the method proposed in this invention, which combines learnable preprocessing with CNN-LSTM networks, are as follows. Figure 7 , Figure 8 As shown, the accuracy is significantly higher than that of traditional preprocessing methods.

[0032] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for millimeter wave radar non-line-of-sight moving target behavior recognition based on learnable pre-processing, characterized in that, The method comprises the following steps: Step 1: obtaining echo data of different behaviors of a target at different positions in a non-line-of-sight area and preprocessing; Step 2: constructing a learnable preprocessing module composed of complex linear transformation layers; Step 3: constructing a CNN-LSTM network to perform feature extraction and behavior recognition on the output of the learnable preprocessing module.

2. The method of claim 1, wherein the method is based on a learnable pre-processing for millimeter wave radar non-line-of-sight moving target behavior recognition. In step 1, the target is located in the non-line-of-sight area of the millimeter wave radar, and the target is in a moving state, the target position changes constantly, and the echo data of the behavior of the target at different positions is obtained.

3. The method of claim 1, wherein the method is based on a learnable pre-processing for millimeter wave radar non-line-of-sight moving target behavior recognition. In step 2, a learnable preprocessing module composed of complex linear layers is constructed, and the complex linear layers are connected in series and correspond to time-frequency transformation in the distance, Doppler and angle dimensions respectively.

4. The method of claim 1, wherein the method is based on a learnable pre-processing for millimeter wave radar non-line-of-sight moving target behavior recognition. In step 2, the distance dimension feature extraction is represented as: ; wherein represents the weight of the distance dimension complex linear layer of the frequency points, and the weight value parameter that best represents the target distance feature is learned through iterative training. The Doppler dimension feature extraction is represented as: ; wherein represents the Doppler complex linear layer of the weight of the Doppler frequency point; The angle dimension feature extraction is represented as: ; wherein is the weight of the kth angle bin of the angular dimension complex linear layer. is the weight of the kth angle bin of the angular dimension complex linear layer.

5. The method of claim 1, wherein the method is based on a learnable pre-processing for millimeter wave radar non-line-of-sight moving target behavior recognition. In step 2, the initial weights of each complex linear layer are set to the discrete Fourier transform basis function, so that the initial transformation of the module is equivalent to the discrete Fourier transform. Through error back propagation in the training process, the weights of each layer of the module are gradually adjusted, and the initial Fourier transform basis function gradually evolves into an optimal basis function combination for a specific data set and task.

6. The method of claim 1, wherein the method is based on a learnable pre-processing for millimeter wave radar non-line-of-sight moving target behavior recognition. In step 3, a behavior recognition network combining CNN and LSTM is adopted, CNN is used to extract spatial features of single frame echo signals, and LSTM is used to capture time sequence features between frames.

Images