A millimeter wave radar-based identity recognition method and system

Abstract

The application discloses a kind of semi-supervised identity recognition method and system based on millimeter wave radar, method includes: obtaining the point cloud data and range-velocity diagram data of millimeter wave radar;Point cloud data and range-velocity diagram data are preprocessed;The point cloud identity recognition model and range-velocity diagram identity recognition model designed are collaboratively trained using the point cloud data and range-velocity diagram data after preprocessing, the prediction probability of point cloud identity recognition model and the prediction probability of range-velocity diagram identity recognition model are obtained;The prediction probability of point cloud identity recognition model and the prediction probability of range-velocity diagram identity recognition model are fused.The application is trained by the method of semi-supervised collaboration to point cloud identity recognition model and range-velocity diagram identity recognition model, each other guide uses unlabelled data, improves model performance;Meanwhile, the prediction of two kinds of identity recognition models is fused, and higher identity recognition accuracy is realized.

Description

Technical Field

[0001] This invention relates to the field of identity recognition technology, and in particular to an identity recognition method and system based on millimeter-wave radar. Background Technology

[0002] With the development of IoT technology, artificial intelligence has been widely applied in caregiving, healthcare, and other applications closely related to users' daily lives. Identity recognition technology can provide customized services for specific individuals, facilitating daily user monitoring and providing a safe and comfortable working and living environment. Identity recognition technology plays a crucial role in information retrieval, human-computer interaction, and automated systems.

[0003] The most common identity verification technologies typically use vision-based methods, which employ cameras to capture images or videos of people for identification. However, vision-based methods are often affected by several factors, sometimes leading to unsatisfactory results. First, they are typically influenced by light intensity and the user's clothing; system performance is affected when these factors change. Second, cameras record a direct video stream, and users may be concerned about privacy and refuse to install them. Radio frequency (RF) based methods emit electromagnetic waves into the environment, recording human movement information in the echoes. This method is unaffected by light, can function normally in darkness, and can protect user privacy.

[0004] Millimeter-wave radar has experienced rapid development in recent years. It is relatively inexpensive and insensitive to changes in lighting conditions. Millimeter-wave radar can detect the position and velocity of a human body in three-dimensional space, offering high range and velocity resolution. It senses the environment and detects objects by emitting electromagnetic waves; features such as body shape and gait are included in the radar echo. Current identity recognition research requires collecting large amounts of data for each user and labeling all of it. However, the data collection method requiring the labeling of a large number of new users hinders the large-scale application of radar-based identity recognition technology. Summary of the Invention

[0005] The purpose of this invention is to provide an identity recognition method and system based on millimeter-wave radar. By co-training a point cloud identity recognition model and a distance-velocity map identity recognition model, the two models can be mutually guided to utilize unlabeled data to improve model performance. At the same time, by integrating the predictions of the two identity recognition models, higher accuracy of identity recognition can be achieved.

[0006] To address the above problems, a first aspect of the present invention provides an identity recognition method based on millimeter-wave radar, comprising:

[0007] Acquire point cloud data and range-velocity map data from millimeter-wave radar;

[0008] The point cloud data and distance-velocity map data are preprocessed;

[0009] The point cloud identity recognition model and the distance-velocity map identity recognition model are co-trained to obtain the prediction probability of the point cloud identity recognition model and the prediction probability of the distance-velocity map identity recognition model.

[0010] The predicted probability of the point cloud identity recognition model and the predicted probability of the distance-velocity map identity recognition model are combined.

[0011] Preferably, acquiring point cloud data and range-velocity map data from millimeter-wave radar includes:

[0012] When the millimeter-wave radar detects the movement of the monitored target, it continuously records the point cloud signal and range-velocity map signal of the millimeter-wave radar, and saves the data sample when the monitored target stops moving;

[0013] The point cloud data is generated by calculating the three-dimensional coordinates and velocity of each point on the monitoring target based on the saved point cloud signal data samples.

[0014] The saved distance-velocity map signal data samples are processed sequentially in both fast and slow time dimensions to form the distance-velocity map data.

[0015] Preferably, the distance-velocity map data is preprocessed, including:

[0016] The distance between the monitored target and the millimeter-wave radar is calculated based on the point cloud data, and the distance-velocity map data is extracted using the distance.

[0017] Preferably, the point cloud identity recognition model and the distance-velocity map identity recognition model are trained collaboratively, including:

[0018] The point cloud identity recognition model and the distance-velocity map identity recognition model were trained using labeled datasets, respectively.

[0019] Make predictions on the unlabeled dataset to obtain a pseudo-labeled dataset;

[0020] Reconstruct the training dataset;

[0021] The point cloud identity recognition model and the distance-velocity map identity recognition model were retrained using the reconstructed training dataset.

[0022] Preferably, the reconstructed training dataset includes: a pseudo-labeled dataset and a labeled dataset.

[0023] Preferably, it further includes: establishing a model prediction loss function and a model reconstruction loss function.

[0024] Cross-entropy is used as the prediction loss function for the model:

[0025]

[0026] Where N is the batch size; j is the category index;

[0027] Labels representing identity, This represents data that is not the target of the monitoring. The representative is the data of the monitoring target;

[0028] f i j For model prediction;

[0029] The mean squared error is used as the model reconstruction loss function:

[0030]

[0031] Where X represents the input distance-velocity map data;

[0032] X′ represents the decoded distance-velocity map data;

[0033] ||*||2 is the L2 norm.

[0034] Preferably, it also includes selecting a pseudo-label dataset H:

[0035]

[0036] in, For point cloud models Predicted probability of the category;

[0037] For velocity-distance graph models Predicted probability of the category;

[0038] α is the weighting coefficient;

[0039] τ is the probability threshold.

[0040] Preferably, it also includes a method for establishing a point cloud identity recognition model, comprising:

[0041] Using the preprocessed point cloud data as input, a feature extraction module is constructed using a multilayer perceptron to extract the attribute features of each point in the point cloud data;

[0042] A spatiotemporal fusion module is constructed to extract the spatiotemporal features of the attribute features of each frame of point cloud data;

[0043] Using the extracted spatiotemporal features, a predictor module is constructed using a multilayer perceptron.

[0044] Preferably, it also includes: a method for establishing a distance-velocity map identity recognition model, comprising:

[0045] Using the preprocessed distance-velocity map data as input, an encoder module is constructed to extract the attribute features of each frame's distance-velocity map;

[0046] Build a decoder module to reconstruct the distance-velocity map;

[0047] The attribute features of the distance-velocity map extracted by the encoder are used as input to construct a predictor module using a multilayer perceptron.

[0048] Preferably, the predicted probabilities of the point cloud identity recognition model and the predicted probabilities of the distance-velocity map identity recognition model are fused by constructing the following function f:

[0049]

[0050] Where a is the prediction weight of the point cloud model, 0 <a<1;

[0051] (1-a) represents the prediction weights of the distance-velocity map model.

[0052] According to another aspect of the present invention, a millimeter-wave radar-based identity recognition system is provided, comprising:

[0053] The data acquisition module is used to acquire point cloud data and range-velocity map data from millimeter-wave radar;

[0054] The data preprocessing module is used to preprocess the point cloud data and distance-velocity map data;

[0055] The model training module is used to collaboratively train the preset point cloud identity recognition model and the distance-velocity map identity recognition model;

[0056] The identity recognition module is used to fuse the predicted probability of the point cloud identity recognition model and the predicted probability of the distance-velocity map identity recognition model to determine the identity of the monitored target.

[0057] Preferably, the data acquisition module includes:

[0058] The target movement monitoring module is used to continuously record the point cloud signal and range-velocity map signal of the millimeter-wave radar when the millimeter-wave radar detects the movement of the monitored target, and save the data when the monitored target stops moving.

[0059] The point cloud data acquisition module calculates the three-dimensional coordinates and velocity of each point on the monitored target based on the data stored by the target movement monitoring module, thereby forming the point cloud data.

[0060] The distance-velocity map data acquisition module processes the stored distance-velocity map signal sequentially using fast time dimension processing and slow time dimension processing based on the data stored by the target movement monitoring module, thereby forming the distance-velocity map data.

[0061] Preferably, the data preprocessing module includes:

[0062] The distance-velocity map data preprocessing module is used to extract the distance-velocity map data based on the distance between the monitored target and the millimeter-wave radar.

[0063] Preferably, the model training module includes:

[0064] The model prediction module is used to predict the unlabeled dataset using the point cloud identity recognition model and the distance-velocity map identity recognition model, which are trained on labeled datasets respectively, to obtain a pseudo-labeled dataset.

[0065] The data reconstruction module is used to reconstruct the training dataset;

[0066] The training module uses the reconstructed training dataset to train the point cloud identity recognition model and the distance-velocity map identity recognition model, respectively.

[0067] Preferably, it further includes: a loss function module, used to establish the model prediction loss function and the model reconstruction loss function.

[0068] Preferably, the point cloud identity recognition model includes:

[0069] The feature extraction module uses a multilayer perceptron to extract the attribute features of each point in the point cloud data;

[0070] The spatiotemporal fusion module takes the attribute features of multi-frame point clouds as input and extracts the spatiotemporal features of the attribute features of each point cloud data sample.

[0071] The point cloud predictor module uses the extracted spatiotemporal features to predict the identity of the detected target.

[0072] Preferably, the distance-velocity map identity recognition model includes:

[0073] The encoder module is used to extract attribute features from the distance-velocity map;

[0074] The decoder module takes the extracted attribute features as input and reconstructs the distance-velocity map;

[0075] The predictor module takes the extracted attribute features as input and predicts the identity of the detected target.

[0076] The above-described technical solution of the present invention has the following beneficial technical effects:

[0077] This invention guides the established point cloud identity recognition model and distance-velocity map identity recognition model to each other, and uses unlabeled data for semi-supervised collaborative training. Only a small amount of labeled data needs to be collected, which reduces the workload of data collection while improving model performance. By obtaining the probability of the point cloud identity recognition model and the distance-velocity map identity recognition model predicting the identity of the monitored target, higher accuracy of identity recognition is achieved. Attached Figure Description

[0078] Figure 1 This is an overall flowchart of an identity recognition method based on millimeter-wave radar according to the present invention;

[0079] Figure 2 This is a flowchart illustrating the acquisition of point cloud data based on millimeter-wave radar echo signals.

[0080] Figure 3 This is a logical block diagram of the collaborative training of the point cloud identity recognition model and the distance-velocity map identity recognition model;

[0081] Figure 4 This is a framework diagram for collaborative training;

[0082] Figure 5 This is a schematic diagram of a distance-velocity map identity recognition model. Detailed Implementation

[0083] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0084] Taking advantage of the ability of millimeter-wave radar signals to accurately obtain information such as the distance, azimuth, and velocity of targets, a target identification model can be constructed using 3D point cloud data based on millimeter-wave radar and trained into a neural network. This can improve the accuracy and reliability of target detection, but it requires the collection of a large amount of labeled target data to train the model.

[0085] Meanwhile, by performing slow-dimensional time processing on millimeter-wave radar signals, a range-velocity map of the target can be obtained when there is relative motion between the target movement and the millimeter-wave radar. Using the target's range and velocity information to construct a target range-velocity map identification model enables target identification. However, for multi-target identification, it is difficult to guarantee accuracy and reliability, and a large amount of labeled target data is still needed to train the model.

[0086] Therefore, the first aspect of this invention discloses an identity recognition method based on millimeter-wave radar, the overall process of which is as follows: Figure 1 As shown, it includes the following steps:

[0087] Step S1: Acquire point cloud data and range-velocity map data from millimeter-wave radar;

[0088] Specifically, when the millimeter-wave radar detects user movement, it continuously records point cloud signals and range-velocity map signals during the movement, and saves the data sample when the user stops moving.

[0089] In this embodiment, a point cloud data sample contains multiple frames of point cloud data, and each frame of point cloud data contains multiple sampling points.

[0090] The data sample consists of two parts: the first part consists of a small number of labeled daily walking signals collected by staff for each monitored user; the second part consists of unlabeled daily walking signals automatically collected by installing millimeter-wave radar in the usage scenario.

[0091] Furthermore, step S1 includes: step S11, acquiring point cloud data from millimeter-wave radar, and step S12, acquiring range-velocity map data;

[0092] Step S11: Acquire point cloud data from millimeter-wave radar:

[0093] In one alternative embodiment, a millimeter-wave radar with three transmitting antennas and four receiving antennas is used, employing time-division multiplexing technology to form a 12-virtual receiving antenna array. When the millimeter-wave radar detects target movement, it continuously records the point cloud signal during the movement.

[0094] Specifically, it includes the following steps, such as Figure 2 As shown:

[0095] Step S11.1: After sampling the radar's transmitted signal for each time using the receiving antenna, perform fast time-dimension FFT processing on each sampled signal; simultaneously, subtract the mean of all signals in a frame from each signal to perform noise removal, as shown in the following formula:

[0096]

[0097] Among them, fi (n) represents the i-th signal in a frame of radar signals on an antenna;

[0098] n represents a sampling point in the fast time dimension;

[0099] In one frame, one antenna carries N signals;

[0100] Step S11.2: Based on the signals received by the eight virtual receiving antennas in the horizontal direction of the millimeter-wave radar, perform capon spectrum estimation to estimate the position of each sampling point on the monitored target in the horizontal direction, and form a 2D-range-horizontal heat map.

[0101] Step S11.3: Use the CFAR algorithm to detect targets in the range direction and horizontal angle direction;

[0102] Step S11.4: Perform Capon spectrum estimation on the signals received by all 12 virtual receiving antennas to estimate the height of the detected monitoring target, form a one-dimensional height heat map, and use the peak search method to determine the height of the detected target, and determine the three-dimensional coordinates (x, y, z) of each sampling point on the monitoring target in each frame.

[0103] Step S11.5: Calculate the beamforming filter based on each detected sampling point and perform beamforming on the 12 antennas using a slow-time dimension FFT. That is, process the signal across the N signals transmitted by the transmitting antennas to calculate the velocity of the sampling points and obtain velocity information.

[0104] Thus, the point cloud data obtained for each frame includes: the three-dimensional coordinates of each sampling point on the monitoring target and the velocity of each sampling point, which can be represented as (x,y,z,v).

[0105] Step S12: Obtain distance-velocity graph data;

[0106] In one alternative embodiment, a millimeter-wave radar with three transmitting antennas and four receiving antennas is used, employing time-division multiplexing technology to form a 12-virtual receiving antenna array. When the millimeter-wave radar detects target movement, it continuously records range-velocity map signal samples during the movement.

[0107] In this embodiment, a distance-velocity map data sample contains multiple frames of distance-velocity map data.

[0108] The echo signals from the millimeter-wave radar generated when the monitored target moves are first processed by fast time dimension FFT, and then by slow time dimension FFT to form the range-velocity map data of the monitored target.

[0109] Step S2: Preprocess the point cloud data and distance-velocity map data;

[0110] In embodiments of the present invention, the steps of preprocessing the collected point cloud data and distance-velocity map data include: step S21, preprocessing the point cloud data; and step S22, preprocessing the distance-velocity map data based on the point cloud data.

[0111] Step S21, the preprocessing of the point cloud data, includes: subtracting the mean value of one frame of point cloud from the x and y axis coordinates of each sampling point on each frame of point cloud in the point cloud data sample to obtain the normalized coordinates of each frame of point cloud.

[0112]

[0113]

[0114] in, The three-dimensional coordinates of the i-th sampling point in a frame of point cloud used as input in the point cloud identity recognition model;

[0115] K represents the number of sampling points in a frame;

[0116] Simultaneously, using the three-dimensional coordinates of each sampling point on each frame of the point cloud before normalization, the distance between the monitored target and the millimeter-wave radar is calculated.

[0117] Step S22: Preprocessing of distance-velocity map data;

[0118] This includes: based on the distance R between the monitored target and the millimeter-wave radar, extracting signals from the range-velocity map data (with distance as the ordinate) where the user's distance interval is less than 60cm, while retaining all signals representing velocity on the lateral axis and discarding the rest. This allows for training the model using the extracted point cloud data belonging to the monitored target and the corresponding range-velocity map data, reducing the impact of others moving simultaneously on the monitored target.

[0119] Step S3: Perform co-training on the point cloud identity recognition model and the distance-velocity map identity recognition model to obtain the prediction probability of the point cloud identity recognition model and the prediction probability of the distance-velocity map identity recognition model.

[0120] In this embodiment, the training dataset consists of two parts: the aforementioned preprocessed labeled dataset D, which is saved when the millimeter-wave radar detects user movement, and collected by staff for each monitored user. L And when millimeter-wave radar is installed in the application scenario, the pre-processed unlabeled dataset D of each monitoring user is automatically collected. U .

[0121] Optionally, 5-10 minutes of tagged daily walking signals are collected for each monitored user and preprocessed as described in step S2.

[0122] Collaborative training includes: step S31, training the point cloud identity recognition model and the distance-velocity map identity recognition model separately; and step S32, training based on the reconstructed training dataset, that is, using the reconstructed training dataset to train the point cloud identity recognition model and the distance-velocity map identity recognition model again in step S31, see [link to relevant documentation]. Figures 3-4 .

[0123] Step S31 includes:

[0124] Step S311: Using the labeled dataset D L Train the point cloud identity recognition model and the distance-velocity map identity recognition model for one cycle respectively;

[0125] Step S312: Use the two models from step S311 to process the unlabeled dataset D. U Make predictions to obtain the pseudo-label dataset H, and complete the first training cycle.

[0126] In this embodiment, the pseudo-label dataset H is selected in the following manner:

[0127]

[0128] in, For point cloud identity recognition models Predicted probability of the category;

[0129] For distance-velocity map identity recognition model Predicted probability of the category;

[0130] α is the weighting coefficient; v is the probability threshold.

[0131] In the prediction process of unlabeled data, because distance-velocity map data and point cloud data contain different information, a collaborative training method that combines the prediction of point cloud identity recognition model and distance-velocity map identity recognition model can be used to guide each other to utilize unlabeled data, thereby improving model performance and making the prediction of unlabeled data more accurate. In subsequent training, using more accurate pseudo-labels to train the model can further improve the accuracy of prediction and further improve model performance.

[0132] Step S32 includes:

[0133] Step S321: Reconstruct the training dataset D train ;

[0134] In the second training cycle, the pseudo-labeled dataset H and the labeled dataset D are used. L Merging as a reconstructed training dataset D train :

[0135] D train =D L ∪H

[0136] Semi-supervised learning uses existing labeled data to label the remaining unlabeled data, thus effectively helping to increase the amount of training data.

[0137] Therefore, this embodiment utilizes the labeled dataset D L and based on the unlabeled dataset D U The generated pseudo-label dataset H is used as a new reconstruction training dataset to perform semi-supervised learning on the point cloud identity recognition model and the distance-velocity map identity recognition model. This can reduce the amount of manual data labeling by training neural networks using a small amount of labeled data.

[0138] Step S322: Reconstruct the training dataset D train Resample the point cloud data and distance-velocity map data, and repeat the training in step S31 until the model converges.

[0139] Specifically, step S322 includes the following steps:

[0140] S3221. Utilize the reconstructed training dataset D train The point cloud identity recognition model and the distance-velocity map identity recognition model, which have completed one cycle of training in step S311, are trained again.

[0141] S3222. Using the two models that have completed one cycle of training in step S3221, train the unlabeled dataset D again. U Make predictions to obtain the pseudo-label dataset H1 and complete the second training cycle;

[0142] S3223, Reconstruct the training dataset D again. train1 Then, begin the third cycle of training, and so on, repeating the learning process.

[0143] At this point, the training dataset D train1 Includes: pseudo-labeled dataset H1 and labeled dataset D L .

[0144] By re-evaluating whether all unlabeled data is a pseudo-label during each training cycle, we can make full use of the unlabeled data and make the prediction of pseudo-labels more accurate by updating the pseudo-labels.

[0145] Meanwhile, from the labeled dataset DL 10%-20% of the data is extracted as a validation set that is not used in training. When the accuracy of the validation set gradually improves and approaches a stable value, the model converges. At this point, the predicted probabilities of the point cloud identity recognition model and the distance-velocity map identity recognition model are obtained.

[0146] Furthermore, step S3 also includes a preliminary step of establishing loss functions for the point cloud identity recognition model and the distance-velocity map identity recognition model, respectively, and a subsequent step of using the loss functions to continue training the two models.

[0147] In this embodiment, a model prediction loss function and a model reconstruction loss function are established for the collaborative training of the point cloud identity recognition model and the distance-velocity map identity recognition model.

[0148] Cross-entropy is used as the model prediction loss function for both the cloud identity recognition model and the distance-velocity map identity recognition model in step S312:

[0149]

[0150] Where N is the batch size; j is the category index;

[0151] Labels representing identity, This represents data that is not the target of the monitoring. Data representing the monitoring target;

[0152] f i j For model prediction.

[0153] The mean squared error is used as the model reconstruction loss function for the distance-velocity map identity recognition model after the training dataset is reconstructed in step S322.

[0154]

[0155] Where X represents the input distance-velocity map data;

[0156] X′ represents the decoded distance-velocity map data;

[0157] ||*||2 is the L2 norm.

[0158] In other embodiments, the point cloud identity recognition model is constructed using the following method:

[0159] Multilayer perceptron is used to extract the attribute features of each sampling point in each frame of preprocessed point cloud data, namely (x,y,z,v);

[0160] The Multilayer Perceptron (MLP) consists of 8 layers, with each pair of layers using a residual connection proposed by ResNet and LeakyReLU as the activation function. For each frame of point cloud data, the maximum value of each feature in the dimension of the number of points in the point cloud is calculated as the feature of a frame of point cloud.

[0161] Using the attribute features of multi-frame point clouds as input, a bidirectional GRU is used to extract the spatiotemporal features of each point cloud data sample. These spatiotemporal features contain gait information of the monitored target walking over a period of time.

[0162] A prediction module is constructed using MLP to obtain spatiotemporal features as input, predicting the identity of the monitored target and obtaining the predicted probability of the point cloud identity recognition model.

[0163] The distance-velocity map identity recognition model is constructed using the following method:

[0164] Using the preprocessed distance-velocity map data as input, the attribute features of the distance-velocity map are extracted;

[0165] Specifically, the encoder module takes the preprocessed distance-velocity map data as input and uses a ResNet network structure to construct eight residual blocks for extracting features from the distance-velocity map.

[0166] A decoder composed of four deconvolutional layers is used to restore the features to the same size as the input signal; by adopting an encoder-decoder network structure, the network extracts a compressed representation of the distance-velocity map, thereby improving the model's performance.

[0167] Based on the features extracted from the distance-velocity map, MLP prediction is used to predict the identity of the detected target, and the prediction probability of the distance-velocity map model is obtained.

[0168] S4. The predicted probability of the fusion point cloud identity recognition model and the predicted probability of the distance-velocity map identity recognition model.

[0169] In one alternative embodiment, the predicted probabilities of the point cloud identity recognition model and the predicted probabilities of the distance-velocity map identity recognition model are fused by constructing a function f:

[0170]

[0171] Where a is the prediction weight of the point cloud model, 0 <a<1;

[0172] (1-a) represents the prediction weights of the distance-velocity map model.

[0173] The predictions from the two models are weighted and summed to obtain the final prediction probability.

[0174] When the final predicted probability f > 0.5, the monitored user is identified.

[0175] This invention aims to protect an identity recognition method based on millimeter-wave radar. By guiding the established point cloud identity recognition model and distance-velocity map identity recognition model to each other and using unlabeled data for semi-supervised collaborative training, the model performance is improved. By obtaining the probability of the point cloud identity recognition model and the distance-velocity map identity recognition model predicting the identity of the monitored target, higher accuracy of identity recognition is achieved.

[0176] To implement the above embodiments, another embodiment of the present invention provides an identity recognition system based on millimeter-wave radar, comprising:

[0177] The data acquisition module is used to acquire point cloud data and range-velocity map data from millimeter-wave radar;

[0178] The data preprocessing module is used to preprocess point cloud data and distance-velocity map data;

[0179] The model training module is used to collaboratively train the preset point cloud identity recognition model and the distance-velocity map identity recognition model;

[0180] The identity recognition module is used to fuse the predicted probabilities of the monitored target obtained from the point cloud identity recognition model and the distance-velocity map identity recognition model obtained from the model training module, and to determine the identity of the monitored target by constructing a probability function.

[0181] In a preferred embodiment, the data acquisition module includes:

[0182] The target movement monitoring module continuously records the point cloud signal and range-velocity map signal samples of the millimeter-wave radar when the millimeter-wave radar detects the movement of the monitored target, and saves the data sample when the monitored target stops moving.

[0183] The point cloud data acquisition module calculates the three-dimensional coordinates of each sampling point on the monitored target and the velocity of each point based on the point cloud signal data samples stored in the target movement monitoring module, thus forming point cloud data.

[0184] The distance-velocity map data acquisition module processes the stored distance-velocity map signals sequentially using fast time dimension processing and slow time dimension processing, based on the distance-velocity map signal samples stored in the target movement monitoring module, to form distance-velocity map data.

[0185] In a preferred embodiment, the data preprocessing module includes: a normalization processing module and a distance-velocity map data preprocessing module.

[0186] The normalization processing module is used to normalize the point cloud data to obtain the coordinates of each frame of point cloud after normalization, and to use the normalized point cloud data for neural network learning.

[0187] The distance-velocity map data preprocessing module calculates the distance R between the monitored target and the millimeter-wave radar based on the three-dimensional coordinates of each sampling point on each frame of the point cloud data. By extracting signals with a user distance interval of less than 60cm from the distance-velocity map data with distance as the vertical axis, and retaining all signals representing velocity on the horizontal axis while discarding the rest, the module achieves the preprocessing of the distance-velocity map data.

[0188] In one optional embodiment, the model training module includes:

[0189] The model prediction module is used to predict the unlabeled dataset using the point cloud identity recognition model and the distance-velocity map identity recognition model trained on the labeled dataset, respectively, to obtain the pseudo-label dataset H.

[0190] The data reconstruction module is used to reconstruct the training dataset.

[0191] Specifically, in the data reconstruction module, the pseudo-labeled dataset H and the labeled dataset D are... L Merge as a new training dataset D train .

[0192] The training module uses the reconstructed training dataset to train the point cloud identity recognition model and the distance-velocity map identity recognition model, respectively.

[0193] Specifically, in the training module, the labeled dataset D is first used. L After training the point cloud identity recognition model and the distance-velocity map identity recognition model for one epoch respectively, the unlabeled dataset D is then tested. U Perform predictions to obtain the pseudo-label dataset H;

[0194] From reconstructing the training dataset D train The point cloud data and distance-velocity map data are resampled. After preprocessing the resampled data, the point cloud identity recognition model and the distance-velocity map identity recognition model are trained in the training module until the model converges.

[0195] In the collaborative training of the point cloud identity recognition model and the distance-velocity map identity recognition model, the loss function module is used to establish the model prediction loss function and reconstruct the model loss function;

[0196] When the model prediction loss function is used to train the point cloud identity recognition model and the distance-velocity map identity recognition model, the error of the point cloud identity recognition model and the distance-velocity map identity recognition model is calculated, and the parameters of the point cloud identity recognition model and the distance-velocity map identity recognition model are optimized based on the error.

[0197] The model reconstruction loss function is used in the training of the distance-velocity map identity recognition model to calculate the error of the distance-velocity map identity recognition model after decoding and reconstruction, and to optimize the parameters of the distance-velocity map identity recognition model based on the error.

[0198] In one optional embodiment, the preset point cloud identity recognition model includes: a feature extraction module, a spatiotemporal fusion module, and a point cloud predictor module;

[0199] The feature extraction module uses a multilayer perceptron to extract the attribute features of each point in each frame of point cloud data in each sample.

[0200] The Multilayer Perceptron (MLP) consists of 8 layers, with each pair of layers using a residual connection proposed by ResNet and LeakyReLU as the activation function. For each frame of point cloud data, the maximum value of each feature in the dimension of the number of points in the point cloud is calculated as the feature of a frame of point cloud.

[0201] The spatiotemporal fusion module takes the attribute features of multi-frame point clouds as input and uses a bidirectional GRU to extract the spatiotemporal features of the attribute features of each point cloud data sample.

[0202] The point cloud predictor module is built using MLP. It takes the extracted spatiotemporal features as input to predict the identity of the monitored target and obtain the predicted probability of the point cloud identity recognition model.

[0203] In one optional embodiment, a preset distance-velocity map identity recognition model, such as Figure 5 As shown, it includes: an encoder module, a decoder module, and a predictor module;

[0204] The encoder module is used to extract attribute features from the distance-velocity map.

[0205] Specifically, the encoder module takes the preprocessed distance-velocity map data as input and uses a ResNet network structure to construct eight residual blocks to extract the attribute features of the distance-velocity map.

[0206] The decoder module consists of four deconvolutional layers. It takes the extracted attribute features as input and restores the features to the same size as the input signal to reconstruct the distance-velocity map.

[0207] By employing an encoder-decoder network structure, the network extracts a compressed representation of the distance-velocity map, thereby improving the model's performance.

[0208] The predictor module, based on the attribute features of the distance-velocity map extracted by the encoding module, uses MLP prediction to predict the identity of the detected target and obtain the prediction probability of the distance-velocity map model.

[0209] After obtaining the predicted probabilities from the point cloud identity recognition model and the distance-velocity map model, a probability function is constructed to determine the identity of the monitored target. When the probability function ultimately predicts a probability greater than 0.5, it is determined that the monitored user has been detected.

[0210] The process of identifying an individual using the millimeter-wave radar-based identity recognition system of the present invention is as follows: During system operation, the data acquisition module acquires the following two parts of data:

[0211] When the millimeter-wave radar detects user movement, staff collect 5-10 minutes of labeled point cloud signal data and distance-velocity map signal data samples for each monitored user. Additionally, the millimeter-wave radar is installed in the usage scenario to automatically collect unlabeled point cloud signal data and distance-velocity map signal data samples for the monitored user.

[0212] The data preprocessing module is used to preprocess the collected labeled and unlabeled point cloud data and distance-velocity map data. At the same time, the distance R between the monitored target and the millimeter-wave radar is obtained. Based on the distance R between the monitored target point and the millimeter-wave radar, signals with a distance of less than 60cm between the user and the surrounding area in the distance-velocity map data before data processing are extracted. All velocity signals are retained and the rest are discarded to reduce the interference of multiple people moving at the same time on the monitored target signal.

[0213] Using the preprocessed data, the model training module is used to perform semi-supervised co-training on the preset point cloud identity recognition model and distance-velocity map identity recognition model. The point cloud identity recognition model and distance-velocity map identity recognition model are trained multiple times by reconstructing the training dataset until the model converges.

[0214] The trained point cloud identity recognition model and distance-velocity map identity recognition model are used to predict the monitored target, and the prediction probability of the point cloud identity recognition model and the prediction probability of the distance-velocity map identity recognition model are obtained.

[0215] The identity recognition module is used to determine the pedestrian identity recognition result of the current frame.

[0216] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. An identity recognition method based on millimeter-wave radar, characterized in that, include: Acquire point cloud data and range-velocity map data from millimeter-wave radar; The point cloud data and distance-velocity map data are preprocessed; The point cloud identity recognition model and the distance-velocity map identity recognition model are co-trained to obtain the prediction probability of the point cloud identity recognition model and the prediction probability of the distance-velocity map identity recognition model. The predicted probability of the point cloud identity recognition model and the predicted probability of the distance-velocity map identity recognition model are combined. The feature is that acquiring point cloud data and range-velocity map data from millimeter-wave radar includes: When the millimeter-wave radar detects the movement of the monitored target, it continuously records the point cloud signal and range-velocity map signal of the millimeter-wave radar, and saves the data sample when the monitored target stops moving; The point cloud data is generated by calculating the three-dimensional coordinates and velocity of each point on the monitoring target based on the saved point cloud signal data samples. The saved distance-velocity map signal data samples are processed sequentially in both fast and slow time dimensions to form the distance-velocity map data. The preprocessing of the distance-velocity map data includes: The distance between the monitored target and the millimeter-wave radar is calculated based on the point cloud data, and the distance-velocity map data is extracted using the distance. The collaborative training of the point cloud identity recognition model and the distance-velocity map identity recognition model includes: The point cloud identity recognition model and the distance-velocity map identity recognition model were trained using labeled datasets, respectively. Make predictions on the unlabeled dataset to obtain a pseudo-labeled dataset; Reconstruct the training dataset; The point cloud identity recognition model and the distance-velocity map identity recognition model were retrained using the reconstructed training dataset.

2. The identity recognition method based on millimeter-wave radar according to claim 1, characterized in that, The reconstructed training dataset includes: a pseudo-labeled dataset and a labeled dataset.

3. The identity recognition method based on millimeter-wave radar according to claim 1, characterized in that, It also includes: establishing the model prediction loss function and the model reconstruction loss function. Cross-entropy is used as the prediction loss function for the model: ； Where N is the batch size; j is the category index; Labels representing identity, This represents data that is not the target of the monitoring. The representative is the data of the monitoring target; For model prediction; The mean squared error is used as the model reconstruction loss function: ； Where X represents the input distance-velocity map data; This is the decoded distance-velocity map data; It is an L2 norm.

4. The identity recognition method based on millimeter-wave radar according to claim 1, characterized in that, It also includes selecting pseudo-label datasets. H : ； in, For point cloud models Predicted probability of the category; For velocity-distance graph models Predicted probability of the category; These are weighting coefficients; This is the probability threshold.

5. The identity recognition method based on millimeter-wave radar according to any one of claims 1-4, characterized in that, It also includes methods for establishing point cloud identity recognition models, including: Using the preprocessed point cloud data as input, a feature extraction module is constructed using a multilayer perceptron to extract the attribute features of each point in the point cloud data; A spatiotemporal fusion module is constructed to extract the spatiotemporal features of the attribute features of each frame of point cloud data; Using the extracted spatiotemporal features, a predictor module is constructed using a multilayer perceptron.

6. The identity recognition method based on millimeter-wave radar according to any one of claims 1-4, characterized in that, Also includes: The method for establishing a distance-velocity map identity recognition model includes: Using the preprocessed distance-velocity map data as input, an encoder module is constructed to extract the attribute features of each frame's distance-velocity map; Build a decoder module to reconstruct the distance-velocity map; The attribute features of the distance-velocity map extracted by the encoder are used as input to construct a predictor module using a multilayer perceptron.

7. The identity recognition method based on millimeter-wave radar according to claim 1, characterized in that, The predicted probabilities of the point cloud identity recognition model and the predicted probabilities of the distance-velocity map identity recognition model are fused by constructing the following function f: （3）； in, These are the prediction weights for the point cloud model. ; (1- ) represents the prediction weights for the distance-velocity map model.

8. An identity recognition system based on millimeter-wave radar, characterized in that, include: The data acquisition module is used to acquire point cloud data and range-velocity map data from millimeter-wave radar; The data preprocessing module is used to preprocess the point cloud data and distance-velocity map data; The model training module is used to collaboratively train the preset point cloud identity recognition model and the distance-velocity map identity recognition model; The identity recognition module is used to fuse the predicted probability of the point cloud identity recognition model and the predicted probability of the distance-velocity map identity recognition model, and determine the identity of the monitored target by constructing a probability function; The data acquisition module includes: The target movement monitoring module is used to continuously record the point cloud signal and range-velocity map signal of the millimeter-wave radar when the millimeter-wave radar detects the movement of the monitored target, and save the data when the monitored target stops moving. The point cloud data acquisition module calculates the three-dimensional coordinates and velocity of each point on the monitored target based on the data stored by the target movement monitoring module, thereby forming the point cloud data. The distance-velocity map data acquisition module processes the stored distance-velocity map signal sequentially using fast time dimension processing and slow time dimension processing based on the data stored by the target movement monitoring module to form the distance-velocity map data. The data preprocessing module includes: A range-velocity map data preprocessing module is used to extract the range-velocity map data based on the distance between the monitored target and the millimeter-wave radar; The model training module includes: The model prediction module is used to predict the unlabeled dataset using the point cloud identity recognition model and the distance-velocity map identity recognition model, which are trained on labeled datasets respectively, to obtain a pseudo-labeled dataset. The data reconstruction module is used to reconstruct the training dataset; The training module uses the reconstructed training dataset to train the point cloud identity recognition model and the distance-velocity map identity recognition model, respectively.

9. The identity recognition system based on millimeter-wave radar according to claim 8, characterized in that, Also includes: The loss function module is used to establish the model prediction loss function and the model reconstruction loss function.

10. The identity recognition system based on millimeter-wave radar according to claim 8, characterized in that, The point cloud identity recognition model includes: The feature extraction module uses a multilayer perceptron to extract the attribute features of each point in each frame of the point cloud data; The spatiotemporal fusion module takes the attribute features of multi-frame point clouds as input and extracts the spatiotemporal features of the attribute features of each point cloud data sample. The point cloud predictor module uses the extracted spatiotemporal features to predict the identity of the detected target.

11. The identity recognition system based on millimeter-wave radar according to claim 8, characterized in that, The distance-velocity map identity recognition model includes: The encoder module is used to extract attribute features from the distance-velocity map; The decoder module takes the extracted attribute features as input and reconstructs the distance-velocity map; The predictor module takes the extracted attribute features as input and predicts the identity of the detected target.

Images