Human gait recognition method based on millimeter wave radar and dynamic sampling neural network

Abstract

The present invention discloses a method for human gait recognition based on millimeter-wave radar and dynamic sampling neural network. The method uses millimeter-wave radar to obtain radar point cloud data of human walking, obtains point cloud dynamic information by differential processing of adjacent point cloud data frames; passes the original point cloud data and the processed point stream data through the point cloud feature extraction network to obtain a feature sequence of point cloud and point stream; samples the point cloud and point stream feature subsequences containing rich gait information through the dynamic frame sampling module; performs feature fusion of the point cloud and point stream feature subsequences through the temporal feature aggregation module, and finally outputs the final human gait recognition result through the average pooling layer and multi-layer perceptron. The present invention designs a novel method for describing radar point cloud dynamic information and proposes a novel and efficient dynamic point cloud frame sampling method, which can significantly reduce the amount of calculation without significantly affecting the recognition performance.

Description

Technical Field

[0001] The present invention relates to the technical field of human gait recognition, and in particular to a human gait recognition method based on millimeter wave radar and dynamic sampling neural network, which combines millimeter wave radar and dynamic sampling neural network to achieve efficient recognition of human gait. Background Art

[0002] As homes become increasingly intelligent, gait recognition has become a crucial component of smart home management. Gait recognition technology enables real-time identification of individuals, enabling personalized customization of home functions and enhancing the user experience. It can also identify whether a pedestrian is a family member, improving home safety.

[0003] Most existing human gait recognition methods are based on camera sensors or wearable sensors. However, camera-based human gait recognition carries the risk of privacy leakage, while wearable sensor-based human gait recognition requires pedestrians to wear the corresponding sensors, limiting usage scenarios and reducing the user experience. Existing pedestrian gait recognition based on millimeter-wave radar sensors mostly relies on raw echo signals, but this can be computationally intensive. Existing pedestrian gait recognition based on radar point clouds uses only raw radar point cloud data, resulting in performance bottlenecks and the continued computational complexity, limiting its applicability. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention provides a method for human gait recognition based on millimeter-wave radar and a dynamic sampling neural network. This method uses millimeter-wave radar and a designed dynamic sampling neural network for gait recognition. This method utilizes point cloud data output by the millimeter-wave radar. Without adding complex feature processing, it performs differential processing on two consecutive millimeter-wave radar point cloud frames to fully exploit the dynamic acceleration information of the human body. Furthermore, dynamic frame sampling significantly reduces computing resource consumption, improves recognition speed, and achieves high-precision real-time gait identification, while minimizing the impact on recognition performance. This method is privacy-friendly, has a wide range of applications, high accuracy, and fast response.

[0005] The object of the present invention is achieved through the following technical solution: a method for human gait recognition based on millimeter wave radar and dynamic sampling neural network, the method comprising the following steps:

[0006] Step 1: Use millimeter-wave radar to collect radar point cloud information of indoor human walking status. Then, use the point cloud clustering algorithm and the inter-frame target association algorithm to filter out background radar points and obtain radar point cloud data of pedestrian targets.

[0007] Step 2: Take the continuous multi-frame radar point cloud sequence as a sample, perform differential processing on adjacent point cloud data frames, and obtain the dynamic information of the point cloud, which is defined as a point stream sequence;

[0008] Step 3: Input the point cloud sequence and point stream sequence into the point cloud feature extraction backbone network respectively to obtain a point cloud feature sequence and a point stream feature sequence;

[0009] Step 4: Through the dynamic frame sampling module, sample the point cloud feature sequence and point stream feature sequence to obtain information-rich point cloud and point stream feature subsequences;

[0010] Step 5: Through the temporal feature aggregation module, the information-rich point cloud and point stream feature subsequence are fused, and the final human gait recognition result is output through the average pooling layer and multi-layer perceptron.

[0011] Furthermore, the step 1 specifically includes:

[0012] Millimeter-wave radar transmits and receives electromagnetic wave reflection signals from the human body, and obtains millimeter-wave radar point cloud information of the human target through two-dimensional Fourier transform and CFAR algorithm. The continuous T-frame radar point cloud sequence is defined as S={P0,P1,…,P T-1}, where the point cloud of the i-th frame Contains N points, each point contains four-dimensional features: in The three-dimensional space coordinates of the representative point, Represents the Doppler velocity information of the point relative to the radar;

[0013] The classic point cloud clustering algorithm and inter-frame target association algorithm are used to cluster and track the radar point cloud of pedestrian targets and filter out background noise points.

[0014] The radar receives electromagnetic wave reflections from the human body, which are mixed to produce an intermediate frequency signal. This signal is then Fourier transformed in the range dimension to produce a range map, and then Fourier transformed in the Doppler dimension to produce a range-Doppler map. The constant false alarm (CFAR) algorithm is used on the range-Doppler map to produce a radar point cloud signal of the reflected target. The DBSCAN clustering algorithm is used to cluster the millimeter-wave radar point cloud, and the Hungarian algorithm is used for inter-frame target association and tracking to produce radar point cloud data of pedestrian targets.

[0015] Furthermore, in step 2, the two adjacent frames of radar point cloud P i and P i+1 In order to obtain the relative change information between the two frames of point cloud, the frame P i Point in In the next frame point cloud P i+1Find the point closest to the relative spatial position of the point And subtract the Doppler velocities of the two points to get the corresponding point flow vector in Point cloud frame P i Perform the above operation for each point in the to get the point stream frame Point stream data can provide more dynamic information about human walking from another dimension, which can help improve the accuracy of gait recognition.

[0016] Because the original millimeter-wave radar point cloud contains four-dimensional features, including one-dimensional Doppler velocity information in addition to the three-dimensional spatial coordinates, it can represent the velocity of different target point clouds relative to the millimeter-wave radar. Since millimeter-wave radar point cloud frames are collected at equal intervals, differential processing of two adjacent radar point cloud frames based on spatial position creates a point stream containing Doppler acceleration information for the radar point cloud. This allows for further exploration of the dynamic information of the millimeter-wave radar point cloud and improves the accuracy of gait recognition.

[0017] Furthermore, in step 3, feature vectors are extracted from each frame of input point cloud data and point stream data through the classic point cloud feature extraction network; for point cloud P i With point flow F i , and obtain the corresponding multidimensional feature vectors and Then the point cloud feature sequence is obtained and point flow feature sequence

[0018] To facilitate subsequent model training and inference, features must first be extracted from the point cloud and point stream data of pedestrian targets. Each point cloud frame can be represented as a multidimensional vector. Classic point cloud feature extraction networks include PointNet, PointNet++, and graph neural network-based networks such as AdaptConv.

[0019] Furthermore, the dynamic frame sampling module can dynamically filter out point cloud and point stream frames with low information content, retaining point cloud frames and point stream frames with rich information, significantly reducing the amount of model calculation while having little impact on classification performance;

[0020] For the point cloud frame P i , first use the multi-layer perceptron to project its features and obtain a projection vector z with a dimension of 1×2 i , the formula is as follows:

[0021]

[0022] Where GeLU represents the GeLU activation function, LN represents the layer normalization operation, and Linear represents the linear layer;

[0023] The projection vectors obtained by the multilayer perceptron for all point cloud frames are concatenated in the first dimension to obtain Z=Concat(z0,z1,…,z T-1 ); then the retention probability matrix is obtained through the Softmax operation Generate a binary mask by sampling π;

[0024] In order to make the sampling process differentiable, the retention probability matrix π is sampled using the reparameter method. During the backward propagation of the model, the gradient of the sampling operation is sampled from the Gumbel distribution. Therefore, the Gumbel-Softmax method is used to obtain the probability of retaining or cropping the point cloud for each frame, which is expressed as:

[0025]

[0026] where π i,j is the probability that the i-th element of π is in state j, π i,0 represents the probability of retaining the i-th element, π i,1 represents the probability of clipping the i-th element, g i,j It is an independent and identically distributed sampling of the Gumbel (0,1) distribution. τ is the temperature parameter. The larger τ is, the closer the output is to the uniform distribution. On the contrary, when τ is smaller, the output is closer to the one-hot distribution. In the training process, τ is gradually decayed so that the calculation results are closer and closer to the one-hot distribution, thus solving the difference problem between forward propagation and backward propagation.

[0027] The above calculations are performed on all point clouds to obtain the matrix Take the first column of matrix G as the binary mask, that is, Mask=G :,0 ∈{0,1} T , Mask is a vector whose elements are 0 or 1, and its length is the same as the number of point cloud frames, where 0 means filtering out the frame and 1 means retaining the frame.

[0028] By multiplying the binary mask with the point cloud feature sequence and the point stream feature sequence respectively, point cloud and point stream feature subsequences containing rich information can be obtained.

[0029] In order to control the ratio of the retained frames to the pre-set sampling ratio t, an additional mask loss function is added to supervise the training of the model. The mask loss function Loss mask The formula is as follows:

[0030]

[0031] Where sum represents the sum of all elements of Mask, that is, the number of elements in the Mask vector that are 1, and len represents the total length of Mask, which is T.

[0032] Because a few radar point cloud frames contain fewer points or only point cloud data corresponding to individual human body parts, they contain fewer valid features. To minimize the computational burden, improve inference speed, and facilitate practical deployment, the present invention uses dynamic frame sampling to automatically filter out radar point cloud frames with less valid information and retain radar point cloud frames rich in human features. This significantly reduces computing resource consumption and improves model inference speed while minimizing the impact on recognition performance.

[0033] Furthermore, the temporal feature aggregation module includes an L-layer standard Transformer model, which can model the long-distance interaction pattern of feature sequences, and thus has a good temporal feature fusion effect; each Transformer model includes a multi-head attention module, a multi-layer perceptron module, and a layer normalization module; through this temporal feature aggregation module, feature fusion of information-rich point cloud features and point stream features can be performed.

[0034] The standard Transformer model, thanks to its self-attention mechanism, can calculate the correlation between any samples in a sequence and capture their dependencies. Compared to recurrent neural networks (RNNs) and long short-term memory (LSTMs), the Transformer fundamentally addresses the problem of long-term dependencies. Furthermore, the Transformer avoids the sequential input method used in RNNs and LSTMs, instead inputting all samples at once, which allows for excellent parallelism and improves computational efficiency.

[0035] The advantages and beneficial effects of the present invention are as follows:

[0036] The present invention uses millimeter-wave radar to collect human gait data, which has the advantages of not infringing on privacy and being applicable to a wide range of scenarios. This method fully mines the dynamic acceleration information of the human body by performing differential processing on continuous millimeter-wave radar point cloud frames, thereby improving recognition accuracy. Through dynamic frame sampling, this method significantly reduces computing resource consumption and improves recognition reasoning speed while having a low impact on recognition performance, thereby achieving high-precision and fast human gait recognition. Different from existing human gait recognition methods, the method proposed in the present invention can mine more human gait dynamic information while taking into account both human gait recognition performance and computing speed, providing great convenience for the popularization of smart homes. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0038] Figure 2 A flowchart of point stream data acquisition provided by an embodiment of the present invention;

[0039] Figure 3 A workflow diagram of the feature fusion process, dynamic frame sampling module, and temporal feature aggregation module provided in an embodiment of the present invention;

[0040] Figure 4 A comparison chart of the accuracy of the dynamic frame sampling module and the random sampling method is provided for the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0042] like Figure 1 As shown, an embodiment of the present invention provides a method for human gait recognition based on millimeter wave radar and dynamic sampling neural network, the method comprising the following steps:

[0043] Step 1: Use millimeter-wave radar to collect radar point cloud information of indoor human walking status. Then, use the point cloud clustering algorithm and the inter-frame target association algorithm to filter out background radar points and obtain radar point cloud data of pedestrian targets.

[0044] Step 2: Take the continuous multi-frame radar point cloud sequence as a sample, perform differential processing on adjacent point cloud data frames, and obtain the dynamic information of the point cloud, which is defined as a point stream sequence;

[0045] Step 3: Input the point cloud sequence and point stream sequence into the point cloud feature extraction backbone network respectively to obtain a point cloud feature sequence and a point stream feature sequence;

[0046] Step 4: Through the dynamic frame sampling module, sample the point cloud feature sequence and point stream feature sequence to obtain information-rich point cloud and point stream feature subsequences;

[0047] Step 5: Through the temporal feature aggregation module, the information-rich point cloud and point stream feature subsequence are fused, and the final human gait recognition result is output through the average pooling layer and multi-layer perceptron.

[0048] Furthermore, the step 1 specifically includes:

[0049] Millimeter-wave radar transmits and receives electromagnetic wave reflection signals from the human body, and obtains millimeter-wave radar point cloud information of the human target through two-dimensional Fourier transform and CFAR algorithm. The continuous T-frame radar point cloud sequence is defined as S={P0,P1,…,P T-1}, where the point cloud of the i-th frame Contains N points, each point contains four-dimensional features: in The three-dimensional space coordinates of the representative point, Represents the Doppler velocity information of the point relative to the radar;

[0050] The classic point cloud clustering algorithm and inter-frame target association algorithm are used to cluster and track the radar point cloud of pedestrian targets and filter out background noise points.

[0051] Furthermore, the step 2 specifically includes:

[0052] For two adjacent frames of radar point cloud P i and P i+1 In order to obtain the relative change information between the two frames of point cloud, the frame P i Point in In the next frame point cloud P i+1 Find the point closest to the relative spatial position of the point And subtract the Doppler velocities of the two points to get the corresponding point flow vector in

[0053] Point cloud frame P i Perform the above operation for each point in the to get the point stream frame Point stream data can provide more dynamic information about human walking from another dimension, which can help improve the accuracy of gait recognition.

[0054] Furthermore, in step 3, feature vectors are extracted from each frame of input point cloud data and point stream data through the classic point cloud feature extraction network; for point cloud P i With point flow F i , and obtain the corresponding multidimensional feature vectors and Then the point cloud feature sequence is obtained and point flow feature sequence

[0055] Furthermore, the dynamic frame sampling module can dynamically filter out point cloud and point stream frames with low information content, retaining point cloud frames and point stream frames with rich information, significantly reducing the amount of model calculation while having little impact on classification performance;

[0056] For the point cloud frame P i, first use the multi-layer perceptron to project its features and obtain a projection vector z with a dimension of 1×2 i , the formula is as follows:

[0057]

[0058] Where GeLU represents the GeLU activation function, LN represents the layer normalization operation, and Linear represents the linear layer;

[0059] The projection vectors obtained by the multilayer perceptron for all point cloud frames are concatenated in the first dimension to obtain Z=Concat(z0,z1,…,z T-1 ); then the retention probability matrix is obtained through the Softmax operation Generate a binary mask by sampling π;

[0060] By multiplying the binary mask with the point cloud feature sequence and the point stream feature sequence respectively, point cloud and point stream feature subsequences containing rich information can be obtained.

[0061] Furthermore, in the dynamic frame sampling module, in order to make the sampling process differentiable, the retention probability matrix π is sampled using a reparameterization method. During the model backpropagation process, the gradient of the sampling operation is sampled from the Gumbel distribution. Therefore, the Gumbel-Softmax method is used to obtain the probability of retaining or cropping the point cloud for each frame, which is expressed as:

[0062]

[0063] where π i,j is the probability that the i-th element of π is in state j, π i,0 represents the probability of retaining the i-th element, π i,1 represents the probability of clipping the i-th element, g i,j It is an independent and identically distributed sampling of the Gumbel (0,1) distribution. τ is the temperature parameter. The larger τ is, the closer the output is to the uniform distribution. On the contrary, when τ is smaller, the output is closer to the one-hot distribution. In the training process, τ is gradually decayed so that the calculation results are closer and closer to the one-hot distribution, thus solving the difference problem between forward propagation and backward propagation.

[0064] The above calculations are performed on all point clouds to obtain the matrix Take the first column of matrix G as the binary mask, that is, Mask=G :,0 ∈{0,1} T , Mask is a vector whose elements are 0 or 1, and its length is the same as the number of point cloud frames, where 0 means filtering out the frame and 1 means retaining the frame.

[0065] Furthermore, in the dynamic frame sampling module, in order to control the ratio of the retained frames to a preset sampling ratio t, a mask loss function is added to supervise the training of the model. The mask loss function Loss mask The formula is as follows:

[0066]

[0067] Where sum represents the sum of all elements of Mask, that is, the number of elements in the Mask vector that are 1, and len represents the total length of Mask, that is, T. Specifically, t can be set as needed and can be set to 0.3-1.

[0068] Furthermore, the temporal feature aggregation module includes an L-layer standard Transformer model, which can model the long-distance interaction pattern of feature sequences, and thus has a good temporal feature fusion effect; each Transformer model includes a multi-head attention module, a multi-layer perceptron module, and a layer normalization module; through this temporal feature aggregation module, feature fusion of information-rich point cloud features and point stream features can be performed.

[0069] The above point flow data acquisition process is as follows Figure 2 shown.

[0070] The above feature fusion process and the workflow of the dynamic frame sampling module and the temporal feature aggregation module are shown in the attached figure. Figure 3 shown.

[0071] The effects of the present invention are further described below in conjunction with the test results of the actually collected millimeter-wave radar point cloud gait data.

[0072] This millimeter-wave radar point cloud gait data was collected in an indoor setting, primarily from the walking gait data of twenty volunteers. The millimeter-wave radar used for data collection was TI's IWR6843, which uses three transmitting antennas and four receiving antennas to generate three-dimensional point cloud data with a sampling interval of 0.1 seconds. The millimeter-wave radar point cloud data was first clustered using the DBSCAN algorithm, followed by inter-frame object association using the Hungarian algorithm. This yielded gait point cloud data for each volunteer. Using a non-overlapping sliding window of length 20, 20 consecutive radar point cloud frames were considered a sample. The point cloud data was then processed to generate point stream data, which was then passed through a point cloud feature extraction backbone network to extract both point cloud and point stream features. The dynamic frame sampling module then cropped radar point cloud frames with less information, while retaining those with more information. The temporal feature aggregation module then generated global features for this sample. Finally, an average pooling layer and a multi-layer perceptron were used to obtain the final human gait recognition results. The proposed method for human gait recognition based on millimeter-wave radar and a dynamic sampling neural network was compared with other models, including mmGaitNet, PL (PointNet combined with LSTM), P+L (PointNet++ combined with LSTM), and SRPNet. The experimental results, shown in Table 1, show that the proposed method for human gait recognition based on millimeter-wave radar and a dynamic sampling neural network outperformed the other models in terms of accuracy, precision, and F1 value, demonstrating superior gait recognition performance.

[0073] Table 1 Comparison of recognition results of the method proposed in this invention and other methods

[0074] method Accuracy Accuracy F1 value mmGaitNet 39.10±1.15 38.74±1.99 37.34±1.76 PL 80.77±3.15 81.52±4.81 79.91±3.28 P+L 81.23±2.98 80.84±3.25 80.84±2.84 SRPNet 84.91±2.86 85.26±2.47 84.67±2.18 The method proposed by the present invention 96.52±1.26 96.48±1.32 96.36±1.37

[0075] The effectiveness of the dynamic frame sampling module under different sampling ratios was then verified. The comparison method was random sampling, that is, according to the set sampling ratio, a binary mask was randomly generated, and the point cloud feature vector and the corresponding point stream feature vector were sampled. The sampling ratio was gradually adjusted from 1.0 to 0.3, with an adjustment interval of 0.1. The performance indicator for comparison was accuracy. The experimental results are shown in the attached figure. Figure 4 As shown in the figure, it can be seen that the accuracy of the two methods will decrease with the decrease of the proportion of radar point cloud frame retention. However, compared with the random sampling method, the dynamic frame sampling module based on millimeter wave radar and dynamic sampling neural network proposed in the present invention has a smaller decrease, and even when the retention ratio is as low as 0.3, the accuracy can still be maintained at a high level.

[0076] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A human gait recognition method based on millimeter wave radar and dynamic sampling neural network, characterized in that: The following steps are involved: Step 1: Use millimeter-wave radar to collect radar point cloud information of indoor human walking status. Then, use the point cloud clustering algorithm and the inter-frame target association algorithm to filter out background radar points and obtain radar point cloud data of pedestrian targets. Step 2: Take the continuous multi-frame radar point cloud sequence as a sample, perform differential processing on adjacent point cloud data frames, and obtain the dynamic information of the point cloud, which is defined as a point stream sequence; Specifically include: For two adjacent frames of radar point cloud P i and P i+1 In order to obtain the relative change information between the two frames of point cloud, the frame P i Point in In the next frame point cloud P i+1 Find the point closest to the relative spatial position of the point And subtract the Doppler velocities of the two points to get the corresponding point flow vector in The three-dimensional space coordinates of the representative point, Represents the Doppler velocity information of the point relative to the radar; for the point cloud frame P i The above Doppler velocity subtraction operation is performed on each point in the image to obtain the point flow frame. Step 3: Input the point cloud sequence and point stream sequence into the point cloud feature extraction backbone network respectively to obtain the point cloud feature sequence and point stream feature sequence; extract the feature vector of each frame of input point cloud data and point stream data through the classic point cloud feature extraction network; for the point cloud P i With point flow F i , and obtain the corresponding multidimensional feature vectors and Then the point cloud feature sequence is obtained and point flow feature sequence Step 4: Using a dynamic frame sampling module, sample the point cloud feature sequence and the point stream feature sequence to obtain information-rich point cloud and point stream feature subsequences; the dynamic frame sampling module can dynamically filter out point cloud and point stream frames with low information content and retain point cloud frames and point stream frames with rich information; For the point cloud frame P i , first use the multi-layer perceptron to project its features and obtain a projection vector z with a dimension of 1×2 i , the formula is as follows: Where GeLU represents the GeLU activation function, LN represents the layer normalization operation, and Linear represents the linear layer; The projection vectors obtained by multi-layer perceptron for all point cloud frames are concatenated in the first dimension to obtain Z=Concat(z0,z1,…,z T-1 ); then the retention probability matrix is obtained through the Softmax operation Generate a binary mask by sampling π; Multiplying the binary mask with the point cloud feature sequence and the point stream feature sequence respectively can obtain point cloud and point stream feature subsequences containing rich information; Step 5: Through the temporal feature aggregation module, the information-rich point cloud and point stream feature subsequence are fused, and the final human gait recognition result is output through the average pooling layer and multi-layer perceptron.

2. The method for human gait recognition based on millimeter wave radar and dynamic sampling neural network according to claim 1, characterized in that: The step 1 specifically includes: Millimeter-wave radar transmits and receives electromagnetic wave reflection signals from the human body, and obtains millimeter-wave radar point cloud information of the human target through two-dimensional Fourier transform and CFAR algorithm. The continuous T-frame radar point cloud sequence is defined as S = {P0, P1, ..., P T-1 }, where the point cloud of the i-th frame Contains N points, each point contains four-dimensional features: in The three-dimensional space coordinates of the representative point, Represents the Doppler velocity information of the point relative to the radar; The point cloud clustering algorithm and the inter-frame target association algorithm are used to cluster and track the radar point cloud of pedestrian targets and filter out background noise points.

3. The method for human gait recognition based on millimeter wave radar and dynamic sampling neural network according to claim 1, characterized in that: In the dynamic frame sampling module, in order to make the sampling process differentiable, the retention probability matrix π is sampled using a reparameterization method. During the model backpropagation process, the gradient of the sampling operation is sampled from the Gumbel distribution. Therefore, the Gumbel-Softmax method is used to obtain the probability of retaining or cropping the point cloud of each frame, which is expressed as: where π i,j is the probability that the i-th element of π is in state j, π i,0 represents the probability of retaining the i-th element, π i,1 represents the probability of clipping the i-th element, g i,j It is an independent and identically distributed sampling of the Gumbel (0,1) distribution. τ is the temperature parameter. The larger τ is, the closer the output is to the uniform distribution. On the contrary, when τ is smaller, the output is closer to the one-hot distribution. In the training process, τ is gradually decayed so that the calculation results are closer and closer to the one-hot distribution, thus solving the difference problem between forward propagation and backward propagation. The above calculations are performed on all point clouds to obtain the matrix Take the first column of matrix G as the binary mask, that is, Mask = G :,0 ∈{0,1} T , Mask is a vector whose elements are 0 or 1, and its length is the same as the number of point cloud frames, where 0 means filtering out the frame and 1 means retaining the frame.

4. The method for human gait recognition based on millimeter wave radar and dynamic sampling neural network according to claim 3, characterized in that: In the dynamic frame sampling module, in order to control the proportion of the retained frames to the preset sampling ratio t, an additional mask loss function is added to supervise the training of the model. The mask loss function Loss mask The formula is as follows: Where sum represents the sum of all elements of Mask, that is, the number of elements in the Mask vector that are 1, and len represents the total length of Mask, which is T.

5. The method for human gait recognition based on millimeter wave radar and dynamic sampling neural network according to claim 1, characterized in that: The temporal feature aggregation module includes an L-layer standard Transformer model, which can model the long-distance interaction pattern of feature sequences; each Transformer model includes a multi-head attention module, a multi-layer perceptron module, and a layer normalization module; through the temporal feature aggregation module, feature fusion of information-rich point cloud features and point stream features is performed.

Images