Human body point cloud analysis method based on millimeter wave radar

Abstract

This invention belongs to the field of radar point cloud recognition and processing technology, and discloses a method for human point cloud parsing based on millimeter-wave radar. This method uses radar point clouds captured from a person as input, identifies the body parts corresponding to each point in the point cloud, groups and parses the point cloud. The point cloud data collected by millimeter-wave radar is first fed into a multilayer perceptron for feature extraction. The features are then learned through an encoding module, followed by a symmetrical decoder structure to decode local features into predicted human body parsing information. After training, the model is saved and used for human body point cloud parsing and subsequent pose estimation tasks. This invention improves the accuracy of point cloud parsing tasks under sparse point clouds, enhances the robustness of the system, reduces network structure complexity, reduces network training overhead, and improves computational efficiency. It can significantly improve the accuracy of human body parsing with fewer training rounds.

Description

Technical Field

[0001] This invention belongs to the field of radar point cloud recognition and processing technology, specifically relating to a method for analyzing human point clouds based on millimeter-wave radar. Background Art

[0002] Human point cloud parsing originates from point cloud segmentation in point cloud tasks. Point cloud segmentation divides the point cloud in a scene into different parts or categories to identify and distinguish different objects, surfaces, or features. In the human point cloud parsing task, radar point clouds captured from the human body are used as input to identify the body part corresponding to each point in the point cloud. That is, which body part each point in the point cloud belongs to. The point cloud with semantic labels is generated as the final output. In fact, various computer vision tasks have demonstrated that incorporating semantic information into loops can significantly improve the accuracy of pose estimation, activity recognition, and person recognition. This advantage is even more pronounced for millimeter-wave radar, because the point clouds generated by this low-cost radio frequency sensor are inherently of lower quality than the images generated by vision sensors. Charles R. Qi et al. provided a unified architecture, PointNet (PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation), for applications ranging from object classification and part segmentation to scene semantic parsing, achieving state-of-the-art performance at the time using only fully connected layers. The subsequently proposed PointNet++ improved upon PointNet by simply using max pooling to aggregate information and lacking the ability to extract local features. By progressively increasing the scanning scale, local features at different scales are obtained. Similarly, to extract local features, Meng-Hao Guo et al. introduced a transformer structure in the encoding layer, resulting in more fine-grained local features (PCT: Point cloud transformer). Xu Ma et al., however, argued that detailed local geometric information might not be key to point cloud analysis, and therefore introduced a pure residual MLP network, PointMLP (Rethinking Network Design and Local Geometry in PointCloud: A Simple Residual MLP Framework). While it did not use a complex local feature extractor, it still achieved good performance and superior inference speed. Guocheng Qia re-examined the structure of PointNet++ and compared it with subsequent PointNet versions, PointMLP and Point Transformer. The results showed that although the accuracy of PointNet++ was significantly surpassed by later versions, this performance improvement was largely due to more advanced data augmentation and optimization techniques and larger models. In terms of architectural innovation, later versions did not actually offer significant improvements.Therefore, while using more advanced optimization techniques, existing technologies have introduced inverse residual bottleneck design and separable MLP to achieve efficient and effective model expansion, and proposed the PointNeXt model (PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies).

[0003] However, tasks involving point cloud segmentation are typically performed on LiDAR, which possesses denser point clouds and is therefore better able to segment different objects. There are few inventions for millimeter-wave radar, which has much sparser point clouds. The sparse point clouds of millimeter-wave radar often fail to fully represent the outline of an object, only depicting a portion of it. Furthermore, noise points generated by multipath effects and specular reflection from the human body have a greater impact on the model under sparse point clouds, thus affecting the model's robustness. Summary of the Invention

[0004] To address the aforementioned technical challenges, this invention proposes a human point cloud parsing method based on millimeter-wave radar. This method uses a multilayer perceptron to extract higher-level features from human point cloud data acquired by millimeter-wave radar. It then employs sampling and grouping algorithms to group these features and further extracts more precise local feature information, allowing the model to focus on finer-grained features. Finally, the data is fed into a neural network for training. After training, this method can achieve human point cloud parsing and improve parsing accuracy. The proposed neural network also enhances the performance of downstream tasks such as pose estimation, person recognition, and gait recognition.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] This invention is a method for analyzing human point clouds based on millimeter-wave radar. The method is implemented using a neural network and specifically includes the following steps:

[0007] Step 1, Radar point cloud data processing and label generation: Collect raw data, use a depth camera to collect human skeletal key point data, use radar to acquire human point cloud data, synchronize with timestamps, and then use the synchronized skeletal key point data.

[0008] Step 2: The human point cloud data acquired by radar is processed by the backbone multilayer perceptron layer of the neural network for global feature extraction. Then, local points are obtained by sampling and grouping the centroid points through the sampling layer and grouping layer of the neural network.

[0009] Step 3: Feed the local points obtained after grouping in Step 3 into the neural network, and use abstract convolutional layers to extract local features related to the local points;

[0010] Step 4: Use the symmetrical decoding layer in the neural network to decode the local features obtained in Step 3. The global features and the local features of each layer correspond to a decoder operation, and residual connections are made to prevent the model from overfitting. Finally, the predicted key points of each point belong to which part of the human body skeleton.

[0011] A further improvement of the present invention is that: the neural network specifically includes an encoder and a decoder, the encoder and decoder using residual connections; the encoder includes a sampling layer, a grouping layer, an abstract convolutional layer, and a reverse residual block; the sampling layer includes a sampling operation with the furthest distance sampling to obtain the sample set as the centroid of the grouping operation; the grouping layer includes a grouping operation using the k-nearest neighbor algorithm to divide the human point cloud data into several groups centered on the sampling machine; the input data of the abstract convolutional layer is the coordinate information of the human point cloud data and local features related to local points; the local features related to local points are obtained by a single two-dimensional convolution combined with batch normalization to obtain a dimension of [B, Local features related to local points [B, 64, N, k] are extracted using a single max pooling operation. The extracted local features and coordinate information from the human point cloud data flow to a reverse residual block. This reverse residual block uses a single 2D convolution combined with batch normalization to obtain a feature vector of dimension [B, 64, N, k]. Then, a separable multilayer perceptron is implemented by first performing a maximum value operation, simplifying the local features and dimension to [B, 64, N] to reduce computation. Feature extraction is then performed through two MLP layers. The first layer quadruples the number of channels, while the second layer restores the number of channels. This inverted bottleneck design enriches feature extraction. The output data flows to the next encoder layer while using residual connections as input to the symmetric decoder.

[0012] A further improvement of the present invention is that step 2 specifically includes the following steps:

[0013] Step 2.1: Perform global feature extraction on the human body point cloud data acquired by radar. The coordinate information of the human body point cloud data acquired by radar is mapped to a higher dimension than three-dimensional space through the backbone multilayer perceptron layer, and global feature data reflecting the global information of the human body is extracted for further learning.

[0014] Step 2.2: Downsample the human point cloud of each frame to obtain n / 4 uniformly distributed centroids in the sampling layer for subsequent grouping.

[0015] Step 2.3: Using the grouping layer of the neural network, with the human point cloud set and the centroid set as input, and with the centroid as the center, select points from the k surrounding human point cloud data for grouping.

[0016] A further improvement of the present invention is that, in step 2.3, let the current human point cloud set be {x1, x2, x3, ..., x...} n Using the centroid as the center, select k points from the surrounding human point cloud data and group them accordingly. The specific steps include:

[0017] Step 2.3.1: Calculate the distance of each point in the human point cloud from the centroid of the point cloud, and select the point in the human point cloud data that is farthest from the centroid and include it in the sampling set;

[0018] Step 2.3.2: Calculate the Euclidean distance between each point in the human point cloud data and the sampling set point, and select the point x in the human point cloud data with the largest distance. i As the next sampling point;

[0019] Step 2.3.3: Repeat step 2.3.2 n / 4 times, when the sampling set {x} l …,x m When there is one or more centroids in the data, for point x in the human point cloud data, calculate the distance {d} for each centroid in turn. i ,…,d m}, select the minimum distance d minx Record the data, and finally select the point x from the human point cloud data with the largest distance. i As the next sampling point.

[0020] A further improvement of the present invention is that, in step 2.3, the grouping specifically involves using the k-nearest neighbor algorithm for the centroid point x. i Select k points x distances from the centroid in sequence. i The nearest neighboring points are grouped together, and the local features between the neighboring points are learned.

[0021] A further improvement of the present invention is that step 3, which involves feeding local points into a neural network for local feature extraction, specifically includes the following steps:

[0022] Step 3.1: The abstract convolutional layer uses three 2D convolutions and batch normalization operations to extract local features from the input centroid coordinates and grouping points;

[0023] Step 3.2: The extracted local features are fed into the inverse residual block, which uses one grouping operation and three 2D convolution operations to enrich the feature dimensions and alleviate the gradient vanishing problem. Step 3.3: The inverted bottleneck design, which uses the first layer MLP to increase the number of channels to 4 times and the second layer MLP to restore the number of channels, increases the output channels of the first layer MLP by 4 times, thus enriching feature extraction.

[0024] Step 3.4: Use residual connections between the encoder and decoder. The decoder then predicts the joint category to which each radar point in the human body belongs.

[0025] A further improvement of the present invention is that, in step 4, a symmetrical decoding layer in the neural network is used to decode the local features obtained in step 3, specifically including the following steps:

[0026] Step 4.1: Use the decoding block to decode local features related to local points;

[0027] Step 4.2: Update the parameter information of the weights and biases, and output the loss function.

[0028] Step 4.3: After the network training is complete, save the training model and test the neural network according to the test samples to verify the effectiveness and accuracy of the neural network.

[0029] A further improvement of the present invention is that step 1 specifically includes the following steps:

[0030] Step 1.1: Use a depth camera to collect key skeletal information of the human body and use radar to acquire human point cloud information;

[0031] Step 1.2: The key point information of the human skeleton and the human point cloud information are synchronized using the timestamp at the time of collection;

[0032] Step 1.3: Fill and sort the human body point cloud information data;

[0033] Step 1.4: Calculate the distances between key skeletal points corresponding to the point distances in the human body point cloud data;

[0034] Step 1.5: Select the category of the skeletal key point that is closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold, the human body point cloud information data point is not related to the human body skeletal key points and is classified as an invalid point.

[0035] A further improvement of the present invention is that, in step 1.1, the human body point cloud information acquired by the radar includes the x, y, z coordinates in three-dimensional space, velocity, signal-to-noise ratio, and timestamp.

[0036] 10. The method for analyzing human point clouds based on millimeter-wave radar according to claim 8, characterized in that: in step 1.3, filling the human point cloud information data in step 1.3 means that each frame of human point cloud information data contains 64 key points. For sparse frames, the centroid of the points in the existing human point cloud information data is calculated, and the centroid is used as the filling point so that each frame contains 64 key points.

[0037] A further improvement of the present invention is that: the sorting of human point cloud information data in step 1.3 is specifically as follows: the input human point cloud information is sorted according to the rules, that is, sorted in ascending order of x, y, z coordinates, that is, first sorted in ascending order of x coordinates of the points in the human point cloud information data, then sorted in ascending order of y coordinates of the points in the human point cloud information data with the same x coordinates, and so on.

[0038] The beneficial effects of the present invention are:

[0039] This invention reduces computational overhead and improves computational efficiency by using a lightweight network.

[0040] Based on millimeter-wave radar design, this invention proposes a series of methods, including k-nearest neighbor sampling, separated MLP, and residual connection, to solve the problem of difficult training and recognition of sparse point clouds.

[0041] This invention successfully applies point cloud classification tasks to human body analysis and demonstrates its effectiveness for downstream visual tasks.

[0042] This invention can significantly improve the accuracy of human body analysis with fewer training rounds. Attached Figure Description

[0043] Figure 1 This is a flowchart of the human body point cloud analysis method of the present invention.

[0044] Figure 2 This is a flowchart of the process of generating skeletal point cloud data and labels in this invention.

[0045] Figure 3 This is a flowchart of the centroid sampling and grouping stage of the present invention.

[0046] Figure 4 This is a flowchart illustrating the process of point cloud local feature encoding, extraction, and decoding in this invention.

[0047] Figure 5 This is a flowchart of the neural network operation of this invention. Detailed Implementation

[0048] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0049] like Figure 1-5As shown, this invention is a method for human body point cloud parsing based on millimeter-wave radar. This method is implemented through a neural network, specifically including an encoder and a decoder connected by residuals. The encoder includes a sampling layer, a grouping layer, an abstract convolutional layer, and inverse residual blocks. The sampling layer includes a farthest-distance sampling operation to obtain a sample set as the centroid of the grouping operation. The grouping layer includes a grouping operation using the k-nearest neighbor algorithm to divide the human body point cloud data into several groups centered on the sampling layer. The input data of the abstract convolutional layer consists of the coordinate information of the human body point cloud data and local features related to local points. The local features related to local points are processed using a single two-dimensional convolution combined with batch processing. The local features are normalized to obtain local features of dimension [B, 64, N, k]. A single max pooling operation is then used to extract these local features and the coordinate information of the human point cloud data, which flow to the inverse residual block. This inverse residual block uses a single 2D convolution combined with batch normalization to obtain a feature vector of dimension [B, 64, N, k]. Subsequently, a maximum value operation is performed to implement a separable multilayer perceptron, simplifying the local features and dimension to [B, 64, N] to reduce computation. Feature extraction is then performed through two MLP layers. The first layer quadruples the number of channels, while the second layer restores the number of channels. This inverted bottleneck design enriches the feature extraction. The output data flows to the next encoder layer while using residual connections as input to the symmetric decoder.

[0050] Specifically, the human body point cloud analysis method includes the following steps:

[0051] Step 1: Radar Point Cloud Data Processing and Label Generation: Raw data is collected, including skeletal keypoint data of the human body obtained using a depth camera, and human point cloud data acquired using radar. Synchronization is performed using timestamps, and then the synchronized skeletal keypoint data is used to generate parsed label data for the human point cloud. This includes the following steps:

[0052] Step 1.1: Use a depth camera to collect key skeletal information of the human body, and use radar to obtain human body point cloud information, including x, y, z coordinates in three-dimensional space, velocity, signal-to-noise ratio, and timestamp.

[0053] Step 1.2: The key point information of the human skeleton and the human point cloud information are synchronized using the timestamp at the time of collection;

[0054] Step 1.3: Fill and sort the human body point cloud information data.

[0055] Filling in the human body point cloud information data means that each frame of human body point cloud information data contains 64 key points. For sparse frames, the centroid of the points in the existing human body point cloud information data is calculated and used as the filling point so that each frame contains 64 key points.

[0056] The specific process of sorting the human point cloud information data is as follows: the input human point cloud information is sorted according to the rules, and sorted in ascending order of x, y, z coordinates. That is, firstly, the points in the human point cloud information data are sorted in ascending order of x coordinates, and then the points in the human point cloud information data with the same x coordinates are sorted in ascending order of y coordinates, and so on.

[0057] Step 1.4: Calculate the distances between key skeletal points corresponding to the point distances in the human body point cloud data;

[0058] Step 1.5: Select the category of the skeletal key point closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold of 30cm, the human body point cloud information data point is not related to the human body skeletal key points and is classified as an invalid point.

[0059] Step 2: The human point cloud data acquired by radar is processed by the backbone multilayer perceptron layer of the neural network for global feature extraction. Then, the centroid points are sampled and grouped by the sampling layer and grouping layer of the neural network to obtain local points for subsequent local feature extraction.

[0060] Specifically, the steps include the following:

[0061] Step 2.1: Perform global feature extraction on the human body point cloud data acquired by radar. The coordinate information of the human body point cloud data acquired by radar is mapped to a higher dimension than three-dimensional space through the backbone multilayer perceptron layer, and global feature data reflecting the global information of the human body is extracted for further learning.

[0062] Step 2.2: Downsample the human point cloud of each frame to obtain n / 4, or 16, uniformly distributed centroids in the sampling layer for subsequent grouping.

[0063] Step 2.3: Using the grouping layer of the neural network, with the human point cloud set and the centroid point set as input, and centering on the centroid, select points from the surrounding k human point cloud data to group them. In this step, let the current human point cloud set be {x1, x2, x3, ..., x...} n Using the centroid as the center, select k points from the surrounding human point cloud data and group them accordingly. The specific steps include:

[0064] Step 2.3.1: Calculate the distance of each point in the human point cloud from the centroid of the point cloud, and select the point in the human point cloud data that is farthest from the centroid and include it in the sampling set;

[0065] Step 2.3.2: Calculate the Euclidean distance between each point in the human point cloud data and the sampling set point, and select the point x in the human point cloud data with the largest distance. i As the next sampling point;

[0066] Step 2.3.3: Repeat step 2.3.2 n / 4 times, when the sampling set {x} l …,x m When there is one or more centroids in the data, for point x in the human point cloud data, calculate the distance {d} for each centroid in turn. i ,…,d m}, select the minimum distance d minx Record the data, and finally select the point x from the human point cloud data with the largest distance. i As the next sampling point.

[0067] In step 2.3, the grouping is specifically as follows: using the k-nearest neighbor algorithm, for the centroid x... i Select k points x distances from the centroid in sequence. i The nearest neighboring points are grouped together, and the local features between the neighboring points are learned.

[0068] Step 3: The local points obtained after grouping in Step 2 are fed into the neural network. Abstract convolutional layers are used to extract local features related to these local points. These local features are more focused on local changes in the human body than global features, thus providing the model with more granular features.

[0069] Step 3 involves feeding local points into a neural network for local feature extraction, specifically including the following steps:

[0070] Step 3.1: The abstract convolutional layer uses three 2D convolutions and batch normalization operations to extract local features from the input centroid coordinates and grouping points;

[0071] Step 3.2: The extracted local features are fed into the inverse residual block. The inverse residual block uses one grouping operation and three two-dimensional convolution operations to enrich the feature dimensions and alleviate the gradient vanishing problem. A residual connection is added between the input and output to alleviate the gradient vanishing problem, and the first two-dimensional convolutional layer is separated from the remaining convolutional layers using the separated multilayer perceptron technique to reduce the amount of computation and enhance point-by-point feature extraction. The inverse residual block separates the multilayer perceptron into a single layer (between the grouping and reduction layers) that computes on the neighborhood features and two layers of point features (after reduction).

[0072] Step 3.3: The inverted bottleneck design, which uses the first layer MLP to expand the number of channels to 4 times the original number and the second layer MLP to restore the number of channels, expands the output channels of the first layer MLP by 4 times, thus enriching feature extraction.

[0073] Step 3.4: Use residual connections between the encoder and decoder. The decoder then predicts the joint category to which each radar point in the human body belongs.

[0074] Step 4: Use a symmetrical decoding layer in the neural network to decode the local features obtained in Step 3. Each global feature and each layer's local features correspond to a decoder operation. Residual connections are used to prevent overfitting. Finally, the predicted skeletal keypoints of each point are obtained. The decoding of the local features obtained in Step 3 using a symmetrical decoding layer in the neural network includes the following steps:

[0075] Step 4.1: Use the decoding block to decode local features related to local points;

[0076] Step 4.2: Update the parameter information of the weights and biases, and output the loss function.

[0077] Step 4.3: After the network training is complete, save the training model and test the neural network according to the test samples to verify the effectiveness and accuracy of the neural network.

[0078] To compare the effectiveness of this method, a comparative experiment was designed to compare it with existing methods. Three model methods were selected: PointNet, PointNet++, and PointNext. The datasets collected by this method and a publicly available human pose point cloud dataset, Mars, were used to classify the human body into three categories (torso, left hand and left leg, right hand and right leg), five categories (torso, left hand, left leg, right hand, right leg), and all categories. Each skeletal keypoint corresponds to a classification label. The comparison results using the Mars dataset and the dataset collected by this method are shown in Tables 1 and 2.

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] As can be seen from the above description, compared with existing technologies, the millimeter-wave-based human body radar point cloud parsing algorithm proposed in this invention accurately distinguishes the human body key point category to which the radar point belongs by using the global and local features of the human body radar point cloud. It uses convolutional neural networks for feature extraction and training, which greatly improves the accuracy of human body radar point cloud parsing. At the same time, this model also has an improving effect on downstream human body tasks, and can significantly enhance the performance of downstream tasks.

[0084] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for analyzing human point clouds based on millimeter-wave radar, characterized in that: The human body point cloud parsing method is implemented through a neural network. Specifically, the human body point cloud parsing method includes the following steps: Step 1, Radar point cloud data processing and label generation: Collect raw data, use a depth camera to collect human skeletal key point data, use radar to acquire human point cloud data, synchronize with timestamps, and then use the synchronized skeletal key point data. Step 2: The human point cloud data acquired by radar is processed by the backbone multilayer perceptron layer of the neural network for global feature extraction. Then, local points are obtained by sampling and grouping the centroid points through the sampling layer and grouping layer of the neural network. Step 3: Feed the local points obtained after grouping in Step 2 into the neural network and use abstract convolutional layers to extract local features related to the local points; Step 4: Use a symmetrical decoding layer in the neural network to decode the local features obtained in Step 3. Each global feature and each layer's local features correspond to a decoder operation, and residual connections are used to prevent overfitting. Finally, the predicted skeletal keypoints of each point in the human body are obtained. The neural network specifically includes an encoder and a decoder, which use residual connections. The encoder includes a sampling layer, a grouping layer, an abstract convolutional layer, and a reverse residual block. The sampling layer includes a farthest-distance sampling operation to obtain a sample set as the centroid of the grouping operation. The grouping layer includes a grouping operation using the k-nearest neighbor algorithm to divide the human point cloud data into several groups centered on the sampling machine. The abstract convolutional layer takes as input the coordinate information of the human point cloud data and local features related to local points. The local features related to local points are processed using a single two-dimensional convolution combined with batch normalization to obtain local features related to local points with dimensions [B, 64, N, k]. A single max pooling operation is then used to extract... The local features related to the extracted local points and the coordinate information of the human point cloud data flow to the inverse residual block. The inverse residual block uses a two-dimensional convolution and batch normalization to obtain a feature vector with a dimension of [B, 64, N, k]. Then, a separable multilayer perceptron is realized by first performing a maximum value operation, which simplifies the local features related to the local points and the dimension to [B, 64, N] to reduce the amount of computation. Then, feature extraction is performed through two layers of MLP, where the number of channels is increased to 4 times the original number in the first layer and the number of channels is restored in the second layer. The output data uses residual connections as the input of the symmetric decoder while flowing to the next layer encoder. Step 2 specifically includes the following steps: Step 2.1: Perform global feature extraction on the human body point cloud data acquired by radar. The coordinate information of the human body point cloud data acquired by radar is mapped to a higher dimension than three-dimensional space through the backbone multilayer perceptron layer, and global feature data reflecting the global information of the human body is extracted. Step 2.2: Downsample the human point cloud of each frame to obtain n / 4 uniformly distributed centroids in the sampling layer for subsequent grouping; Step 2.3: Using the grouping layer of the neural network, with the human point cloud set and the centroid point set as input, and with the centroid as the center, select points from the k surrounding human point cloud data for grouping based on the centroid. Step 3 involves feeding local points into a neural network for local feature extraction, specifically including the following steps: Step 3.1: The abstract convolutional layer uses three 2D convolutions and batch normalization operations to extract local features from the input centroid coordinates and grouping points; Step 3.2: Input the extracted local features into the inverse residual block. The inverse residual block uses one grouping operation and three two-dimensional convolution operations. Step 3.3: The inverted bottleneck design, which uses the first layer MLP to increase the number of channels to 4 times the original number and the second layer MLP to restore the number of channels, increases the output channels of the first layer MLP by 4 times. Step 3.4: Use residual connections between the encoder and decoder. The decoder then predicts the joint category to which each radar point in the human body belongs.

2. The method for human point cloud analysis based on millimeter-wave radar according to claim 1, characterized in that: In step 2.3, let the current human point cloud set be {x1, x2, x3, ..., x...} n Using the centroid as the center, select k points from the surrounding human point cloud data and group them accordingly. The specific steps include: Step 2.3.1: Calculate the distance of each point in the human point cloud from the centroid of the point cloud, and select the point in the human point cloud data that is farthest from the centroid and include it in the sampling set; Step 2.3.2: Calculate the Euclidean distance between each point in the human point cloud data and the sampling set point, and select the point x in the human point cloud data with the largest distance. i As the next sampling point; Step 2.3.3: Repeat step 2.3.2 n / 4 times, when the sampling set {x} l …,x m When there is one or more centroids in the data, for point x in the human point cloud data, calculate the distance {d} for each centroid in turn. i ,…,d m }, select the minimum distance d minx Record the data, and finally select the point x from the human point cloud data with the largest distance. i As the next sampling point.

3. The method for human point cloud analysis based on millimeter-wave radar according to claim 1, characterized in that: In step 2.3, the grouping is specifically as follows: using the k-nearest neighbor algorithm, for the centroid x... i Select k points x distances from the centroid in sequence. i The nearest neighboring points are grouped together, and the local features between the neighboring points are learned.

4. The method for human body point cloud analysis based on millimeter-wave radar according to claim 1, characterized in that: In step 4, the local features obtained in step 3 are decoded using a symmetrical decoding layer in the neural network. This specifically includes the following steps: Step 4.1: Use the decoding block to decode local features related to local points; Step 4.2: Update the parameter information of the weights and biases, and output the loss function. Step 4.3: After the network training is complete, save the training model and test the neural network according to the test samples to verify the effectiveness and accuracy of the neural network.

5. The method for human body point cloud analysis based on millimeter-wave radar according to any one of claims 1-4, characterized in that: Step 1 specifically includes the following steps: Step 1.1: Use a depth camera to collect key skeletal information of the human body and use radar to acquire human point cloud information; Step 1.2: The key point information of the human skeleton and the human point cloud information are synchronized using the timestamp at the time of collection; Step 1.3: Fill and sort the human body point cloud information data; Step 1.4: Calculate the distances between key skeletal points corresponding to the point distances in the human body point cloud data; Step 1.5: Select the category of the skeletal key point that is closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold, the human body point cloud information data point is not related to the human body skeletal key points and is classified as an invalid point.

6. The method for human point cloud analysis based on millimeter-wave radar according to claim 5, characterized in that: In step 1.1, the human body point cloud information acquired by the radar includes the x, y, z coordinates in three-dimensional space, velocity, signal-to-noise ratio, and timestamp.

7. The method for human point cloud analysis based on millimeter-wave radar according to claim 5, characterized in that: In step 1.3, filling the human body point cloud information data means that each frame of human body point cloud information data contains 64 key points. For sparse frames, the centroid of the points in the existing human body point cloud information data is calculated and the centroid is used as the filling point so that each frame contains 64 key points.

Images