Human body point cloud analysis method based on millimeter wave radar

Abstract

The invention belongs to the technical field of radar point cloud identification processing, and discloses a millimeter wave radar-based human body point cloud analysis method, which comprises the following steps of: taking radar point cloud captured from a human body as input, identifying a body part corresponding to each point in the point cloud, grouping and analyzing the point cloud, and obtaining a point cloud group; point cloud data collected by a millimeter-wave radar is firstly transmitted into a multi-layer perceptron for feature extraction, then features are learned through an encoding module, then a symmetric decoder structure is adopted, local features are decoded into predicted human body analysis information, and the predicted human body analysis information is analyzed. And after training is finished, the model is stored and is used for analyzing the human body point cloud and performing a subsequent posture estimation task. According to the method, the accuracy of the point cloud analysis task of the model under the sparse point cloud is improved, and the robustness of the system is improved; according to the method, the network structure complexity is reduced, the network training overhead is reduced, the operation efficiency is improved, and the human body analysis precision can be greatly improved under the condition of fewer training rounds.

Description

Technical Field

[0001] The invention belongs to the technical field of radar point cloud recognition and processing, and specifically relates to a human body point cloud analysis method based on millimeter wave radar. Background Art

[0002] Human point cloud parsing originates from point cloud segmentation in point cloud tasks, which divides point clouds within a scene into different parts or categories to identify and distinguish different objects, surfaces or features. In the human point cloud parsing task, the radar point cloud captured from the human body is used as input to identify the body part corresponding to each point in the point cloud. That is, which part of the body each point in the point cloud belongs to. A point cloud with semantic labels will be generated as the final output. In fact, various computer vision tasks have confirmed that adding semantic information to the loop can significantly improve the accuracy of posture estimation, activity recognition, and personnel recognition. This advantage is even more prominent for millimeter wave radars, because the point cloud produced by this low-cost RF sensor is inherently lower in quality than the image produced by visual sensors. Charles R. Qi et al. provided a unified architecture Pointnet (PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation) for applications ranging from object classification, part segmentation to scene semantic parsing, achieving the best performance at the time with only fully connected layers. The PointNet++ proposed later improved the problem that Pointnet simply used maximum pooling to aggregate information and lacked the ability to extract local features. By gradually increasing the scanning scale, local features at different scales are obtained. Similarly, in order to extract local features, Meng-Hao Guo et al. introduced the transformer structure in the encoding layer to obtain more fine-grained local features (PCT: Point cloud transformer). Xu Ma et al. believed that detailed local geometric information may not be the key to point cloud analysis, so they introduced a pure residual MLP network PointMLP (Rethinking Network Design and Local Geometry in PointCloud: A Simple Residual MLP Framework), which does not use complex local feature extractors, but still has good performance and better reasoning speed. Guocheng Qia re-examined the structure of PointNet++ and compared it with subsequent Pointnet versions PointMLP and Point Transformer. The results show that although the accuracy of PointNet++ has been greatly surpassed by subsequent versions, this performance improvement is largely due to more advanced data enhancement and optimization techniques and larger models, and in terms of architectural innovation, subsequent versions do not actually have obvious improvements.Therefore, while using more advanced optimization techniques, the existing technology introduces reverse residual bottleneck design and separable MLP to achieve efficient and effective model expansion, and proposes the PointNeXt model (PointNeXt: Revisiting PointNet++ with Improved Training and ScalingStrategies).

[0003] However, this type of point cloud segmentation task is usually performed on LiDAR, which has a denser point cloud and can therefore segment different objects well. However, there are few related inventions for millimeter-wave radar, which has a sparser point cloud. The sparse point cloud of millimeter-wave radar usually cannot fully reflect the outline of an object, but can only depict part of the object; at the same time, the noise points generated by multipath effects and human body mirror reflections will have a greater impact on the model under sparse point clouds, thereby affecting the robustness of the model. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a human body point cloud parsing method based on millimeter wave radar, which extracts human body point cloud data collected by millimeter wave radar into more advanced features through a multi-layer perceptron; uses a sampling and grouping algorithm to group the features, and further extracts more accurate local feature information so that the model can focus on finer-grained features; finally, it is sent to a neural network for training. After the training is completed, the human body point cloud can be parsed and the parsing accuracy can be improved; the proposed neural network is used to improve the performance of downstream tasks such as posture estimation, personnel recognition and gait recognition.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention is a human body point cloud parsing method based on millimeter wave radar, and the human body point cloud parsing method is implemented by a neural network. Specifically, the human body point cloud parsing method specifically includes the following steps:

[0007] Step 1, radar point cloud data processing and label generation: collect raw data, use the depth camera to collect the human skeleton key point data, use the radar to obtain the human point cloud data, synchronize through the timestamp, and then use the synchronized skeleton key point data;

[0008] Step 2: Use the backbone multi-layer perceptron layer of the neural network to extract global features from the human body point cloud data acquired by the radar, and then perform centroid sampling and grouping through the sampling layer and grouping layer of the neural network to obtain local points;

[0009] Step 3: Send the local points obtained after grouping in step 3 into the neural network, and use the abstract convolution layer to extract local features related to the local points;

[0010] Step 4: Use the symmetrical decoding layer in the neural network to decode the local features obtained in step 3. The global features and the local features of each layer correspond to a decoder operation, and residual connections are performed to prevent overfitting of the model. Finally, it is obtained which part of the human body the predicted bone key point belongs to for each point.

[0011] A further improvement of the present invention is that: the neural network specifically includes an encoder and a decoder, the encoder and the decoder use residual connection, the encoder includes a sampling layer, a grouping layer, an abstract convolution layer and a reverse residual block, the sampling layer includes a sampling operation of the farthest distance sampling to obtain a sampling set as the centroid point of the grouping operation, the grouping layer includes a grouping operation using a k-nearest neighbor algorithm to divide the human point cloud data into several groups with the sampler as the center, the input data of the abstract convolution layer is the coordinate information of the human point cloud data and the local features related to the local points, the local features related to the local points use a two-dimensional convolution and cooperate with batch normalization to obtain a dimension of [B, 64,N,k] local features related to local points, and use a maximum pooling, extract the local features related to local points and the coordinate information of human point cloud data to the reverse residual block, the reverse residual block uses a two-dimensional convolution and batch normalization to obtain a feature vector with a dimension of [B,64,N,k], and then implements a separable multi-layer perceptron by first taking the maximum value operation, simplifying the local features and dimensions related to local points to [B,64,N] to reduce the amount of calculation, and then extracting features through two layers of MLP, where the first layer expands the number of channels to 4 times the original, and the second layer restores the number of channels. Such an inverted bottleneck design can enrich feature extraction. The output data uses a residual connection as the input of the symmetric decoder while flowing to the next layer of encoder.

[0012] A further improvement of the present invention is that the step 2 specifically comprises the following steps:

[0013] Step 2.1, perform global feature extraction on the human body point cloud data acquired by the radar, map the coordinate information of the human body point cloud data acquired by the radar to a dimension higher than the three-dimensional space through the backbone multi-layer perception machine layer, and extract the global feature data reflecting the global information of the human body; for further learning.

[0014] Step 2.2, downsample the human body point cloud of each frame, and obtain n / 4 evenly distributed centroid points in the sampling layer for subsequent grouping;

[0015] Step 2.3: Use the grouping layer of the neural network, take the human body point cloud set and the centroid point set as input, take the centroid as the center, and select the points in the surrounding k human body point cloud data according to the centroid for grouping.

[0016] A further improvement of the present invention is that: in step 2.3, let the current human body point cloud set {x 1 ,x 2 ,x 3 ,…,x n}, taking the centroid as the center, select the points in the surrounding k human point cloud data for grouping according to the centroid, which specifically includes the following steps:

[0017] Step 2.3.1, calculate the distance between each point in the human body point cloud and the centroid of the point cloud, select the point in the human body point cloud data farthest from the centroid, and include it in the sampling set;

[0018] Step 2.3.2, calculate the Euclidean distance between each point in the human point cloud data and the point in the sampling set, and select the point x in the human point cloud data with the largest distance i As the next sampling point;

[0019] Step 2.3.3, repeat step 2.3.2n / 4 times, when the sampling set {x l …,x m}, when there is one or more centroid points in the human body point cloud data, the distance {d i ,…,d m}, select the smallest distance d minx Record and finally select the point x in the human point cloud data with the largest distance i as the next sampling point.

[0020] A further improvement of the present invention is that in step 2.3, the grouping is specifically: using the k nearest neighbor algorithm, for the centroid point x i , select k distance points x from the centroid in turn i The nearest neighboring points are grouped together to learn local features between neighboring points.

[0021] A further improvement of the present invention is that: the step 3 sends the local points to the neural network for local feature extraction, which specifically includes the following steps:

[0022] Step 3.1, the abstract convolution layer uses three-dimensional convolution and batch normalization operations to extract local features of the incoming centroid coordinates and grouping points;

[0023] Step 3.2: The extracted local features are passed into the reverse residual block. The reverse residual block uses one grouping operation and three two-dimensional convolution operations to enrich the feature dimension and alleviate the problem of gradient disappearance. Step 3.3: The first layer of MLP is used to expand the number of channels to 4 times the original number. The second layer of MLP restores the number of channels by an inverted bottleneck design to expand the output channels of the first layer of MLP by 4 times to enrich feature extraction.

[0024] Step 3.4: Use the residual to connect the encoder and decoder, and the decoder finally predicts the joint category to which each radar point of the human body belongs.

[0025] A further improvement of the present invention is that in step 4, a symmetrical decoding layer in a neural network is used to perform a decoding operation on the local features obtained in step 3, which specifically includes the following steps:

[0026] Step 4.1, use the decoding block to decode the local features related to the local points;

[0027] Step 4.2: Update the parameter information of weights and bias parameters, and output the loss function;

[0028] Step 4.3: After network training is completed, save the training model and test the neural network based on the test samples to verify the effectiveness and accuracy of the neural network.

[0029] A further improvement of the present invention is that the step 1 specifically comprises the following steps:

[0030] Step 1.1: Use a depth camera to collect the key points of the human skeleton, and use a radar to obtain the point cloud information of the human body;

[0031] Step 1.2, the human skeleton key point information and the human body point cloud information are synchronized through the timestamp when collected;

[0032] Step 1.3, filling and sorting the human body point cloud information data;

[0033] Step 1.4, calculating the distance between the key points of the skeleton corresponding to the point distance in the human body point cloud information data;

[0034] Step 1.5, select the skeleton key point category closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold, the human body point cloud information data point is irrelevant to the human body's skeleton key point and is classified as an invalid point.

[0035] A further improvement of the present invention is that in step 1.1, the human body point cloud information acquired by the radar includes x, y, z coordinates, speed, signal-to-noise ratio and timestamp in three-dimensional space.

[0036] 10. According to the millimeter wave radar-based human body point cloud analysis method of claim 8, it is characterized in that: in step 1.3, filling the human body point cloud information data in step 1.3 means that: each frame of human body point cloud information data contains 64 key points, and for sparse frames, the center of mass of the points in the existing human body point cloud information data is calculated, and the center of mass is used as the filling point so that each frame contains 64 key points.

[0037] A further improvement of the present invention is that the human body point cloud information data is sorted in step 1.3 as follows: the input human body point cloud information is sorted according to the rules, and sorted in ascending order of x, y, and z coordinates, that is, first sorted in ascending order according to the x coordinates of the points in the human body point cloud information data, and the points in the human body point cloud information data with the same x coordinate are sorted in ascending order of the y coordinates, and so on.

[0038] The beneficial effects of the present invention are:

[0039] The present invention reduces computing overhead and improves computing efficiency through a lightweight network.

[0040] Based on millimeter-wave radar design, the present invention proposes a series of methods such as k-nearest neighbor sampling, separation MLP and residual connection to solve the problem that sparse point clouds are difficult to train and identify.

[0041] This paper successfully applies the point cloud classification task to human body analysis and proves its effectiveness on downstream visual tasks.

[0042] The present invention can significantly improve the accuracy of human body analysis with fewer training rounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the human body point cloud parsing method of the present invention.

[0044] Figure 2 It is a workflow diagram of the skeleton point cloud data and label generation stage of the present invention.

[0045] Figure 3 It is a workflow diagram of the centroid point sampling and grouping stage of the present invention.

[0046] Figure 4 It is a workflow diagram of the point cloud local feature encoding extraction and decoding stage of the present invention.

[0047] Figure 5 It is a workflow diagram of the neural network of the present invention. DETAILED DESCRIPTION

[0048] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0049] like Figure 1-5As shown, the present invention is a human point cloud parsing method based on millimeter wave radar, which is implemented by a neural network. The neural network specifically includes an encoder and a decoder. The encoder and the decoder use residual connections. The encoder includes a sampling layer, a grouping layer, an abstract convolution layer and a reverse residual block. The sampling layer includes a sampling operation of the farthest distance sampling to obtain a sampling set as the centroid point of the grouping operation. The grouping layer includes a grouping operation using a k-nearest neighbor algorithm to divide the human point cloud data into several groups with the sampler as the center. The input data of the abstract convolution layer is the coordinate information of the human point cloud data and the local features related to the local points. The local features related to the local points use a two-dimensional convolution and cooperate with batch The local features related to the local points are normalized to obtain the local features of dimension [B, 64, N, k], and the local features related to the local points are extracted using a maximum pooling. The coordinate information of the human point cloud data flows to the reverse residual block, which uses a two-dimensional convolution and batch normalization to obtain a feature vector of dimension [B, 64, N, k]. Then, a separable multi-layer perceptron is realized by first performing a maximum operation, simplifying the local features and dimensions related to the local points to [B, 64, N] to reduce the amount of calculation, and then extracting features through two layers of MLP, in which the first layer expands the number of channels to 4 times the original number, and the second layer restores the number of channels. Such an inverted bottleneck design can enrich feature extraction. The output data uses a residual connection as the input of the symmetric decoder while flowing to the next layer of encoder.

[0050] Specifically, the human body point cloud parsing method includes the following steps:

[0051] Step 1: Radar point cloud data processing and label generation: Collect raw data, use a depth camera to collect human skeleton key point data, use radar to obtain human point cloud data, synchronize through timestamps, and then use the synchronized skeleton key point data to generate parsed label data for human point cloud. The specific steps include:

[0052] Step 1.1: Use a depth camera to collect the key points of the human skeleton, and use a radar to obtain the human point cloud information, including the x, y, z coordinates, speed, signal-to-noise ratio, and timestamp in three-dimensional space.

[0053] Step 1.2, the human skeleton key point information and the human body point cloud information are synchronized through the timestamp when collected;

[0054] Step 1.3: Fill and sort the human body point cloud information data.

[0055] Among them, filling the human body point cloud information data means: each frame of human body point cloud information data contains 64 key points. For sparse frames, the centroid of the points in the existing human body point cloud information data is calculated, and the centroid is used as the filling point so that each frame contains 64 key points.

[0056] Among them, sorting the human body point cloud information data is specifically as follows: sorting the input human body point cloud information according to the rules, and sorting it in ascending order according to the x, y, and z coordinates, that is, first sorting the points in the human body point cloud information data in ascending order according to the x coordinates, and sorting the points in the human body point cloud information data with the same x coordinate in ascending order according to the y coordinates, and so on.

[0057] Step 1.4, calculating the distance between the key points of the skeleton corresponding to the point distance in the human body point cloud information data;

[0058] Step 1.5, select the skeleton key point category closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold, that is, 30cm, the human body point cloud information data point is irrelevant to the human body's skeleton key point and is classified as an invalid point.

[0059] Step 2: Use the multi-layer perceptron layer of the neural network backbone to extract global features from the human body point cloud data acquired by the radar, and then use the sampling layer and grouping layer of the neural network to sample and group the centroid points to obtain local points for subsequent local feature extraction.

[0060] The specific steps include:

[0061] Step 2.1, perform global feature extraction on the human body point cloud data acquired by the radar, map the coordinate information of the human body point cloud data acquired by the radar to a dimension higher than the three-dimensional space through the backbone multi-layer perception machine layer, and extract the global feature data reflecting the global information of the human body; for further learning.

[0062] Step 2.2, downsample the human body point cloud of each frame, and obtain n / 4, i.e., 16 evenly distributed centroid points in the sampling layer for subsequent grouping;

[0063] Step 2.3: Through the grouping layer of the neural network, the human body point cloud set and the centroid point set are taken as input, and the points in the surrounding k human body point cloud data are selected according to the centroid to group them. In this step, let the current human body point cloud set {x 1 ,x 2 ,x 3 ,…,x n}, taking the centroid as the center, select the points in the surrounding k human point cloud data for grouping according to the centroid, which specifically includes the following steps:

[0064] Step 2.3.1, calculate the distance between each point in the human body point cloud and the centroid of the point cloud, select the point in the human body point cloud data farthest from the centroid, and include it in the sampling set;

[0065] Step 2.3.2, calculate the Euclidean distance between each point in the human point cloud data and the point in the sampling set, and select the point x in the human point cloud data with the largest distance i As the next sampling point;

[0066] Step 2.3.3, repeat step 2.3.2n / 4 times, when the sampling set {x l …,x m}, when there is one or more centroid points in the human body point cloud data, the distance {d i ,…,d m}, select the smallest distance d minx Record and finally select the point x in the human point cloud data with the largest distance i as the next sampling point.

[0067] In step 2.3, the grouping is as follows: using the k nearest neighbor algorithm, for the centroid point x i , select k distance points x from the centroid in turn i The nearest neighboring points are grouped together to learn local features between neighboring points.

[0068] Step 3: Send the local points obtained after grouping in step 2 to the neural network, and use the abstract convolution layer to extract local features related to the local points. These local features pay more attention to the local changes of the human body than the global features, thereby providing more fine-grained features for the model.

[0069] The step 3 sends the local points to the neural network for local feature extraction, which specifically includes the following steps:

[0070] Step 3.1, the abstract convolution layer uses three-dimensional convolution and batch normalization operations to extract local features of the incoming centroid coordinates and grouping points;

[0071] Step 3.2: The extracted local features are passed to the reverse residual block. The reverse residual block uses one grouping operation and three two-dimensional convolution operations to enrich the feature dimension and alleviate the problem of gradient disappearance. A residual connection is added between the input and output to alleviate the gradient disappearance problem, and the first layer of two-dimensional convolution is separated from the remaining convolution layers using the separation multi-layer perceptron technology to reduce the amount of calculation and strengthen point-by-point feature extraction. The reverse residual block separates the multi-layer perceptron into a single layer calculated on the neighborhood features (between the grouping and reduction layers) and two layers of point features (after reduction);

[0072] Step 3.3: Use the first layer of MLP to expand the number of channels to 4 times the original number, and the second layer of MLP restores the number of channels. The inverted bottleneck design expands the output channels of the first layer of MLP by 4 times to enrich feature extraction.

[0073] Step 3.4: Use the residual to connect the encoder and decoder, and the decoder finally predicts the joint category to which each radar point of the human body belongs.

[0074] Step 4: Use the symmetrical decoding layer in the neural network to decode the local features obtained in step 3. The global features and the local features of each layer correspond to a decoder operation, and residual connections are performed to prevent overfitting of the model. Finally, it is obtained which part of the bone key point of the human body each predicted point belongs to. The decoding operation of the local features obtained in step 3 using the symmetrical decoding layer in the neural network specifically includes the following steps:

[0075] Step 4.1, use the decoding block to decode the local features related to the local points;

[0076] Step 4.2: Update the parameter information of weights and bias parameters, and output the loss function;

[0077] Step 4.3: After network training is completed, save the training model and test the neural network based on the test samples to verify the effectiveness and accuracy of the neural network.

[0078] In order to compare the effectiveness of this method, a comparative experiment was designed to compare it with existing methods. Three model methods, pointnet, pointnet++ and pointnext, were selected; the datasets collected by this method and a public human posture point cloud dataset, mars, were used respectively to divide the human body into three categories, namely, torso, left hand and left leg, right hand and right leg, five categories, namely, torso, left hand, left leg, right hand, right leg and full category, that is, each skeletal key point corresponds to a classification label. The comparison results using the mars dataset and the dataset collected by this method are shown in Tables 1 and 2.

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] From the above description, it can be seen that compared with the existing technology, the millimeter-wave based human body radar point cloud parsing algorithm proposed in the present invention accurately distinguishes the human body key point category to which the radar point belongs through the global and local features of the human body radar point cloud, and uses a convolutional neural network for feature extraction and training, which greatly improves the accuracy of human body radar point cloud parsing. At the same time, the model also has an improvement effect on downstream human body tasks, and can significantly enhance the performance of downstream tasks.

[0084] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A human body point cloud analysis method based on millimeter wave radar, characterized in that: The human body point cloud parsing method is implemented by a neural network. Specifically, the human body point cloud parsing method comprises the following steps: Step 1, radar point cloud data processing and label generation: collect raw data, use the depth camera to collect the human skeleton key point data, use the radar to obtain the human point cloud data, synchronize through the timestamp, and then use the synchronized skeleton key point data; Step 2: Use the backbone multi-layer perceptron layer of the neural network to extract global features from the human body point cloud data acquired by the radar, and then perform centroid sampling and grouping through the sampling layer and grouping layer of the neural network to obtain local points; Step 3: Send the local points obtained after grouping in step 2 into the neural network, and use the abstract convolution layer to extract local features related to the local points; Step 4: Use the symmetrical decoding layer in the neural network to decode the local features obtained in step 3. The global features and the local features of each layer correspond to a decoder operation, and residual connections are performed to prevent overfitting. Finally, it is obtained to which part of the human body each predicted point belongs to the corresponding bone key point.

2. The human body point cloud analysis method based on millimeter wave radar according to claim 1, characterized in that: The neural network specifically includes an encoder and a decoder, and the encoder and the decoder use residual connections. The encoder includes a sampling layer, a grouping layer, an abstract convolution layer and a reverse residual block. The sampling layer includes a sampling operation of the farthest distance sampling to obtain a sampling set as the centroid point of the grouping operation. The grouping layer includes a grouping operation using a k-nearest neighbor algorithm to divide the human point cloud data into several groups with the sampler as the center. The input data of the abstract convolution layer is the coordinate information of the human point cloud data and the local features related to the local points. The local features related to the local points use a two-dimensional convolution and cooperate with batch normalization to obtain the local features related to the local points with a dimension of [B, 64, N, k], and use a maximum pooling to obtain The local features related to the local points and the coordinate information of the human body point cloud data flow to the reverse residual block. The reverse residual block uses a two-dimensional convolution and cooperates with batch normalization to obtain a feature vector with a dimension of [B, 64, N, k]. Subsequently, a separable multi-layer perceptron is realized by first performing a maximum operation, and the local features and dimensions related to the local points are simplified to [B, 64, N] to reduce the amount of calculation. Feature extraction is then performed through two layers of MLP, in which the first layer expands the number of channels to 4 times the original number, and the second layer restores the number of channels. The output data uses a residual connection as the input of the symmetric decoder while flowing to the next layer of encoder.

3. The human body point cloud analysis method based on millimeter wave radar according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2.1, extracting global features from the human body point cloud data acquired by the radar, mapping the coordinate information of the human body point cloud data acquired by the radar to a dimension higher than the three-dimensional space through the backbone multi-layer perception machine layer, and extracting global feature data reflecting the global information of the human body; Step 2.2, downsample the human body point cloud of each frame, and obtain n / 4 evenly distributed centroid points in the sampling layer for subsequent grouping; Step 2.3: Use the grouping layer of the neural network, take the human body point cloud set and the centroid point set as input, take the centroid as the center, and select the points in the surrounding k human body point cloud data according to the centroid for grouping.

4. The human body point cloud analysis method based on millimeter wave radar according to claim 1, characterized in that: In step 2.3, let the current human body point cloud set {x1,x2,x3,…,x n }, taking the centroid as the center, select the points in the surrounding k human point cloud data for grouping according to the centroid, which specifically includes the following steps: Step 2.3.1, calculate the distance between each point in the human body point cloud and the centroid of the point cloud, select the point in the human body point cloud data farthest from the centroid, and include it in the sampling set; Step 2.3.2, calculate the Euclidean distance between each point in the human point cloud data and the point in the sampling set, and select the point x in the human point cloud data with the largest distance i As the next sampling point; Step 2.3.3, repeat step 2.3.2n / 4 times, when the sampling set {x l …,x m }, when there is one or more centroid points in the human body point cloud data, the distance {d i ,…,d m }, select the smallest distance d minx Record and finally select the point x in the human point cloud data with the largest distance i as the next sampling point.

5. The human body point cloud analysis method based on millimeter wave radar according to claim 1, characterized in that: In step 2.3, the grouping is as follows: using the k nearest neighbor algorithm, for the centroid point x i , select k distance points x from the centroid in turn i The nearest neighboring points are grouped together to learn local features between neighboring points.

6. The human body point cloud analysis method based on millimeter wave radar according to claim 1, characterized in that: The step 3 sends the local points to the neural network for local feature extraction, which specifically includes the following steps: Step 3.1, the abstract convolution layer uses three-dimensional convolution and batch normalization operations to extract local features of the incoming centroid coordinates and grouping points; Step 3.2: The extracted local features are passed into the reverse residual block, which uses one grouping operation and three two-dimensional convolution operations; Step 3.3, use the first layer MLP to expand the number of channels to 4 times the original number, and the second layer MLP restores the number of channels by an inverted bottleneck design to expand the output channels of the first layer MLP by 4 times; Step 3.4: Use the residual to connect the encoder and decoder, and the decoder finally predicts the joint category to which each radar point of the human body belongs.

7. The method for analyzing human body point cloud based on millimeter wave radar according to claim 1, characterized in that: In step 4, the local features obtained in step 3 are decoded using the symmetrical decoding layer in the neural network, which specifically includes the following steps: Step 4.1, use the decoding block to decode the local features related to the local points; Step 4.2: Update the parameter information of weights and bias parameters, and output the loss function; Step 4.3: After network training is completed, save the training model and test the neural network based on the test samples to verify the effectiveness and accuracy of the neural network.

8. The method for analyzing human body point cloud based on millimeter wave radar according to any one of claims 1 to 7, characterized in that: The step 1 specifically includes the following steps: Step 1.1: Use a depth camera to collect the key points of the human skeleton, and use a radar to obtain the point cloud information of the human body; Step 1.2, the human skeleton key point information and the human body point cloud information are synchronized through the timestamp when collected; Step 1.3, filling and sorting the human body point cloud information data; Step 1.4, calculating the distance between the key points of the skeleton corresponding to the point distance in the human body point cloud information data; Step 1.5, select the skeleton key point category closest to the point in the human body point cloud information data as the category label of the human body point cloud information data point. If the closest distance is greater than the set threshold, the human body point cloud information data point is irrelevant to the human body's skeleton key point and is classified as an invalid point.

9. The method for analyzing human body point cloud based on millimeter wave radar according to claim 8, characterized in that: In step 1.1, the human body point cloud information acquired by the radar includes the x, y, z coordinates, speed, signal-to-noise ratio and timestamp in three-dimensional space.

10. The method for analyzing human body point cloud based on millimeter wave radar according to claim 8, characterized in that: In step 1.3, filling the human body point cloud information data in step 1.3 means: each frame of human body point cloud information data contains 64 key points. For sparse frames, the center of mass of the points in the existing human body point cloud information data is calculated, and the center of mass is used as the filling point so that each frame contains 64 key points.

Images