A Human Posture and Behavior Recognition Method Based on 4D Millimeter-Wave Radar

Abstract

The present invention relates to the field of posture recognition and provides a method for human posture behavior recognition based on 4D millimeter-wave radar, the method comprising: obtaining point cloud data to be recognized through a 4D millimeter-wave radar, the point cloud data to be recognized including human posture information; zero-padding the point cloud data to be recognized to obtain a human posture vector; generating a fully connected graph based on the human posture vector; using a graph convolution model with a multi-layer perceptron as the core to iteratively update the vertex features of the fully connected graph several times; fusing the updated vertex features to obtain a human posture recognition result. In order to address the risk of privacy leakage when using a camera for human posture recognition in related technologies, the solution of the present application can obtain target point cloud data, and multiple data processing of the point cloud data can obtain more accurate human posture recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of posture recognition, and in particular to a method for human posture behavior recognition based on 4D millimeter wave radar. Background Art

[0002] Rail transit is one of the indispensable public infrastructures in cities. Currently, rail transit carriages are generally equipped with environmental perception sensors to monitor passenger behavior. For example, if an elderly person suddenly falls in the train toilet, the environmental perception sensor can automatically alarm, thus shortening the rescue time.

[0003] Currently, the most commonly used environmental perception sensor in related technologies is the camera, which can obtain high-resolution environmental information and is conducive to the classification of human posture and movement. However, the camera has the risk of privacy leakage, and the image information obtained by the camera only provides semantic features of the 2D plane, which makes it difficult to obtain accurate depth information of the target. In addition, in insufficient lighting, false detection and missed detection are inevitable. Summary of the Invention

[0004] The present invention provides a method for human posture behavior recognition based on 4D millimeter-wave radar, which is used to solve the risk of privacy leakage when using cameras for human posture recognition in related technologies. In the solution of this application, point cloud data of the target can be obtained, and multiple data processing of the point cloud data can obtain more accurate human posture recognition results.

[0005] The present invention provides a human body posture behavior recognition method based on 4D millimeter wave radar, comprising:

[0006] Acquire point cloud data to be identified through 4D millimeter-wave radar;

[0007] Obtaining a human body posture vector based on zero filling of the point cloud data to be identified;

[0008] generating a fully connected graph based on the human posture vector;

[0009] Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model;

[0010] Fusing the updated vertex features, edge features, and global features to obtain a human posture recognition result;

[0011] The step of obtaining the point cloud data to be identified by the 4D millimeter wave radar includes:

[0012] Acquire original point cloud data, and perform filtering processing on the original point cloud data;

[0013] Clustering the filtered original point cloud data to determine a recognition range of human posture;

[0014] Eliminating the original point cloud data outside the recognition range to obtain the point cloud data to be recognized;

[0015] Clustering the filtered original point cloud data to determine the recognition range of human postures includes:

[0016] Generating a corresponding three-dimensional bounding box for each frame of the original point cloud data;

[0017] For any one of the three-dimensional bounding boxes, if the left boundary and the right boundary of the three-dimensional bounding box are within a preset recognition range, the recognition range is updated, and the coordinate value of the left boundary of the updated recognition range is the maximum value between the left boundary of the three-dimensional bounding box and the left boundary of the recognition range before the update, and the coordinate value of the right boundary of the updated recognition range is the minimum value between the right boundary of the three-dimensional bounding box and the right boundary of the recognition range before the update.

[0018] According to a human posture behavior recognition method based on 4D millimeter wave radar provided by the present invention, generating a fully connected graph based on the human posture vector includes:

[0019] For each frame of point cloud in the human body posture vector, construct an adjacency matrix;

[0020] Based on all the adjacency matrices, a fully connected graph of the human posture vector is generated.

[0021] According to a human posture behavior recognition method based on 4D millimeter wave radar provided by the present invention, generating a fully connected graph based on the human posture vector further includes:

[0022] Updating edge features of the fully connected graph based on vertex features of the fully connected graph;

[0023] The vertex features of the fully connected graph are updated based on the updated edge features.

[0024] According to a human posture behavior recognition method based on 4D millimeter-wave radar provided by the present invention, the fully connected graph is input into a graph convolution model with a multi-layer perceptron as the core, and updated vertex features, edge features, and global features output by the graph convolution model are obtained, including:

[0025] During each iterative update, the weight value of each vertex feature of the fully connected graph is calculated through the self-attention mechanism, and the vertex features are weighted based on the weight values of the vertex features.

[0026] According to a human posture behavior recognition method based on 4D millimeter wave radar provided by the present invention, clustering the raw point cloud data after filtering to determine the recognition range of human posture, and then further comprising:

[0027] If the number of point cloud data included in the neighborhood of the target point cloud data exceeds a preset density threshold, the target point cloud data is determined to be a core point, and the target point cloud data is any point cloud data in the original point cloud data;

[0028] If the number of point cloud data included in the neighborhood of the target point cloud data does not exceed a preset density threshold, but the neighborhood of the target point cloud data includes at least one core point, the target point cloud data is determined to be a boundary point;

[0029] If the target point cloud data is neither a core point nor a boundary point, determining that the target point cloud data is a noise point;

[0030] Noise points in the original point cloud data are removed.

[0031] According to a human posture behavior recognition method based on 4D millimeter wave radar provided by the present invention, the filtering processing of the original point cloud data includes:

[0032] Eliminate noise points in the original point cloud data by statistical filtering;

[0033] Point cloud data above a specific height in the original point cloud data is removed through straight-through filtering.

[0034] The present invention provides a method for human posture behavior recognition based on 4D millimeter-wave radar, which uses point cloud data for posture recognition. Compared with the camera used to obtain image data in related technologies, it can protect user privacy. Point cloud data is three-dimensional data, which contains a larger amount of data and is more comprehensive. In addition, graph convolution can be used in this solution to iteratively update the fully connected graph constructed from point cloud data multiple times, which can further eliminate irrelevant data and improve the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 4D millimeter-wave radar-based human body posture behavior recognition method provided by an embodiment of the present invention;

[0037] Figure 2 4D millimeter-wave radar-based human body posture behavior recognition system provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0040] Figure 1 This is a flow chart of a method for human posture behavior recognition based on 4D millimeter-wave radar provided in an embodiment of the present invention.

[0041] like Figure 1 As shown, this embodiment provides a human posture behavior recognition method based on 4D millimeter wave radar, including:

[0042] Step 101: Acquire point cloud data to be identified through a 4D millimeter-wave radar;

[0043] In this step, point cloud data can be obtained through 4D millimeter-wave radar. Compared with 3D millimeter-wave radar, the 4D millimeter-wave radar used in this step can also focus on the human body height corresponding to the point cloud data. Human body height data is also crucial for human posture recognition. For example, when a person suddenly falls, the height of the person will definitely have a sudden change. Therefore, the point cloud data obtained by using the 4D millimeter-wave radar in this embodiment can contain richer information, which also provides support for accurate human posture recognition.

[0044] The point cloud data obtained in this embodiment may include human posture information, and human posture may include limb movements and hand gestures.

[0045] Step 102, obtaining a human body posture vector based on zero padding of the point cloud data to be identified;

[0046] In practical applications, when using 4D millimeter-wave radar to collect human posture data, the point cloud is relatively sparse, resulting in a relatively small number of points in a frame of point cloud. In this step, the collected point cloud data can be padded to enhance the key point information of the human body collected by the millimeter wave. Specifically, 42 points can be selected in each frame of the point cloud for subsequent steps to construct a fully connected graph. For point clouds with less than 42 points, the number of points in each frame of action can be increased to 42 through a zero-padding strategy to obtain a vector representing the human posture. The zero-padding strategy refers to inserting zero-value elements at the edges of the data to keep the size or shape of the data unchanged.

[0047] After completing the zero-filling, you can create an LMDB database to facilitate data storage, reading, and obtaining the number of samples.

[0048] Step 103, generating a fully connected graph based on the human body posture vector;

[0049] A fully connected graph means that all pairs of nodes in the graph are connected by corresponding edges. Usually, these edges are also assigned corresponding weights. The edges between nodes with high similarity generally have large weights, while the edges between nodes with low similarity generally have small weights.

[0050] Step 104: input the fully connected graph into a graph convolution model with a multi-layer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model;

[0051] The Multilayer Perceptron (MLP) model is a feedforward neural network model. In practical applications, the MLP model can be used for multiple iterations to fully propagate forward and backward all training data. Residual connections are performed in the point cloud, and point cloud training and reasoning are performed separately. It can be fully trained, achieve rapid inference, and improve the accuracy of human pose classification. It effectively utilizes the relative positional relationship between points and the attention mechanism, and through multiple iterations and enhancements, it predicts human pose as accurately as possible, improving the recognition accuracy of human pose. In each round of training, the model calculates the prediction results based on the input data, then compares them with the actual labels, calculates the loss value, and adjusts the model parameters based on the loss value to make the prediction more accurate. By minimizing the cross-entropy loss, the model can learn to predict the probability of each category as accurately as possible, thereby gradually improving the accuracy of human pose recognition. At the same time, during the model training process, the optimizer can also calculate the mean and variance of the gradient of each parameter to adjust the learning rate of each parameter, effectively handling sparse gradients and different parameter scales.

[0052] In practical applications, the graph convolutional network model provided by the solution of this application conforms to the following formula:

[0053]

[0054] in, is a hard activation function, is the self-attention mechanism, 、 and are query, key, and value matrices, is global information, Represents the importance of the edge, For side information, S Represents the aggregated output information, W and b denote the weight and bias matrices respectively, represents the sigmoid activation function, Represents the proportional weight of new and old information.

[0055] Step 105 , fusing the updated vertex features, edge features, and global features to obtain a human posture recognition result.

[0056] In practical applications, in step 104, after the last iteration, the state value of the updated vertex feature can be input into an MLP model to expand the vertex feature and map the state vector of each point to a higher dimension in order to retain more spatial and semantic information and improve the expression ability of the point. The maximum pooling function is used to aggregate these high-dimensional features and input them into the Bi-LSTM. The bidirectional LSTM can take into account the information of the previous and next time series at the same time, which is very effective for tasks that need to capture time series, such as human motion and posture recognition. The Max function is then used to aggregate the information of all points.

[0057] For each frame of point cloud data, the extended vector can be fused in the time dimension to obtain the human posture recognition result. In the human activity recognition task, the input is Where n=60 is the size of the sliding window, and the extended vector is obtained after the graph convolution network. , a bidirectional LSTM with a size of 16 hidden units is used to process and directly output the confidence of different categories of human activities.

[0058] In the human posture recognition method provided in this embodiment, point cloud data is used for posture recognition. Compared with the image data commonly used in related technologies, user privacy can be protected. Point cloud data is three-dimensional data, which contains a larger amount of data and is more comprehensive. In addition, in this solution, a graph convolution model can be applied to perform multiple iterative updates on the fully connected graph constructed from the point cloud data, which can further eliminate irrelevant data and improve the accuracy of detection.

[0059] The step of obtaining the point cloud data to be identified by the 4D millimeter wave radar includes:

[0060] Acquire original point cloud data, and perform filtering processing on the original point cloud data;

[0061] Clustering the filtered original point cloud data to determine a recognition range of human posture;

[0062] The original point cloud data outside the recognition range is eliminated to obtain the point cloud data to be recognized.

[0063] Raw point cloud data refers to data directly acquired using 4D millimeter-wave radar. Since the raw data may contain a lot of chaotic noise data, the first step after acquiring the raw point cloud data is to perform filtering to remove data in the raw point cloud data that is not related to human posture recognition.

[0064] Furthermore, since the area for collecting human postures is relatively concentrated, in implementation, the recognition range of the human point cloud can be defined in a fixed area where the human body performs movements. However, in order to avoid the problem of imperfect monitoring due to setting a rigid detection area, a method of dynamically determining the recognition range is adopted in this embodiment, that is, the point cloud data is clustered by a clustering algorithm, and then the recognition range is determined. In practical applications, the DBSCAN clustering algorithm can be used.

[0065] After the recognition range is determined by the clustering algorithm, the point cloud data outside the recognition range is considered noise data. Therefore, the point cloud data outside the recognition range can also be eliminated. In this way, only the data related to human posture recognition is retained, which can make human posture recognition more accurate. Moreover, due to the small amount of data, the data processing volume of the calculation process can be significantly reduced.

[0066] Clustering the filtered original point cloud data to determine the recognition range of human postures includes:

[0067] Generating a corresponding three-dimensional bounding box for each frame of the original point cloud data;

[0068] For any one of the three-dimensional bounding boxes, if the left boundary and the right boundary of the three-dimensional bounding box are within a preset recognition range, the recognition range is updated, and the coordinate value of the left boundary of the updated recognition range is the maximum value between the left boundary of the three-dimensional bounding box and the left boundary of the recognition range before the update, and the coordinate value of the right boundary of the updated recognition range is the minimum value between the right boundary of the three-dimensional bounding box and the right boundary of the recognition range before the update.

[0069] The three-dimensional bounding box may also be referred to as a 3D box. In this embodiment, the 3D box may be a bounding box around a human body formed by point cloud data. The range of the 3D box is also the range of the human body to be identified.

[0070] The following is a specific example of the method for determining the recognition range of a human body posture provided in this application, which includes the following steps:

[0071] S1. First set a minimum detection range on both sides. , During initialization, the left and right segmentation boundaries of the point cloud are initialized to the corresponding minimum detection ranges on both sides, as shown in the following formula:

[0072]

[0073]

[0074] S2. Use the DBSCN clustering algorithm to cluster the point cloud of the current frame and obtain the current action representation based on the previous and next frames.

[0075] S3, traverse the 3D box corresponding to each cluster of the current frame in turn, and determine whether the currently traversed 3D box is in Inside.

[0076] S4. If the 3Dbox frame currently being viewed is If the point cloud range is within the range, the point cloud range is modified according to the Y-axis range of the current 3Dbox frame, so that the point information is changed accordingly, as shown in the following formula:

[0077]

[0078]

[0079] Among them is the left border of the 3Dbox box, The right border of the 3Dbox.

[0080] S5. If the 3Dbox currently being viewed is not If it is within, continue the traversal in step S3.

[0081] In practical applications, radar signals can also be emitted to the target. Static targets and moving targets will produce different echo signals for the radar signal. Therefore, static targets and moving targets can be distinguished based on the received echo signals, and the moving targets can be further determined as the identification range.

[0082] In an exemplary embodiment, generating a fully connected graph based on the human body posture vector includes:

[0083] For each frame of point cloud in the human body posture vector, construct an adjacency matrix;

[0084] Based on all the adjacency matrices, a fully connected graph of the human posture vector is generated.

[0085] For each point cloud data , is the coordinate data, This is state data, including (x, y, z, range, velocity, doppler_bin, bearing, intensity). x, y, and z represent the position of each point in the Cartesian coordinate system. Velocity represents the relative velocity of the target, indicating the rate of motion between the target and the radar. Millimeter-wave radar calculates target velocity through the Doppler effect, which is the frequency shift of the radar signal. The doppler_bin Doppler shift is the difference between the radar echo signal frequency and the transmitted signal frequency, reflecting the target's relative velocity to the radar. doppler_bin typically represents the index value assigned to different frequency bands after discretizing the Doppler shift. It represents the target's velocity component along the radar's line of sight in the radar coordinate system. bearing represents the azimuth, and intensity represents the signal strength.

[0086] Set the adjacency matrix for each point cloud data P, initialize it to 1, and then initialize a fully connected graph , ,when , the adjacency matrix .

[0087] In an exemplary embodiment, the generating of a fully connected graph based on the human body posture vector further includes:

[0088] Updating edge features of the fully connected graph based on vertex features of the fully connected graph;

[0089] The vertex features of the fully connected graph are updated based on the updated edge features.

[0090] Wherein, the vertex features of the fully connected graph are updated based on the updated edge features, satisfying the following formula:

[0091]

[0092] in, and are the edge features and vertex features of (t+1) iterations respectively, Calculate the edge features of the vertices, Is an aggregation function used to perform Max or Mean operations on the edge features of each vertex. Use the aggregated edge features to update vertex features.

[0093] During implementation, you can also use Extract edge features to ensure translation invariance of relative coordinates. The relative coordinate displacement generated according to the vertex state value is an automatic registration mechanism, and we iterate with the center vertex.

[0094] In an exemplary embodiment, inputting the fully connected graph into a graph convolution model with a multilayer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model includes:

[0095] During each iterative update, the weight value of each vertex feature of the fully connected graph is calculated through the self-attention mechanism, and the vertex features are weighted based on the weight values of the vertex features.

[0096] In this process, the new information obtained by aggregating neighbor edge features is combined with the current node state to dynamically adjust the degree of node state update, ensuring that the network can flexibly integrate information of varying degrees from neighbors and combine it with the current state through a weighted sum. The introduction of residual connections ensures that the model maintains information continuity during updates, preventing complete loss of historical information.

[0097] The self-attention mechanism calculates a weight for each point, enhancing informative points and suppressing unimportant ones. In point cloud data, the spatial relationship between points has a significant impact on the final classification, so the introduction of the self-attention mechanism helps capture this local-global relationship.

[0098] In an exemplary embodiment, clustering the filtered raw point cloud data to determine a recognition range of a human posture may further include:

[0099] If the number of point cloud data included in the neighborhood of the target point cloud data exceeds a preset density threshold, the target point cloud data is determined to be a core point, and the target point cloud data is any point cloud data in the original point cloud data;

[0100] If the number of point cloud data included in the neighborhood of the target point cloud data does not exceed a preset density threshold, but the neighborhood of the target point cloud data includes at least one core point, the target point cloud data is determined to be a boundary point;

[0101] If the target point cloud data is neither a core point nor a boundary point, determining that the target point cloud data is a noise point;

[0102] Noise points in the original point cloud data are removed.

[0103] In practice, we can start from any unvisited point. If it is a core point, we expand it with it as the center until all points that can be connected by density are marked as the same cluster. If a point is a boundary point, it is added to the cluster of the nearest core point. If a point is neither a core point nor a boundary point, it is marked as a noise point. The above process is repeated until all points in a frame of point cloud have been visited. Noise points are removed to prevent them from affecting the subsequent point cloud convolution results.

[0104] In an exemplary embodiment, filtering the raw point cloud data includes:

[0105] Eliminate noise points in the original point cloud data by statistical filtering;

[0106] Point cloud data above a specific height in the original point cloud data is removed through straight-through filtering.

[0107] In implementation, straight-through filtering can be used to filter point clouds that are 1.5 times higher than the human body height. In practical applications, the human body height can be averaged based on experience, such as 1.6 meters.

[0108] The human body posture behavior recognition system based on 4D millimeter wave radar provided by the present invention is described below. The human body posture behavior recognition system based on 4D millimeter wave radar described below and the human body posture behavior recognition method based on 4D millimeter wave radar described above can be referenced to each other.

[0109] Figure 2 4D millimeter-wave radar-based human body posture behavior recognition system provided in an embodiment of the present invention.

[0110] like Figure 2 As shown, the human posture behavior recognition system based on 4D millimeter wave radar provided in this embodiment includes:

[0111] The data acquisition module 201 is used to acquire target point cloud data through 4D millimeter wave radar;

[0112] A data filling module 202 is configured to obtain a human body posture vector based on zero filling of the point cloud data to be identified;

[0113] A graph construction module 203, configured to generate a fully connected graph based on the human posture vector;

[0114] A graph updating module 204 is configured to input the fully connected graph into a graph convolutional model with a multi-layer perceptron as its core, and obtain updated vertex features, edge features, and global features output by the graph convolutional model;

[0115] The posture recognition module 205 is used to fuse the updated vertex features, edge features and global features to obtain a human posture recognition result.

[0116] The specific implementation method of the human posture behavior recognition system based on 4D millimeter wave radar provided in this embodiment can be implemented with reference to the above embodiment and will not be repeated here.

[0117] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the human body posture behavior recognition method based on 4D millimeter wave radar, which includes:

[0118] Acquire point cloud data to be identified through 4D millimeter-wave radar;

[0119] Obtaining a human body posture vector based on zero filling of the point cloud data to be identified;

[0120] generating a fully connected graph based on the human posture vector;

[0121] Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model;

[0122] The updated vertex features, edge features and global features are fused to obtain a human posture recognition result.

[0123] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the human posture behavior recognition method based on 4D millimeter wave radar provided by the above methods, which includes:

[0125] Acquire point cloud data to be identified through 4D millimeter-wave radar;

[0126] Obtaining a human body posture vector based on zero filling of the point cloud data to be identified;

[0127] generating a fully connected graph based on the human posture vector;

[0128] Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model;

[0129] The updated vertex features, edge features and global features are fused to obtain a human posture recognition result.

[0130] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for human posture behavior recognition based on 4D millimeter-wave radar provided by the above methods is implemented, and the method includes:

[0131] Acquire point cloud data to be identified through 4D millimeter-wave radar;

[0132] Obtaining a human body posture vector based on zero filling of the point cloud data to be identified;

[0133] generating a fully connected graph based on the human posture vector;

[0134] Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron as the core to obtain updated vertex features, edge features, and global features output by the graph convolution model;

[0135] The updated vertex features, edge features and global features are fused to obtain a human posture recognition result.

[0136] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0137] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods of each embodiment or certain portions of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A human body posture behavior recognition method based on 4D millimeter wave radar, characterized in that: include: Acquire point cloud data to be identified through 4D millimeter-wave radar; Obtaining a human body posture vector based on zero filling of the point cloud data to be identified; generating a fully connected graph based on the human posture vector; Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron (MLP) as its core to obtain updated vertex features, edge features, and global features output by the graph convolution model; Fusing the updated vertex features, edge features, and global features to obtain a human posture recognition result; The graph convolution model conforms to the following formula: ； in, is a hard activation function, is the self-attention mechanism, Q 、 K and V are query, key, and value matrices, is global information, Represents the importance of the edge, For side information, S Represents the aggregated output information, W and b denote the weight and bias matrices respectively, represents the sigmoid activation function, Represents the proportional weight of new and old information; The step of obtaining the point cloud data to be identified by the 4D millimeter wave radar includes: Acquire original point cloud data, and perform filtering processing on the original point cloud data; Clustering the filtered original point cloud data to determine a recognition range of human posture; Eliminating the original point cloud data outside the recognition range to obtain the point cloud data to be recognized; Clustering the filtered original point cloud data to determine the recognition range of human postures includes: S1. First set a minimum detection range on both sides. , During initialization, the left and right segmentation boundaries of the point cloud are initialized to the corresponding minimum detection ranges on both sides, as shown in the following formula: ； ； S2. Use the DBSCN clustering algorithm to cluster the point cloud of the current frame and obtain the current action representation based on the previous and next frames; S3, traverse the 3D box corresponding to each cluster of the current frame in turn, and determine whether the currently traversed 3D box is in Inside; S4. If the 3Dbox frame currently being viewed is If the point cloud range is within the range, the point cloud range is modified according to the Y-axis range of the current 3Dbox frame, so that the point information is changed, as shown in the following formula: ； ； Among them is the left border of the 3Dbox box, The right border of the 3Dbox; S5. If the 3Dbox currently being viewed is not If it is within, continue the traversal in step S3.

2. The method for human posture behavior recognition based on 4D millimeter wave radar according to claim 1, characterized in that: Generating a fully connected graph based on the human body posture vector includes: For each frame of point cloud in the human body posture vector, construct an adjacency matrix; Based on all the adjacency matrices, a fully connected graph of the human posture vector is generated.

3. The human body posture behavior recognition method based on 4D millimeter wave radar according to claim 1 is characterized in that: The method further comprises generating a fully connected graph based on the human body posture vector, and then further comprising: Updating edge features of the fully connected graph based on vertex features of the fully connected graph; The vertex features of the fully connected graph are updated based on the updated edge features.

4. The method for human posture behavior recognition based on 4D millimeter wave radar according to claim 1, characterized in that: Inputting the fully connected graph into a graph convolution model with a multi-layer perceptron (MLP) as its core to obtain updated vertex features, edge features, and global features output by the graph convolution model includes: During each iterative update, the weight value of each vertex feature of the fully connected graph is calculated through the self-attention mechanism, and the vertex features are weighted based on the weight values of the vertex features.

5. The method for human posture behavior recognition based on 4D millimeter wave radar according to claim 1, characterized in that: The original point cloud data after filtering is clustered to determine the recognition range of human posture, and then further includes: If the number of point cloud data included in the neighborhood of the target point cloud data exceeds a preset density threshold, the target point cloud data is determined to be a core point, and the target point cloud data is any point cloud data in the original point cloud data; If the number of point cloud data included in the neighborhood of the target point cloud data does not exceed a preset density threshold, but the neighborhood of the target point cloud data includes at least one core point, the target point cloud data is determined to be a boundary point; If the target point cloud data is neither a core point nor a boundary point, determining that the target point cloud data is a noise point; Noise points in the original point cloud data are removed.

6. The method for human posture behavior recognition based on 4D millimeter wave radar according to claim 1, characterized in that: The filtering process on the original point cloud data includes: Eliminate noise points in the original point cloud data by statistical filtering; Point cloud data above a specific height in the original point cloud data is removed through straight-through filtering.

Images