Human body tumble identification method based on millimeter wave radar perception

Abstract

The invention discloses a human body tumble identification method based on millimeter wave radar perception. The method comprises the following steps: acquiring echo data of various human body actions by using a millimeter wave radar; processing the echo data to respectively obtain point cloud data of human body actions; a DBSCAN clustering method is used for clustering point cloud data, point clouds are divided into different clusters based on a central point, then the obtained point cloud data are input into a deep neural network, space structure and dependency relationship information of the point cloud data is captured, the category of human body actions is recognized according to the space structure and dependency relationship information, and a human body tumble recognition result is output. According to the invention, a deep neural network model fusing a PointNet network and a Transform module is designed, and the model can enhance the description of the spatial relationship of point clouds, thereby improving the judgment of human body tumble recognition, and reducing the false alarm and false alarm rate. According to the invention, human body postures and actions are sensed through millimeter wave radar signals, human body tumble recognition is realized, and the system has the advantages of good anti-interference performance, high recognition accuracy and the like.

Description

Technical Field

[0001] The present invention belongs to the field of human fall recognition, and in particular relates to a human fall recognition method based on millimeter wave radar perception. Background Art

[0002] In recent years, with the aging of the population in my country, the demand for industries related to elderly care and health has been increasing. Falling is one of the main risks to the health of the elderly. Traditional fall recognition methods rely on visual technology, such as capturing the user's body movements and postures through cameras, and analyzing and identifying them. However, this method is greatly affected by factors such as lighting conditions, background interference, and occlusion in actual applications, and the recognition effect is not stable enough. In addition, the use of visual cameras also poses a risk of privacy leakage.

[0003] Millimeter-wave radar can work in the absence of light. It is small in size, low in power consumption, and easy to integrate into the system. It has certain advantages in monitoring the fall of the elderly. However, the existing human fall recognition method based on millimeter-wave radar has a high error rate, especially when there are multiple people present, it is easy to be interfered and cannot accurately detect the fall phenomenon. Summary of the invention

[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, a method for identifying human falls based on millimeter-wave radar sensing is provided. The method senses human posture and movement through millimeter-wave radar signals to realize the identification of human falls. The method has many advantages such as good anti-interference performance and high recognition accuracy.

[0005] Technical solution: To achieve the above purpose, the present invention provides a method for human fall recognition based on millimeter wave radar perception, comprising the following steps:

[0006] S1: Use millimeter wave radar to collect echo data of various human body movements;

[0007] S2: Process the echo data to obtain point cloud data of human body movements;

[0008] S3: Cluster the point cloud data obtained in step S2 using the DBSCAN clustering method, divide the point cloud into different clusters based on the center point, and then input the obtained point cloud data into the constructed and trained deep neural network;

[0009] S4: The deep neural network includes PointNet network and Transformer module, which can better capture the deep spatial structure relationship between point cloud data.

[0010] Furthermore, in step S1, an FMCW millimeter wave radar is used to collect echo data of various human body movements, and the FMCW millimeter wave radar includes a transmitting antenna (RX), a receiving antenna (TX), a radio frequency signal module, a mixer and an analog-to-digital converter (ADC);

[0011] The FMCW millimeter-wave radar collects echo data of various human movements in the following way: the RF signal module generates a continuous frequency modulation signal, which is transmitted through the transmitting antenna after frequency multiplication. The reflection of the transmitted signal after encountering an obstruction is captured by the receiving antenna. The receiving signal is mixed with the transmitting signal to obtain an intermediate frequency signal. Finally, the intermediate frequency signal is converted into digital form and the subsequent signal is processed.

[0012] The frequency modulated continuous waveform FMCW of the millimeter wave radar transmission signal is expressed as:

[0013]

[0014] Among them, A T is the amplitude of the transmitted signal; f c is the starting frequency of the carrier signal; B is the bandwidth of the frequency modulated signal, S is the slope of the frequency modulated continuous wave signal; T c is the period of the frequency modulated signal;

[0015] After being reflected by human body movements, the signal received by the radar is expressed as follows:

[0016]

[0017] Among them, A R is the amplitude of the received echo signal; τ is the delay from the transmission to the reception of the FM signal; Δf d is the Doppler frequency shift value;

[0018] The intermediate frequency signal is obtained by mixing the received signal with the transmitted signal. The intermediate frequency signal is expressed as:

[0019]

[0020] Furthermore, in step S2, the echo data is generated into a point cloud by using a signal processing technique, specifically:

[0021] A1: Use one-dimensional fast Fourier transform on each frame of echo data in the fast time dimension;

[0022] A2: Calculate the mean of the results obtained in step A1 in each distance unit in the slow time dimension;

[0023] A3: Using the result of step A1, subtract the mean value corresponding to the distance unit calculated in step A2 from each distance unit;

[0024] A4: Perform one-dimensional fast Fourier transform on the result of step A3 in the slow time dimension to obtain the distance-speed information of the moving target;

[0025] A5: The system performs constant false alarm rate (CFAR) detection in distance and horizontal direction to distinguish background noise from actual target signals to ensure the accuracy of target detection;

[0026] A6: After completing the horizontal processing, the data undergoes vertical beamforming and vertical angle estimation to obtain the target's position information on the vertical plane;

[0027] A7: The above operations generate the point cloud data of the target, which contains information on distance, angle, radial velocity, and amplitude, expressed as follows:

[0028] It is obtained by Fourier transforming a single frequency sweep time, that is, performing distance-FFT along each row; through the principle of distance measurement, searching according to different separated peak spectra, the frequency point information of the target and the distance information between the target and the radar are obtained;

[0029]

[0030] Where d represents the distance between the target object and the radar sensor, c represents the speed of light, S is the slope of the FM continuous wave signal, τ is the delay from the transmission to the reception of the FM signal, and f IF is the frequency of the IF signal;

[0031] The target object speed is v, l is the distance between the receiving antennas, the wavelength is λ, T c is the frame period, and the intermediate frequency signal phase difference Δφ is generated according to the two Chirp signals. The target speed and angle are calculated respectively.

[0032]

[0033]

[0034] Furthermore, in step S3, the point cloud data is clustered using the DBSCAN clustering method, and two parameters (neighborhood radius and minimum number of points) of the DBSCAN clustering algorithm are set to adapt to the data characteristics and clustering requirements of the millimeter wave radar. The pre-processed three-dimensional point cloud data is clustered using the DBSCAN algorithm, and several center points of the point cloud data are found through the above parameters and steps. The point cloud is divided into different clusters based on the center points, and the obtained point cloud data is then input into the constructed and trained deep neural network.

[0035] Furthermore, the neural network in step S4 is composed of a PointNet network and a Transformer module. The PointNet network is composed of several repeated modules and a maximum pooling layer. Each module contains a T-Net and a multi-layer perceptron. The Transformer module is connected after the PointNet network and contains multiple modules such as point input embedding, position encoding, multi-head attention and feedforward network.

[0036] The operation mode of the PointNet network is as follows: PointNet receives a point cloud data frame as input, maintains the invariance of spatial transformation through T-Net matrix transformation, then uses a multi-layer perceptron (MLP) to extract features from the point cloud data, and finally extracts the overall features of the three-dimensional point cloud through the maximum pooling layer.

[0037] The input of the PointNet network is a set of point cloud coordinate data, represented as an N×3 two-dimensional tensor, where N represents the number of points per frame and 3 represents the three-dimensional coordinates (x, y, z) of each point.

[0038] The T-Net module is a transformation layer used to learn the rotation matrix and align different data. The T-Net module maps the three-dimensional features of the input point cloud data to a 1024-dimensional space through the MLP (Multi-layer Perceptron) module, extracts global features through the maximum pooling layer, and then uses the MLP module for downsampling. The 9 features obtained by downsampling are recombined to obtain a 3×3 rotation matrix. The T-Net module not only solves the problem of rotation invariance of three-dimensional point clouds in space, but also improves the robustness and performance of the network.

[0039] The U-max pooling layer utilizes the invariance of symmetric functions, can process the disordered characteristics of point cloud data, and effectively captures global information by extracting the maximum value.

[0040] The operation mode of the Transformer module in step S4 is as follows: the global features or the features of each point extracted by the PointNet network are serialized through the input embedding and position encoding modules, the self-attention of the point features is calculated through the multi-head attention module to capture the complex relationship between the points, and finally the softmax model is used for classification.

[0041] The specific method of calculating the self-attention of point features through the multi-head attention module to capture the complex relationship between points is:

[0042] The attention mechanism of the multi-head attention module consists of query Q, key K and value V vectors, which are expressed as query vectors mapped to a series of key-value pairs; the attention mechanism first calculates the similarity between the query vector Q and the key vector K through a scaled dot product operation to obtain the contribution of each input point cloud data to the output result, and divides it by dk to maintain the stability of the gradient; secondly, the attention weight is converted into a probability distribution through softmax normalization, so that the sum of the attention weights is equal to 1; finally, each value vector V is multiplied by the corresponding attention weight and summed to obtain a weighted sum to represent the key information in the input sequence; the attention calculation method is expressed as:

[0043]

[0044] The multi-head attention mechanism used is a variant of the attention mechanism. The multi-head attention mechanism introduces multiple attention heads, each with its own query, key, and value matrix. The input frame point cloud sequence is mapped to a different representation space through linear transformation, and the score value of the i attention heads is calculated. The calculation method is expressed as:

[0045]

[0046] Finally, a linear transformation is introduced to concatenate the outputs of multiple attention heads and multiply them with the random parameter matrix. This can fuse the information of different heads and generate the final attention score value. See the formula:

[0047] MultiHead(Q,K,V)=Concat(H 1 ,H 2 …,H n )W Q

[0048] Masked multi-head attention is used in the decoder of the Transformer model to mask the input point cloud data, so that even if all the results are given to the Transformer at once during training, the effect is equivalent to giving them one by one.

[0049] Beneficial effects: Compared with the prior art, the present invention designs a deep neural network model that integrates the PointNet network and the Transformer module. The use of this model can enhance the description of the spatial relationship of the point cloud, thereby improving the judgment of human fall recognition and reducing the false alarm and false alarm rate. The present invention senses the posture and movement of the human body through millimeter wave radar signals to realize the recognition of human falls, and has many advantages such as good anti-interference performance and high recognition accuracy. The present invention can effectively protect privacy and can be deployed in various places such as homes, nursing homes, and public areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is an overall flow chart of the method of the present invention;

[0051] Figure 2 This is the operation flow chart of DBSCAN clustering;

[0052] Figure 3 Schematic diagram of DBSCAN algorithm processing point cloud clustering;

[0053] Figure 4 Design a flow chart for the human fall recognition method;

[0054] Figure 5 This is the PointNet network structure diagram;

[0055] Figure 6 This is the Transformer model structure diagram;

[0056] Figure 7 This is a curve chart of the learning rate changes during model training;

[0057] Figure 8 This is a curve chart of the change in loss during training;

[0058] Fig. 9 It is a confusion matrix diagram. DETAILED DESCRIPTION

[0059] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0060] like Figure 1 As shown, the present invention provides a method for human fall recognition based on millimeter wave radar perception, comprising the following steps:

[0061] S1: Use millimeter wave radar to collect echo data of various human body movements;

[0062] The FMCW millimeter wave radar is used to collect echo data of various human body movements. The FMCW millimeter wave radar consists of a transmitting antenna (RX), a receiving antenna (TX), a radio frequency signal module, a mixer and an analog-to-digital converter (ADC);

[0063] The FMCW millimeter-wave radar collects echo data of various human movements in the following way: the RF signal module generates a continuous frequency modulation signal, which is transmitted through the transmitting antenna after frequency multiplication. The reflection of the transmitted signal after encountering an obstruction is captured by the receiving antenna. The receiving signal is mixed with the transmitting signal to obtain an intermediate frequency signal. Finally, the intermediate frequency signal is converted into digital form and the subsequent signal is processed.

[0064] The frequency modulated continuous waveform FMCW of the millimeter wave radar transmission signal is expressed as:

[0065]

[0066] Among them, A T is the amplitude of the transmitted signal; f c is the starting frequency of the carrier signal; B is the bandwidth of the frequency modulated signal, S is the slope of the frequency modulated continuous wave signal; T c is the period of the frequency modulated signal;

[0067] After being reflected by human body movements, the signal received by the radar is expressed as follows:

[0068]

[0069] Among them, A R is the amplitude of the received echo signal; τ is the delay from the transmission to the reception of the FM signal; Δf d is the Doppler frequency shift value;

[0070] The intermediate frequency signal is obtained by mixing the received signal with the transmitted signal. The intermediate frequency signal is expressed as:

[0071]

[0072] S2: Process the echo data to obtain point cloud data of human body movements;

[0073] First, a one-dimensional fast Fourier transform is applied to each frame of echo data in the fast time dimension; then, the mean of each range unit in the slow time dimension is calculated and subtracted from the result obtained in the previous step; then, this result is again subjected to a one-dimensional fast Fourier transform in the slow time dimension to extract the range-velocity information of the moving target; the system then performs a constant false alarm rate (CFAR) detection to distinguish between background noise and actual target signals to ensure the accuracy of target detection; after completing the horizontal processing, the data will then undergo vertical beamforming and vertical angle estimation to obtain the position information of the target on the vertical plane; finally, the point cloud data generated by these steps will contain information such as the distance, angle, and Doppler velocity of the target, specifically:

[0074] It is obtained by Fourier transforming a single sweep time (i.e., the radar sweep signal in the fast time domain), that is, performing distance-FFT along each row; through the principle of distance measurement, searching according to different separated peak spectra, thereby obtaining the frequency point information of the target and the distance information between the target and the radar;

[0075]

[0076] Where d represents the distance between the target object and the radar sensor, c represents the speed of light, S is the slope of the FM continuous wave signal, τ is the delay from the transmission to the reception of the FM signal, and f IF is the frequency of the IF signal;

[0077] The target object speed is v, l is the distance between the receiving antennas, the wavelength is λ, T c is the frame period, and the intermediate frequency signal phase difference Δφ is generated according to the two Chirp signals. The target speed and angle are calculated respectively.

[0078]

[0079]

[0080] S3: Perform DBSCAN clustering on the point cloud data obtained in step S2;

[0081] DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based clustering algorithm. The purpose of DBSCAN clustering design in the present invention is to identify clusters of arbitrary shapes in the data and effectively handle noise and outliers. The core idea of ​​DBSCAN is to define clusters by density. Specifically, it relies on the local density of data points to divide clusters, rather than relying on the global number of clusters or cluster shapes. DBSCAN does not have to specify the number of clusters to use it, and can also optimize its performance according to specific data features by adjusting parameters, which enables DBSCAN to better handle complex data distribution in human fall scenarios and discover complex data structures and noise.

[0082] The working principle of DBSCAN in processing point cloud data is based on the following key points:

[0083] (1) Epsilon Neighborhood: For a given point p, its epsilon neighborhood refers to all points in a circular area with a radius of epsilon and a center at the point. epsilon is a user-defined parameter that represents the radius of the neighborhood.

[0084] (2) Minimum number of points (MinPts): MinPts is another user-defined parameter that specifies the minimum number of points that must exist within the ε neighborhood of a point in order for the point to be marked as a core point.

[0085] (3) Core Point: If the number of points in the ε neighborhood of a point (including the point itself) is greater than or equal to MinPts, then the point is called a core point.

[0086] (4) Border Point: If a point is not a core point but is within the ε neighborhood of a core point, then this point is called a border point.

[0087] (5) Noise Point: If a point is neither a core point nor a boundary point, it is marked as a noise point.

[0088] The clustering steps of DBSCAN are: (1) Identification of core points. For each point, calculate the number of points in its ε neighborhood. If the number of points in the neighborhood of the point is greater than or equal to MinPts, then the point is a core point. (2) Cluster expansion: Starting from each core point, check all points in its neighborhood. If these points are core points, add them to the cluster and continue to expand the cluster. This process continues until the cluster can no longer be expanded. (3) Allocation of boundary points: If a point is in the ε neighborhood of a core point but does not meet the core point conditions, then it is assigned to the corresponding cluster. (4) Processing of noise points. Points that do not belong to any cluster are marked as noise points.

[0089] Reference Figure 2 and Figure 3 , use the DBSCAN clustering method to cluster the point cloud data, and set the two parameters of the DBSCAN clustering algorithm (neighborhood radius ε and minimum number of points MinPts) to adapt to the data characteristics and clustering requirements of the millimeter wave radar; use the DBSCAN algorithm to cluster the preprocessed three-dimensional point cloud data, and find several center points of the point cloud data through the above parameters and steps, and divide the point cloud into different clusters based on the center points.

[0090] S4: Figure 4 As shown, the clustered point cloud data is input into the constructed and trained neural network to output the human fall recognition result. The neural network consists of a PointNet network and a Transformer module.

[0091] The neural network consists of PointNet network and Transformer modules;

[0092] like Figure 5 As shown in the figure, the PointNet network consists of several repeated modules and a maximum pooling layer. Each module contains a T-Net and a multi-layer perceptron MLP. The Transformer module is connected after the PointNet network and contains multiple modules such as point input embedding, position encoding, multi-head attention and feedforward network.

[0093] First, the point cloud data is input into the PointNet network, and the invariance of spatial transformation is maintained through the T-Net matrix transformation in the PointNet network; then MLP is used to extract features of the point cloud data, and finally the overall features of the three-dimensional point cloud are extracted through the maximum pooling layer.

[0094] The input to the PointNet network is a set of point cloud coordinate data, represented as an N×3 two-dimensional tensor, where N represents the number of points per frame and 3 represents the three-dimensional coordinates (x, y, z) of each point; the T-Net module in PointNet is a transformation layer that learns a transformation matrix through a neural network. This matrix is ​​used to perform geometric transformations such as rotation and scaling on the point cloud. The output of the T-Net module is a 3x3 transformation matrix used to rotate the point cloud in three-dimensional space. The learned transformation matrix will be applied to the point cloud data to ensure that the representation of the point cloud remains consistent after rotation. This step makes PointNet invariant to the rotation of the point cloud. By applying the transformation matrix between the input and output of the network, T-Net can reduce the impact of point cloud rotation on feature extraction and classification, so that the final classification and regression results are not affected by the spatial position and orientation of the point cloud. In the human fall scenario, the T-Net module enables PointNet to maintain the stability and consistency of its features when facing point clouds of different directions and postures, thereby improving the robustness and performance of the model. Next, the 3D features of the input point cloud are mapped to a 1024-dimensional space through MLP, and then the global features are extracted using the maximum pooling layer. Next, the MLP module performs downsampling and recombines the obtained 9 features into a 3×3 rotation matrix.

[0095] The purpose of using multiple T-Net modules in PointNet is to let each T-Net be responsible for learning the transformation matrices of different levels of point cloud data. In this way, the PointNet model can capture the transformations of different levels of point cloud data, thereby enhancing the robustness to point cloud rotation and transformation.

[0096] In PointNet, the max pooling operation is used as a symmetric function to handle the permutation invariance of point cloud data. Max pooling has the properties of symmetry and rotation invariance, as well as the function of feature aggregation. Since the order of points in the point cloud is not fixed, max pooling helps the network keep the order of these points unchanged and obtain a stable global feature representation. (1) Symmetry: Max pooling selects the maximum value of the features of all points in the feature dimension instead of relying on the order of the points, thus ensuring that the result of max pooling remains unchanged regardless of how the order of the input points changes; (2) Rotation invariance: By performing max pooling on the point cloud, the network can ignore the order of the points in the point cloud, ensuring that the extracted global features are invariant to the rotation and position changes of the points; (3) Feature aggregation: Applying max pooling on the feature vector of each point compresses the features of all points into a global feature vector of fixed length, thereby generating a representation of the global features of the point cloud.

[0097] like Figure 6 As shown in the figure, the Transformer module is connected after the PointNet network, which includes multiple modules such as point input embedding, position encoding, multi-head attention and feedforward network. The role of input embedding and position encoding is to serialize the global features or features of each point extracted by the PointNet network so that they can be input into the Transformer network. The multi-head attention module is used to calculate the self-attention of point features to capture the complex relationship between points. The attention mechanism allows the model to focus more on certain parts when processing radar point cloud information. The attention mechanism consists of query (Query, Q), key (Key, K) and value (Value, V) vectors, which can essentially be expressed as a query vector mapped to a series of key-value pairs. The attention mechanism first calculates the similarity between the query vector Q and the key vector K through a scaled dot product operation to obtain the contribution of each input point cloud data to the output result, and divides it by dk to maintain the stability of the gradient. Secondly, the attention weight is converted into a probability distribution through softmax normalization, so that the sum of the attention weights is equal to 1. Finally, each value vector V is multiplied by the corresponding attention weight and summed to obtain a weighted sum to represent the key information in the input sequence. The final attention calculation method is expressed as:

[0098]

[0099] The multi-head attention mechanism used in the present invention is a variant of the attention mechanism. Compared with the attention mechanism, the multi-head attention mechanism introduces multiple attention heads, each of which has its own query, key and value matrix. The input frame point cloud sequence is mapped to different representation spaces through linear transformation, and the score values ​​of i attention heads are calculated. The calculation method is expressed as:

[0100]

[0101] Finally, a linear transformation is introduced to concatenate the outputs of multiple attention heads and multiply them with the random parameter matrix. This can fuse the information of different heads and generate the final attention score value. See the formula:

[0102] MultiHead(Q,K,V)=Concat(H 1 ,H 2 …,H n )W Q

[0103] Masked Multi-Head Attention is a special form of multi-head attention. It is used in the decoder of the Transformer model to mask the input point cloud data, so that even if all the results are given to the Transformer at once during training, the effect is equivalent to giving them one by one. The purpose of this masking mechanism is to prevent the leakage of future information during the training process. Finally, the Transformer module uses softmax to classify the features and output the human fall recognition results.

[0104] In order to verify the effectiveness and effect of the method of the present invention, the following experiments were performed in this embodiment for verification, as follows:

[0105] 1. Experimental equipment and parameters

[0106] The experiment used a Vayyar vBlu radio frequency imaging module with 46 antenna arrays, where each transmitting antenna corresponds to multiple receiving antennas. The echo data was processed using a computer with an Intel i510400F processor, 8G memory, and an NVIDIA GTX1070 graphics card (8G video memory). The deep learning framework Pytorch was used for human motion recognition.

[0107] 2. Dataset Collection

[0108] In this experiment, 6 types of human actions are set, which are represented by serial numbers 1 to 6 in the data set, as shown in Table 1. Actions 1-6 represent side fall, back fall, front fall, forward jump, sit down, and walk. The data set is completed by 10 experimenters, and the personnel information is shown in Table 2. Each action in the collected action serial numbers 1-6 is collected 50 times, generating a total of 50×6×10=3000 data.

[0109] Table 1 Description of human actions

[0110]

[0111] Table 2 Experimenter information table

[0112]

[0113]

[0114] 3. Data Processing

[0115] Signal processing technology generates point clouds from echo data, which are briefly described below: First, a one-dimensional fast Fourier transform is performed on the fast time dimension for each frame of echo data; then the mean of each distance unit in the slow time dimension is calculated, and then this mean is subtracted from the result calculated in the previous step, and the result is then subjected to a one-dimensional fast Fourier transform in the slow time dimension to obtain the distance-speed information of the moving target; then, the system performs constant false alarm rate (CFAR) detection to distinguish between background noise and actual target signals to ensure the accuracy of target detection. After the horizontal processing is completed, the data undergoes vertical beamforming and angle estimation to obtain the position information of the target on the vertical plane; finally, these processing steps generate target point cloud data containing distance, angle, radial velocity and amplitude.

[0116] 4. Model training and testing

[0117] When training the model, Adam is used as the optimization method, where the initial value of the learning rate is set to 0.0125, the parameter weight decay is set to 0.09, and it decays every 2 epochs. Figure 7 This is the learning rate curve during model training. At the beginning of training, the learning rate is usually set high so that the model parameters can be updated quickly to escape from the undesirable local minimum. As the training progresses, the learning rate gradually decreases to refine the adjustment of the model parameters so that the model can be fine-tuned near the optimal solution. Eventually, the learning rate will decrease to a very small value, making the model parameter update very small, the loss function tends to be stable, and the model performance gradually stabilizes. Figure 8 The loss curve during training is shown in Figure 2. In the early stage of training, the model has not yet adapted to the point cloud data well, so the loss value will be high. As the model parameters are updated, the loss value will drop rapidly, indicating that the model is rapidly learning the patterns in the point cloud data. As the training progresses, the loss curve fluctuates after the 20th epoch. As the training continues, the loss curve gradually stabilizes, the model begins to stabilize, and the learning process is close to convergence. The average prediction accuracy of the model's human action category reaches 96.9%, and the confusion matrix is ​​shown in Figure 2. Fig. 9 As shown in the figure, the model has a better effect in recognizing backward falls, but a poor effect in recognizing forward jumps. Further analysis and optimization of the data set model and other aspects are needed to achieve similar results in recognizing forward falls, side falls, walking, and sitting down.

Claims

1. A method for human fall recognition based on millimeter wave radar perception, characterized in that: The steps include: S1: Use millimeter wave radar to collect echo data of human motion; S2: Process the echo data to obtain point cloud data corresponding to the human body movement; S3: Clustering point cloud data using clustering algorithm, dividing point cloud data into different clusters based on center points, and using cluster centers to describe human target information; S4: Input the point cloud data after clustering in step S2 into the constructed and trained neural network to capture the spatial structure and dependency information of the point cloud data, and use it to identify the category of human body movements and output the human fall recognition result.

2. The method for human fall recognition based on millimeter wave radar perception according to claim 1 is characterized in that: In the step S1, an FMCW millimeter wave radar is used to collect echo data of various human body movements, and the FMCW millimeter wave radar includes a transmitting antenna, a receiving antenna, a radio frequency signal module, a mixer and an analog-to-digital converter; The FMCW millimeter-wave radar collects echo data of various human movements in the following way: the RF signal module generates a continuous frequency modulation signal, which is transmitted through the transmitting antenna after frequency multiplication. The reflection of the transmitted signal after encountering an obstruction is captured by the receiving antenna. The receiving signal is mixed with the transmitting signal to obtain an intermediate frequency signal. Finally, the intermediate frequency signal is converted into digital form and the subsequent signal is processed. The frequency modulated continuous waveform FMCW of the millimeter wave radar transmission signal is expressed as: Among them, A T is the amplitude of the transmitted signal; f c is the starting frequency of the carrier signal; B is the bandwidth of the frequency modulated signal, S is the slope of the frequency modulated continuous wave signal; T c is the period of the frequency modulated signal; After being reflected by human motion, the signal received by the radar is expressed as follows: Among them, A R is the amplitude of the received echo signal; τ is the delay from the transmission to the reception of the FM signal; Δf d is the Doppler frequency shift value; The intermediate frequency signal is obtained by mixing the received signal with the transmitted signal. The intermediate frequency signal is expressed as:

3. The method for human fall recognition based on millimeter wave radar perception according to claim 1, characterized in that: In step S2, the echo data is generated into a point cloud by using signal processing technology, specifically: A1: Use one-dimensional fast Fourier transform on each frame of echo data in the fast time dimension; A2: Calculate the mean of the results obtained in step A1 in each distance unit in the slow time dimension; A3: Using the result of step A1, subtract the mean value corresponding to the distance unit calculated in step A2 from each distance unit; A4: Perform one-dimensional fast Fourier transform on the result of step A3 in the slow time dimension to obtain the distance-speed information of the moving target; A5: The system performs constant false alarm rate (CFAR) detection in distance and horizontal direction to distinguish background noise from actual target signals to ensure the accuracy of target detection; A6: After completing the horizontal processing, the data undergoes vertical beamforming and vertical angle estimation to obtain the target's position information on the vertical plane; A7: The above operations generate point cloud data of the target, including information on distance, angle, radial velocity, and amplitude.

4. The method for human fall recognition based on millimeter wave radar perception according to claim 3 is characterized in that: The point cloud data of the target in step A7 is expressed as follows: It is obtained by Fourier transforming a single frequency sweep time, that is, performing distance-FFT along each row; through the principle of distance measurement, searching according to different separated peak spectra, the frequency point information of the target and the distance information between the target and the radar are obtained; Where d represents the distance between the target object and the radar sensor, c represents the speed of light, S is the slope of the FM continuous wave signal, τ is the delay from the transmission to the reception of the FM signal, and f IF is the frequency of the IF signal; The target object speed is v, l is the distance between the receiving antennas, the wavelength is λ, T c is the frame period, and the intermediate frequency signal phase difference Δφ is generated according to the two Chirp signals. The target speed and angle are calculated respectively.

5. The method for human fall recognition based on millimeter wave radar perception according to claim 1, characterized in that: In step S3, the point cloud data is clustered using the DBSCAN clustering method, and the neighborhood radius and the minimum number of points of the DBSCAN clustering algorithm are set to adapt to the data characteristics and clustering requirements of the millimeter wave radar; the preprocessed three-dimensional point cloud data is clustered using the DBSCAN algorithm, and several center points of the point cloud data are found through the above parameters and steps, and the point cloud is divided into different clusters based on the center points.

6. The method for human fall recognition based on millimeter wave radar perception according to claim 1, characterized in that: In step S4, the neural network consists of a PointNet network and a Transformer module. The PointNet network consists of several repeated modules and a maximum pooling layer. Each module contains a T-Net and a multi-layer perceptron. The Transformer module is connected after the PointNet network and includes point input embedding, position encoding, multi-head attention and feedforward network modules.

7. The method for human fall recognition based on millimeter wave radar perception according to claim 6 is characterized in that: The operation mode of the PointNet network in step S4 is as follows: PointNet receives a point cloud data frame as input, maintains the invariance of spatial transformation through T-Net matrix transformation, then uses a multi-layer perceptron MLP to extract features from the point cloud data, and finally extracts the overall features of the three-dimensional point cloud through a maximum pooling layer.

8. The method for human fall recognition based on millimeter wave radar perception according to claim 7 is characterized in that: The input of the PointNet network in step S4 is a set of point cloud coordinate data, represented as an N×3 two-dimensional tensor, where N represents the number of points per frame and 3 represents the three-dimensional coordinates (x, y, z) of each point; The T-Net module maps the three-dimensional features of the input point cloud data to a 1024-dimensional space through the MLP module, extracts global features through the maximum pooling layer, and then downsamples using the MLP module. The 9 features obtained by downsampling are recombined to obtain a 3×3 rotation matrix.

9. The method for human fall recognition based on millimeter wave radar perception according to claim 6, characterized in that: The operation mode of the Transformer module in step S4 is as follows: the global features or the features of each point extracted by the PointNet network are serialized through the input embedding and position encoding modules, the self-attention of the point features is calculated through the multi-head attention module to capture the complex relationship between the points, and finally the softmax model is used for classification.

10. The method for human fall recognition based on millimeter wave radar perception according to claim 9, characterized in that: The specific method of calculating the self-attention of point features by the multi-head attention module in step S4 to capture the complex relationship between points is: The attention mechanism of the multi-head attention module consists of query Q, key K and value V vectors, which are expressed as query vectors mapped to a series of key-value pairs; the attention mechanism first calculates the similarity between the query vector Q and the key vector K through a scaled dot product operation to obtain the contribution of each input point cloud data to the output result, and divides it by dk to maintain the stability of the gradient; secondly, the attention weight is converted into a probability distribution through softmax normalization, so that the sum of the attention weights is equal to 1; finally, each value vector V is multiplied by the corresponding attention weight and summed to obtain a weighted sum to represent the key information in the input sequence; the attention calculation method is expressed as: The multi-head attention mechanism used is a variant of the attention mechanism. The multi-head attention mechanism introduces multiple attention heads, each with its own query, key, and value matrix. The input frame point cloud sequence is mapped to a different representation space through linear transformation, and the score value of the i attention heads is calculated. The calculation method is expressed as: Finally, a linear transformation is introduced to concatenate the outputs of multiple attention heads and multiply them with the random parameter matrix. This can fuse the information of different heads and generate the final attention score value. See the formula: MultiHead(Q,K,V)=Concat(H1,H2…,H n )W Q Masked multi-head attention is used in the decoder of the Transformer model to mask the input point cloud data, so that even if all the results are given to the Transformer at once during training, the effect is equivalent to giving them one by one.

Images