A radar point cloud data enhancement method for posture recognition

Abstract

The present invention discloses a radar point cloud data enhancement method for posture recognition. A human body posture point cloud dataset is collected via radar; data enhancement is performed on the point cloud in the distance, angle, and speed dimensions in sequence; the point cloud is divided into voxel blocks in the form of a three-dimensional matrix through voxelization; the main parts of the human body are divided according to the characteristics of the human body posture, and a neural network for human body posture is used for training. By inputting real point cloud data, the present invention can enhance data in the distance, angle, and speed dimensions, generate virtual point cloud data of different distances, angles, and speeds, and use a neural network to classify human body postures after data preprocessing according to the posture characteristics. This alleviates the problem of small sample datasets in the field of radar posture, enriches the radar dataset, improves recognition accuracy, and facilitates subsequent research at the deep learning level.

Description

Technical Field

[0001] The present invention relates to the field of data enhancement, and in particular to a radar point cloud data enhancement method for gesture recognition. Background Art

[0002] Deep learning has developed rapidly in recent years and has been widely applied in various fields. The quality of training models in deep learning is closely related to the size of the dataset. Sufficient data can also prevent overfitting during training. However, obtaining high-quality datasets has become a bottleneck restricting the effectiveness of deep learning in some fields.

[0003] Numerous open-source datasets exist for image pose data, and data collection is relatively easy. However, in the radar field, public datasets are scarce due to difficulties in data collection and generalizability, and small sample sizes are common. Applying deep learning methods to pose recognition using radar point cloud data can easily lead to problems such as network overfitting.

[0004] Therefore, how to combine the characteristics of radar signals with sensor parameters to use radar point cloud data for reliable data enhancement and expand the data set for posture recognition is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a radar point cloud data enhancement method for gesture recognition.

[0006] The object of the present invention is achieved through the following technical solution: a radar point cloud data enhancement method for gesture recognition, comprising the following steps:

[0007] Step 1: Obtain a human body posture point cloud dataset through radar as the first data subset, and calculate the point cloud signal-to-noise ratio;

[0008] Step 2: Perform distance dimension enhancement on the first data subset. After translating the point cloud outward by a fixed distance, update the signal-to-noise ratio of the point cloud, and then add noise to the signal-to-noise ratio to improve generalization performance. Based on the radar detection performance, remove abnormal points that do not conform to physical laws to obtain a subset after data enhancement, which is recorded as the second data subset.

[0009] Step 3: Perform angle dimension enhancement on the first data subset, update the signal-to-noise ratio of the angle-adjusted point cloud, and then add noise to the signal-to-noise ratio to improve generalization performance. Based on the radar detection performance, remove abnormal points that do not conform to physical laws to obtain a subset after data enhancement, which is recorded as the third data subset.

[0010] Step 4: Perform velocity dimension enhancement on the first data subset, perform downsampling on the point cloud dataset and adjust the velocity value of the point cloud to obtain the fourth data subset;

[0011] Step 5: Preprocess the enhanced human body posture point cloud data using a voxelization method, and divide the human body posture point cloud data into a three-dimensional grid matrix of M×N×L;

[0012] Step 6: Based on the characteristics of human body posture and movement, the three-dimensional grid matrix obtained in step 5 is segmented according to the human body structure, and then sent to the neural network model for human posture recognition for training.

[0013] Furthermore, in step 1, the point cloud acquired by the radar is divided into a point cloud in a polar coordinate system and a point cloud in a rectangular coordinate system. The point cloud in the rectangular coordinate system needs to be converted into a point cloud in a polar coordinate system. The formula is as follows:

[0014]

[0015] Where x, y, and z are the coordinates of the x, y, and z axes in the rectangular coordinate system, R is the distance from the radar to the point cloud, θ is the horizontal angle, and φ is the pitch angle.

[0016] Furthermore, in step 1, calculating the signal-to-noise ratio information of the point cloud includes:

[0017] Calculate the power density S at the point cloud:

[0018]

[0019] Among them, P t is the radar transmission power, R is the distance from the radar to the point cloud;

[0020] Radars usually use directional antennas. The relationship between antenna gain G, effective area A, and radar wavelength λ is as follows:

[0021]

[0022] When the radar transmitting antenna gain is G t In the radiation direction, the power density S1 at the point cloud with a distance R from the radar is:

[0023]

[0024] If it is assumed that the echo signal received by the human body at the point cloud is lossless omnidirectional radiation, the radar receiving antenna echo power density S2 is:

[0025]

[0026] Where α is the radar cross section;

[0027] According to the effective receiving area A of the radar receiving antenna r Calculate the received echo power P r :

[0028]

[0029] Where λ is the radar wavelength, G r is the receiving antenna gain;

[0030] Taking into account the internal noise of the radar and the interference of the external environment, the signal-to-noise ratio (SNR) of the point cloud is finally obtained:

[0031]

[0032] Where T meas is the total measurement time, k is the Boltz constant, T is the antenna temperature, and F is the noise figure.

[0033] Furthermore, in step 2, based on the known point cloud signal-to-noise ratio SNR1 at the distance d1, the point cloud signal-to-noise ratio SNR2 at the distance d2 is calculated;

[0034]

[0035] Therefore, after translating the point cloud outward by a fixed distance d, the signal-to-noise ratio SNR′ after translation is obtained d ;

[0036]

[0037] where d o is the distance between the point cloud and the radar before translation, SNR d is the signal-to-noise ratio before translation.

[0038] Furthermore, in step 3, for a single-channel radar, the signal-to-noise ratio SNR′ after angle dimension enhancement is a The calculation formula is as follows:

[0039]

[0040] Among them, G o G is the gain of the radar transceiver antenna pair before angle adjustment. v is the gain of the radar transmitting and receiving antenna pair after angle adjustment, SNR a is the signal-to-noise ratio before angle adjustment.

[0041] Furthermore, in step 3, antenna simulation and antenna measurement are performed on the radar in sequence to obtain the directional pattern gain characteristics of the transmitting and receiving antenna pairs, respectively. The directional pattern gain obtained by simulation is used to correct the measured directional pattern gain deviation to obtain the corrected directional pattern gain. If the radar has multiple transmitting and receiving antenna pairs, the corrected directional pattern gain characteristics of different transmitting and receiving antenna pairs are fitted according to the relationship between the pairwise combinations of the transmitting and receiving antennas to obtain the gain function G(θ)={G1(θ),G2(θ),…,Gn (θ)}, θ is the new angle after adjustment, G i (θ) is the gain function of the i-th group of transmitting and receiving antenna pairs; after the angle adjustment, the transmitting and receiving antenna gains need to be accumulated, and the signal-to-noise ratio SNR′ after the angle adjustment b The calculation formula is as follows:

[0042]

[0043] Where n is the total number of transmit and receive antenna pairs, G oi is the gain of the i-th antenna transceiver pair before angle adjustment, G vi is the gain of the i-th antenna transceiver pair after angle adjustment, SNR b is the signal-to-noise ratio before angle adjustment.

[0044] Furthermore, in step 3, if the radar supports detection of pitch angles and horizontal angles, the point cloud data is adjusted in terms of horizontal angle dimension and pitch angle dimension at the same time, and then the two are combined to obtain the human posture point cloud data with expanded angle dimension; for human postures with large vertical changes such as jumping and squatting, the pitch angle dimension is mainly used, and the horizontal angle dimension is used as an auxiliary correction; for human postures with large horizontal changes such as walking and punching, the horizontal angle dimension is mainly used, and the pitch angle dimension is used as an auxiliary correction.

[0045] Furthermore, in steps 2 and 3, the noise added is Gaussian noise. Taking the point cloud as the origin, the Gaussian noise at the adjacent positions of the point cloud is as follows:

[0046]

[0047] SNR′ (x,y,z) =SNR (x,y,z) +H x,y,z

[0048] Where σ is the point cloud variance, (x, y, z) is the point cloud coordinate, is the coordinate of the adjacent position of the point cloud, H x,y,z is the Gaussian noise intensity of the coordinate (x, y, z) point cloud, SNR (x,y,z) ,SNR′ (x,y,z) The signal-to-noise ratio before and after adding noise to the coordinate (x, y, z) point cloud.

[0049] Furthermore, in steps 2 and 3, outliers that do not conform to physical laws are removed based on the radar detection performance. The conditions are as follows: the minimum signal-to-noise ratio (SNR) of the point cloud that the radar can actually detect is min As a benchmark, after a series of operations, the signal-to-noise ratio strength is less than SNR min The points are removed from the point cloud dataset.

[0050] Furthermore, in steps 2 and 3, a density-based clustering algorithm is used to cluster the human body posture point cloud to eliminate abnormal interference points that do not belong to the human body. Based on the characteristics of the human body posture point cloud being closely associated in the horizontal plane and highly dispersed in the vertical plane, the improved Euclidean distance is used instead of the traditional Euclidean distance as the distance parameter in the density-based clustering algorithm to reduce the influence of the z-axis in the clustering process. The formula is as follows:

[0051] D(q i ,q j )=(x i -x j ) 2 +(y i -y j ) 2 +0.25*(z i -z j ) 2

[0052] Where D(q i ,q j ) is the point cloud q i and point cloud q j Improved Euclidean distance, x i ,y i ,z i and x j ,y j ,z j Point cloud q i and point cloud q j The three-dimensional coordinates of .

[0053] Furthermore, in step 4, since the radar collects data at a fixed frame rate, when the human body accelerates, the number of frames collected for the same posture action will decrease; the collected point cloud data set is distributed in time sequence, represented by F = {f1, f2, ..., f h}, where f i represents the human body posture point cloud dataset of the i-th frame, h represents the total number of frames collected; a random sampling method is used to select frames with a ratio of p from F to form a new temporal distribution human body posture point cloud dataset F′={f1,f2,…,f m},in The speed of updating the point cloud in the point cloud dataset at the same time:

[0054]

[0055] Where v represents the original velocity of the point cloud, and v′ represents the updated velocity of the point cloud.

[0056] Furthermore, the second, third, and fourth data subsets are combined and expanded, that is, the methods of step 2, step 3, and step 4 are arbitrarily combined as needed to form point cloud data sets of different distances, angles, and speeds; according to the characteristics of human posture point cloud data, speed dimension enhancement can provide the neural network with human posture point cloud data of different speeds containing more information, while distance dimension translation has less effect on the distribution of human posture point clouds than angle dimension rotation. Therefore, the selection order of the three dimensions is: speed dimension, angle dimension, and distance dimension. When combined and expanded, the allocated weights decrease in sequence.

[0057] Furthermore, in step 5, a voxelization operation is performed to traverse all point clouds of the human body to obtain the maximum and minimum three-dimensional coordinates of the human body posture point cloud, which are x min ,x max ,y min ,y max ,z min ,z max Obtain the length X, width Y, and height Z of the human body area, divide the human body area evenly into M×N×L individual blocks, and calculate the intensity value of each block; the intensity value of the block is calculated by the number of point clouds in the block, the sum of the signal-to-noise ratio intensity of all point clouds in the block, and the sum of the velocities of all point clouds in the block; finally, obtain the three-dimensional grid matrix π;

[0058]

[0059] Among them I x , I y , I z Indicates the block numbers on the x-axis, y-axis, and z-axis respectively.

[0060] Furthermore, in step 6, for common human postures, the main moving parts of the human body are divided into three parts: the left arm, the torso, and the right arm. The left arm, the torso, the right arm, and the human body as a whole are processed using a long short-term memory network. At the same time, the human posture point clouds of all frames in the time window are superimposed and aggregated and then processed using a convolutional neural network. The post-decision fusion method is adopted to adjust the weights of these five parts through the attention mechanism module to obtain the human posture recognition result.

[0061] The beneficial effects of the present invention are as follows: The present invention proposes a radar point cloud data enhancement method for posture recognition. Based on real human posture point cloud data collected by radar, it can perform data enhancement in the distance, angle, and speed dimensions. In combination with the characteristics of the radar itself, it can screen point clouds that conform to physical characteristics. A density-based clustering algorithm is used to cluster the human posture point clouds to eliminate abnormal interference points that do not belong to the human body. The point cloud is divided into voxel blocks in the form of a three-dimensional matrix through voxelization processing; the main parts of the human body are divided according to the characteristics of human posture, and a neural network for human posture is used for training. This method can greatly expand the size of the data set, alleviate the problem of insufficient radar signal data sets, and effectively improve the performance of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flowchart of a method provided by an exemplary embodiment;

[0063] Figure 2 is a schematic diagram of an original point cloud provided by an exemplary embodiment;

[0064] Figure 3 is a schematic diagram of a point cloud after distance dimension enhancement provided by an exemplary embodiment;

[0065] Figure 4 is a schematic diagram of a point cloud after angle dimension enhancement provided by an exemplary embodiment;

[0066] Figure 5 is a schematic diagram of a point cloud after velocity dimension enhancement provided by an exemplary embodiment;

[0067] Figure 6 is a schematic diagram of radar installation provided by an exemplary embodiment;

[0068] Figure 7 is a schematic diagram of point cloud voxelization provided by an exemplary embodiment;

[0069] Figure 8 It is a schematic diagram of a neural network structure provided by an exemplary embodiment. DETAILED DESCRIPTION

[0070] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.

[0071] Without loss of generality, this embodiment provides a radar point cloud data enhancement method for gesture recognition, the process is as follows Figure 1As shown in the figure, an FMCW millimeter wave sensor with a frequency range of 60 GHz to 64 GHz is used. The transmission signal frame rate is 20 frames per second. Each frame of data is divided into 288 chirp signals, and each chirp signal has 96 sampling points.

[0072] Step 1: Install the radar at a height of 1.5m above the ground. Figure 6 As shown. The human body posture point cloud dataset is obtained by radar as the first data subset. If the point cloud obtained is in a rectangular coordinate system, it needs to be converted into a polar coordinate system. The formula is as follows:

[0073]

[0074] Where x, y, and z are the coordinates of the x, y, and z axes in the rectangular coordinate system, R is the distance from the radar to the point cloud, θ is the horizontal angle, and φ is the pitch angle.

[0075] Calculate the power density S at the point cloud:

[0076]

[0077] Among them, P t is the radar transmission power, R is the distance from the radar to the point cloud;

[0078] Radars usually use directional antennas. The relationship between antenna gain G, effective area A, and radar wavelength λ is as follows:

[0079]

[0080] When the radar transmitting antenna gain is G t In the radiation direction, the power density S1 at the point cloud with a distance R from the radar is:

[0081]

[0082] If it is assumed that the echo signal received by the human body at the point cloud is lossless omnidirectional radiation, the radar receiving antenna echo power density S2 is:

[0083]

[0084] Where α is the radar cross section;

[0085] According to the effective receiving area A of the radar receiving antenna r Calculate the received echo power P r :

[0086]

[0087] Where λ is the radar wavelength, G r is the receiving antenna gain;

[0088] Taking into account the internal noise of the radar and the interference of the external environment, the signal-to-noise ratio (SNR) of the point cloud is finally obtained:

[0089]

[0090] Where T meas is the total measurement time, k is the Boltz constant, T is the antenna temperature, and F is the noise figure.

[0091] Step 2: Perform distance dimension enhancement on the first data subset. When the distance changes, the signal strength will change significantly. Based on the known point cloud signal-to-noise ratio SNR1 at distance d1, calculate the point cloud signal-to-noise ratio SNR2 at distance d2.

[0092]

[0093] Therefore, after translating the point cloud outward by a fixed distance d, the signal-to-noise ratio SNR′ after translation is obtained d ;

[0094]

[0095] where d o is the distance between the point cloud and the radar before translation, SNR d is the signal-to-noise ratio before translation.

[0096] Then, we add Gaussian noise to the signal-to-noise ratio to improve the generalization performance. The Gaussian noise is added, with the point cloud as the origin, and the Gaussian noise at the adjacent positions of the point cloud is as follows:

[0097]

[0098] SNR′ (x,y,z) =SNR (x,y,z) +H x,y,z

[0099] Where σ is the point cloud variance, (x, y, z) is the point cloud coordinate, is the coordinate of the adjacent position of the point cloud, H x,y,z is the Gaussian noise intensity of the coordinate (x, y, z) point cloud, SNR (x,y,z) ,SNR′ (x,y,z) The signal-to-noise ratio before and after adding noise to the coordinate (x, y, z) point cloud.

[0100] Then, based on the radar detection capability, the minimum signal-to-noise ratio (SNR) of the point cloud that the radar can actually detect is used. min As a benchmark, after a series of operations, the signal-to-noise ratio strength is less than SNR minThe points are removed from the point cloud dataset. A density-based clustering algorithm is used to cluster the human body posture point cloud, eliminating abnormal interference points that do not belong to the human body. Based on the characteristics of the human body posture point cloud being closely associated in the horizontal plane and highly dispersed in the vertical plane, the improved Euclidean distance is used instead of the traditional Euclidean distance as the distance parameter in the density-based clustering algorithm to reduce the influence of the z-axis in the clustering process, and obtain a data-enhanced subset, which is recorded as the second data subset.

[0101] D(q i ,q j )=(x i -x j ) 2 +(y i -y j ) 2 +0.25*(z i -z j ) 2

[0102] Where D(q i ,q j ) is the point cloud q i and point cloud q j Improved Euclidean distance, x i ,y i ,z i and x j ,y j ,z j Point cloud q i and point cloud q j The three-dimensional coordinates of .

[0103] Step 3: Perform angle dimension enhancement on the first data subset. For a single-channel radar, the signal-to-noise ratio (SNR) after angle dimension enhancement is a The calculation formula is as follows:

[0104]

[0105] Among them, G o G is the gain of the radar transceiver antenna pair before angle adjustment. v is the gain of the radar transmitting and receiving antenna pair after angle adjustment, SNR a is the signal-to-noise ratio before angle adjustment.

[0106] In this embodiment, the radar has four receiving antennas and three transmitting antennas, and uses multiple-input multiple-output technology to form 12 transceiver antenna pairs. Antenna simulation and antenna measurement are performed on the radar in sequence to obtain the directional pattern gain characteristics of the transceiver antenna pairs. The simulated directional pattern gain is used to correct the measured directional pattern gain deviation to obtain the corrected directional pattern gain. For radars with multiple transceiver antenna pairs, the corrected directional pattern gain characteristics of different transceiver antenna pairs are fitted according to the relationship between the two combinations of transceiver antennas to obtain the transceiver antenna pair gain function G(θ) = {G1(θ), G2(θ), …, G n (θ)}, θ is the new angle after adjustment, G i (θ) is the gain function of the i-th group of transmitting and receiving antenna pairs; after the angle adjustment, the transmitting and receiving antenna gains need to be accumulated, and the signal-to-noise ratio SNR′ after the angle adjustment b The calculation formula is as follows:

[0107]

[0108] Where n is the total number of transmit and receive antenna pairs, G oi is the gain of the i-th antenna transceiver pair before angle adjustment, G vi is the gain of the i-th antenna transceiver pair after angle adjustment, SNR b is the signal-to-noise ratio before angle adjustment.

[0109] Similar to step 2, noise is added to the signal-to-noise ratio to improve generalization performance. Based on the radar's detection capabilities, outliers that do not conform to physical laws are removed. A density-based clustering algorithm is used to cluster the human posture point cloud, eliminating abnormal interference points that do not belong to the human body, and obtaining a data-enhanced subset, which is recorded as the third data subset. In this embodiment, the radar supports the detection of pitch angles and horizontal angles, so the point cloud data can be enhanced in both the horizontal angle dimension and the pitch angle dimension. The two are then combined to obtain human posture point cloud data with expanded angle dimensions. For human postures with large vertical changes such as jumping and squatting, the pitch angle dimension is mainly used, and the horizontal angle dimension is used as an auxiliary correction. For human postures with large horizontal changes such as walking and punching, the horizontal angle dimension is mainly used, and the pitch angle dimension is used as an auxiliary correction. The signal-to-noise ratio after updating the angle adjustment is used as the third data subset.

[0110] Step 4: Perform velocity dimension enhancement on the first data subset. Since the radar collects data at a fixed frame rate, when the human body accelerates, the number of frames collected for the same posture action will decrease; the collected point cloud data set is distributed in time sequence, expressed as F = {f1, f2, ..., f h}, where f irepresents the human body posture point cloud dataset of the i-th frame, h represents the total number of frames collected; using random sampling, a frame with a ratio of p = 0.8 is selected from F to form a new temporal distribution human body posture point cloud dataset F′ = {f1,f2,…,f m},in The speed of updating the point cloud in the point cloud dataset at the same time:

[0111]

[0112] Where v represents the original velocity of the point cloud, and v′ represents the updated velocity of the point cloud.

[0113] Figure 2-Figure 5 Schematic diagrams of the original point cloud, the point cloud after distance dimension enhancement, the point cloud after angle dimension enhancement, and the point cloud after velocity dimension enhancement provided for an implementation example respectively.

[0114] Step 5: Combine and expand the second, third, and fourth data subsets. That is, the methods of Steps 2, 3, and 4 are arbitrarily combined as needed to construct point cloud datasets of varying distances, angles, and velocities. Given the characteristics of human pose point cloud data, velocity enhancement can provide the neural network with more informative human pose point cloud data at varying speeds. Distance translation, compared to angle rotation, has a smaller effect on the distribution of human pose point clouds. Therefore, the three dimensions are selected in the following order: velocity, angle, and distance, with decreasing weights assigned to them during combined expansion.

[0115] The point cloud is preprocessed by voxelization method. The schematic diagram of point cloud voxelization is as follows Figure 7 Traverse all the point clouds of the human body to obtain the maximum and minimum three-dimensional coordinates of the human body posture point cloud, which are x min ,x max ,y min ,y max ,z min ,z max ; Get the length X, width Y, and height Z of the human body area, divide the human body area evenly into 32×32×10 blocks, and calculate the sum of the signal-to-noise ratios of all point clouds in the block as the intensity value of the block, and finally get the three-dimensional grid matrix π.

[0116]

[0117] Among them I x , I y , I z Indicates the block numbers on the x-axis, y-axis, and z-axis respectively.

[0118] Step 6. For common human postures, the main moving parts of the human body are divided into three parts: the left arm, the torso, and the right arm. The long short-term memory network is used to process the left arm, the torso, the right arm, and the human body as a whole. At the same time, the human posture point clouds of all frames in the time window are superimposed and aggregated and then processed using a convolutional neural network. The post-decision fusion method is adopted to adjust the weights of these five parts through the attention mechanism module to obtain the human posture recognition results. Figure 8 This is a schematic diagram of a neural network structure provided by an implementation example.

[0119] The following is an application scenario for radar point cloud data enhancement, but it is not limited to this. A radar sensor is placed indoors, and the original point cloud data of a specific posture (jumping, walking, squatting, punching) is collected at a fixed distance and angle. The enhancement parameters, including translation distance, rotation angle, and random sampling ratio, are set to generate an enhanced data set. After voxelization preprocessing, it is sent to Figure 8 The neural network model shown in Figure 2 was trained and the classification results were compared. The enhanced human posture recognition accuracy of the present invention increased from 90.42% of the original point cloud data to 93.29%. This completes the data enhancement, greatly expanding the data set while improving the accuracy.

[0120] In summary, the present invention proposes a radar point cloud data enhancement method for gesture recognition. This method collects radar point cloud data and reliably enhances distance, angle, velocity, and any combination of dimensions. After voxelizing the point cloud, the human torso data is divided according to the characteristics of human posture and trained using a specific neural network. This method significantly expands the dataset size and improves recognition accuracy, providing sufficient data for subsequent research. This alleviates the current data shortage in radar gesture datasets and facilitates deep learning research.

[0121] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the scope of protection of the invention.

Claims

1. A radar point cloud data enhancement method for gesture recognition, characterized in that: The method comprises the following steps: Step 1: Obtain a human body posture point cloud dataset through radar as the first data subset, and calculate the point cloud signal-to-noise ratio (SNR). The calculation formula of the point cloud signal-to-noise ratio (SNR) is: Where T meas is the total measurement time, k is the Boltz constant, T is the antenna temperature, F is the noise figure, P t is the radar transmission power, R is the distance from the radar to the point cloud, G t is the radar transmitting antenna gain, α is the radar scattering cross section, λ is the radar wavelength, G r is the receiving antenna gain; Step 2: Perform distance dimension enhancement on the first data subset, translate the point cloud outward by a fixed distance, update the signal-to-noise ratio of the point cloud, and then add noise to the signal-to-noise ratio to improve the generalization performance; translate the point cloud outward by a fixed distance d to obtain the signal-to-noise ratio SNR′ after translation d ; where d o is the distance between the point cloud and the radar before translation, SNR d is the signal-to-noise ratio before translation; According to the radar detection performance, outliers that do not conform to physical laws are eliminated to obtain a subset after data enhancement, which is recorded as the second data subset; Step 3: Perform angle dimension enhancement on the first data subset, update the signal-to-noise ratio of the point cloud after angle adjustment, and then add noise to the signal-to-noise ratio to improve the generalization performance; according to the radar detection performance, remove abnormal points that do not conform to physical laws to obtain the subset after data enhancement, which is recorded as the third data subset; for single-channel radar, the signal-to-noise ratio SNR′ after angle dimension enhancement a The calculation formula is as follows: Among them, G o G is the gain of the radar transceiver antenna pair before angle adjustment. v is the gain of the radar transmitting and receiving antenna pair after angle adjustment, SNR a is the signal-to-noise ratio before angle adjustment; The radar antenna is simulated and measured in turn to obtain the pattern gain characteristics of the transmitting and receiving antenna pairs. The simulated pattern gain is used to correct the measured pattern gain deviation to obtain the corrected pattern gain. If the radar has multiple transmitting and receiving antenna pairs, the corrected pattern gain characteristics of different transmitting and receiving antenna pairs are fitted according to the relationship between the two combinations of transmitting and receiving antennas, and the gain function of the transmitting and receiving antenna pairs G(θ)={G1(θ),G2(θ),…,G n (θ)}, θ is the new angle after adjustment, G i (θ) is the gain function of the i-th group of transmitting and receiving antenna pairs; after the angle adjustment, the transmitting and receiving antenna gains need to be accumulated, and the signal-to-noise ratio SNR′ after the angle adjustment b The calculation formula is as follows: Where n is the total number of transmit and receive antenna pairs, G oi is the gain of the i-th antenna transceiver pair before angle adjustment, G vi is the gain of the i-th antenna receiving and transmitting pair after angle adjustment, SNR b is the signal-to-noise ratio before angle adjustment; If the radar supports the detection of pitch and horizontal angles, the point cloud data is adjusted for both horizontal and pitch angles, and then the two are combined to obtain the human posture point cloud data with expanded angle dimensions. For human postures with large vertical variations, the pitch angle dimension is used as the primary correction, with the horizontal angle dimension used as an auxiliary correction. For human postures with large horizontal variations, the horizontal angle dimension is used as the primary correction, with the pitch angle dimension used as an auxiliary correction. Step 4: Enhance the velocity dimension of the first data subset, downsample the point cloud dataset and adjust the velocity value of the point cloud to obtain the fourth data subset. Since the radar collects data at a fixed frame rate, when the human body accelerates, the number of frames collected for the same posture action will decrease. The collected point cloud dataset is distributed in time series, expressed as F = {f1, f2, ..., f h }, where f i represents the human body posture point cloud dataset of the i-th frame, h represents the total number of frames collected; using random sampling, a frame with a ratio of p is selected from F to form a new temporal distribution human body posture point cloud dataset F ′ ={f1,f2,…,f m },in The speed of updating the point cloud in the point cloud dataset at the same time: Where v represents the original velocity of the point cloud, and v′ represents the updated velocity of the point cloud; Step 5: Preprocess the enhanced human body posture point cloud data using a voxelization method, and divide the human body posture point cloud data into a three-dimensional grid matrix of M×N×L; Step 6: Based on the characteristics of human body posture and movement, the three-dimensional grid matrix obtained in step 5 is segmented according to the human body structure, and then sent to the neural network model for human posture recognition for training.

2. The radar point cloud data enhancement method for gesture recognition according to claim 1, characterized in that: In steps 2 and 3, the noise added is Gaussian noise. Taking the point cloud as the origin, the Gaussian noise at the adjacent positions of the point cloud is as follows: SNR′ (x,y,z) =SNR (x,y,z) +H x,y,z Where σ is the point cloud variance, (x, y, z) is the point cloud coordinate, is the coordinate of the adjacent position of the point cloud, H x,y,z is the Gaussian noise intensity of the coordinate (x, y, z) point cloud, SNR (x,y,z) ,SNR′ (x,y,z) The signal-to-noise ratio before and after adding noise to the coordinate (x, y, z) point cloud.

3. The radar point cloud data enhancement method for gesture recognition according to claim 1, characterized in that: In steps 2 and 3, outliers that do not conform to physical laws are removed based on the radar detection performance. The conditions are as follows: the minimum signal-to-noise ratio (SNR) of the point cloud that the radar can actually detect is min As a benchmark, after a series of operations, the signal-to-noise ratio strength is less than SNR min The points are removed from the point cloud dataset; A density-based clustering algorithm is used to cluster the human body posture point cloud and eliminate abnormal interference points that do not belong to the human body. According to the characteristics of the human body posture point cloud that is closely associated in the horizontal plane and highly dispersed in the vertical plane, the improved Euclidean distance is used instead of the traditional Euclidean distance as the distance parameter in the density-based clustering algorithm to reduce the influence of the z-axis in the clustering process. The formula is as follows: D(q i ,q j )=(x i -x j ) 2 +(y i -y j ) 2 +0.25*(z i -z j ) 2 Where D(q i ,q j ) is the point cloud q i and point cloud q j Improved Euclidean distance, x i ,y i ,z i and x j ,y j ,z j Point cloud q i and point cloud q j The three-dimensional coordinates of .

4. The radar point cloud data enhancement method for gesture recognition according to claim 1, characterized in that: The second, third and fourth data subsets are combined and expanded, that is, the methods of step 2, step 3 and step 4 are arbitrarily combined as needed to form a point cloud data set of different distances, angles and speeds; according to the characteristics of human posture point cloud data, speed dimension enhancement can provide the neural network with human posture point cloud data of different speeds containing more information, while distance dimension translation has less effect on the distribution of human posture point cloud than angle dimension rotation. Therefore, the selection order of the three dimensions is: speed dimension, angle dimension, distance dimension. When combined and expanded, the allocated weights decrease in sequence.

5. The radar point cloud data enhancement method for gesture recognition according to claim 1, characterized in that: In step 5, voxelization is performed to traverse all point clouds of the human body to obtain the maximum and minimum three-dimensional coordinates of the human body posture point cloud, which are x min ,x max ,y min ,y max ,z min ,z max ; Get the length X, width Y, and height Z of the human body area, divide the human body area evenly into M×N×L individual blocks, and calculate the intensity value of each block; The intensity value of the block is calculated by the following methods: the number of point clouds in the block, the sum of the signal-to-noise ratio intensities of all point clouds in the block, and the sum of the velocities of all point clouds in the block; finally, the three-dimensional grid matrix π is obtained; Among them I x , I y , I z Indicates the block numbers on the x-axis, y-axis, and z-axis respectively.

6. The radar point cloud data enhancement method for gesture recognition according to claim 1, characterized in that: In step 6, for common human postures, the main moving parts of the human body are divided into three parts: the left arm, the torso, and the right arm. The left arm, the torso, the right arm, and the human body as a whole are processed using a long short-term memory network. At the same time, the human posture point clouds of all frames in the time window are superimposed and aggregated and then processed using a convolutional neural network. The post-decision fusion method is adopted to adjust the weights of these five parts through the attention mechanism module to obtain the human posture recognition result.

Images