Omnidirectional human motion feature normalization method based on space-time motion path compensation

Abstract

The invention discloses an omnidirectional human motion feature normalization method based on space-time motion path compensation. The method comprises the following steps: firstly, processing radar echo signals, constructing a three-dimensional feature point cloud in a distance-angle-time space, and estimating a motion path based on a space-time fitting motion path estimation algorithm; and finally, compensating the features of the omni-directional human body actions into radial human body action features through a feature compensation algorithm based on a motion direction angle, and realizing normalization of the omni-directional human body action features. According to the scheme of the invention, the method starts from the feature distortion of the radar echo signal of the human motion caused by the motion direction, analyzes and solves the angle sensitivity problem from the angle of signal processing, restores the feature distortion caused by the motion angle, compensates the omnidirectional human motion features into radial features, and improves the detection precision. The method eliminates the influence of a motion angle on a radar measurement signal, improves the representation efficiency of features, is superior to a traditional feature extraction algorithm in accuracy, efficiency and robustness, and improves the utilization efficiency of information.

Description

Technical Field

[0001] The invention belongs to the field of radar human motion recognition, and in particular relates to an omnidirectional human motion feature normalization method based on space-time motion path compensation. Background Art

[0002] The application of human motion recognition in public security monitoring, smart home, medical health monitoring and human-computer interaction is increasing. With the aging of society, the public's demand for healthy and safe life is increasing. In this context, human motion will cause changes in the frequency of radar echo signals, a phenomenon known as the micro-Doppler effect. The micro-Doppler effect can reveal the unique characteristics of different human motions, thus laying a solid foundation for radar-based human motion recognition technology. Compared with contact sensors, radar sensors do not need to be in close contact with the surface of human skin to obtain data and can be used for long-term monitoring; compared with optical sensors, radar-based human motion recognition methods can overcome environmental influences such as darkness and obstacles, and are less restricted by the range of motion; in particular, since the radar echo signal only carries the characteristic information of the target and does not capture the visual form of the target, it can protect the personal privacy of the subject. Radar has been widely studied and applied in the field of human motion recognition due to its low environmental sensitivity (background, light), no contact, all-weather operation, privacy protection, safety and reliability, and low cost.

[0003] Existing human motion recognition methods based on radar sensors mainly focus on radial human motion, that is, human motion is approaching or moving away from the radar along the direction of the radar main beam. They realize human motion recognition by extracting the micro-Doppler and distance features of human motion. However, in practical applications, the movement direction of human motion is not limited to the radial direction. When there is an angle between the movement direction of human motion and the direction of the radar main beam, the radar detects the components of the micro-Doppler and distance information of the real motion in the radial direction. This causes deviations and distortions between the features measured by the radar and the real features. This deviation and distortion is closely related to the direction of human motion, and also significantly reduces the effect of human motion recognition.

[0004] Therefore, omnidirectional human motion recognition is a key issue for human motion recognition to move towards practical application. Existing omnidirectional motion recognition methods are mainly divided into methods based on multi-base and single-base radar systems. Multi-base radar systems can obtain multi-channel information in different directions through multiple systems in different locations. This deployment can monitor and fuse target information from different angles and ensure that human activities are effectively recorded from a favorable angle. By integrating information from radar echo signals in different directions, the missing information caused by the existence of motion angles in single-base radars can be supplemented, which can provide good classification performance for omnidirectional human motion recognition. However, methods based on multi-base radar systems still have shortcomings in practical applications, including long time consumption, complex deployment, and site occupation. At the same time, calibration between multiple systems needs to be considered in applications, which is not conducive to application promotion. Therefore, methods based on single-base radar systems have received great attention. Among the existing research methods, although most studies based on single-base radar systems consider angle information, they mainly focus on multiple acquisitions, building large data sets covering all directions as much as possible, or designing deep learning models to try to solve the problem. They all require a large amount of radar data as support, and lack in-depth analysis of the specific impact of motion direction on human motion characteristics. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide an omnidirectional human motion feature normalization method based on space-time motion path compensation.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A method for normalizing omnidirectional human motion features based on space-time motion path compensation comprises the following steps:

[0008] Step 1: Obtain the target's multi-transmit and multi-receive radar reflected echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and the distance-time data matrix RT based on human body movements;

[0009] Step 2: Based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body motion, the effective frame is judged by the Doppler envelope, and the Doppler feature D and the distance feature R of the signal changing with time, as well as the target azimuth α are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the distance-angle-time space is constructed;

[0010] Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using a motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected;

[0011] Step 4: Based on the geometric relationship between the motion path and the direction of the radar main beam, the motion direction angle θ corresponding to the human motion feature is obtained, and the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm. corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] (1) The omnidirectional human motion feature normalization method based on space-time motion path compensation proposed in the present invention starts from the distortion of human motion radar echo signal characteristics caused by motion direction, based on a single-base radar system, analyzes and solves the angle sensitivity problem from the perspective of signal processing, restores the feature distortion caused by the motion angle, and compensates the omnidirectional human motion features into radial features, eliminating the influence of the motion angle on the radar measurement signal, improving the representation efficiency of the features, and is superior to traditional feature extraction algorithms in terms of accuracy, efficiency and robustness;

[0014] (2) Compared with the traditional path estimation algorithm which usually only considers the path in the two-dimensional plane, the motion path estimation algorithm of the present invention based on space-time fitting pays attention to and considers the time relationship between feature points, tracks the human body motion path in the distance-angle-time three-dimensional feature point cloud, corrects and utilizes abnormal data, avoids abnormal changes in features, ensures that the path estimation is logically smooth, and improves the efficiency of information utilization;

[0015] (3) The present invention extracts a sequence of feature points extracted from human motion for use in human motion recognition, which meets the discrimination of human motion and effectively removes redundant information. Compared with traditional spectrogram features, the present method has a smaller data volume and higher representation efficiency, which greatly benefits the efficiency and accuracy of subsequent network processing. At the same time, the present invention uses a feature compensation algorithm based on the moving direction to compensate the features of omnidirectional activities into radial features to suppress feature distortion. It can use only the recorded data of radial angles and generalize it to all angles, which greatly reduces the cost of radar data collection and can further improve the economic benefits and operational efficiency of the system.

[0016] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure is a flow chart of the method for normalizing omnidirectional human motion features based on space-time motion path compensation of the present invention.

[0018] Figure 2 It is a schematic diagram of echo data preprocessing of the present invention.

[0019] Figure 3 It is a schematic diagram of feature extraction and three-dimensional feature point cloud construction of the present invention.

[0020] Figure 4 It is a schematic diagram of the motion path estimation effect in an embodiment of the present invention.

[0021] Figure 5 It is the geometric relationship between the motion path of the present invention and the direction of the radar main beam.

[0022] Figure 6 Schematic diagram for comparing the effects of feature compensation algorithms in embodiments of the present invention. DETAILED DESCRIPTION

[0023] A method for normalizing omnidirectional human motion features based on space-time motion path compensation comprises the following steps:

[0024] Step 1: Obtain the target's multi-input multi-output (MIMO) radar reflection echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and distance-time data matrix RT based on human body movements:

[0025] Step 1-1, obtain radar echo signal R rec , for the radar echo signal R rec Rearrange the data:

[0026] According to the number of single chirp sampling points M, the number of antenna channels C, and the number of chirps N, the radar echo signal R rec Rearrange the entire echo signal into a three-dimensional radar echo data matrix R0 of M×N×C, where m=[1,2,...,M] represents fast time sampling points, n=[1,2,...,N] represents slow time sampling points, and c=[1,2,...,C] represents a virtual antenna channel sequence;

[0027] Step 1-2: Based on the radar echo data matrix R0, perform N fft Fast Fourier operation of points to get the data matrix RPC temp , due to RPC temp It is symmetric in the fast time dimension. Take half of it to obtain the distance-time-channel three-dimensional data matrix RPC:

[0028]

[0029] RPC(r,n,c)=RPC temp (N fft / 2:N fft ,n,c)

[0030] Among them, r temp =[1,2,...,N fft ] indicates RPC temp The range gate index, r = [1,2,...,N fft / 2] represents the range gate index of RPC; the symbol "A:B" represents indexing the elements in the range from sequence number A to sequence number B in a specific dimension, N fft The number of Fourier operation points to be set;

[0031] Step 1-3: Perform moving target indicator (MTI) clutter elimination on the range-time-channel three-dimensional data matrix RPC, perform differential subtraction along the slow time dimension, eliminate the stationary target echo with the same amplitude and phase, and obtain the range gate signal data matrix RPC after removing the stationary target. filtOut , select RPC filtOut A single antenna channel is obtained to obtain the RT data matrix;

[0032]

[0033] RT(r,n)=RPC filiOut (r,n,1)

[0034] Step 1-4: Based on the radar echo data matrix R0, select a single virtual antenna channel data matrix, calculate the average value of each fast time sampling point in the slow time dimension as the reference value of the background noise, and obtain the single channel echo data matrix TPC after noise removal. filtOut :

[0035]

[0036] TPC filtOut (m,n)=R0(m,n)-noise(m,n)

[0037] Step 1-5: TPC based on single channel echo data filtOut , select a single fast time sampling point sequence and perform N in the slow time dimension stft The short-time Fourier transform (STFT) of the points is used to obtain the DT data matrix:

[0038] Data dt (n) = TPC filtOut (pIdx,n)

[0039]

[0040] Where m1=[1,2,…,Nstft ],n1=[1,2,…,N / WIN stft ], WIN stft is the window size of the STFT transform.

[0041] Step 2: Based on the Doppler-Time (DT) data matrix DT and the Range-Time (RT) data matrix RT of human motion, the valid frame is judged by the Doppler envelope, and the Doppler feature D and the range feature R of the signal changing with time, as well as the target azimuth α are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the range-angle-time space is constructed:

[0042] Step 2-1: Based on the Doppler-time data matrix DT of human motion, find all the values ​​in each time column that are greater than the set energy threshold P. th The maximum and minimum indexes of the data are taken as the upper and lower envelopes Eu and Ed respectively:

[0043] I(m1,n1)={1|DT(m1,n1)>P th}

[0044] Eu(n1)=maxidx(I(:,n1))

[0045] Ed(n1)=minidx(I(:,n1))

[0046] Among them, minidx(I(:,n1)) and maxidx(I(:,n1)) are the minimum and maximum index values ​​in index I(:,n1);

[0047] Step 2-2: Divide the upper and lower envelopes Eu and Ed into N f sub-segments, according to and As well as the upper envelope minimum offset offset1 and the lower envelope minimum offset offst2, the corresponding frames that meet the conditions are taken as valid frames for subsequent calculations:

[0048]

[0049] Ed i =Ed((i-1)*V+1:V*i)

[0050] Eu i =Eu((i-1)*V+1:V*i)

[0051] where n f =[1,2,...,N f ] represents the divided data frame sequence, N fTo divide the number of data frames, V = N / WIN stft / N f Indicates the length of each sub-segment. When effFrame(n f ) is 1, it is considered a valid frame;

[0052] Step 2-3, select the distance-time data matrix RT corresponding to the valid frame, obtain the corresponding distance by finding the maximum row and the corresponding number of rows, find the corresponding maximum envelope, and obtain the corresponding Doppler feature D and distance feature R:

[0053]

[0054] r norm =20log 10 (|r frame |)

[0055]

[0056] R(index)=(maxrow+256)*ΔR

[0057] D(index)=max(Eu index )

[0058] Where index is the valid frame number obtained from effFrame in step 2-2; N chirp is the number of chirps corresponding to each frame in the data matrix; the symbol argmax represents the index corresponding to the maximum value; ΔR is the range gate width;

[0059] Step 2-4: According to the radar antenna arrangement, select the corresponding virtual channel in TPC to form an azimuth data matrix, perform angle estimation through the Multiple Signal Classification (MUSIC) algorithm, and obtain the target azimuth α of the corresponding frame;

[0060] Data A1 (n,c array )=R0(pIdx,n,C array1 )

[0061] Data A2 (n,c array )=R0(pIdx,n,C array2 )

[0062]

[0063] Rxx=U·S·V

[0064]

[0065] Among them, c array =[1,2,…,z] is the virtual antenna channel sequence; C array1 , C array2 Respectively represent two sets of virtual channel numbers used for azimuth detection; N s To detect the number of targets, Rxx is the array manifold vector of the channel domain data matrix; is the array manifold vector; α is the average of the angles calculated by the two channel domain data matrices, Data Ai Data A1 or Data A2 ;

[0066] Step 2-5: Determine the plane motion sequence point coordinates XY according to the distance feature R and the azimuth angle α, generate the relative time axis coordinates according to the number of valid frames, and construct the three-dimensional feature point cloud XYZ in the distance-angle-time space:

[0067] X(index)=R(index)*sin(α(index))

[0068] Y(index)=R(index)*cos(α(index))

[0069] Z(index)=t start +0.2*index

[0070] XYZ = [X; Y; Z]

[0071] Among them, t start The starting time point for the setting.

[0072] Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using the motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected:

[0073] Step 3-1, initialize and set the neighborhood width gate of the straight line, set the minimum core point ratio to rate, and the maximum offset degree of the point set to bestd;

[0074] Step 3-2: Randomly select any two points A and B in the three-dimensional feature point cloud XYZ and connect them into a straight line L k , calculate the distance from the remaining points in the three-dimensional feature point cloud XYZ to the straight line:

[0075]

[0076] Where A and B represent two points in the kth combination; k = [1, 2, ..., K] represents the index of all point combinations; L krepresents the straight line formed by the k-th combination of points; is the i-th valid point except A and B, x i ,y i 、z i are the coordinates of the point respectively; the symbol ‖·‖ represents the magnitude of the vector;

[0077] Step 3-3: From each point obtained in step 3-2 to the straight line L k Distance, determine whether each point is the straight line L k Neighborhood points of:

[0078] N in ={XYZ i |d i ≤gate}

[0079] N out ={XYZ i |d i >gate}

[0080] Among them, N in represents a set that contains all the i The points with ≤gate are normal points. out It indicates an outlier group;

[0081] Step 3-4: From each point obtained in step 3-2 to the straight line L k By excluding the data points with the largest and second largest distances, the influence of outliers on the fitted straight line is reduced, and the degree of deviation between the point set and the straight line is obtained:

[0082] distance=Σd i -d max1 -d max2

[0083] Among them, d max1 For all i The maximum value among d max2 For all i The second largest value in ;

[0084] Step 3-5: The point set N obtained according to steps 3-3 and 3-4 in and distance, determine the straight line L k Is it the current optimal straight line? If it is the optimal straight line, update the parameters. If it is not the optimal straight line, jump to step 3-2 for iteration until all point set combinations are traversed to obtain the optimal combination point S. b1 and S b2 :

[0085] if|Nin |≥|XYZ|*rate and distance <bestd then

[0086] bestd=distance

[0087] elsecontinue

[0088] Here, the symbol |·| represents the number of point sets;

[0089] Step 3-6: Determine the optimal combination point S based on the result of step 3-5 b1 and S b2 , the outlier correction method based on space-time is used to realize the outlier correction. The plane perpendicular to the time axis and corresponding to the outlier timestamp is selected as the correction plane to realize the correction point. out The space-time dimension is corrected through its time characteristics to obtain the correct point set N c :

[0090] N out =N(x out ,y out ,z out )

[0091] z out =z corr

[0092]

[0093] N corr =N(x corr ,y corr ,z corr )

[0094] N c =N in +N corr

[0095] Among them, N corr is the outlier group N out The corrected point group, y corr 、x corr The horizontal and vertical coordinates of the correction point obtained by solving the collinear point relationship in three-dimensional space;

[0096] Step 3-7: The corrected point set N obtained through step 3-6 c =N(x c ,y c ,z c ), according to N c The XY coordinates of each valid point are used to obtain the azimuth angle α corresponding to each valid point. The optimal fitting combination point S is obtained according to steps 3-5.b1 and S b2 , get the motion path angle β:

[0097]

[0098]

[0099] α=A c ·μ

[0100]

[0101] Where μ is the sign correction factor, which is used to determine the correct sign of the azimuth, A c N c The azimuth of each point in the point group.

[0102] Step 4: Based on the geometric relationship between the motion path and the direction of the radar main beam, the motion direction angle θ corresponding to the human motion feature is obtained, and the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm. corr and R corr , compensating the omnidirectional motion features of the human body into radial features:

[0103] Step 4-1: Based on the corrected point set N obtained in step 3-7 c , calculate N c The distance between each point in the motion sequence and the first point is used to obtain the actual distance span of the target movement, thereby compensating the original distance feature R to the radial distance feature R corr :

[0104]

[0105] Among them, x c1 ,y c1 The point set N c The horizontal and vertical coordinates of the first point in ;

[0106] Step 4-2: According to the azimuth angle α and the motion path direction angle β corresponding to each valid point, the corresponding motion direction angle θ is obtained according to the geometric relationship between the motion path and the radar main beam direction, and the Doppler feature is compensated by θ to obtain the radial Doppler feature D corr :

[0107] θ=α+β

[0108] D corr =D / cos(θ)

[0109] The present invention also provides an omnidirectional human motion feature normalization system based on space-time motion path compensation, comprising the following modules:

[0110] Radar echo signal processing module: used to obtain the target's multi-transmit and multi-receive radar reflected echo signals, pre-process the echo signals, remove clutter information in the signals, and obtain the Doppler-time data matrix DT and distance-time data matrix RT based on human body movements;

[0111] Feature extraction module: It is used to determine the valid frame through the Doppler envelope based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body movements, extract the Doppler feature D and the distance feature R of the signal changing with time, and the target azimuth α, and construct the three-dimensional feature point cloud XYZ in the distance-angle-time space based on R, D and α;

[0112] Path estimation module: used to estimate the motion path based on the three-dimensional feature point cloud XYZ, using the motion path estimation algorithm based on space-time fitting, obtain the motion path direction angle β, and correct the data abnormal points;

[0113] Radial feature restoration module: used to obtain the motion direction angle θ corresponding to the human motion feature based on the geometric relationship between the motion path and the radar main beam direction, and restore the Doppler feature D and distance feature R to the radial feature distribution range D through the feature compensation algorithm corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

[0114] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0115] Step 1: Obtain the target's multi-transmit and multi-receive radar reflected echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and the distance-time data matrix RT based on human body movements;

[0116] Step 2: Based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body motion, the effective frame is judged by the Doppler envelope, and the Doppler feature D and the distance feature r of the signal changing with time, as well as the target azimuth α are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the distance-angle-time space is constructed;

[0117] Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using a motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected;

[0118] Step 4: Based on the geometric relationship between the motion path and the direction of the radar main beam, the motion direction angle θ corresponding to the human motion feature is obtained, and the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm. corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

[0119] A computer storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0120] Step 1: Obtain the target's multi-transmit and multi-receive radar reflected echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and the distance-time data matrix RT based on human body movements;

[0121] Step 2: Based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body motion, the effective frame is judged by the Doppler envelope, and the Doppler feature D and the distance feature R of the signal changing with time, as well as the target azimuth α are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the distance-angle-time space is constructed;

[0122] Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using a motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected;

[0123] Step 4: Based on the geometric relationship between the motion path and the direction of the radar main beam, the motion direction angle θ corresponding to the human motion feature is obtained, and the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm. corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

[0124] Example

[0125] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0126] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0127] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0128] Combination Figure 1 , a method for normalizing omnidirectional human motion features based on space-time motion path compensation, comprising the following steps:

[0129] Step 1: Raise the radar system to 0.85 meters above the ground. Volunteers face the radar and fall in different directions to obtain the target's multi-input multi-output (MIMO) radar reflection echo signal. The echo signal is pre-processed, including data rearrangement, removal of DC components, and MTI processing (removal of static clutter interference). The human motion echo signal with characteristic distortion due to the non-radar radial motion direction is obtained, such as Figure 2 As shown;

[0130] Remove the clutter information in the signal and obtain the Doppler-time data matrix DT and distance-time data matrix RT based on human motion:

[0131] Step 1-1, obtain radar echo signal R rec , for the radar echo signal R rec Rearrange the data:

[0132] Each time the radar platform sends a signal, it is called a chirp signal. The received radar echo signal R recIt is composed of two LVDS channels in a non-interleaved format and stored in a binary file. Each chirp sampling point is a 16-bit unsigned integer and requires two bytes to store. According to the number of single chirp sampling points M = 64, the number of antenna channels C = 2*4 = 8, the number of chirps N = 250*128 = 32000, the radar echo signal R rec Rearrange the entire echo signal into a three-dimensional radar echo data matrix R0 of M×N×C, where m=[1,2,...,64] represents fast time sampling points, n=[1,2,...,32000] represents slow time sampling points, and c=[1,2,...,8] represents a virtual antenna channel sequence;

[0133] Step 1-2: Based on the radar echo data matrix R0, perform N fft =512-point fast Fourier transform FFT operation to obtain the data matrix RPC temp , due to RPC temp It is symmetric in the fast time dimension. Take half of it to obtain the distance-time-channel three-dimensional data matrix RPC:

[0134]

[0135] RPC(r,n,c)=RPC temp (N fft / 2:N fft ,n,c)

[0136] Among them, r temp =[1,2,...,N fft ] indicates RPC temp The range gate index, r = [1,2,...,N fft / 2] represents the range gate index of RPC; the symbol "A:B" represents indexing the elements in the range from sequence number A to sequence number B in a specific dimension, N fft The number of Fourier operation points to be set;

[0137] Step 1-3, perform moving target indicator (MTI) clutter elimination on the distance-time-channel three-dimensional data matrix RPC, perform differential subtraction along the slow time dimension, and eliminate the stationary target echo with the same amplitude and phase. Since the human body is always in motion (breathing, etc.), the amplitude and phase of the reflected signal are different. By subtracting time by time, the stationary object can be removed, and the changed signal is retained to obtain the range gate signal data matrix RPC after removing the stationary target. filtOut , select RPC filtOut A single antenna channel is obtained to obtain the RT data matrix;

[0138]

[0139] RT(r,n)=RPC filtOut (r,n,1)

[0140] Step 1-4: Based on the radar echo data matrix R0, select a single virtual antenna channel data matrix, calculate the average value of each fast time sampling point in the slow time dimension as the reference value of the background noise, and obtain the single channel echo data matrix TPC after noise removal. filtOut :

[0141]

[0142] TPC filtOut (m,n)=R0(m,n)-noise(m,n)

[0143] Step 1-5: TPC based on single channel echo data filtOut , select a single fast time sampling point sequence and perform N in the slow time dimension stft The short-time Fourier transform (STFT) of the points is used to obtain the DT data matrix:

[0144] Data dt (n) = TPC filtout (pIdx,n)

[0145]

[0146] Where m1=[1,2,…,N stft ],n1=[1,2,…,N / WIN stft ], WIN stft is the window size of STFT transformation, pIdx=50 is the number of fast time sampling points in the selected single chirp signal, WIN stft =128 is the sliding window size of STFT transformation; N stft =640 is the number of FFT points for each sliding window of STFT;

[0147] Step 2: Based on the Doppler-time (DT) data matrix DT and the range-time (RT) data matrix RT of human motion, the effective frame is judged by the Doppler envelope, and the Doppler feature D and range feature R of the signal changing with time, as well as the target azimuth α, are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the range-angle-time space is constructed, as shown in Figure 3 As shown;

[0148] Step 2-1: Based on the Doppler-time data matrix DT of human motion, find all the values ​​in each time column that are greater than the set energy threshold P. th From the qualified data, take the maximum index and the minimum index as the upper and lower envelopes Eu and Ed respectively:

[0149] I(m1,n1)={1|DT(m t ,n1)>P th}

[0150] Eu(n1)=maxidx(I(:,n1))

[0151] Ed(n1)=minidx(I(:,n1))

[0152] Among them, minidx(I(:,n1)) and maxidx(I(:,n1)) are the minimum and maximum index values ​​in index I(:,n1);

[0153] Step 2-2: Divide the upper and lower envelopes Eu and Ed into N f = 23 sub-segments, according to and As well as the upper envelope minimum offset offset1 and the lower envelope minimum offset offset2, the corresponding frames that meet the conditions are taken as valid frames for subsequent calculations:

[0154]

[0155]

[0156] Ed i =Ed((i-1)*V+1:V*i)

[0157] Eu i =Eu((i-1)*V+1:V*i)

[0158] where n f =[1,2,...,N f ] represents the divided data frame sequence, N f To divide the number of data frames, V = N / WIN stft / N f Indicates the length of each sub-segment. When effFrame(n f ) is 1, it is considered a valid frame;

[0159] In this embodiment, offset1=100, offset2=-80 are valid frame determination thresholds;

[0160] Step 2-3, select the distance-time data matrix RT corresponding to the valid frame, obtain the corresponding distance by finding the maximum row and the corresponding number of rows, find the corresponding maximum envelope, and obtain the corresponding Doppler feature D and distance feature R:

[0161]

[0162] r norm =20log 10 (|r frame |)

[0163]

[0164] R(index)=(maxrow+256)*ΔR

[0165] D(index)=max(Eu index )

[0166] Where index is the valid frame number obtained from effFrame in step 2-2; N chirp =2560 is the number of chirps corresponding to each frame in the data matrix; the symbol argmax indicates the index corresponding to the maximum value; is the distance door width;

[0167] Step 2-4: According to the radar antenna arrangement, select the corresponding virtual channel in TPC to form an azimuth data matrix, perform angle estimation through the Multiple Signal Classification (MUSIC) algorithm, and obtain the target azimuth α of the corresponding frame;

[0168] Data A1 (n,c array )=R0(pIdx,n,C arra )

[0169] Data A2 (n,c array )=R0(pIdx,n,C array2 )

[0170]

[0171] Rxx=U·S·V

[0172]

[0173] Among them, c array =[1,2,3,4] is the virtual antenna channel sequence; C arra =[1,4,5,8], C array2= [2, 3, 6, 7] respectively represent the two sets of virtual channel numbers used for azimuth detection; N s To detect the number of targets, Rxx is the array manifold vector of the channel domain data matrix; is the array manifold vector; α is the average of the angles calculated by the two channel domain data matrices, Data Ai Data A1 or Data A2 ;

[0174] Step 2-5: Determine the plane motion sequence point coordinates XY according to the distance feature R and the azimuth angle α, generate the relative time axis coordinates according to the number of valid frames, and construct the three-dimensional feature point cloud XYZ in the distance-angle-time space:

[0175] X(index)=R(index)*sin(α(index))

[0176] Y(index)=R(index)*cos(α(index))

[0177] Z(index)=t start +0.2*index

[0178] XYZ = [X; Y; Z]

[0179] Among them, t start =0.4 is the starting time point for setting.

[0180] Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using the motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected, such as Figure 4 As shown:

[0181] Step 3-1, initialize and set the neighborhood width of the line gate = 0.15, set the minimum core point ratio to rate = 0.5, and the maximum offset degree of the point set to bestd = 100;

[0182] Step 3-2: Randomly select any two points A and B in the three-dimensional feature point cloud XYZ and connect them into a straight line L k , calculate the distance from the remaining points in the three-dimensional feature point cloud XYZ to the straight line:

[0183]

[0184]

[0185] Where A and B represent two points in the kth combination; k = [1, 2, ..., K] represents the index of all point combinations; L krepresents the straight line formed by the k-th combination of points; is the i-th valid point except A and B, x i ,y i 、z i are the coordinates of the point respectively; the symbol ‖·‖ represents the magnitude of the vector;

[0186] Step 3-3: From each point obtained in step 3-2 to the straight line L k Distance, determine whether each point is the straight line L k Neighborhood points of:

[0187] N in ={XYZ i |d i ≤gate}

[0188] N out ={XYZ i |d i >gate}

[0189] Among them, N in represents a set that contains all the i The points with ≤gate are normal points. out It indicates an outlier group;

[0190] Step 3-4: From each point obtained in step 3-2 to the straight line L k By excluding the data points with the largest and second largest distances, the influence of outliers on the fitted straight line is reduced, and the degree of deviation between the point set and the straight line is obtained:

[0191] distance=∑d i -d max1 -d ma

[0192] Among them, d max1 For all i The maximum value among d max2 For all i The second largest value in ;

[0193] Step 3-5: The point set N obtained according to steps 3-3 and 3-4 in and distance, determine the straight line L k Is it the current optimal straight line? If it is the optimal straight line, update the parameters. If it is not the optimal straight line, jump to step 3-2 for iteration until all point set combinations are traversed to obtain the optimal combination point S. b1 and S b2 :

[0194] if|Nin |≥|XYZ|*rateanddistance <bestdthen

[0195] bestd=distance

[0196] elsecontinue

[0197] The symbol |·| represents the number (i.e., length) of the point set;

[0198] Step 3-6: Determine the optimal combination point S based on the result of step 3-5 b1 and S b2 , the outlier correction method based on space-time is used to realize the outlier correction. The plane perpendicular to the time axis and corresponding to the outlier timestamp is selected as the correction plane to realize the correction point. out The space-time dimension is corrected through its time characteristics to obtain the correct point set N c :

[0199] N out =N(x out ,y out ,z out )

[0200] z out =z corr

[0201]

[0202] N corr =N(x corr ,y corr ,z corr )

[0203] N c =N in +N corr

[0204] Among them, N corr is the outlier group N out The corrected point group, y corr 、x corr The horizontal and vertical coordinates of the correction point obtained by solving the collinear point relationship in three-dimensional space;

[0205] Step 3-7: The corrected point set N obtained through step 3-6 c =N(x c ,y c ,z c ), according to N cThe XY coordinates of each valid point are used to obtain the azimuth angle α corresponding to each valid point. The optimal fitting combination point S is obtained according to steps 3-5. b1 and S b2 , get the motion path angle β:

[0206]

[0207] α=A c ·μ

[0208]

[0209] Where μ is the sign correction factor, which is used to determine the correct sign of the azimuth, A c N c The azimuth of each point in the point group.

[0210] Step 4: Based on the geometric relationship between the motion path and the radar main beam direction, obtain the motion direction angle θ corresponding to the human motion feature, as follows: Figure 5 As shown; the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm corr and R corr , compensating the omnidirectional motion features of the human body into radial features, such as Figure 6 As shown:

[0211] Step 4-1, set the distance of the action starting point R0 = 3, so that the initial distance is a fixed value, highlighting the characteristics of the distance feature changing according to different actions, based on the corrected point set N obtained in step 3-7 c , calculate N c The distance between each point in the motion sequence and the first point is used to obtain the actual distance span of the target movement, thereby compensating the original distance feature R to the radial distance feature R corr :

[0212]

[0213]

[0214] Among them, x c1 ,y c1 The point set N c The horizontal and vertical coordinates of the first point in ;

[0215] Step 4-2: According to the azimuth angle α and the motion path direction angle β corresponding to each valid point, the corresponding motion direction angle θ is obtained according to the geometric relationship between the motion path and the radar main beam direction, and the Doppler feature is compensated by θ to obtain the radial Doppler feature D corr :

[0216] θ=α+β

[0217] D corr =D / cos(θ)

[0218] pass Figure 4 and Figure 6 It can be seen that after the omnidirectional human motion feature normalization method based on space-time motion path compensation proposed by the present invention, the human motion features (Doppler and distance features) restored by the feature compensation algorithm are more consistent with the radial motion feature distribution. Figure 6 The blue in the middle is the feature before compensation, the orange is the feature after compensation, and the yellow is the feature measured in the radial case. It can be seen that when the motion angle is not the radar main beam, the Doppler and distance features of the human body motion before compensation are distorted, and the Doppler is attenuated to half of the true value, while the Doppler and distance features of the human body motion after compensation return to the normal range. Figure 4 The figure is a schematic diagram of motion path estimation. In the figure, blue points are normal points, purple points are abnormal points, red points are correction points of abnormal points, and the green straight line is the obtained fitting straight line. It can be proved from the figure that the motion path estimation based on space-time fitting can correctly extract the angle of human motion path, and at the same time, the timing characteristics of human motion can be used to correct the abnormal points in space and logic.

[0219] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for normalizing omnidirectional human motion features based on space-time motion path compensation, characterized in that: The following steps are involved: Step 1: Obtain the target's multi-transmit and multi-receive radar reflected echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and the distance-time data matrix RT based on human body movements; Step 2: Based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body motion, the effective frame is judged by the Doppler envelope, and the Doppler feature D and the distance feature R of the signal changing with time, as well as the target azimuth α are extracted. Based on R, D and α, a three-dimensional feature point cloud XYZ in the distance-angle-time space is constructed; Step 3: Based on the three-dimensional feature point cloud XYZ, the motion path is estimated using a motion path estimation algorithm based on space-time fitting, the motion path direction angle β is obtained, and the data abnormal points are corrected; Step 4: Based on the geometric relationship between the motion path and the direction of the radar main beam, the motion direction angle θ corresponding to the human motion feature is obtained, and the Doppler feature D and the distance feature R are restored to the radial feature distribution range D through the feature compensation algorithm. corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

2. The omnidirectional human motion feature normalization method based on space-time motion path compensation according to claim 1, characterized in that: The radar echo signal is preprocessed in step 1, specifically: Step 1-1, obtain radar echo signal R rec , for the radar echo signal R rec Rearrange the data: According to the number of single chirp sampling points M, the number of antenna channels C, and the number of chirps N, the radar echo signal R rec Rearrange the entire echo signal into a three-dimensional radar echo data matrix R0 of M×N×C, where m=[1, 2, ..., M] represents fast time sampling points, n=[1, 2, ..., N] represents slow time sampling points, and c=[1, 2, ..., C] represents a virtual antenna channel sequence; Step 1-2: Based on the radar echo data matrix R0, perform Nff along the fast time dimension t Fast Fourier operation of points to get the data matrix RPC temp , due to RPC temp It is symmetric in the fast time dimension. Take half of it to obtain the distance-time-channel three-dimensional data matrix RPC: RPC(r,n,c)=RPC temp (N fft / 2:N fft ,n,c) Among them, r temp =[1, 2, ..., N fft ] indicates RPC temp The range gate index, r = [1, 2, ..., N fft / 2] represents the range gate index of RPC; the symbol "A:B" represents indexing the elements in the range from sequence number A to sequence number B in a specific dimension, N fft The number of Fourier operation points to be set; Step 1-3: Perform moving target display clutter elimination on the range-time-channel three-dimensional data matrix RPC, perform differential subtraction along the slow time dimension, eliminate the stationary target echo with the same amplitude and phase, and obtain the range gate signal data matrix RPC after removing the stationary target filtOut , select RPC filtOut A single antenna channel is obtained to obtain the RT data matrix; RT(r,n)=RPC filtOut (r,n,1) Step 1-4: Based on the radar echo data matrix R0, select a single virtual antenna channel data matrix, calculate the average value of each fast time sampling point in the slow time dimension as the reference value of the background noise, and obtain the single channel echo data matrix TPC after noise removal. filtOut : TPC filtOut (m,n)=R0(m,n)-noise(m,n) Step 1-5: TPC based on single channel echo data filtOut , select a single fast time sampling point sequence and perform N in the slow time dimension stft Short-time Fourier transform of the points gives the DT data matrix: Data dt (n)=TPC filtOut (pIdx,n) Where m1 = [1, 2, ..., N stft ],n1=[1,2,...,N / WIN stft ], WIN stft is the window size of the STFT transform.

3. The omnidirectional human motion feature normalization method based on space-time motion path compensation according to claim 2 is characterized in that: The step 2 of constructing a three-dimensional feature point cloud in the distance-angle-time space is specifically as follows: Step 2-1: Based on the Doppler-time data matrix DT of human motion, find all the values ​​in each time column that are greater than the set energy threshold P. th The maximum and minimum indexes of the data are taken as the upper and lower envelopes Eu and Ed respectively: I(m1,n1)={1|DT(m1,n1)>P th } Eu(n1)=maxidx(I(:,n1)) Ed(n1)=minidx(I(:,n1)) Among them, minidx(I(:,n1)) and maxidx(I(:,n1)) are the minimum and maximum index values ​​in index I(:,n1); Step 2-2, divide the upper and lower envelopes Eu and Ed into Nf sub-segments, according to and As well as the upper envelope minimum offset offset1 and the lower envelope minimum offset offset2, the corresponding frames that meet the conditions are taken as valid frames for subsequent calculations: Ed i =Ed((i-1)*V+1:V*i) I i =I((i-1)*V+1:V*i) Where n f =[1, 2, ..., Nf] represents the divided data frame sequence, N f To divide the number of data frames, V = N / WIN stft / N f Indicates the length of each sub-segment. When effFrame(n f ) is 1, it is considered to be a valid frame; Step 2-3, select the distance-time data matrix RT corresponding to the valid frame, obtain the corresponding distance by finding the maximum row and the corresponding number of rows, find the corresponding maximum envelope, and obtain the corresponding Doppler feature D and distance feature R: r norm =20log 10 (|r frame |) R(index)=(maxrow+256)*ΔR D(index)=max(Eu index ) Where index is the valid frame number obtained from effFrame in step 2-2; N chirp is the number of chirps corresponding to each frame in the data matrix; the symbol argmax represents the index corresponding to the maximum value; ΔR is the range gate width; Step 2-4: According to the radar antenna arrangement, select the corresponding virtual channel in TPC to form an azimuth data matrix, perform angle estimation through a multiple signal classification algorithm, and obtain the target azimuth α of the corresponding frame; Data A1 (n,c array )=R0(pIdx,n,C array1 ) Data A2 (n,c array )=R0(pIdx,n,C array ) Rxx=U·S·V Among them, c array =[1, 2, ..., z] is the virtual antenna channel sequence; C array , C array2 Respectively represent two sets of virtual channel numbers used for azimuth detection; N s is the number of detected targets, Rxx is the array manifold vector of the channel domain data matrix; a(θ) is the array manifold vector; α is the average of the angles calculated by the two channel domain data matrices, Data Ai Data A1 or Data A2 ; Step 2-5: Determine the plane motion sequence point coordinates XY according to the distance feature R and the azimuth angle α, generate the relative time axis coordinates according to the number of valid frames, and construct the three-dimensional feature point cloud XYZ in the distance-angle-time space: X(index)=R(index)*sin(α(index)) Y(index)=R(index)*cOs(α(index)) Z(index)=t start +0.2*index XYZ = [X; Y; Z] Among them, t start The starting time point for the setting.

4. The omnidirectional human motion feature normalization method based on space-time motion path compensation according to claim 1, characterized in that: The step 3 of obtaining the motion path direction angle β is specifically: Step 3-1, initialize and set the neighborhood width gate of the straight line, set the minimum core point ratio to rate, and the maximum offset degree of the point set to bestd; Step 3-2: Randomly select any two points A and B in the three-dimensional feature point cloud XYZ and connect them into a straight line L k , calculate the distance from the remaining points in the three-dimensional feature point cloud XYZ to the straight line: Where A and B represent two points in the kth combination; k = [1, 2, ..., K] represents the index of all point combinations; L k represents the straight line formed by the k-th combination of points; is the i-th valid point except A and B, x i ,y i 、z i are the coordinates of the point respectively; the symbol ||·|| represents the modulus of the vector; Step 3-3: From each point obtained in step 3-2 to the straight line L k The distance between each point and the straight line L k Neighborhood points of: N in ={XYZ i |d i ≤gate} N out ={XYZ i |d i >gate} Among them, N in represents a set that contains all the i The points with ≤gate are normal points. out It indicates an outlier group; Step 3-4: From each point obtained in step 3-2 to the straight line L k By excluding the data points with the largest and second largest distances, the influence of outliers on the fitted straight line is reduced, and the degree of deviation between the point set and the straight line is obtained: distance=∑d i -d max1 -d max2 Among them, d max1 For all i The maximum value among d max2 For all i The second largest value in ; Step 3-5: The point set N obtained according to steps 3-3 and 3-4 in and distance, determine the straight line L k Is it the current optimal straight line? If it is the optimal straight line, update the parameters. If it is not the optimal straight line, jump to step 3-2 for iteration until all point set combinations are traversed to obtain the optimal combination point S. b1 and S b2 : if|N in |≥|XYZ|*rate and distance<bestd then bestd=distance elsecontinue Among them, the symbol |·| represents the number of point sets; Step 3-6: Determine the optimal combination point S based on the result of step 3-5 b1 and S b2 , the outlier correction method based on space-time is used to realize the outlier correction. The plane perpendicular to the time axis and corresponding to the outlier timestamp is selected as the correction plane to realize the correction point. out The space-time dimension is corrected through its time characteristics to obtain the correct point set N c : N out =N(x out y out ,z out ) With out =from corr N corr =N(x corr y corr ,z corr ) N c =N in +N corr Among them, N corr is the outlier group N out The corrected point group, y corr 、x corr The horizontal and vertical coordinates of the correction point obtained by solving the collinear point relationship in three-dimensional space; Step 3-7: The corrected point set N obtained through step 3-6 c =N(x c ,y c , z c ), according to N c The XY coordinates of each valid point are used to obtain the azimuth angle α corresponding to each valid point. The optimal fitting combination point S is obtained according to steps 3-5. b1 and S b2 , get the motion path angle β: α=A c ·m Where μ is the sign correction factor, which is used to determine the correct sign of the azimuth, A c N c The azimuth of each point in the point group.

5. The omnidirectional human motion feature normalization method based on space-time motion path compensation according to claim 4 is characterized in that: In step 4, the Doppler feature D and the distance feature R are restored to the radial feature distribution range D by the feature compensation algorithm. corr and R corr , the omnidirectional motion characteristics of the human body are compensated into radial characteristics, specifically: Step 4-1: Based on the corrected point set N obtained in step 3-7 c , calculate N c The distance between each point in the motion sequence and the first point is used to obtain the actual distance span of the target movement, thereby compensating the original distance feature R to the radial distance feature R corr : Among them, x c1 ,y c1 The point set N c The horizontal and vertical coordinates of the first point in ; Step 4-2: According to the azimuth angle α and the motion path direction angle β corresponding to each valid point, the corresponding motion direction angle θ is obtained according to the geometric relationship between the motion path and the radar main beam direction, and the Doppler feature is compensated by θ to obtain the radial Doppler feature D corr : θ=α+B D corr = D / cos(θ).

6. An omnidirectional human motion feature normalization system based on space-time motion path compensation, characterized in that: Includes the following modules: Radar echo signal processing module; It is used to obtain the target's multi-transmit and multi-receive radar reflected echo signal, pre-process the echo signal, remove the clutter information in the signal, and obtain the Doppler-time data matrix DT and distance-time data matrix RT based on human body movements; Feature extraction module: It is used to determine the valid frame through the Doppler envelope based on the Doppler-time data matrix DT and the distance-time data matrix RT of human body movements, extract the Doppler feature D and the distance feature R of the signal changing with time, and the target azimuth α, and construct the three-dimensional feature point cloud XYZ in the distance-angle-time space based on R, D and α; Path estimation module: used to estimate the motion path based on the three-dimensional feature point cloud XYZ, using the motion path estimation algorithm based on space-time fitting, obtain the motion path direction angle β, and correct the data abnormal points; Radial feature restoration module: used to obtain the motion direction angle θ corresponding to the human motion feature based on the geometric relationship between the motion path and the radar main beam direction, and restore the Doppler feature D and distance feature R to the radial feature distribution range D through the feature compensation algorithm corr and R corr , compensating the omnidirectional motion characteristics of the human body into radial characteristics.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to claims 1 to 5 are implemented.

8. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claims 1 to 5 are implemented.

Images