A human body posture recognition method based on wireless network

Abstract

The present invention relates to a method for human posture recognition based on wireless network. Since the sensor is cumbersome to wear, the camera is easily affected by problems such as illumination, obstacles, monitoring blind spots and privacy leakage, so this paper proposes an effective supplementary technology for human posture recognition based on segment posture eigenvalue (Channel State Information-Segment Posture Eigenvalue, CSI-SPE). The method collects human posture information (running, jumping, sitting, standing, going up and down stairs) through a WIFI routing device, uses a Butterworth filter and a Hampel algorithm to perform denoising and outlier processing, then constructs segment posture eigenvalues, and then passes them into a support vector machine (Support Vector Machine, SVM) for recognition. Experiments have shown that the comprehensive recognition accuracy of the method based on CSI-SPE is 98%, and the experimental results show that the CSI-SPE method has high robustness and recognition accuracy.

Description

Technical Field

[0001] The invention belongs to the fields of wireless networks, human body posture recognition or machine learning. Background Art

[0002] At present, the research on human posture recognition in my country has become increasingly close to daily life, and the research results of human posture recognition are widely used in the fields of action recognition, human-computer interaction and clothing analysis. In the existing research on human posture recognition, reference [1] designed a WLAN-based personnel perception and intrusion detection system, which utilized the similarity difference of different CSI phase matrices. However, this method of using eigenvalue mutation has the defect of being easily affected by the surrounding environment and unstable. Reference [2] extracts the maximum and second largest eigenvalues ​​of the correlation matrix of CSI amplitude and phase to form a four-dimensional feature vector, but whether the recognition rate will increase with the increase of feature vectors remains to be verified. Reference [3] proposes to extract the subcarrier feature variance corresponding to different postures, and then use the SRC algorithm to classify human behavior, but the stability of this method is poor, and the detection effect is not ideal when the human activity is not intense. Reference [4] omits the data preprocessing process to reduce costs. It extracts the mean and variance of the amplitude of the original CSI data as features and uses SVM for personnel perception. However, there is interference information in the original CSI data, so the effect of this method is not ideal. The current problems and challenges in human posture recognition are:

[0003] (1) As people’s requirements for action recognition continue to increase, how to achieve low cost, fine granularity and high precision.

[0004] (2) In indoor environments, how to reduce the multipath effect and how to deal with various environmental noises brought by WIFI signals and handle complex application environments. Summary of the invention

[0005] 1. Technical problems that the present invention needs and can solve:

[0006] The present invention provides a human posture recognition method based on a wireless network, which can overcome the influence of environmental noise. Compared with other recognition methods, it has the advantages of simple feature extraction and accurate feature recognition rate. The granularity is fine and the multipath effect can be reduced, thereby improving the accuracy of human posture recognition, making human posture estimation more efficient and less costly.

[0007] 2. Specific technical solutions of the present invention:

[0008] 1. Collect raw data

[0009] 2. Perform matrix transformation on the collected original data set.

[0010] 3. Clean and process the data set after matrix transformation.

[0011] 4. Perform feature extraction on the cleaned and processed dataset.

[0012] 5. Perform human posture recognition on the extracted features to obtain posture recognition results.

[0013] A method for human body posture recognition based on wireless network, characterized in that:

[0014] 1) Collect the original data set:

[0015] (2) Perform matrix transformation on the collected original data set;

[0016] Extract subcarriers through MATLAB software, where data_f, data_s, and data_t represent the subcarrier data of the first antenna, the second antenna, and the third antenna;

[0017] F(x)=x T (1)

[0018] Where x represents the incoming subcarrier data, T represents transposition, and data_f, data_s, and data_t are passed in for matrix transposition; the three sets of data obtained after transposition are flattened and reduced to (1, n) dimensional data, where n represents the number of columns of data; then the stack function of the numpy library is used to splice them into a (3, n) matrix, where n represents the number of columns of data; the user number, user posture, and timestamp are spliced ​​into the matrix to construct (6, n), and then formula 1 is used to construct the original data set (r, 6), where r is the number of rows;

[0019]

[0020] Normalize the data to convert it to data between 0 and 1, and use formula (2), where x is the input matrix data, MIN is the minimum value of each column of data, and MAX is the maximum value of each column of data;

[0021] (3) Clean and process the data set after matrix transformation:

[0022] A low-pass filter is designed. During the acquisition process, the filter designed in this paper sends a frequency of 50Hz. After checking, the environmental noise is 250Hz to 500Hz; therefore, the low-pass filter is designed with a cutoff frequency f p =30Hz, maximum passband attenuation α p =3dB, stopband start frequency f s =250Hz, stop band minimum attenuation α s =30dB;

[0023] The normalized frequency λ is calculated by formula (3): p is the cutoff frequency, λ s is the stopband start frequency, where Ω p =2πf p ,Ω p Passband cutoff frequency, Ω s =2πf s ,Ω s Passband start frequency;

[0024]

[0025] Calculate the Butterworth filter order N and parameters C

[0026]

[0027] The calculation of formula (4) shows that parameter C is 1, and α is the intermediate variable;

[0028]

[0029]

[0030] The calculation shows that the filter order N is 5th order;

[0031] According to the butter transfer function formula (7) in the software, the parameters B and A are calculated.

[0032] [B,A]=butter(N,λ s ,λ p ,C) (7)

[0033] y is the output signal sequence, where x is the input signal, that is, the three subcarrier sequences of the data set, where B is the numerator of the Butterworth filter, A is the denominator of the Butterworth filter, and filter is the filter function in the software;

[0034] y=filter(B,A,x) (8)

[0035] The filtered signal is output in formula (8), and the input vector (three subcarriers) is subjected to outlier removal. The Hampel function calculates the median of the window consisting of the sample and its surrounding six samples, three on each side. The median absolute value is used to estimate the standard deviation of the median of each sample pair to replace the outliers, and a cleaned and processed data set is obtained. The filter and Hampel function designed in this paper for this type of task improve the cleanliness of the data set compared with other methods.

[0036] (4) Feature extraction of the cleaned and processed data set:

[0037] Construct input feature values ​​and create a time window of 90 times 50ms, that is, 4.5 seconds. Move forward 45 records each time, using a semi-overlapping method. Then create input data. Each group contains 90 consecutive data of three subcarriers, which is called a segment posture feature value. The user activities in the 90 data are counted, and the one with the most occurrences is the label name of this group of data.

[0038] (5) Perform human posture recognition on the extracted features to obtain posture recognition results

[0039] The data set is divided into training set, validation set and test set; the specific SVM algorithm used in this article is:

[0040] The training set data T is expressed as:

[0041] T={(x1,y1),(x2,y2),...,(xn,yn)} (9)

[0042] xi belongs to a multidimensional space, and the value of y is one of 0, 1, 2, 3, 4, and n = 1, 2, ..., n; n is an integer greater than or equal to 1; xi is the i-th feature vector, yi is the label name (running 0, sitting 1, standing 2, going up and down stairs 3, walking 4);

[0043] Construct and solve a convex quadratic programming problem, where α i ,α j Indicates the corresponding y i y j The coefficient, α * represents the optimal solution. The physical meaning of α is to find an optimal solution that satisfies formula (10). By continuously optimizing α, the optimal hyperplane is found, thereby obtaining the boundary of the classification problem faced by SVM:

[0044]

[0045]

[0046] 0<α i <M,i=1,2,3...n (12)

[0047] Formula 10, 11, 12 to obtain the optimal solution

[0048]

[0049] in Represents the function y i The corresponding independent variable x i There are n optimal coefficients in total. Formula 13 calculates the coefficient b * , b* Represents the function y i There are n optimal solutions for the corresponding constants in ;

[0050]

[0051] The calculated α * , b * Substitute into formula (15)

[0052]

[0053] Use Gaussian kernel function, penalty coefficient M = 8, to solve α * Using the gradient descent method, the Gaussian kernel function K(x i ,x j ) as shown in the formula; the physical meaning of K is the function that refers to the following equation, with two independent variables; among them, σ 2 is x i ,x j The standard deviation of

[0054]

[0055] Solve f(x) and compare the result with the true value label. When f(x) = 0, it proves that the matching training result is running. When f(x) = 1, it proves that the matching training result is sitting. When f(x) = 2, it proves that the matching training result is standing. When f(x) = 3, it proves that the matching training result is going up and down stairs. When f(x) = 4, it proves that the matching training result is walking.

[0056] 3. The effects that can be achieved by the present invention:

[0057] From an economic perspective: This method greatly reduces costs compared to traditional sensor systems and computer vision systems. Especially with the popularization of wireless networks and 5G technology, this technology can penetrate into every household. It is simple to set up, and the required equipment is basically available in ordinary households.

[0058] From a social perspective: Compared with traditional sensor systems and computer vision systems, this method greatly protects personal privacy. In computer vision systems, video streams carry personal facial features, body features, and environmental features. If the video stream is leaked, all personal feature information will be exposed to the outside world, which poses a great risk. However, this method only needs to collect personal postures through wireless networks, and does not need to collect facial features, environmental features, etc. It has strong security.

[0059] From a technical perspective: Compared with traditional sensor systems and computer vision systems, this method is simple to build and can be constructed by users who are familiar with Linux and Windows systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Experimental environment

[0061] Figure 2 Experimental Procedure

[0062] Figure 3 Confusion Matrix DETAILED DESCRIPTION

[0063] The experimental environment is an ordinary empty laboratory, where the distance between the transmitter and the receiver is 1.5 meters. The experimenter only needs to perform gesture expression between the transmitter and the receiver.

[0064] Figure 3 The image shows the recognition results of five postures: running 0, sitting 1, standing 2, going up and down stairs 3, and walking 4, which reflects a very accurate recognition result.

[0065] Table 1 Accuracy of different classifiers

[0066]

[0067] Table 1 shows the accuracy results obtained using different classifiers, among which SVC has the best classification effect. It is also the classifier used in this paper.

[0068] Table 2 Comparison of different methods

[0069]

[0070]

[0071] Table 2 shows the accuracy of different experiments, among which the CSI-SPE method based on the segment pose eigenvalues ​​in this paper has the highest accuracy, which is significantly better than other methods.

[0072] Table 3 Dataset format after cleaning and processing

[0073]

[0074] Table 3 shows the data set format after implementation of solution (2) of the present invention.

[0075] 1) Collect the original data set:

[0076] 1. Prepare a host with Linux Ubuntu installed, an Intel 5300 network card, three receiving antennas, and a router that supports 802.11n protocol. Install CSI TOOLS on the host, and then configure the router to AP mode. Then use the ping command from the host to send packets, and receive data through the antenna and store it in a file.

[0077] (2) Perform matrix transformation on the collected original data set.

[0078] The subcarriers are extracted through MATLAB software, where data_f, data_s, and data_t represent the subcarrier data of the first antenna, the second antenna, and the third antenna.

[0079] F(x)=x T (1)

[0080] Where x represents the incoming subcarrier data, T represents transposition, and data_f, data_s, and data_t are passed in for matrix transposition. The three sets of data obtained after transposition are flattened and reduced to (1, n) dimensional data, where n represents the number of columns of data. Then, through the stack function of the numpy library, it is spliced ​​into a (3, n) matrix, where n represents the number of columns of data. Further, the user number (UserId), user posture (Activity), and timestamp (Timestamp) are spliced ​​into the matrix to construct (6, n), and then formula 1 is used to construct the original data set (n, 6), where n is the number of rows, and n is 30,000 in this data set.

[0081]

[0082] Furthermore, the data is normalized and converted to data between 0 and 1, using Formula 2, where x is the input matrix data, MIN is the minimum value of each column of data, and MAX is the maximum value of each column of data.

[0083] (3) Clean and process the data set after matrix transformation:

[0084] Furthermore, the data obtained in (2) is cleaned and processed. In this paper, a low-pass filter is designed. During the acquisition process, the filter designed in this paper sends a frequency of 50Hz. After checking, the environmental noise is 250Hz to 500Hz. Therefore, the low-pass filter is designed with a cutoff frequency f p =30Hz, maximum passband attenuation α p =3dB, stopband start frequency f s =250Hz, stop band minimum attenuation α s =30dB.

[0085] Furthermore, the normalized frequency λ is calculated by formula 3 p is the cutoff frequency, λ s is the stopband start frequency, where Ω p =2πf p ,Ω p Passband cutoff frequency, Ωs =2πf s ,Ω s Passband start frequency.

[0086]

[0087] Further calculation of Butterworth filter order N and parameter C

[0088]

[0089] The calculation of formula 4 shows that parameter C is 1, and α is an intermediate variable.

[0090]

[0091]

[0092] The filter order N is calculated to be 5th order by using formula 5 and formula 6.

[0093] Further, the butter transfer function (Formula 7) in the software is used to calculate the parameters B and A.

[0094] [B,A]=butter(N,λ s ,λ p ,C) (7)

[0095] Further, y is the output signal sequence, where x is the input signal, that is, the three subcarrier sequences of the data set, where B is the numerator of the Butterworth filter, A is the denominator of the Butterworth filter, and filter is the filter function in the software.

[0096] y=filter(B,A,x) (8)

[0097] Furthermore, the filtered signal is output in formula 8, and the input vector (three subcarriers) is subjected to outlier removal. The Hampel function calculates the median of a window consisting of a sample and six samples around it, three on each side. The standard deviation of the median of each sample pair is estimated using the absolute value of the median to replace the outliers, and a cleaned and processed data set is obtained. The filter and Hampel function designed in this paper for this type of task improve the cleanliness of the data set compared to other methods.

[0098] (4) Feature extraction of the cleaned and processed data set:

[0099] Further, based on (3), we construct the input feature value and create a time window of 90 times 50ms, that is, 4.5 seconds, and advance 45 records each time, using a semi-overlapping method. Then we create input data, each group of which contains 90 consecutive data of three subcarriers, called a segment posture feature value (Channel State Information-Segment PostureEigenvalue, CSI-SPE), where the user activities in the 90 data are counted, and the one with the most occurrences is the label name of this group of data.

[0100] (5) Perform human posture recognition on the extracted features to obtain posture recognition results

[0101] Furthermore, based on (4), the data set is divided into training set, validation set, and test set according to 7:2:1. The specific SVM algorithm used in this paper is:

[0102] The training set data T is expressed as:

[0103] T={(x1,y1),(x2,y2),...,(xn,yn)} (9)

[0104] In formula 9, xi belongs to an n-dimensional space, and the value of y is one of 0, 1, 2, 3, 4, and n = 1, 2, ..., n; n is an integer greater than or equal to 1; xi is the i-th feature vector, and yi is the label name (running 0, sitting 1, standing 2, going up and down stairs 3, walking 4). In the present invention, the composition of the feature vector is the segment posture feature value (Channel State Information-SegmentPosture Eigenvalue, CSI-SPE)

[0105] 2. Select an appropriate kernel function and penalty parameter M>0, construct and solve a convex quadratic programming problem, where α i ,α j Indicates the corresponding y i y j The coefficient, α * represents the optimal solution. The physical meaning of α is to find an optimal solution that satisfies Formula 10. By continuously optimizing α, the optimal hyperplane is found, thereby obtaining the boundary of the classification problem faced by SVM:

[0106]

[0107]

[0108] 0<α i <M,i=1,2,3...n (12)

[0109] Formulas 10, 11, and 12 are used to find the optimal solution (generally using the gradient descent method)

[0110]

[0111] in Represents the function y i The corresponding independent variable x i There are n optimal coefficients in total. Formula 13 calculates the coefficient b * , b * Represents the function y i There are n optimal solutions for the corresponding constants in .

[0112]

[0113] The α calculated by formula 13 and 14 * , b * Substitute into formula 15

[0114]

[0115] The present invention uses a Gaussian kernel function with a penalty coefficient of M = 8 to solve α * Using the gradient descent method, the Gaussian kernel function K(x i ,x j ) as shown in Formula 16. The physical meaning of K is that the function refers to the following equation, with two independent variables. 2 is x i ,x j The standard deviation of .

[0116]

[0117] Solve f(x) and compare the result with the true value label. When f(x) = 0, it proves that the matching training result is running. When f(x) = 1, it proves that the matching training result is sitting. When f(x) = 2, it proves that the matching training result is standing. When f(x) = 3, it proves that the matching training result is going up and down stairs. When f(x) = 4, it proves that the matching training result is walking.

[0118] (6) Verify the results:

[0119] This paper uses confusion matrix and secondary indicators of confusion matrix for verification. Compared with other technologies referenced in this paper, they have obvious advantages. The filter and feature extraction method designed in this paper are better than other methods. After many experiments, the accuracy of the results reached about 98%.

[0120] References:

[0121] [1].LI L

[0122] [2].QIAN K,WU C,YANG Z,et al.Enabling Contactless Detection of MovingHumans with Dynamic Speeds Using CSI[J].ACM Transactions on EmbeddedComputing Systems,2018,17(2):1-18.

[0123] [3].XIAO L,PAN H.Human motion recognition system based on Wi-Fisignal[J].Journal of Beijing University of Posts and Telecommunications,2018,41(3):119-124.

[0124] [4].WANG T,YANG DD,ZHANG SQ,et al.Wi-Alarm:Low-Cost PassiveIntrusion DetectionUsing Wi-Fi[J].Sensors,2019,19(10):2335.

[0125] [5]. Hao Zhanjun, Zhang Daiyang, Dang Xiaochao, Duan Yu. A contactless personnel action recognition method based on CSI[J / OL]. Computer Engineering: 1-13[2021-03-07]. https: / / doi.org / 10.19678 / j.issn.1000-3428.0057612.

[0126] [6]. Yin Genglei, Ding Wenchao, Zhang Junbao. CSI action recognition based on temporal convolutional network[J]. Journal of Zhongyuan University of Technology, 2020, 31(05): 59-65.

[0127] [7]. Guo Aying, Xu Zhimeng, Chen Liangqin. A human motion recognition method based on WiFi channel state information[J]. Journal of Sensor Technology, 2019, 32(11): 1688-1693.

Claims

1. A method for human posture recognition based on a wireless network, characterized in that: 1) Collect the original data set: (2) Perform matrix transformation on the collected original data set; Extract subcarriers through MATLAB software, where data_f, data_s, and data_t represent the subcarrier data of the first antenna, the second antenna, and the third antenna; F(x)=x T (1) Where x represents the incoming subcarrier data, T represents transposition, and data_f, data_s, and data_t are passed in for matrix transposition; the three sets of data obtained after transposition are flattened and reduced to (1, n) dimensional data, where n represents the number of columns of data; then, the stack function of the numpy library is used to splice them into a (3, n) matrix, where n represents the number of columns of data; the user number, user posture, and timestamp are spliced ​​into the matrix to construct (6, n), and then formula 1 is used to construct the original data set (r, 6), where r is the number of rows; Normalize the data to convert it to data between 0 and 1, and use formula (2), where x is the input matrix data, MIN is the minimum value of each column of data, and MAX is the maximum value of each column of data; (3) Clean and process the data set after matrix transformation: A low-pass filter was designed. During the acquisition process, the designed filter transmission frequency was 50Hz. After checking, the environmental noise was 250Hz to 500Hz; therefore, the low-pass filter was designed with a cutoff frequency f p =30Hz, maximum passband attenuation α p =3dB, stopband start frequency f s =250Hz, stop band minimum attenuation α s =30dB; The normalized frequency λ is calculated by formula (3): p is the cutoff frequency, λ s is the stopband start frequency, where Ω p =2πf p ,Ω p Passband cutoff frequency, Ω s =2πf s ,Ω s Passband start frequency; Calculate the Butterworth filter order N and parameter C The calculation of formula (4) shows that parameter C is 1, and α is the intermediate variable; The calculation shows that the filter order N is 5th order; According to the butter transfer function formula (7) in the software, the parameters B and A are calculated. [B,A]=butter(N,λ s ,l p ,C) (7) y is the output signal sequence, where x is the input signal, that is, the three subcarrier sequences of the data set, where B is the numerator of the Butterworth filter, A is the denominator of the Butterworth filter, and filter is the filter function in the software; y=filter(B,A,x) (8) The filtered signal is output in formula (8), and the outlier removal operation is performed on the input vector, i.e., the three subcarriers; the Hampel function calculates the median of the window consisting of the sample and its surrounding six samples, three on each side; and the standard deviation of the median of each sample pair is estimated by the absolute value of the median to replace the outliers, and the cleaned and processed data set is obtained; (4) Feature extraction of the cleaned and processed data set: Construct input feature values ​​and create a time window of 90 times 50ms, that is, 4.5 seconds. Move forward 45 records each time, using a semi-overlapping method. Then create input data. Each group contains 90 consecutive data of three subcarriers, which is called a segment posture feature value. The user activities in the 90 data are counted, and the one with the most occurrences is the label name of this group of data. (5) Perform human posture recognition on the extracted features to obtain posture recognition results The data set is divided into training set, validation set and test set; the specific SVM algorithm used is: The training set data T is expressed as: T={(x1,y1),(x2,y2),...,(xn,yn)} (9) xi belongs to a multidimensional space, and the value of y is one of 0, 1, 2, 3, 4, and n = 1, 2, ..., n; n is an integer greater than or equal to 1; xi is the i-th feature vector, yi is the label name, specifically running 0, sitting 1, standing 2, going up and down stairs 3, walking 4; Construct and solve a convex quadratic programming problem, where α i ,α j Indicates the corresponding y i y j The coefficient, α * represents the optimal solution. The physical meaning of α is to find an optimal solution that satisfies formula (10). By continuously optimizing α, the optimal hyperplane is found, thereby obtaining the boundary of the classification problem faced by SVM: 0<α i <M,i=1,2,3...n (12) Formula 10, 11, 12 to obtain the optimal solution in Represents the function y i The corresponding independent variable x i There are n optimal coefficients in total. Formula 13 calculates the coefficient b * , b * Represents the function y i There are n optimal solutions for the corresponding constants in ; The calculated α * , b * Substitute into formula (15) Use Gaussian kernel function, penalty coefficient M = 8, to solve α * Using the gradient descent method, the Gaussian kernel function K(x i ,x j ) as shown in the formula; the physical meaning of K is the function that refers to the following equation, with two independent variables; among them, σ 2 is x i ,x j The standard deviation of Solve f(x) and compare the result with the true value label. When f(x) = 0, it proves that the matching training result is running. When f(x) = 1, it proves that the matching training result is sitting. When f(x) = 2, it proves that the matching training result is standing. When f(x) = 3, it proves that the matching training result is going up and down stairs. When f(x) = 4, it proves that the matching training result is walking.

Images