Fall detection method based on wifi channel state information

Abstract

The application provides a fall detection method based on WiFi channel state information. The method comprises a training stage and a detection stage. The training stage comprises: obtaining CSI data and video data of WiFi in the same time period; extracting a human mask in the video data, then matching the CSI data at the same time with the human mask to obtain a training data pair according to a time stamp, and further constructing a training data set; iteratively updating a GAN network using the training data set; using a CNN-AE model to extract features of a normal human contour picture, and then using the extracted CNN-AE features to train an OCSVM model; the detection stage comprises: obtaining CSI data in real time; using the trained generator to generate a scene picture according to the CSI data obtained in real time; using the CNN-AE model to extract features of the scene picture, and inputting the extracted features into the trained OCSVM model to detect whether a fall behavior occurs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of behavior recognition, and particularly relates to a fall detection method based on WiFi channel state information. BACKGROUND

[0002] Traditional fall detection methods are mainly divided into contact sensing detection and non-contact sensing detection. The contact sensing detection obtains user limb behavior and movement mode information by wearing various customized acceleration sensors, gyroscopes and the like. Since this detection method requires users to carry specific equipment in real time, it consumes a large amount of cost and is difficult to operate stably for a long time, so this detection method does not meet the development needs of intelligent sensing. The non-contact sensing detection (mainly refers to fall detection based on computer vision) mainly relies on cameras and various sensors. This detection method realizes data collection by collecting video image signals, physical data parameters and user computing information, then extracts human foreground targets, extracts features, and then uses machine learning algorithms to simulate the fall decision-making process to achieve the effect of fall detection. However, this detection method requires sufficient light conditions (it cannot be accurately recognized in poor light conditions such as heavy fog), and may cause indoor privacy problems (such as not applicable to scenes such as bathhouses and toilets), and is affected by the visual distance in the indoor situation, such as some dead angles may not be monitored. In addition, ultra-wideband, ZigBee, and radio frequency signals have good performance in trajectory tracking and position information acquisition, but the acquisition of such signals requires special equipment, and the specific deployment method and high cost limit its application scenarios. SUMMARY

[0003] In order to protect user privacy while improving the accurate detection of the fall state, the present application provides a fall detection method based on WiFi channel state information.

[0004] The present application provides a fall detection method based on WiFi channel state information, comprising: a training stage and a detection stage.

[0005] The training stage comprises:

[0006] Obtain the CSI data and video data of WiFi in the same time period;

[0007] Extract the human mask in the video data, then match the CSI data at the same time with the human mask according to the time stamp to obtain a training data pair, and then construct a training data set.

[0008] updating the GAN network iteratively using the training data set; wherein each iteration updating process comprises: generating a fake picture according to the CSI data by the generator, and performing adversarial training by taking the human body mask corresponding to the CSI data and the fake picture as inputs of the adversarial network;

[0009] extracting features of the normal human body contour picture by using the CNN-AE model, and then training a one-class support vector machine (OCSVM) model by using the extracted CNN-AE features;

[0010] the detection stage comprises:

[0011] real-time acquisition of CSI data, and generation of a scene picture according to the real-time acquired CSI data by using the trained generator;

[0012] feature extraction of the scene picture by using the CNN-AE model, and input of the extracted features into the trained OCSVM model to detect whether a fall behavior occurs.

[0013] Further, the human body mask in the video data is extracted by using a Mask R-CNN framework.

[0014] Further, in the training stage and the detection stage, the picture is subjected to background subtraction before being subjected to feature extraction by using the CNN-AE model.

[0015] Further, the feature extraction of the scene picture by using the CNN-AE model, and the input of the extracted features into the trained OCSVM model to detect whether a fall behavior occurs, specifically comprises:

[0016] Given a video sequence, a frame-by-frame processing manner is adopted to perform background subtraction on a current frame scene picture to extract a human body contour picture from the current frame scene picture; then, the CNN-AE model is used to extract contour features x from the human body contour picture, and the contour features x are input into the OCSVM model; if a probability value output by the OCSVM model exceeds a set threshold value, the current frame is considered as a normal frame, otherwise, the current frame is considered as an abnormal frame, and the abnormal frame is marked as a fall behavior; finally, whether the number of abnormal frames exceeds a set threshold value is determined, and if yes, it is considered that a fall occurs.

[0017] The present application has the following advantages:

[0018] (1) Different from the existing WiFi channel state information, the amplitude and phase features are extracted, and the human body dynamic path is used to solve the fall detection problem, the present application generates a human body activity posture image through WiFi channel state information, and then realizes fall detection through a machine learning algorithm.

[0019] (2) In order to generate accurate human activity posture images based on WiFi channel state information, a method for generating images based on GAN network is proposed, which can improve the imaging accuracy. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the fall detection method based on WiFi channel state information provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the CNN-AE model provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] With the development of the Internet of Things (IoT), wireless routing devices have been integrated into thousands of households, and Wi-Fi sensing-based applications are being widely used in life. This invention is based on the channel state information of Wi-Fi signals and combines deep learning and machine learning technologies to realize indoor human fall event detection.

[0024] like Figure 1 As shown, this embodiment of the invention provides a fall detection method based on WiFi channel state information, including: a training phase and a detection phase;

[0025] The training phase includes:

[0026] Acquire CSI data and video data of WiFi within the same time period;

[0027] Specifically, in this embodiment of the invention, on the hardware device, two computers equipped with Intel 5300 network cards are used, one computer as the transmitter with two antennas and the other computer as the receiver with three antennas; on the software device, the open-source CSI tool deployed on the Ubuntu 14.04 system is used; CSI data is collected based on the above software and hardware configuration, and the CSI data format is a 2*3*30 matrix.

[0028] The video data is collected using a 360 camera; it should be noted that the collected video data is only used for data annotation and will not leak user privacy.

[0029] extract the human body mask in the video data, and then match the CSI data at the same time according to the timestamp with the human body mask to obtain a training data pair, and then a training data set is constructed;

[0030] Specifically, the human body mask in the video data is extracted by using the Mask R-CNN framework. In the embodiment of the application, 5 CSI data packets at one time (for example, per second) are matched with a human body mask to obtain a training data pair, and a 150*2*3 matrix is formed.

[0031] The training data set is used for iterative updating of the GAN network, wherein each iteration updating process comprises: generating a fake picture according to the CSI data by using the generator, and inputting the human body mask corresponding to the CSI data and the fake picture into the discriminator as inputs for adversarial training.

[0032] Specifically, in the training stage, the CSI matrix needs to be up-sampled first, and then the up-sampled result is input into the generator to generate a fake picture. The stopping condition of the iterative updating can be that the number of iterations reaches a given number of iterations, or the accuracy of the generator reaches a given accuracy threshold. On the basis of the original CSI matrix, the up-sampling algorithm is used to insert appropriate data on the original data, which can partially solve the sparsity of the sample, so that the generator can more easily obtain rich features to generate the original image.

[0033] The CNN-AE model is used for feature extraction of normal human contour pictures, and then the extracted CNN-AE features are used for training of the one-class support vector machine (OCSVM) model.

[0034] Specifically, the normal human contour picture refers to a scene image of other daily behavior activities except the falling behavior, for example, a walking scene image and a normal sitting scene image. The purpose of this step is to make the one-class support vector machine (OCSVM) model learn the normal posture in daily life, so as to distinguish the normal posture from the falling behavior.

[0035] Generally, there is a large amount of invalid background information in the scene image. In order to eliminate image redundancy and reduce the amount of feature extraction calculation, the background is removed from the picture before the picture is subjected to feature extraction by using the CNN-AE model. In this embodiment, the background removal process also becomes a human contour extraction process, which specifically comprises: first extracting the minimum bounding rectangle of each contour, then maintaining the aspect ratio of the rectangle region, scaling it to a template with a size of 30*30 pixels, and finally normalizing each pixel point.

[0036] The structure of the CNN-AE model used in the embodiment of the application is as follows: Figure 2The CNN-AE is an unsupervised learning algorithm that learns the hidden features of the original input data through a neural network. It uses convolutional layers instead of fully connected layers in artificial neural networks, which are more suitable for reconstructing two-dimensional image information. The specific process is as follows: for the input data x, the intermediate semantic vector x* is obtained through the automatic encoder, and then the intermediate semantic vector x* enters the decoder to obtain the output result, thereby completing the reconstruction of the feature x*. A loss function is established between the input and output of the neural network, and the stochastic gradient descent (SGD) optimization algorithm is used to optimize the model parameters through multiple iterations to realize feature extraction.

[0037] wherein the feature map calculation process of the lth convolutional layer is taken as an example, which can be expressed by formula (1):

[0038]

[0039] wherein, represents the pixel point of the kth feature map in the lth convolutional layer, represents the corresponding input, and f(·) represents a Sigmoid activation function, represents the corresponding convolution kernel matrix (d, m, n) of the kth feature map in the lth convolutional layer, represents the corresponding bias value.

[0040] Compared with the original automatic encoder and the stacked automatic encoder, the CNN-AE requires fewer parameters to be adjusted, making it easier to train with relatively less training data set.

[0041] The detection stage includes:

[0042] real-time acquisition of CSI data; using the trained generator to generate scene pictures according to the real-time acquired CSI data;

[0043] using the CNN-AE model to extract features from the scene pictures, and inputting the extracted features into the trained OCSVM model to detect whether a fall behavior occurs.

[0044] Specifically, given a video sequence, the scene picture of the current frame is background subtracted, i.e., the human body contour map is extracted, in a frame-by-frame processing manner, and then the contour feature x is extracted from the human body contour map by using the CNN-AE model, and finally whether the posture in the frame image is a normal posture is judged by the OCSVM model. The OCSVM model outputs a specific value according to the input contour feature x, that is, the probability that the posture in the frame is a normal posture. When the probability value exceeds the set threshold, it is considered to be a normal frame, otherwise it is an abnormal frame, and the abnormal frame is marked as a fall behavior; finally, whether the number of abnormal frames exceeds the set threshold is counted, if so, it is considered that a fall occurs, and finally the human fall event detection is realized.

[0045] The specific calculation process of the OCSVM model for fall recognition is as follows: assuming that the coordinate origin is an abnormal point, all data points x = [x1, x2, … xn] are separated from the origin in the feature space, and the distance of the maximum separation hyperplane to the origin is maximized. A decision function f(x) is established by the data points x, as shown in formula (2).

[0046]

[0047] Wherein, x represents a data sample; is a mapping function that maps data to a high-dimensional space, also known as a kernel function, and the selection of the kernel function and the setting of the parameters play a key role in the classification result of the model.

[0048] The Gaussian kernel function k Guassian (x,y) is modeled with different parameter γ values, and the definition of the Gaussian kernel function is formula (3):

[0049] k Guassian (x,y) = exp(-γ‖x-y‖) (3)

[0050] f(x) obtains the probability density area of the data in the feature space. When the input x is located inside the probability density area of the training samples, f(x) is greater than the threshold, and vice versa

[0051] The parameters ω and ρ need to be estimated in formula (2), in order to obtain the parameters ω and ρ, the dual form of formula (4) is obtained by introducing the Lagrange multiplier:

[0052]

[0053] Wherein, α i represents a Lagrange multiplier, and the relationship between the optimal function parameters (ω and ρ) and the optimal solution α is obtained, and the model parameters ω and ρ are optimized.

[0054] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A fall detection method based on WiFi channel state information, characterized in that, The application relates to a fall detection method based on WiFi channel state information (CSI) and video data. The method comprises a training stage and a detection stage. The training stage comprises the following steps: obtaining CSI data and video data of the same time period; extracting a human body mask in the video data, and then matching the CSI data and the human body mask at the same time according to a time stamp to obtain a training data pair, and further constructing a training data set; updating a GAN network by iteration using the training data set; wherein each iteration updating process comprises the following steps: generating a false picture according to the CSI data by using a generator, and inputting the human body mask corresponding to the CSI data and the false picture into an adversarial trainer for adversarial training; extracting features of a normal human body contour picture by using a CNN-AE model, and then training an OCSVM model by using the extracted CNN-AE features; the detection stage comprises the following steps: obtaining CSI data in real time, and generating a scene picture according to the real-time obtained CSI data by using the trained generator; extracting features of the scene picture by using the CNN-AE model, and inputting the extracted features into the trained OCSVM model to detect whether a fall behavior occurs. 2.The fall detection method based on WiFi channel state information according to claim 1, characterized in that, The human body mask in the video data is extracted by using a Mask R-CNN framework. 3.The WiFi channel state information based fall detection method of claim 1, wherein, In the training stage and the detection stage, the picture is subjected to background subtraction before being subjected to feature extraction by using the CNN-AE model. 4.The fall detection method based on WiFi channel state information according to claim 3, characterized in that, The method for extracting features of the scene picture by using the CNN-AE model, and inputting the extracted features into the trained OCSVM model to detect whether a fall behavior occurs comprises the following steps: given a video sequence, a current frame scene picture is subjected to background subtraction to extract a human body contour picture in a frame-by-frame processing mode; the contour feature x is extracted from the human body contour picture by using the CNN-AE model, and the contour feature x is input into the OCSVM model; if the probability value output by the OCSVM model exceeds a set threshold value, the current frame is considered to be a normal frame, otherwise, the current frame is considered to be an abnormal frame, and the abnormal frame is marked as a fall behavior; finally, whether the number of abnormal frames exceeds a set threshold value is determined, and if yes, it is considered that a fall occurs.

Images