Human activity fine-grained identification method based on human body RFID skeleton

Abstract

The invention provides a human activity fine-grained identification method based on a human body RFID skeleton, and belongs to the field of deep learning. According to the method, an advanced identification method and existing problems are considered, firstly, 16 RFID flexible tags are arranged at main joints of a human body by using an RFID technology and distribution of a human skeleton, a new human activity data set is acquired by reading the tags, the data set is an activity data set with most human activity types in the HA R field of bound RFID so far, and the data set is a new human activity data set with most human activity types in the HA R field of bound RFID. The problems that at present, the HA R based on the binding type RFID mainly utilizes human body trunk movement information, four-limb movement information related to human activities is lacked, human activity characteristics cannot be effectively captured, and the types and the number of human activities are small are solved. Then, based on the sampling data, a space-time skeleton diagram of human body activities is designed, and the problem that at present, HAR based on binding type RFID cannot utilize a graphic interaction network to classify graphs is solved. And finally, designing a space-time skeleton diagram convolutional network for human activity identification, and classifying the space-time skeleton diagram to realize human activity identification, thereby solving the problems that the HAR based on binding type RFID uses the traditional machine learning as an identification method at present, the HAR is limited in modeling capability for complex activities and low in accuracy. Experiments show that the method is a simple and feasible fine-grained human activity identification method.

Description

Technical Field

[0001] The content of this method belongs to the field of deep learning and is specifically used for human activity recognition. Human activity recognition (HAR) has always been an important topic in the field of pattern recognition and ubiquitous computing. It is a very challenging time series classification task. Human activity recognition methods are not only applicable to active health services, but also can be used in human-computer interaction, virtual reality, behavior analysis, animation production, traffic safety and other fields. The identified activities can be ubiquitous activities such as walking, standing, sitting, and lying in a general environment, or specific activities in special scenes (such as sports fields and factories). Background Art

[0002] Radio frequency identification (RFID) is a wireless communication technology that uses radio signals to identify target objects and obtain corresponding information. In recent years, it has been widely used in the field of human activity recognition due to its outstanding advantages such as privacy protection, ease of use, identifiability, lightweight, and fine-grainedness. Using RFID to monitor human activities in real time, especially fall detection for the elderly, is of great significance in smart healthcare. RFID-based HAR can effectively solve the three core key issues in the field of the Internet of Things, namely object recognition, location positioning, and state perception.

[0003] There are already some RFID-based HAR solutions, but compared with the relatively complete computer vision-based HAR methods, the existing RFID-based HAR methods still have a lot of room for improvement in classification granularity and accuracy. How to use the significant perception capabilities of RFID technology to achieve more detailed activity recognition while ensuring recognition accuracy has become an important issue in current RFID-based HAR research.

[0004] RFID-based HAR can be divided into two types: non-binding and binding. Non-binding RFID HAR cannot identify active individuals. Depending on whether the tag is moving, non-binding RFID HAR methods can be further divided into three types: The first is to deploy tags on daily life objects. During human activities, the tags will be in a moving state. This method identifies human activities by monitoring the movement of tags on these objects [1]. This method only identifies human activities on the used objects with tags, and the types of activities identified are limited. The second is that the tags exist in the environment in the form of an array, which is a fixed deployment of tags. In the environment, the subject moves in the space between the tag and the antenna, and the activity is identified by detecting the occlusion between the person and the tag [2]. This method has edge blur and cannot perform fine-grained detection. The third is to combine the first two, monitor the movement and response signals of the tags blocked by the human body, and identify the activities of people using objects and other activities respectively [3].

[0005] HAR bound to RFID can identify active individuals. It deploys tags on the human body, and researchers analyze activity categories by identifying the movement of the tags [4-5]. Compared with traditional wearable sensors, this method has the advantage of being easy to use due to the low power consumption and light weight of the tags. At the same time, this method can use the identifiability of RFID to identify active individuals.

[0006] From the above, it can be seen that RFID-bound HAR has outstanding advantages in human activity recognition methods because it is easy to use and identify active individuals. References: [1] [2] L.Yao, QZSheng, [3] L.Yang, Y.Qiu, F.Lin, B.Wang, and X.Wu, "Optimized Algorithm for RFID-Based Activity Recognition of the Elderly," in IEEE Access, vol.7, pp.136777-136782, 2019. [4] L.Wang, T.Gu, [5] Summary of the invention

[0007] The content of the present invention is to provide a fine-grained human activity recognition method based on human RFID skeleton, which can overcome some shortcomings of the prior art. The method takes into account advanced recognition methods and existing problems. First, RFID technology and the distribution of human skeleton are used to arrange 16 RFID flexible tags at the main joints of the human body. By reading these tags, a new human activity data set is collected. This data set is the activity data set with the most types of human activities in the field of HAR of binding RFID so far, which solves the problem that the current HAR based on binding RFID mainly uses human trunk motion information, lacks limb motion information involved in human activities, cannot effectively capture human activity characteristics, and has a small number of human activities. Then, based on these sampled data, a spatiotemporal skeleton graph of human activities is designed, which solves the problem that the current HAR based on binding RFID fails to classify graphics using a graphic interactive network. Finally, a spatiotemporal skeleton graph convolutional network for human activity recognition is designed. By classifying the spatiotemporal skeleton graph, human activity recognition is realized, which solves the problem that the current HAR based on binding RFID uses traditional machine learning as a recognition method, and these HARs have limited modeling capabilities for complex activities and low accuracy. Experiments show that this method is a concise and feasible method for fine-grained human activity recognition.

[0008] A fine-grained human activity recognition method based on human RFID skeleton is characterized by: designing activity categories including 21 kinds of human daily activities, considering the superiority of RFID technology and the distribution of human skeleton, 16 RFID flexible tags are arranged at the main joints of the human body, and by reading these tags, the activity data set with the largest number of human activities in the HAR field of binding RFID is collected so far, which improves the application scenario of this type of recognition method. Using the idea of ​​classifying graphics with a graph interactive network, a human RFID skeleton spatiotemporal graph that can reflect the spatiotemporal characteristics of human activities is designed. Further, a spatiotemporal skeleton graph convolution network for human activity recognition is designed using a graph convolutional network, and the spatiotemporal skeleton graph is classified to realize human activity recognition. Experiments show that the probability that the network can accurately identify activities is between 48% and 100%, the accuracy of most activities is above 76%, and nearly half of the activities can be identified with an accuracy of above 90%. The low recognition rate of some activities is due to too many missing values ​​in some activities. In general, the network solves the problem that the current human activity recognition model based on binding RFID has limited modeling capabilities for complex activities and cannot effectively capture activity information.

[0009] The human activity fine-grained recognition method based on human RFID skeleton includes the following steps:

[0010] 1. Design a data collection plan

[0011] 2. Data Collection

[0012] 3. Data Processing

[0013] 4. Designing a Human Activity Recognition Model

[0014] 5. Perform human activity recognition to obtain recognition accuracy BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of the actual deployment of antennas;

[0016] Figure 2 This is a schematic diagram of antenna angle correction;

[0017] Figure 3 This is a schematic diagram of the human RFID skeleton structure;

[0018] Figure 4 It is a space-time skeleton diagram; DETAILED DESCRIPTION

[0019] Step 1. Design a data collection plan

[0020] The hardware devices used in this method are shown in the following table.

[0021] like Figure 1 As shown, each subject needs to carry 16 RFID flexible tags. Most of these tags are deployed at the main joints of the human body, which can effectively monitor the movement of the corresponding deployment positions. In order to make the experimental environment suitable for most subjects of different heights, this method sets the antenna height to 2m. In order to make the antenna reading range cover the tags closest to the ground as much as possible, the angle between the antenna plane and the bracket column is set to 35°. At this time, the highest boundary of the antenna reading range is exactly parallel to the ground and 2m above the ground. The antenna angle correction is shown in the figure. Figure 2 In this scheme, theoretically, when the person being measured is directly in front of the antenna, the possible blind spot for measurement only exists within 0.73m in front of the antenna. This method numbers the node points (tags), and the numbering is as follows Figure 3 As shown, 0, 1, 8, and 9 represent the trunk, 2-7 represent the upper limbs, and 10-15 represent the lower limbs.

[0022] Step 2. Data Collection

[0023] In order to ensure the diversity of human activity data based on human RFID skeleton, 15 volunteers of different ages, genders and body shapes were recruited, with a male-female ratio of 8:6, a height of 1.55-1.85 meters, and an age of 18-30 years old. According to the data collection plan, we collected human activity data of these volunteers on different dates and time periods. Each volunteer repeated each human activity 40 times, and each completed human activity was an activity data sample. The time to complete an activity sample is also called the duration of the human activity. We reasonably set the duration to 5 seconds. When 15 volunteers completed 21 activities, we obtained 1.27×10 4 Activity samples. Each activity sample includes five values: electronic product code (EPC), timestamp (Timestamp), Doppler frequency (DopplerFrequency: DF), received signal strength (RSSI), and phase (Phase).

[0024] Step 3. Data processing

[0025] The DF value expresses the moving speed and direction of human activities, the RSSI value expresses the large displacement of human activities, and the Phase value expresses the small displacement of human activities. Therefore, the three values ​​of DF, RSSI, and Phase together constitute a set of tag response characteristic data used to express human activities. We assume that the average duration of a human activity is 5s, and the tag generates at most h sets of tag response characteristic data during this duration.

[0026] Due to the different physical characteristics (such as height, weight, etc.) and different positions of volunteers, the size of the tag response characteristic data of the same activity often varies during the data collection process. In order to ensure that the tag response characteristic data of the same activity have similar scales and ranges, the HAR model can be better trained and the classification accuracy can be improved. To this end, it is necessary to normalize the tag response characteristic data. The size of the normalized tag response characteristic data is distributed between 0 and 1, eliminating the difference in the tag response characteristic data and having the same activity expression ability as the original tag response characteristic data. In addition, the RFID system uses a multi-tag random reading method to collect data. The number of obtained tag response records is different and random. This difference and randomness will cause the number of original tag response characteristic data of the 16 tags to be different. The normalized tag response characteristic data cannot be directly used to train the HAR model. Therefore, it is necessary to regularize the normalized tag response characteristic data of the 16 tags. These two processes are called standardization, and all the normalized tag response characteristic data are used to construct the RFID skeleton characteristic matrix.

[0027] In the RFID skeleton characteristic matrix, we use h groups of tag response characteristic data of 16 node tags to construct h human RFID skeletons. We propose that a human RFID skeleton represents a human posture in human activities, and each human activity is represented by h continuously changing human postures. Therefore, we use h human RFID skeletons to jointly construct an RFID skeleton activity graph that can fully express human activities. Since a simple RFID skeleton activity graph only has the spatial position connection of the tags, it does not have the time information of each node. In order to make the RFID skeleton activity graph have the spatiotemporal information of the nodes at the same time, we constructed a spatiotemporal skeleton graph to represent an activity of the human body. The spatiotemporal skeleton graph is as follows: Figure 4 shown.

[0028] Step 4. Design a human activity recognition model

[0029] Traditional machine learning methods usually treat data from different sources as independent and uniformly distributed data. When these data are considered to be related, they need to be fused to express the correlation and improve learning efficiency. In contrast, the node data in the graph are interconnected through edges. Traditional machine learning has difficulty in handling graph computing tasks, and graph machine learning models have emerged. Therefore, we constructed a spatiotemporal skeleton graph convolutional network for activity recognition and used it for spatiotemporal skeleton graph classification. The network consists of a spatiotemporal skeleton graph batch processing module, two skeleton graph ensemble convolution modules, a convolution result pooling module, an activity classification module, and a softmax function.

[0030] The batch processing module combines a batch of spatiotemporal skeleton graphs in a discontinuous form into a large spatiotemporal skeleton graph to reduce the number of model training times, shorten the processing time of each spatiotemporal skeleton graph, and improve the efficiency of activity recognition. According to the structure of the spatiotemporal skeleton graph, it can be seen that only two convolutions are required to aggregate the feature data of each node from other nodes. Therefore, the network consists of two convolution modules. Each convolution module contains a hidden layer and an activation function, where the hidden layer is used to aggregate the feature data of adjacent nodes in the graph, and the activation function remaps the aggregated feature data to make the data linearly separable.

[0030] Step 5. Perform human activity recognition to obtain recognition accuracy

[0031] All deep learning models used in this method are implemented on the Pytorch platform, and the graphics card configuration is RTX3090 (24G). According to the 80 / 20 rule, we take 80% of the activity samples in the human activity dataset as the training set (10173 activity samples), and 20% of the activity samples in the human activity dataset as the test set (2535 activity samples). The initialization value of the model training parameters is the system default value: the initial learning rate is set to 0.001, and the ratio is reduced by a factor of 0.1 every 30 training stages (epochs), and a total of 50 epochs are trained. In the model parameters, Bs is set to 64 and the number of convolution nodes is set to 256.

Claims

1. Fine-grained recognition method of human activities based on human RFID skeleton Considering advanced recognition methods and existing problems, RFID technology and the distribution of human skeleton are first used to arrange 16 RFID flexible tags at the main joints of the human body. By reading these tags, a new human activity data set is collected. This data set is the activity data set with the largest number of human activities in the field of HAR based on binding RFID so far. It solves the problem that the current HAR based on binding RFID mainly uses the human trunk movement information, lacks the limb movement information involved in human activities, cannot effectively capture the characteristics of human activities, and the types and number of human activities are small. Then, based on these sampled data, a spatiotemporal skeleton graph of human activities is designed, which solves the problem that the current HAR based on binding RFID fails to classify graphics using graphic interaction networks. Finally, a spatiotemporal skeleton graph convolutional network for human activity recognition is designed. Human activity recognition is realized by classifying the spatiotemporal skeleton graph, which solves the problem that the current HAR based on binding RFID uses traditional machine learning as the recognition method. These HARs have limited modeling capabilities for complex activities and low accuracy. Experiments show that this method is a concise and feasible fine-grained human activity recognition method.

2. A fine-grained human activity recognition method based on human RFID skeleton, characterized by: An activity category containing 21 kinds of daily human activities was designed. Considering the superiority of RFID technology and the distribution of the human skeleton, 16 RFID flexible tags were arranged at the main joints of the human body. By reading these tags, the activity data set with the largest number of human activities in the HAR field of binding RFID was collected so far, which improved the application scenarios of this type of recognition method. Using the idea of ​​classifying graphics with a graph interaction network, a human RFID skeleton spatiotemporal graph that can reflect the spatiotemporal characteristics of human activities was designed. Further, a spatiotemporal skeleton graph convolution network for human activity recognition was designed using a graph convolution network, and the spatiotemporal skeleton graph was classified to achieve human activity recognition. Experiments have shown that the probability that the network can accurately identify activities is between 48% and 100%, the accuracy of most activities is above 76%, and nearly half of the activities can be identified with an accuracy of above 90%. The low recognition rate of some activities is due to too many missing values ​​in some activities. In general, the network solves the problem that the current human activity recognition model based on binding RFID has limited modeling capabilities for complex activities and cannot effectively capture activity information. The fine-grained recognition method of human activities based on human RFID skeleton includes the following steps: Designing a data collection plan Data collection Data processing Designing a human activity recognition model Human activity recognition is performed to obtain the recognition accuracy.

Images