Multi-modal human activity recognition method based on generative adversarial network

Abstract

The invention discloses a multi-mode human activity recognition method based on a generative adversarial network. The multi-mode human activity recognition method specifically comprises the followingsteps: 1) preprocessing activity data acquired by using a wearable device, and constructing a training data set; 2) generating diversified activity data by using a modal generator based on a full connection network according to the category label of the real activity data; and 3) for the real activity data and the generated activity data, carrying out joint training of a discrimination task and anactivity recognition task by utilizing a hierarchical discriminator and a classifier based on a shared convolutional layer to obtain a classifier with relatively high generalization capability. The multi-mode human activity recognition method introduces the activity recognition model into the generative adversarial network, generates activity data with diversity, obtains the multi-modal activityrecognition model with high generalization ability through joint learning of the discrimination task and the recognition task, improves the activity recognition performance, and has wide application prospects in the fields of medical care, motion monitoring and the like.

Description

technical field [0001] The invention relates to the field of activity recognition, in particular to a multi-modal human activity recognition method based on a generative confrontation network. Background technique [0002] Human activity recognition based on wearable devices is one of the important research fields of ubiquitous and mobile computing. Wearable devices placed on different body parts are used to collect activity data, and the discovered data changes are used to identify the types of activities performed. . Many application scenarios in today's life are realized by human activity recognition based on wearable devices, such as sports tracking and training, health care and work assistance. [0003] Early research on human activity recognition based on wearable devices was mainly based on artificially defined features, either from a single sensor modality, or from multimodal data. Most of the artificially defined features can be divided into time-domain features (...

AI Technical Summary

Problems solved by technology

However, the first type of model tends to generate data samples with rich modality details, but may lose global consistency; the second type of model can maintain global consistency, but may not be able to capture diverse modality details

The above existing models cannot balance the global consistency and modality details of the generated data, and cannot meet the requirements of deep multimodal activity recognition

Images