Mixed reality open set human body posture recognition method based on deep learning

Abstract

The invention discloses a mixed reality open-set human body posture recognition method based on deep learning. The method includes using a neural network of an open-set human body posture recognition algorithm based on mixed reality, and setting a training loss function through a decision boundary based on a sample number imbalance problem to obtain a trained neural network; and migrating the domain invariant features learned from the virtual domain to the real domain to guide the real domain to carry out feature extraction, thereby realizing mixed reality human body posture recognition. According to the invention, unknown activities which do not appear in the training set can be detected while known human activities are accurately identified.

Description

technical field

[0001] The present invention relates to a technology in the field of wearable sensors, in particular to a mixed reality open-set human gesture recognition method based on wearable sensor equipment and deep learning. Background technique

[0002] Wearable sensors such as accelerometers and gyroscopes and other inertial measurement units (IMUs for short) are widely used in Human Activity Recognition (HAR for short) due to their low price and flexibility. Since there will be a variety of unpredictable and unknown behaviors in real life, the existing human gesture recognition based on wearable sensor devices can only solve the closed set problem, that is, it can only recognize human activities that have appeared in the training set. When there are unknown gestures in the test set that have not appeared in the training set, the existing closed-set algorithm will inevitably mistake the unknown class as a certain class in the known actions, which will cause a signif...

Examples

Embodiment Construction

[0022] Such as image 3 As shown, this embodiment relates to a mixed reality open-set human gesture recognition method based on wearable sensor devices and deep learning, and the specific steps include:

[0023] Step 1: Build a neural network.

[0024] The input data of the neural network of the mixed reality-based open-set human body posture recognition algorithm adopted in this embodiment are the human body posture data generated by the virtual IMU and the real human body posture data collected by wearable sensors, and are migrated from the virtual domain IMU data to The mixed reality method of real domain IMU data only uses a small amount of labeled real IMU data, and the adversarial training strategy enables the network model to accurately recognize known actions while including the ability to detect unknown human behavior actions.

[0025] Such as figure 2 As shown, the neural network of the open-set human gesture recognition algorithm based on mixed reality includes: ...

technical field

[0001] The present invention relates to a technology in the field of wearable sensors, in particular to a mixed reality open-set human gesture recognition method based on wearable sensor equipment and deep learning. Background technique

[0002] Wearable sensors such as accelerometers and gyroscopes and other inertial measurement units (IMUs for short) are widely used in Human Activity Recognition (HAR for short) due to their low price and flexibility. Since there will be a variety of unpredictable and unknown behaviors in real life, the existing human gesture recognition based on wearable sensor devices can only solve the closed set problem, that is, it can only recognize human activities that have appeared in the training set. When there are unknown gestures in the test set that have not appeared in the training set, the existing closed-set algorithm will inevitably mistake the unknown class as a certain class in the known actions, which will cause a signif...

Examples

Embodiment Construction

[0022] Such as image 3 As shown, this embodiment relates to a mixed reality open-set human gesture recognition method based on wearable sensor devices and deep learning, and the specific steps include:

[0023] Step 1: Build a neural network.

[0024] The input data of the neural network of the mixed reality-based open-set human body posture recognition algorithm adopted in this embodiment are the human body posture data generated by the virtual IMU and the real human body posture data collected by wearable sensors, and are migrated from the virtual domain IMU data to The mixed reality method of real domain IMU data only uses a small amount of labeled real IMU data, and the adversarial training strategy enables the network model to accurately recognize known actions while including the ability to detect unknown human behavior actions.

[0025] Such as figure 2 As shown, the neural network of the open-set human gesture recognition algorithm based on mixed reality includes: ...

Images