Human behavior recognition method and system based on multi-mode deep Boltzmann machine

Abstract

The invention discloses a human behavior recognition method and system based on a multi-mode deep Boltzmann machine. The method includes the following steps: 1) acquiring visual and wearable sensor data; 2) building visual data and wearable sensor multi-mode fusion model; 3) carrying out heterogeneous transfer learning by using a deep neural network to reconstruct missing data; 4) performing classification by using a softmax regression model classifier; 5) adaptively adjusting a deep network model generated by common sample data according to user individual characteristics. The human behaviorrecognition method and system can improve the accuracy of human body recognition in the event of complex scenes and missing data.

Description

technical field [0001] The invention relates to the technical fields of artificial intelligence and behavior recognition, in particular to a human behavior recognition method and system based on a multimodal deep Boltzmann machine. Background technique [0002] In recent years, the robot industry has shown explosive growth, and the era of "full application" of robots is coming. On the one hand, robots appear in families and daily life. On the other hand, with the development of industrial robots, robots are widely used in various industries such as automobile manufacturing and metal manufacturing to realize human-machine collaboration. Human behavior recognition is widely used in human-computer interaction, human-computer collaboration and other fields. Robots need to understand and recognize human behavior from various abstract layers. The accuracy of recognition will play a major role in the application and development of robot technology. Human behavior recognition by ro...

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and propose a human behavior recognition method and system based on a multimodal deep Boltzmann machine with high recognition accuracy and strong usability, aiming to build a human behavior recognition system based on vision The multimodal deep neural network model of wearable sensors can improve the accuracy of behavior recognition in complex scenes; the deep Boltzmann machine network is used in the multimodal deep learning model to reduce the impact of missing data on the accuracy of behavior recognition The influence caused by this problem; a method of adjusting the network structure to establish an adaptive common model combined with individual characteristics is proposed, so as to improve the accuracy of the robot's recognition of the specific owner's behavior

Images