Indoor human pose recognition method based on multi-sensor fusion

Abstract

The invention provides an indoor human pose recognition method based on multi-sensor fusion and belongs to an indoor scene recognition method of a mobile robot. The method comprises the following steps: 1) deploying an optical sensor, a depth sensor and an infrared sensor; 2) acquiring images of the three sensors in the same scene and calibrating the images of the three sensors; 3) recognizing thecurrent real-time human pose with a pre-trained deep neural network. Fusion processing and comprehensive decision making are performed on data of the multiple sensors with machine learning and a computer vision algorithm, various limitations and bottlenecks of the traditional recognition method based on single sensor are solved, and the recognition accuracy and robustness for various indoor humanposes are improved greatly.

Description

technical field [0001] The invention designs a human body posture recognition method based on multi-sensor fusion of a deep neural network, which belongs to an indoor scene recognition method of a mobile robot. Background technique [0002] Among the many key technologies of indoor mobile robots, scene perception and interaction is one of the important contents, and it is also an important research direction of computer vision. Among them, human body posture recognition is a typical application in indoor scenes. Through the recognition of human body posture in the scene, the mobile robot can make further interactive responses, such as approaching or following for voice interaction and other follow-up operations. Among the many current gesture recognition methods, they can be classified into two categories in terms of input form. One type uses traditional optical sensors as the main action video, or directly performs single-frame image recognition, or considers the continuity...

AI Technical Summary

Problems solved by technology

In order to achieve high recognition accuracy, the traditional single-sensor human body posture recognition method usually has strict restrictions on the external environment and scenes, and cannot achieve stable recognition of full-time complex scenes

These limitations are mainly caused by the characteristics of the sensor itself. For example, although the optical sensor has the advantages of low cost and deployment and installation, it has high requirements for light and angles, especially in weak light conditions, and the recognition effect is significantly reduced.

Although the depth sensor can contain three-dimensional space information such as depth and is more robust to illumination changes compared with the optical sensor, its recognition range is small (within 5 meters), its own resolution is low, and the edge of the object is prone to holes. Problems such as large delay also limit its application range; infrared sensors are also more sensitive to illumination and angle changes than optical sensors, but still have low resolution, lack of texture and color, and are susceptible to noise from various heat sources Impact and other issues

Images