The invention discloses a three-dimensional
human body posture
estimation method and device based on multi-mode and
occlusion perception, and relates to the field of
computer vision, and the method comprises the steps: S1, obtaining a multi-mode
video sequence comprising an RGB sequence, a depth sequence and an
infrared sequence; s2, carrying out preprocessing to obtain a multi-
modal video sequence with aligned data; s3, constructing a three-dimensional
human body posture
estimation model comprising a spatial-temporal
feature extraction network, a shielding detection network, a
time sequence coordinate classification module, a Z-axis depth prediction module and a three-dimensional coordinate splicing module; s4, training a three-dimensional
human body posture
estimation model by using the data-aligned multi-
modal video sequence; and S5, performing three-dimensional human
body posture estimation by using the trained model. According to the method, through RGB, depth and
infrared three-mode fusion, X-axis and Y-axis coordinates are predicted by adopting a
time sequence coordinate classification method of
occlusion perception, and Z-axis depth prediction based on bucket separation and multi-mode and multi-resolution probability fusion is adopted, so that the problem of three-dimensional human
body posture estimation in a complex
occlusion scene can be effectively solved.