The invention discloses a wild animal attitude
estimation method based on dynamic prompt and semi-supervised multi-mode learning, and relates to the technical field of
computer vision and
animal behavior analysis, and the method comprises the steps: firstly carrying out the motion state analysis of an input video, so as to extract parameters such as the
centroid velocity, the acceleration and the
steering angle of a moving material; the method comprises the following steps: generating a
dynamic text prompt containing directional
semantics, constructing a cross-
modal fusion
time sequence convolutional network, fusing visual features and text prompt features through a space alignment
mask mechanism and a cross attention module, capturing
time sequence dependence by using a bidirectional ConvGRU network to realize attitude
estimation, and finally, based on a semi-supervised multi-
task learning framework, carrying out
dynamic text prompt
processing on the basis of the semi-supervised multi-
task learning framework. A joint
loss function including supervision loss,
time difference loss, attitude PCA reconstruction loss, multi-
modal alignment loss and motion consistency loss is adopted, a
loss weight is adaptively adjusted in combination with prediction uncertainty to optimize network parameters, a smooth 2D / 3D animal skeleton is finally output, real-time reasoning can be realized on edge equipment, and the real-time reasoning efficiency is improved. Ecological
analysis software is compatible, high precision is still kept under the condition of low-
label data, and the performance is remarkably improved especially in an animal sharp turning scene.