The invention discloses a
pose-driven controllable video generation method based on a
diffusion model, and the method comprises the following steps: obtaining a reference appearance image set and a
pose sequence, generating an observation
tensor, and building a skeleton template; calling a micro SoftPOSIT to obtain a soft
distribution matrix, a frame-level
pose initial result, a
visibility weight and a matching confidence coefficient; alternately updating and applying
bone length invariance, joint amplitude limiting and
kinematic chain consistency constraint, and outputting frame-level pose
estimation; organizing an initial attitude sequence according to the matching confidence, and generating a
visibility weight sequence and a soft
distribution matrix sequence; establishing a cross-frame
factor graph, outputting a time consistent attitude sequence and generating a
factor graph state; integrating the quantities into a video condition sequence; in
diffusion denoising, temperature parameters and
noise intensity are synchronously adjusted and matched, attitude consistent guidance is calculated, and the state of a
factor graph is incrementally updated; and generating a video
frame sequence according to posture consistency guidance. According to the method, the attitude consistency and the
time sequence stability are improved, and the
controllability is enhanced.