The invention discloses a multi-style picture book generation method based on a
diffusion model. The method comprises the following steps: S1, constructing a
data set; s2, constructing a picture book generation model; the preprocessing module encodes the picture book image and the story text to obtain text embedding, role
mask and role image; s3, importing a style
reference image to a style consistency module, and obtaining style feature embedding; s4, importing text embedding, role masks and role images to a role consistency module, and obtaining role embedding and
layout embedding; s5, importing the picture book image, performing style feature embedding, role embedding and
layout embedding into a Unet module to perform picture book
image noise prediction, and obtaining a predicted picture book image; and S6, constructing a
loss function. According to the method, a
diffusion model and an image style migration
algorithm in
deep learning are combined, a coherent picture book scene with various artistic styles can be generated according to the input
reference image, and the method can be widely applied to the fields of
animation production, image
creative design,
digital content generation and the like.