Multi-object visual content generation model training method, generation method and device

Abstract

The invention relates to a multi-object visual content generation model training method, a multi-object visual content generation method and a multi-object visual content generation device. The training method comprises the steps of obtaining a first local visual feature representation and a local text feature representation corresponding to each target object based on a training data pair; respectively embedding an image position code and a text position code in each target object through a position coding module to obtain a second local visual feature representation and a second local text feature representation; inputting the second local visual feature representation and the second local text feature representation of each target object into a cross attention module and a diffusion model, so that the diffusion model performs denoising according to fusion features output by the cross attention module; and fixing network parameters of each layer of the diffusion model, and iterating network parameters of the cross attention module and the position coding module to obtain a trained visual content generation model. According to the scheme provided by the invention, the consistency and the control precision of the visual content in the multi-object generation scene can be ensured.

Images