Scene text processing method, apparatus, device, and product

Abstract

The application discloses a scene text processing method and device, equipment and product, and relates to the technical field of data processing. The method comprises the following steps: acquiring a multi-modal control signal of a to-be-edited text image, wherein the multi-modal control signal comprises a source image, an erasing mask image, a structure guide image, a style reference image and a target editing instruction; performing feature coding on the multi-modal control signal based on a preset variational autoencoder and a visual language model to obtain corresponding mixed latent features; and performing fusion processing on the mixed latent features based on a preset diffusion model to generate an edited target text image, wherein the preset diffusion model comprises a plurality of diffusion model modules connected in series, and each diffusion model module comprises a superimposed reference position coding mechanism and a region adaptive suppression mechanism. The application realizes decoupling of space layout control and style control of the to-be-edited text image through the above scheme, and can output a high-quality and natural scene text processing result at one time.

Images