A method for semantically multimodal image generation

Abstract

The invention discloses a method for generating semantically multimodal images. The present invention proposes a new task on the task of segmenting the label to generate an image, and the new task is to be able to individually control the category of the generated image by dividing the category of the label. On this task, the present invention proposes a solution that uses grouped convolution for the encoder to separate the encoding of different classes in the natural image, so that changing the encoding can change the style of the class in the generated image. In the generator part, the present invention adopts grouped convolution with gradually decreasing number of groups, which can improve the ability of independent control and reduce the occupation of video memory, so this method can be applied to data sets with many categories. At the same time, the present invention can be applied to many image generation applications, such as image editing and the like. Compared with the previous related work of segmenting tags to generate natural images, the present invention can individually control the categories in the generated images as much as possible.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and more specifically relates to a method for generating semantically multi-modal images. Background technique [0002] In recent years, the image quality generated by GAN has been getting higher and higher, and many interesting applications have been produced, such as image editing, image completion, etc. Another application is to generate natural images through semantic segmentation labels of images. This can be seen as the inverse process of image segmentation. Generating natural images by segmenting labels can facilitate the creation of various images. Only need to draw the segmentation map that needs to generate images, and then can generate images that are close to real through the network, simplifying the operation process and enhancing the control over images . [0003] Generating natural images through segmentation labels is a one-to-many problem, that is, one segmentation map ...

AI Technical Summary

Problems solved by technology

These methods either generate natural images that are fixed, or can only make global changes to the image, and cannot individually control the generation effect of each category.

Images