Remote sensing image style conversion method based on text data

Abstract

The invention provides a remote sensing image style conversion method based on text data, and the method comprises the steps: constructing a data set, and obtaining a text data set and a to-be-converted image data set; generating a low-resolution image, extracting sentence features according to the text data, and generating a low-resolution remote sensing image and corresponding image features incombination with noise; generating a high-resolution image, extracting word features according to the text data, and generating a high-resolution remote sensing image and image features of the next layer by combining the low-resolution features of the previous layer; calculating a loss function, detecting the matching degree of the generated image and the text, and generating a corresponding lossfunction; and image style conversion: taking the generated high-resolution image as a reference style image, and performing style conversion according to a cyclic consistency principle and an adversarial loss function. The method has the beneficial effects that the high-resolution images are generated layer by layer from the text data, so that the generation precision from the text to the images is greatly improved, and the vacancy of style conversion of the text data is made up.

Description

technical field [0001] The invention relates to the field of image generation, in particular to a text data-based remote sensing image style conversion method. Background technique [0002] Image generation is one of the research hotspots in the field of artificial intelligence. At present, the application of Generative Adversarial Network (GAN) extends to many fields such as video, image, text, voice, etc., especially in the field of image generation, which has achieved good results. However, there are still research gaps in style transfer from text data to images. [0003] Currently, image generation mainly includes image-to-image generation and text-to-image generation. In the original GAN, because the output only depends on random noise, it is impossible to control the content to be generated, so M. Mirza et al. proposed the CGAN algorithm in 2014. As for the text-to-image generation, the rationality and authenticity of its generation become the judging criteria of th...

AI Technical Summary

Problems solved by technology

Although StackGANs can generate images based on textual descriptions, they cannot capture the localization constraints of objects in images

Image conversion is mainly through existing images, such as the pix2pix-based data generation technology proposed by Phillip Isola et al. in 2018. This technology uses the idea of ​​generative confrontation network to achieve data style conversion, but the core of the technology is The principle is to use the mapping relationship between pixels of the same scene image, so this also requires that the training data must be input in pairs, which is simply impossible for remote sensing images

Images