Multi-domain image conversion method and system based on conditional generative adversarial network, and medium

Abstract

The invention relates to the field of deep learning image generation, in particular to a multi-domain image conversion method and system based on a conditional generative adversarial network, and a medium, and the method comprises the implementation steps: inputting to-be-converted x-modal original images x and y-modal original images y; adopting a pre-trained condition extractor to generate an xmodal condition Cx and a y modal condition Cy; and inputting the original image x, the original image y, the x modal condition Cx and the x modal condition y into a pre-trained condition generative adversarial network to obtain a corresponding image conversion result. According to the method, the characteristics of the original image are extracted by utilizing the characteristic extractor, the condition matrix is obtained through up-sampling and splicing with the zero matrix on the channel, and the semantic information of each modal input is kept under the condition of higher independence; andtraining is flexible, the number of domains to be converted is not limited, and few parameters are needed.

Description

technical field [0001] The invention relates to the field of deep learning image generation, in particular to a conditional generative confrontation network-based multi-domain image conversion method, system and medium, especially suitable for image style transfer and multi-modal conversion of medical images. Background technique [0002] Convolutional neural network is an important research direction in the field of deep learning (DL, Deep Learning), and has now become part of the most influential innovation in the field of computer vision. The main feature of the convolutional neural network is the convolution operation, which is good at matrix operations, and the channel of the image can generally be expressed as a two-dimensional matrix, so the convolutional neural network performs excellently on image-related tasks. [0003] Since the Generative Adversarial Network (GAN) was proposed in 2014, the image neighborhood has made great progress, many in classification, segmen...

AI Technical Summary

Problems solved by technology

[0004] We know that classic GAN can only generate random pictures, but not specified pictures

For example, if we want to use GAN to generate a picture of a car of a specified color, GAN can only ensure that the generated picture is a car, but cannot guarantee that the generated car must be of the color we specified

Images