Multi-domain image conversion method, system and medium based on conditional generative confrontation network

A conditional generation and image conversion technology, which is applied in the field of image style transfer and multi-modal conversion of medical images, can solve the problems of not being able to guarantee cars, not being able to generate pictures, etc., and achieve the effect of flexible training

Active Publication Date: 2021-07-09
SUN YAT SEN UNIV
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] We know that classic GAN can only generate random pictures, but not specified pictures
For example, if we want to use GAN to generate a picture of a car of a specified color, GAN can only ensure that the generated picture is a car, but cannot guarantee that the generated car must be of the color we specified

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-domain image conversion method, system and medium based on conditional generative confrontation network
  • Multi-domain image conversion method, system and medium based on conditional generative confrontation network
  • Multi-domain image conversion method, system and medium based on conditional generative confrontation network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] The following will take the two-domain conversion of two modalities of x and y as an example to further describe in detail the multi-domain image conversion method and system of the present invention that can realize multi-modal conversion of medical images.

[0060] Such as figure 1 As shown, the implementation steps of the multi-domain image conversion method based on conditional generative confrontation network in this embodiment include:

[0061] 1) Input the original image x of the x mode to be converted, the original image y of the y mode;

[0062] 2) For the original image x, use a pre-trained condition extractor to produce x modal condition C x , using a pre-trained condition extractor for the original image y to produce the y-mode condition C y ;

[0063] 3) The original image x, original image y, x modal condition C x , y mode condition C y Input the pre-trained conditional generative adversarial network to get the corresponding image conversion result. ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention relates to the field of deep learning image generation, and in particular to a multi-domain image conversion method, system and medium based on conditional generative confrontation networks. Original image y; use a pre-trained condition extractor to produce x modal condition C x and the y-mode condition C y ;The original image x, original image y, x modal condition C x , y modality inputs the pre-trained conditional generative adversarial network to obtain the corresponding image conversion result. The present invention uses a feature extractor to extract the features of the original image, obtains a conditional matrix through upsampling and splicing with the zero matrix on the channel, and retains the semantic information of each modal input in the case of high independence; The invention is flexible in training, has no limit to the number of domains to be converted, and requires few parameters.

Description

technical field [0001] The invention relates to the field of deep learning image generation, in particular to a conditional generative confrontation network-based multi-domain image conversion method, system and medium, especially suitable for image style transfer and multi-modal conversion of medical images. Background technique [0002] Convolutional neural network is an important research direction in the field of deep learning (DL, Deep Learning), and has now become part of the most influential innovation in the field of computer vision. The main feature of the convolutional neural network is the convolution operation, which is good at matrix operations, and the channel of the image can generally be expressed as a two-dimensional matrix, so the convolutional neural network performs excellently on image-related tasks. [0003] Since the Generative Adversarial Network (GAN) was proposed in 2014, the image neighborhood has made great progress, many in classification, segmen...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06T3/40G06T7/33G06T9/00G06K9/62
CPCG06T3/4038G06T7/33G06T9/002G06T2207/20081G06T2207/20084G06F18/214
Inventor 邓楚富肖侬卢宇彤陈志广瞿毅力苏婉琪王莹
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products