Image attribute editing method based on structured scene and text description

An attribute editing and structuring technology, applied in the field of image processing, can solve problems such as complex and redundant network structure, long training time, excessive modification of semantically irrelevant parts, etc., to avoid excessive modification, solve unstable operation, improve efficiency and accuracy degree of effect

Active Publication Date: 2021-10-08
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF11 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to address the above-mentioned deficiencies in the prior art, and propose a method for editing image attributes based on structured scenes and text descriptions, and adopt a network structure, fusion module a

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image attribute editing method based on structured scene and text description
  • Image attribute editing method based on structured scene and text description
  • Image attribute editing method based on structured scene and text description

Examples

Experimental program
Comparison scheme
Effect test

experiment example 1

[0135] The experimental conditions are set as follows: system: Ubuntu 18.04, software: Python 3.6, processor: Intel Xeon(R) CPU E5-2620 v4@2.10GHz×2, memory: 256GB.

[0136] Simulation experiment 1: use the method of the present invention to use the pictures and modified descriptions of flowers and birds as input to generate corresponding real pictures, the results are as follows Figure 4 shown;

[0137] Simulation experiment 2: Using the method of the present invention to use animals, vehicles and modified descriptions as input to generate corresponding real pictures, the results are as follows Figure 5 shown;

[0138] Simulation experiment 3: using the method of the present invention to use the bird and different modified descriptions as input to generate corresponding real pictures, the results are as follows Image 6 shown.

[0139] from Figure 4 It can be seen that the real image generated by the method of the present invention has a clear picture, rich and reasona...

experiment example 2

[0141] Experimental conditions: System: Ubuntu 18.04, Software: Python 3.6, Processor: Intel Xeon(R) CPUE5-2620 v4@2.10GHz×2, Memory: 256GB.

[0142] The data sets used in this experiment example exist in the form of graphic-text pairs, that is, a real picture corresponds to several text descriptions, and each algorithm is used in turn to train the training set in the data set. After the training is completed, each algorithm is used to match the text description of the real picture on the test set of the data set to generate the corresponding picture.

[0143] In the experiment, the test set is randomly divided into several batches, and each batch includes 100 image-text pairs (x, t). In each batch, a pairwise generation Unpaired generation in There is no corresponding real sample in the dataset. Repeat the experiment 10 times on the test set and take the average.

[0144] (1) Investigate the quality and diversity of generated images:

[0145] Pre-trained on the ImageN...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an image attribute editing method based on a structured scene and text description, and provides an implementation scheme of first structuring a picture and then editing the picture for the first time, the picture modification efficiency and accuracy can be effectively improved through understanding and decoupling of a picture scene, and a semantic irrelevant part is prevented from being excessively modified. According to the method, a multi-scale feature fusion mechanism more suitable for the task is adopted on a network structure, and return segmentation loss, hierarchical semantic matching perception, adversarial loss of an image quality discriminator, image consistency loss and image-text similarity loss are combined on a loss function, so that the final sum performance of the whole method is ensured; the problems that an existing image attribute editing method is unstable in operation, difficult to expand and large in resource overhead are solved.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to the design of an image attribute editing method based on structured scenes and text descriptions. Background technique [0002] With the development of deep neural networks, technologies such as image classification, image segmentation, and image object detection have been relatively mature and widely used. However, due to the high quality and high resolution requirements of image generation related technologies, there are often problems such as long model training time, high overhead, and unstable training, and have not been widely applied. [0003] Among them, the text-guided image editing task is a task with strong functionality and broad application prospects. It uses the text description given by the user to edit the input image, keeps the overall style unchanged, and effectively performs semantic-related attributes. modification to generate the corresp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/11G06T3/00G06K9/62G06N3/04G06N3/08G06F16/532G06F16/583
CPCG06T7/11G06T3/0012G06N3/08G06F16/532G06F16/583G06T2207/10004G06T2207/20081G06T2207/20084G06T2207/30168G06N3/045G06F18/22G06F18/214G06F18/253Y02T10/40
Inventor 高联丽赵启轲朱俊臣苏思桐申恒涛
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap