Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks

A self-encoding and sketching technology, applied to biological neural network models, using manual input to modify/create images, neural architectures, etc., can solve problems such as the inability to capture long-distance dependencies between strokes, small receptive fields, and few data, etc., to achieve Generate effects that are diverse and realistic, avoid possibilities, and improve quality

Pending Publication Date: 2020-07-31
HUAZHONG UNIV OF SCI & TECH
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The research on sketches is mainly in recognition, and achieved good results to a certain extent, but there is only a small amount of research on sketch generation.
In 2018, a seq2seq variational self-encoder (VAE) model called Sketch-RNN proposed by Google, which can generate sketches based on manual input, was introduced as the first artificial intelligence technology and is known as the most advanced technology ; but the model also has its own limitations, including: it can have a better output when learning a single category sketch, but when multiple categories are input into the model, the quality of the generated sketch will decrease; when a sketch is composed of multiple components , Sketch-RNN cannot capture the relative positional relationship between components very well, resulting in low quality of generated sketches
However, the quality of the sketch generated by this model needs to be improved. The encoder part of the Sketch-pix2seq model has a small receptive field, which can only learn the stroke relationship in a local range, but cannot capture the long-distance dependency between strokes, resulting in captured The effective information is limited, the sketch quality is reduced, and the Sketch-RNN and Sketch-pix2seq models are both based on the VAE generation framework
[0003] To sum up, the problems existing in the existing technology are: (1) The existing sketch generation schemes are all based on the VAE model, and the generation framework is single
[0004] (2) At present, Sketch-pix2seq, the best model for generating sketches, cannot learn the long-distance relationship between strokes when learning sketch features, and cannot better grasp the feature information of sketches from a global perspective.
[0005] The difficulty of solving the above technical problems: At present, the research on hand-drawn sketches has become a branch of computer vision. Previous research has mainly focused on the recognition of hand-drawn sketches, and achieved good results to a certain extent, but in hand-drawn sketches There is only a small amount of research on the generation of sketches, and there are few verifiable materials; in addition, there is no clear standard for the quality evaluation of sketches, and the evaluation of sketches is mainly based on human subjective evaluation.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks
  • Hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks
  • Hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0090] Such as figure 1 and figure 2 As shown, the embodiment of the present invention provides a sketch generation method, including:

[0091] (1) Obtain a vector format sketch of a certain category or categories, and perform format conversion to obtain the corresponding raster format sketch, and form a new data set from the vector format sketch and the raster format sketch;

[0092] Specifically, the sketch data in vector format provided by the embodiment of the present invention refers to a picture composed of quintuples consisting of data point offset and change status information. No matter whether the picture in this format is enlarged, reduced or rotated, it will not be distorted. ; The present invention uses the QuickDraw data set to download different types of vector format pictures, and utilizes the cairoSVG graphics library to perform format conversion, thereby obtaining the required data sets for model training.

[0093] (2) The new data set, using convolutiona...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of image data processing, and discloses a hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks. The method comprises the following steps: obtaining a vector format sketch, performing format conversion, and constructing a new data set; obtaining global and local feature structures of stroke vectors inthe new data set; utilizing a decoder to obtain stroke normal distribution parameter data and stroke state classification distribution probability data of the new data set; randomly sampling from thenormal distribution and calculating a stroke state to obtain prediction output of each time step, and obtaining a sketch generated by prediction; obtaining true and false information of the input data by using a discriminator to obtain a sketch generation model fusing VAE and GAN; and inputting the data into a generation model to obtain a prediction generation sketch. According to the sketch generation method, the generated sketch is high in quality, and generation of multiple types of sketches is supported.

Description

technical field [0001] The invention belongs to the technical field of image data processing, and in particular relates to a hand-drawn sketch generation method based on variational self-encoding and generative confrontation network. Background technique [0002] As early as ancient times, sketches have played an important role in assisting human communication and creative design. At the same time, vector graphics require only the least space for storage and transmission during transmission. This situation promotes the development of sketches in artificial intelligence technology. develop. The research on sketches is mainly on recognition, and has achieved good results to a certain extent, but there is only a small amount of research on sketch generation. In 2018, a seq2seq variational self-encoder (VAE) model called Sketch-RNN proposed by Google, which can generate sketches based on manual input, was introduced as the first artificial intelligence technology and is known a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T11/80G06N3/04
CPCG06T11/80G06N3/044G06N3/045
Inventor 李红云王海卫曾志豪王荣耀
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products