Unlock instant, AI-driven research and patent intelligence for your innovation.

Image synthesis method and device, electronic equipment and computer readable storage medium

An image synthesis and sub-network technology, applied in the field of image processing, can solve the problems of inconsistent color of left and right eyebrows, asymmetry of left and right eyes, affecting the display effect of natural images, etc.

Pending Publication Date: 2021-05-18
MEGVII BEIJINGTECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the labels corresponding to each part in the semantic segmentation image are different, when synthesizing natural images, the processing methods of each part are also different, resulting in large differences between similar parts in the final synthesized natural image, for example The left and right eyes are asymmetrical, the color of the left and right eyebrows is inconsistent, etc., which affect the display effect of natural images

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image synthesis method and device, electronic equipment and computer readable storage medium
  • Image synthesis method and device, electronic equipment and computer readable storage medium
  • Image synthesis method and device, electronic equipment and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

preparation example Construction

[0049] First, please refer to figure 1 , the embodiment of the present application provides an image synthesis method, which may include the following steps:

[0050]Step S110: Obtain a semantic segmentation map including a plurality of local regions and a sampling noise map of the same scale as the semantic segmentation map, each of the local regions has a class label for representing its category.

[0051] Step S120: Convert the semantic segmentation map into semantic and appearance-related semantic vectors according to the pre-acquired similarity information between objects represented by each class label.

[0052] Step S130: Obtain a composite image by processing the semantic vector and the sampled noise map.

[0053] Among them, when the method can be applied to image synthesis network.

[0054] like figure 2 As shown, the image synthesis network 200 may include a semantic vector generation sub-network 210 and a semantic rendering sub-network 220 .

[0055] Among the...

Embodiment approach

[0073] As mentioned above, the semantic vector generation sub-network 210 of the image synthesis network 200 may include a feature extraction unit 211 , and correspondingly, the semantic rendering sub-network 220 of the image synthesis network 200 also includes a residual calculation unit 221 . In this embodiment, the target semantic segmentation map can be directly input into the image synthesis network 200 (equivalent to downsampling the target semantic segmentation map with a preset multiple of 1), at this time, the feature extraction unit 211 obtains The first input information of is the target semantic segmentation map.

[0074] Furthermore, in some implementations, when the semantic vector generation sub-network 210 includes N (for example, 3) feature extraction units 211 , correspondingly, the semantic rendering sub-network 220 also includes 3 residual calculation units 221 . In this embodiment, scale conversion can be performed on the target semantic segmentation map i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to an image synthesis method and device, electronic equipment and a computer readable storage medium, and belongs to the field of image processing. According to the method, the feature value of the feature map is adjusted according to the similarity information between the objects represented by the class labels, so that corresponding scalar quantities used for representing similar areas (such as a left eye area and a right eye area) in the feature map in a subsequently generated semantic vector are similar. Therefore, the subsequent processing parameters of the residual calculation unit for processing the similar area are similar, so that the similar area presents a similar effect in the finally obtained composite image, and the display effect can be improved.

Description

technical field [0001] The present application belongs to the field of image processing, and in particular relates to an image synthesis method, device, electronic equipment and computer-readable storage medium. Background technique [0002] The semantically segmented image is input to the adversarial network to convert the semantically segmented image into a more realistic natural image. Since the labels corresponding to each part in the semantic segmentation image are different, when synthesizing natural images, the processing methods of each part are also different, resulting in large differences between similar parts in the final synthesized natural image, for example The left and right eyes are asymmetrical, and the color of the left and right eyebrows is inconsistent, etc., which affect the display effect of natural images. [0003] Through the appearance similarity, a connection is established between each local area, so that when each local area is converted subsequ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/34G06K9/46G06K9/62G06N3/04G06N3/08
CPCG06N3/084G06V10/267G06V10/44G06N3/048G06N3/045G06F18/22
Inventor 王毅亓鲁张祥雨
Owner MEGVII BEIJINGTECH CO LTD