Figure clothing conversion method and system

Abstract

The invention discloses a figure clothing conversion method and system, and the method comprises the steps: carrying out the corresponding shape change of an original segmentation image of an originalimage according to an input statement, and converting the original segmentation image into a target segmentation image; processing a synthesis problem by using a second-level generative adversarial network: training a generator to learn multi-domain mapping from an original image to a target image by taking the target segmentation image as a semantic guidance and shape limitation condition together with an input statement so as to synthesize the target image, and completing figure and costume conversion; the second-level generation network integrates the following steps: enhancing the relevance between the target picture and the input statement by adopting a soft attention layer; adopting a self-attention layer to explicitly capture long-distance correlation on the image; a stylized attention layer is adopted to establish a dependency relationship between features through channel-by-channel inner product and feature map recalibration. According to the figure clothing conversion methodand system, three attention layers are fused, and high-quality costume generation is achieved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method and system for character clothing conversion. Background technique [0002] Character clothing conversion is a very challenging task in the field of computer vision. Its goal is to convert the character clothing in the original image according to the input text description, while keeping the character's posture, identity, body shape and other information unchanged. . This character has a wide range of applications and can be extended to many emerging application scenarios such as photo editing, film production, and virtual fitting. Although generative adversarial networks have achieved excellent performance in domain transfer tasks such as face attribute conversion and makeup conversion in recent years, there is still a lot of room for improvement in character clothing conversion tasks. [0003] The challenge of the clothing conversion task is firstly reflected...

AI Technical Summary

Problems solved by technology

[0004] Second, the existing methods are difficult to meet the high-quality clothing conversion generation

The existing character clothing conversion method still uses the traditional full convolution generator. This network structure has very limited ability to capture long-distance correlation and cannot meet the requirements of high-quality generation.

Not only that, the existing methods use the overall representation of the input sentence as conditional information to train the network, which does not fully utilize the semantic information at the word level, so it is not enough to support fine-grained texture and color generation

In addition, since the transformation of character clothing may require the network to perform large-scale inference and imagination, for example, when switching from long sleeves to short sleeves, the network needs to generate new arm parts, so how to generate information that is not in the original image is also the same. However, existing methods have not explored this problem deeply enough.

Images