Image style migration model training method, system and device and storage medium

Abstract

The invention discloses a training method, system and device of an image style migration model and a storage medium. Three parts of losses are designed to train the image style migration model: 1) the optimal balance of a generator and a discriminator can be achieved through the total confrontation loss; the method has the advantages that the method is simple, the reconstruction effect of the generator can be guaranteed through the total circulation consistency loss, the fidelity degree of the output image of the generator can be improved through the comparison loss, the image style migration model is trained by integrating the three losses, the image style migration capability can be improved, and the better image after style migration can be obtained.

Description

technical field [0001] The present invention relates to the technical field of image style transfer, in particular to a training method, system, device and storage medium for an image style transfer model. Background technique [0002] With the development of computer science and the improvement of modeling capabilities, computers can simulate virtual scenes that are very similar to real scenes, thereby batch-producing simulated virtual images and labels that can be used for other computer vision tasks. However, due to the limitations of related technologies and the complexity of real scenes and many other factors, it is difficult for virtual images to be completely consistent with the style of real images, which will not only affect the user's visual experience, but also reduce its performance in many downstream tasks to a certain extent. applications, such as target detection, semantic segmentation, etc. Therefore, the style transfer task came into being, by preserving th...

AI Technical Summary

Problems solved by technology

Specifically, the method based on cycle consistency usually encodes the source domain image into a low-dimensional latent space, transforms the domain-related image information from the source domain to the target domain in the low-dimensional latent space, and uses the transformed image information to reconstruct the target Domain-style images. In this process, two pairs of generators and discriminators are used, and the images migrated to the target domain are required to be converted back to the source domain. Most current methods based on cycle consistency use pixel-level constraints, so , most of them have problems such as deformation and style disorder

Decoupling-based methods tend to decouple the source and target domain images into domain-invariant content features that remain unchanged during the transformation process and domain-specific style features that change during the transformation process, and the transformation is achieved by preserving the content features and replacing the style features. purpose, but there is a problem of content leakage

[0004] In the Chinese patent application with publication number CN113808011A "A Style Migration Method, Device and Related Components Based on Feature Fusion", the decoupling-based style migration method is used to encode the style and content of the image separately, and then the required The content features and style features of this method are fused, and the final migration result is output by the decoder, which can improve the quality of content details and the color similarity with the target domain. There is the problem of content leakage; in the Chinese patent application "Image Migration Method Based on Mean Standard Deviation" with the publication number CN113837926A, the features are normalized in different levels of feature spaces, and PSNR and SSIM are performed through feature maps and source images In contrast, it reduces the time required to train the model and reduces the distortion and artifacts of image features. However, this method focuses on improving the training efficiency and does not significantly improve the quality of the generated image; in the Chinese patent application with the publication number CN107705242A In "An Image Stylization Migration Method Combining Deep Learning and Depth Perception", the content loss, style loss and depth of field loss are calculated on the output feature maps of the perceptual loss network and the depth perceptual network respectively, which improves the three-dimensionality of the generated image. But there are still some situations like corresponding semantic content mismatch and object distortion, so overall, the transfer ability of this scheme is not good

Images