Dynamic-to-static scene conversion method based on conditional generative adversarial network

Abstract

The invention relates to a dynamic-to-static scene conversion method based on a conditional generative adversarial network, and the method is characterized in that a generator is a cascade two-stage network from coarse to fine, and discriminators are PatchGAN and SN-PatchGAN. According to the method, a dynamic region binary mask is deduced according to the difference between the output result of the coarse network and the pixel value of the dynamic scene, and then static recovery is performed on the dynamic region through the fine network. After deep-layer and shallow-layer features of a finenetwork coding area are extracted, a context attention mechanism is adopted to optimize dynamic area scene generation. Compared with a traditional discriminator, the discriminator adopted by the invention can pay more attention to image details, and the training process is more stable. Compared with a traditional dynamic-to-static scene conversion method, the method has the advantages that the extracted dynamic target area is more accurate, and the generated dynamic scene image is rich in texture and closer to the real situation.

Description

technical field [0001] The invention relates to a conversion method, in particular to a dynamic-to-static scene conversion method based on a conditional generative confrontation network, and belongs to the technical field of deep learning and image generation. Background technique [0002] Dynamic scenes pose a high challenge to vision-based robot pose estimation and localization tasks. The traditional solution is to use dynamic object detection technology to distinguish the effectiveness of image regions, and only use static regions for pose estimation or positioning by discarding dynamic region information. However, when the detection of the dynamic target area is inaccurate or the dynamic area is too large, this method will lead to inaccurate or too little effective information, which in turn will lead to increased pose estimation or localization errors. In order to improve this phenomenon, a dynamic-to-static scene conversion method was proposed (Bescos B, Neira J, Sieg...

AI Technical Summary

Problems solved by technology

The existing conversion method is to use the conditional generation confrontation network (P.Isola, J.-Y.Zhu, T.Zhou, and A.A.Efros. Image-to-image translation with conditional adversarial networks [C]. CVPR, 2017), but this This method will generate all the pixels of the static scene image, although the average pixel error between the whole image and the real image can be small, but due to the influence of the mean value, the method performs poorly in image texture and details, and in robot pose Image details and texture information are particularly important in estimation and localization tasks

Images