Multi-modal image enhancement method based on orthogonal element space

Abstract

The invention discloses a multi-modal image enhancement method based on orthogonal element space, and belongs to the field of computer vision. The method is characterized by comprising the following steps: 1, extracting a style and a content code of a high-aesthetic-quality image by using a coder-decoder and a mutual information optimization strategy; 2, mapping the style code of the reference image into a style element space formed by a group of orthogonal base sheets; 3, improving the style and content coding decoupling of the reference image by using a self-adaptive instance standardizationmodule and a mutual information optimization feature decoupling method, and constructing a generative adversarial network based on coding-decoding to perform model training; and 4, in a test stage, inputting any common image into the trained model, extracting content codes by a content encoder, randomly sampling a plurality of style codes in a style meta-space, fusing the content codes and the style codes, and sending the fused codes into a generator to obtain a multi-modal enhanced image. According to the invention, the enhanced image with various styles in the aspects of aesthetic characteristics such as brightness, contrast, color and the like can be obtained.

Description

Technical field [0001] The invention belongs to the field of deep learning and computer vision, and particularly relates to a multi-modal image enhancement method based on orthogonal element space. Background technique [0002] With the popularity of smart phones, people prefer to take pictures with portable devices such as mobile phone cameras. However, since the current non-professional equipment such as mobile phone cameras need to take into account factors such as convenience and low cost, the sensors and lenses are relatively small, and there are inevitable limitations in hardware design. More importantly, due to the variability of lighting and scenes, the photos taken may contain noise, incomplete color distribution, and limited resolution and dynamic range. In view of these factors, non-professional users may be disappointed with their unsightly photos because they do not match their desired visual experience. Therefore, it is of great significance to reconstruct and enh...

AI Technical Summary

Problems solved by technology

Another method of feature decoupling through the gradient inversion method modifies the gradient of the encoder-decoder, and the forced constraint model cannot reconstruct the effect of the original image according to a single encoding, but there are problems such as difficulty in training

Images