Autonomous image enhancement system

Abstract

The invention relates to an autonomous image enhancement system, and aims to solve the problems that an existing image denoising and super-resolution technology is too high in dependence on large-scale annotation data, insufficient in model generalization ability and insufficient in detail recovery. The method comprises the following steps: firstly, aiming at an input original image, generating a plurality of different views through a data enhancement technology, and inputting the views into a ResNet encoder to extract multi-scale features; thirdly, constructing positive and negative sample pairs by utilizing a comparative learning module, maximizing the similarity of the positive sample pairs and minimizing the similarity of the negative sample pairs through an InfoNCE loss function, and completing self-supervised feature learning; then, the U-Net decoder performs up-sampling layer by layer and combines jump connection to recover image details and generate a de-noised or super-resolution image. And finally, combining the comparison loss and the reconstruction loss to optimize feature learning and image reconstruction performance. The method can be applied to processing of medical images, remote sensing images and video monitoring images, and the image quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an autonomous image enhancement system. Background Art

[0002] With the rapid development of computer vision technology, image denoising and super-resolution technologies have been widely used and received attention in the fields of medical image processing, remote sensing image analysis, and video surveillance. The inventors understand that the main methods currently used in image denoising and super-resolution tasks at home and abroad are as follows: The first is based on traditional image processing methods, such as Gaussian filtering, bilateral filtering, and non-local means (NLM), which use fixed image filters to remove noise or perform image enhancement. Although these methods can achieve certain results in some cases, their performance is limited because they cannot capture complex image structure features. The second is deep learning methods based on supervised learning, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs). These methods are usually trained through large-scale annotated data sets (such as clean-noise image pairs or low-resolution-high-resolution image pairs), which can extract multi-level feature information and effectively improve image quality. However, training these deep learning models relies on a large amount of high-quality annotated data, and obtaining this data is costly and time-consuming, especially in some specific fields (such as medical imaging and remote sensing images) where annotated data is relatively scarce. The third is the image enhancement method based on weak supervision or semi-supervision. This type of method is trained with limited labeled data and combined with unlabeled data to improve the generalization ability of the model. However, the weak supervision method still relies on a certain amount of labeled data, and the denoising or super-resolution effect of the model is unstable when facing complex image scenes. Among the above methods, although the first method can remove some noise by filtering, it is easy to cause loss of details. The second method performs well when there is large-scale annotated data, but its dependence on annotated data limits the scope of application. The third method has limited performance when there is insufficient annotated data and cannot fully tap the potential of the deep learning model. The present invention combines contrastive learning with the ResNet-UNet architecture to address the deficiencies in the prior art, improve the performance of image denoising and super-resolution by using self-supervised learning, reduce dependence on annotated data, and achieve a wider range of application scenarios. Summary of the invention

[0003] The purpose of the present invention is to avoid the shortcomings of existing image denoising and super-resolution technologies, such as high dependence on large-scale annotated data, insufficient model generalization ability, and limited detail recovery ability, and propose an autonomous image enhancement system. Contrastive learning is introduced into the ResNet-UNet framework, the joint loss function is optimized, and in the architecture combining ResNet and U-Net, the effective combination of deep and shallow layer features is guaranteed through the jump connection mechanism, which significantly improves the ability to restore image details, and is particularly suitable for image processing of complex scenes. The present invention uses unlabeled data for denoising and super-resolution processing through self-supervised learning, effectively reducing the dependence on labeled data and improving the generalization ability and effect of the model in different scenarios. The technical solution is as follows: An autonomous image enhancement system, the steps of which are as follows: (1) Image preprocessing and view enhancement: Unlabeled image data is read from the original image set, and data enhancement is performed on each input image. The enhancement methods include random cropping, rotation, noise, blur, etc., to generate multiple different views. The generated views will be used as positive sample pairs, and the views of other different images will be used as negative sample pairs for comparative learning. (2) ResNet encoder: Each enhanced image is input into the ResNet encoder to extract the feature representation of the image. The jump connection of each residual block ensures the transmission of shallow and deep features. Through multiple residual blocks, the deep features of the image are effectively extracted. (3) Contrastive learning module: The InfoNCE loss function is used to calculate the contrast loss, which guides the model to learn the potential structural features of the image by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs: Among them, z i and z j is the feature representation of the positive sample pair, z k is the feature representation of negative samples, sim represents the similarity between features, and τ is the temperature coefficient. Among them, z i and z j is the feature representation of the positive sample pair, z k is the feature representation of negative samples, sim represents the similarity between features, and τ is the temperature coefficient. (4) U-Net decoder: The output feature map of the ResNet encoder is input to the U-Net decoder through a skip connection. U-Net gradually restores the details of the image by upsampling layer by layer. The skip connection ensures that the high-resolution features in the encoder can be directly passed to the corresponding layer of the decoder to help restore the details of the image. The U-Net decoder combines the high-resolution features of the shallow layer and the global features of the deep layer, restores the details of the image through multi-scale fusion, and finally outputs a high-resolution denoised image or a super-resolution image. (5) Joint loss function design: First, we use perceptual loss to extract high-level features of the image through the pre-trained network VGG to measure the perceptual differences: Among them, Φ represents the high-level features extracted by the pre-trained network. Then, the contrast loss and reconstruction loss are jointly optimized, and feature learning and image generation are comprehensively optimized. The total loss function is: Among them, α and β are weight parameters used to adjust the balance between contrast loss and reconstruction loss. (6) Iterative training: In each iteration, unlabeled image data is input and the model parameters are continuously optimized through the above steps until convergence. The beneficial effects of the present invention are: The present invention can reduce the reliance on large-scale annotated data and achieve efficient self-supervised learning by innovatively combining the contrastive learning method with the ResNet-UNet architecture. The model has strong generalization ability and can effectively restore image details under different noise conditions and resolutions, significantly improving the effects of denoising and super-resolution. Through multi-view generation and joint loss optimization, the present invention can not only improve training efficiency, but also enhance the adaptability of the model, so it is expected to be widely used in various image processing scenarios such as medical imaging, remote sensing and video surveillance. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a network structure diagram of the present invention; DETAILED DESCRIPTION

[0005] To make the technical solution of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. The present invention is specifically implemented according to the following steps: (1) Preprocessing of input images and generation of multi-views: Read the original image set X = {x 1 , x 2 , …, x N}, for each input image x i Perform various data enhancement operations to generate different views Vi = {v i1 , v i2 , …, v ik}, where each v ij The images generated by data augmentation operations include random cropping, random rotation, noise addition, and Gaussian blur. Each image x i Generate two views v through data augmentation i1 =t 1 (x i ) and v i2 =t 2 (x i ), where t 1 and t 2 is a randomly selected data augmentation operation. (2) ResNet encoder feature extraction: Input image view V i = {v i1 , v i2 , …, v ik} to the ResNet encoder to extract its deep feature representation. ResNet consists of several residual blocks, each of which has the following formula: in, is the input feature of the lth layer, F(·) is the convolution operation, and w (l) is the weight parameter of the lth layer. ResNet extracts multi-scale features layer by layer, and the shallower features Capture local information of the image, deeper features Capturing global information, the output feature is expressed as: z ij =ResNet(v ij ) Among them, v ij is the jth view, z ij is the feature vector of the view. (3) Contrastive learning module: Contrasting different views of the same image (v i1 , v i2 ) as a positive sample pair and calculate its feature representation (z i1 , z i2 ); the view pairs of different images (v i1 , v j2 ), where i≠j, is a negative sample pair, and its feature representation (z i1 , z j2 ), and use cosine similarity to measure the similarity between positive and negative sample pairs: Among them, z 1 and z 2 is the feature representation of the two image views. InfoNCE loss is used to optimize the contrastive learning process, with the goal of maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs: Here, τ is a temperature parameter that controls the sensitivity to similarity. (4) Decoding and image reconstruction: The feature map output by the ResNet encoder is input into the U-Net decoder, and the feature map is gradually restored to the original resolution image. There is a jump connection between each convolution layer of the U-Net and the corresponding decoding layer, which directly passes the multi-scale features of the encoder to the decoder to ensure that the detailed information can be preserved during the decoding process. The deconvolution layer is used to upsample layer by layer to gradually restore the low-resolution feature map to a high-resolution image. Each time the decoder upsamples, it fuses the current feature map with the corresponding jump connection feature map to generate a more refined image. Finally, the decoder outputs a high-resolution image I resolved . (5) Joint loss function design: Using perceptual loss, we extract high-level features of the image through the pre-trained network VGG to measure the perceptual differences: Among them, Φ represents the high-level features extracted by the pre-trained network. The final loss function is composed of a weighted combination of contrast loss and reconstruction loss, taking into account both feature learning and image restoration tasks: Among them, α and β are weight parameters used to adjust the balance between contrast loss and reconstruction loss. (6) Iterative training: After multiple iterative training, when the joint loss function When it converges to the minimum value, the model is trained.

Claims

1. An autonomous image enhancement system, characterized in that: The system uses a self-supervised learning framework, uses the ResNet encoder to extract multi-scale features, constructs positive and negative sample pairs for comparative learning, and combines the U-Net decoder to gradually restore image details through jump connections. It uses a joint loss function to achieve efficient image denoising and super-resolution processing without labeled data: (1) Image preprocessing and view enhancement: Unlabeled image data is read from the original image set, and data enhancement is performed on each input image. The enhancement methods include random cropping, rotation, noise, blur, etc., to generate multiple different views. The generated views will be used as positive sample pairs, and the views of other different images will be used as negative sample pairs for comparative learning. (2) ResNet encoder: Each enhanced image is input into the ResNet encoder to extract the feature representation of the image. The jump connection of each residual block ensures the transmission of shallow and deep features. Through multiple residual blocks, the deep features of the image are effectively extracted. (3) Contrastive learning module: The InfoNCE loss function is used to calculate the contrast loss, which guides the model to learn the potential structural features of the image by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs: Among them, z i and z j is the feature representation of the positive sample pair, z k is the feature representation of negative samples, sim represents the similarity between features, and τ is the temperature coefficient. (4) U-Net decoder: The output feature map of the ResNet encoder is input to the U-Net decoder through a skip connection. U-Net gradually restores the details of the image by upsampling layer by layer. The skip connection ensures that the high-resolution features in the encoder can be directly passed to the corresponding layer of the decoder to help restore the details of the image. The U-Net decoder combines the high-resolution features of the shallow layer and the global features of the deep layer, restores the details of the image through multi-scale fusion, and finally outputs a high-resolution denoised image or a super-resolution image. (5) Joint loss function design: First, we use perceptual loss to extract high-level features of the image through the pre-trained network VGG to measure the perceptual differences: Among them, Φ represents the high-level features extracted by the pre-trained network. Then, the contrast loss and reconstruction loss are jointly optimized, and feature learning and image generation are comprehensively optimized. The total loss function is: Among them, α and β are weight parameters used to adjust the balance between contrast loss and reconstruction loss. (6) Iterative training: Input unlabeled image data and iterate round by round. Optimize the model parameters through the above steps until convergence, and obtain the best model for testing and predictive analysis.

Images