An improved Retinex-Net vignetting image correction method
Through the improved Retinex-Net vignetting image correction method, the decomposition and correction network combines the dense residual network to process vignetting images, solving problems such as noise enhancement and color distortion in the prior art, and achieving high-quality image correction effects.
Patent Information
- Application Number
- CN202310498094.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-05
- Publication Date
- 2025-05-13
- Estimated Expiration
- 2043-05-05
AI Technical Summary
The prior art has problems such as increased noise, excessive exposure, color distortion and increased noise in vignetting image correction, and traditional algorithms require manual parameter adjustment and cannot adapt to images in different scenarios.
Using the improved Retinex-Net vignetting image correction method, the vignetting image is decomposed into illumination components and reflective components by decomposing the network, and enhanced and denoising processing is performed using the correction network, and denoising is performed through the dense residual network, and the two components after processing are finally reconstructed.
High-quality vignetting image correction is achieved, noise and color distortion is reduced, correction effect and processing performance are improved, and the corrected image is close to the original image, with higher practicality and robustness.
Smart Images
Figure CN116579941B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of digital image processing, and in particular to an improved Retinex-Net vignetting image correction method. Background Art
[0002] High-quality images play a vital role in computer vision tasks. Images captured by cameras often degrade when not properly processed. Vignetting is a manifestation of image degradation. Image vignetting is mainly divided into internal vignetting and physical vignetting. Physical vignetting is manifested as a sudden strong darkening in the corners of the image. The main reason is that there is a large area of occlusion in front of the lens or the lens hood is used incorrectly when taking pictures. Internal vignetting usually manifests as a slight darkening from the center to the four corners of the image. This vignetting is caused by the inside of the lens or the body. Generally, zoom lenses are prone to this vignetting. The presence of different degrees of vignetting will cause the edge of the picture to lose many details, blur the edge information, and cause large color deviations. Vignetting is unavoidable when industrial cameras collect images.
[0003] In recent years, through the continuous research of scholars at home and abroad, several methods for correcting image vignetting have been proposed. Generally, these methods are divided into: vignetting correction processing algorithm based on histogram equalization, vignetting image enhancement processing based on gamma correction, etc. However, these correction methods have some disadvantages: constraints or imperfect prior knowledge cause noise enhancement, overexposure, color distortion, increased noise, etc., and cannot achieve satisfactory results.
[0004] In the field of vignetting image correction, traditional correction algorithms require manual parameter adjustment and cannot adapt to vignetted images collected in different scenes. Every time the image of a different scene is changed, the algorithm has to recalculate the parameters, which leads to a significant reduction in the accuracy and flexibility of the correction algorithm. With the development of machine learning, the introduction of convolutional neural networks has largely solved the problems existing in the field of image processing. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] In view of the deficiencies in the prior art, the present invention provides an improved Retinex-Net vignetting image correction method, which solves the problems raised in the above background technology.
[0007] (II) Technical solution
[0008] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:
[0009] An improved Retinex-Net vignetting image correction method includes the following steps:
[0010] Step 1: Image preprocessing, batch processing of the size and format of the collected image set;
[0011] Step 2: Input the paired vignetted and control images into the trained improved Retinex-Net model;
[0012] Step 3: Pass the vignetted image and the control image input in step 2 through the decomposition network to obtain the illumination component I after decomposition of the control image normal and the reflected component R normal , the illumination component I after the vignetted image is decomposed Vignetting and the reflected component R Vignetting . Will I Vignetting , R Vignetting The input is sent to the correction network for enhancement and denoising respectively;
[0013] Step 4: The correction network is a multi-branch network, and the I input in step 3 Vignetting The enhanced image I of the illumination component is obtained by enhancement processing through the correction network enhance , R Vignetting The enhanced image R of the reflected component after denoising is obtained through the residual dense block (RDB) denoising network. enhance ;
[0014] Step 5: The two component images R processed in step 4 enhance and I enhance Perform fusion reconstruction to obtain the restored image after correction.
[0015] Furthermore, the 1000 pairs of vignetted images and control images collected in steps 1 and 2 are batch-processed into RGB images with a size of 21.17 cm×14.11 cm and a png format, which are input into the improved Retinex-Net model for training.
[0016] Furthermore, in step 3, the network structure of the decomposition module is as follows: the paired vignetted image and the control image input in the input layer share weights, and the two images are decomposed into corresponding illumination component images and reflection component images using two constraints. The specific process is as follows: in the hidden layer, the first layer first performs feature extraction through convolution, and then passes through two layers of convolution + ReLu activation layers, and uses the constraint function of the two images to obtain the illumination component images I of the two images. normal and I Vignetting and the reflected component image R normal and R VignettingIn the output layer, the extracted multi-channel features are projected into illumination and reflection components through a convolutional layer of size 3x 3, and then the Sigmoid function is used to constrain the variables to ensure that the threshold is positive.
[0017] Furthermore, in step 4, the correction network includes two modules;
[0018] The first module in the correction network is the enhancement module, which inputs R in the input layer Vignetting Image and I Vignetting Image, hidden layer has five layers, first through two layers of convolution ReLu activation layer, the purpose is to perform downsampling operation, clarify the illumination component of large-scale image, and at the same time make jump connection with mirror upsampling layer, reconstruct local illumination distribution of image, let the network force to learn residual, then through a layer of 1x 1 convolution layer to simplify the multi-channel features after multi-scale splicing, and finally through a 3x 3 convolution output layer, reconstruct illumination component image;
[0019] The second module in the correction network is the hole convolution network module, which has eight layers of convolution, and its purpose is to improve the receptive field. Vignetting The image passes through the dilated convolutional network and the enhanced sub-network in parallel, and finally merges through a convolutional layer of size 1x 1 to obtain the enhanced image I of the illumination component. enhance .
[0020] Furthermore, in step 4, in the reflection denoising network, in order to make full use of all features in the reflection component image and improve network performance, the present invention uses a dense residual network in the denoising network;
[0021] The first layer of the RDB network is convolution + ReLu activation function. After convolution extraction, the features are activated by ReLu to form a nonlinear feature map. The second layer of the network is a residual block, which contains three dense blocks. Each dense block contains 4 layers of convolution + ReLu activation function. Each convolution layer obtains the output of all previous convolution layers in the module. The adjacent convolutions are short-connected, and all convolution layers are jump-connected. The structure of the third to eleventh layers is convolution + ReLu activation + BN, that is, feature mapping is performed after feature extraction, and the features are normalized. The final output layer is a convolution layer of size 3x 3 to obtain the denoised reflection component after training.
[0022] (III) Beneficial effects
[0023] Compared with the prior art, the present invention provides an improved Retinex-Net vignetting image correction method, which has the following beneficial effects:
[0024] The present invention combines Retinex theory and convolutional neural network, decomposes the vignetted image into an illumination component image and a reflection component image by using constraint conditions, enhances the illumination component image, expands the receptive field through a dilated convolutional network, inputs the reflection component image into a dense residual network for denoising, and then fuses and reconstructs the two processed components to obtain a corrected restored image. Compared with other correction algorithms, the algorithm of the present invention has superior processing effect and processing performance. The corrected restored image is very close to the original image, and its practicality and robustness are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a flowchart of the present invention;
[0026] Figure 2 This is the improved Retinex-Net network model diagram of the present invention;
[0027] Figure 3 This is a schematic diagram of the dense residual denoising network structure of the present invention;
[0028] Figure 4 A schematic diagram of the present invention for processing vignetting image correction;
[0029] Figure 5 It is a schematic diagram of image evaluation indexes of result images of different vignetting correction methods of the present invention. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0031] Example
[0032] like Figure 1-5 As shown, an improved Retinex-Net vignetting image correction method proposed in one embodiment of the present invention includes the following steps:
[0033] Step 1: Image preprocessing, batch processing of the size and format of the collected image set;
[0034] The collected 1000 pairs of vignetted images and control images were batch-processed into RGB images with a size of 21.17 cm × 14.11 cm and a png format.
[0035] Step 2: Input the paired vignetted and control images into the trained improved Retinex-Net model.
[0036] like Figure 1 As shown, an improved Retinex-Net vignetting image correction method also includes: the main idea is to train the vignetted image end-to-end through three sub-networks: a decomposition network, a correction network, and a fusion network, so as to obtain a restored image after vignetting correction.
[0037] The network structure of the decomposition module is as follows: the paired vignetted image and the contrast image are input in the input layer, and the two images share weights. The two images are decomposed into corresponding illumination component images and reflection component images using two constraints. The specific process is as follows: in the hidden layer, the first layer performs feature extraction through convolution, and then passes through two layers of convolution + ReLu activation layers, and uses the constraint function of the two images to obtain the illumination component images I of the two images. normal and I Vignetting and the reflected component image R normal and R Vignetting ; In the output layer, the extracted multi-channel features are projected into the illumination component and the reflection component through a convolutional layer of size 3x 3, and then the Sigmoid function is used to constrain the variables to ensure that the threshold is positive.
[0038] Step 3: Pass the vignetted image and the control image input in step 2 through the decomposition network to obtain the illumination component I after decomposition of the control image normal and the reflected component R normal , the illumination component I after the vignetted image is decomposed Vignetting and the reflected component R Vignetting . Will I Vignetting , R Vignetting The input is sent to the correction network for enhancement and denoising respectively.
[0039] Step 4: The correction network is a multi-branch network, and the I input in step 3 Vignetting The enhanced image I of the illumination component is obtained by enhancement processing through the correction network enhance , R Vignetting The enhanced image R of the reflected component after denoising is obtained through the residual dense block (RDB) denoising network. enhance .
[0040] In the correction network, there are two parts: the enhancement part and the denoising part.
[0041] S1. The first part in the correction network is the enhancement part. Input R in the input layer Vignetting Image and I VignettingThe image has five hidden layers. It first passes through two convolutional ReLu activation layers to perform downsampling operations to clarify the illumination component of the large-scale image. At the same time, it makes jump connections with the mirror upsampling layer to reconstruct the local illumination distribution of the image, so that the network is forced to learn the residual. Then, a 1x 1 convolutional layer is used to simplify the multi-channel features after multi-scale splicing. Finally, a 3x 3 convolutional output layer is used to reconstruct the illumination component image.
[0042] S2, the second part of the correction network is the hole convolution network part. The network structure is eight layers of convolution, and its purpose is to improve the receptive field. Vignetting The image passes through the dilated convolutional network and the enhanced sub-network in parallel, and finally merges through a convolutional layer of size 1x 1 to obtain the enhanced image I of the illumination component. enhance .
[0043] S3. In the reflection denoising network, in order to make full use of all the features in the reflection component image and improve the network performance, the present invention uses a dense residual network in the denoising network. The RDB extracts local features in a densely connected manner, and further performs local feature fusion. At the same time, the RDB network also has a feedforward property. The output of each layer can be directly connected to all subsequent layers, and each layer can also read the information of all previous layers, which closely connects the shallow network and the deep network together, has a continuous memory mechanism, and retains the accumulated features.
[0044] The first layer of the RDB network is convolution + ReLu activation function. After convolution extraction, the features are activated by ReLu to form a nonlinear feature map. The second layer of the network is a residual block, which contains three dense blocks. Each dense block contains 4 layers of convolution + ReLu activation function. Each convolution layer obtains the output of all previous convolution layers in the module. The adjacent convolutions are short-connected, and all convolution layers are jump-connected. The structure of the third to eleventh layers is convolution + ReLu activation + BN, that is, feature mapping is performed after feature extraction, and the features are normalized. The final output layer is a convolution layer of size 3x3 to obtain the denoised reflection component after training.
[0045] S4, the decomposition network uses its characteristics to learn image decomposition, and its loss function is composed of multiple losses;
[0046] First, the reconstruction loss is used. Paired images are input. The R decomposed from the vignetted image is multiplied by I and the vignetted image is counted as a loss. The R decomposed from the control image is multiplied by I and the control image is also counted as a loss. The R decomposed from the vignetted image is multiplied by I decomposed from the control image and the control image is counted as a loss. The R decomposed from the control image is multiplied by I decomposed from the vignetted image and the vignetted image is counted as a loss. The formula is as follows:
[0047]
[0048] The second loss is the loss of the reflected component image, that is, the R obtained by decomposing the vignetted image. vignetting R obtained after decomposition of the control image normal After subtraction, find the 1 norm.
[0049] L2=||R vignetting -R normal ||1
[0050] The third loss is the loss of the illumination component image. In the decomposition network, we want to remove the texture of I while retaining its boundary information. Therefore, the gradient is minimized selectively. We cannot directly minimize the total variation of the gradients of all illumination components, which will lose the boundaries of the illumination components.
[0051]
[0052] In L3, the negative logarithm of the reflection component is used to weight the variational loss of the illumination component. The loss in places with strong gradients becomes smaller because of the logarithmic weight, so that the structural information of the strong gradient area can be retained.
[0053] The total loss function is:
[0054] L=L1+λ2L2+λ3L3
[0055] Among them, λ2 is the consistent balance coefficient of the reflection component, and λ3 is the smoothing coefficient of the illumination component.
[0056] Step 5: The two component images R processed in step 4 enhance and I enhance Perform fusion reconstruction to obtain the restored image after correction.
[0057] like Figure 2 As shown, in some embodiments, the network structure of the decomposition network is: a pair of vignetted images and control images input in the input layer share weights, and two constraints are used to decompose the two images into corresponding illumination component images and reflection component images. In the correction network, the two parts included are the enhancement part and the denoising part.
[0058] like Figure 3 As shown, in some embodiments, in the reflection denoising network, in order to fully utilize all features in the reflection component image and improve network performance, the present invention uses a dense residual network in the denoising network.
[0059] like Figure 4As shown, in some embodiments, the improved Retinex-Net vignetting image correction method proposed in this article corrects the vignetting image with better effect.
[0060] like Figure 5 As shown, in some embodiments, the present invention provides a vignetting image correction method and other vignetting image correction methods to process images simultaneously. The present invention uses qualitative measurement indicators to quantitatively evaluate the effect of image correction. In the field of image processing, three evaluation methods are generally used: peak signal-to-noise ratio (PSNR), root mean square error (RMSE), and structural similarity (SSIM). The sample images selected for result verification are randomly selected from 1000 pairs of vignetting images and control images provided in this hair style.
[0061] Other vignetting image correction methods selected by the present invention for comparative verification of correction results are: Reference [1] - [Zhou Siyu, Bao Guoqi, Liu Kai, Image vignetting correction based on constrained logarithmic intensity entropy under low-pass filtering. Journal of Computer Applications, 2020.], Reference [2] - [Zhou Qingsong, Huang Song, Chen Honglei, Multi-scale image vignetting correction based on low adaptive compensation Retinex algorithm. Computer Simulation, 2021.], Reference [3] - [Li Zhaolong, Shen Tongsheng, Lou Shuli. Infrared system vignetting effect correction method based on polynomial approximation. Infrared and Laser Engineering, 2016.].
[0062] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An improved Retinex-Net vignetting image correction method, characterized by: The following steps are included: Step 1: Image preprocessing, batch processing of the size and format of the collected image set; Step 2: Input the paired vignetted and control images into the trained improved Retinex-Net model; Step 3: Pass the vignetted image and the control image input in step 2 through the decomposition network to obtain the illumination component I after decomposition of the control image normal and the reflected component R normal , the illumination component I after the vignetted image is decomposed Vignetting and the reflected component R Vignetting ; will I Vignetting , R Vignetting The input is sent to the correction network for enhancement and denoising respectively; Step 4: The correction network is a multi-branch network, and the I input in step 3 Vignetting The enhanced image I of the illumination component is obtained by enhancement processing through the correction network enhance , R Vignetting The enhanced image R of the reflected component after denoising is obtained through the dense residual denoising RDB network. enhance ; In step 4, the correction network includes two modules; The first module in the correction network is the enhancement module, which inputs R in the input layer Vignetting Image and I Vignetting Image, hidden layer has five layers, first through two layers of convolution ReLu activation layer, the purpose is to perform downsampling operation, clarify the illumination component of large-scale image, and at the same time make jump connection with mirror upsampling layer, reconstruct local illumination distribution of image, let the network force to learn residual, then through a layer of 1x 1 convolution layer to simplify the multi-channel features after multi-scale splicing, and finally through a 3x 3 convolution output layer, reconstruct illumination component image; The second module in the correction network is the hole convolution network module, which has eight layers of convolution, and its purpose is to improve the receptive field. Vignetting The image passes through the dilated convolutional network and the enhanced sub-network in parallel, and finally merges through a convolutional layer of size 1x 1 to obtain the enhanced image I of the illumination component. enhance ; In step 4, in the reflection denoising network, in order to make full use of all features in the reflection component image and improve network performance, a dense residual network is used in the denoising network; The first layer of the RDB network is convolution + ReLu activation function. After the convolution extracts the features, ReLu activation is used to form a nonlinear feature map. The second layer of the network is a residual block, which contains three dense blocks. Each denseblock contains 4 layers of convolution + ReLu activation function. Each convolution layer obtains the output of all previous convolution layers in the module. The adjacent convolutions are connected in a short connection manner, and all convolution layers are connected together in a jump manner. The structure of the third to eleventh layers is convolution + ReLu activation + BN, that is, feature mapping is performed after feature extraction, and the features are normalized. The final output layer is a convolution layer of size 3x 3, and the denoised reflection component after training is obtained. Step 5: The two component images R processed in step 4 enhance and I enhance Perform fusion reconstruction to obtain the restored image after correction.
2. The improved Retinex-Net vignetting image correction method according to claim 1, characterized in that: In the steps 1 and 2, the 1000 pairs of vignetted images and control images collected are batch-processed into RGB images with a size of 21.17 cm×14.11 cm and a png format, which are input into the improved Retinex-Net model for training.
3. The improved Retinex-Net vignetting image correction method according to claim 1, characterized in that: In step 3, the network structure of the decomposition module is: the paired vignetted image and the control image input in the input layer share weights, and the two images are decomposed into corresponding illumination component images and reflection component images using two constraints. The specific process is: in the hidden layer, the first layer first extracts features through convolution, and then passes through two layers of convolution + ReLu activation layers, and uses the constraint function of the two images to obtain the illumination component images I of the two images. normal and I Vignetting and the reflected component image R normal and R Vignetting In the output layer, the extracted multi-channel features are projected into illumination and reflection components through a convolutional layer of size 3x 3, and then the Sigmoid function is used to constrain the variables to ensure that the threshold is positive.