Image denoising and adaptive enhancement method based on pixel discretization and illumination guidance

Abstract

The present invention discloses an image denoising and adaptive enhancement method based on pixel discretization and illumination guidance, which relates to the field of image processing technology. The method comprises: downsampling and denoising an input image, and obtaining a preliminary denoised image through deconvolution, fuzzy pixel discretization, and residual continuity processing; estimating the illumination component from the preliminary denoised image using an illumination estimation network; calculating a reflected image based on the estimated illumination component and the preliminary denoised image and according to image enhancement theory; splicing the reflected image and the illumination component in the channel dimension based on a dynamic splicing mechanism to obtain a spliced ​​image; denoising and enhancing the spliced ​​image using a reflection denoising network, with the preservation of image detail and brightness information as the criterion, to output a denoised and enhanced image. The present invention achieves high-quality restoration of low-light images, improving the intelligence level of power inspection systems.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image denoising and adaptive enhancement method based on pixel discretization and illumination guidance. Background Art

[0002] Power inspections play a vital role in the operation and maintenance of power systems. Power equipment is widely distributed in diverse environments, and inspections in some areas must be conducted at night or in low-light environments, such as indoor distribution rooms. With the development of intelligent power inspection technology, image acquisition and analysis techniques are being widely used in power inspections. Capturing images of power equipment through cameras can quickly and accurately identify potential equipment failures and safety hazards, ensuring the stable operation of the power system. However, images captured in low-light environments often suffer from serious quality issues. Due to insufficient illumination, a large number of noisy pixels appear in the image, resulting in blurred images and difficulty distinguishing detailed features of the equipment. This not only increases the difficulty for inspectors to manually interpret the images, but also affects the accuracy and reliability of subsequent intelligent diagnostic systems based on image analysis. This can lead to missed or false fault detections, posing a threat to the safe and stable operation of the power system.

[0003] Looking back at the technological developments over the years, traditional image denoising methods can reduce the noise level to a certain extent when processing low-light power inspection images, but they usually lead to the loss of image detail information, making the key features of the equipment blurred, which is not conducive to the accurate identification of equipment faults. At the same time, these methods do not fully consider the characteristics of low-light images and cannot effectively denoise the noise characteristics in low-light environments. Existing image enhancement technology, when applied to low-light power inspection images, tends to amplify the noise in the image, further deteriorating the image quality that already has noise problems. In addition, traditional image denoising and enhancement technologies are often performed independently, without achieving an organic combination of the two. It is difficult to effectively enhance the image while denoising, and cannot meet the actual needs of power inspection work for high-quality processing of low-light images.

[0004] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0005] In response to the problems in the related art, the present invention proposes an image denoising and adaptive enhancement method based on pixel discretization and illumination guidance to overcome the above-mentioned technical problems existing in the existing related art.

[0006] To this end, the specific technical solutions adopted in the present invention are as follows:

[0007] Image denoising and adaptive enhancement methods based on pixel discretization and illumination guidance, including:

[0008] The input image is downsampled and denoised, and a preliminary denoised image is obtained through deconvolution, fuzzy pixel discretization, and residual continuity processing. The illumination component is estimated from the preliminary denoised image using an illumination estimation network.

[0009] Based on the estimated illumination components and the preliminarily denoised image, the reflected image is calculated according to image enhancement theory. Based on the dynamic stitching mechanism, the reflected image and the illumination components are stitched in the channel dimension to obtain the stitched image.

[0010] With the principle of maintaining image details and brightness information, the spliced ​​image is denoised and enhanced through the reflection denoising network, and the denoised and enhanced image is output.

[0011] Furthermore, the input image is downsampled and denoised, and through deconvolution, fuzzy pixel discretization and residual continuity processing, the preliminary denoised image is obtained, including:

[0012] The input image is downsampled into non-overlapping image blocks, and each block is input into the low-light denoising network for denoising to obtain the denoised image blocks;

[0013] Using the illumination basic kernel, deconvolution operation is performed on the denoised image block to generate a deconvolution image;

[0014] Extract the noise features of the deconvolution image and determine the noise category based on the type and intensity of the noise features;

[0015] Based on the noise category and the deconvolution class image, a noise segmentation map is obtained; the continuous residual corresponding to the noise segmentation map is obtained by using the multi-layer convolutional network of the mapping model, and is added to the input image to obtain a preliminary denoised image.

[0016] Furthermore, based on the noise category and the deconvolution class image, a noise segmentation map is obtained; the continuous residual corresponding to the noise segmentation map is obtained by using the multi-layer convolutional network of the mapping model, and is added to the input image to obtain the preliminary denoised image including:

[0017] Combine the category indices of all pixels in the deconvolution image according to the spatial position of the input image to generate a noise segmentation map with the same size as the input image;

[0018] The noise area in the noise segmentation map is discretized into different categories and the discretized residual is obtained;

[0019] Through the multi-layer convolutional network of the mapping model, the mapping relationship between the discretized residual and the continuous residual is learned; the discretized residual is converted into a continuous residual, and the preliminary denoised image is estimated by adding the continuous residual and the input image.

[0020] Furthermore, the expression of the discretized residual is:

[0021] ;

[0022] Where, I discrete Represents the discretized residual image;

[0023] c k Represents the residual value of the kth category;

[0024] M k represents the position of the kth category in the image, and K represents the total number of categories.

[0025] Furthermore, the illumination estimation network is used to estimate the illumination components from the preliminary denoised image, including:

[0026] The brightness distribution adjustment algorithm is used to perform global brightness adjustment on the image after preliminary denoising; each pixel in the image after preliminary denoising is enhanced by the adaptive enhancement algorithm, and the image to be processed is obtained after global brightness adjustment and enhancement processing on the image after preliminary denoising;

[0027] The illumination estimation network is used to extract and decode the estimated illumination components from the image to be processed.

[0028] Furthermore, each pixel in the image after preliminary denoising is enhanced by an adaptive enhancement algorithm, including:

[0029] The image after preliminary denoising is divided into multiple local regions, the local contrast of each pixel is calculated using a local contrast enhancement algorithm, and the local details of the image are enhanced by limiting the contrast of the histogram in each local region;

[0030] In each local area, the adaptive enhancement coefficient of each pixel is calculated according to the brightness value and local contrast of the pixel, and the calculation formula of the adaptive enhancement coefficient is:

[0031] ;

[0032] Where, Represents the adaptive enhancement coefficient, I max Indicates the maximum brightness value of the pixel, Represents the brightness value of the pixel;

[0033] β represents the scaling factor, and τ represent different exponential factors;

[0034] C max represents the local contrast maximum, Represents the local contrast of the pixel;

[0035] An adaptive enhancement coefficient is applied to each pixel to achieve enhancement processing in each local area.

[0036] Furthermore, the illumination estimation network is used to extract and decode the estimated illumination components from the image to be processed, including:

[0037] Add the illumination smoothness loss function to the total loss function, and backpropagate and optimize the illumination estimation network;

[0038] Based on the decoder in the optimized illumination estimation network, the preliminary illumination estimation is extracted and decoded, and the estimated illumination components are obtained after constraining the gradient change and neighborhood consistency of the illumination through the illumination smoothness loss function.

[0039] Among them, the formula of the illumination smoothness loss function is:

[0040] ;

[0041] Where, L smooth represents the illumination smoothness loss function;

[0042] R, G, B represent the three color channels, and c represents the three color channels of R, G, and B;

[0043] and Represents the gradient of light in the horizontal and vertical directions respectively;

[0044] represents the neighborhood of pixel i;

[0045] Represents the weight of the neighborhood pixels;

[0046] λ represents the hyperparameter that balances the two terms, and N represents the total number of pixels in the image;

[0047] and denote the estimated illumination values ​​at pixel i and pixel j, respectively.

[0048] Furthermore, based on the estimated illumination components and the image after preliminary denoising, the reflected image is calculated according to image enhancement theory, including:

[0049] According to the image enhancement theory, the estimated illumination components and the image after preliminary denoising are subjected to inverse operation to obtain the reflected image;

[0050] After obtaining the reflected image, the estimated illumination component and the corresponding reflected image are paired to obtain paired data.

[0051] Furthermore, based on the dynamic stitching mechanism, the reflection image and the illumination component are stitched in the channel dimension, and the stitched image includes:

[0052] Build a hybrid architecture model and use the convolutional neural network in the hybrid architecture model to extract the local texture and noise features of the reflected image, as well as the spatial distribution of the illumination components; align the scale and number of channels of the feature map through cross-modal convolution;

[0053] The deep learning module in the hybrid architecture model is used to learn the association between the reflection images of different regions and the estimated illumination components, and to control the fusion ratio of different channels;

[0054] By iteratively updating the hybrid architecture model parameters and minimizing the loss function, the hybrid architecture model learns the optimal stitching strategy between the reflected image and the estimated illumination component. Using the optimal stitching strategy, the reflected image and the illumination component are stitched in the channel dimension to obtain a stitched image.

[0055] Among them, combined with meta-learning, when facing new scenarios, some samples are used to fine-tune the hybrid architecture model and adjust the splicing strategy parameters; the loss is calculated on the query set, and the initial parameters of the hybrid architecture model are optimized through back propagation to make the hybrid architecture model adapt to new scenarios.

[0056] Furthermore, with the preservation of image details and brightness information as the criterion, the spliced ​​image is denoised and enhanced through the reflection denoising network, and the output denoised and enhanced image includes:

[0057] The loss function of the reflection denoising network is constructed by combining residual loss, consistency loss, and illumination consistency loss. The parameters of the reflection denoising network are optimized through backpropagation to ensure that the details and brightness information of the output image are preserved.

[0058] The spliced ​​image is input into the optimized reflection denoising network, and the denoised enhanced image is output;

[0059] Among them, the loss function of the reflection denoising network is:

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] Where, L total represents the loss function of the reflection denoising network, L residual represents the residual loss, L consistency represents the consistency loss, Llight Indicates the loss of lighting consistency;

[0065] represents the output image, R clean represents an ideal clean image;

[0066] represents the estimated illumination component, represents the image after preliminary denoising, and ▽ represents the gradient operator.

[0067] The beneficial effects of the present invention are:

[0068] Beneficial effects of the present invention:

[0069] 1. This invention integrates the low-light image denoising and enhancement processes into a unified framework. By jointly optimizing the denoising and brightness enhancement tasks, it solves the problems of low step-by-step processing efficiency and detail loss in traditional methods, significantly improving the quality of low-light images.

[0070] 2. An innovative discrete denoising scheme is proposed. Through a denoising pixel discretizer and a discrete-to-continuous converter, the denoising problem is discretized and the processing flow is simplified, avoiding the complex operations of predicting convolution kernels and deconvolution in traditional methods. While reducing computational complexity, it also reduces dependence on training data and noise distribution, and enhances the generalization ability of the denoising model.

[0071] 3. Combining the illumination estimation network and the reflected image enhancement module, it accurately estimates the illumination components and optimizes the reflected image to avoid overexposure or under-enhancement, adapting to a variety of low-light scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0073] Figure 1 is a flowchart of an image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to an embodiment of the present invention;

[0074] Figure 2 is a logic flow chart according to an embodiment of the present invention;

[0075] Figure 3 4 is a network structure diagram of low-light image denoising and brightness enhancement according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0077] According to an embodiment of the present invention, an image denoising and adaptive enhancement method based on pixel discretization and illumination guidance is provided.

[0078] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, the image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to an embodiment of the present invention includes:

[0079] S1. Downsample and denoise the input image, and obtain a preliminary denoised image through deconvolution, fuzzy pixel discretization and residual continuity processing; use the illumination estimation network to estimate the illumination component from the preliminary denoised image.

[0080] S2. Based on the estimated illumination component and the image after preliminary denoising, the reflected image is calculated according to image enhancement theory. Based on the dynamic stitching mechanism, the reflected image and the illumination component are stitched in the channel dimension to obtain a stitched image.

[0081] S3. Based on the principle of maintaining image details and brightness information, the spliced ​​image is denoised and enhanced through the reflection denoising network, and the denoised and enhanced image is output.

[0082] In one embodiment, the input image is downsampled and denoised, and a deconvolution operation, fuzzy pixel discretization, and residual continuity processing are performed to obtain a preliminary denoised image, including:

[0083] The input image is downsampled into non-overlapping image blocks and respectively input into the low-light denoising network for denoising to obtain denoised image blocks; the denoised image blocks are deconvolved using the illumination basis kernel to generate a deconvolved image; the noise features of the deconvolved image are extracted, and the noise category is determined based on the type and intensity of the noise features; based on the noise category and the deconvolved image, a noise segmentation map is obtained; the continuous residual corresponding to the noise segmentation map is obtained using the multi-layer convolutional network of the mapping model, and added to the input image to obtain a preliminary denoised image.

[0084] In one embodiment, based on the noise category and the deconvolution class image, a noise segmentation map is obtained; using a multi-layer convolutional network of a mapping model, a continuous residual corresponding to the noise segmentation map is obtained and added to the input image to obtain a preliminary denoised image, including:

[0085] The category indexes of all pixels in the deconvolution image are combined according to the spatial position of the input image to generate a noise segmentation map with the same size as the input image; the noise area in the noise segmentation map is discretized into different categories, and the discretized residuals are obtained; the mapping relationship between the discretized residuals and the continuous residuals is learned through the multi-layer convolutional network of the mapping model; the discretized residuals are converted into continuous residuals, and the preliminary denoised image is estimated by adding the continuous residuals and the input image.

[0086] In one embodiment, the expression of the discretized residual is:

[0087] ;

[0088] Where, I discrete represents the discretized residual image; c k Represents the residual value of the kth category; M k represents the position of the kth category in the image, and K represents the total number of categories.

[0089] In one embodiment, estimating illumination components from the preliminarily denoised image using an illumination estimation network includes:

[0090] The brightness distribution adjustment algorithm is used to perform global brightness adjustment on the image after preliminary denoising; the adaptive enhancement algorithm is used to enhance each pixel in the image after preliminary denoising, and the image to be processed is obtained after global brightness adjustment and enhancement processing on the image after preliminary denoising; the illumination estimation network is used to extract and decode the estimated illumination component from the image to be processed.

[0091] In one embodiment, performing enhancement processing on each pixel in the image after preliminary denoising using an adaptive enhancement algorithm includes:

[0092] The image after preliminary denoising is divided into multiple local areas. The local contrast of each pixel is calculated using the local contrast enhancement algorithm. The local details of the image are enhanced by limiting the contrast of the histogram in each local area. In each local area, the adaptive enhancement coefficient of each pixel is calculated based on the brightness value and local contrast of the pixel. The calculation formula of the adaptive enhancement coefficient is:

[0093] ;

[0094] Where, Represents the adaptive enhancement coefficient, Imax Indicates the maximum brightness value of the pixel, represents the brightness value of the pixel; β represents the scaling factor, and τ represent different exponential factors; C max represents the local contrast maximum, Represents the local contrast of a pixel; an adaptive enhancement coefficient is applied to each pixel to achieve enhancement processing in each local area.

[0095] In one embodiment, extracting and decoding estimated illumination components from an image to be processed using an illumination estimation network includes:

[0096] The illumination smoothness loss function is added to the total loss function, and the illumination estimation network is back-propagated and optimized. The decoder in the optimized illumination estimation network extracts and decodes the preliminary illumination estimate, and the illumination smoothness loss function is used to constrain the gradient change and neighborhood consistency of the illumination to obtain the estimated illumination component. The formula of the illumination smoothness loss function is:

[0097] ;

[0098] Where, L smooth Represents the illumination smoothness loss function; R, G, B represent the three color channels, and c represents the three color channels of R, G, and B; and Represents the gradient of light in the horizontal and vertical directions respectively; represents the neighborhood of pixel i; represents the weight of the neighborhood pixels; λ represents the hyperparameter for balancing the two terms, and N represents the total number of pixels in the image; and denote the estimated illumination values ​​at pixel i and pixel j, respectively.

[0099] In one embodiment, calculating the reflected image based on the estimated illumination components and the preliminarily denoised image and according to image enhancement theory includes:

[0100] According to image enhancement theory, the estimated illumination component and the image after preliminary denoising are subjected to inverse operation to obtain a reflected image. After obtaining the reflected image, the estimated illumination component and the corresponding reflected image are paired to obtain paired data.

[0101] In one embodiment, based on a dynamic stitching mechanism, the reflection image and the illumination component are stitched in the channel dimension, and the stitched image includes:

[0102] A hybrid architecture model is constructed, and the convolutional neural network in the hybrid architecture model is used to extract the local texture and noise features of the reflected image, as well as the spatial distribution of the illumination component; the scale and number of channels of the feature map are aligned through cross-modal convolution; the deep learning module in the hybrid architecture model is used to learn the association between the reflected images in different regions and the estimated illumination components, and to control the fusion ratio of different channels; the hybrid architecture model parameters are iteratively updated and the loss function is minimized so that the hybrid architecture model learns the optimal splicing strategy between the reflected image and the estimated illumination component; the reflected image and the illumination component are spliced ​​in the channel dimension using the optimal splicing strategy to obtain the spliced ​​image; in which, combined with meta-learning, when faced with a new scene, the hybrid architecture model is fine-tuned using some samples and the splicing strategy parameters are adjusted; the loss is calculated on the query set, and the initial parameters of the hybrid architecture model are optimized through back propagation to make the hybrid architecture model adapt to the new scene.

[0103] In one embodiment, the spliced ​​image is denoised and enhanced by a reflection denoising network based on the principle of maintaining image details and brightness information, and the output denoised and enhanced image includes:

[0104] The loss function of the reflection denoising network is constructed by combining residual loss, consistency loss, and illumination consistency loss. The parameters of the reflection denoising network are optimized through backpropagation to ensure that the details and brightness information of the output image are preserved. The spliced ​​image is input into the optimized reflection denoising network, and the denoised enhanced image is output. The loss function of the reflection denoising network is:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] Where, L total represents the loss function of the reflection denoising network, L residual represents the residual loss, L consistency represents the consistency loss, L light Indicates the loss of lighting consistency; represents the output image, R clean represents an ideal clean image; represents the estimated illumination component, represents the image after preliminary denoising, and ▽ represents the gradient operator.

[0110] In order to facilitate understanding of the above technical solutions of the present invention, the working principle of the present invention in actual process is described in detail below.

[0111] In low-light environments, through the joint processing of multi-stage denoising and brightness adaptive enhancement methods, the influence of training data and noise distribution is optimized, and high-quality restoration of low-light images is achieved, which improves the intelligence level of the power inspection system and has important practical significance for ensuring the safe and stable operation of the power system.

[0112] High-quality image restoration is achieved through multi-stage processing. Figure 2 As shown, the input image is first downsampled and denoised using a low-light denoising network. Deconvolution and discretization of blurred pixels are then combined to generate a latently sharp image. The discrete residual is then converted to a continuous form using a D2C model. Secondly, the IE-Net network (illumination estimation network) is used to estimate illumination components. Brightness adjustment and adaptive enhancement are achieved through gamma correction and the CLAHE algorithm (contrast-limited adaptive histogram equalization, an improved version of traditional histogram equalization). An illumination smoothness loss is introduced to optimize illumination continuity. Finally, the reflected image is calculated based on image enhancement theory (Retinex theory). This image is then concatenated with the illumination and fed into the RD-Net network (encoder-decoder structure) for denoising and enhancement, resulting in a high-quality output image.

[0113] Figure 3 This is the network structure diagram for low-light image denoising and brightness enhancement, including input image, noise pixel discretizer, noise segmentation map, D2C converter, image residual, preliminary denoising, reconstructed image, illumination estimation network, illumination estimation, reflected image and channel stitching. The input image, illumination estimation and reflected image are divided into several parts.

[0114] The D2C model is usually composed of multiple convolutional layers, activation layers, and residual blocks. The network establishes a nonlinear mapping relationship between discrete residuals (multiple convolutional layers are processed in sequence, with the output of the previous layer used as the input of the next layer, gradually extracting features from elementary to advanced levels to form a multi-level expression of the discretized residuals) and continuous residuals through layer-by-layer convolution and nonlinear activation functions (each convolutional layer calculates the feature response of the local area of ​​the input data through the convolution kernel, and introduces a nonlinear activation function to perform a nonlinear transformation on the convolution result, thereby enhancing the model's learning ability for complex features and enabling the network to capture more abstract feature representations in the discretized residuals). The backpropagation algorithm is used in the training process, using the mean square error to minimize the pixel-level difference between the discrete residuals and the continuous residuals as the loss function, and paired data (discrete residuals and continuous residuals) are used to train the model end-to-end. Finally, the model outputs the final clear image .

[0115] Specifically include the following:

[0116] Step 1: Perform preliminary denoising on the input low-light noise image, and obtain a potential clear image through downsampling, deconvolution, blur pixel discretization and residual continuity processing:

[0117] The input low-light image is downsampled and denoised by the low-light denoising network. A deconvolution-like image is generated using the illumination base kernel, and a clear image is estimated through a fuzzy pixel discretizer and a D2C model: the input low-light image I is downsampled into two non-overlapping image blocks, which are respectively input into the low-light denoising network for denoising to obtain the denoised image blocks.

[0118] Use a set of predefined lighting base kernels , where n represents the number of basic cores, and each basic core It is a two-dimensional matrix that represents different illumination restoration features. The deconvolution operation is performed on the denoised image using this set of illumination basic kernels in turn to generate a series of deconvolution images. . Each deconvolution class image All represent specific basic cores The restored latent clear image, different Due to the use of different basic kernels, different illumination restoration image enhancement effects are shown.

[0119] For multi-channel images, the deconvolved image The pixel value of the cth channel is expressed as:

[0120] ;

[0121] in, Represents the basic kernel The part corresponding to the c-th channel, Represents the pixel coordinates in the deconvolution image. m and n represent the two-dimensional indexes inside the convolution kernel, representing the row and column indices of the kernel matrix, respectively. h and w represent the height and width of the image, respectively.

[0122] Secondly, the deconvolution image is passed through the convolution layer to extract noise features, and the noise category is defined according to the noise type and intensity. The fuzzy pixel discretizer selects the category with the highest probability as the label for each pixel. Finally, the category indexes of all pixels are combined to generate a noise segmentation map with the same size as the input image. Each pixel value is the corresponding noise category index. Specifically:

[0123] The deconvolution image is processed through the convolution layer to extract noise features. These features contain the expression of information about the noise type, intensity, etc. for each pixel in the image and are input into the fuzzy pixel discretizer.

[0124] Define noise categories based on noise type (Gaussian, salt and pepper, Poisson, etc.) and intensity level (low, medium, high).

[0125] For each pixel's noise feature vector, the discretizer uses a classification model (the pixel is converted into a probability distribution using a softmax function, with each probability value in the range [0, 1] and summing to 1) to calculate the probability P(k) that the pixel belongs to each predefined noise category. For each pixel, the noise category with the highest probability is selected as its label. This step converts the fuzzy probability distribution into a clear category index, achieving discretized pixel classification.

[0126] Combine the category indexes of all pixels according to the spatial position of the original image to generate a noise segmentation map with the same size as the input image. Each pixel value corresponds to the noise category index to which it belongs, and a binary mask is generated. , Represents image Belongs to category k, otherwise it is 0, thus completing the category division of the noise area in the image.

[0127] The noise area in the noise segmentation map generated by the fuzzy pixel discretizer is discretized into different categories, each category corresponds to a noise type and level, and the discretization residual is expressed as:

[0128] ;

[0129] Among them, I discrete represents the discretized residual image, c k represents the residual value of the kth category, Represents a binary mask indicating the location of the k-th category in the image, and K represents the total number of categories.

[0130] The generated discretized residual I discrete Input D2C model, learn the mapping relationship between discretized residual and continuous residual through the multi-layer convolutional network of D2C model, and transform the discretized residual I discrete Convert to continuous residual I continous Finally, the continuous residual is added to the original image to estimate the final clear image :

[0131] .

[0132] Step 2: Use the illumination estimation network IE-Net to estimate the illumination components from the image after preliminary denoising, and adaptively optimize the image quality through brightness adjustment. That is, use the illumination estimation network to estimate the illumination components from the image output by the low-light denoising network, and perform brightness adjustment and adaptive image enhancement:

[0133] The pixel values ​​of low-light images are usually low, resulting in insufficient overall brightness. Therefore, the gamma correction method is used to adjust the global brightness distribution of the image, so that the overall brightness of the low-light image after preliminary denoising is increased to a level close to that of normal light.

[0134] Based on the overall brightness adjustment, an adaptive enhancement algorithm is used to locally adjust each pixel to prevent overexposure or underexposure. The core concept of adaptive enhancement is to dynamically adjust the enhancement amplitude based on the brightness value of each pixel, ensuring that dark areas are fully enhanced while bright areas are not overexposed.

[0135] The image is divided into multiple local regions, and the local contrast enhancement algorithm CLAHE is used to calculate the local contrast of each pixel. , and limit the contrast of the histogram in each region to enhance the local details of the image.

[0136] According to the brightness value of the pixel and local contrast , calculate the adaptive enhancement coefficient for each pixel :

[0137] ;

[0138] Among them, I max Represents the maximum brightness value of the image (usually 255), β represents a scaling factor used to control the magnitude of the enhancement, ,τ represents an exponential factor used to adjust the enhanced nonlinear characteristics, C max Indicates the local contrast maximum (set according to the image dynamic range).

[0139] By adjusting β, The values ​​of and τ ensure that the brightness enhancement of pixels in dark areas is large, while the brightness enhancement of pixels in bright areas is small, avoiding overexposure and under-enhancement. The adaptive enhancement coefficient Applied to each pixel, the pixel value adaptively enhanced image I is obtained enhanced .

[0140] Get a preliminary estimate of the lighting from the illumination estimation network IE-Net , L1 regularization is performed on the horizontal and vertical gradients of each color channel, and Gaussian weights are used to calculate the illumination difference of neighboring pixels. Finally, since the illumination changes in natural images are usually continuous and smooth, the illumination smoothness loss function L is introduced smooth Add a total loss function, backpropagate, and optimize IE-Net parameters. The illumination estimation network is the illumination estimation network. The illumination estimation network extracts the global brightness distribution and local illumination change characteristics of the illumination through the convolutional layer in the encoder, and the decoder generates a preliminary illumination estimate.

[0141] The illumination smoothness loss function mainly consists of two parts: the gradient constraint term and the neighborhood smoothness constraint term. By constraining the gradient change and neighborhood consistency of illumination, the estimated illumination components are ensured. With continuity and smoothness:

[0142] ;

[0143] Among them, c represents the three channels of RGB color; 、 Represents the gradient of light in the horizontal and vertical directions respectively; represents the neighborhood of pixel i (usually a 5×5 window); Represents the weight of the neighborhood pixels, calculated by the Gaussian kernel function; represents the hyperparameter for balancing the two terms, represents the total number of pixels in the image, 、 Represents the estimated illumination value at pixel i and j. The first half is the spatial gradient constraint, which sums the L1 norm of the horizontal and vertical gradients of the three color channels. The purpose is to force the illumination to transition smoothly in space by directly penalizing the drastic changes in illumination. The second half of the formula is the neighborhood consistency constraint, which is for each pixel i in the image, in its neighborhood According to the neighborhood pixel weight Calculate the weighted sum of the differences between the neighboring pixels and the estimated illumination value of the pixel, and constrain the smoothness of the local area by comparing the illumination value difference between the pixel and its neighboring pixels.

[0144] By jointly optimizing the gradient term and the neighborhood term, the model learns a lighting distribution that is both globally smooth and locally adaptive, improving the accuracy and reliability of overall lighting estimation. λ is a hyperparameter used to balance the influence of the first and second terms.

[0145] Step 3: Calculate the reflected image based on the estimated illumination components, and denoise and enhance the reflected image through the reflection denoising network RD-Net, and finally output a high-quality denoised and enhanced image:

[0146] According to image enhancement theory (Retinex theory), the estimated illumination components And the image after preliminary denoising , calculate the reflected image R by inverse operation:

[0147] ;

[0148] in, Represents the inverse operation of the lighting component.

[0149] The reflected image represents the inherent properties of the low-light image and contains noise. Splicing is performed on the channel dimension to form As the input of the reflection denoising network RD-Net, the noise characteristics are kept unchanged during the denoising process by maintaining the consistency of the illumination.

[0150] Widely collect low-light images from multiple scenes (such as indoor, outdoor, night, etc.), and use the illumination estimation network IE-Net to obtain the corresponding illumination components and through Calculate the reflected image R to form a large number of Paired data.

[0151] Using a CNN-Transformer hybrid architecture, the CNN (convolutional neural network) module extracts the local texture and noise features of the reflected image and the spatial distribution of the illumination components through multi-layer convolution, and aligns the scale and number of channels of the two feature maps through cross-modal convolution. The Transformer (a deep learning model architecture) module processes global feature dependencies, divides the fused feature map into a sequence of patches (sub-images), models long-range dependencies through multi-head self-attention, and generates a weight matrix in the last fully connected layer to learn the R and The association controls the fusion ratio of different channels:

[0152] ;

[0153] in, Represents the feature map after channel splicing, is a dynamically generated weight matrix.

[0154] Iteratively update the model parameters to minimize the loss function (PSNR, SSIM, MSE) so that the model can learn Mapping of optimal splicing strategies.

[0155] At the same time, combined with meta-learning, the dataset is divided into multiple subtasks according to the scene type, and each task contains a specific scene. When faced with new scene data, we fine-tune the above model with a small number of samples, adjust the splicing strategy parameters, calculate the loss on the query set, and optimize the initial parameters of the model through back propagation, so that it has the ability to quickly adapt to new tasks and generate splicing weights suitable for the scene.

[0156] The reflection denoising network is used to combine the reflection component and illumination component Denoising and enhancement are performed, and residual loss, consistency loss, and illumination consistency loss are used to ensure that the denoised image maintains high-quality details and brightness information, and outputs a fully denoised and enhanced high-quality reflection image. .

[0157] The spliced ​​reflection image and the illumination component The reflection denoising network (RD-Net) uses an encoder-decoder architecture. The encoder is responsible for feature extraction, gradually extracting multi-scale features of the reflection image through multiple convolutional layers. This layer adjusts the number of channels and captures local details and noise characteristics of the image. Furthermore, the residual block and attention mechanism modules further learn deep features. The residual block allows the network to focus on learning noise residuals and enhance feature reuse capabilities. The self-attention module calculates the correlation weights between pixels, allowing the network to focus on noise-prone areas and enhance denoising effectiveness.

[0158] The decoder is used for feature reconstruction. The deconvolution layer / upsampling layer gradually restores the image resolution and combines jump connections to fuse shallow details with deep semantic information. The nonlinear mapping layer ReLU is embedded in the decoding process to dynamically adjust the local contrast and brightness.

[0159] The loss function combines residual loss, consistency loss, and illumination consistency loss, optimizes network parameters through backpropagation, ensures that details and brightness information are preserved, and finally outputs a denoised and enhanced reflection image. , the size is consistent with the input:

[0160] ;

[0161] Residual loss calculation output image and ideal clean image Pixel-level differences constrain denoising accuracy:

[0162] ;

[0163] The consistency loss ensures that the image structure is consistent before and after denoising:

[0164] ;

[0165] Lighting consistency loss, constraining the gradient consistency between the enhanced image and the original image:

[0166] .

[0167] The loss function of the reflection denoising network jointly optimizes the residual loss, consistency loss, and illumination consistency loss. It constrains the model from three dimensions: pixel-level reconstruction, physical constraints, and structure preservation. This ensures that the denoised image retains detail and brightness while conforming to the physical decomposition of illumination and reflection.

[0168] Residual loss constrains the pixel-level difference between the output image and the ideal clean image based on the L1 norm, measuring the gap in pixel values ​​between the output image and the ideal clean image.

[0169] Consistency loss,constrains that the product of the illumination component and the denoised,reflection component can accurately reconstruct the preliminary denoised image,,that is, the consistency with the preliminary denoised image, ensuring the,physical rationality of the decomposition results and avoiding the,illumination or reflection components from deviating from the theoretical constraints.

[0170] Lighting consistency loss constrains the consistency of the gradient of the illumination-reflection combination with the gradient of the initial denoised image. That is, the high-frequency information such as edges and textures of the two must be aligned to maintain the integrity of the image structure and prevent artifacts or blurring introduced by lighting adjustment.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Image denoising and adaptive enhancement method based on pixel discretization and illumination guidance, characterized by: include: Using the illumination basic kernel, deconvolution operation is performed on the denoised image block to generate a deconvolution image; Extract the noise features of the deconvolution image and determine the noise category based on the type and intensity of the noise features; Combine the category indices of all pixels in the deconvolution image according to the spatial position of the input image to generate a noise segmentation map with the same size as the input image; The noise regions in the noise segmentation map are discretized into different categories to obtain the discretized residuals. The mapping relationship between the discretized residuals and the continuous residuals is learned through the multi-layer convolutional network of the mapping model. The discretized residuals are converted into continuous residuals, and the initial denoised image is estimated by adding the continuous residuals to the input image. The illumination component is estimated from the initial denoised image using the illumination estimation network. Based on the estimated illumination components and the initially denoised image, the reflected image is calculated according to image enhancement theory. The convolutional neural network in the constructed hybrid architecture model is used to extract the local texture and noise characteristics of the reflected image, as well as the spatial distribution of the illumination components. The scale and number of channels of the feature map are aligned through cross-modal convolution. The deep learning module in the hybrid architecture model is used to learn the association between the reflected images of different regions and the estimated illumination components, thereby controlling the fusion ratio of different channels. By iteratively updating the hybrid architecture model parameters and minimizing the loss function, the hybrid architecture model learns the optimal stitching strategy between the reflected image and the estimated illumination component; Using the optimal stitching strategy, the reflection image and the illumination component are stitched in the channel dimension; With the principle of maintaining image details and brightness information, the spliced ​​image is denoised and enhanced through the reflection denoising network, and the denoised and enhanced image is output.

2. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: The method further includes performing a deconvolution operation on the denoised image block using the illumination basic kernel and generating a deconvolution image before the deconvolution image is generated: The input image is downsampled into non-overlapping image blocks, and each block is input into the low-light denoising network for denoising to obtain the denoised image blocks.

3. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: The expression of the discretized residual is: Where, I discrete Represents the discretized residual image; c k Represents the residual value of the kth category; M k represents the position of the kth category in the image, and K represents the total number of categories.

4. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: The method of estimating the illumination component from the image after preliminary denoising using the illumination estimation network includes: The brightness distribution adjustment algorithm is used to perform global brightness adjustment on the image after preliminary denoising; each pixel in the image after preliminary denoising is enhanced by the adaptive enhancement algorithm, and the image to be processed is obtained after global brightness adjustment and enhancement processing on the image after preliminary denoising; The illumination estimation network is used to extract and decode the estimated illumination components from the image to be processed.

5. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 4, characterized in that: The enhancing process of each pixel in the image after preliminary denoising by using the adaptive enhancement algorithm includes: The image after preliminary denoising is divided into multiple local regions, the local contrast of each pixel is calculated using a local contrast enhancement algorithm, and the local details of the image are enhanced by limiting the contrast of the histogram in each local region; In each local area, the adaptive enhancement coefficient of each pixel is calculated according to the brightness value and local contrast of the pixel, and the calculation formula of the adaptive enhancement coefficient is: Where α(x, y) represents the adaptive enhancement coefficient, I max represents the maximum brightness value of the pixel, and I(x, y) represents the brightness value of the pixel; β represents the scaling factor, γ and τ represent different exponential factors; C max represents the local contrast maximum, and C(x, y) represents the local contrast of the pixel; An adaptive enhancement coefficient is applied to each pixel to achieve enhancement processing in each local area.

6. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 4, characterized in that: The method of extracting and decoding the estimated illumination components from the image to be processed using the illumination estimation network includes: Add the illumination smoothness loss function to the total loss function, and backpropagate and optimize the illumination estimation network; Based on the decoder in the optimized illumination estimation network, the preliminary illumination estimation is extracted and decoded, and the estimated illumination components are obtained after constraining the gradient change and neighborhood consistency of the illumination through the illumination smoothness loss function. Among them, the formula of the illumination smoothness loss function is: Where, L smooth represents the illumination smoothness loss function; R, G, B represent the three color channels, and c represents the three color channels of R, G, and B; and Represents the gradient of light in the horizontal and vertical directions respectively; represents the neighborhood of pixel i; ω i，j Represents the weight of the neighborhood pixels; λ represents the hyperparameter that balances the two terms, and N represents the total number of pixels in the image; and denote the estimated illumination values ​​at pixel i and pixel j, respectively.

7. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: The method of calculating the reflected image based on the estimated illumination components and the image after preliminary denoising and according to image enhancement theory includes: According to the image enhancement theory, the estimated illumination components and the image after preliminary denoising are subjected to inverse operation to obtain the reflected image; After obtaining the reflected image, the estimated illumination component and the corresponding reflected image are paired to obtain paired data.

8. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: Combined with meta-learning, when faced with new scenarios, some samples are used to fine-tune the hybrid architecture model and adjust the splicing strategy parameters; the loss is calculated on the query set, and the initial parameters of the hybrid architecture model are optimized through backpropagation to make the hybrid architecture model adapt to the new scenario.

9. The image denoising and adaptive enhancement method based on pixel discretization and illumination guidance according to claim 1, characterized in that: The method uses a reflection denoising network to denoise and enhance the spliced ​​image based on the principle of maintaining image details and brightness information, and outputs a denoised and enhanced image including: The loss function of the reflection denoising network is constructed by combining residual loss, consistency loss, and illumination consistency loss. The parameters of the reflection denoising network are optimized through backpropagation to ensure that the details and brightness information of the output image are preserved. The spliced ​​image is input into the optimized reflection denoising network, and the denoised enhanced image is output; Among them, the loss function of the reflection denoising network is: L total =L residual +L consistency +L light . Where, L total represents the loss function of the reflection denoising network, L residual represents the residual loss, L consistency represents the consistency loss, L light Indicates the loss of lighting consistency; represents the output image, R clean represents an ideal clean image; represents the estimated illumination component, represents the image after preliminary denoising, Represents a gradient operator.

Images