Image quality enhancement method based on residual diffusion model and residual estimation model

Abstract

The invention discloses an image quality enhancement method and system based on a residual diffusion model and a residual estimation model, and the method comprises the steps: carrying out the random sampling in a global range of a large-scale image, taking each sampling point as a center, carrying out the image cutting, and obtaining a plurality of local image blocks; then performing enhancement processing on the local image blocks, wherein the enhancement process comprises the steps of firstly decomposing an input low-quality image I into an illumination image L and a reflection image R through a convolutional neural network; then, based on a residual diffusion model, noise is gradually eliminated from the illumination image in an iteration mode, so that a high-quality clear illumination image is recovered; reflection image enhancement is realized based on a residual error estimation model, and layer-by-layer residual error estimation and residual error connection are carried out; and carrying out pixel-by-pixel fusion on the enhanced illumination image and reflection image to generate a final enhanced image. And finally, a global color mapping curve is constructed by using the local image blocks and the corresponding enhancement results, so that the processing efficiency of the high-resolution image is remarkably improved while the enhancement effect is ensured.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image quality enhancement, and relates to an image quality enhancement method and system, in particular to an image quality enhancement method and system based on a residual diffusion model and a residual estimation model. BACKGROUND

[0002] Image quality enhancement is a key technology in the field of computer vision and digital image processing, aiming to improve the quality of the original input image, thereby facilitating downstream detection, recognition and other tasks. In the actual camera data acquisition process, affected by factors such as lighting conditions and camera settings, the image quality may be poor, and problems such as overall insufficient brightness and insufficient contrast may occur. The existence of these problems may lead to the failure of downstream detection and recognition tasks. For example, in a self-service vending cabinet, when the image quality captured by the camera is poor, it may lead to the failure of subsequent product detection and recognition tasks.

[0003] To solve the problem of dark light image quality enhancement, traditional methods such as histogram equalization, adaptive histogram equalization and Gamma correction mainly adjust the overall intensity distribution of the image to improve the contrast and brightness. However, these methods have significant limitations when dealing with complex scenes. They lack local adaptive ability, are prone to amplify noise, appear block artifacts and cause detail distortion, and are difficult to retain bright and dark details in high dynamic range or unevenly illuminated images, resulting in local overexposure or underexposure. In recent years, image enhancement methods based on deep learning have also been proposed. Some methods use a direct end-to-end regression approach to obtain high-quality images, ignoring the physical model of imaging, resulting in poor performance and weak generalization. Currently, some methods combine imaging mechanism models and deep learning models, such as the Retinex model, to enhance images, resulting in better performance. However, the existing methods still have two problems: 1) the original image brightness information is severely missing, and the existing neural network architecture is difficult to obtain satisfactory enhanced images, and the network generation ability is limited; 2) for large-scale images, due to limited computing resources, the existing methods are difficult to directly process, and usually need to be divided into many overlapping image blocks for processing, resulting in low efficiency and inconsistency between image blocks. The above problems limit the usability of deep learning-based image enhancement methods in practice, and new methods are urgently needed to solve them. SUMMARY

[0004] To solve the problem of image quality under dark light conditions, the present application generates an image taken under normal light from a dark light image, improves the contrast and brightness of the image, and improves the accuracy of subsequent image target detection, recognition and other tasks. The present application proposes an image stitching quality enhancement method and system based on a residual diffusion model and a residual estimation model.

[0005] The technical scheme of the method of the present application is as follows: an image quality enhancement method based on a residual diffusion model and a residual estimation model, comprising the following steps: Step 1: randomly sampling in the global range of a large-scale image, and performing image cropping with each sampling point as the center to obtain a plurality of local image blocks; Step 2: performing enhancement processing on the local image blocks to obtain the pixel value results before and after enhancement of each image block; The specific implementation includes the following sub-steps: Step 2.1: decomposing the input image block I into an illumination map L and a reflection map R through a convolutional neural network to simulate the imaging mechanism of illumination-reflection; Step 2.2: based on the residual diffusion model, iteratively removing noise from the illumination map to recover a high-quality clear illumination map; Step 2.3: based on the residual estimation model, realizing reflection map enhancement, and synchronously realizing noise suppression and detail information preservation in the process of enhancing reflection through layer-by-layer residual estimation and residual connection; Step 2.4: performing pixel-by-pixel point multiplication fusion on the enhanced illumination map and reflection map to generate the final enhanced image ; Step 3: using the local image blocks and their corresponding enhancement results to construct a global color mapping curve to significantly improve the processing efficiency of high-resolution images while ensuring the enhancement effect.

[0006] As a preferred, in step 1, first, random sampling points are generated in the global range of a high-resolution image , wherein Y represents the number of sampling points; then a rectangular frame with a length and a width of 256 is constructed with each sampling point as the center, and image cropping is performed according to the coverage range of each rectangular frame, thereby obtaining a local image set containing Y cropped blocks .

[0007] As a preferred, in step 2.1, the illumination-reflection map decomposition module based on the convolutional neural network decomposes the input low-quality image into an illumination map and a reflection map; the illumination-reflection map decomposition module includes two branches: a reflection map extraction branch and an illumination map extraction branch, which are respectively used to predict the illumination map and the reflection map of the image to realize significant enhancement of the image in two feature levels; The reflection map extraction branch, for the input low-quality image , first obtains multi-scale features through UNet network layer-by-layer encoding and decoding, then integrates the multi-scale features through a two-dimensional convolution layer, and finally normalizes the output to the value space of [0, 1] using the Sigmoid activation function to obtain the reflection map wherein, represents the layer-by-layer encoding and decoding operations of UNet network, w and b represent the weight and bias parameters of convolution layer, represents the activation function in convolution layer; H, W represent the width and height of image, R represents the feature vector space, and 3 represents the number of channels; The illumination map extraction branch first extracts preliminary illumination features from the image I by a light convolution module composed of two convolution layers R After that, the preliminary illumination features are spliced with the feature output by the reflectance map extraction branch , and a convolution layer is connected to integrate the features, and finally a Sigmoid activation function is used to adjust the output to the value range of [0, 1] to obtain the illumination map wherein, represents the light convolution module, represents the connection operation of illumination features and reflection features.

[0008] As preferred, in step 2.2, the residual diffusion model recovers a high-quality clear illumination map by illumination component encoding and initial noise adding, illumination component step-by-step denoising, and illumination component decoding and enhancement result acquisition; The illumination component encoding and initial noise adding first expand the illumination map L into a three-channel feature map and input it into the pre-trained VQGAN encoder to obtain its latent space representation ; then a standard Gaussian noise is added to to generate the initial sampling state T at time step wherein, represents the feature channel expansion operation, represents the feature processing process of VQGAN encoder, represents the standard Gaussian noise distribution; The illumination component step-by-step denoising uses a denoising neural network SwinUNet to gradually remove noise from each intermediate sampling state of the illumination map in a decreasing manner from time step T to 1; The illumination component decoding and enhancement result acquisition, when the denoising process of the last step is iterated to t= 1, the final initial state encoding tensor obtained is the latent representation of the enhanced illumination map; input into the VQGAN decoder to recover to the pixel space, and take the average value of the three-channel feature map to obtain the final enhanced illumination map wherein, the feature processing process of VQGAN decoder, This indicates an average operation across the three channels.

[0009] Preferably, the noise removal is performed step by step on each intermediate sampling state of the illumination map, decreasing from T to 1 in time steps; wherein the sampling process corresponding to each time step includes the following two sub-steps: (1) Initial state estimation; For each time step, represent the current sampling state. Encoding of low-quality illumination maps and time step t The data is fed into the denoising neural network SwinUNet to estimate the initial state after complete noise removal. in, The encoding and decoding process of a window-based self-attention mechanism represents a denoising neural network. This represents a multilayer perceptron used to generate temporal vector embeddings; Represents a join operation; (2) Previous state estimation; Combined with the initial state estimated in the previous step Current status and time step t Construct the mean term of the probability distribution of all sampled states And sample the state representation of the previous time step. in, and This represents learnable parameters used to achieve a smooth transition between different time steps; This represents standard Gaussian noise.

[0010] Preferably, in step 2.3, the residual estimation model achieves reflection map enhancement through shallow feature extraction of the reflection map, reflection map residual estimation, and reflection map enhancement result acquisition. The shallow feature extraction of the reflection map involves inputting the reflection map R into a convolutional layer to extract shallow features. = MLP(R); This represents a multilayer perceptron used to generate temporal vector embeddings; The reflection map residual estimation is performed by estimating the reflection residual layer by layer through several cascaded RSTB modules. Each RSTB module consists of several cascaded STL layers and a convolutional layer. The enhanced reflection map result is obtained by performing residual estimation through M RSTB modules, and then connected to a convolutional layer. Feature integration is then achieved, followed by integration with the initial shallow features. The connections are made, and finally, the mixture passes through a convolutional layer. Obtain an enhanced reflection map in, This represents the output of the Mth RSTB module. This represents a join operation.

[0011] Preferably, for each STL layer, the input feature map is divided into the reflection map residual estimation. There are 3 local window blocks, and the feature input of each window block is denoted as . Where H and W represent the width and height of the image, M represents the size of the window, and C is the feature dimension; The corresponding query vector is obtained through linear projection. Key vector Sum value vector ,in, , and This represents a multilayer perceptron (MLP) that generates corresponding vectors through projection; subsequently, a self-attention operation is performed. Where s represents the learnable scaling factor and b represents the learnable positional encoding; subsequently, after batch normalization layer processing and residual connections, and then after processing and residual connections through a multilayer perceptron, the fused features based on window self-attention are obtained. Among them, LN Indicates batch normalization layer, MLP Represents a multilayer perceptron; For the i Each RSTB module, after being processed by L STL layers, yields intermediate features. , ;in, Indicates the first i The first RSTB module j STL layer; then add a convolutional layer and implement residual connections before outputting. in, Indicates the first i A convolutional layer of RSTB, Indicates the first i The output characteristics of each RSTB.

[0012] As a preferred embodiment, step 3 includes the following sub-steps: Step 3.1: Enhancement processing of local image patches; firstly, random sampling points are generated globally within the high-resolution image. ,in Y This represents the number of sampling points; then, for each sampling point... Centered on a rectangle with dimensions of 256, a frame is constructed, and the image is cropped according to the coverage area of ​​each rectangle, thus obtaining a result containing... Y A collection of local images of each cropped block. Finally, for each cutout image block... Enhancement processing is performed, and the pixel value results of each image block before and after enhancement are recorded; Step 3.2: Color mapping curve construction; the color mapping curve is solved through the gray scale distribution before and after local image block enhancement, a monotonically increasing RGB three-channel color mapping lookup table (LUT), that is, the color mapping curve, is constructed, and color values with a statistical quantity less than a preset value are removed to improve the robustness and global consistency of the mapping curve; at the same time, for the problem of lack of mapping for some pixels, an interpolation and linear mapping strategy is adopted to ensure that the mapping curve covers all gray scale values; Step 3.3: Global image enhancement; the color mapping curve is used to perform pixel-by-pixel color transformation on the entire high-resolution image, thereby obtaining the enhancement result of the entire image.

[0013] As a preferred, the color mapping curve construction specifically includes the following sub-steps: (1) LUT initialization and pixel threshold calculation; First, the gray scale level is initialized to L = 256, the three-channel and the corresponding counting array are initialized, respectively; then the total number of pixels of all cropped images is counted , the statistical elimination proportion parameter of the pixel value is set to , the pixel counting threshold is obtained, and the gray scale values with a count exceeding T are used for LUT fitting; (2) Statistics of pixel pairs before and after enhancement and calculation of average mapping values; First, for each pair of original image block and enhanced image block, the pixel value gray scale change and the occurrence number are counted channel by channel, wherein , o is the pixel value of the original image block, e is the pixel value of the corresponding position of the enhanced image block; then for the gray scale value satisfying i , the average value is obtained to obtain the average enhanced pixel value corresponding to the original pixel i

[0014] (3) Missing value interpolation and high gray scale linear extension; If some gray values lack sufficient statistics, the mapping results of adjacent gray values are found, the LUT filling of the missing pixel values is completed by one-dimensional linear interpolation, and then monotonicity correction is performed to ensure that the mapping curve is strictly monotonically increasing; in addition, if the neighborhood of the missing pixel value has no statistics, the mapping result of the gray value in the LUT is calculated by using the method of linear extension of gray scale; specifically, assuming that the maximum valid gray index is , the mapping value is , then , the LUT mapping value of is calculated The linear mapping method is used to complete the smooth transition of the high gray area in the original image.

[0015] The technical scheme of the system of the application is: an image quality enhancement system based on a residual diffusion model and a residual estimation model, comprising: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image quality enhancement method based on the residual diffusion model and the residual estimation model.

[0016] Compared with the prior art, the beneficial effects of the application include: (1) The application designs an illumination-reflectance map decomposition module based on the Retinex theory, decomposes the input image into an illumination map and a reflection map through a convolutional neural network, and simulates the imaging mechanism of illumination-reflectance.

[0017] (2) The application removes noise from the illumination map decomposed from the low-quality image through iteration to recover a high-quality clear illumination map based on the residual diffusion model. The diffusion model has strong generation capability and can generate a high-quality brightness map from severely degraded brightness information.

[0018] (3) The application realizes reflection map enhancement based on the residual estimation model, and synchronously realizes noise suppression and detail information preservation in the process of enhancing the reflection through layer-by-layer residual estimation and residual connection.

[0019] (4) The application performs pixel-by-pixel fusion on the enhanced illumination map and reflection map to generate the final image enhancement result.

[0020] (5) For high-resolution image processing, the application further proposes an enhancement method based on a color mapping curve, uses a randomly cropped image block and its corresponding enhancement result to construct a global color mapping curve, and significantly improves the processing efficiency of high-resolution images while ensuring the enhancement effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] The technical solutions of the present application are further described below using examples and specific embodiments. In addition, some drawings are also used in the description of the technical solutions. For those skilled in the art, other drawings and the intention of the present application can also be obtained from these drawings without creative labor.

[0022] Figure 1 Method flowchart of the embodiment of the present application; Figure 2 Structure diagram of the illumination-reflection map decomposition module of the embodiment of the present application; Figure 3 Illumination enhancement processing flowchart based on the residual diffusion model of the embodiment of the present application; Figure 4 Reflection enhancement processing flowchart based on the residual estimation model of the embodiment of the present application; Figure 5 High-resolution image enhancement method processing flowchart of the embodiment of the present application; Figure 6 Comparison diagram of the results of the method and other methods on low-quality image enhancement of the embodiment of the present application; Figure 7 Comparison diagram of the high-resolution image enhancement effect of the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to facilitate those skilled in the art to understand and implement the present application, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0024] The present application provides an image stitching quality enhancement method based on residual diffusion model and residual estimation model, and the overall flowchart is as shown in Figure 1 The present method takes low-quality images (such as images with low light, low contrast, noise interference, and detail blur degradation) as input, aiming to significantly improve the visual quality and feature richness, and to provide clear and consistent brightness high-quality input for subsequent image stitching. Specifically, the following steps are included: Step 1: Reflection map and illumination map decomposition; In one embodiment, an illumination-reflection map decomposition module based on convolutional neural network is designed, as shown in Figure 2 The module takes low-quality images as input, and the whole contains two branches: reflection map extraction branch and illumination map extraction branch, which are respectively used to predict the illumination map and reflection map of the image, so as to realize the significant enhancement of the image in two feature levels.

[0025] In an embodiment, the reflectance map extraction branch adopts a classical 5-layer UNet structure as the reflectance map extraction branch. For the input low-quality image The present embodiment obtains multi-scale features through layer-by-layer encoding and decoding of the UNet network. Subsequently, the present embodiment integrates the multi-scale features through a standard two-dimensional convolution layer, and finally normalizes the output to the value space of [0, 1] using a Sigmoid activation function to obtain the reflectance component map R This process can be formulated as: (1) wherein, represents the layer-by-layer encoding and decoding operation of the UNet network, w and b represent the weight and bias parameters of the convolution layer, represents the activation function in the convolution layer. H and W represent the width and height of the image, R represents the feature vector space, and 3 represents the number of channels.

[0026] In an embodiment, the illumination map extraction branch first extracts preliminary illumination features of the image I through an illumination convolution module composed of the first two convolution layers, then splices the illumination features with the output features of the corresponding position of the reflectance branch R , connects a convolution layer for feature integration, and finally adjusts the output to the value range of [0, 1] using a Sigmoid activation function to obtain the illumination component map L This process can be specifically expressed by the following formula: (2) wherein, represents the illumination convolution module, represents the connection operation of the illumination features and the reflectance features.

[0027] Step 2: Illumination enhancement based on residual diffusion model; As the brightness information in low-light images is severely degraded, it is difficult to obtain high-quality brightness information if the existing convolutional neural network paradigm is used for learning and training. Therefore, in an embodiment, the present embodiment will utilize the powerful generation capability of the diffusion model to generate high-quality brightness from the degraded brightness information. The iterative generation paradigm of the traditional residual diffusion model (Denoising diffusion probabilistic models, DDPM) exhibits strong capability in modeling complex distributions and can effectively solve the problem of image quality degradation. In practical applications, an already degraded image is usually used as a conditional input to guide the diffusion model to gradually remove noise and restore high-quality clear images. However, the traditional diffusion model starts from completely random noise for iterative sampling, although it can generate high-quality results, but usually requires hundreds or even thousands of iterations. To solve this problem, the present embodiment uses the degraded image (the illumination component map extracted in step 1.1) as the result of the forward process, that is, it is regarded as the initial value of the sampling process, instead of pure random noise, aiming to improve the efficiency of the diffusion model while maintaining the generation quality. The network structure of the residual diffusion model is shown in Figure 3 , and the enhancement process of the illumination component map includes the following sub-steps: (1) Illumination component encoding and initial noise addition; After the illumination component map L is expanded into a three-channel feature map, it is input into the pre-trained VQGAN encoder to obtain its latent space representation . Then, a standard Gaussian noise is added to to generate the initial sampling state T at time step . Its processing process can be formulated as: (3) where , represents the feature processing process of the VQGAN encoder, represents a standard Gaussian noise distribution.

[0028] (2) Step-by-step denoising of the illumination component. The present embodiment uses a UNet network improved based on SwinTransformer, namely SwinUNet, to gradually remove noise from each intermediate sampling state of the illumination component in a time step decreasing (from T to 1) manner. Each time step corresponds to the following two sub-steps: 1) Initial state estimation. Specifically, for each time step, the present embodiment estimates the sampling state at the current time, the encoding of the low-quality illumination map , and the time stept is input into the denoising neural network SwinUNet to estimate the initial state after complete removal of noise This process can be formulated as: (4) where, represents the encoding and decoding process of the denoising network based on the window self-attention mechanism, represents a multi-layer perceptron for generating time vector embedding.

[0029] 2) Previous state estimation. The initial state estimated in the previous step is combined with the current state , the current state and the time step t to construct the mean item of the probability distribution of all sampled states , and the state representation of the previous time step is sampled, and this process can be formulated as: (5) (6) where, and represent learnable parameters to achieve smooth transition between different time steps. represents standard Gaussian noise.

[0030] (3) Illumination component decoding and enhanced result acquisition. When the denoising process of the previous step is iterated to t= 1, the final initial state encoding tensor is obtained, which is the latent representation of the enhanced illumination map. Thereafter, the is input into the VQGAN decoder to restore to the pixel space, and the average value of the three-channel feature map is taken to obtain the final enhanced illumination map . This process can be formulated as: #(7) where, the feature processing process of the VQGAN decoder, represents a three-channel average operation.

[0031] At this point, the enhanced illumination map after enhancement processing has been recovered from the low-quality input image.

[0032] Step 3: Reflection enhancement based on residual estimation model; The reflection component can be considered as the superposition of the normal reflection image and the noise image. In an embodiment, a residual estimation network based on Swin Transformer is used to model and remove the noise component therein, and to enhance the feature information therein, thereby recovering a clear reflection image. The residual estimation model shown in FIG. 8 is used to achieve noise suppression and structure information enhancement in the reflection map. Specifically, the following sub-steps are included: Figure 4 (1) Reflection map shallow feature extraction. In this embodiment, the reflection map R obtained by decomposition is input into a convolutional layer to extract shallow features This process can be formulated as: = MLP(R);(8) (2) Reflection map residual estimation based on RSTB module. In this embodiment, a residual Swin Transformer module (RSTB) is designed to estimate the reflection residual layer by layer. Each RSTB module is composed of multiple Swin Transformer layers (STL) and convolutional layers. The processing process includes the following sub-steps: 1) Single-layer STL processing. For each STL, the input feature map is divided into local window blocks (H and W represent the width and height of the image, and M represents the size of the window), and the feature input of each window block can be represented as (C is the feature dimension). The corresponding query vector Q, key vector K, and value vector V are obtained through linear projection, which can be expressed by the following formula: (9) wherein , and represent the MLPs for generating the corresponding vectors. Thereafter, self-attention operation is performed, which can be formulated as: (10) wherein s represents a learnable scaling coefficient, and b represents a learnable position encoding. Subsequently, batch normalization layer processing and residual connection are performed, followed by multi-layer perceptron processing and residual connection, to obtain the window self-attention-based fusion features This process can be expressed by the following formula: (11) wherein LN represents a batch normalization layer, and MLP represents a multi-layer perceptron.

[0033] 2) Residual connection and feature integration. For the i th RSTB module, after L STL layer processing, the intermediate feature ​This process can be formalized as follows: (12) in, Indicates the first i The first RSTB module j STL layer. Afterwards, a convolutional layer is added and residual connections are implemented; this process can be expressed by the following formula: #(13) in, Indicates the first i A convolutional layer of RSTB, Indicates the first i The output characteristics of each RSTB.

[0034] (3) Obtaining the reflection map enhancement result. After residual estimation by M RSTB modules, a convolutional layer is then added to achieve feature integration, which is then combined with the initial shallow features. The layers are connected, and finally, an enhanced reflection map is obtained through a convolutional layer. This process can be formulated as follows: (14) in, This represents the output of the Mth RSTB module. This represents the final result of the enhanced reflection image.

[0035] Step 4: Enhance image synthesis.

[0036] Based on the enhanced illumination map and enhanced reflectance map obtained in steps 2 and 3, this embodiment performs pixel-by-pixel dot product fusion on the two to generate the final enhanced image. This process can be expressed by the following formula: (15) Thus, this embodiment has successfully achieved the enhancement processing of low-quality images, effectively solving the quality degradation problems of insufficient luminosity, low contrast, and missing details in some stitched images, and providing strong support for obtaining high-quality image stitching results.

[0037] Considering the limitations of video memory when processing high-resolution stitched images, it is impossible to directly input the entire image into a neural network for enhancement processing. While common block-based processing strategies are feasible, they easily lead to color inconsistencies between blocks. Although weighted smoothing can alleviate this problem, it cannot completely solve it. Furthermore, a large number of blocks significantly reduces processing speed. To overcome this challenge, one implementation proposes a high-resolution image enhancement method based on color mapping curve adjustment, such as... Figure 5The advantages of this method are that only a small number of image blocks need to be enhanced, and by solving a unified color mapping curve, high-quality and color-coordinated enhanced results can be obtained at a global scale, significantly improving the processing efficiency of high-resolution images. The processing process specifically includes the following sub-steps: (1) Enhancement processing of local image blocks. First, the present embodiment generates random sampling points in the global range of the high-resolution image , wherein M represents the number of sampling points. Thereafter, a rectangular frame with a length and width of 256 is constructed around each sampling point , and image cropping is performed according to the coverage of each rectangular frame, thereby obtaining a local image set M containing cropped blocks. Thereafter, the present embodiment performs enhancement processing on each cropped image block using the single low-resolution image enhancement method mentioned in steps 1-4, and records the pixel value results before and after enhancement of each image block.

[0038] (2) Color mapping curve construction. The color mapping curve is solved by the gray scale distribution before and after the enhancement of the local image blocks, a monotonically increasing RGB three-channel color mapping lookup table (LUT) is constructed, that is, the color mapping curve, and color values with insufficient statistics are eliminated to improve the robustness and global consistency of the mapping curve. In addition, for the problem of lack of mapping for some pixels, an interpolation and linear mapping strategy is adopted to ensure that the mapping curve covers all gray values. Specifically, the following sub-steps are included: 1) LUT initialization and pixel threshold calculation. The gray scale level is initialized to L = 256, and the three channels of LUT and the corresponding counting array are initialized, which can be expressed by the following formula: .(16) Thereafter, the total number of pixels of all cropped images is counted, and the following formula is used to calculate: , (17) Set the pixel value statistical elimination ratio parameter , then the pixel counting threshold can be calculated by the following formula: , (18) It is worth noting that the present application only uses the gray values with a count exceeding T for LUT fitting.

[0039] 2) Statistics of enhanced and unenhanced pixel pairs and calculation of average mapping value. For each pair of original image blocks and enhanced image blocks, the pixel value gray scale change and occurrence frequency are counted channel by channel, which can be expressed by the following formula: (19) in, , o These are the pixel values ​​of the original image block. e To enhance the pixel values ​​at corresponding locations in the post-image block. Subsequently, for those satisfying... grayscale value i The average enhanced pixel value corresponding to the original pixel i is obtained by averaging, which can be expressed by the following formula: (20) 3) Missing value interpolation and high grayscale linear extension. If some grayscale values ​​lack sufficient statistical values, the mapping results of adjacent grayscale values ​​are found, and the missing pixel values ​​are completed through one-dimensional linear interpolation. LUT Filling is performed, followed by monotonicity correction to ensure the mapping curve is strictly monotonically increasing. Furthermore, if the neighborhood of a missing pixel value has no statistical value (often appearing in high grayscale regions), this invention uses a grayscale linear extension method to calculate the grayscale value in... LUT The mapping result in the image. Specifically, let the maximum effective grayscale index be... Its mapping value is Then for Calculate it according to the following formula LUT Mapping value: (twenty one) This linear mapping method achieves a smooth transition of high grayscale areas in the original image.

[0040] (3) Global image enhancement.

[0041] The three-channel LUT color mapping curve obtained through the above steps maintains the authenticity of local statistics while achieving monotonically increasing and color consistency globally, making it suitable for global color enhancement of large-format images. Therefore, this embodiment performs pixel-by-pixel color transformation on the entire high-resolution image according to this color mapping curve, thereby obtaining the enhanced result of the entire image. Thus, this embodiment achieves the goal of high-resolution image enhancement, significantly improving the computational efficiency of the enhancement process while ensuring the accuracy of the enhancement result.

[0042] The present invention will be further illustrated by the following experiments.

[0043] This invention conducted experiments on low-quality image enhancement on the publicly available dataset VE-LOL-L. The comparison results between the proposed method and currently widely used methods are as follows: Figure 6The evaluation index and visual result of the method are optimal, which fully verifies that the method can effectively solve the image quality degradation problem. Regarding the high-resolution image enhancement, the comparison results of the method and the current mainstream method before and after enhancement are shown in FIG. 6. The method of the present application also achieves the optimal enhancement performance. The above experimental results fully verify that the present application can effectively realize the quality enhancement in image stitching, and provide strong technical support for generating high-quality stitched image results. Figure 7 The evaluation index and visual result of the method are optimal, which fully verifies that the method can effectively solve the image quality degradation problem. Regarding the high-resolution image enhancement, the comparison results of the method and the current mainstream method before and after enhancement are shown in FIG. 6. The method of the present application also achieves the optimal enhancement performance. The above experimental results fully verify that the present application can effectively realize the quality enhancement in image stitching, and provide strong technical support for generating high-quality stitched image results.

[0044] It should be understood that the above-described embodiments are part of the embodiments of the present application, rather than all the embodiments. In addition, the technical features in each embodiment or single embodiment provided by the present application can be combined with each other to form a feasible technical solution, and such combination is not restricted by the sequence of steps and / or structure mode, but should be based on the realization by the ordinary skilled in the art. When the combination of technical solutions appears contradictory or unfeasible, it should be considered that such combination of technical solutions does not exist, and is not within the protection scope of the present application.

[0045] It should be understood that the above description of the preferred embodiments is more detailed, and therefore should not be considered as a limitation on the scope of patent protection of the present application. The ordinary skilled in the art can make substitutions or modifications under the inspiration of the present application without departing from the scope of protection claimed by the present application, which falls within the protection scope of the present application. The protection scope of the present application should be subject to the appended claims.

Claims

1. An image quality enhancement method based on a residual diffusion model and a residual estimation model, characterized in that, The method comprises the following steps: Step 1: Randomly sampling in the global range of large-scale images, and performing image cropping centered on each sampling point to obtain a plurality of local image blocks; Step 2: Performing enhancement processing on the local image blocks to obtain pixel value results before and after enhancement of each image block; The specific implementation comprises the following sub-steps: Step 2.1: decomposing the input image block I into an illumination map L and a reflection map R through a convolutional neural network to simulate the imaging mechanism of illumination-reflection; Step 2.2: based on a residual diffusion model, iteratively remove noise from the illumination map to recover a high-quality clear illumination map; Step 2.3: based on a residual estimation model, implement reflection map enhancement, and through layer-by-layer residual estimation and residual connection, simultaneously achieve noise suppression and detail information preservation in the process of enhancing the reflection; Step 2.4: Perform a pixel-wise dot product fusion of the enhanced illumination map and the reflection map to generate the final enhanced imagery ; Step 3: using the local image blocks and their corresponding enhancement results to construct a global color mapping curve, which significantly improves the processing efficiency of high-resolution images while ensuring the enhancement effect.

2. The method of claim 1, wherein the method is based on a residual diffusion model and a residual estimation model. In step 1, first generate random sampling points in the global range of high-resolution images , wherein Y represents the number of sampling points; then, as the center of each sampling point , construct a rectangular frame with a length and width of 256, and perform image cropping according to the coverage range of each rectangular frame, so as to obtain a local image set containing Y cropped blocks .

3. The method of claim 1, wherein the method further comprises: determining a residual diffusion model and a residual estimation model based on the image quality enhancement model; and applying the residual diffusion model and the residual estimation model to the image to generate the enhanced image. In step 2.1, the illumination-reflection map decomposition module based on the convolutional neural network decomposes the input low-quality image into an illumination map and a reflection map; the illumination-reflection map decomposition module includes two branches: a reflection map extraction branch and an illumination map extraction branch, which are used to predict the illumination map and the reflection map of the image respectively to achieve significant enhancement of the image at two feature levels; The reflection map extraction branch, for the input low-quality image First, the multi-scale features are obtained by layer-by-layer encoding and decoding through the UNet network, then the multi-scale features are integrated through the two-dimensional convolution layer, and finally the output is normalized to the value space of [0, 1] by using the Sigmoid activation function to obtain the reflection map Wherein, represents the layer-by-layer encoding and decoding operation of the UNet network, and w and b represent the weight and bias parameters of the convolution layer, ) represents the activation function in the convolution layer; H and W represent the width and height of the image; The illumination map extraction branch first uses an illumination convolution module composed of two convolutional layers to process the image. I Extract preliminary illumination features, and then combine these preliminary illumination features with the output features of the reflectance extraction branch. R Then concatenate the layers and connect them to a convolutional layer. Feature integration is performed, and finally, the Sigmoid activation function is used to adjust the output to the range of [0,1] to obtain the illumination map. in, Represents the illumination convolution module. This represents the connection operation between illumination characteristics and reflection characteristics.

4. The method of claim 1, wherein the method further comprises: determining a residual diffusion model and a residual estimation model based on the image quality enhancement model; and applying the residual diffusion model and the residual estimation model to the image to generate the enhanced image. In step 2.2, the residual diffusion model recovers a high-quality clear illumination map through illumination component coding and initial noise addition, illumination component step-by-step denoising, and illumination component decoding and enhancement result acquisition; The illumination component encoding and initial noise addition first add the illumination map L to the initial state , and then add a standard Gaussian noise to the extended three-channel feature map , to generate the initial sampling state T of the time step wherein, represents a feature channel expansion operation, represents a feature processing procedure of the VQGAN encoder, represents a standard Gaussian noise distribution. The illumination component step-by-step denoising uses a denoising neural network SwinUNet to gradually remove noise from each intermediate sampling state of the illumination map in a decreasing manner from T to 1; The acquisition of the illumination component decoding and enhancement results, when the previous denoising process iterates to... t= At time 1, the final obtained initial state encoding tensor That is, the latent representation of the enhanced illumination map; The input is fed into the VQGAN decoder to restore the pixel space, and the feature maps of the three channels are averaged to obtain the final enhanced illumination map. in, The feature processing procedure of the VQGAN decoder This indicates an average operation across the three channels.

5. The method of claim 4, wherein the method further comprises: The decreasing manner from T to 1 gradually removes noise from each intermediate sampling state of the illumination map; Each time step corresponds to a sampling process comprising the following two sub-steps: (1) initial state estimation; For each time step, the sampled state representation at the current time step , encoding of low-quality light maps and time steps t are input into a denoising neural network SwinUNet to estimate the initial state after complete removal of noise wherein denotes the encoding and decoding process of the denoising neural network based on the window self-attention mechanism, denotes a multi-layer perceptron for generating a temporal vector embedding; denotes a concatenation operation; (2) previous state estimation; the initial state estimated in the previous step , the current state and the time step t Construct the mean term of the probability distribution of all sampled states , and sample the state representation of the previous time step where and denote learnable parameters to achieve smooth transition between different time steps; denotes standard Gaussian noise.

6. The method of claim 1, wherein the method further comprises: determining a residual diffusion model and a residual estimation model based on the image quality enhancement model; and applying the residual diffusion model and the residual estimation model to the image to generate the enhanced image. In step 2.3, the residual estimation model achieves reflection map enhancement through reflection map shallow feature extraction, reflection map residual estimation, and reflection map enhancement result acquisition; The reflection map shallow layer feature extraction inputs the reflection map R into a convolutional layer to extract shallow layer features = MLP(R); represents a multi-layer perceptron for generating a temporal vector embedding; The reflection map residual error is estimated layer by layer through a plurality of serial RSTB modules, each RSTB module being composed of a plurality of serial STL layers and a convolution layer; for each STL layer, the input feature map is divided into a plurality of local window blocks, the feature input of each window block being denoted as ; wherein H and W represent the width and height of the image, M represents the size of the window, and C is the feature dimension; The corresponding query vector , key vector and value vector are obtained through linear projection, wherein , , and represent a multi-layer perception MLP for generating the corresponding vectors; thereafter, a self-attention operation is performed wherein s represents a learnable scaling coefficient and b represents a learnable position encoding; subsequently, batch normalization layer processing and residual connection are performed, and then multi-layer perception processing and residual connection are performed to obtain the fusion feature based on window self-attention wherein LN represents a batch normalization layer, and MLP represents a multi-layer perception; The reflection map enhancement result is obtained through residual estimation of M RSTB modules, and then accessed to a convolution layer The feature integration is implemented, and then combined with the initial shallow features The connection is performed, and finally accessed to a convolution layer The enhanced reflection map is obtained Wherein, represents the output result of the Mth RSTB module, represents the connection operation.

7. The method of claim 6, wherein the method further comprises: determining a residual error between the reconstructed image and the original image; and updating the residual error diffusion model and the residual error estimation model based on the determined residual error. The reflection map residual estimation is obtained by processing the first i RSTB module through L STL layers to obtain intermediate features , ; wherein represents the first i STL layer of the first j RSTB module; after a convolution layer is added and a residual connection is implemented, output wherein represents a convolution layer of the first i RSTB, represents output features of the first i RSTB.

8. The method of claim 1-7, wherein, The specific implementation of step 3 comprises the following sub-steps: Step 3.1: enhancement processing of local image blocks; first, generate random sampling points in the global range of high-resolution images , wherein Y represents the number of sampling points; then, construct a rectangular frame with a length and width of 256 centered on each sampling point , and perform image cropping according to the coverage range of each rectangular frame, thereby obtaining a local image set containing Y cropped blocks ; finally, perform enhancement processing on each cropped image block , and record the pixel value results before and after the enhancement of each image block. Step 3.2: color mapping curve construction; solve the color mapping curve by the gray scale distribution before and after the enhancement of the local image block, construct a monotonically increasing RGB three-channel color mapping lookup table (LUT), i.e., the color mapping curve, and remove color values with a statistical quantity less than a preset value to improve the robustness and global consistency of the mapping curve; at the same time, for the problem of lack of mapping for some pixels, an interpolation and linear mapping strategy is adopted to ensure that the mapping curve covers all gray values; Step 3.1: global image enhancement; perform pixel-by-pixel color transformation on the entire high-resolution image according to the color mapping curve to obtain the enhancement result of the entire image.

9. The image quality enhancement method based on the residual diffusion model and the residual estimation model according to claim 8, characterized in that: The color mapping curve construction comprises the following sub-steps: (1) LUT initialization and pixel threshold calculation; First, initialize the gray scale as L = 256, initialize the three channels and the corresponding counting array respectively; then count the total number of pixels of all cropped images , set the pixel value statistical elimination ratio parameter , then the pixel counting threshold , and use the gray values whose counts exceed T for LUT fitting; (2) statistical enhancement before and after the pixel pair and average mapping value calculation; First, for each pair of original image block and enhanced image block, the pixel value gray level change is counted channel by channel and the number of occurrences , wherein , o is the pixel value of the original image block, e is the pixel value of the corresponding position of the enhanced image block; then for the gray level satisfying i , the average value is obtained to get the average enhanced pixel value corresponding to the original pixel i (3) missing value interpolation and high gray level linear extension; If some gray value lacks enough statistical value, the mapping result of adjacent gray value is found, the LUT filling of missing pixel value is completed by one-dimensional linear interpolation, and then monotonicity correction is carried out to ensure that the mapping curve is strictly monotonically increasing. In addition, if the neighborhood of the missing pixel value has no statistical value, the mapping result of the gray value in the LUT is calculated by using the method of linear extension of gray scale. Specifically, let the maximum valid gray index be , and the mapping value be , then , the LUT mapping value of is calculated The smooth transition of the high gray area in the original image is completed by this linear mapping method.

10. An image quality enhancement system based on a residual diffusion model and a residual estimation model, characterized by, comprise: one or more processors; a memory device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method for enhancing image quality based on residual diffusion model and residual estimation model according to any one of claims 1 to 9.

Images