Multi-degradation type adaptive image enhancement method and device

Abstract

This invention provides an adaptive image enhancement method and apparatus for multiple degradation types. The method includes: acquiring a degraded image and extracting degradation features from the degraded image to generate a degradation vector; dynamically adjusting the preset weight coefficients of the multi-scale feature extraction branches based on the degradation vector, and performing weighted fusion of the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features; generating detail masks and noise masks based on the degradation vector and fused features, optimizing the fused features to obtain optimized fused features; mapping the optimized fused features to a preliminary enhanced image, and performing color correction on the preliminary enhanced image based on the color statistics information of the degraded image to output the final enhanced image. The method and apparatus of this invention, through adaptive degradation perception and dynamic weighted fusion, can adapt to multiple degradation coexistence scenarios with a single algorithm, overcoming the limitations of traditional serial processing.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an adaptive image enhancement method and apparatus for multiple degradation types. Background Technology

[0002] In the fields of image processing and computer vision, enhancement techniques for degraded images have always been a research hotspot. Traditional methods are typically designed and optimized for a single type of image degradation. For example, algorithms such as RetinexNet focus on brightening images under low-light conditions, while networks like RCAN (Residual Channel Attention Networks) primarily address image super-resolution and deblurring problems. However, images captured in real-world scenarios, such as surveillance videos and footage from mobile devices, are often simultaneously affected by a combination of degradation factors, typically manifesting as "coexistence of low light, high noise, and compression block effects" or "motion blur accompanied by color distortion."

[0003] Existing technical solutions have significant shortcomings when facing complex scenarios with "coexistence of multiple degradations": if multiple single-function enhancement algorithms are simply chained together, the final result will suffer from serious loss of detail, improper amplification of noise, or color artifacts due to conflicts and error accumulation between processing steps; while a few all-in-one restoration models proposed in recent years, such as SwinIR based on Swin Transformer, can handle multiple degradations, but their model structure is large and computationally complex, making it difficult to meet the stringent low-latency requirements of scenarios such as real-time video stream enhancement.

[0004] Therefore, there is a lack of image enhancement methods in the existing technology that can intelligently perceive multiple types and degrees of degradation and perform adaptive and efficient processing based on this. Summary of the Invention

[0005] Therefore, it is necessary to provide an adaptive image enhancement method and apparatus for multiple degradation types to address the aforementioned technical problems.

[0006] In a first aspect, the present invention provides an adaptive image enhancement method for multiple degradation types, comprising:

[0007] Acquire the degraded image and extract the degradation features from the degraded image to generate a degradation vector, wherein the degradation vector includes quantized values ​​of noise intensity, blur degree, low illumination degree and compression distortion degree;

[0008] Based on the degenerate vector, the preset weight coefficients of the multi-scale feature extraction branch are dynamically adjusted, and the extracted multi-scale features are weighted and fused according to the adjusted preset weight coefficients to obtain the fused features.

[0009] Based on the degenerate vector and the fused features, a detail mask and a noise mask are generated, and the fused features are optimized to obtain the optimized fused features.

[0010] The optimized fused feature map is used to create a preliminary enhanced image, and the color of the preliminary enhanced image is corrected based on the color statistics of the degraded image to output the final enhanced image.

[0011] Optionally, degradation features are extracted from the degraded image, including:

[0012] Convert the degraded image to a grayscale image;

[0013] Divide the grayscale image into multiple first image blocks of the same size;

[0014] Calculate the pixel variance value of each first image block, and based on the pixel variance values ​​of all first image blocks, determine the image block with the smallest variance value of the first preset ratio;

[0015] Calculate the arithmetic mean of the pixel variance values ​​corresponding to the image block with the smallest variance value, and use it as the variance estimate of the flat region.

[0016] The variance estimate of the flat region is normalized to obtain the quantized value of the noise intensity.

[0017] Optionally, degradation features are extracted from the degraded image, including:

[0018] Convert the degraded image to a grayscale image;

[0019] Gradient calculation is performed on the grayscale image to obtain the gradient magnitude map;

[0020] Count the number of first pixels whose magnitude is greater than a preset gradient threshold in the gradient magnitude map, and calculate the proportion of the number of first pixels in the total number of pixels in the gradient magnitude map;

[0021] The quantization value of the blur level is calculated based on the proportion of the first pixel in the total number of pixels in the gradient magnitude map.

[0022] Optionally, degradation features are extracted from the degraded image, including:

[0023] Extracting the luminance component from a degraded image;

[0024] Calculate the global brightness mean of the luminance component;

[0025] The global brightness mean is normalized to obtain the normalized global brightness mean. The normalized global brightness mean is then subtracted from the normalized global brightness mean by a preset constant to obtain the quantized value of the low illumination level.

[0026] Optionally, degradation features are extracted from the degraded image, including:

[0027] Convert the degraded image to a grayscale image;

[0028] Divide the grayscale image into multiple second image blocks of the same size;

[0029] Detect the pixel transition intensity at the boundary of each second image patch;

[0030] The number of second image blocks whose pixel transition intensity is greater than a preset transition threshold is counted;

[0031] The first ratio of the number of second image patches to the total number of second image patches is calculated as a quantization value of the degree of compression distortion.

[0032] Optionally, based on the degradation vector, the preset weight coefficients of the multi-scale feature extraction branch are dynamically adjusted, and the extracted multi-scale features are weighted and fused according to the adjusted preset weight coefficients to obtain fused features, including:

[0033] Set corresponding preset weight coefficients for each degradation type and each feature scale. The preset weight coefficients include preset noise weight coefficient, preset blur weight coefficient, preset low light weight coefficient and preset compression distortion weight coefficient.

[0034] Based on the quantization value of each degradation type in the degradation vector and the corresponding preset weight coefficient, calculate the contribution value of each degradation type to each feature scale;

[0035] The contribution values ​​of each feature scale are fused and normalized to generate the final weight coefficients corresponding to each feature scale.

[0036] Based on the final weight coefficients, the extracted multi-scale features are weighted and fused to obtain the fused features.

[0037] Optionally, based on the degenerate vector and the fused features, detail masks and noise masks are generated, and the fused features are optimized to obtain optimized fused features, including:

[0038] Edge features are extracted from the fused features to obtain edge features;

[0039] Based on edge features and a preset dynamic detail threshold, a detail mask is generated, whereby the detail mask is used to identify the regions of detail features that need to be retained.

[0040] The fused features are smoothed to obtain smoothed features;

[0041] The feature difference between the fused feature and the smoothed feature is calculated, and a noise mask is dynamically generated based on the feature difference. The noise mask is used to identify the noise feature regions that need to be suppressed.

[0042] Based on the detail mask and noise mask, the pixel values ​​of corresponding regions in the fused features are selectively adjusted to obtain the optimized fused features.

[0043] Optionally, the feature difference between the fused feature and the smoothed feature is calculated, and a noise mask is dynamically generated based on the feature difference, including:

[0044] Calculate the feature differences between fused features and smoothed features;

[0045] The noise threshold is dynamically determined based on the quantized value of the noise intensity in the degradation vector.

[0046] A noise mask is generated based on feature differences and a noise threshold.

[0047] Optionally, the optimized fused features are mapped to a preliminary enhanced image, and color correction is performed on the preliminary enhanced image based on the color statistics of the degraded image to output the final enhanced image, including:

[0048] The optimized fused features are mapped to a preliminary enhanced image;

[0049] Obtain first color statistics of the degraded image, wherein the first color statistics include the first mean and first variance of the degraded image in each color channel;

[0050] Obtain the second color statistics of the preliminary enhanced image, wherein the second color statistics include the second mean and second variance of the preliminary enhanced image in each color channel;

[0051] Based on the first color statistics of the degraded image and the second color statistics of the preliminary enhanced image, channel-by-channel color correction is performed on the preliminary enhanced image to obtain the final enhanced image.

[0052] In a second aspect, the present invention provides a multi-degradation type adaptive image enhancement apparatus, comprising:

[0053] The degradation perception module is used to acquire degraded images, extract degradation features from the degraded images, and generate degradation vectors. The degradation vectors include quantized values ​​of noise intensity, blur level, low illumination level, and compression distortion level.

[0054] The feature extraction and fusion module, connected to the degradation perception module, is used to dynamically adjust the preset weight coefficients of the multi-scale feature extraction branches based on the degradation vector, and to perform weighted fusion of the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features.

[0055] The optimization module, connected to the feature extraction and fusion module, is used to generate detail masks and noise masks based on the degradation vector and fused features, and to optimize the fused features to obtain optimized fused features.

[0056] The color correction module, connected to the optimization module, is used to map the optimized fused features into a preliminary enhanced image, and to perform color correction on the preliminary enhanced image based on the color statistics of the degraded image, outputting the final enhanced image.

[0057] The adaptive image enhancement method and apparatus for multiple degradation types provided by this invention, through an integrated degradation perception module, can simultaneously and accurately quantify multiple degradation types and their degrees, such as noise, blur, low illumination, and compression distortion, and dynamically adjust the processing strategy accordingly. This achieves efficient adaptation of a single model to scenarios with multiple degradation types coexisting, solving the problems of low efficiency and conflicting effects caused by the need to string together multiple dedicated algorithms in traditional solutions. Furthermore, it employs a detail and noise co-optimization mechanism, using dynamically generated double masks to accurately distinguish and process the detailed structures and noise interference in the image, effectively suppressing noise while significantly improving the preservation and clarity of key details such as texture and edges. In addition, the correction technique based on the color statistics of the original degraded image ensures the realism and consistency of the colors in the final enhanced image, avoiding common color distortion problems. Attached Figure Description

[0058] Figure 1a This is a flowchart illustrating a multi-degradation type adaptive image enhancement method provided in an embodiment of the present invention;

[0059] Figure 1b This is another flowchart illustrating the adaptive image enhancement method for multiple degradation types provided in this embodiment of the invention;

[0060] Figure 1c This is another flowchart illustrating the adaptive image enhancement method for multiple degradation types provided in this embodiment of the invention.

[0061] Figure 1d This is another schematic diagram of the multi-degradation type adaptive image enhancement method provided in the embodiments of the present invention;

[0062] Figure 2 A schematic diagram of the circuit module structure of the adaptive image enhancement device with multiple degradation types provided in an embodiment of the present invention;

[0063] Figure 3 This is an internal structural diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0065] like Figure 1a As shown, the present invention provides an adaptive image enhancement method for multiple degradation types, comprising:

[0066] Step S11: Obtain the degraded image and extract the degradation features from the degraded image to generate a degradation vector, wherein the degradation vector includes quantized values ​​of noise intensity, blur degree, low illumination degree and compression distortion degree;

[0067] Optionally, the quantization value of the noise intensity in the degradation features of the degradation image in step S11 includes:

[0068] Convert the degraded image to a grayscale image;

[0069] Divide the grayscale image into multiple first image blocks of the same size;

[0070] Calculate the pixel variance value of each first image block, and based on the pixel variance values ​​of all first image blocks, determine the image block with the smallest variance value of the first preset ratio;

[0071] Calculate the arithmetic mean of the pixel variance values ​​corresponding to the image block with the smallest variance value, and use it as the variance estimate of the flat region.

[0072] The variance estimate of the flat region is normalized to obtain the quantized value of the noise intensity.

[0073] The size of the first image patch can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the size of the first image patch is 8×8 pixels. This size provides a sample size of 64 pixels, which is sufficient to stably calculate the pixel variance value of the first image patch, effectively characterize the flat areas in the degraded image, and avoid the problems of smaller patches being easily interfered with by a single pixel or larger patches containing too much image structure.

[0074] The first preset ratio can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the first preset ratio is 10%.

[0075] For example, when calculating the degradation feature of "quantized noise intensity" in a degraded image, firstly, the color degraded image is converted to a corresponding grayscale image using OpenCV's cvtColor function; secondly, the grayscale image is divided into multiple 8×8 pixel first image blocks; thirdly, the pixel variance value of each first image block is calculated using NumPy's np.var() function, and NumPy's np.sort() is used to sort all pixel variance values ​​in ascending order, extracting the top 10% of image blocks with the smallest variance values; then, np.mean() is used to calculate the arithmetic mean of the pixel variance values ​​corresponding to the extracted image blocks with the smallest variance values, which is used as the variance estimate of the flat region; finally, the variance estimate of the flat region is normalized by dividing by 255 to obtain the quantized noise intensity value N, where the quantized noise intensity value is in the range [0,1].

[0076] Optionally, the step S11 of extracting the quantization value of the blur degree in the degradation features of the degradation image includes:

[0077] Convert the degraded image to a grayscale image;

[0078] Gradient calculation is performed on the grayscale image to obtain the gradient magnitude map;

[0079] Count the number of first pixels whose magnitude is greater than a preset gradient threshold in the gradient magnitude map, and calculate the proportion of the number of first pixels in the total number of pixels in the gradient magnitude map;

[0080] The quantization value of the blur level is calculated based on the proportion of the first pixel in the total number of pixels in the gradient magnitude map.

[0081] The preset gradient threshold can be flexibly selected by those skilled in the art according to actual needs, and is not limited here. Preferably, the preset gradient threshold is 20.

[0082] For example, when calculating the degradation feature of "blurry quantization value" in a degraded image, firstly, the color degraded image is converted to a corresponding grayscale image using OpenCV's cvtColor function; secondly, the gradient magnitude map of the grayscale image is calculated using the Sobel gradient operator to characterize the local changes in pixel intensity; thirdly, the number of first pixels in the gradient magnitude map that are greater than a preset gradient threshold (usually set to 20, based on a grayscale range of 0-255) is counted, and its proportion P in the total number of pixels in the gradient magnitude map is calculated; finally, based on the proportion P of the first pixel in the total number of pixels in the gradient magnitude map, the blurry quantization value is calculated using the following formula: B = 1 - P, where B is the blurry quantization value, and its value range is [0,1]. The larger the value of B, the less strong edge information there is in the degraded image, that is, the more blurred the degraded image is; P is the proportion of the first pixel in the total number of pixels in the gradient magnitude map.

[0083] Optionally, the extraction of quantization values ​​of low illumination levels from degradation features in the degradation image in step S11 includes:

[0084] Extracting the luminance component from a degraded image;

[0085] Calculate the global brightness mean of the luminance component;

[0086] The global brightness mean is normalized to obtain the normalized global brightness mean. The normalized global brightness mean is then subtracted from the normalized global brightness mean by a preset constant to obtain the quantized value of the low illumination level.

[0087] For example, when calculating the degradation feature of "quantization value of low illumination" in a degraded image, firstly, the color degraded image is converted from the BGR color space to the YUV color space using the OpenCV cv2.cvtColor function to obtain a YUV image, and the Y channel is extracted from the YUV image as the luminance component; secondly, the average value of all pixels in the luminance channel is calculated using the NumPy np.mean() function, that is, the global luminance mean is calculated; thirdly, the global luminance mean is divided by 255 to obtain the normalized global luminance mean, and then calculated according to the formula L=1-NOR. mean The quantization value of the low illumination level is calculated; where L is the quantization value of the low illumination level, and its value ranges from [0,1]. The larger the L value, the darker the degraded image and the more severe the low illumination level; the smaller the L value, the brighter the degraded image and the milder the low illumination level. NOR mean The normalized global average brightness value; 1 is a preset constant.

[0088] It should be noted that if the degraded image is a grayscale image, the grayscale values ​​of the grayscale image can be used directly as the luminance component without color space conversion.

[0089] Optionally, the quantization value of the compression distortion degree in the degradation features of the degradation image in step S11 includes:

[0090] Convert the degraded image to a grayscale image;

[0091] Divide the grayscale image into multiple second image blocks of the same size;

[0092] Detect the pixel transition intensity at the boundary of each second image patch;

[0093] The number of second image blocks whose pixel transition intensity is greater than a preset transition threshold is counted;

[0094] The first ratio of the number of second image patches to the total number of second image patches is calculated as a quantization value of the degree of compression distortion.

[0095] The size of the second image block can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the size of the second image block is 4×4 pixels.

[0096] The preset transition threshold can be flexibly selected by those skilled in the art according to actual needs, and is not limited here. Preferably, the preset transition threshold is 15.

[0097] For example, when calculating the degradation feature of "quantization value of compression distortion" in a degraded image, firstly, the color degraded image is converted to a corresponding grayscale image using OpenCV's cvtColor function; secondly, the grayscale image is divided into multiple 4×4 pixel second image blocks; then, the pixel jump intensity at the boundary of each second image block is detected, specifically by calculating the brightness difference between adjacent pixels in the horizontal and vertical directions within the second image block, and selecting the maximum value as the pixel jump intensity characterization of the second image block; next, the number of second image blocks with jump intensity greater than a preset jump threshold (usually set to 15, based on a grayscale range of 0-255) is counted; finally, the first ratio of the number of second image blocks with jump intensity greater than the preset jump threshold to the total number of second image blocks is calculated as the quantization value C of the compression distortion degree, where the quantization value of the compression distortion degree ranges from [0,1]. The larger the C value, the more severe the compression distortion degree of the degraded image; the smaller the C value, the less severe the compression distortion degree of the degraded image. This method is based on the principle that block compression algorithms (such as JPEG) can easily introduce discontinuous transitions at block boundaries. It objectively assesses the severity of compression distortion by quantizing the proportion of boundary transitions. The calculation process is efficient and does not rely on frequency domain transformation.

[0098] Step S12: Based on the degenerate vector, dynamically adjust the preset weight coefficients of the multi-scale feature extraction branch, and perform weighted fusion of the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features;

[0099] In one optional embodiment of the present invention, such as Figure 1b As shown, step S12 specifically includes:

[0100] Step S121: Set corresponding preset weight coefficients for each degradation type and each feature scale, wherein the preset weight coefficients include preset noise weight coefficients, preset blur weight coefficients, preset low illumination weight coefficients and preset compression distortion weight coefficients;

[0101] For the preset weighting coefficients, those skilled in the art can flexibly select them according to actual needs, and there is no limitation here. For example, the preset weighting coefficients are as follows: preset noise weighting coefficient is [0.6, 0.3, 0.1], preset blur weighting coefficient is [0.2, 0.3, 0.5], preset low light weighting coefficient is [0.7, 0.2, 0.1], and preset compression distortion weighting coefficient is [0.5, 0.3, 0.2].

[0102] Step S122: Calculate the contribution of each degradation type to each feature scale based on the quantization value of each degradation type in the degradation vector and the corresponding preset weight coefficient;

[0103] Step S123: The contribution values ​​of each feature scale are fused and normalized to generate the final weight coefficients corresponding to each feature scale.

[0104] Step S124: Based on the final weight coefficients, the extracted multi-scale features are weighted and fused to obtain the fused features.

[0105] Optionally, before step S121, the method of the present invention further includes: using convolution kernels of different sizes to extract features at different scales from the degraded image to obtain multi-scale features.

[0106] The size and number of convolutional kernels can be flexibly selected by those skilled in the art according to actual needs, and are not limited here. For example, a 3×3 pixel convolutional kernel (64 channels, stride 1, padding 1) can be used to extract fine-scale features, a 5×5 pixel convolutional kernel (64 channels, stride 1, padding 2) can be used to extract medium-scale features, and a 7×7 pixel convolutional kernel (64 channels, stride 1, padding 3) can be used to extract coarse-scale features.

[0107] Assuming that three convolutional kernels of different sizes are used to extract fine-scale, medium-scale, and coarse-scale features from the degraded image, the corresponding fine-scale features, medium-scale features, and coarse-scale features are obtained. The degradation vector of the degraded image is D=[N,B,L,C]=[0.4,0.3,0.8,0.2]. The preset weight coefficients are as follows: preset noise weight coefficient is [0.6,0.3,0.1], preset blur weight coefficient is [0.2,0.3,0.5], preset low light weight coefficient is [0.7,0.2,0.1], and preset compression distortion weight coefficient is [0.5,0.3,0.2].

[0108] First, based on the quantized values ​​of each degradation type in the degradation vector and the corresponding preset weight coefficients, the contribution of each degradation type to each feature scale is calculated: the contribution of the quantized value N of noise intensity is [0.4×0.6=0.24, 0.4×0.3=0.12, 0.4×0.1=0.04]; the contribution of the quantized value B of blurriness is [0.3×0.2=0.06, 0.3×0.3=0.09, 0.3×0.5=0.15]; the contribution of the quantized value L of low illumination is [0.8×0.7=0.56, 0.8×0.2=0.16, 0.8×0.1=0.08]; the contribution of the quantized value C of compression distortion is [0.2×0.5=0.1, 0.2×0.3=0.06, 0.2×0.2=0.04].

[0109] Then, the contribution values ​​of each feature scale are added together to obtain the total contribution of the fine scale as 0.24+0.06+0.56+0.1=0.96, the total contribution of the mesoscale as 0.12+0.09+0.16+0.06=0.43, and the total contribution of the coarse scale as 0.04+0.15+0.08+0.04=0.31. After normalization, the final weight coefficients are obtained, including: final weight coefficient of fine scale W1=0.96 / (0.96+0.43+0.31)≈0.5647, final weight coefficient of mesoscale W2=0.43 / (0.96+0.43+0.31)≈0.2529, and final weight coefficient of coarse scale W3=0.31 / (0.96+0.43+0.31)≈0.1824.

[0110] Finally, the extracted fine-scale features F1, mesoscale features F2, and coarse-scale features F3 are weighted and fused according to the final weight coefficients to form F = W1×F1 + W2×F2 + W3×F3 = 0.5647×F1 + 0.2529×F2 + 0.1824×F3.

[0111] Step S13: Based on the degenerate vector and the fused features, generate detail mask and noise mask, optimize the fused features, and obtain the optimized fused features;

[0112] In one optional embodiment of the present invention, such as Figure 1c As shown, step S13 specifically includes:

[0113] Step S131: Extract edge features from the fused features to obtain edge features;

[0114] Step S132: Generate a detail mask based on edge features and a preset dynamic detail threshold, wherein the detail mask is used to identify the detail feature regions that need to be retained;

[0115] Step S133: Smooth the fused features to obtain smoothed features;

[0116] Step S134: Calculate the feature difference between the fused feature and the smoothed feature, and dynamically generate a noise mask based on the feature difference, wherein the noise mask is used to identify the noise feature regions that need to be suppressed;

[0117] Optionally, step S134 specifically includes:

[0118] Step S1341: Calculate the feature differences between the fused features and the smoothed features;

[0119] Step S1342: Dynamically determine the noise threshold based on the quantized noise intensity value in the degradation vector;

[0120] Step S1343: Generate a noise mask based on feature differences and a noise threshold.

[0121] It should be noted that steps S1341 to S1343 are not shown in the figure for ease of description only.

[0122] Step S135: Based on the detail mask and the noise mask, selectively adjust the pixel values ​​of the corresponding regions in the fused features to obtain the optimized fused features.

[0123] In one specific embodiment of the present invention, Laplacian edge detection is first performed on the input fused features to extract edge features E, and a preset dynamic detail threshold, namely the preset dynamic detail threshold T, is dynamically set according to a second preset proportion (e.g., 30%) of the maximum value in the edge features. detail =Second preset ratio × max(E) = 30% × max(E), generate detail mask M to identify areas where details need to be preserved. detail (For example, perform point-by-point judgment on each pixel position in edge feature E: if the position of...) Value greater than or equal to This position is set to 1, indicating that details need to be preserved; if this position is... Value less than The position is set to 0).

[0124] Subsequently, Gaussian filtering is applied to the fused features to obtain smoothed features, and the absolute difference map (i.e., feature difference) between the fused features and the smoothed features is calculated. ,in, As a feature of fusion, (For smoothing features), and dynamically adjust the noise threshold (i.e., the noise threshold) based on the quantized value of the noise intensity in the degradation vector. In other words, the larger the quantization value of the noise intensity, the lower the noise threshold, thus generating a noise mask M that identifies the area where noise needs to be suppressed. noise (For example, perform point-by-point judgment on each pixel position in the feature difference ∆F: if the position of...) Value less than This position is set to 1, indicating that noise suppression is required; if this position is... Value greater than or equal to The position is set to 0).

[0125] Finally, based on the detail mask M detail With noise mask M noise Selective adjustments are made to the fused features: the original features are preserved in the detail masked areas to maintain sharpness, while the noise masked and non-detailed areas are replaced with smooth features to suppress noise, thus obtaining an optimized fused feature (i.e., optimized fused feature F) that simultaneously improves detail preservation and noise suppression. opt =F×M detail +F smooth ×M noise This process achieves precise differentiation and coordinated processing of details and noise through an adaptive thresholding mechanism.

[0126] Step S14: Map the optimized fused features to a preliminary enhanced image, and perform color correction on the preliminary enhanced image based on the color statistics of the degraded image to output the final enhanced image.

[0127] In one optional embodiment of the present invention, such as Figure 1d As shown, step S14 specifically includes:

[0128] Step S141: Map the optimized fused features to a preliminary enhanced image;

[0129] Step S142: Obtain the first color statistics of the degraded image, wherein the first color statistics include the first mean and the first variance of the degraded image in each color channel;

[0130] Step S143: Obtain the second color statistics of the preliminary enhanced image, wherein the second color statistics include the second mean and second variance of the preliminary enhanced image in each color channel;

[0131] Step S144: Based on the first color statistical information of the degraded image and the second color statistical information of the preliminary enhanced image, perform channel-by-channel color correction on the preliminary enhanced image to obtain the final enhanced image.

[0132] In one specific embodiment of the present invention, firstly, the first mean value (i.e., Rmax) of the degraded image in the RGB three color channels is calculated respectively. avg G avg B avg ) and the first variance (i.e. R) var G var B var ), which serves as the primary color statistical information.

[0133] Subsequently, the optimized fused features are mapped to the preliminary enhanced image I using a 1×1 pixel convolution kernel. temp (i.e., RGB image), and obtain the second mean (i.e., R') of the preliminarily enhanced image in the RGB three color channels. avg G' avg B' avg ) and the second variance (i.e., R') var G' var B' var ), as second color statistical information.

[0134] Finally, based on the first and second color statistical information, a channel-by-channel correction is performed on the initially enhanced image, specifically applying the correction formula to each channel value of each pixel: I corr(x,y,c) =(I temp(x,y,c) -M temp-c )×(Var degraded-c / Var temp-c )+M degraded-c Where c∈{R,G,B}; I corr(x,y,c) This represents the c-channel value of a pixel (x, y) in the initially enhanced image after correction; I temp(x,y,c) M represents the c-channel value of a pixel (x, y) in the initially enhanced image; temp-c This represents the second mean of the c channel of the initially enhanced image; Var degraded-c The first variance of the c channel of the degraded image; Var temp-c M represents the second variance of the c channel of the initially enhanced image; degraded-c This represents the first mean of the c channel of the degraded image.

[0135] It should be noted that when c represents the R channel, the correction formula is: I corr(x,y,R) =(I temp(x,y,R) -M temp-R )×(Var degraded-R / Var temp-R )+Mdegraded-R =(I temp(x,y,R) -R' avg )×(R var / R' var )+R avg Other cases follow the same principle and will not be elaborated here.

[0136] Furthermore, the degraded image in this invention is a natural image, not a solid color image. Therefore, the first variance of each channel of the degraded image and the second variance of each channel of the preliminary enhanced image are not 0, i.e., R0 var G var B var 、R' var G' var B' var Var temp-c and Var temp-R None of them are 0.

[0137] The linear transformation described above ensures that the brightness distribution (mean) and contrast variation (variance) of each color channel in the corrected image remain consistent with the original degraded image. This improves the visual quality of the image while strictly maintaining the authenticity of the original colors, effectively avoiding the color distortion problems commonly encountered during the enhancement process.

[0138] This invention provides an adaptive image enhancement method for multiple degradation types. Through an integrated degradation perception module, it can simultaneously and accurately quantify the various degradation types and their degrees, such as noise, blur, low illumination, and compression distortion, and dynamically adjust the processing strategy accordingly. This achieves efficient adaptation of a single model to scenarios with multiple degradation types coexisting, solving the problems of inefficiency and conflicting results caused by the need to string together multiple dedicated algorithms in traditional solutions. Furthermore, it employs a detail and noise co-optimization mechanism, using dynamically generated double masks to accurately distinguish and process the detailed structures and noise interference in the image. This effectively suppresses noise while significantly improving the preservation and clarity of key details such as texture and edges. In addition, a correction technique based on the color statistics of the original degraded image ensures the realism and consistency of the colors in the final enhanced image, avoiding common color distortion problems.

[0139] Based on the same inventive concept, embodiments of the present invention also provide a multi-degradation type adaptive image enhancement apparatus for implementing the multi-degradation type adaptive image enhancement method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the multi-degradation type adaptive image enhancement apparatus provided below can be found in the limitations of the multi-degradation type adaptive image enhancement method described above, and will not be repeated here.

[0140] like Figure 2As shown, the present invention provides an adaptive image enhancement device for multiple degradation types, comprising:

[0141] The degradation perception module 21 is used to acquire the degradation image, extract the degradation features in the degradation image, and generate a degradation vector, wherein the degradation vector includes quantized values ​​of noise intensity, blur degree, low illumination degree and compression distortion degree;

[0142] The feature extraction and fusion module 22 is connected to the degradation perception module 21. It is used to dynamically adjust the preset weight coefficients of the multi-scale feature extraction branch based on the degradation vector, and to perform weighted fusion of the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features.

[0143] The optimization module 23, connected to the feature extraction and fusion module 22, is used to generate detail masks and noise masks based on the degradation vector and fusion features, optimize the fusion features, and obtain optimized fusion features.

[0144] The color correction module 24, connected to the optimization module 23, is used to map the optimized fused features into a preliminary enhanced image, and to perform color correction on the preliminary enhanced image based on the color statistics of the degraded image, and output the final enhanced image.

[0145] Optionally, the degradation perception module 21 is specifically used for: converting the degraded image into a grayscale image; dividing the grayscale image into multiple first image blocks of the same size; calculating the pixel variance value of each first image block, and determining the image block with the smallest variance value of a first preset ratio based on the pixel variance values ​​of all first image blocks; calculating the arithmetic mean of the pixel variance values ​​corresponding to the image block with the smallest variance value as the variance estimate of the flat region; and normalizing the variance estimate of the flat region to obtain the quantized value of the noise intensity.

[0146] Optionally, the degradation perception module 21 is specifically used to: convert the degraded image into a grayscale image; perform gradient operation on the grayscale image to obtain a gradient magnitude map; count the number of first pixels in the gradient magnitude map whose magnitude is greater than a preset gradient threshold, and calculate the proportion of the number of first pixels in the total number of pixels in the gradient magnitude map; and calculate the quantization value of the blur level based on the proportion of the number of first pixels in the total number of pixels in the gradient magnitude map.

[0147] Optionally, the degradation perception module 21 is specifically used to: extract the luminance component from the degraded image; calculate the global luminance mean of the luminance component; normalize the global luminance mean to obtain the normalized global luminance mean, and subtract the normalized global luminance mean from the preset constant to obtain the quantized value of the low illumination level.

[0148] Optionally, the degradation perception module 21 is specifically used to: convert the degraded image into a grayscale image; divide the grayscale image into multiple second image blocks of the same size; detect the pixel transition intensity of the boundary of each second image block; count the number of second image blocks whose pixel transition intensity is greater than a preset transition threshold; and calculate a first ratio of the number of second image blocks to the total number of second image blocks as a quantization value of the degree of compression distortion.

[0149] Optionally, the feature extraction and fusion module 22 is specifically used for: setting corresponding preset weight coefficients for each degradation type and each feature scale, wherein the preset weight coefficients include preset noise weight coefficients, preset blur weight coefficients, preset low illumination weight coefficients, and preset compression distortion weight coefficients; calculating the contribution value of each degradation type to each feature scale based on the quantization value of each degradation type in the degradation vector and the corresponding preset weight coefficients; fusing and normalizing the contribution values ​​of each feature scale to generate the final weight coefficients corresponding to each feature scale; and performing weighted fusion of the extracted multi-scale features based on the final weight coefficients to obtain the fused features.

[0150] Optionally, the optimization module 23 is specifically used for: extracting edge features from the fused features to obtain edge features; generating a detail mask based on the edge features and a preset dynamic detail threshold, wherein the detail mask is used to identify the detail feature regions that need to be retained; smoothing the fused features to obtain smooth features; calculating the feature difference between the fused features and the smooth features, and dynamically generating a noise mask based on the feature difference, wherein the noise mask is used to identify the noise feature regions that need to be suppressed; and selectively adjusting the pixel values ​​of corresponding regions in the fused features according to the detail mask and the noise mask to obtain the optimized fused features.

[0151] The process of calculating the feature difference between the fused feature and the smoothed feature, and dynamically generating a noise mask based on the feature difference, may include: calculating the feature difference between the fused feature and the smoothed feature; dynamically determining the noise threshold based on the noise intensity quantization value in the degradation vector; and generating a noise mask based on the feature difference and the noise threshold.

[0152] Optionally, the color correction module 24 is specifically used to: obtain first color statistical information of the degraded image, wherein the optimized fusion features are mapped to a preliminary enhanced image; the first color statistical information includes the first mean and first variance of the degraded image in each color channel; obtain second color statistical information of the preliminary enhanced image, wherein the second color statistical information includes the second mean and second variance of the preliminary enhanced image in each color channel; and perform channel-by-channel color correction on the preliminary enhanced image based on the first color statistical information of the degraded image and the second color statistical information of the preliminary enhanced image to obtain the final enhanced image.

[0153] The multi-degradation type adaptive image enhancement device provided by this invention, through an integrated degradation perception module, can simultaneously and accurately quantify multiple degradation types and their degrees, such as noise, blur, low illumination, and compression distortion, and dynamically adjust the processing strategy accordingly. This achieves efficient adaptation of a single model to scenarios with multiple coexisting degradations, solving the inefficiency and conflicting effects caused by the need for multiple dedicated algorithms in traditional solutions. Furthermore, it employs a detail and noise co-optimization mechanism, using dynamically generated double masks to accurately distinguish and process detailed structures and noise interference in the image, effectively suppressing noise while significantly improving the preservation and clarity of key details such as texture and edges. In addition, a correction technique based on the color statistics of the original degraded image ensures the realism and consistency of the colors in the final enhanced image, avoiding common color distortion problems.

[0154] It should be noted that "multiple" in this invention includes two or more.

[0155] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0156] Each module in the devices of this invention can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0157] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data required for or generated by the aforementioned multi-degradation type adaptive image enhancement method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-degradation type adaptive image enhancement method.

[0158] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a multi-degradation type adaptive image enhancement method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0159] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0164] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0165] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A multi-degradation type adaptive image enhancement method, characterized in that, include: Acquire the degraded image and extract the degradation features from the degraded image to generate a degradation vector, wherein the degradation vector includes quantized values ​​of noise intensity, blur degree, low illumination degree and compression distortion degree; Based on the degenerate vector, the preset weight coefficients of the multi-scale feature extraction branch are dynamically adjusted, and the extracted multi-scale features are weighted and fused according to the adjusted preset weight coefficients to obtain the fused features. Based on the degenerate vector and the fused features, a detail mask and a noise mask are generated, and the fused features are optimized to obtain the optimized fused features. The optimized fused feature map is used to create a preliminary enhanced image, and the color of the preliminary enhanced image is corrected based on the color statistics of the degraded image to output the final enhanced image.

2. The method according to claim 1, characterized in that, The extraction of degradation features from the degraded image includes: Convert the degraded image to a grayscale image; Divide the grayscale image into multiple first image blocks of the same size; Calculate the pixel variance value of each first image block, and based on the pixel variance values ​​of all first image blocks, determine the image block with the smallest variance value of the first preset ratio; Calculate the arithmetic mean of the pixel variance values ​​corresponding to the image block with the smallest variance value, and use it as the variance estimate of the flat region. The variance estimate of the flat region is normalized to obtain the quantized value of the noise intensity.

3. The method according to claim 1, characterized in that, The extraction of degradation features from the degraded image includes: Convert the degraded image to a grayscale image; Gradient calculation is performed on the grayscale image to obtain the gradient magnitude map; Count the number of first pixels whose magnitude is greater than a preset gradient threshold in the gradient magnitude map, and calculate the proportion of the number of first pixels in the total number of pixels in the gradient magnitude map; The quantization value of the blur level is calculated based on the proportion of the first pixel in the total number of pixels in the gradient magnitude map.

4. The method according to claim 1, characterized in that, The extraction of degradation features from the degraded image includes: Extracting the luminance component from a degraded image; Calculate the global brightness mean of the luminance component; The global brightness mean is normalized to obtain the normalized global brightness mean. The normalized global brightness mean is then subtracted from the normalized global brightness mean by a preset constant to obtain the quantized value of the low illumination level.

5. The method according to claim 1, characterized in that, The extraction of degradation features from the degraded image includes: Convert the degraded image to a grayscale image; Divide the grayscale image into multiple second image blocks of the same size; Detect the pixel transition intensity at the boundary of each second image patch; The number of second image blocks whose pixel transition intensity is greater than a preset transition threshold is counted; The first ratio of the number of second image patches to the total number of second image patches is calculated as a quantization value of the degree of compression distortion.

6. The method according to claim 1, characterized in that, The process involves dynamically adjusting the preset weight coefficients of the multi-scale feature extraction branches based on the degenerate vector, and then weighting and fusing the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features, including: Set corresponding preset weight coefficients for each degradation type and each feature scale. The preset weight coefficients include preset noise weight coefficient, preset blur weight coefficient, preset low light weight coefficient and preset compression distortion weight coefficient. Based on the quantization value of each degradation type in the degradation vector and the corresponding preset weight coefficient, calculate the contribution value of each degradation type to each feature scale; The contribution values ​​of each feature scale are fused and normalized to generate the final weight coefficients corresponding to each feature scale. Based on the final weight coefficients, the extracted multi-scale features are weighted and fused to obtain the fused features.

7. The method according to claim 1, characterized in that, The process involves generating detail masks and noise masks based on the degenerate vector and fused features, and optimizing the fused features to obtain optimized fused features, including: Edge features are extracted from the fused features to obtain edge features; Based on edge features and a preset dynamic detail threshold, a detail mask is generated, whereby the detail mask is used to identify the regions of detail features that need to be retained. The fused features are smoothed to obtain smoothed features; The feature difference between the fused feature and the smoothed feature is calculated, and a noise mask is dynamically generated based on the feature difference. The noise mask is used to identify the noise feature regions that need to be suppressed. Based on the detail mask and noise mask, the pixel values ​​of corresponding regions in the fused features are selectively adjusted to obtain the optimized fused features.

8. The method according to claim 7, characterized in that, The calculation of the feature difference between the fused feature and the smoothed feature, and the dynamic generation of a noise mask based on the feature difference, includes: Calculate the feature differences between fused features and smoothed features; The noise threshold is dynamically determined based on the quantized value of the noise intensity in the degradation vector; A noise mask is generated based on feature differences and a noise threshold.

9. The method according to claim 1, characterized in that, The process of mapping the optimized fused features to a preliminary enhanced image, performing color correction on the preliminary enhanced image based on the color statistics of the degraded image, and outputting the final enhanced image includes: The optimized fused features are mapped to a preliminary enhanced image; Obtain first color statistics of the degraded image, wherein the first color statistics include the first mean and first variance of the degraded image in each color channel; Obtain the second color statistics of the preliminary enhanced image, wherein the second color statistics include the second mean and second variance of the preliminary enhanced image in each color channel; Based on the first color statistics of the degraded image and the second color statistics of the preliminary enhanced image, channel-by-channel color correction is performed on the preliminary enhanced image to obtain the final enhanced image.

10. A multi-degradation type adaptive image enhancement device, characterized in that, include: The degradation perception module is used to acquire degraded images, extract degradation features from the degraded images, and generate degradation vectors. The degradation vectors include quantized values ​​of noise intensity, blur level, low illumination level, and compression distortion level. The feature extraction and fusion module, connected to the degradation perception module, is used to dynamically adjust the preset weight coefficients of the multi-scale feature extraction branches based on the degradation vector, and to perform weighted fusion of the extracted multi-scale features according to the adjusted preset weight coefficients to obtain fused features. The optimization module, connected to the feature extraction and fusion module, is used to generate detail masks and noise masks based on the degradation vector and fused features, and to optimize the fused features to obtain optimized fused features. The color correction module, connected to the optimization module, is used to map the optimized fused features into a preliminary enhanced image, and to perform color correction on the preliminary enhanced image based on the color statistics of the degraded image, outputting the final enhanced image.

Images