High-resolution image enhancement method and system based on multi-scale feature fusion

Abstract

This invention discloses a high-resolution image enhancement method and system based on multi-scale feature fusion, belonging to the field of image processing technology. Multi-scale feature extraction generates images at four scales; an improved residual network is used to extract features and upsample them; cross-scale feature fusion is performed by fusing features through skip connections and channel attention mechanisms to compress dimensionality; detail enhancement and noise suppression are processed in blocks and targeted enhancement, filtering, and residual fusion are applied; super-resolution reconstruction involves sub-pixel convolutional amplification, regularization, and cropping; quality assessment and optimization are performed, and parameters are adjusted iteratively to optimize when performance targets are not met, before outputting the final result. This invention enhances high-resolution images effectively, preserving more details, reducing noise, maintaining color consistency, adapting to various scenarios, balancing efficiency and quality, and demonstrating strong practicality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a high-resolution image enhancement method and system based on multi-scale feature fusion. Background Technology

[0002] High-resolution image enhancement technology has significant application value in fields such as remote sensing monitoring, medical imaging, and security surveillance. Its core requirement is to effectively preserve detail and suppress noise while increasing image resolution. Traditional single-scale enhancement methods rely solely on the original image, making it difficult to handle scenes with complex textures and multi-scale features. For example, in remote sensing images where building edges and vegetation textures coexist, issues such as blurred details or over-sharpening can easily occur. While interpolation-based super-resolution algorithms are computationally efficient, they only reconstruct images based on local pixel correlations, lacking utilization of global features, leading to jagged edges or artifacts in the enhancement results.

[0003] Existing deep learning-based methods extract features through convolutional neural networks, improving enhancement results to some extent, but limitations remain. Most networks employ single-scale feature extraction, making it difficult to consider both the global structure and local details of an image. For example, when processing low-light images, they either lose details in dark areas or amplify noise. Cross-scale feature fusion strategies often use simple concatenation or weighted summation, failing to consider the differences in the importance of features at different scales. This leads to redundant information interfering with effective features, especially causing color inconsistencies in multi-light source scenes. Furthermore, existing methods lack adaptability to image content; the same set of parameters is difficult to adapt to different scenes such as night scenes, portraits, and landscapes, resulting in unstable enhancement effects. For instance, edge enhancement in night scene images may be excessive, while smooth areas in portraits may appear grainy.

[0004] With the diversification of image acquisition devices and the expansion of application scenarios, the requirements for image enhancement technologies are constantly increasing. On the one hand, there is a need to process higher resolution inputs (such as 4K and 8K images), posing a challenge to the computational efficiency of algorithms. On the other hand, there is a need to maintain detail clarity and color fidelity in complex environments (such as low light and high noise), which traditional methods can no longer meet. Current technologies do not employ sufficiently refined feature attention mechanisms, paying insufficient attention to key regions (such as edges and textures) and lacking dynamic adjustment mechanisms, resulting in room for improvement in both subjective quality and objective metrics. Therefore, there is an urgent need for a high-resolution image enhancement method that can effectively integrate multi-scale features, adaptively adjust enhancement strategies, and balance efficiency and effectiveness to adapt to diverse application scenarios and complex image content. Summary of the Invention

[0005] The present invention proposes a high-resolution image enhancement method and system based on multi-scale feature fusion to solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a high-resolution image enhancement method based on multi-scale feature fusion, comprising: Multi-scale feature extraction steps: The input low-resolution image is sampled at multiple scales with scaling factors of 1.0, 0.75, 0.5, and 0.25, generating four images at different scales. For example, if the input low-resolution image is 1024×768, four sets of images are generated: 1024×768, 768×576, 512×384, and 256×192. For each scale image, a residual network is used to extract features. The network contains five convolutional blocks, each consisting of two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function. The first convolutional block outputs a passthrough... The number of channels is 64, which extracts low-level edge features. The number of output channels of the 2nd to 4th convolutional blocks are 128, 256 and 512 respectively, which extract texture features such as vegetation texture and skin texture. The number of output channels of the 5th convolutional block is 1024, which extracts global structural features such as building outlines and overall image layout. The feature maps at each scale are upsampled to the size of the low-resolution image (or 1.5 times the original size) using bilinear interpolation to form a multi-scale feature set. The feature set is a set of feature maps corresponding to edge, texture and structural features at each scale. The feature map size is uniform and the number of channels covers 64-1024 levels. Cross-scale feature fusion steps: Features at different scales in the multi-scale feature set are fused to construct a feature pyramid fusion structure. The bottom-level features (1024 channels) and middle-level features (512 channels) are fused by adding them together via skip connections, while the middle-level features and high-level features (256 channels) are fused by weighted fusion via a channel attention mechanism. This channel attention mechanism calculates the weights of different feature channels using a squeeze-excitation module (first, global average pooling is performed on each channel feature to compress spatial information into channel descriptors, then two fully connected layers output weight values ​​in the range of 0-1). High weights (wc values ​​0.6-0.9) are assigned to effective feature channels such as edges and textures, while low weights (wc values ​​0.1-0.3) are assigned to noise-dominated channels. After fusion, the number of channels in the feature map is uniformly 512, and a 1×1 convolution is used to compress the dimension while retaining key feature information. Detail enhancement and noise suppression steps: The fused feature map is divided into 32×32 blocks (step size 16, zero-padding at edges), and the local contrast (based on the standard deviation of gray values) and noise intensity (based on Gaussian filter residuals) of each block are calculated. Different image block features are processed: for high-contrast regions (standard deviation > 20), the Laplacian operator is used to enhance edge details; for high-noise regions (residual > 15), a 3×3 to 7×7 dynamically switching filter kernel is used for adaptive noise reduction. The processed blocks are then stitched together and fused with the original input image via skip connections, preserving more than 85% of the structural information of the original image. Super-resolution reconstruction steps: The feature map after detail enhancement and noise suppression is upscaled to the target high resolution using subpixel convolutional layers (supporting 2×, 4×, and 8× magnification; for example, the target resolution is 4 times that of the input low-resolution image when magnified by 4×). The subpixel convolutional kernel size is 3×3, and the number of channels before pixel rearrangement is the product of the square of the target magnification factor and the RGB channels (3 channels). An edge-preserving regularization term is added during reconstruction (achieved by calculating the L1 norm of the image gradient). For each pixel (i,j) in the reconstructed image, the gradient difference between it and its neighboring pixels (3×3 neighborhood) is constrained to not exceed 1.2 times the gradient difference of the corresponding region in the original low-resolution image. For textured smooth regions (gradient variance < 5), the constraint is relaxed to 1.5 times to reduce over-sharpening. After reconstruction, bilateral filtering (spatial standard deviation 2.0, grayscale standard deviation 50) is used for post-processing. The high-resolution image is cropped with grayscale values ​​of 0-255 before output to avoid pixel value overflow. Quality assessment and parameter optimization steps: The enhancement results are scored using no-reference image quality assessment metrics (including information entropy, gradient magnitude, and structural similarity); when information entropy < 7.5 or structural similarity < 0.9, the multi-scale sampling factor (increase or decrease by 0.1) and attention weight coefficient (increase or decrease by 0.05) are adjusted in feedback; the evaluation metrics are optimized through iterative optimization (up to 5 times) to achieve the target, and the final high-resolution image is output; the maximum number of iterations is set to 2 for images with clear textures, and a maximum of 5 iterations is retained for images with blurred textures to balance the enhancement effect and computational efficiency.

[0007] Furthermore, it also includes: Dynamic weight adjustment and color consistency correction steps: Initial weight values ​​are preset for images in different scenes. The weight for enhancing edges in night scene images is increased by 20%, and the weight for smoothing areas in portrait images is increased by 15%. The weights are adjusted in real time by calculating color saturation and texture complexity through image content analysis. Areas with high texture complexity have increased detail enhancement weights, and areas with low color saturation have increased color enhancement weights. In the color consistency correction step, the color deviation (mean difference, variance ratio) of feature maps at each scale is calculated. Feature maps with deviations exceeding the threshold (mean difference > 10 or variance ratio > 1.5) are subjected to gamma correction, and the correction coefficient is dynamically adjusted according to the degree of deviation (range 0.5-2.0). After fusion, the image is adjusted by white balance to eliminate color cast caused by cross-scale fusion and ensure that the mean deviation of the R, G, and B channels is within ±5.

[0008] Feature attention optimization and adaptive iteration termination steps: Before cross-scale fusion, a spatial attention mask is generated for each feature map. The mask value is normalized to 0-1 using the Sigmoid function to highlight important regional features. The mask value for low-light regions (average gray value < 50) is boosted (multiplied by a coefficient of 1.5) to enhance the ability to extract details in dark areas. In the adaptive iteration termination step, the difference in image quality evaluation index between two consecutive iterations is calculated. When the difference is < 0.02 and continues for two iterations, the optimization process is terminated early. For images with simple textures (such as cartoons), the maximum number of iterations is set to 2, while for images with complex textures (such as landscapes), a maximum of 5 iterations is retained to balance the enhancement effect and computational efficiency.

[0009] Furthermore, in the cross-scale feature fusion step, the weight calculation formula for the channel attention mechanism is: wc=σ(W2・δ(W1・Fc)), where wc is the attention weight of the c-th channel, σ is the Sigmoid activation function, δ is the ReLU activation function, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and Fc is the global average pooling result of the c-th channel. In the calculation process, firstly, global average pooling is performed on each channel feature to compress the spatial information into channel descriptors; the number of channels is reduced from C to C / r through W1, and after ReLU activation, it is restored to the original number of channels C through W2; finally, the output is mapped to the 0-1 interval using the Sigmoid function to achieve adaptive allocation of channel weights; for channels with obvious edge features, the wc value is between 0.6 and 0.9, and the wc value of channels with dominant noise is suppressed to 0.1-0.3, making the fusion process more focused on effective features.

[0010] Furthermore, in the color consistency correction step, the gamma correction coefficient is calculated as follows: γ = k1 × (μt / μs) + k2 × (σt / σs), where γ is the gamma correction coefficient, k1 and k2 are weighting coefficients, μt is the target color mean, μs is the color mean of the current feature map, σt is the target color variance, σs is the color variance of the current feature map, and the average variance of the multi-scale feature maps is taken. During calculation, μs and σs are first calculated for the R, G, and B channels of each scale feature map, and then the average value of all scales is taken to obtain μt and σt. If the current feature map is too dark, μt / μs > 1.25, and the γ value is increased accordingly to improve brightness. When the feature map is too bright, μt / μs < 0.83, and the γ value is decreased to reduce brightness. For variance deviation, it is corrected by the σt / σs term.

[0011] Furthermore, in the feature attention optimization step, the spatial attention mask generation formula is: Ms(i,j)=(1-α)×G(i,j)+α×L(i,j), where Ms(i,j) is the spatial attention mask value at position (i,j) in the image, α is the fusion coefficient, G(i,j) is the gradient magnitude at position (i,j), and L(i,j) is the illumination intensity at position (i,j). During calculation, G(i,j) is extracted using the Sobel operator, and L(i,j) is the average gray value of the RGB three channels. For edge regions, regardless of the size of L(i,j), Ms(i,j) is maintained above 0.7, ensuring that edge features are preferentially preserved. For low-light regions, even if G(i,j) is low, α is adjusted to increase Ms(i,j) to above 0.5, enhancing the attention to details in dark areas.

[0012] Furthermore, in the super-resolution reconstruction step, the edge preservation regularization term is implemented by calculating the L1 norm of the image gradient. For each pixel (i,j) of the reconstructed image, the regularization term constrains the gradient difference between it and its neighboring pixels to not exceed 1.2 times the gradient difference of the corresponding region in the original low-resolution image. For textured smooth regions, the constraint is relaxed to 1.5 times to reduce over-sharpening. After reconstruction, bilateral filtering is used for post-processing to suppress noise while preserving the edges.

[0013] A system for implementing the aforementioned high-resolution image enhancement method based on multi-scale feature fusion includes: Feature extraction module: includes image scaling unit, feature extraction network, and upsampling unit; the feature extraction network outputs feature maps at four scales; Cross-scale feature fusion module: consists of skip connection unit, channel attention unit, and feature compression unit; skip connection unit supports the addition of features with different number of channels.

[0014] Furthermore, the detail enhancement and noise suppression module includes an image segmentation unit, a region analysis unit, a detail enhancement unit, and an adaptive filtering unit; The super-resolution reconstruction module includes subpixel convolution units, regularization units, and grayscale cropping units; the subpixel convolution units have a reconstruction speed of ≥15fps, the regularization units reduce edge blur by 20%-30%, and the grayscale cropping units prevent pixel value overflow.

[0015] Furthermore, the quality assessment and parameter optimization module includes an index calculation unit, a parameter adjustment unit, and an iteration control unit; the parameter adjustment unit improves the pass rate of the assessment index, and the iteration control unit reduces the number of iterations for simple texture images, achieving a balance between effect and efficiency; The dynamic weight adjustment module includes a scene recognition unit to distinguish between night scenes, portraits, and landscapes, and a weight correction unit to adjust the weights based on texture complexity and color saturation. The color consistency correction module includes a color deviation analysis unit to calculate the mean difference and variance ratio, a gamma correction unit to adjust the coefficients, and a white balance unit to eliminate cross-scale fusion color shift.

[0016] Compared with existing technologies, the beneficial effects of this invention are: Multi-scale feature extraction captures both the global structure and local details of an image simultaneously, avoiding the information loss problem of single-scale methods in complex scenes. This ensures that the enhancement results maintain overall consistency while presenting rich details. Cross-scale feature fusion employs a channel attention mechanism, which adaptively allocates weights to different channels, highlighting key features and suppressing redundant information. This effectively solves the feature interference problem in traditional fusion methods, and performs particularly well when processing images with multiple textures.

[0017] This method enhances adaptability to different scenes through dynamic weight adjustment and color consistency correction. Dark details in night scene images are clearly presented, skin tones in portrait images are more natural, and color gradations in landscape images are richer, avoiding the instability caused by single-parameter processing. Feature attention optimization further improves attention to edges and low-light areas, maintaining image balance and reducing overexposure or underexposure even under complex lighting conditions such as backlighting. Edge regularization during super-resolution reconstruction effectively suppresses noise amplification and artifact generation, improving both the subjective visual effect and objective quality indicators of the enhanced image.

[0018] The collaborative work of each module in the system improves processing efficiency. The multi-scale feature extraction and fusion strategy, while ensuring high-quality results, reduces computational load through reasonable channel compression and iterative control, making it suitable for real-time processing of high-resolution images. The closed-loop mechanism for quality assessment and parameter optimization ensures the stability and reliability of the enhanced results, reducing reliance on manual adjustments. Overall, the method and system demonstrate excellent performance in detail preservation, noise suppression, and color consistency, exhibiting strong adaptability and applicability to various scenarios, thus possessing significant practical value. Attached Figure Description

[0019] Figure 1 This is a schematic block diagram of the high-resolution image enhancement system based on multi-scale feature fusion proposed in this invention; Figure 2 This is a schematic block diagram of the high-resolution image enhancement method based on multi-scale feature fusion proposed in this invention; Figure 3 This is a schematic diagram comparing the detail retention of different methods for high-resolution image enhancement based on multi-scale feature fusion proposed in this invention; Figure 4 This is a schematic diagram of the channel attention weight distribution of the high-resolution image enhancement method based on multi-scale feature fusion proposed in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0022] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The invention will now be described in further detail with reference to the accompanying drawings.

[0023] Reference Figures 1 to 4 A high-resolution image enhancement method based on multi-scale feature fusion includes the following steps: Multi-scale feature extraction steps: The input low-resolution image is sampled at multiple scales to generate four images at different scales (scaling factors of 1.0, 0.75, 0.5, and 0.25, respectively). For each scale image, an improved residual network is used to extract features. This network contains five convolutional blocks (each convolutional block consists of two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function). The number of output channels for the first to fourth convolutional blocks are 64, 128, 256, and 512, respectively, and the number of output channels for the fifth convolutional block is 1024. The feature maps at each scale are upsampled (using bilinear interpolation) to the original image size to form a multi-scale feature set (the feature map size is 1.5 times that of the input image).

[0024] Cross-scale feature fusion steps: Construct a feature pyramid fusion structure. The bottom layer features (1024 channels) and the middle layer features (512 channels) are added through skip connections. The middle layer features and the high layer features (256 channels) are weighted and fused through a channel attention mechanism. The attention weights are calculated through the squeeze-excitation module (first global average pooling, then output weight values ​​through 2 fully connected layers), with a weight range of 0-1. After fusion, the number of channels in the feature map is unified to 512, and 1×1 convolution is used to compress the dimension and retain key feature information.

[0025] Detail enhancement and noise suppression steps: The fused feature map is processed in blocks (block size 32×32, stride 16), and the local contrast (based on the standard deviation of gray values) and noise intensity (based on Gaussian filter residual) of each block are calculated; edge details are enhanced in high-contrast regions (standard deviation > 20) using the Laplacian operator, and adaptive filtering is performed on high-noise regions (residual > 15) (the filter kernel size is dynamically adjusted from 3×3 to 7×7 according to the noise intensity); the processed blocks are stitched together and fused with the residual of the original image through skip connections to preserve the structural information of the original image.

[0026] Super-resolution reconstruction steps: Subpixel convolutional layers are used to upscale the fused feature map to the target resolution (supporting 2×, 4×, and 8× magnification). The subpixel convolutional kernel size is 3×3, and the number of channels before pixel rearrangement is the product of the square of the target magnification factor and 3 (RGB channels). An edge preservation regularization term is added during the reconstruction process to suppress high-frequency noise amplification and enhance image sharpness. The high-resolution image is cropped with gray values ​​of 0-255 before output to ensure that the pixel values ​​are within the effective range.

[0027] Quality assessment and parameter optimization steps: The enhancement results are scored using no-reference image quality assessment metrics (including information entropy, gradient magnitude, and structural similarity); when information entropy < 7.5 or structural similarity < 0.9, the multi-scale sampling factor (increase or decrease by 0.1) and attention weight coefficient (increase or decrease by 0.05) are adjusted in feedback; the evaluation metrics are optimized through iterative optimization (up to 5 times) to achieve the target, and the final enhanced image is output.

[0028] This invention also includes: The dynamic weight adjustment and color consistency correction steps are as follows: In the dynamic weight adjustment step, initial weight values ​​are preset for different scene images (such as night scenes, portraits, and landscapes). The weight for enhancing edges in night scene images is increased by 20%, and the weight for smooth areas in portrait images is increased by 15%. The weights are corrected in real time through image content analysis (calculating color saturation and texture complexity). Regions with high texture complexity (gradient mean > 30) receive increased detail enhancement weights, while regions with low color saturation (saturation mean < 0.2) receive increased color enhancement weights. In the color consistency correction step, the color deviation (mean difference, variance ratio) of feature maps at each scale is calculated. Feature maps with deviations exceeding the threshold (mean difference > 10 or variance ratio > 1.5) undergo gamma correction, and the correction coefficient is dynamically adjusted according to the degree of deviation (range 0.5-2.0). After fusion, the image is adjusted for white balance (based on the gray-scale world assumption) to eliminate color cast caused by cross-scale fusion, ensuring that the mean deviation of the R, G, and B channels is within ±5.

[0029] The feature attention optimization step and the adaptive iteration termination step are as follows: In the feature attention optimization step, a spatial attention mask (with the same size as the feature map) is generated for each feature map before cross-scale fusion. The mask value is normalized to 0-1 using the Sigmoid function to highlight important regional features. The mask value of low-light regions (average gray value < 50) is boosted (multiplied by a coefficient of 1.5) to enhance the ability to extract details in dark areas. In the adaptive iteration termination step, the difference in image quality evaluation index between two consecutive iterations is calculated. When the difference is < 0.02 and continues for 2 iterations, the optimization process is terminated early. For images with simple textures (such as cartoons), the maximum number of iterations is set to 2, while for images with complex textures (such as landscapes), a maximum of 5 iterations is retained to balance the enhancement effect and computational efficiency.

[0030] In this invention, the weight calculation formula for the channel attention mechanism in the cross-scale feature fusion step is: wc=σ(W2・δ(W1・Fc)), where wc is the attention weight (0-1) of the c-th channel, σ is the Sigmoid activation function, δ is the ReLU activation function, W1 is the weight matrix of the first fully connected layer (dimension C / r×C, r is the dimensionality reduction coefficient, with a value of 16), W2 is the weight matrix of the second fully connected layer (dimension C×C / r), and Fc is the global average pooling result of the c-th channel (dimension 1×1×1). During the calculation process, firstly, global average pooling is performed on each channel feature to compress spatial information into channel descriptors; the number of channels is reduced from C to C / r using W1 (reducing computational load), and after ReLU activation, it is restored to the original number of channels C using W2; finally, the Sigmoid function is used to map the output to the 0-1 interval, achieving adaptive allocation of channel weights. For channels with obvious edge features (such as channels containing high-frequency information), the wc value is usually between 0.6 and 0.9, while for channels with noise dominance (such as high-frequency noise channels in smooth regions), the wc value is suppressed to 0.1-0.3, making the fusion process more focused on effective features and reducing redundant information interference. This can significantly improve the edge preservation effect, especially in low-light image enhancement.

[0031] In this invention, the formula for calculating the gamma correction coefficient in the color consistency correction step is: γ = k1 × (μt / μs) + k2 × (σt / σs), where γ is the gamma correction coefficient (0.5-2.0), k1 and k2 are weighting coefficients (k1=0.6, k2=0.4), μt is the target color mean (the average mean of the multi-scale feature maps), μs is the color mean of the current feature map, σt is the target color variance (the average variance of the multi-scale feature maps), and σs is the color variance of the current feature map. During calculation, μs and σs are first calculated for the R, G, and B channels of the feature map at each scale, and then the average value of all scales is taken to obtain μt and σt. When the current feature map is dark (μs < μt × 0.8), μt / μs > 1.25, and the γ value is increased accordingly (e.g., 1.3-2.0) to improve brightness. When the feature map is bright (μs > μt × 1.2), μt / μs < 0.83, and the γ value is decreased (e.g., 0.5-0.8) to reduce brightness. For variance bias (σs and σt deviation > 30%), further correction is made through the σt / σs term to ensure consistent color dynamic range. After correction, the mean deviation of the three channels of the feature map at each scale is controlled within ±3, and the variance bias is < 15%, effectively avoiding color cast problems after cross-scale fusion, which is especially suitable for image enhancement in multi-light source scenes.

[0032] In this invention, the spatial attention mask generation formula in the feature attention optimization step is: Ms(i,j)=(1-α)×G(i,j)+α×L(i,j), where Ms(i,j) is the spatial attention mask value (0-1) at position (i,j) of the image, α is the fusion coefficient (0.3-0.7, α=0.7 for complex texture regions), G(i,j) is the gradient magnitude at position (i,j) (normalized to 0-1), and L(i,j) is the illumination intensity at position (i,j) (normalized to 0-1). During calculation, G(i,j) is extracted using the Sobel operator (the square root of the sum of squares of the horizontal and vertical gradients), and L(i,j) is the average gray value of the RGB three channels (normalized to 0-1). For edge regions (G(i,j) > 0.6), Ms(i,j) remains above 0.7 regardless of the size of L(i,j), ensuring that edge features are preferentially preserved. For low-light regions (L(i,j) < 0.3), even if G(i,j) is low (e.g., 0.2-0.4), Ms(i,j) is increased to above 0.5 through α adjustment (α=0.5 in low-light regions), enhancing the focus on details in dark areas. After the mask is generated, it is multiplied pixel by pixel with the feature map, increasing the feature weight of important regions by 20%-50% and decreasing the feature weight of secondary regions by 10%-30%. This effectively balances the problem of overexposure in bright areas and underexposure in dark areas in backlit image enhancement.

[0033] In this invention, the edge preservation regularization term in the super-resolution reconstruction step is implemented by calculating the L1 norm of the image gradient. For each pixel (i,j) of the reconstructed image, the regularization term constrains the gradient difference between it and its neighboring pixels (3×3 neighborhood) to not exceed 1.2 times the gradient difference of the corresponding region in the original low-resolution image. For textured smooth regions (gradient variance < 5), the constraint is relaxed to 1.5 times to reduce over-sharpening. After reconstruction, bilateral filtering (spatial standard deviation 2.0, grayscale standard deviation 50) is used for post-processing to further suppress noise while preserving the edges, thereby improving the peak signal-to-noise ratio (PSNR) of the enhanced image by 1.5-2.0 dB.

[0034] This invention also discloses a system for a high-resolution image enhancement method based on multi-scale feature fusion, comprising: Multi-scale feature extraction module: includes an image scaling unit (supporting scaling of 1.0, 0.75, 0.5, and 0.25 times, using bicubic interpolation), a feature extraction network (5 convolutional blocks, including 3×3 convolutional layers, batch normalization layers, and ReLU activation function), and an upsampling unit (bilinear interpolation to the original size); the image scaling unit has a processing latency of <50ms, the feature extraction network outputs feature maps at 4 scales (64-1024 channels), and the upsampling unit has an error rate of <1%.

[0035] The cross-scale feature fusion module consists of a skip connection unit (which adds features from the bottom and middle layers), a channel attention unit (a squeezing-excitation module with a dimensionality reduction factor of 16), and a feature compression unit (1×1 convolution, compressing the number of channels to 512). The skip connection unit supports the addition of features with different numbers of channels (automatically filling in the dimensions), the channel attention unit takes less than 10ms to calculate the weights, and the feature compression unit retains more than 90% of the key feature information.

[0036] In this invention, the detail enhancement and noise suppression module includes an image segmentation unit (32×32 blocks, step size 16, supporting edge zero-padding), a region analysis unit (calculating local contrast and noise intensity, accuracy ±2), a detail enhancement unit (Laplacian operator, edge gain 1.2-1.5), and an adaptive filtering unit (3×3 to 7×7 variable kernel, switching according to noise intensity); the image segmentation unit has a processing speed ≥30fps (1080P image), the region analysis unit has an accuracy of ≥95% in recognizing high-contrast regions, and the adaptive filtering unit has a noise suppression rate >30% and an edge preservation rate >85%.

[0037] Super-resolution reconstruction module: includes subpixel convolutional units (3×3 convolutional kernels, supporting 2×, 4×, 8× magnification), regularization units (edge ​​preservation constraints, gradient difference threshold of 1.2x), and grayscale cropping units (0-255 range limit); the subpixel convolutional units have a reconstruction speed of ≥15fps (4K output), the regularization units reduce edge blur by 20%-30%, and the grayscale cropping units prevent pixel value overflow (error rate of 0).

[0038] In this invention, the quality assessment and parameter optimization module includes an index calculation unit (calculating information entropy, gradient magnitude, and structural similarity with an accuracy of ±0.01), a parameter adjustment unit (correcting sampling factors and attention weights with step sizes of 0.1 and 0.05), and an iteration control unit (maximum 5 iterations, supporting early termination). The index calculation unit takes less than 20ms to process, the parameter adjustment unit improves the evaluation index compliance rate to over 98%, and the iteration control unit reduces the number of iterations for simple texture images by 60%, balancing performance and efficiency.

[0039] The dynamic weight adjustment module (corresponding to claim 2) includes a scene recognition unit (distinguishing between night scenes, portraits, and landscapes with an accuracy of ≥92%) and a weight correction unit (adjusting weights based on texture complexity and color saturation); the scene recognition unit has a response time of <30ms, and the weight correction unit improves scene adaptability by 25%-35%.

[0040] Color consistency correction module (corresponding to claim 2): includes a color deviation analysis unit (calculates the mean difference and variance ratio), a gamma correction unit (adjusts the coefficient according to the formula of claim 5), and a white balance unit (assuming a gray world, color deviation correction rate > 80%); the color deviation analysis unit has an accuracy of ±1, and the gamma correction unit reduces cross-scale color deviation by 40%-50%.

[0041] Specific implementation of a high-resolution image enhancement method and system based on multi-scale feature fusion: Example 1: High-resolution enhancement processing of remote sensing images This embodiment targets a remote sensing image with a resolution of 1024×768 (containing various land features such as buildings, vegetation, and roads). It employs the high-resolution image enhancement method and system based on multi-scale feature fusion described in this patent, aiming to output a 4× magnified 4096×3072 resolution image. The specific steps are as follows: Multi-scale feature extraction: The input image is sampled at multiple scales to generate four images with scaling factors of 1.0 (1024×768), 0.75 (768×576), 0.5 (512×384), and 0.25 (256×192). An improved residual network is used to extract features. The first convolutional block (64 channels) processes low-level edges, the second to fourth convolutional blocks (128, 256, and 512 channels) extract texture features, and the fifth convolutional block (1024 channels) outputs global structural features. The feature maps at each scale are upsampled to 1536×1152 (1.5 times the original size) through bilinear interpolation to form a multi-scale feature set.

[0042] Cross-scale feature fusion: A feature pyramid is constructed. The bottom layer (1024 channels) and the middle layer (512 channels) features are added through skip connections (automatically filling in dimensions). The middle and top layers (256 channels) features are fused using a channel attention mechanism. The channel attention weight calculation formula is as follows: wc=σ(W2・δ(W1・Fc)) Where C=512 (number of mid-layer feature channels), r=16 (dimensionality reduction coefficient), W1 dimension is 32×512, W2 dimension is 512×32, and Fc is the global average pooling result of the c-th channel. For the building edge channel (c=120-150), wc=0.7-0.85 was calculated; for the vegetation texture channel (c=200-230), wc=0.6-0.75; and for the noise channel (c=400-420), wc=0.1-0.25. After fusion, the channel is compressed to 512 channels using a 1×1 convolution, retaining 92% of the key features.

[0043] Detail enhancement and noise suppression: The fused feature map is divided into 32×32 blocks (step size 16, zero-padding at edges). Local contrast (standard deviation 25-35 for building areas, 15-20 for vegetation areas) and noise intensity (residual 5-8 for road areas, 12-18 for shadow areas) are calculated. High-contrast building edges (standard deviation > 30) are enhanced using the Laplacian operator (gain 1.4), high-noise shadow areas (residual > 15) are filtered using a 7×7 kernel, and low-noise road areas are filtered using a 3×3 kernel. After block stitching, the residuals are fused with the original image, preserving 85% of the original structure.

[0044] Super-resolution reconstruction: 3×3 subpixel convolutional layers, number of channels before pixel rearrangement = 4 2 ×3=48, upscale to 4096×3072. Edge preservation regularization constraint: gradient difference of 3×3 neighborhood ≤ 1.2 times (building edges) and 1.5 times (vegetation areas) of the original image. After reconstruction, bilateral filtering (spatial standard deviation 2.0, grayscale standard deviation 50) is applied, and the image is cropped from 0 to 255.

[0045] Quality assessment and optimization: Initial assessment showed information entropy of 7.2 (<7.5) and structural similarity of 0.88 (<0.9). The sampling factor was adjusted (0.75→0.85) and the attention weight coefficient was increased by 0.05. After 3 iterations, the information entropy reached 7.6 and the structural similarity reached 0.92, meeting the output target.

[0046] Example 2: Low-light portrait image enhancement This example processes a low-light portrait with a resolution of 1920×1080 (uneven lighting, blurred details in dark areas), and enlarges the target by 2× to 3840×2160. The steps are as follows: Multi-scale feature extraction: sampling factors of 1.0, 0.75, 0.5, and 0.25, residual network extracts features, the 5th convolutional block outputs 1024 channels of features, upsampled to 2880×1620.

[0047] Cross-scale fusion and color correction: In the channel attention weight calculation, skin tone channel (c=50-70) wc=0.7-0.8, hair texture channel (c=100-120) wc=0.6-0.7, and background channel (c=300-320) wc=0.3-0.4. Color consistency correction uses the gamma coefficient calculation formula: γ=k1×(μt / μs)+k2×(σt / σs) Where μt=55 (multi-scale average mean), σt=30 (multi-scale average variance), for the dark areas of the face μs=30, σs=15, we get γ=0.6×(55 / 30)+0.4×(30 / 15)=1.1+0.8=1.9 (enhancing brightness); for the bright areas μs=80, σs=40, γ=0.6×(55 / 80)+0.4×(30 / 40)=0.41+0.3=0.71 (reducing brightness). The mean deviation of the three channels after correction is ±2.

[0048] Feature attention optimization: Spatial attention mask generative formula: Ms(i,j)=(1-α)×G(i,j)+α×L(i,j) For the facial region, α=0.5, G(i,j)=0.3-0.5 (low gradient), L(i,j)=0.2-0.3 (dark area), Ms(i,j)=0.5×0.4+0.5×0.25=0.325, which is increased to 0.488 after multiplying by 1.5, thus enhancing the attention to the dark area.

[0049] Super-resolution reconstruction and iterative optimization: 2× magnification, sub-pixel convolution channel count = 2 2 ×3=12, regularization constraint: facial gradient difference ≤ 1.2 times. After 2 iterations, information entropy is 7.7, structural similarity is 0.93, output.

[0050] The performance comparison data is shown in the table below: index Traditional interpolation method Single-scale CNN method Example 1 of this application Example 2 of this application Detail retention Low middle high high Noise suppression effect Difference middle excellent excellent Color consistency middle Difference excellent excellent Subjective visual quality Vague Local over-sharpening Clear and natural Soft and delicate Processing time (s) 1.2 4.5 5.8 6.2 The table data shows that the method in this application significantly outperforms traditional methods in terms of detail preservation and noise suppression, especially in terms of color consistency, solving the color cast problem of single-scale CNN methods. In Example 1, the building edges and vegetation textures in the remote sensing image are clearly distinguishable, and in Example 2, the dark details of the portrait in low light (such as eyelashes and hair strands) are presented and the skin tone is natural. Although the processing time is slightly longer than that of traditional methods, the multi-scale fusion and attention mechanism achieve a qualitative improvement in the effect, and the subjective visual quality is more in line with human visual perception. The increased processing time is due to multi-scale feature processing, but efficiency is balanced by iterative control (only 2 iterations in Example 2), making it suitable for scenarios with high quality requirements.

[0051] Reference Figure 3The figure visually demonstrates the advantages of the method of this invention in terms of detail preservation. Data shows that the detail preservation rates of Embodiments 1 and 2 of this patent (90%-92%) are significantly higher than those of traditional interpolation methods (65%) and single-scale CNN methods (78%), approaching the ideal value. This is due to the comprehensive capture of details at different levels by multi-scale feature extraction, and the high weight allocation of edge channels during cross-scale fusion (e.g., the weight of building edge channels in remote sensing images is 0.7-0.85). In low-light portrait processing, spatial attention masks enhance the attention to dark areas, allowing subtle features such as eyelashes and hair strands to be preserved, solving the problem of blurred details in traditional methods and verifying the effective ability of the multi-scale fusion strategy to preserve detail information.

[0052] Reference Figure 4 The line graph illustrates the differentiated weighting effect of the channel attention mechanism on different types of channels. Key feature channels (such as edges and textures) have weights concentrated between 0.6 and 0.8, significantly higher than noise and background channels (0.15-0.35). This high weight allocation focuses the fusion process on effective information; for example, the weight of the building edge channel in remote sensing images is 0.8, enhancing contour clarity. Low weights suppress noise interference, resulting in cleaner smooth areas in the image. This adaptive weighting mechanism solves the problem of feature redundancy in traditional fusion, improving the effectiveness of the fused features.

[0053] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A high-resolution image enhancement method based on multi-scale feature fusion, characterized in that, include: Multi-scale feature extraction step: Sample the input low-resolution image to generate multiple images at different scales; A deep residual network is used to extract features from images at each scale, and the extracted features are upsampled to the size of the low-resolution image to form a multi-scale feature set. Cross-scale feature fusion step: Fusing features of different scales in the multi-scale feature set, wherein low-level features and mid-level features are fused through skip connections, and mid-level features and high-level features are fused through a channel attention mechanism with weights; the channel attention mechanism is used to calculate the weights of different feature channels; Detail enhancement and noise suppression steps: The fused feature map is divided into blocks, and targeted enhancement or filtering is performed on the features of different image blocks. The processed image blocks are then fused with the original input image using residual fusion. Super-resolution reconstruction steps: Upsample the feature map after detail enhancement and noise suppression to the target high resolution, and add regularization constraints during the reconstruction process to output the final high-resolution image; Quality assessment and parameter optimization steps: Upsample the feature map after detail enhancement and noise suppression to the target high resolution, and add regularization constraints during the reconstruction process to output the final high-resolution image.

2. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 1, characterized in that, Also includes: Dynamic weight adjustment steps and color consistency correction steps: preset initial weight values ​​for images in different scenes; The weights are adjusted in real time through image content analysis. Areas with high texture complexity are given more detail enhancement weights, while areas with low color saturation are given more color enhancement weights. In the color consistency correction step, the color deviation of feature maps at each scale is calculated, and gamma correction is performed on feature maps with deviations exceeding the threshold. The correction coefficient is dynamically adjusted according to the degree of deviation.

3. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 1, characterized in that, Also includes: Feature attention optimization step and adaptive iteration termination step: Before cross-scale fusion, a spatial attention mask is generated for each feature map. The mask value is normalized to 0-1 by the Sigmoid function to highlight the features of important regions. The mask value of low-light regions is boosted to enhance the ability to extract details in dark areas. In the adaptive iteration termination step, the difference in image quality evaluation index between two consecutive iterations is calculated. When the difference is less than a preset difference threshold and continues for a preset number of iterations, the optimization process is terminated in advance. The maximum number of iterations for adaptation is set according to the image texture complexity to balance the enhancement effect and computational efficiency.

4. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 1, characterized in that, In the cross-scale feature fusion step, the weight calculation formula for the channel attention mechanism is: wc=σ(W2・δ(W1・Fc)), where wc is the attention weight of the c-th channel, σ is the Sigmoid activation function, δ is the ReLU activation function, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, and Fc is the global average pooling result of the c-th channel. In the calculation process, firstly, global average pooling is performed on each channel feature to compress the spatial information into channel descriptors; then, the number of channels is reduced from C to C / r by W1, activated by ReLU, and then restored to the original number of channels C by W2; finally, the Sigmoid function is used to map the output to the 0-1 interval. High weights are assigned to effective feature channels, and low weights are assigned to noise-dominated channels.

5. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 2, characterized in that, In the color consistency correction step, the gamma correction coefficient is calculated as follows: γ = k1 × (μt / μs) + k2 × (σt / σs), where γ is the gamma correction coefficient, k1 and k2 are weighting coefficients, μt is the target color mean, μs is the color mean of the current feature map, σt is the target color variance, and σs is the color variance of the current feature map. During the calculation, μs and σs are first calculated for the R, G, and B channels of the feature map at each scale, and then the average value of all scales is taken to obtain μt and σt. The γ value is dynamically adjusted to correct the color based on the brightness deviation and variance deviation between the current feature map and the target color.

6. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 3, characterized in that, In the feature attention optimization step, the spatial attention mask generation formula is: Ms(i,j)=(1-α)×G(i,j)+α×L(i,j), where Ms(i,j) is the spatial attention mask value at position (i,j) of the image, α is the fusion coefficient, G(i,j) is the gradient magnitude at position (i,j), and L(i,j) is the illumination intensity at position (i,j). During calculation, G(i,j) is extracted using a preset operator, and the average grayscale value of the RGB three channels is taken as L(i,j); Maintain an adaptive range of mask values ​​for edge-preserving regions, and increase mask values ​​for low-light regions through α adjustment.

7. The high-resolution image enhancement method based on multi-scale feature fusion according to claim 1, characterized in that, In the super-resolution reconstruction step, the edge preservation regularization term is implemented by calculating the L1 norm of the image gradient. For each pixel (i,j) of the reconstructed image, the regularization term constrains the gradient difference between it and its neighboring pixels to not exceed a preset threshold of the gradient difference of the corresponding region in the original low-resolution image. For textured smooth regions, the constraint threshold is relaxed to reduce over-sharpening. After reconstruction, bilateral filtering is used for post-processing to suppress noise while preserving the edges.

8. A system for implementing the high-resolution image enhancement method based on multi-scale feature fusion as described in any one of claims 1-7, characterized in that, include: Feature extraction module: includes image scaling unit, feature extraction network, and upsampling unit; the feature extraction network outputs feature maps at multiple scales; Cross-scale feature fusion module: consists of skip connection unit, channel attention unit, and feature compression unit; skip connection unit supports the addition of features with different number of channels, channel attention unit calculates weights through preset attention mechanism, and feature compression unit compresses feature dimension through 1×1 convolution.

9. The high-resolution image enhancement system based on multi-scale feature fusion according to claim 8, characterized in that, Also includes: The detail enhancement and noise suppression module includes an image segmentation unit, a region analysis unit, a detail enhancement unit, and an adaptive filtering unit. The region analysis unit calculates the local contrast and noise intensity of the fused feature map segments, the detail enhancement unit enhances the edges of high-contrast regions, and the adaptive filtering unit reduces noise in high-noise regions. The super-resolution reconstruction module includes a sub-pixel convolution unit, a regularization unit, and a grayscale cropping unit. The reconstruction speed of the sub-pixel convolution unit meets the preset real-time requirements, the regularization unit reduces edge blurring through edge preservation constraints, and the grayscale cropping unit reduces pixel value overflow.

10. The high-resolution image enhancement system based on multi-scale feature fusion according to claim 8, characterized in that, Also includes: The quality assessment and parameter optimization module includes an index calculation unit, a parameter adjustment unit, and an iterative control unit; the parameter adjustment unit provides feedback to adjust the sampling factor and attention weight, and the iterative control unit reduces the number of iterations for simple texture images; The dynamic weight adjustment module includes a scene recognition unit to distinguish preset type scenes and a weight correction unit to adjust the weights according to texture complexity and color saturation. The color consistency correction module includes a color deviation analysis unit that calculates the mean difference and variance ratio, a gamma correction unit that adjusts the correction coefficient, and a white balance unit that eliminates cross-scale fusion color bias.

Images