Image super-resolution reconstruction method based on deep learning

Abstract

The invention discloses an image super-resolution reconstruction method based on deep learning, and belongs to the technical field of image signal processing. Comprising the following steps: S1, realizing multi-dimensional feature representation of a low-resolution input image by using a deep learning network; s2, realizing accurate alignment and lossless mapping of cross-scale features, and effectively solving the problems of feature dimension inconsistency and information distortion; s3, explicitly learning an increment mapping relation between low-resolution input and a reconstructed high-resolution image, and relieving the gradient disappearance problem of a deep learning network structure; s4, enhancing the naturalness and detail fidelity of the reconstructed image through adversarial training; s5, integrating pixel-level reconstruction loss, perception loss and structural similarity loss, and effectively balancing objective indexes and subjective visual quality of the reconstructed image; and S6, realizing joint optimization of deep learning network parameters through a unified gradient back propagation mechanism, and finally outputting a high-quality super-resolution reconstructed image.

Description

Technical Field

[0001] The present application relates to the technical field of image signal processing, and more specifically, to an image super-resolution reconstruction method based on deep learning. Background Art

[0002] With the advent of the digital age, images are increasingly used in various fields. However, the quality problem of low-resolution images is particularly prominent in many application scenarios. Although traditional image super-resolution reconstruction methods, such as bilinear interpolation and bicubic interpolation, are simple and easy to use, they often introduce blur and distortion while improving image resolution, making it difficult to meet the needs of high-precision image reconstruction.

[0003] The rapid development of deep learning technology has provided a new solution for image super-resolution reconstruction. Deep learning methods can extract deep feature information from low-resolution images by constructing complex neural networks, and can accurately predict and reconstruct high-resolution images by training a large amount of data. These methods have shown great potential in restoring image details and improving visual effects. However, existing deep learning methods still face many challenges in feature extraction, cross-scale feature alignment, and training efficiency, resulting in insufficient detail fidelity and naturalness of reconstructed images.

[0004] At present, the demand for image data and its application scope are constantly expanding. From medical imaging, satellite remote sensing to video surveillance, and advanced visual applications, high-quality image data support is required. Therefore, designing an efficient and accurate deep learning model to achieve reconstruction from low-resolution images to high-resolution images has become an important research direction in the field of image processing.

[0005] In summary, how to achieve super-resolution reconstruction of images based on deep learning technology and improve the resolution and detail fidelity of images has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to overcome a series of defects in the prior art, the purpose of this application is to provide an image super-resolution reconstruction method based on deep learning for the above problems, comprising the following steps:

[0007] S1, using deep learning network to achieve multi-dimensional feature representation of low-resolution input images;

[0008] S2, achieves accurate alignment and lossless mapping of cross-scale features, effectively solving the problem of inconsistent feature dimensions and information distortion;

[0009] S3, explicitly learns the incremental mapping relationship between low-resolution input and reconstructed high-resolution image, alleviating the gradient vanishing problem of deep learning network structure;

[0010] S4, enhancing the naturalness and detail fidelity of reconstructed images through adversarial training;

[0011] S5, integrates pixel-level reconstruction loss, perceptual loss and structural similarity loss to effectively balance the objective indicators and subjective visual quality of the reconstructed image;

[0012] S6, through a unified gradient back-propagation mechanism, realizes the joint optimization of deep learning network parameters and finally outputs high-quality super-resolution reconstructed images.

[0013] Further, S1 includes the following steps:

[0014] By designing convolutional layers with different kernel sizes, local and global texture features of images are captured simultaneously in the shallow and deep layers of the deep learning network, improving the understanding of low-resolution images;

[0015] By stacking convolution blocks and up / down sampling modules, multi-level features are extracted to retain texture details while learning deep semantic information;

[0016] The shallow texture features and deep semantic features are fused across scales through cascading and element-by-element addition to ensure that details are preserved while capturing high-level semantic information;

[0017] Introducing deformable convolution to dynamically adjust the receptive field and sampling position of the convolution kernel, more accurately capture the local structural features of the image, and improve the ability to recognize texture details;

[0018] Through multi-scale feature extraction and fusion, a semantically powerful and detail-rich feature representation is constructed to provide more comprehensive feature information for super-resolution reconstruction.

[0019] Furthermore, the cross-scale fusion is performed by cascading and element-by-element addition, which is expressed as: F f =α·F s +(1-α)·(W p *Cpncat(F s ,F d )+b p ), where F f It is the result obtained by cascading shallow and deep features, which contains the detailed features and semantic information of the image; α is the dynamically learned weight coefficient, which is used to control the relative importance of shallow features and deep features in the final fusion; F s Represents shallow texture features; W p is the convolution kernel weight used to transform features; * indicates convolution operation; Concat(F s ,F d ) represents the cascade operation, which converts the shallow texture feature F sand deep semantic features F d Concatenate in the channel dimension; b p is the bias term after the convolution operation; F d Represents deep semantic features.

[0020] Further, S2 includes the following steps:

[0021] Map features of different scales to a unified query-key-value space, realize nonlinear associations between features through attention weight calculation, and effectively capture complex dependencies between feature dimensions;

[0022] Through learnable channel conversion layers and normalization operations, the number of channels and scales of features of different scales are dynamically adjusted, and channel attention and spatial attention mechanisms are used to adaptively balance and reconstruct feature information;

[0023] The feature alignment strategy is gradually refined at different levels of abstraction, and the semantic bias and scale distortion in feature representation are gradually eliminated through adaptive weight learning and cross-scale feature interaction.

[0024] By introducing an additional cross-scale consistency loss, the semantic matching of features at different scales is further optimized to ensure more accurate feature alignment;

[0025] By introducing instance normalization and conditional normalization techniques, the distribution characteristics of features are dynamically adjusted to eliminate statistical differences between different scales and channels, thereby enhancing the stability and consistency of feature mapping;

[0026] Through feature consistency constraints, the quality of feature mapping is gradually optimized, making the generated feature representation closer to the ideal cross-scale consistent expression.

[0027] Furthermore, S3 includes the following steps:

[0028] Through the deep residual learning network and multi-branch convolutional architecture, an incremental mapping learning mechanism is constructed to accumulate residual information layer by layer, capture subtle changes from low-resolution to high-resolution images, and avoid information loss;

[0029] Introducing a learnable channel attention mechanism to dynamically adjust the importance of different residual branches, optimize residual propagation, alleviate the gradient vanishing problem, and enhance the ability to reconstruct image details;

[0030] Through multi-scale residual fusion strategy and skip connection, the underlying texture features and high-level semantic information are preserved, enhancing the understanding of the global and local structures of the image.

[0031] Residual gradient normalization and adaptive gradient clipping strategies are used to control the residual propagation range and prevent gradient explosion and disappearance. At the same time, sparsity constraints are introduced to improve the compactness and efficiency of incremental mapping representation.

[0032] Based on the recurrent neural network, a gated residual unit is constructed to iteratively refine the residual information, gradually enhance the reconstruction of high-frequency details, and accurately capture the nonlinear mapping relationship from low-resolution to high-resolution images;

[0033] An adaptive residual generator is constructed based on meta-learning, and the network structure and parameters are dynamically adjusted through the residual learning strategy to achieve adaptive matching with the feature distribution of the input image.

[0034] Further, S4 includes the following steps:

[0035] Design a generator and discriminator network, where the generator is responsible for reconstructing the image and the discriminator is used to distinguish the reconstructed image from the real image;

[0036] Design feature matching loss in the intermediate feature layer of the generator. By freezing the parameters of the discriminator, calculate the mean and variance difference between the generated image and the real image at the feature layer, and encourage the generator to generate a reconstruction result that is closer to the real image distribution.

[0037] Through the minimum-maximum game optimization strategy, the generator minimizes the discriminant loss of the discriminator, and the discriminator maximizes the probability of correct classification, realizing a competitive training process;

[0038] The generator and discriminator are optimized alternately to gradually improve the image quality, making the generator close to the real image in terms of visual quality, detail fidelity, and perceptual consistency.

[0039] Furthermore, the difference in mean and variance between the image and the real image at the feature layer is expressed as: Among them, μ G is the mean of the generated image in the intermediate feature layer; μ R is the mean of the real image in the intermediate feature layer; λ1 is the control mean difference (μ G -μ R ) in the loss; λ2 controls the variance difference (σ G -σ R ) weight in the loss; σ G is the variance of the generated image at the intermediate feature layer; σ R is the variance of the real image in the intermediate feature layer; is the square of the Euclidean distance, which means the sum of the squares of the differences between the elements of the vectors; L feat It is the feature matching loss, which is used to measure the statistical difference between the generated image and the real image at the feature layer.

[0040] Further, S5 includes the following steps:

[0041] Calculate the pixel-level difference between the reconstructed image and the original image to quantify the accuracy of the reconstruction;

[0042] Extract the feature representation of the original and reconstructed images at the intermediate layer, calculate their mean square error, and evaluate the similarity of the images at the perceptual and semantic levels;

[0043] The structural similarity index is introduced as the loss function, which focuses on evaluating the structure, contrast and brightness information of the reconstructed image and effectively balances the texture details and overall structural integrity of the image.

[0044] Design an adaptive loss weight strategy to dynamically balance the relative importance of pixel-level reconstruction loss, perceptual loss, and similarity loss;

[0045] By simultaneously evaluating the reconstruction quality at both the coarse-grained and fine-grained scales of the image, the local texture details and global structural features of the image are fully captured, further improving the objective indicators and subjective visual effects of the reconstructed image;

[0046] By adding a regularization term to the loss function to form a comprehensive loss function, the reconstruction artifacts are reduced and the overall reconstruction quality is improved.

[0047] Furthermore, the comprehensive loss function is expressed as: L total =β1·MSE(L orig ,L recom )+β2·L perceptual +β3·(1-SSIM(I orig ,I recon ))+β4·L reg , where L total is a comprehensive loss function that combines pixel-level difference, perceptual loss, structural similarity loss and regularization term to comprehensively evaluate the quality of the reconstructed image; β1 is the weight coefficient of pixel-level reconstruction loss; β2 is the weight coefficient of perceptual loss term; β3 is the weight coefficient of structural similarity loss term; β4 is the weight coefficient of regularization term, which adjusts the contribution of regularization term to the total loss function; MSE(I orig ,L recon ) is the original image I orig and reconstructed image I recon The mean square error between them is used to quantify the difference between images at the pixel level and measure the accuracy of the reconstructed image. The formula is: Where N is the number of pixels in the image; I orig (i) is the original image I orig The pixel value at the i-th pixel position; I recon (u) is the reconstructed image I recon The pixel value at the i-th pixel position; L perceptual It is the perceptual loss, which is calculated based on the feature representation of the intermediate layer and is used to quantify the perceptual difference between the original image and the reconstructed image. The formula is: Among them, M is the number of selected intermediate layers used to calculate the perceptual loss; φ m (I orig ) is the original image I orig Feature representation at the mth layer; φ m (I recon ) is the reconstructed image I recon Feature representation at the mth layer; SSIM(I orig ,I recon ) is the structural similarity index, which is used to evaluate the original image I orig and reconstructed image I recon Similarity in structure, contrast and brightness, the formula is: Among them, μ orig is the original image I orig The mean value of recon To reconstruct image I recon The mean of; c1 is the first constant in the structural similarity index, which prevents the denominator from being zero and usually takes a small constant value; σ orig is the original image I orig The variance of recon To reconstruct image I recon The variance of L; c2 is the second constant in the structural similarity index, which is usually used to stabilize the denominator to prevent the variance from being very small; L reg is the regularization loss, which is used to control the smoothness of network parameters, prevent artifacts and improve the reconstruction quality of the image. The formula is: in, To reconstruct image I recon The gradient at the i-th pixel.

[0048] Further, S6 includes the following steps:

[0049] Construct the corresponding error signal based on the current output of the deep learning network and the preset goal;

[0050] Based on the constructed error signal, the gradient back propagation algorithm is used to calculate the error gradient of each parameter to clarify the impact of the parameter on the output deviation;

[0051] Pass the error gradient information from subsequent layers layer by layer to ensure that each layer receives gradient feedback to guide the direction and magnitude of parameter adjustment;

[0052] According to the transmitted gradient information, all relevant parameters are jointly optimized and adjusted so that the network gradually approaches the output of high-quality super-resolution images;

[0053] Repeatedly perform gradient calculation, information transfer, and joint parameter optimization to gradually narrow the gap between the output and the requirements of high-quality super-resolution reconstructed images, thereby improving the quality of image reconstruction.

[0054] When the output image reaches the expected high quality level, it is output as the final super-resolution reconstructed image, completing the entire super-resolution reconstruction process.

[0055] Compared with the prior art, this application has the following beneficial effects:

[0056] This application significantly improves the reconstruction quality of low-resolution images through deep learning networks and multi-dimensional feature representation. It solves the problem of inconsistent feature dimensions and information distortion, and enhances the naturalness and detail fidelity of images through adversarial training, and comprehensively balances various loss functions to ultimately output high-quality super-resolution reconstructed images with high objective indicators and excellent subjective visual quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of an image super-resolution reconstruction method based on deep learning disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the implementation of the present invention clearer, the technical scheme in the embodiment of the present invention will be described in more detail below in conjunction with the drawings in the embodiment of the present invention. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present invention, not all of the embodiments.

[0059] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0060] The embodiments and directional terms described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0061] like Figure 1 As shown, a deep learning-based image super-resolution reconstruction method comprises the following steps:

[0062] S1, using deep learning network to achieve multi-dimensional feature representation of low-resolution input images;

[0063] S2, achieves accurate alignment and lossless mapping of cross-scale features, effectively solving the problem of inconsistent feature dimensions and information distortion;

[0064] S3, explicitly learns the incremental mapping relationship between low-resolution input and reconstructed high-resolution image, alleviating the gradient vanishing problem of deep learning network structure;

[0065] S4, enhancing the naturalness and detail fidelity of reconstructed images through adversarial training;

[0066] S5, integrates pixel-level reconstruction loss, perceptual loss and structural similarity loss to effectively balance the objective indicators and subjective visual quality of the reconstructed image;

[0067] S6, through a unified gradient back-propagation mechanism, realizes the joint optimization of deep learning network parameters and finally outputs high-quality super-resolution reconstructed images.

[0068] In step S1, the role of the deep learning network is to extract multi-dimensional features from the low-resolution input image, specifically by extracting features from the image through deep learning models such as convolutional neural networks (CNN). Compared with high-resolution images, low-resolution images usually contain fewer details and information, and the information distribution is relatively sparse, so it is necessary to use deep learning algorithms to mine the potential features of the image. The technical effect of this step is that the deep learning model can extract feature information of different scales and levels in the image layer by layer through multi-layer convolution operations, including multi-dimensional features such as edges, textures, and color distribution. These features not only help the network understand the basic structure of the image, but also provide strong feature support for subsequent super-resolution reconstruction. At this stage, the deep learning network can extract higher-dimensional representations from complex low-resolution images, thereby laying the foundation for subsequent image reconstruction and enhancement. The success or failure of this step determines the effect of all subsequent reconstruction operations, so its importance cannot be ignored.

[0069] Low-resolution images suffer from a loss in spatial resolution, so it is necessary to accurately align multi-scale features to ensure that important information is not lost during the reconstruction process. Step S2 uses the alignment mechanism designed in the deep learning model to achieve accurate alignment and lossless mapping of information between multi-scale levels. Especially when processing low-resolution images, features of different scales often have different spatial information, so alignment operations are required to ensure the effective fusion of cross-scale information. In terms of technical effects, by aligning cross-scale features in the feature extraction stage, the deep learning network can avoid information loss or distortion problems, ensuring that all kinds of details of low-resolution images are fully preserved and restored during high-resolution reconstruction. This process can effectively solve the problem of information distortion caused by scale differences, so that features at different levels remain consistent in the network, thereby enhancing the quality and detail fidelity of the reconstructed image.

[0070] The core goal of step S3 is to explicitly learn the incremental mapping relationship between the low-resolution input and the high-resolution reconstructed image. Traditional super-resolution methods usually only focus on the reconstruction of the image itself, while ignoring the subtle differences between the low-resolution image and the high-resolution image. By designing an incremental mapping mechanism in the deep learning network, the network can learn the parts of the low-resolution image that need to be enhanced when converting it to a high-resolution image. This incremental mapping can effectively alleviate the gradient vanishing problem, which is a challenge often encountered by deep learning models during training. The incremental mapping mechanism enables the network to more accurately learn and reconstruct the specific details of the low-resolution image by gradually enhancing the information in the process of strengthening detail recovery, thereby avoiding the phenomenon of gradient vanishing and information degradation, and ensuring high-quality output of the reconstruction results. This process improves the reconstruction accuracy of the image, especially in restoring image details and textures.

[0071] Adversarial training is an advanced technology in deep learning and is widely used in image generation and reconstruction tasks. In super-resolution reconstruction, adversarial training using a generative adversarial network (GAN) can significantly enhance the naturalness and detail fidelity of the image. Step S4 trains the generator to generate more natural and realistic high-resolution images by constructing a game process between the generator and the discriminator. The generator is responsible for generating high-resolution images, while the discriminator judges whether the generated images are realistic enough. The two compete and promote each other, and the final generated image can approach the distribution of the real image. This training method effectively avoids the image distortion problem that may exist in traditional reconstruction methods, enhances the details and texture performance of the reconstructed image, and makes the visual effect of the super-resolution image closer to the real high-resolution image.

[0072] Step S5 optimizes the quality of the reconstructed image by combining different loss functions. Specifically, pixel-level reconstruction loss (such as mean square error), perceptual loss (such as feature loss in VGG network) and structural similarity loss (SSIM) are combined to measure and optimize the quality of the reconstructed image from multiple levels. Pixel-level reconstruction loss focuses on the pixel-level accuracy of the image, ensuring that the reconstructed image is as close to the real image as possible in pixels. Perceptual loss focuses on high-level features of the image, such as texture and structure, to improve the perceived quality of the image. Structural similarity loss optimizes the global consistency of the image by measuring the structural information of the image. Through the combination of these three, the model can not only ensure the fidelity of image details and textures, but also improve the visual effect of the image, avoiding visual unnatural phenomena caused by over-optimization of pixel-level details. The comprehensive loss function can balance the objective quality (such as PSNR) and subjective quality (such as the perceived quality of the human eye) of the reconstructed image, thereby improving the overall effect of image reconstruction.

[0073] In step S6, the entire deep learning network jointly optimizes all parameters through a unified gradient back-propagation mechanism to ensure that the various components in the entire super-resolution reconstruction process can work together to output the best high-quality super-resolution image. Through back-propagation, the network can adjust the parameters of each layer according to the feedback information of each loss function, so that every detail in the image reconstruction process can be optimized. This process enables the network to simultaneously optimize the low-level features (such as pixel accuracy) and high-level features (such as visual effects) of the image, ensuring that the reconstructed image achieves the best results in multiple aspects. The technical effect of this step is that the joint optimization of all network parameters can eliminate potential training errors and ensure that the final output of the network can comprehensively consider all optimization goals, thereby generating clear, natural and detailed super-resolution images.

[0074] In summary, the image super-resolution reconstruction method based on deep learning uses multi-level technical optimization, from feature extraction of low-resolution input images, to precise alignment of image cross-scale features, to explicit learning of incremental mapping, and finally to adversarial training to enhance the naturalness and detail fidelity of images. Each step revolves around improving the quality and visual effect of image reconstruction. By comprehensively using pixel-level reconstruction loss, perceptual loss, and structural similarity loss, the network parameters are finally jointly optimized through the gradient back-propagation mechanism to ensure high-quality output of super-resolution images. This multi-dimensional technical integration not only improves the objective quality of images, but also enhances the subjective visual effect of images, providing strong theoretical and practical support for the application of image super-resolution reconstruction technology.

[0075] Further, S1 includes the following steps:

[0076] By designing convolutional layers with different kernel sizes, local and global texture features of images are captured simultaneously in the shallow and deep layers of the deep learning network, improving the understanding of low-resolution images;

[0077] By stacking convolution blocks and up / down sampling modules, multi-level features are extracted to retain texture details while learning deep semantic information;

[0078] The shallow texture features and deep semantic features are fused across scales through cascading and element-by-element addition to ensure that details are preserved while capturing high-level semantic information;

[0079] Introducing deformable convolution to dynamically adjust the receptive field and sampling position of the convolution kernel, more accurately capture the local structural features of the image, and improve the ability to recognize texture details;

[0080] Through multi-scale feature extraction and fusion, a semantically powerful and detail-rich feature representation is constructed to provide more comprehensive feature information for super-resolution reconstruction.

[0081] By introducing a variety of technical means, this deep learning-based image super-resolution reconstruction method has achieved significant improvements in detail restoration and semantic understanding. From convolutional layers with different kernel sizes to capture local and global texture features, to the extraction and fusion of multi-level features, to the introduction of deformable convolution, the technical effects of these steps complement each other and jointly build a powerful feature representation system. These technologies not only effectively improve the texture details of the image, but also enhance the semantic understanding ability of the image, making the super-resolution reconstructed image more natural, delicate and realistic. Through multi-scale feature extraction and fusion, the network can comprehensively utilize information at various levels to provide more comprehensive support for image reconstruction, and finally output high-quality super-resolution images. The innovation of this method not only improves image quality, but also provides new ideas for the application of deep learning in image reconstruction.

[0082] Furthermore, the cross-scale fusion is performed by cascading and element-by-element addition, which is expressed as: F f =α·F s +(1-α)·(W p *Concat(F s ,F d )+b p ), where F f It is the result obtained by cascading shallow and deep features, which contains the detailed features and semantic information of the image; α is the dynamically learned weight coefficient, which is used to control the relative importance of shallow features and deep features in the final fusion; F s Represents shallow texture features; W p is the convolution kernel weight used to transform features; * indicates convolution operation; Concat(F s ,F d ) represents the cascade operation, which converts the shallow texture feature F s and deep semantic features F d Concatenate in the channel dimension; b p is the bias term after the convolution operation; F d Represents deep semantic features.

[0083] In summary, by cascading and element-by-element addition to perform cross-scale feature fusion, combined with dynamically learned weight coefficients, the deep learning network can effectively balance detail recovery and semantic preservation when processing low-resolution images. The fusion of shallow texture features and deep semantic features not only improves the accuracy of image details, but also enhances the overall structural understanding of the image. The dynamically adjusted weight coefficients make the network highly adaptable under different input conditions and can flexibly respond to various image characteristics. The innovativeness of this technical method improves the quality of image super-resolution reconstruction and provides a more efficient and accurate solution for image reconstruction tasks. The final output image has both rich details and good semantic consistency.

[0084] Further, S2 includes the following steps:

[0085] Map features of different scales to a unified query-key-value space, realize nonlinear associations between features through attention weight calculation, and effectively capture complex dependencies between feature dimensions;

[0086] Through learnable channel conversion layers and normalization operations, the number of channels and scales of features of different scales are dynamically adjusted, and channel attention and spatial attention mechanisms are used to adaptively balance and reconstruct feature information;

[0087] The feature alignment strategy is gradually refined at different levels of abstraction, and the semantic bias and scale distortion in feature representation are gradually eliminated through adaptive weight learning and cross-scale feature interaction.

[0088] By introducing an additional cross-scale consistency loss, the semantic matching of features at different scales is further optimized to ensure more accurate feature alignment;

[0089] By introducing instance normalization and conditional normalization techniques, the distribution characteristics of features are dynamically adjusted to eliminate statistical differences between different scales and channels, thereby enhancing the stability and consistency of feature mapping;

[0090] Through feature consistency constraints, the quality of feature mapping is gradually optimized, making the generated feature representation closer to the ideal cross-scale consistent expression.

[0091] In summary, the technical design and implementation of step S2 demonstrate how to solve the problem of cross-scale feature mismatch and information loss in image super-resolution reconstruction through fine feature alignment and optimization. Through attention mechanism, dynamic adjustment of channels and scales, cross-scale consistency loss, normalization technology, and feature consistency constraints, the network can effectively learn and optimize the relationship between different scales, so that the reconstructed image can maintain details and semantic consistency while improving the overall image quality. The introduction of these technical solutions not only enhances the ability to restore image details, but also improves the stability and adaptability of the network in multi-scale feature fusion. Finally, through the optimization of these strategies, the generated super-resolution images have higher realism and visual quality, providing strong support for image processing tasks in practical applications.

[0092] Furthermore, S3 includes the following steps:

[0093] Through the deep residual learning network and multi-branch convolutional architecture, an incremental mapping learning mechanism is constructed to accumulate residual information layer by layer, capture subtle changes from low-resolution to high-resolution images, and avoid information loss;

[0094] Introducing a learnable channel attention mechanism to dynamically adjust the importance of different residual branches, optimize residual propagation, alleviate the gradient vanishing problem, and enhance the ability to reconstruct image details;

[0095] Through multi-scale residual fusion strategy and skip connection, the underlying texture features and high-level semantic information are preserved, enhancing the understanding of the global and local structures of the image.

[0096] Residual gradient normalization and adaptive gradient clipping strategies are used to control the residual propagation range and prevent gradient explosion and disappearance. At the same time, sparsity constraints are introduced to improve the compactness and efficiency of incremental mapping representation.

[0097] Based on the recurrent neural network, a gated residual unit is constructed to iteratively refine the residual information, gradually enhance the reconstruction of high-frequency details, and accurately capture the nonlinear mapping relationship from low-resolution to high-resolution images;

[0098] An adaptive residual generator is constructed based on meta-learning, and the network structure and parameters are dynamically adjusted through the residual learning strategy to achieve adaptive matching with the feature distribution of the input image.

[0099] In summary, step S3 establishes an incremental mapping learning mechanism through a series of innovative technical means, which solves the problems of gradient vanishing, information loss and detail recovery faced in the process of reconstructing low-resolution images to high-resolution images. The deep residual learning network and multi-branch convolutional architecture ensure that the details of the image can be restored layer by layer, while the channel attention mechanism and residual optimization further enhance the network's detail reconstruction ability. The introduction of multi-scale residual fusion and jump connection enables the underlying texture features and high-level semantic information to communicate effectively, improving the global and local structural understanding of the image. In addition, through residual gradient normalization, adaptive gradient clipping and RNN-based gated residual units, S3 optimizes the network training process and avoids common problems in traditional deep networks. The introduction of meta-learning provides the network with the ability to adaptively adjust, so that the network can dynamically adjust its structure and parameters according to the feature distribution of different inputs, thereby improving the accuracy and generalization of image reconstruction.

[0100] Further, S4 includes the following steps:

[0101] Design a generator and a discriminator network, where the generator is responsible for reconstructing the image and the discriminator is used to distinguish the reconstructed image from the real image;

[0102] Design feature matching loss in the intermediate feature layer of the generator. By freezing the parameters of the discriminator, calculate the mean and variance difference between the generated image and the real image at the feature layer, and encourage the generator to generate a reconstruction result that is closer to the real image distribution.

[0103] Through the minimum-maximum game optimization strategy, the generator minimizes the discriminant loss of the discriminator, and the discriminator maximizes the probability of correct classification, realizing a competitive training process;

[0104] The generator and discriminator are optimized alternately to gradually improve the image quality, making the generator close to the real image in terms of visual quality, detail fidelity, and perceptual consistency.

[0105] In summary, step S4 significantly improves the quality of super-resolution image reconstruction through the adversarial training mechanism. The mutual game between the generator and the discriminator promotes the generator to generate more realistic images by minimizing the discriminant loss of the discriminator; the feature matching loss enhances the detail fidelity and naturalness of the image by reducing the difference between the generated image and the real image at the feature layer; the minimum-maximum game optimization strategy creates a benign competition between the generator and the discriminator, further improving the image quality; the alternating optimization strategy ensures that the generator and the discriminator continue to improve during the game process, and finally generate high-quality images. Through this series of adversarial training methods, S4 not only improves the details and clarity of the reconstructed image, but also improves the visual perception quality of the image, providing stronger technical support for the task of super-resolution image reconstruction.

[0106] Furthermore, the difference in mean and variance between the image and the real image at the feature layer is expressed as: Among them, μ G is the mean of the generated image in the intermediate feature layer; μ R is the mean of the real image in the intermediate feature layer; λ1 is the control mean difference (μ G -μ R ) in the loss; λ2 controls the variance difference (σ G -σ R ) weight in the loss; σ G is the variance of the generated image at the intermediate feature layer; σ R is the variance of the real image in the intermediate feature layer; is the square of the Euclidean distance, which means the sum of the squares of the differences between the elements of the vectors; L feat It is the feature matching loss, which is used to measure the statistical difference between the generated image and the real image at the feature layer.

[0107] Further, S5 includes the following steps:

[0108] Calculate the pixel-level difference between the reconstructed image and the original image to quantify the accuracy of the reconstruction;

[0109] Extract the feature representation of the original and reconstructed images at the intermediate layer, calculate their mean square error, and evaluate the similarity of the images at the perceptual and semantic levels;

[0110] The structural similarity index is introduced as the loss function, which focuses on evaluating the structure, contrast and brightness information of the reconstructed image and effectively balances the texture details and overall structural integrity of the image.

[0111] Design an adaptive loss weight strategy to dynamically balance the relative importance of pixel-level reconstruction loss, perceptual loss, and similarity loss;

[0112] By simultaneously evaluating the reconstruction quality at both the coarse-grained and fine-grained scales of the image, the local texture details and global structural features of the image are fully captured, further improving the objective indicators and subjective visual effects of the reconstructed image;

[0113] By adding a regularization term to the loss function to form a comprehensive loss function, the reconstruction artifacts are reduced and the overall reconstruction quality is improved.

[0114] In summary, S5 optimizes the image reconstruction quality at multiple levels by comprehensively introducing pixel-level loss, perceptual loss, structural similarity loss, and adaptive loss weight strategy. By accurately evaluating the differences at the pixel level, perceptual level, and structural level, S5 can generate more natural and delicate high-resolution images. In particular, S5 has demonstrated strong capabilities in detail recovery and structural consistency. The introduction of adaptive loss weight strategy and multi-scale evaluation balances the quality of the reconstructed image at all levels. The final generated image not only performs well in objective quality, but also has a strong visual effect and sense of reality.

[0115] Furthermore, the comprehensive loss function is expressed as: L total =β1·MSE(I orig ,I recon )+β2·L perceptuakl +β3·(1-SSIM(I orig ,I recon ))+β4·L reg , where L total is a comprehensive loss function that combines pixel-level difference, perceptual loss, structural similarity loss and regularization term to comprehensively evaluate the quality of the reconstructed image; β1 is the weight coefficient of pixel-level reconstruction loss; β2 is the weight coefficient of perceptual loss term; β3 is the weight coefficient of structural similarity loss term; β4 is the weight coefficient of regularization term, which adjusts the contribution of regularization term to the total loss function; MSE(I orig ,I recon ) is the original image I orig and reconstructed image I recon The mean square error between them is used to quantify the difference between images at the pixel level and measure the accuracy of the reconstructed image. The formula is: Where N is the number of pixels in the image; I orig (i) is the original image I orig The pixel value at the i-th pixel position; I recon (i) is the reconstructed image I recon The pixel value at the i-th pixel position; L perceptual It is the perceptual loss, which is calculated based on the feature representation of the intermediate layer and is used to quantify the perceptual difference between the original image and the reconstructed image. The formula is: Among them, M is the number of selected intermediate layers used to calculate the perceptual loss; φ m (I orig ) is the original image I orig Feature representation at the mth layer; φ m (I recon ) is the reconstructed image I recon Feature representation at the mth layer; SSIM(I orig ,I recon) is the structural similarity index, which is used to evaluate the original image I orig and reconstructed image I recon Similarity in structure, contrast and brightness, the formula is: Among them, μ orig is the original image I orig The mean value of recon To reconstruct image I recon The mean of; c1 is the first constant in the structural similarity index, which prevents the denominator from being zero and usually takes a small constant value; σ orig is the original image I orig The variance of recon To reconstruct image I recon The variance of L; c2 is the second constant in the structural similarity index, which is usually used to stabilize the denominator to prevent the variance from being very small; L reg is the regularization loss, which is used to control the smoothness of network parameters, prevent artifacts and improve the reconstruction quality of the image. The formula is: in, To reconstruct image I recon The gradient at the i-th pixel.

[0116] Further, S6 includes the following steps:

[0117] Construct the corresponding error signal based on the current output of the deep learning network and the preset goal;

[0118] Based on the constructed error signal, the gradient back propagation algorithm is used to calculate the error gradient of each parameter to clarify the impact of the parameter on the output deviation;

[0119] Pass the error gradient information from subsequent layers layer by layer to ensure that each layer receives gradient feedback to guide the direction and magnitude of parameter adjustment;

[0120] According to the transmitted gradient information, all relevant parameters are jointly optimized and adjusted so that the network gradually approaches the output of high-quality super-resolution images;

[0121] Repeatedly perform gradient calculation, information transfer, and joint parameter optimization to gradually narrow the gap between the output and the requirements of high-quality super-resolution reconstructed images, thereby improving the quality of image reconstruction.

[0122] When the output image reaches the expected high quality level, it is output as the final super-resolution reconstructed image, completing the entire super-resolution reconstruction process.

[0123] In summary, S6 achieves gradual optimization in the process of super-resolution image reconstruction through the back-propagation algorithm and gradient transfer mechanism. It can guide the adjustment of network parameters through error feedback and gradient calculation, thereby gradually narrowing the gap between the generated image and the target image. Repeated gradient calculation and information transfer ensure that each level of the deep learning model is finely optimized, so that the final super-resolution image can achieve a high quality level in terms of detail recovery and structural consistency. Finally, through multiple rounds of iteration and optimization, S6 ensures that the quality of the generated image meets expectations and completes the entire super-resolution reconstruction task.

[0124] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image super-resolution reconstruction based on deep learning, characterized in that: The following steps are involved: S1, using deep learning network to achieve multi-dimensional feature representation of low-resolution input images; S2, achieves accurate alignment and lossless mapping of cross-scale features, effectively solving the problem of inconsistent feature dimensions and information distortion; S3, explicitly learns the incremental mapping relationship between low-resolution input and reconstructed high-resolution image, alleviating the gradient vanishing problem of deep learning network structure; S4, enhancing the naturalness and detail fidelity of reconstructed images through adversarial training; S5, integrates pixel-level reconstruction loss, perceptual loss and structural similarity loss to effectively balance the objective indicators and subjective visual quality of the reconstructed image; S6, through a unified gradient back-propagation mechanism, realizes the joint optimization of deep learning network parameters and finally outputs high-quality super-resolution reconstructed images.

2. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S1 includes the following steps: By designing convolutional layers with different kernel sizes, local and global texture features of images are captured simultaneously in the shallow and deep layers of the deep learning network, improving the understanding of low-resolution images; By stacking convolution blocks and up / down sampling modules, multi-level features are extracted to retain texture details while learning deep semantic information; The shallow texture features and deep semantic features are fused across scales through cascading and element-by-element addition to ensure that details are preserved while capturing high-level semantic information; Introducing deformable convolution to dynamically adjust the receptive field and sampling position of the convolution kernel, more accurately capture the local structural features of the image, and improve the ability to recognize texture details; Through multi-scale feature extraction and fusion, a semantically powerful and detail-rich feature representation is constructed to provide more comprehensive feature information for super-resolution reconstruction.

3. The image super-resolution reconstruction method based on deep learning according to claim 2, characterized in that: The cross-scale fusion through cascading and element-by-element addition is expressed as: F f =α·F s +(1-α)·(W p *Concat(F s ,F d )+b p ), where F f It is the result obtained by cascading shallow and deep features, which contains the detailed features and semantic information of the image; α is the dynamically learned weight coefficient, which is used to control the relative importance of shallow features and deep features in the final fusion; F s Represents shallow texture features; W p is the convolution kernel weight used to transform features; * indicates convolution operation; Concat(F s ,F d ) represents the cascade operation, which converts the shallow texture feature F s and deep semantic features F d Splicing in the channel dimension; b p is the bias term after the convolution operation; F d Represents deep semantic features.

4. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S2 includes the following steps: Map features of different scales to a unified query-key-value space, realize nonlinear associations between features through attention weight calculation, and effectively capture complex dependencies between feature dimensions; Through learnable channel conversion layers and normalization operations, the number of channels and scales of features of different scales are dynamically adjusted, and channel attention and spatial attention mechanisms are used to adaptively balance and reconstruct feature information; The feature alignment strategy is gradually refined at different levels of abstraction, and the semantic bias and scale distortion in feature representation are gradually eliminated through adaptive weight learning and cross-scale feature interaction. By introducing an additional cross-scale consistency loss, the semantic matching of features at different scales is further optimized to ensure more accurate feature alignment; By introducing instance normalization and conditional normalization techniques, the distribution characteristics of features are dynamically adjusted to eliminate statistical differences between different scales and channels, thereby enhancing the stability and consistency of feature mapping; Through feature consistency constraints, the quality of feature mapping is gradually optimized, making the generated feature representation closer to the ideal cross-scale consistent expression.

5. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S3 includes the following steps: Through the deep residual learning network and multi-branch convolutional architecture, an incremental mapping learning mechanism is constructed to accumulate residual information layer by layer, capture subtle changes from low-resolution to high-resolution images, and avoid information loss; Introducing a learnable channel attention mechanism to dynamically adjust the importance of different residual branches, optimize residual propagation, alleviate the gradient vanishing problem, and enhance the ability to reconstruct image details; Through multi-scale residual fusion strategy and skip connection, the underlying texture features and high-level semantic information are preserved, enhancing the understanding of the global and local structures of the image. Residual gradient normalization and adaptive gradient clipping strategies are used to control the residual propagation range and prevent gradient explosion and disappearance. At the same time, sparsity constraints are introduced to improve the compactness and efficiency of incremental mapping representation. Based on the recurrent neural network, a gated residual unit is constructed to iteratively refine the residual information, gradually enhance the reconstruction of high-frequency details, and accurately capture the nonlinear mapping relationship from low-resolution to high-resolution images; An adaptive residual generator is constructed based on meta-learning, and the network structure and parameters are dynamically adjusted through the residual learning strategy to achieve adaptive matching with the feature distribution of the input image.

6. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S4 includes the following steps: Design a generator and discriminator network, where the generator is responsible for reconstructing the image and the discriminator is used to distinguish the reconstructed image from the real image; Design feature matching loss in the intermediate feature layer of the generator. By freezing the parameters of the discriminator, calculate the mean and variance difference between the generated image and the real image at the feature layer, and encourage the generator to generate a reconstruction result that is closer to the real image distribution. Through the minimum-maximum game optimization strategy, the generator minimizes the discriminant loss of the discriminator, and the discriminator maximizes the probability of correct classification, realizing a competitive training process; The generator and discriminator are optimized alternately to gradually improve the image quality, making the generator close to the real image in terms of visual quality, detail fidelity, and perceptual consistency.

7. The image super-resolution reconstruction method based on deep learning according to claim 6, characterized in that: The difference in mean and variance between the image and the real image at the feature layer is expressed as: Among them, μ G is the mean of the generated image in the intermediate feature layer; μ R is the mean of the real image in the intermediate feature layer; λ1 is the control mean difference (μ G -μ R ) in the loss; λ2 controls the variance difference (σ G -σ R ) weight in the loss; σ G is the variance of the generated image at the intermediate feature layer; σ R is the variance of the real image in the intermediate feature layer; is the square of the Euclidean distance, which means the sum of the squares of the differences between the elements of the vectors; L feat It is the feature matching loss, which is used to measure the statistical difference between the generated image and the real image at the feature layer.

8. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S5 includes the following steps: Calculate the pixel-level difference between the reconstructed image and the original image to quantify the accuracy of the reconstruction; Extract the feature representation of the original and reconstructed images at the intermediate layer, calculate their mean square error, and evaluate the similarity of the images at the perceptual and semantic levels; The structural similarity index is introduced as the loss function, which focuses on evaluating the structure, contrast and brightness information of the reconstructed image and effectively balances the texture details and overall structural integrity of the image. Design an adaptive loss weight strategy to dynamically balance the relative importance of pixel-level reconstruction loss, perceptual loss, and similarity loss; By simultaneously evaluating the reconstruction quality at both the coarse-grained and fine-grained scales of the image, the local texture details and global structural features of the image are fully captured, further improving the objective indicators and subjective visual effects of the reconstructed image; By adding a regularization term to the loss function to form a comprehensive loss function, the reconstruction artifacts are reduced and the overall reconstruction quality is improved.

9. The image super-resolution reconstruction method based on deep learning according to claim 8, characterized in that: The comprehensive loss function is expressed as: L total =β1·MSE(I orig ,I recon )+β2·L perceptual +β3·(1-SSIM(I orig ,I recon ))+β4·L reg , where L total is a comprehensive loss function that combines pixel-level difference, perceptual loss, structural similarity loss and regularization term to comprehensively evaluate the quality of the reconstructed image; β1 is the weight coefficient of pixel-level reconstruction loss; β2 is the weight coefficient of perceptual loss term; β3 is the weight coefficient of structural similarity loss term; β4 is the weight coefficient of regularization term, which adjusts the contribution of regularization term to the total loss function; MSE(I orig ,I recon ) is the original image I orig and reconstructed image I recon The mean square error between them is used to quantify the difference between images at the pixel level and measure the accuracy of the reconstructed image. The formula is: Where N is the number of pixels in the image; I orig (i) is the original image I orig The pixel value at the i-th pixel position; I recon (i) is the reconstructed image I recon The pixel value at the i-th pixel position; L perceptual It is the perceptual loss, which is calculated based on the feature representation of the intermediate layer and is used to quantify the perceptual difference between the original image and the reconstructed image. The formula is: Among them, M is the number of selected intermediate layers used to calculate the perceptual loss; φ m (I orig ) is the original image I orig Feature representation at the mth layer; φ m (I recon ) is the reconstructed image I recon Feature representation at the mth layer; SSIM(I orig ,I recon ) is the structural similarity index, which is used to evaluate the original image I orig and reconstructed image I recon Similarity in structure, contrast and brightness, the formula is: Among them, μ orig is the original image I orig The mean value of recon To reconstruct image I recon The mean of; c1 is the first constant in the structural similarity index, which prevents the denominator from being zero and usually takes a small constant value; σ orig is the original image I orig The variance of recon To reconstruct image I recon The variance of L; c2 is the second constant in the structural similarity index, which is usually used to stabilize the denominator to prevent the variance from being very small; L reg is the regularization loss, which is used to control the smoothness of network parameters, prevent artifacts and improve the reconstruction quality of the image. The formula is: in, To reconstruct image I recon The gradient at the i-th pixel.

10. The image super-resolution reconstruction method based on deep learning according to claim 1, characterized in that: S6 includes the following steps: Construct the corresponding error signal based on the current output of the deep learning network and the preset goal; Based on the constructed error signal, the gradient back propagation algorithm is used to calculate the error gradient of each parameter to clarify the impact of the parameter on the output deviation; Pass the error gradient information from subsequent layers layer by layer to ensure that each layer receives gradient feedback to guide the direction and magnitude of parameter adjustment; According to the transmitted gradient information, all relevant parameters are jointly optimized and adjusted so that the network gradually approaches the output of high-quality super-resolution images; Repeatedly perform gradient calculation, information transfer, and joint parameter optimization to gradually narrow the gap between the output and the requirements of high-quality super-resolution reconstructed images, thereby improving the quality of image reconstruction. When the output image reaches the expected high quality level, it is output as the final super-resolution reconstructed image, completing the entire super-resolution reconstruction process.

Images