Real image super-resolution method guided by reliable semantics and uncertainty
Patent Information
- Application Number
- CN202611296621.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-10-09
AI Technical Summary
现有语义引导依赖低分辨率输入的特征表示，模糊、噪声、压缩等退化因素会改变特征分布，导致语义提示偏移，进而造成类别错误、颜色偏移、结构不一致及错误纹理等问题
1.本发明通过构建退化特征差异矩阵并进行奇异值分解获得表征退化偏差的退化子空间，将语义提取器的权重投影至退化子空间的正交补空间，从而抑制退化相关特征对语义提取过程的干扰，保留更稳定的语义信息，降低类别误判、颜色偏移及虚假目标生成等语义错误，提高超分辨率结果的语义保真度。
Smart Images
Figure CN122887676A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image data processing technology, specifically relating to a real image super-resolution method jointly guided by reliable semantics and uncertainty. Background Technology
[0002] Image super-resolution aims to recover higher-resolution details from low-resolution images. In real-world scenarios, low-resolution images are not simply ideal downsampled results, but are affected by various unknown degradation factors such as blurring, noise, and compression. In recent years, super-resolution methods based on pre-trained diffusion models have become a research focus. These methods utilize the image generation prior of the pre-trained model to extract semantic features from the low-resolution image as cues, guiding the model to recover missing textures and structures. Existing methods include high-level semantic extraction, degradation-aware cues, and model distillation to reduce inference costs.
[0003] However, existing technologies still have the following shortcomings: First, degraded information easily interferes with semantic extraction. Existing semantic guidance relies on low-resolution input feature representations. Degraded factors such as blurring, noise, and compression can alter the feature distribution, leading to semantic cues shift and consequently causing problems such as category errors, color shifts, structural inconsistencies, and incorrect textures. Simply adding semantic cues or degradation-aware cues cannot distinguish between degraded and valid semantics at the feature level.
[0004] Secondly, the difficulty of restoration varies across different regions, but targeted modeling is lacking. Real-world image degradation exhibits spatial non-uniformity, with textured and edge regions being more difficult to restore than flat regions. Existing methods employ a uniform generation strategy, making it difficult to dynamically allocate feature representation capabilities according to regional difficulty, easily leading to texture loss, blurred edges, and excessive smoothing. While existing uncertainty modeling can describe partial restoration confidence, it does not utilize frequency domain reconstruction errors to characterize the spatial restoration difficulty.
[0005] Finally, it is difficult to balance semantic consistency and detail realism. Semantic guidance mainly addresses the generation of content and semantics, while uncertainty- or frequency-aware methods mainly focus on local detail recovery. The lack of an effective joint mechanism between the two leads to semantic errors under severe degradation and insufficient detail recovery in complex texture areas.
[0006] Therefore, there is an urgent need for a super-resolution method for real-world images that can address both semantic reliability and the difficulty of region recovery, in order to solve the aforementioned technical problems. Summary of the Invention
[0007] This invention provides the following technical solution: a real image super-resolution method jointly guided by reliable semantics and uncertainty, comprising the following steps: Step S1: Perform feature difference analysis on the paired high-resolution and low-resolution images, construct a degradation feature difference matrix, and perform singular value decomposition on the degradation feature difference matrix to obtain the degradation subspace.
[0008] Step S2: Determine the corresponding orthogonal complement space based on the degenerate subspace, and project the weights in the semantic extractor onto the orthogonal complement space to obtain a robust semantic extractor based on degenerate subspace decomposition. The robust semantic extractor is used to output reliable semantic hints.
[0009] Step S3: Perform wavelet transform on the super-resolution image and the corresponding high-resolution image, construct uncertainty supervision information based on the reconstruction error of different frequency sub-bands, train the uncertainty prediction network, and use the uncertainty prediction network to output pixel-level uncertainty map.
[0010] Step S4: Input reliable semantic prompts as conditional information into the diffusion model UNet, introduce pixel-level uncertainty maps into the feature modulation and spatial attention process of the diffusion model UNet, adaptively adjust the network features, and train to obtain the diffusion super-resolution network to generate super-resolution images.
[0011] Preferably, step S1 specifically includes: inputting the paired high-resolution image and low-resolution image into the pre-trained visual encoder respectively, and obtaining the corresponding visual features from different network layers; aggregating the visual features in the token dimension of the features, calculating the difference between the low-resolution features and the high-resolution features, and obtaining the degradation feature difference matrix.
[0012] In step S1, after performing singular value decomposition on the degenerate feature difference matrix, the right singular vectors corresponding to the first k largest singular values are selected to construct the degenerate subspace, where k is a positive integer.
[0013] Preferably, step S2 specifically includes: according to Calculate the first The orthogonal complement space corresponding to the degenerate subspace of the layer, where It is the identity matrix. For the first Degenerate subspace of the layer; The first The weights of the MLP in the layer semantic extractor are projected onto the orthogonal complement space to obtain the edited weights. ,in These are the original weights.
[0014] Preferably, in step S3, discrete wavelet transform is performed on the super-resolution image and the corresponding high-resolution image to obtain their respective low-frequency sub-bands and multiple high-frequency sub-bands; the reconstruction error of the super-resolution image and the high-resolution image in each wavelet sub-band is calculated respectively.
[0015] Preferably, in step S3, the uncertainty supervision information is obtained by assigning different weights to the reconstruction errors of different frequency sub-bands and then stitching them along the channel dimension, wherein the weight of the high-frequency sub-band is greater than the weight of the low-frequency sub-band.
[0016] Preferably, in step S3, the uncertainty prediction network takes the wavelet domain features corresponding to the initial super-resolution result of the low-resolution image as input and outputs a multi-channel uncertainty response; the multi-channel uncertainty response is averaged and aggregated to obtain a pixel-level uncertainty map.
[0017] Preferably, in step S4, the feature modulation process of introducing the pixel-level uncertainty map into the diffusion model UNet specifically includes: in the upsampling stage of UNet, using an uncertainty encoder to extract weights from the pixel-level uncertainty map and generate scale modulation parameters. and offset parameters ;according to The UNet features are normalized and affine modulated, where F represents the features from the UNet upsampling stage. and Let F represent the mean and standard deviation of feature F, respectively.
[0018] Preferably, in step S4, the process of introducing the pixel-level uncertainty map into the spatial attention process of the diffusion model UNet specifically includes: The pixel-level uncertainty map is used as an additional spatial prior and fused with the average pooling and max pooling results of UNet features. After convolution and activation, a spatial attention map is obtained. Based on the spatial attention map, the uncertainty-modulated features are dynamically fused with the original UNet features. Regions with larger spatial attention values adopt more uncertainty-modulated features, while regions with smaller spatial attention values retain more original UNet features.
[0019] Preferably, in step S4, the loss function used to train the diffusion super-resolution network is a weighted combination of mean squared error loss, perceptual loss, and generative adversarial loss.
[0020] Preferably, during the inference stage, only the low-resolution image to be processed needs to be input. Reliable semantic cues and pixel-level uncertainty maps are obtained through a robust semantic extractor and an uncertainty prediction network, respectively. The two are then input into the trained diffusion super-resolution network to generate the final super-resolution image.
[0021] The beneficial effects of this invention are: 1. This invention constructs a degradation feature difference matrix and performs singular value decomposition to obtain a degradation subspace representing degradation bias. The weights of the semantic extractor are projected onto the orthogonal complement space of the degradation subspace, thereby suppressing the interference of degradation-related features on the semantic extraction process, preserving more stable semantic information, reducing semantic errors such as misclassification, color shift and false target generation, and improving the semantic fidelity of super-resolution results.
[0022] 2. This invention utilizes wavelet domain high-frequency reconstruction errors to construct pixel-level uncertainty information, characterizing the recovery difficulty of different spatial regions. The uncertainty information is then introduced into the feature modulation and spatial attention processes of the diffusion model, enabling the model to dynamically adjust the feature response according to the region recovery difficulty. Targeted enhancement of texture, edge, and structural details is performed on high-uncertainty regions, thereby improving the problems of detail loss, edge blurring, and over-smoothing in complex regions.
[0023] 3. As the degradation becomes more severe, the semantic information of the input image is more significantly interfered with, and existing methods that directly extract semantic information from degraded images are more prone to semantic shift. This invention effectively suppresses the influence of degradation direction through degradation subspace modeling and orthogonal weight editing, maintaining more stable semantic guidance capabilities in complex and severe degradation scenarios, and exhibiting more significant performance advantages.
[0024] 4. By combining reliable semantic information and spatial uncertainty information, this invention ensures the overall content and semantic consistency while focusing on enhancing areas that are difficult to recover, thereby improving the texture and structural realism of the generated image and reducing false textures and details that do not conform to the real image structure that are prone to occur during the diffusion model generation process.
[0025] 5. Compared with related methods that rely on large-scale generation priors for computationally intensive generation, this invention provides targeted guidance for the diffusion generation process through reliable semantic prompts and regional uncertainty information. While ensuring the realism of the reconstruction, it helps to reduce the generation of invalid textures and improve the efficiency of image reconstruction. Attached Figure Description
[0026] Figure 1 This is a flowchart of the weight editing process for the degenerate robust semantic extractor in the real image super-resolution method jointly guided by reliable semantics and uncertainty of the present invention; Figure 2 This is a flowchart of the training process for the wavelet decomposition-based uncertain guidance module of the present invention. Figure 3 This is the overall flowchart of the present invention. Detailed Implementation
[0027] The relevant technologies of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0028] like Figures 1-3 As shown, the real image super-resolution method jointly guided by reliable semantics and uncertainty in this embodiment includes the following steps: Feature difference analysis is performed on paired high-resolution and low-resolution images, and singular value decomposition is performed on the resulting degradation feature difference matrix to obtain the degradation subspace.
[0029] The corresponding orthogonal complement space is determined based on the degenerate subspace, and the weights in the semantic extractor are projected onto the orthogonal complement space to obtain a robust semantic extractor based on degenerate subspace decomposition.
[0030] Wavelet transform is performed on the super-resolution image and the corresponding high-resolution image, and uncertainty supervision information is constructed based on the reconstruction error of different frequency sub-bands to train an uncertainty prediction network.
[0031] The robust semantic extractor generates reliable semantic cues, and the uncertainty information generated by the uncertainty prediction network is introduced into the diffusion model UNet. The network features are adaptively adjusted through feature modulation and spatial attention mechanisms to obtain the trained diffusion super-resolution network and generate super-resolution images.
[0032] Furthermore, the step of obtaining the high-resolution image specifically includes: Feature difference analysis is performed on paired high-resolution and low-resolution images, and singular value decomposition is performed on the resulting degradation feature difference matrix to obtain the degradation subspace.
[0033] Obtain high-resolution images with corresponding relationships. and low-resolution images The low-resolution image and the high-resolution image are respectively input into the pre-trained visual encoder. Figure 1 (1, 3) Obtain corresponding visual features from different network layers. and .
[0034] ( Figure 1 The second module (in the code) aggregates features along the token dimension, denoted as... The features obtained from the layer are Where N represents the number of tokens and D represents the embedding dimension. Next, the difference between the low-resolution and high-resolution features is calculated to obtain the difference matrix:
[0035] This difference matrix reflects the degradation bias introduced by the low-resolution image compared to the high-resolution image. Next, singular value decomposition is performed on the degradation feature difference matrix:
[0036] in, , These are the left and right singular vector matrices, respectively. This is a diagonal matrix of singular values. The directions with larger singular values correspond to the dominant directions where energy is more concentrated in the degenerate feature difference matrix, representing the main offset direction of low-resolution features relative to high-resolution features. According to the properties of singular value decomposition, larger singular values correspond to the dominant directions where energy is more concentrated in the residual matrix. Therefore, this invention selects the right singular vectors corresponding to the k largest singular values. These are constructed as degenerate subspaces (DegraSpace), which characterize the main differences between true features and degenerate features.
[0037] The corresponding orthogonal complement space is determined based on the degenerate subspace, and the weights in the semantic extractor are projected onto the orthogonal complement space to obtain a robust semantic extractor based on degenerate subspace decomposition.
[0038] Further, calculate the first The complement space of the corresponding degenerate subspace of the layer is obtained as the complement space:
[0039] in, Represents the identity matrix. This represents the degenerate subspace defined in step 1.
[0040] Next, the present invention will... Weights of layer MLP Projected to The edited weights are obtained by filling in the missing space:
[0041] After weight editing, the semantic extractor will be constrained by the degradation direction during feature transformation, which will suppress the feature components corresponding to DegraSpace, while the stable semantic direction orthogonal to it will be preserved.
[0042] After completing the weight editing, the edited semantic extractor is denoted as DSPE. For a given low-resolution input... First, DSPE extracts its semantic features and performs feature mapping through an MLP layer that has undergone degenerate subspace editing, thereby obtaining a semantic representation after suppressing degenerate interference. Subsequently, DSPE outputs a Prompt, which is passed as a conditional input to the UNet of the diffusion model.
[0043] Compared to directly extracting semantics from degraded LR images, DSPE explicitly models the degraded subspace and suppresses its corresponding feature orientation, making semantic cues focus more on stable content information, thereby reducing semantic errors such as misclassification, color shift, and false target generation. Figure 1 (4) Finally, reliable semantic cues, as conditional information of the diffusion model, participate in super-resolution reconstruction together with the uncertainty information generated by subsequent WUG, achieving a synergistic improvement in semantic reliability and local detail recovery capability.
[0044] Wavelet transform is performed on the super-resolution image and the corresponding high-resolution image, and uncertainty supervision information is constructed based on the reconstruction error of different frequency sub-bands to train an uncertainty prediction network.
[0045] To address the issue of varying recovery difficulty across different regions of a real image, discrete wavelet transforms are performed on both the super-resolution image and its corresponding high-resolution image.
[0046] For the input image Perform over-scoring to obtain the initial over-scoring result. .
[0047] Super-resolution results and corresponding real high-resolution images Perform wavelet decomposition separately ( Figure 2 From 1), we get:
[0048]
[0049] Among them, LL represents the low-frequency subband, which mainly contains the overall structure of the image and low-frequency information; LH, HL and HH represent the high-frequency subband, which reflect the texture, edge and local detail information in different directions, respectively.
[0050] Next, the reconstruction errors of the super-resolution results and the high-resolution image in different wavelet subbands were calculated:
[0051] Considering that high-frequency information is of greater importance to texture details and edge structure, different weights are assigned to different frequency sub-bands to obtain the uncertainty supervision target.
[0052] The objective of monitoring uncertainty can be expressed as:
[0053] in, This indicates splicing along the channel dimension. The target for monitoring uncertainty in four channels.
[0054] Next, an uncertainty prediction network is constructed, using the wavelet domain features corresponding to the upsampled low-resolution image as input. The input can be represented as: The corresponding uncertainty features are obtained through the uncertainty prediction network:
[0055] in, This represents an uncertain prediction network.
[0056] An uncertainty prediction network is trained using an uncertainty-supervised objective, enabling it to predict the recovery uncertainty of the corresponding spatial location based on an input low-resolution image. The loss formula is expressed as:
[0057] By inputting the initial super-resolution result of the low-resolution input image into the trained uncertainty network, the four-channel uncertainty response can be directly predicted. Mean aggregation is then performed on the four frequency channels.
[0058] in, This represents the uncertainty response corresponding to the c-th frequency sub-band. This is the final pixel-level uncertainty map. (Higher) The value indicates that the model has higher uncertainty in the recovery of this region, that is, the region is more likely to have reconstruction difficulties such as missing texture and blurred structure.
[0059] Through the training method described above, the uncertainty network can learn to predict the recovery difficulty corresponding to different frequency components from low-resolution input and its initial reconstruction results, thereby establishing a mapping relationship from frequency domain reconstruction error to spatial uncertainty and generating an uncertainty map.
[0060] The robust semantic extractor generates reliable semantic cues, and the uncertainty information generated by the uncertainty prediction network is introduced into the diffusion model UNet. The network features are adaptively adjusted through feature modulation and spatial attention mechanisms to obtain the trained diffusion super-resolution network and generate super-resolution images.
[0061] The low-resolution image undergoes initial super-resolution followed by wavelet decomposition. Figure 3 1), Input uncertain prediction network ( Figure 3(2) In this step, multi-channel uncertainty features are obtained, the features are fused, and they are mapped to the image space to obtain a single-channel pixel-level uncertainty map. .in, and These represent the spatial height and width of the image, respectively. Regions with larger response values in the uncertainty graph indicate that the model has a higher difficulty in recovering that region, typically corresponding to areas with rich texture, distinct edges, or severe degradation.
[0062] The resulting uncertainty map is input into the uncertainty-aware feature modulation module. Considering that the upsampling stage of the diffusion model UNet mainly undertakes the tasks of spatial structure restoration and high-frequency detail generation, uncertainty feature modulation is introduced in the UNet upsampling stage.
[0063] Let the features of the UNet upsampling stage be... First, an uncertainty encoder is used to... Perform weight extraction ( Figure 3 (3) and generate scale modulation parameters. and offset parameters :
[0064] in, This represents the weight extraction module. Subsequently, the UNet features are normalized and affine modulated:
[0065] in, and Let F represent the mean and standard deviation of feature F, respectively.
[0066] Through the aforementioned feature modulation, uncertain information can participate in the feature propagation process of the diffusion model.
[0067] To further enhance the diffusion model's ability to perceive the recovery difficulty of different spatial regions, an uncertainty graph is introduced into the spatial attention computation process. This uncertainty graph is used as an additional spatial prior and fused with the average pooling and max pooling results of the features. Figure 3 (4 in the middle).
[0068] Spatial attention map can be represented as:
[0069] in, This represents the activation function. This represents the convolution operation. and Representing the features The results after performing average pooling and max pooling. This represents a pixel-level uncertainty map.
[0070] Using the obtained spatial attention map, the features modulated by uncertainty are dynamically fused with the original UNet features:
[0071] in, Larger regions will utilize more features modulated by uncertainty, thereby enhancing the feature representation of complex regions; Smaller areas retain more of their original features to avoid over-enhancing structurally stable regions.
[0072] This invention will Weights of layer MLP Projected to In the fill space, after weight editing ( Figure 3 In step 5 of the process, for a given low-resolution image, DSPE outputs a Prompt, while simultaneously incorporating the uncertainty map predicted by WUG into the UNet feature modulation and spatial attention computation process.
[0073] During training, a loss function is used to optimize the diffusion super-resolution network. This loss function is a weighted combination of mean squared error loss, perceptual loss, and generative adversarial loss, expressed as follows:
[0074] in Represents a high-resolution image. Indicates super-resolution image, Indicates the weighting coefficient. Indicates the mean square error loss. Indicates perceived loss. This represents the generative adversarial loss. During the training of the diffusion super-resolution network, reliable semantic cues are input as conditional information into the diffusion model to constrain the content and semantic information of the super-resolution image. Simultaneously, a pixel-level uncertainty map is input into the diffusion model's UNet, and the UNet features are adaptively adjusted through feature modulation and spatial attention mechanisms. This allows the diffusion model to dynamically allocate feature representation capabilities according to the recovery difficulty of different spatial regions, thereby enhancing the recovery of texture, edge, and structural details in high-uncertainty regions while maintaining overall semantic consistency.
[0075] After gradual denoising and feature reconstruction using the diffusion model, the final super-resolution image is obtained.
[0076] During the inference process, only the low-resolution image to be processed needs to be input. Reliable semantic cues and pixel-level uncertainty maps can be obtained by a robust semantic extractor based on degenerate subspace decomposition and an uncertainty prediction network, respectively. The reliable semantic cues and pixel-level uncertainty maps are then input into the trained diffusion super-resolution network to generate the final super-resolution image, without the need for additional manual semantic annotation or other auxiliary information.
[0077] This invention is not limited to the above embodiments, and the following equivalent solutions may also be adopted: 1. Replacement of semantic feature extractors The embodiments of the present invention use a pre-trained RAM model to obtain features of high and low resolution images, but are not limited to a specific visual encoder.
[0078] Visual language models, image encoders, visual Transformers, convolutional neural networks, or other networks capable of extracting visual semantic features can be used as alternatives. As long as the corresponding representations of low-resolution and high-resolution images in the feature space can be obtained, they can be used to construct degradation feature differences.
[0079] 2. Alternatives to the degenerate subspace construction method This invention employs singular value decomposition to obtain the main direction of degradation features. In other embodiments, feature covariance analysis, principal component analysis, feature spectral decomposition, or other low-rank subspace estimation methods can also be used to obtain the degradation direction. The core of all these methods lies in determining the main degradation direction from the difference between low-resolution and high-resolution features and using it to construct a degradation subspace.
[0080] 3. Replacement of frequency decomposition method This invention uses discrete wavelet transform for frequency decomposition, but is not limited to a specific wavelet basis or number of decomposition layers. Other frequency domain transform methods that can separate low-frequency structural information from high-frequency texture information can also be used, such as different types of wavelet transform, discrete cosine transform, Fourier transform, or other frequency domain decomposition methods.
[0081] The key innovation of this invention lies in: 1. A degradation feature difference matrix is established based on the feature differences between high and low resolution images, and the degradation subspace is obtained through singular value decomposition.
[0082] 2. By projecting and editing the weights in the semantic extractor using the orthogonal complement space of the degenerate subspace, the degenerate-related feature directions are suppressed, resulting in a degenerate robust semantic extractor.
[0083] 3. Uncertainty supervision information with frequency difference weights is constructed by utilizing the wavelet domain reconstruction error between the super-resolution results and the high-resolution image, and an uncertainty prediction network is trained to obtain a pixel-level spatial uncertainty map.
[0084] 4. Pixel-level uncertainty maps are introduced into the diffusion model UNet, and the feature responses of different regions are adaptively adjusted through uncertainty-aware feature modulation and spatial attention mechanisms.
[0085] 5. By using degenerate robust semantic cues and spatial uncertainty information together as conditions for the diffusion model, a joint guidance of semantic reliability and region adaptive detail recovery is achieved.
[0086] In summary, this invention proposes a real-image super-resolution method jointly guided by reliable semantics and uncertainty. By explicitly modeling the degradation process and suppressing degradation interference, the semantic consistency of the super-resolution results is improved. Simultaneously, uncertainty guidance enables adaptive feature adjustment for regions with varying recovery difficulty, effectively enhancing the texture detail reconstruction of complex regions. Compared to traditional super-resolution methods that rely solely on pixel reconstruction loss, and existing schemes that focus only on semantic constraints or spatial recovery difficulty guidance, this invention can simultaneously address common semantic errors and detail blurring issues in real-world super-resolution. It achieves superior quantitative metrics and visual effects compared to existing mainstream methods on publicly available benchmark datasets. This method can be applied to image resolution enhancement tasks in multiple fields such as security monitoring, medical imaging, and remote sensing image processing, demonstrating promising application prospects and widespread application value.
[0087] It should be emphasized that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.
Claims
1. A real image super-resolution method jointly guided by reliable semantics and uncertainty, characterized in that, Includes the following steps: Step S1: Perform feature difference analysis on the paired high-resolution and low-resolution images, construct a degradation feature difference matrix, and perform singular value decomposition on the degradation feature difference matrix to obtain the degradation subspace. Step S2: Determine the corresponding orthogonal complement space based on the degenerate subspace, project the weights in the semantic extractor onto the orthogonal complement space, and obtain a robust semantic extractor based on degenerate subspace decomposition. The robust semantic extractor is used to output reliable semantic hints. Step S3: Perform wavelet transform on the super-resolution image and the corresponding high-resolution image, construct uncertainty supervision information based on the reconstruction error of different frequency sub-bands, and train an uncertainty prediction network. The uncertainty prediction network is used to output a pixel-level uncertainty map. Step S4: Input the reliable semantic prompt as conditional information into the diffusion model UNet, introduce the pixel-level uncertainty map into the feature modulation and spatial attention process of the diffusion model UNet, adaptively adjust the network features, and train to obtain the diffusion super-resolution network to generate super-resolution images.
2. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, Step S1 specifically includes: inputting the paired high-resolution image and low-resolution image into the pre-trained visual encoder respectively, and obtaining the corresponding visual features from different network layers; aggregating the visual features in the token dimension of the features, calculating the difference between the low-resolution features and the high-resolution features, and obtaining the degradation feature difference matrix. In step S1, after performing singular value decomposition on the degenerate feature difference matrix, the right singular vectors corresponding to the first k largest singular values are selected to construct the degenerate subspace, where k is a positive integer.
3. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, Step S2 specifically includes: according to Calculate the first The orthogonal complement space corresponding to the degenerate subspace of the layer, where It is the identity matrix. For the first Degenerate subspace of the layer; The first The weights of the MLP in the layer semantic extractor are projected onto the orthogonal complement space to obtain the edited weights. ,in These are the original weights.
4. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S3, discrete wavelet transform is performed on the super-resolution image and the corresponding high-resolution image to obtain their respective low-frequency sub-bands and multiple high-frequency sub-bands; the reconstruction error of the super-resolution image and the high-resolution image in each wavelet sub-band is calculated respectively.
5. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S3, the uncertainty supervision information is obtained by assigning different weights to the reconstruction errors of different frequency sub-bands and then stitching them along the channel dimension, wherein the weight of the high-frequency sub-band is greater than the weight of the low-frequency sub-band.
6. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S3, the uncertainty prediction network takes the wavelet domain features corresponding to the initial super-resolution result of the low-resolution image as input and outputs a multi-channel uncertainty response; the multi-channel uncertainty response is averaged and aggregated to obtain the pixel-level uncertainty map.
7. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S4, the feature modulation process of introducing the pixel-level uncertainty map into the diffusion model UNet specifically includes: in the upsampling stage of UNet, using an uncertainty encoder to extract weights from the pixel-level uncertainty map and generate scale modulation parameters. and offset parameters ;according to The UNet features are normalized and affine modulated, where F represents the features from the UNet upsampling stage. and Let F represent the mean and standard deviation of feature F, respectively.
8. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S4, the process of introducing the pixel-level uncertainty map into the spatial attention of the diffusion model UNet specifically includes: The pixel-level uncertainty map is used as an additional spatial prior and fused with the average pooling and max pooling results of the UNet features. After convolution and activation, a spatial attention map is obtained. Based on the spatial attention map, the uncertainty-modulated features and the original UNet features are dynamically fused. Regions with larger spatial attention values adopt more uncertainty-modulated features, while regions with smaller spatial attention values retain more original UNet features.
9. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, In step S4, the loss function used to train the diffusion super-resolution network is a weighted combination of mean squared error loss, perceptual loss, and generative adversarial loss.
10. The real image super-resolution method jointly guided by reliable semantics and uncertainty according to claim 1, characterized in that, During the inference phase, only the low-resolution image to be processed needs to be input. The robust semantic extractor and the uncertainty prediction network obtain reliable semantic cues and pixel-level uncertainty maps, respectively. The two are then input into the trained diffusion super-resolution network to generate the final super-resolution image.