Power scene image super-resolution method and system based on diffusion model

Abstract

This invention discloses a method and system for super-resolution reconstruction of power scene images based on a diffusion model. The method includes: acquiring a low-resolution image of a power scene; performing noise addition, enhancement, and magnification processing on the image using multiple branch networks based on a diffusion model; generating a noisy image through a first branch network; obtaining high- and low-frequency information by performing a two-dimensional wavelet transform on the image through a second branch network, fusing the processed high- and low-frequency information, and obtaining an enhanced image through an inverse two-dimensional wavelet transform; obtaining a magnified image by performing bicubic interpolation upsampling through a third branch network; encoding and compressing the noisy image, magnified image, and enhanced image; inputting the compressed image into a UNet denoising network finely tuned with LoRA, and gradually removing image noise through a reverse inference process; and decoding the denoised image to obtain the target super-resolution power scene image. This invention can achieve high-quality super-resolution reconstruction of power scene images.

Description

Technical Field

[0001] This invention relates to the field of power scene image processing technology, specifically to a power scene image super-resolution method and system based on a diffusion model. Background Technology

[0002] With the goal of "three types of networks and two grids, world-class standards" being proposed, comprehensive perception of power grid operation information, fault types, and equipment status has become a key aspect of the construction of new power systems. Against this backdrop, online monitoring equipment for power scenarios such as transmission lines, substations, and distribution equipment is being deployed at an accelerated pace. Image perception technology can perform real-time detection of power equipment and provide important data for defect identification, fault early warning, and health status assessment, thus possessing significant application value in the power Internet of Things (IoT).

[0003] However, due to limitations in hardware costs, imaging device manufacturing processes, and deployment conditions, high-resolution detectors typically require smaller pixel sizes or larger chip areas, resulting in higher equipment costs and hindering their widespread application in large-scale power monitoring scenarios. In contrast, while low-resolution imaging devices are less expensive, the images they acquire generally suffer from insufficient resolution, low signal-to-noise ratio, blurred edges, and lack of texture. Especially in complex power scenarios, targets such as conductors, insulators, hardware fittings, and equipment defects often have elongated structures, multi-scale distributions, and weak detail information, making low-quality images prone to affecting subsequent identification and detection results. Therefore, there is a strong engineering application demand for improving the resolution of power scenario images using software methods without significantly increasing hardware costs.

[0004] Single-image super-resolution reconstruction aims to recover a corresponding high-resolution image from a low-resolution image. In recent years, deep learning has driven the development of image super-resolution technology. While traditional convolutional neural network super-resolution methods can improve resolution, they often focus more on optimizing pixel reconstruction errors, leading to over-smoothing of the generated results and difficulty in effectively recovering high-frequency textures and edge details. Generative adversarial network-based super-resolution methods, although improving visual perception quality, suffer from training instability, artifacts, and repetitive textures. Diffusion models, which generate images through progressive noise addition and iterative denoising, offer advantages over traditional methods in texture realism and uncertainty modeling, and are therefore increasingly being applied to image super-resolution tasks.

[0005] However, most existing diffusion super-resolution methods still primarily model images uniformly within the spatial domain, lacking specific processing mechanisms for different frequency components of the image. For power scene images, the low-frequency components mainly characterize equipment outlines, structural layout, and overall brightness distribution, while the high-frequency components contain more key information such as edges, textures, and defect details. If the same processing method is used for high-frequency and low-frequency information, it can easily lead to insufficient detail recovery or unstable structural preservation, making it difficult to balance global structural consistency and local texture clarity.

[0006] Furthermore, power scene images often contain long-range dependencies and complex background interference, such as conductors extending across regions, discrete equipment structures, and a small proportion of fine-grained defect targets. This places higher demands on the global modeling capabilities of the model. Existing convolution-based feature extraction methods have limited receptive fields and are insufficient in modeling long-range dependencies. While some Transformer-based structures can enhance global modeling capabilities, their computational complexity is high, easily leading to significant training and inference overhead in high-resolution image processing scenarios. Therefore, how to improve the modeling ability of low-frequency global structural information while controlling model complexity, reducing subsequent false detections caused by low resolution, and improving model efficiency and generalization performance are also urgent problems to be solved in existing technologies. There is an urgent need for a technical solution suitable for super-resolution reconstruction of power scene images, which can both target and enhance high-frequency detail information during diffusion denoising and effectively model long-range dependencies in low-frequency structures, thereby improving the reconstruction quality of power scene images and the effectiveness of subsequent detection applications. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method and system for super-resolution reconstruction of power scene images based on a diffusion model, so as to achieve high-quality super-resolution reconstruction of power scene images with low hardware cost.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A super-resolution method for power scene images based on a diffusion model includes the following steps: S1, acquire low-resolution images of the power scene; S2, based on a diffusion model, multiple branch networks perform noise addition, enhancement, and magnification processing on the low-resolution image. Specifically, the first branch network performs a forward diffusion noise addition process, progressively adding Gaussian noise to the low-resolution image to generate a noisy image. The second branch network performs a two-dimensional wavelet transform on the low-resolution image to obtain low-frequency and high-frequency information. Global features are extracted from the low-frequency information, and detailed features are restored from the high-frequency information. The processed low-frequency and high-frequency information are then fused and subjected to an inverse two-dimensional wavelet transform to obtain the enhanced image. The third branch network performs bicubic interpolation upsampling on the low-resolution image to obtain the magnified image. S3, input the noisy image, magnified image and enhanced image into the encoder for encoding and compression; S4. The compressed image is input into the UNet denoising network that has been fine-tuned by LoRA, and the image noise is gradually removed through the backward inference process of the diffusion model. S5 inputs the denoised image into the decoder for decoding to obtain the target super-resolution power scene image.

[0009] Furthermore, the UNet denoising network is an improved Swin-UNet, which includes an encoder, a decoder, a frequency domain enhancement module, a skip connection module, and an output layer, wherein: The encoder includes a multi-layer Swing Transformer module and a feature compression module, which are used to downsample the input features layer by layer so that the feature resolution decreases layer by layer, the number of channels increases layer by layer, and multi-scale semantic features are extracted. The decoder includes a multi-layer Swing Transformer module and a feature expansion module, which are used to upsample the features processed by the encoder layer by layer to gradually restore the spatial resolution of the features. The skip connection module and the frequency domain enhancement module are set between the corresponding layers of the encoder and the decoder. They are used to enhance the features output by each downsampling layer of the encoder in the frequency domain and then concatenate them with the input features of the corresponding upsampling layer of the decoder in the channel dimension, so as to fuse the high-resolution low semantic features of the encoder with the low-resolution high semantic features of the decoder. The output layer includes a feature expansion module and a linear mapping module, which are used to perform channel mapping on the final features output by the decoder to obtain denoised output features.

[0010] Furthermore, the frequency domain enhancement module includes a two-dimensional discrete wavelet transform unit, an attention processing unit, a residual update unit, and a two-dimensional inverse discrete wavelet transform unit, wherein: The two-dimensional discrete wavelet transform unit is used to decompose the features output by the encoder into low-frequency components and multiple high-frequency components. The attention processing unit is used to add multiple high-frequency components to obtain a key vector K and a value vector V, and to use each high-frequency component as a query vector Q, and to perform self-attention calculation on each high-frequency component. The residual update unit is used to perform residual connection between each high-frequency component calculated by self-attention and the corresponding input high-frequency component to obtain the updated high-frequency component. The two-dimensional inverse discrete wavelet transform unit is used to fuse the updated high-frequency components with the low-frequency components and then perform an inverse transform to obtain the enhanced features.

[0011] Furthermore, the LoRA fine-tuning is used to freeze the original weight matrix of the layer to be fine-tuned in the pre-trained model. A low-rank adapter is set only in the bypass path of the layer to be fine-tuned, so as to simulate the change in weights through the adapter of low-rank decomposition. The forward propagation of the layer to be fine-tuned satisfies the following equation: ; In the above formula, h represents the output feature of the layer to be fine-tuned, and x represents the input feature of the layer to be fine-tuned. and This represents the low-rank matrix used for low-rank decomposition and training. This refers to the original weight matrix of the fully connected layer or attention layer in the pre-trained model. This refers to the weight changes introduced through LoRA fine-tuning. d represents the rank of the low-rank adapter, d represents the output feature dimension, and k represents the input feature dimension.

[0012] Furthermore, the second branch network is a frequency-domain Mamba network, which includes a two-dimensional wavelet transform module, a low-frequency processing module, a high-frequency processing module, and a two-dimensional wavelet inverse transform module, wherein: The two-dimensional wavelet transform module is used to perform two-dimensional wavelet transform decomposition on the input power scene image to obtain low-frequency information and high-frequency information. The high-frequency information includes horizontal features, vertical features and diagonal features. The low-frequency processing module includes a convolutional unit, a GELU activation unit, a Mamba module, and a channel-adjusting convolutional unit. The convolutional unit is used to perform preliminary feature extraction on low-frequency information. The GELU activation unit is used to perform nonlinear transformation on the preliminary extracted features. The Mamba module is used to perform global feature extraction on the low-frequency features after nonlinear transformation. The channel-adjusting convolutional unit is used to adjust the number of channels of the low-frequency features. The high-frequency processing module is used to concatenate the horizontal, vertical and diagonal features according to the channel dimension, and to extract detailed features and adjust channels through multi-scale convolution; The two-dimensional wavelet inverse transform module is used to merge the processed low-frequency features with high-frequency features, and obtain the frequency-domain enhanced power scene image through two-dimensional wavelet inverse transform, which serves as one of the inputs to the UNet denoising network.

[0013] Furthermore, the Mamba module includes a batch normalization unit, a Mamba model unit, a multilayer perceptron unit, and a residual connection unit, wherein: The batch normalization unit is used to standardize the input data; The Mamba model unit is used to scan and model the standardized input data and extract low-frequency spatial features. The multilayer perceptron unit includes an input layer, a hidden layer, and an output layer, and is used to integrate the spatial features extracted by the Mamba model unit; The residual connection unit is used to add the input of each layer to the corresponding output and then use the result as the output of that layer. The Mamba model unit processes the input sequence using the following steps: The input sequence is normalized using root mean square (RMS); the normalized input sequence is linearly projected to expand the feature channels; the linearly projected features are processed by one-dimensional convolution and input into a state-space model for sequence modeling; simultaneously, a residual branch is introduced to process the input sequence using linear projection and activation functions; the features processed by the state-space model are fused with the features output by the residual branch; the fused features are added to the input sequence using residual summation; the features after residual summation are normalized and linearly regressed to obtain the output sequence.

[0014] Further, in step S2, the forward diffusion noise addition process uses a Markov chain to gradually add Gaussian noise to the high-resolution image from the real data distribution over T iterations. The forward diffusion noise addition process satisfies the following equation: ; In the above formula, , The images are the noise images at time steps t and (t-1) respectively. It is a time series. Let be the noise scheduling hyperparameter at time step t. The Gaussian noise generated at time step t. This indicates that it follows a standard Gaussian distribution.

[0015] Further, in step S4, the reverse reasoning process reverses the forward diffusion noise addition process by following a Markov chain joint distribution to recover the image from noise samples with a standard Gaussian distribution, wherein the noise samples... Predict the previous step noise sample The conditional probability follows the following Gaussian distribution: ; In the above formula, This indicates that the hyperparameters for scheduling at time step t are determined by noise. A defined fidelity factor, This represents the cumulative product of the fidelity coefficients up to the t-th time step; The reverse denoising process satisfies the following formula:

[0016] ; In the above formula, Z represents the Gaussian noise sampled during the reverse denoising process. This indicates that the UNet denoising network denoises the current noisy image. The noise component predicted at time step t.

[0017] A diffusion-based super-resolution system for power scene images includes an interconnected microprocessor and a memory, the microprocessor being programmed or configured to execute steps of a diffusion-based super-resolution method for power scene images.

[0018] A computer-readable storage medium storing a computer program / instructions programmed or configured to perform steps of a diffusion-based power scene image super-resolution method via a processor.

[0019] Compared with the prior art, the advantages of the present invention are as follows: This invention performs frequency domain enhancement processing on low-resolution power scene images through a second branch network. First, a two-dimensional wavelet transform is used to decompose the image into low-frequency and high-frequency information, which are then processed separately. The low-frequency information is used to extract global structural features, while the high-frequency information is used to restore edge and texture details. This improves the enhanced image's ability to constrain the subsequent diffusion denoising process, reducing structural distortion and detail loss during reconstruction. This invention employs a LoRA-tuned UNet denoising network for backpropagation, enabling it to adapt to power scene image features with lower training costs. It progressively predicts and removes noise, maintaining consistency in equipment outlines and scene structure while more effectively restoring details such as defect areas and edge textures. This improves the clarity, realism, and applicability of the super-resolution reconstruction results to power inspection scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart of a power scene image super-resolution method based on a diffusion model, according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the network structure of the diffusion model super-resolution system corresponding to the method in the embodiments of the present invention.

[0022] Figure 3 This is a schematic diagram of the network structure of the Mamba module used for low-frequency feature extraction in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the network structure of the Mamba model in the Mamba module of this invention.

[0024] Figure 5 This is a schematic diagram of the structure of the UNet denoising network in an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram of the network structure of the frequency domain enhancement module in an embodiment of the present invention. Detailed Implementation

[0026] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0027] like Figure 1 and Figure 2 As shown, the power scene image super-resolution method based on the diffusion model in this embodiment includes the following steps: S1, acquire low-resolution images of the power scene; S2, based on a diffusion model, multiple branch networks perform noise addition, enhancement, and magnification processing on the low-resolution image. Specifically, the first branch network performs a forward diffusion noise addition process, progressively adding Gaussian noise to the low-resolution image to generate a noisy image. The second branch network performs a two-dimensional wavelet transform on the low-resolution image to obtain low-frequency and high-frequency information. Global features are extracted from the low-frequency information, and detailed features are restored from the high-frequency information. The processed low-frequency and high-frequency information are then fused and subjected to an inverse two-dimensional wavelet transform to obtain the enhanced image. The third branch network performs bicubic interpolation upsampling on the low-resolution image to obtain the magnified image. S3 inputs the noisy image, magnified image, and enhanced image into the encoder for encoding and compression; S4. The compressed image is input into the UNet denoising network that has been fine-tuned by LoRA, and the image noise is gradually removed through the backward inference process of the diffusion model. S5 inputs the denoised image into the decoder for decoding to obtain the target super-resolution power scene image.

[0028] In specific application embodiments, such as Figure 2As shown, the diffusion model processes the low-resolution input power scene image by designing three branch networks. The branch networks respectively perform noise addition, image enhancement, and residual connection introduction on the input image, as detailed below: The first branch network includes a forward diffusion noise addition process, which gradually adds Gaussian white noise to the input power scene image through a diffusion model until the input image is completely transformed into Gaussian white noise; The second branch network includes a frequency domain Mamba network, comprising a two-dimensional wavelet transform module, a Mamba module, a multi-scale convolution module, and a two-dimensional wavelet inverse transform module. It decomposes the input power scene image through two-dimensional wavelet transform to obtain low-frequency and high-frequency information. The low-frequency information is processed by the Mamba module to scan global information, while the high-frequency information is processed by the multi-scale convolution module to recover detail information. The high-frequency and low-frequency information are then merged and processed by the two-dimensional wavelet inverse transform module to obtain the enhanced image. The third branch network includes adding low-resolution images via skip connections and using bicubic interpolation to upscale the low-resolution images; The noisy image is encoded by a sine and cosine encoder. The bicubic interpolated image, the noisy image, the encoded position information, and the wavelet-enhanced image are combined and input into the VQGAN encoder to compress the original image by a factor of four in the spatial dimension. The compressed image is then fed into the UNet denoising network, which has been fine-tuned by LoRA. The image noise is gradually removed through the backward inference process of the diffusion model. Finally, the image is decoded to obtain the super-resolution image.

[0029] Specifically, this embodiment utilizes a diffusion model for super-resolution of power scene images. During the diffusion training phase, a UNet denoising network is trained by calculating the MSE loss between predicted noise and actual noise. During the diffusion inference phase, the trained UNet network is used to gradually denoise the image, starting with pure noise and guided by a low-resolution image, to generate the corresponding high-resolution image. The low-resolution image specifies the direction of model generation. The frequency domain Mamba module uses the enhanced low-resolution image as a condition to guide the model in generating the corresponding high-resolution image. The frequency domain Mamba module enhances the global features and local details of the image, enabling the diffusion model to recover richer details and improve image quality. This embodiment addresses the problem of low resolution power scene images and the relatively smooth image generation of traditional neural network super-resolution methods, achieving better super-resolution results.

[0030] In this embodiment, in step S2, the forward diffusion noise addition process uses a Markov chain to gradually add Gaussian noise to the high-resolution image from the real data distribution over T iterations. The forward diffusion noise addition process satisfies the following equation: (1) In the above formula, , The images are the noise images at time steps t and (t-1) respectively. It is a time series, and the range of values ​​is... , Let be the noise scheduling hyperparameter at time step t. The Gaussian noise generated at time step t. This indicates that it follows a standard Gaussian distribution.

[0031] In step S4, the reverse reasoning process reverses the forward diffusion noise addition process by following the joint distribution of a Markov chain to recover the image from noise samples with a standard Gaussian distribution, wherein the noise samples... Predict the previous step noise sample The conditional probability follows the following Gaussian distribution: (2) In the above formula, This indicates that the hyperparameters for scheduling at time step t are determined by noise. A defined fidelity factor, This represents the cumulative product of the fidelity coefficients up to the t-th time step; The reverse denoising process satisfies the following formula:

[0032] (3) In the above formula, Z represents the Gaussian noise sampled during the reverse denoising process. This indicates that the UNet denoising network denoises the current noisy image. The noise component predicted at time step t.

[0033] In specific application embodiments, the iterative process of the diffusion model includes forward noise addition and backward inference. Forward noise addition involves continuously adding Gaussian noise to transform the image into a noisy one, and then reversing this noise addition process to learn and recover the data. The forward noise addition process follows a Markov chain, which gradually adds Gaussian noise to the high-resolution image over T iterations from the real data distribution. The noise addition process is defined as shown in formula (1), where the noise addition process can be simplified as:

[0034] (4) In the above formula, Represents the original resolution image. Let represent the fidelity coefficient at the k-th time step, and Z represent random noise following a standard Gaussian distribution. This formula can be used to obtain the original image... One step becomes noise This reduces the time it takes for the original image to be converted into noise.

[0035] The diffusion model's reverse inference reverses the above process by following the joint distribution of a Markov chain, recovering the image from noise samples with a standard Gaussian distribution. Predict the previous step noise sample The conditional probability follows a Gaussian distribution as shown in formula (2), and the reverse denoising process satisfies formula (3). A clear image can be gradually recovered from a noisy image using formula (3).

[0036] It is understood that the power scene image super-resolution method based on diffusion model in this embodiment introduces diffusion model into the super-resolution reconstruction of power scene images. Through the generation method of progressive denoising, it can learn the probability distribution of specific features of power equipment, lines and other features from a large amount of data, and generate high-frequency details with realistic texture and in accordance with physical laws.

[0037] In this embodiment, as Figure 2 As shown, the second branch network is a frequency domain Mamba network. The frequency domain Mamba network includes a two-dimensional wavelet transform module, a low-frequency processing module, a high-frequency processing module, and a two-dimensional wavelet inverse transform module, wherein: The two-dimensional wavelet transform module is used to perform two-dimensional wavelet transform decomposition on the input power scene image to obtain low-frequency and high-frequency information. The high-frequency information includes horizontal features, vertical features and diagonal features. The low-frequency processing module includes a convolutional unit, a GELU activation unit, a Mamba module, and a channel-adjusting convolutional unit. The convolutional unit is used to perform preliminary feature extraction on the low-frequency information, the GELU activation unit is used to perform nonlinear transformation on the preliminary extracted features, the Mamba module is used to perform global feature extraction on the low-frequency features after nonlinear transformation, and the channel-adjusting convolutional unit is used to adjust the number of channels of the low-frequency features. The high-frequency processing module is used to concatenate horizontal, vertical, and diagonal features according to channel dimensions, and to extract detailed features and adjust channels through multi-scale convolution; The two-dimensional wavelet inverse transform module is used to merge the processed low-frequency features with high-frequency features, and then obtain the frequency-domain enhanced power scene image through two-dimensional wavelet inverse transform, which serves as one of the inputs to the UNet denoising network.

[0038] like Figure 3 As shown, the Mamba module includes a batch normalization unit, a Mamba model unit, a multilayer perceptron unit, and a residual connection unit, wherein: The batch normalization unit is used to standardize the input data; The Mamba model unit is used to scan and model the standardized input data and extract low-frequency spatial features; The multilayer perceptron unit includes an input layer, a hidden layer, and an output layer, which are used to integrate the spatial features extracted by the Mamba model unit; The residual connection unit is used to add the input of each layer to the corresponding output and then use the result as the output of that layer. Among them, such as Figure 4 As shown, the Mamba model unit processes the input sequence through the following steps: The input sequence is normalized using root mean square (RMS); the normalized input sequence is then linearly projected to expand the feature channels; the linearly projected features are processed by one-dimensional convolution and input into a state-space model for sequence modeling; a residual branch is introduced to process the input sequence using linear projection and activation functions; the features processed by the state-space model are fused with the features output by the residual branch; the fused features are then summed with the residuals of the input sequence; the summed residuals are then normalized and linearly regressed to obtain the output sequence.

[0039] In specific application embodiments, such as Figure 2 As shown, the diffusion network includes a wavelet transform module. The input power scene image is first processed by the wavelet transform module to obtain low-frequency information and high-frequency information in three directions: horizontal, vertical, and diagonal features. The low-frequency and high-frequency features are then processed by different modules and merged before being processed by the inverse wavelet transform module to obtain the frequency-domain enhanced image. Wavelet transform provides a new approach to diffusion super-resolution because the conversion from the spatial domain to the wavelet domain is lossless and offers significant advantages, as the spatial size of the image can be reduced to one-quarter of its original size, allowing for faster diffusion during the training and inference phases. Specifically, the Haar wavelet transform function can be used. Haar wavelets are composed of simple rectangular functions, requiring only simple addition and subtraction operations for calculation, without complex numerical calculations, resulting in fast computation speed. Haar wavelet functions are orthogonal, avoiding information redundancy. Haar wavelet features have the ability to encode visual information such as image edges and lines, and can extract global and detailed information from the image.

[0040] Frequency domain Mamba networks also include The convolutional module and GELU activation function decompose the input power scene image into high- and low-frequency information through two-dimensional wavelet transform. The high-frequency information includes horizontal, vertical, and diagonal features. The low-frequency information is then processed... Convolutional processing initially extracts features, and an activation function is added to enhance feature nonlinearity. After initial processing, the Mamba module extracts global features, and finally, the convolutional module adjusts the number of channels in the low-frequency region. High-frequency information in the horizontal, vertical, and diagonal directions is concatenated according to channel dimensions, and then a multi-scale convolutional module restores the image's detailed information and adjusts the number of channels in the high-frequency region. The processed high-frequency and low-frequency features are merged and then subjected to a two-dimensional wavelet inverse transform to obtain a frequency-enhanced power scene image, which is then embedded into the diffusion model denoising conditional network.

[0041] The core idea of ​​the Mamba model is to leverage a selective mechanism to achieve more efficient and flexible sequence modeling, compressing long sequence data into a more compact representation, thereby more effectively capturing long-distance dependencies. Its Selective State-Space Model (SSM) allows parameters to be dynamically adjusted based on the input, enabling the model to selectively propagate or forget information based on the current data, better handling discrete and information-intensive data. While conventional SSM models have achieved sequence context capture in linear time complexity, they are constrained by static parameterization and cannot perform content-based reasoning. To address this issue, the improved SSM (i.e., Mamba) selectively propagates or forgets the length information of the current sequence by simply setting the parameters of the SSM as a function of the input. In the Mamba model, parameters are computed directly from the input sequence, thus achieving sequence-aware parameterization.

[0042] like Figure 3 As shown, the Mamba module includes batch normalization, the Mamba model, a multilayer perceptron, and residual connections. Batch normalization standardizes the input data, making the training of intermediate layer models more stable. The Mamba model can scan the data, effectively capturing the dependencies between distant pixels in the image and scanning and recovering low-frequency information. The multilayer perceptron includes an input layer, hidden layers, an output layer, and activation functions, integrating and refining the spatial features extracted by the Mamba model to achieve feature transformation and dimensionality recovery, ultimately generating prediction results. Residual connections add the input of a layer to its output and use it as the final output of that layer, maintaining data flow, making the network easier to train, and maintaining good performance in deep networks, effectively avoiding the problems of gradient vanishing and gradient exploding.

[0043] like Figure 3 As shown, let This represents the feature input to the Mamba module. After passing through the normalization layer, the features enter the Mamba network. The initial features are stacked through skip connections, i.e. The output is then passed through a normalization layer before being input into a multilayer perceptron (MLP), with skip connections used to prevent feature omission. The final feature output can be represented as follows: (5) in Representing the Mamba model, Represents a multilayer perceptron. Represents the normalization layer. and These represent the input and output of the Mamba module, respectively. and This represents two parallel paths.

[0044] like Figure 4 As shown, the Mamba model includes root mean square normalization, a projection module, an activation function, a state-space model, and a linear regression layer. The Mamba model first processes the input sequence through a normalization layer. Normalization is performed, and then a linear projection is applied to the normalized sequence to expand the feature channels. The projection layer is further processed by one-dimensional convolution, and the state-space model SSM is used for computation. Simultaneously, another branch is introduced through residual connections, which, after passing through a linear projection layer and an activation function, yields... Then and Combined Processed by the projection layer Input sequence connected to residuals The summation is then passed through a normalization layer and a linear regression layer to finally obtain the output sequence. The Mamba model utilizes a selective mechanism to achieve more efficient and flexible sequence modeling. It captures long-distance dependencies more effectively by compressing long sequence data into a more compact representation. Its selective state-space model (SSMs) allows parameters to be dynamically adjusted based on the input, enabling the model to selectively pass or forget information based on the current data, thus better handling discrete and information-intensive data.

[0045] It is understood that the diffusion-based super-resolution method for power scene images in this embodiment proposes a wavelet transform module to decompose the high-frequency and low-frequency information of the image in order to better extract image features. Differential feature extraction strategies are adopted for their different characteristics, achieving efficient fusion of local and global frequency domain features, fully utilizing frequency domain information, significantly improving computational efficiency, and effectively enhancing super-resolution quality. Furthermore, the addition of the Mamba module when extracting low-frequency information allows for performance comparable to or even better than the Transformer with a smaller model size, reducing the number of model parameters and computational complexity, thereby lowering training and inference costs and improving model efficiency.

[0046] In this embodiment, as Figure 5 As shown, the UNet denoising network is an improved version of Swin-UNet. The UNet denoising network includes an encoder, a decoder, a frequency domain enhancement module, a skip connection module, and an output layer, wherein: The encoder includes a multi-layer Swing Transformer module and a feature compression module, which are used to downsample the input features layer by layer so that the feature resolution decreases layer by layer, the number of channels increases layer by layer, and multi-scale semantic features are extracted. The decoder includes a multi-layer Swing Transformer module and a feature expansion module, which are used to upsample the features processed by the encoder layer by layer to gradually restore the spatial resolution of the features. The skip connection module and the frequency domain enhancement module are set between the corresponding layers of the encoder and the decoder. They are used to enhance the features output by each downsampling layer of the encoder in the frequency domain and then concatenate them with the input features of the corresponding upsampling layer of the decoder in the channel dimension, so as to fuse the high-resolution low semantic features of the encoder with the low-resolution high semantic features of the decoder. The output layer includes a feature expansion module and a linear mapping module, which are used to perform channel mapping on the final features output by the decoder to obtain denoised output features.

[0047] like Figure 6 As shown, the frequency domain enhancement module includes a two-dimensional discrete wavelet transform unit, an attention processing unit, a residual update unit, and a two-dimensional inverse discrete wavelet transform unit, wherein: The two-dimensional discrete wavelet transform unit is used to decompose the features of the encoder output into low-frequency components and multiple high-frequency components. The attention processing unit is used to add multiple high-frequency components to obtain the key vector K and the value vector V, and to use each high-frequency component as a query vector Q, and to perform self-attention calculation on each high-frequency component. The residual update unit is used to perform residual connection between each high-frequency component calculated by self-attention and the corresponding input high-frequency component to obtain the updated high-frequency component. The two-dimensional inverse discrete wavelet transform unit is used to fuse the updated high-frequency components with the low-frequency components and then perform an inverse transform to obtain the enhanced features.

[0048] In specific application embodiments, such as Figure 5As shown, the UNet network is an improved Swin-UNet, a symmetric encoder-decoder structure that replaces the CNN with a Swin Transformer to build an end-to-end pure Transformer model. It also incorporates a frequency domain enhancement module to improve information interaction between the encoder and decoder. Its core modules include: encoder, decoder, frequency domain enhancement module, and skip connections. The encoder consists of multiple Swin Transformer modules and a feature compression module, halving the resolution and doubling the number of channels, progressively reducing feature resolution and enhancing semantics layer by layer, gradually capturing global dependencies through the Transformer module. The decoder progressively restores spatial resolution and locates target details, including multiple Swin Transformer modules and a feature expansion module, restoring to the original resolution. The skip connections and frequency domain enhancement module concatenate the feature maps of corresponding layers in the encoder and decoder, solving the spatial information loss problem caused by downsampling by concatenating the output features of each downsampled layer to the input of the corresponding upsampled layer, thus fusing the high-resolution, low-semantic features of the encoder with the low-resolution, high-semantic features of the decoder. The output layer includes a feature expansion module and a linear mapping module, mapping the number of channels in the final feature map to complete the network output.

[0049] like Figure 6 As shown, the frequency domain enhancement module includes a two-dimensional wavelet transform and an attention network. The encoder features are decomposed into low-frequency components (LL) and high-frequency components (LH, HL, HH) through a two-dimensional discrete wavelet transform. The three high-frequency components are added together to obtain the attention values ​​K and V, and the Q value is the value of each component. Self-attention operation is performed on each high-frequency component (LH, HL, HH), and then each high-frequency component is updated through residual connections. Finally, the updated high-frequency components are combined with the low-frequency components and subjected to a two-dimensional inverse discrete wavelet transform, so that the encoder features are enhanced in the wavelet domain, enhancing the high-frequency details (edges, textures) of the image without affecting the low-frequency structure, improving the information flow between the encoder and decoder, and avoiding the loss of details.

[0050] In this embodiment, LoRA fine-tuning is used to freeze the original weight matrix of the layer to be fine-tuned in the pre-trained model. A low-rank adapter is set only in the bypass path of the layer to be fine-tuned, so as to simulate the change in weights through the adapter of low-rank decomposition. The forward propagation of the layer to be fine-tuned satisfies the following equation: (6) In the above formula, h represents the output feature of the layer to be fine-tuned, and x represents the input feature of the layer to be fine-tuned. and Let represent the low-rank matrix used for low-rank decomposition and participating in training. It is usually initialized using a random Gaussian distribution. Initialize as a zero matrix. This refers to the original weight matrix of the fully connected layer or attention layer in the pre-trained model. Frozen and not participating in gradient updates. This refers to the weight changes introduced through LoRA fine-tuning. Describe the rank of the low-rank adapter. r is a key hyperparameter that determines the adapter's capacity and number of parameters, d represents the output feature dimension, and k represents the input feature dimension.

[0051] In specific application examples, LoRA fine-tuning learns the knowledge of the fine-tuning task by freezing the original weights of the pre-trained model and adding only a tiny, trainable "low-rank adapter" layer in the side path. It consists of a pre-trained base model, a LoRA adapter module, a training framework, and hyperparameters. For the weight matrix of a fully connected layer or attention layer in a pre-trained model... Its forward propagation calculation is as follows The change in weights is simulated using an adapter based on low-rank decomposition. As shown in Equation (6), by freezing the original weights, the general knowledge learned during pre-training is preserved, while the low-rank structure is equivalent to regularization, reducing the risk of overfitting and achieving the transfer from natural images to power images with a smaller training cost.

[0052] It is understood that the super-resolution method for power scene images based on the diffusion model in this embodiment uses LoRA fine-tuning when loading pre-trained weights, which can achieve the transfer from natural images to power images with a small training cost and reduce the risk of overfitting. At the same time, a frequency domain enhancement module is added to the Swin-UNet network. The Swin Transformer module captures the long-distance dependence of the image, while the frequency domain enhancement module captures local details and spatial frequency features, achieving complementarity.

[0053] It should be noted that the super-resolution method in this embodiment adopts a diffusion model, in which the low-frequency feature extraction Mamba network corresponds to... Figure 3 The Mamba model corresponds to Figure 4 UNet denoising network corresponds to Figure 5In addition, encoders with other structures can be used as needed. The VQGAN encoder in this embodiment was pre-trained on other natural images. During training, to improve training efficiency and ensure the stability of the results, the VQGAN encoder in this embodiment was kept frozen. The UNet network in this embodiment uses 4x super-resolution pre-trained weights from ResSharp, achieving the conversion between high-resolution and low-resolution images by shifting the residual between them. It can recover a clear image in four inference steps, greatly reducing inference time and improving the model's speed and accuracy.

[0054] The present invention further provides a power scene image super-resolution system based on a diffusion model, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the steps of a power scene image super-resolution method based on a diffusion model.

[0055] The present invention further provides a computer-readable storage medium storing a computer program / instructions, the computer program / instructions being programmed or configured to execute, via a processor, the steps of a diffusion-based power scene image super-resolution method.

[0056] Compared with the prior art, the present invention has the following beneficial effects: This invention employs a diffusion model to achieve super-resolution of power scene images, while introducing a frequency domain decomposition module and a Mamba module. The power scene image super-resolution network includes a wavelet frequency processing module, a Mamba module, a multi-scale convolution module, LoRA fine-tuning, a UNet denoising network, a frequency domain enhancement module, a sine / cosine encoder, and a VQGAN encoder. The wavelet frequency module decomposes the high- and low-frequency features of the image, obtaining high- and low-frequency features from the power scene image. Low-frequency features are processed through convolution and activation functions and input to the Mamba module. After normalization in the Mamba module, they are sent to the Mamba model for scanning and extraction of low-frequency features. After normalization, the output is processed by a multilayer perceptron. High-frequency information in the horizontal, vertical, and diagonal directions is processed by initial convolution and then input to the multi-scale convolution module for processing and recovery of high-frequency features. The high-frequency and low-frequency features are combined and then recovered from the frequency domain to the time domain via the wavelet inverse transform module. The sine and cosine encoders introduce sine and cosine functions to encode the noisy image generated by the diffusion model, facilitating the UNet network to identify the noise added at each step. The VQGAN encoder compresses the spatial dimension and aggregates information by gradually reducing the spatial size of the feature map and increasing the number of channels, which is used to compress the image. The pre-trained UNet network, fine-tuned by LoRA, is loaded. The Transformer module in the encoder and decoder captures the long-distance dependence of the image, and the frequency domain enhancement module captures local details and spatial frequency features, achieving complementarity, which is used to predict and denoise the encoded and compressed noisy image. The image enhanced by the wavelet frequency module is conditionally embedded into the UNet network. The features processed by the UNet network are decoded to obtain the final super-resolution image. In terms of image super-resolution, by introducing a diffusion model, the super-resolution image has a finer granularity, which is convenient for subsequent detection tasks. By separating high-frequency and low-frequency information and using different processing methods for high-frequency and low-frequency information, it is possible to recover the global features and details of the image. The enhanced image is conditionally embedded into the diffusion model, which specifies the direction of image recovery for the model and facilitates the prediction and removal of noise.

[0057] The system and medium of the present invention, corresponding to the methods described above, also have the advantages described above.

[0058] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0059] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A super-resolution method for power scene images based on a diffusion model, characterized in that, Including the following steps: S1, acquire low-resolution images of the power scene; S2, based on multiple branch networks of the diffusion model, performs noise addition, enhancement, and magnification processing on low-resolution images respectively; The process involves: 1) Forward diffusion noise addition, where Gaussian noise is gradually added to the low-resolution image to generate a noisy image; 2) Two-dimensional wavelet transform, where the low-resolution image is processed by a second branch network to obtain low-frequency and high-frequency information; 3) Global features are extracted from the low-frequency information and detailed features are recovered from the high-frequency information; 4) The processed low-frequency and high-frequency information are fused and then subjected to inverse two-dimensional wavelet transform to obtain an enhanced image; and 5) Bicubic interpolation upsampling, where the low-resolution image is processed by a third branch network to obtain a magnified image. S3, input the noisy image, magnified image and enhanced image into the encoder for encoding and compression; S4. The compressed image is input into the UNet denoising network that has been fine-tuned by LoRA, and the image noise is gradually removed through the backward inference process of the diffusion model. S5 inputs the denoised image into the decoder for decoding to obtain the target super-resolution power scene image.

2. The super-resolution method for power scene images based on a diffusion model according to claim 1, characterized in that, The UNet denoising network is an improved Swin-UNet, which includes an encoder, a decoder, a frequency domain enhancement module, a skip connection module, and an output layer, wherein: The encoder includes a multi-layer Swing Transformer module and a feature compression module, which are used to downsample the input features layer by layer so that the feature resolution decreases layer by layer, the number of channels increases layer by layer, and multi-scale semantic features are extracted. The decoder includes a multi-layer Swing Transformer module and a feature expansion module, which are used to upsample the features processed by the encoder layer by layer to gradually restore the spatial resolution of the features. The skip connection module and the frequency domain enhancement module are set between the corresponding layers of the encoder and the decoder. They are used to enhance the features output by each downsampling layer of the encoder in the frequency domain and then concatenate them with the input features of the corresponding upsampling layer of the decoder in the channel dimension, so as to fuse the high-resolution low semantic features of the encoder with the low-resolution high semantic features of the decoder. The output layer includes a feature expansion module and a linear mapping module, which are used to perform channel mapping on the final features output by the decoder to obtain denoised output features.

3. The super-resolution method for power scene images based on a diffusion model according to claim 2, characterized in that, The frequency domain enhancement module includes a two-dimensional discrete wavelet transform unit, an attention processing unit, a residual update unit, and a two-dimensional inverse discrete wavelet transform unit, wherein: The two-dimensional discrete wavelet transform unit is used to decompose the features output by the encoder into low-frequency components and multiple high-frequency components. The attention processing unit is used to add multiple high-frequency components to obtain a key vector K and a value vector V, and to use each high-frequency component as a query vector Q, and to perform self-attention calculation on each high-frequency component. The residual update unit is used to perform residual connection between each high-frequency component calculated by self-attention and the corresponding input high-frequency component to obtain the updated high-frequency component. The two-dimensional inverse discrete wavelet transform unit is used to fuse the updated high-frequency components with the low-frequency components and then perform an inverse transform to obtain the enhanced features.

4. The super-resolution method for power scene images based on a diffusion model according to claim 1, characterized in that, The LoRA fine-tuning is used to freeze the original weight matrix of the layer to be fine-tuned in the pre-trained model. A low-rank adapter is set only in the bypass path of the layer to be fine-tuned, so as to simulate the change in weights through the adapter of low-rank decomposition. The forward propagation of the layer to be fine-tuned satisfies the following equation: ； In the above formula, h represents the output feature of the layer to be fine-tuned, and x represents the input feature of the layer to be fine-tuned. and This represents the low-rank matrix used for low-rank decomposition and training. This refers to the original weight matrix of the fully connected layer or attention layer in the pre-trained model. This refers to the weight changes introduced through LoRA fine-tuning. d represents the rank of the low-rank adapter, d represents the output feature dimension, and k represents the input feature dimension.

5. The super-resolution method for power scene images based on a diffusion model according to claim 1, characterized in that, The second branch network is a frequency-domain Mamba network, which includes a two-dimensional wavelet transform module, a low-frequency processing module, a high-frequency processing module, and a two-dimensional wavelet inverse transform module, wherein: The two-dimensional wavelet transform module is used to perform two-dimensional wavelet transform decomposition on the input power scene image to obtain low-frequency information and high-frequency information. The high-frequency information includes horizontal features, vertical features and diagonal features. The low-frequency processing module includes a convolutional unit, a GELU activation unit, a Mamba module, and a channel-adjusting convolutional unit. The convolutional unit is used to perform preliminary feature extraction on low-frequency information. The GELU activation unit is used to perform nonlinear transformation on the preliminary extracted features. The Mamba module is used to perform global feature extraction on the low-frequency features after nonlinear transformation. The channel-adjusting convolutional unit is used to adjust the number of channels of the low-frequency features. The high-frequency processing module is used to concatenate the horizontal, vertical and diagonal features according to the channel dimension, and to extract detailed features and adjust channels through multi-scale convolution; The two-dimensional wavelet inverse transform module is used to merge the processed low-frequency features with high-frequency features, and obtain the frequency-domain enhanced power scene image through two-dimensional wavelet inverse transform, which serves as one of the inputs to the UNet denoising network.

6. The super-resolution method for power scene images based on a diffusion model according to claim 5, characterized in that, The Mamba module includes a batch normalization unit, a Mamba model unit, a multilayer perceptron unit, and a residual connection unit, wherein: The batch normalization unit is used to standardize the input data; The Mamba model unit is used to scan and model the standardized input data and extract low-frequency spatial features. The multilayer perceptron unit includes an input layer, a hidden layer, and an output layer, and is used to integrate the spatial features extracted by the Mamba model unit; The residual connection unit is used to add the input of each layer to the corresponding output and then use the result as the output of that layer. The Mamba model unit processes the input sequence using the following steps: The input sequence is normalized using root mean square (RMS); the normalized input sequence is linearly projected to expand the feature channels; the linearly projected features are processed by one-dimensional convolution and input into a state-space model for sequence modeling; simultaneously, a residual branch is introduced to process the input sequence using linear projection and activation functions; the features processed by the state-space model are fused with the features output by the residual branch; the fused features are added to the input sequence using residual summation; the features after residual summation are normalized and linearly regressed to obtain the output sequence.

7. The super-resolution method for power scene images based on a diffusion model according to claim 1, characterized in that, In step S2, the forward diffusion noise addition process uses a Markov chain to gradually add Gaussian noise to the high-resolution image over T iterations from the real data distribution. The forward diffusion noise addition process satisfies the following equation: ； In the above formula, , The images are the noise images at time steps t and (t-1) respectively. It is a time series. Let be the noise scheduling hyperparameter at time step t. The Gaussian noise generated at time step t. This indicates that it follows a standard Gaussian distribution.

8. The super-resolution method for power scene images based on a diffusion model according to claim 1, characterized in that, In step S4, the reverse reasoning process reverses the forward diffusion noise addition process by following the joint distribution of a Markov chain to recover the image from noise samples with a standard Gaussian distribution, wherein the noise samples... Predict the previous step noise sample The conditional probability follows the following Gaussian distribution: ； In the above formula, This indicates that the hyperparameters for scheduling at time step t are determined by noise. A defined fidelity factor, This represents the cumulative product of the fidelity coefficients up to the t-th time step; The reverse denoising process satisfies the following formula: ； In the above formula, Z represents the Gaussian noise sampled during the reverse denoising process. This indicates that the UNet denoising network denoises the current noisy image. The noise component predicted at time step t.

9. A super-resolution system for power scene images based on a diffusion model, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the steps of the diffusion-based super-resolution method for power scene images as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program / instructions, characterized in that, The computer program / instructions are programmed or configured to execute the steps of the diffusion-based power scene image super-resolution method according to any one of claims 1 to 8 via a processor.

Images