Image diffusion enhancement method and device based on multi-scale feature extraction and fusion

Abstract

The application provides an image diffusion enhancement method and device based on multi-scale feature extraction and fusion, which comprises the following steps: obtaining a to-be-processed image; performing multi-scale feature extraction on the to-be-processed image to obtain a multi-scale feature map; performing pyramid feature fusion on the multi-scale feature map to obtain a fused feature map; performing compression block detection on the to-be-processed image and generating a block boundary mask; based on a preset retrieval algorithm, retrieving a similar reference image of the to-be-processed image from a natural image reference library and extracting a reference feature of the similar reference image; taking the fused feature map, the block boundary mask and the reference feature as conditions, using a conditional latent space diffusion model to enhance the to-be-processed image and output an enhanced image. The method and device of the application can effectively restore the brightness, contrast and detail texture of a downhole low-light-intensity compressed image without relying on paired training data, significantly suppresses compression artifacts such as color blocks and ringing, and generates a high-quality enhanced image with rich details and natural structure.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an image diffusion enhancement method and apparatus based on multi-scale feature extraction and fusion. Background Technology

[0002] Underground safety monitoring relies heavily on image and video data. However, underground shooting environments typically suffer from insufficient lighting, high noise levels, low contrast, and severe compression artifacts (such as color blocks, banding, and ringing) introduced by efficient transmission and storage. These combined degradation factors result in significant loss of detail in the acquired images, directly impacting the accuracy and reliability of subsequent manual judgment and computer vision-based automated detection (such as equipment fault identification and safety hazard investigation).

[0003] To address the above problems, existing technologies have proposed various image enhancement and restoration methods, but all of them have obvious limitations:

[0004] First, traditional image enhancement methods, such as histogram equalization, CLAHE, Retinex theoretical models, and Gamma correction, are simple to implement and have low computational cost. However, their enhancement strategies are relatively global, with limited ability to recover local compression artifacts and details. Furthermore, while enhancing contrast, they are prone to amplifying noise and artifacts, and may even introduce new distortions.

[0005] Second, traditional denoising and deblocking methods, such as BM3D filtering and JPEG deblocking algorithms, can suppress specific types of degradation (such as noise or block boundaries) to a certain extent. However, they are insufficient for the mixed degradation mode of "low illumination + high noise + strong compression" in downhole environments, and it is difficult to improve the overall image quality while restoring details.

[0006] Third, there are deep learning-based image restoration methods, such as ARCNN (Allied Recurrent and Convolutional Neural Network), DnCNN (Denoising Convolutional Neural Network), ESRGAN (Enhanced Super-Resolution Generative Adversarial Network), and SwinIR (SwinTransformer-based Image Restoration Network). These methods have achieved significant results in many image super-resolution and denoising tasks, but their performance is highly dependent on large-scale, high-quality paired training data (i.e., low-quality input images and corresponding high-quality target images). In the specific scenario of underground mining, obtaining a large number of realistic, matching, high-quality paired samples is extremely difficult, which greatly limits the training and application effects of this type of model.

[0007] Fourth, generative methods, such as Generative Adversarial Networks (GANs) and diffusion models. These methods demonstrate powerful capabilities in generating high-quality, high-detail textures. However, general generative models lack specific constraints on the degradation priors (especially block artifacts) unique to downhole images. During the generation process, they may "over-imagine," redrawing textures that do not conform to the actual geometric structure or semantic information of the scene, or failing to effectively eliminate the original compression artifact boundaries, resulting in distorted generation results.

[0008] In summary, existing technologies face several bottlenecks when processing complex degraded downhole images, including strong dependence on pairwise data, poor adaptability to mixed degradation patterns, and insufficient reliability and realism of the generated results. Therefore, there is an urgent need for an image diffusion enhancement method and apparatus based on multi-scale feature extraction and fusion that can effectively integrate prior knowledge of the downhole scene without relying on pairwise data, achieving brightness enhancement, noise suppression, artifact removal, and detail reconstruction. Summary of the Invention

[0009] Therefore, it is necessary to provide an image diffusion enhancement method and apparatus based on multi-scale feature extraction and fusion to address the aforementioned technical problems.

[0010] In a first aspect, the present invention provides an image diffusion enhancement method based on multi-scale feature extraction and fusion, the method comprising:

[0011] Obtain the image to be processed;

[0012] Multi-scale feature extraction is performed on the image to be processed to obtain a multi-scale feature map;

[0013] Pyramid feature fusion is performed on the multi-scale feature maps to obtain a fused feature map;

[0014] Perform compressed block detection on the image to be processed and generate block boundary masks;

[0015] Based on a preset retrieval algorithm, similar reference images of the image to be processed are retrieved from a natural image reference library, and reference features of the similar reference images are extracted.

[0016] By using the fused feature map, block boundary mask, and reference features as conditions, the conditional latent space diffusion model is used to enhance the image to be processed, and the enhanced image is output.

[0017] Optionally, multi-scale feature extraction is performed on the image to be processed to obtain a multi-scale feature map, including:

[0018] Based on a pre-defined lightweight network model, U is calculated. 2 The target weight values ​​of each convolutional layer in the Net network model;

[0019] Based on the weight values ​​of each objective, and using U 2 The Net network model calculates the dynamic convolution kernels of each convolutional layer and extracts multi-scale features from the image to be processed based on these dynamic convolution kernels, resulting in a multi-scale feature map. The dynamic convolution kernels are calculated using the following formula:

[0020]

[0021] in, For the dynamic convolution kernel of the i-th convolutional layer, Let be the target weight values ​​for the i-th convolutional layer. The image to be processed. For the first One basic convolutional kernel, The number of basic convolutional kernels.

[0022] Optionally, pyramid feature fusion is performed on the multi-scale feature maps to obtain a fused feature map, including:

[0023] The multi-scale feature maps are sorted from high to low spatial resolution to obtain the multi-scale feature map sequence [F1, F2, ..., F]. L ], where F1 is the shallowest feature map with the highest resolution, F L This is the deep feature map with the lowest resolution;

[0024] From the deepest feature map F L Begin by following these steps sequentially from top to bottom:

[0025] Upsample the feature map of the current level so that the spatial size of the feature map of the current level is the same as that of the feature map of the next level.

[0026] Adjust the channel dimension of the feature map at the current level using a 1×1 convolution kernel to ensure that it has the same number of channels as the feature map at the next level.

[0027] The feature map of the current level, after upsampling and channel adjustment, is added element by element to the feature map of the next level to obtain the fused feature map of the current level.

[0028] Repeat the above steps until all levels of feature maps have been processed to obtain a multi-scale fused feature pyramid.

[0029] Spatial attention weights are generated for each feature map in the multi-scale fused feature pyramid using a small attention network;

[0030] Based on spatial attention weights, the multi-scale fusion feature pyramid is weighted and summed pixel by pixel to obtain the final fusion feature map.

[0031] Optionally, compressed block detection is performed on the image to be processed, and a block boundary mask is generated, including:

[0032] Compressed detection is performed on the image to be processed;

[0033] If the image to be processed conforms to the preset compression mode, the image to be processed is converted to grayscale, and the processed image to be processed is divided into pixel blocks of a preset size;

[0034] If the image to be processed does not conform to the preset compression mode, the image to be processed is compressed and grayscaled sequentially according to the preset compression mode, and the processed image to be processed is divided into pixel blocks of a preset size.

[0035] Perform a discrete cosine transform on each pixel block to obtain the corresponding discrete cosine coefficient matrix;

[0036] Based on each discrete cosine coefficient matrix, the sum of the absolute values ​​of the high-frequency regions in each pixel block is calculated as the high-frequency energy value of the corresponding pixel block;

[0037] Traverse the adjacent boundaries of all adjacent pixel blocks in the image to be processed, and calculate the absolute difference of the high-frequency energy values ​​of all two pixel blocks with adjacent boundaries.

[0038] Each absolute difference is mapped to a preset interval through a preset function to serve as the corresponding block boundary mask.

[0039] Optionally, based on a preset retrieval algorithm, similar reference images of the image to be processed are retrieved from a natural image reference database, and reference features of the similar reference images are extracted, including:

[0040] Based on the attribute conditions of the image to be processed, construction attribute conditions are selected, and a natural image reference library is constructed according to the construction attribute conditions.

[0041] Based on a preset image retrieval algorithm, similar reference images that are similar to the image to be processed are retrieved from a natural image reference library;

[0042] A pre-trained image recognition and classification model is used to extract reference features from similar reference images.

[0043] Optionally, the fused feature map, block boundary mask, and reference features are used as conditions to enhance the image to be processed using a conditional latent space diffusion model, outputting an enhanced image, including:

[0044] The image to be processed is encoded into the latent space using a pre-trained encoder to obtain an initial latent space representation;

[0045] Based on the conditional fusion mechanism, the fusion feature map, block boundary mask and reference features are fused to obtain the fusion conditions;

[0046] The initial latent space representation and fusion conditions are input together into the U-Net prediction network in the conditional latent space diffusion model for inverse denoising, and the denoised latent space representation is predicted.

[0047] Using a pre-trained decoder, the denoised latent space representation is decoded to output an enhanced image.

[0048] Optionally, the method further includes:

[0049] The reference features are input into the cross-attention conditional injection module to calculate the attention weights between the current layer latent features and the reference features of the U-Net prediction network, and to generate latent features enhanced by the reference features.

[0050] The enhanced latent features are then used as conditional inputs into the U-Net prediction network for the next layer of latent feature enhancement.

[0051] Optionally, the method further includes:

[0052] A training sample library is constructed by selecting a preset number of unpaired first low-quality images and first high-quality images, or by selecting a preset number of paired second low-quality images and second high-quality images.

[0053] Multi-scale feature extraction is performed on the first or second low-quality image in the training sample library to obtain the first multi-scale feature map;

[0054] Pyramid feature fusion is performed on the first multi-scale feature map to obtain the first fused feature map;

[0055] Perform compressed block detection on the first low-quality image or the second low-quality image, and generate the first block boundary mask;

[0056] Based on a preset retrieval algorithm, a first similar reference image of a first low-quality image or a second low-quality image is retrieved from a natural image reference library, and a first reference feature of the first similar reference image is extracted.

[0057] Using the first fused feature map, the first boundary mask, and the first reference feature as conditions, the conditional latent space diffusion model is used to enhance the image to be processed, and the enhanced training image is output.

[0058] Based on the enhanced training images, a hierarchical loss function is calculated, which includes one or more of the following: conditional latent space diffusion loss, pixel loss, perceptual loss, frequency domain loss, edge loss, block artifact suppression loss, and perceptual style alignment loss.

[0059] The hierarchical loss function is input into the preset model optimizer to calculate the optimization parameters of the conditional latent space diffusion model. Based on the optimization parameters, the parameters of the conditional latent space diffusion model are adjusted to obtain the optimized conditional latent space diffusion model.

[0060] Optionally, the method further includes:

[0061] Based on preset attribute conditions, select the second highest quality image;

[0062] The second high-quality image is sequentially subjected to low-light, noise, blur, and compression processing to obtain the second low-quality image.

[0063] Pair the second low-quality image with the second high-quality image to obtain a paired second low-quality image and a paired second high-quality image.

[0064] Secondly, the present invention provides an image diffusion enhancement device based on multi-scale feature extraction and fusion, comprising:

[0065] The image acquisition module is used to acquire the image to be processed.

[0066] The first extraction module, connected to the image acquisition module, is used to extract multi-scale features from the image to be processed, and obtain a multi-scale feature map.

[0067] The fusion module, connected to the first extraction module, is used to perform pyramid feature fusion on multi-scale feature maps to obtain a fused feature map.

[0068] The compression detection and mask generation module, connected to the image acquisition module, is used to perform compression block detection on the image to be processed and generate block boundary masks.

[0069] The second extraction module, connected to the image acquisition module, is used to retrieve similar reference images of the image to be processed from the natural image reference library based on a preset retrieval algorithm, and extract reference features of the similar reference images.

[0070] The conditional latent space diffusion model is connected to the image acquisition module, fusion module, compression detection and mask generation module, and second extraction module, respectively. It is used to enhance the image to be processed by taking the fused feature map, block boundary mask and reference features as conditions and output the enhanced image.

[0071] This invention provides an image diffusion enhancement method and apparatus based on multi-scale feature extraction and fusion. By introducing multi-scale feature extraction and pyramid fusion mechanisms, combined with compressed block-aware masks and natural image reference priors, a conditional latent space diffusion enhancement framework independent of paired training data is constructed. This framework enables synergistic optimization of image brightness enhancement, noise suppression, artifact removal, and detail reconstruction in composite degradation scenarios characterized by low illumination, high noise, and strong compression. This method not only significantly improves the visual quality of downhole monitoring images but also provides a more reliable data foundation for subsequent image-based intelligent analysis and decision-making. Attached Figure Description

[0072] Figure 1a This is a schematic diagram of a process for an image diffusion enhancement method based on multi-scale feature extraction and fusion provided in an embodiment of the present invention;

[0073] Figure 1b This is another flowchart illustrating the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in an embodiment of the present invention;

[0074] Figure 1c This is another flowchart illustrating the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in this embodiment of the invention.

[0075] Figure 1d The image to be processed is a low-quality downhole image that has not been processed by the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in the embodiments of the present invention.

[0076] Figure 1e To Figure 1d The image shown is an enhanced image after the low-quality downhole image to be processed is processed by the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in this embodiment of the invention.

[0077] Figure 2a This is a schematic diagram of a circuit module structure of an image diffusion enhancement device based on multi-scale feature extraction and fusion provided in an embodiment of the present invention;

[0078] Figure 2b This is a schematic diagram of another circuit module structure of the image diffusion enhancement device based on multi-scale feature extraction and fusion provided in an embodiment of the present invention;

[0079] Figure 3 This is an internal structural diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0081] like Figure 1a As shown, this invention provides an image diffusion enhancement method based on multi-scale feature extraction and fusion, the method comprising:

[0082] Step S11: Obtain the image to be processed;

[0083] Step S12: Perform multi-scale feature extraction on the image to be processed to obtain a multi-scale feature map;

[0084] Optionally, such as Figure 1b As shown, step S12 specifically includes:

[0085] Step S121: Calculate U based on the preset lightweight network model. 2 The target weight values ​​of each convolutional layer in the Net network model;

[0086] The preset lightweight network model can be an existing lightweight network model, such as the MobileNetV2 lightweight convolutional neural network model. Those skilled in the art can choose flexibly according to actual needs; no limitation is made here. U 2 The Net network model is a salient object detection model based on a deeply nested U-shaped structure, which achieves multi-scale feature fusion through residual U modules (RSU).

[0087] Step S122: Based on the weight values ​​of each target, and using U... 2 The Net network model calculates the dynamic convolution kernels of each convolutional layer and extracts multi-scale features from the image to be processed based on these dynamic convolution kernels, resulting in a multi-scale feature map. The dynamic convolution kernels are calculated using the following formula:

[0088]

[0089] in, For the dynamic convolution kernel of the i-th convolutional layer, Let be the target weight values ​​for the i-th convolutional layer. The image to be processed. For the first One basic convolutional kernel, The number of basic convolutional kernels.

[0090] Step S13: Perform pyramid feature fusion on the multi-scale feature maps to obtain a fused feature map;

[0091] Optionally, step S13 specifically includes:

[0092] Step S131: Sort the multi-scale feature maps in descending order of spatial resolution to obtain the multi-scale feature map sequence [F1, F2, ..., F...]. L ], where F1 is the shallowest feature map with the highest resolution, F L This is the deep feature map with the lowest resolution;

[0093] From the deepest feature map F L Begin by following these steps sequentially from top to bottom:

[0094] Step S132: Upsample the feature map of the current level so that the spatial size of the feature map of the current level is the same as that of the feature map of the next level.

[0095] Step S132: Adjust the channel dimension of the feature map of the current layer using a 1×1 convolution kernel to ensure that it has the same number of channels as the feature map of the next layer;

[0096] Step S133: Add the upsampled and channel-adjusted feature map of the current level to the feature map of the next level element by element to obtain the fused feature map of the current level;

[0097] Repeat steps S132 to S133 until all levels of feature maps have been processed to obtain a multi-scale fused feature pyramid.

[0098] Step S134: Generate spatial attention weights for each feature map in the multi-scale fused feature pyramid using a small attention network;

[0099] Step S135: Based on spatial attention weights, perform pixel-wise weighted summation on the multi-scale fusion feature pyramid to obtain the final fusion feature map.

[0100] For example, assuming the image to be processed is a 1280×720×3 underground monitoring image, after multi-scale feature extraction, four feature maps of different scales are obtained: F1: 1280×720×64 (highest resolution, retaining the most spatial details); F2: 640×360×128 (1 / 2 downsampling); F3: 320×180×256 (1 / 4 downsampling); F4: 160×90×512 (1 / 8 downsampling, deepest semantic features); the pyramid feature fusion in step S13 is as follows:

[0101] ① Using bilinear interpolation, F4 (160×90×512) is upsampled by 2 times to obtain a feature map of 320×180×512; the number of channels is adjusted by a 1×1 convolution kernel to obtain a feature map of 320×180×256, ensuring that the number of channels is the same as that of F3; the adjusted F4 (320×180×256) and F3 (320×180×256) are added element by element to obtain the fused feature map F3' (320×180×256).

[0102] ② Using the bilinear interpolation algorithm, F3' (320×180×256) is upsampled by 2 times to obtain a feature map of 640×360×256; the number of channels is adjusted by a 1×1 convolution kernel to obtain 640×360×128, ensuring that the number of channels is the same as that of F2; the adjusted F3' (640×360×128) and F2 (640×360×128) are added element by element to obtain the fused feature map F2' (640×360×128).

[0103] ③ Using the bilinear interpolation algorithm, F2' (640×360×128) is upsampled by 2 times to obtain a feature map of 1280×720×128; the number of channels is adjusted by a 1×1 convolution kernel to obtain a feature map of 1280×720×64, ensuring that the number of channels is the same as that of F1; the adjusted F2' (1280×720×64) and F1 (1280×720×64) are added element by element to obtain F1' (1280×720×64).

[0104] Four levels of fused feature maps are obtained from ① to ③: F1': 1280×720×64 (highest resolution); F2': 640×360×128; F3': 320×180×256; F4' (i.e. F4): 160×90×512 (lowest resolution).

[0105] ④ Generate spatial attention weights for each feature map using a small attention network (e.g., containing two 3×3 convolutional layers and a sigmoid activation function): For F1', output attention weight map A1: 1280×720×1; for F2', output attention weight map A2: 640×360×1; for F3', output attention weight map A3: 320×180×1; for F4', output attention weight map A4: 160×90×1.

[0106] ⑤ Multiply F1' and A1 element by element, i.e., F1'×A1; Upsample F2' and A2 to 1280×720 and multiply them element by element, i.e., upsample(F2')×upsample(A2); Upsample F3' and A3 to 1280×720 and multiply them element by element, i.e., upsample(F3')×upsample(A3); Upsample F4' and A4 to 1280×720 and multiply them element by element, i.e., upsample(F4')×upsample(A4).

[0107] ⑥ Add all the weighted feature maps pixel by pixel to obtain the final fused feature map (1280×720×64).

[0108] Through this pyramid feature fusion mechanism, the final 1280×720×64 fused feature map not only retains the shallow high-frequency texture details, but also incorporates deep semantic structure information, providing rich multi-scale conditional features for subsequent conditional latent space diffusion enhancement.

[0109] Step S14: Perform compressed block detection on the image to be processed and generate block boundary masks;

[0110] Optionally, step S14 specifically includes:

[0111] Compressed detection is performed on the image to be processed;

[0112] If the image to be processed conforms to the preset compression mode, the image to be processed is converted to grayscale, and the processed image to be processed is divided into pixel blocks of a preset size;

[0113] If the image to be processed does not conform to the preset compression mode, the image to be processed is compressed and grayscaled sequentially according to the preset compression mode, and the processed image to be processed is divided into pixel blocks of a preset size.

[0114] In this invention, the preset compression mode can be JPEG compression mode, and the compression and grayscale processing methods can be existing compression and grayscale processing methods. The preset size of the pixel block is preferably an 8×8 pixel block. Those skilled in the art can choose according to actual needs, and there is no limitation here.

[0115] Perform a discrete cosine transform on each pixel block to obtain the corresponding discrete cosine coefficient matrix;

[0116] Based on each discrete cosine coefficient matrix, the sum of the absolute values ​​of the high-frequency regions in each pixel block is calculated as the high-frequency energy value of the corresponding pixel block;

[0117] Traverse the adjacent boundaries of all adjacent pixel blocks in the image to be processed, and calculate the absolute difference of the high-frequency energy values ​​of all two pixel blocks with adjacent boundaries.

[0118] Each absolute difference is mapped to a preset interval through a preset function to serve as the corresponding block boundary mask.

[0119] To enable those skilled in the art to clearly understand the specific implementation process of step S14, a detailed description will be provided below using specific embodiments, as follows:

[0120] By checking the header information of the image to be processed, it can be determined whether the image is compressed: JPEG header information check: Read the first 2 bytes (16 bits) of the image file to be processed and check if it is 0xFFD8, which is the standard start marker of JPEG image files. If 0xFFD8 is detected, it is confirmed to be in JPEG compression format.

[0121] The appropriate processing should be carried out based on the compression test results:

[0122] Scenario 1: If the image to be processed conforms to JPEG compression mode, directly perform grayscale processing on the image to be processed, and divide the grayscale processed image into 8×8 pixel blocks. For a 1280×720 image to be processed: number of horizontal blocks 1280÷8=160, number of vertical blocks 720÷8=90, total number of blocks 160×90=14,400 blocks.

[0123] Scenario 2: If the image to be processed does not conform to the JPEG compression mode, the image to be processed is compressed and grayscaled sequentially according to the JPEG compression mode, and the grayscaled image to be processed is divided into 8×8 pixel blocks.

[0124] Perform a two-dimensional discrete cosine transform (DCT) on each 8×8 pixel block to obtain an 8×8 discrete cosine coefficient matrix.

[0125] Based on the 8×8 discrete cosine coefficient matrix, the sum of the absolute values ​​of the high-frequency regions in each 8×8 pixel block is calculated as the high-frequency energy value of the corresponding pixel block. A region in the discrete cosine coefficient matrix that meets any of the following conditions is a high-frequency region:

[0126] or ,

[0127] in, For vertical frequency component index, Index for horizontal frequency components.

[0128] Traverse the adjacent boundaries of all adjacent 8×8 pixel blocks in the image to be processed, and calculate the absolute difference of the high-frequency energy values ​​between any two 8×8 pixel blocks with adjacent boundaries. Assume there are pixel blocks A, B, C, and D. Pixel block A is adjacent to pixel block B, pixel block B is adjacent to pixel block D, pixel block C is adjacent to pixel block A, and pixel block D is adjacent to pixel block C. Therefore, it is necessary to calculate the absolute difference of the high-frequency energy values ​​between pixel blocks A and B. The absolute difference in high-frequency energy values ​​between pixel blocks B and D The absolute difference in high-frequency energy values ​​between pixel blocks C and A The absolute difference in high-frequency energy values ​​between D-pixel blocks and C-pixel blocks .

[0129] Each absolute difference is mapped to the [0,1] interval using a Sigmoid function to serve as the corresponding block boundary mask.

[0130] Step S15: Based on the preset retrieval algorithm, retrieve similar reference images of the image to be processed from the natural image reference library, and extract the reference features of the similar reference images;

[0131] Optionally, such as Figure 1c As shown, step S15 specifically includes:

[0132] Step S151: Based on the attribute conditions of the image to be processed, select the construction attribute conditions, and construct the natural image reference library according to the construction attribute conditions;

[0133] The attribute conditions can be texture type or lighting conditions, which can be flexibly selected by those skilled in the art according to actual needs, and are not limited here.

[0134] Step S152: Based on a preset image retrieval algorithm, retrieve similar reference images that are similar to the image to be processed from the natural image reference library;

[0135] The preset image retrieval algorithm can be the existing FAISS similarity retrieval algorithm or CLIP (Constrastive Language-Image Pre-training) algorithm. Those skilled in the art can choose flexibly according to actual needs, and there is no limitation here.

[0136] Step S153: Use a pre-trained image recognition classification model to extract reference features from similar reference images.

[0137] The pre-trained image recognition and classification model can be the existing VGG16 model.

[0138] Specifically, based on the attribute conditions of the image to be processed (such as texture type and lighting conditions), a natural image reference library is constructed, containing 1000 high-quality natural images (such as rock walls and metal surfaces), all with a resolution of 1280×720. The FAISS similarity retrieval algorithm or CLIP algorithm is used to retrieve the top 3 reference images most similar to the image to be processed from the reference library. The pre-trained VGG16 model is used to extract reference features (512-dimensional feature vectors) from the reference images.

[0139] Step S16: Using the fused feature map, block boundary mask, and reference features as conditions, the conditional latent space diffusion model is used to enhance the image to be processed, and the enhanced image is output.

[0140] Optionally, step S16 specifically includes:

[0141] Step S161: Encode the image to be processed into the latent space using a pre-trained encoder to obtain an initial latent space representation;

[0142] Step S162: Based on the conditional fusion mechanism, the fusion feature map, block boundary mask and reference features are fused to obtain the fusion conditions;

[0143] The conditional fusion mechanism can be implemented using a feature linear adjustment mechanism (FiLM mechanism) and / or a cross-attention mechanism, without any limitation here.

[0144] Step S163: Input the initial latent space representation and fusion conditions into the U-Net prediction network in the conditional latent space diffusion model for reverse denoising, and predict the denoised latent space representation.

[0145] Step S164: Use the pre-trained decoder to decode the denoised latent space representation and output the enhanced image.

[0146] In this invention, the encoder, U-Net prediction network and decoder in the conditional latent space diffusion model are all existing encoders, U-Net prediction networks and decoders. Those skilled in the art can flexibly choose according to actual needs, and no limitation is made here.

[0147] Specifically, the image to be processed is encoded into the latent space using the encoder in the pre-trained VAE (Variational Autoencoder) to obtain the initial latent space representation. The dimensions are 160×90×4 (corresponding to a 1 / 8 downsampled version of the original image of 1280×720), where, , The image to be processed. The process involves encoding the image to be processed; based on a conditional fusion mechanism, the fusion feature map, block boundary mask, and reference features are concatenated and fused to obtain the fusion conditions; the initial latent space representation and the fusion conditions are input into the U-Net prediction network in the conditional latent space diffusion model for inverse denoising, predicting the denoised latent space representation. diffusion steps =1000, noise scheduling adopts a linear strategy; the decoder in the pre-trained VAE is used to decode the denoised latent space representation, outputting an enhanced image with the resolution restored to 1280×720.

[0148] in, ,in, The image with Gaussian noise added in the positive direction. For diffusion steps, This represents the noisy latent space representation at diffusion step number t. This indicates that, given a high-quality image cond, the U-Net prediction network predicts the current noisy latent space representation. Given the diffusion steps t, predict the denoised target image. , This represents the L2 norm or the square of the L2 loss. This represents the squared value of the L2 norm after encoding the difference between the forward-added Gaussian noise image and the target image.

[0149] It should be noted that steps S161 to S164 are not shown in the figure, but are only for the purpose of description.

[0150] In an optional embodiment of the present invention, the method further includes:

[0151] The reference features are input into the cross-attention conditional injection module to calculate the attention weights between the current layer latent features and the reference features of the U-Net prediction network, and to generate latent features enhanced by the reference features.

[0152] The enhanced latent features are then used as conditional inputs into the U-Net prediction network for the next layer of latent feature enhancement.

[0153] By introducing a cross-attention conditional injection mechanism, reference features are dynamically aligned and fused with the latent features of the current layer of the U-Net prediction network, enabling precise detail guidance and semantic awareness enhancement. This mechanism allows the model to adaptively retrieve the most relevant texture and structural information from high-quality images during the generation process and inject it into the latent features. This not only significantly improves the realism and texture richness of the output image but also effectively avoids geometric distortion or semantic inconsistencies caused by unconditional generation, ultimately achieving high-quality image enhancement that preserves detail and structure.

[0154] In an optional embodiment of the present invention, the method further includes:

[0155] A training sample library is constructed by selecting a preset number of unpaired first low-quality images and first high-quality images, or by selecting a preset number of paired second low-quality images and second high-quality images.

[0156] Multi-scale feature extraction is performed on the first or second low-quality image in the training sample library to obtain the first multi-scale feature map;

[0157] Pyramid feature fusion is performed on the first multi-scale feature map to obtain the first fused feature map;

[0158] Perform compressed block detection on the first low-quality image or the second low-quality image, and generate the first block boundary mask;

[0159] Based on a preset retrieval algorithm, a first similar reference image of a first low-quality image or a second low-quality image is retrieved from a natural image reference library, and a first reference feature of the first similar reference image is extracted.

[0160] Using the first fused feature map, the first boundary mask, and the first reference feature as conditions, the conditional latent space diffusion model is used to enhance the image to be processed, and the enhanced training image is output.

[0161] Based on the enhanced training images, a hierarchical loss function is calculated, which includes one or more of the following: conditional latent space diffusion loss, pixel loss, perceptual loss, frequency domain loss, edge loss, block artifact suppression loss, and perceptual style alignment loss.

[0162] The hierarchical loss function can be calculated using the following formula: ,in, For each loss weight, For each loss.

[0163] For example, if the hierarchical loss function includes: conditional latent space diffusion loss, pixel loss, and perceptual loss, then ,in, For the conditional latent space diffusion loss weight, For conditional latent space diffusion loss, For pixel loss weights, For pixel loss, To perceive loss weights, To perceive loss.

[0164] Among them, the conditional latent space diffusion loss, pixel loss, perceptual loss, frequency domain loss, edge loss, block effect suppression loss and perceptual style alignment loss can all be calculated using methods in the prior art. Those skilled in the art can flexibly choose according to actual needs, and no limitation is made here.

[0165] In one alternative implementation, the above-mentioned loss can be calculated using the following formula:

[0166] The calculation method for conditional latent space diffusion loss is as described above. The calculation method will not be elaborated here.

[0167] Pixel loss: ,in, To enhance the image, For the first high-quality image, Indicates the L1 norm or L1 loss. This represents the L1 norm value after encoding the difference between the enhanced image and the first high-quality image;

[0168] Perceived loss: ,in, For the first Hierarchical weights To enhance the image, For the second highest quality image, The first convolution after enhancing the image Layer features, The first convolution is performed on the second high-quality image. Layer reference features, Indicates the L2 norm or L2 loss;

[0169] Frequency domain loss: ,in, For frequency band weighting, For Fast Fourier Transform loss, , This represents the absolute value after performing a Fast Fourier Transform on the enhanced image. This represents the absolute value after performing a Fast Fourier Transform on the second high-quality image. It should be noted that for the frequency band weights... Those skilled in the art can set it flexibly according to actual needs, and no limitation is made here.

[0170] Edge loss: , To enhance the image, For the first high-quality image, This indicates that convolution is performed on the enhanced image. This indicates that convolution is performed on the first high-quality image. Indicates the L1 norm or L1 loss;

[0171] Block effect suppresses loss: Where b is a compressed block in the enhanced image, This represents the b-th compressed block in the enhanced image. Indicates the second-order difference. Indicates the L1 norm or L1 loss;

[0172] Perceived style alignment loss: ,in, To enhance the image, For the second highest quality image, The first convolution after enhancing the image Layer features, The first convolution is performed on the second high-quality image. Layer reference features, For Gram matrix operations, This represents the L2 norm or the square of the L2 loss.

[0173] In this invention, the first high-quality image and the second high-quality image can be images selected from a natural image reference library based on attributes such as image contrast and image resolution. Those skilled in the art can pre-set attribute thresholds such as image contrast and image resolution, and determine images that are greater than or equal to the attribute thresholds as high-quality images and images that are less than the attribute thresholds as low-quality images. No limitation is made here.

[0174] The hierarchical loss function is input into the preset model optimizer to calculate the optimization parameters of the conditional latent space diffusion model. Based on the optimization parameters, the parameters of the conditional latent space diffusion model are adjusted to obtain the optimized conditional latent space diffusion model.

[0175] In an optional embodiment of the present invention, the method further includes:

[0176] Based on preset attribute conditions, select the second highest quality image;

[0177] The second high-quality image is sequentially subjected to low-light, noise, blur, and compression processing to obtain the second low-quality image.

[0178] Pair the second low-quality image with the second high-quality image to obtain a paired second low-quality image and a paired second high-quality image.

[0179] For example, based on preset attribute conditions (such as the material of the underground scene), 5000 second high-quality images are selected, all with a resolution of 1280×720; the second high-quality images are then subjected to low-light processing (brightness reduction of 70%), Gaussian noise addition (σ=0.1), Gaussian blur (kernel size=5), and JPEG compression (Q=15) in sequence to obtain second low-quality images; the second low-quality images and the second high-quality images are then paired to obtain paired training samples (i.e., paired second low-quality images and second high-quality images).

[0180] like Figure 1d and Figure 1e As shown, Figure 1d For low-quality downhole images that have not been processed by the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in this embodiment of the invention, Figure 1d Significant compression artifacts exist in the areas marked by the first dashed box 100 and the second dashed box 101, specifically manifested as color blocks, color banding, and ringing effects, severely affecting the structural continuity and detail discernibility of the image. Furthermore, regarding... Figure 1d The low-quality downhole image to be processed shown is obtained after applying the image diffusion enhancement method based on multi-scale feature extraction and fusion provided in this embodiment of the invention. Figure 1e The enhanced image shown ( Figure 1e The areas marked by the first dashed box 100 and the second dashed box 101 shown are... Figure 1d The areas marked by the first dashed box 100 and the second dashed box 101 correspond one-to-one. This enhanced image shows significant improvements in overall brightness, contrast, and detail, especially in... Figure 1d In the areas marked by the first dashed box 100 and the second dashed box 101, the original compression artifacts are effectively suppressed, the image edges are smoother and more natural, the texture details are restored realistically, and the scene structure remains intact, fully demonstrating the superior performance of the method of the present invention in restoring image quality under complex degradation conditions.

[0181] This invention presents an image diffusion enhancement method based on multi-scale feature extraction and fusion. By introducing multi-scale feature extraction and pyramid fusion mechanisms, combined with compressed block-aware masks and natural image reference priors, a conditional latent space diffusion enhancement framework independent of pairwise training data is constructed. This framework enables synergistic optimization of image brightness enhancement, noise suppression, artifact removal, and detail reconstruction in composite degradation scenarios characterized by low illumination, high noise, and strong compression. This method not only significantly improves the visual quality of downhole monitoring images but also provides a more reliable data foundation for subsequent image-based intelligent analysis and decision-making.

[0182] Based on the same inventive concept, embodiments of the present invention also provide an image diffusion enhancement device based on multi-scale feature extraction and fusion for implementing the image diffusion enhancement method based on multi-scale feature extraction and fusion described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more embodiments of the image diffusion enhancement device based on multi-scale feature extraction and fusion provided below can be found in the limitations of the image diffusion enhancement method based on multi-scale feature extraction and fusion described above, and will not be repeated here.

[0183] like Figure 2a As shown, this invention provides an image diffusion enhancement device based on multi-scale feature extraction and fusion, comprising: an image acquisition module 21, a first extraction module 22, a fusion module 23, a compression detection and mask generation module 24, a second extraction module 25, and a conditional latent space diffusion model 26; wherein,

[0184] Image acquisition module 21 is used to acquire the image to be processed;

[0185] The first extraction module 22 is connected to the image acquisition module 21 and is used to extract multi-scale features from the image to be processed to obtain a multi-scale feature map.

[0186] The fusion module 23, connected to the first extraction module 22, is used to perform pyramid feature fusion on the multi-scale feature map to obtain a fused feature map.

[0187] The compression detection and mask generation module 24 is connected to the image acquisition module 21 and is used to perform compression block detection on the image to be processed and generate block boundary masks.

[0188] The second extraction module 25, connected to the image acquisition module 21, is used to retrieve similar reference images of the image to be processed from the natural image reference library based on a preset retrieval algorithm, and extract reference features of the similar reference images.

[0189] The conditional latent space diffusion model 26 is connected to the image acquisition module 21, the fusion module 23, the compression detection and mask generation module 24, and the second extraction module 25, respectively. It is used to enhance the image to be processed by using the fused feature map, block boundary mask and reference features as conditions, and output the enhanced image.

[0190] Optionally, the first extraction module 22 is specifically used for:

[0191] Based on a pre-defined lightweight network model, U is calculated. 2 The target weight values ​​of each convolutional layer in the Net network model;

[0192] Based on the weight values ​​of each objective, and using U 2The Net network model calculates the dynamic convolution kernels of each convolutional layer and extracts multi-scale features from the image to be processed based on these dynamic convolution kernels, resulting in a multi-scale feature map. The dynamic convolution kernels are calculated using the following formula:

[0193]

[0194] in, For the dynamic convolution kernel of the i-th convolutional layer, Let be the target weight values ​​for the i-th convolutional layer. The image to be processed. For the first One basic convolutional kernel, The number of basic convolutional kernels.

[0195] Optionally, the fusion module 23 is specifically used for:

[0196] The multi-scale feature maps are sorted from high to low spatial resolution to obtain the multi-scale feature map sequence [F1, F2, ..., F]. L ], where F1 is the shallowest feature map with the highest resolution, F L This is the deep feature map with the lowest resolution;

[0197] From the deepest feature map F L Begin by following these steps sequentially from top to bottom:

[0198] Upsample the feature map of the current level so that the spatial size of the feature map of the current level is the same as that of the feature map of the next level.

[0199] Adjust the channel dimension of the feature map at the current level using a 1×1 convolution kernel to ensure that it has the same number of channels as the feature map at the next level.

[0200] The feature map of the current level, after upsampling and channel adjustment, is added element by element to the feature map of the next level to obtain the fused feature map of the current level.

[0201] Repeat the above steps until all levels of feature maps have been processed to obtain a multi-scale fused feature pyramid.

[0202] Spatial attention weights are generated for each feature map in the multi-scale fused feature pyramid using a small attention network;

[0203] Based on spatial attention weights, the multi-scale fusion feature pyramid is weighted and summed pixel by pixel to obtain the final fusion feature map.

[0204] Optionally, the compression detection and mask generation module 24 is specifically used for:

[0205] Compressed detection is performed on the image to be processed;

[0206] If the image to be processed conforms to the preset compression mode, the image to be processed is converted to grayscale, and the processed image to be processed is divided into pixel blocks of a preset size;

[0207] If the image to be processed does not conform to the preset compression mode, the image to be processed is compressed and grayscaled sequentially according to the preset compression mode, and the processed image to be processed is divided into pixel blocks of a preset size.

[0208] Perform a discrete cosine transform on each pixel block to obtain the corresponding discrete cosine coefficient matrix;

[0209] Based on each discrete cosine coefficient matrix, the sum of the absolute values ​​of the high-frequency regions in each pixel block is calculated as the high-frequency energy value of the corresponding pixel block;

[0210] Traverse the adjacent boundaries of all adjacent pixel blocks in the image to be processed, and calculate the absolute difference of the high-frequency energy values ​​of all two pixel blocks with adjacent boundaries.

[0211] Each absolute difference is mapped to a preset interval through a preset function to serve as the corresponding block boundary mask.

[0212] Optionally, the second extraction module 25 is specifically used for:

[0213] Based on the attribute conditions of the image to be processed, construction attribute conditions are selected, and a natural image reference library is constructed according to the construction attribute conditions.

[0214] Based on a preset image retrieval algorithm, similar reference images that are similar to the image to be processed are retrieved from a natural image reference library;

[0215] A pre-trained image recognition and classification model is used to extract reference features from similar reference images.

[0216] Optionally, the conditional latent space diffusion model 26 is specifically used for:

[0217] The image to be processed is encoded into the latent space using a pre-trained encoder to obtain an initial latent space representation;

[0218] Based on the conditional fusion mechanism, the fusion feature map, block boundary mask and reference features are fused to obtain the fusion conditions;

[0219] The initial latent space representation and fusion conditions are input together into the U-Net prediction network in the conditional latent space diffusion model for inverse denoising, and the denoised latent space representation is predicted.

[0220] Using a pre-trained decoder, the denoised latent space representation is decoded to output an enhanced image.

[0221] Optionally, such as Figure 2b As shown, the device also includes: a cross-attention conditional injection module 27, which is connected to the second extraction module 25 and the conditional latent space diffusion model 26, respectively, for:

[0222] The reference features are input into the cross-attention conditional injection module 27, and the attention weights between the current layer latent features and the reference features in the U-Net prediction network in the conditional latent space diffusion model 26 are calculated to generate latent features enhanced by the reference features.

[0223] The enhanced latent features are used as conditional inputs into the U-Net prediction network in the conditional latent space diffusion model 26 for the next layer of latent feature enhancement.

[0224] Optionally, the device further includes: a pre-training module (not shown in the figure), a loss calculation module (not shown in the figure), and a model optimizer (not shown in the figure); wherein,

[0225] The pre-training module is connected to the first extraction module, the compression detection and mask generation module, the second extraction module, and the conditional latent space diffusion model. It is used to select a preset number of unpaired first low-quality images and first high-quality images or to select a preset number of paired second low-quality images and second high-quality images to construct a training sample library.

[0226] At this time, the first extraction module is also used to: perform multi-scale feature extraction on the first low-quality image or the second low-quality image in the training sample library to obtain the first multi-scale feature map;

[0227] The fusion module is also used to: perform pyramid feature fusion on the first multi-scale feature map to obtain the first fused feature map;

[0228] The compression detection and mask generation module is also used to: perform compressed block detection on the first low-quality image or the second low-quality image, and generate the first block boundary mask;

[0229] The second extraction module is also used to: retrieve a first similar reference image of the first low-quality image or the second low-quality image from the natural image reference library based on a preset retrieval algorithm, and extract the first reference feature of the first similar reference image;

[0230] The conditional latent space diffusion model is also used to: use the first fused feature map, the first boundary mask and the first reference feature as conditions, and use the conditional latent space diffusion model to enhance the image to be processed, and output the enhanced training image;

[0231] The loss calculation module is connected to the fusion module, the compression detection and mask generation module, the second extraction module, and the conditional latent space diffusion model, respectively. It is used to calculate the hierarchical loss function based on the enhanced training image. The hierarchical loss function includes one or more of the following: conditional latent space diffusion loss, pixel loss, perceptual loss, frequency domain loss, edge loss, block artifact suppression loss, and perceptual style alignment loss.

[0232] The model optimizer is connected to the loss calculation module and the conditional latent space diffusion model respectively. After receiving the hierarchical loss function, it calculates the optimization parameters of the conditional latent space diffusion model and adjusts the parameters of the conditional latent space diffusion model according to the optimization parameters to obtain the optimized conditional latent space diffusion model.

[0233] Optionally, the pre-training module is also used to: select a second high-quality image based on preset attribute conditions; sequentially process the second high-quality image with low illumination, noise, blur, and compression to obtain a second low-quality image; and pair the second low-quality image with the second high-quality image to obtain a paired second low-quality image and a second high-quality image.

[0234] This invention presents an image diffusion enhancement device based on multi-scale feature extraction and fusion. By introducing multi-scale feature extraction and pyramid fusion mechanisms, combined with compressed block-aware masks and natural image reference priors, it constructs a conditional latent space diffusion enhancement framework that does not rely on pairwise training data. This framework enables the coordinated optimization of image brightness enhancement, noise suppression, artifact removal, and detail reconstruction in composite degradation scenarios characterized by low illumination, high noise, and strong compression. This method not only significantly improves the visual quality of downhole monitoring images but also provides a more reliable data foundation for subsequent image-based intelligent analysis and decision-making.

[0235] It should be noted that "multiple" in this invention includes two or more.

[0236] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0237] Each module in the devices of this invention can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0238] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data required for or generated by the image diffusion enhancement method based on multi-scale feature extraction and fusion. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements an image diffusion enhancement method based on multi-scale feature extraction and fusion.

[0239] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image diffusion enhancement method based on multi-scale feature extraction and fusion. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0240] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0241] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0242] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0243] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0244] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0245] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0246] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0247] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An image diffusion enhancement method based on multi-scale feature extraction and fusion, characterized in that, The method includes: Obtain the image to be processed; Multi-scale feature extraction is performed on the image to be processed to obtain a multi-scale feature map; Pyramid feature fusion is performed on the multi-scale feature maps to obtain a fused feature map; The process involves detecting compressed blocks in the image to be processed and generating block boundary masks, including: performing compression detection on the image to be processed; if the image to be processed conforms to a preset compression mode, converting the image to grayscale and dividing the processed image into pixel blocks of a preset size; if the image to be processed does not conform to the preset compression mode, compressing and converting the image to grayscale sequentially according to the preset compression mode, and dividing the processed image into pixel blocks of a preset size; performing discrete cosine transform on each pixel block to obtain the corresponding discrete cosine coefficient matrix; calculating the sum of the absolute values ​​of the high-frequency regions in each pixel block as the high-frequency energy value of the corresponding pixel block based on each discrete cosine coefficient matrix; traversing the adjacent boundaries of all adjacent pixel blocks in the image to be processed and calculating the absolute difference of the high-frequency energy values ​​of all two pixel blocks with adjacent boundaries; and mapping each absolute difference to a preset interval through a preset function as the corresponding block boundary mask. Based on a preset retrieval algorithm, similar reference images of the image to be processed are retrieved from a natural image reference library, and reference features of the similar reference images are extracted. Using fused feature maps, block boundary masks, and reference features as conditions, a conditional latent space diffusion model is employed to enhance the image to be processed, outputting an enhanced image. The process includes: encoding the image to be processed into the latent space using a pre-trained encoder to obtain an initial latent space representation; fusing the fused feature maps, block boundary masks, and reference features based on a conditional fusion mechanism to obtain fusion conditions; inputting the initial latent space representation and fusion conditions into the U-Net prediction network within the conditional latent space diffusion model for inverse denoising to predict a denoised latent space representation; and decoding the denoised latent space representation using a pre-trained decoder to output the enhanced image. The method further includes: inputting reference features into the cross-attention conditional injection module, calculating the attention weights between the current layer latent features and the reference features of the U-Net prediction network, generating latent features enhanced by the reference features; and inputting the enhanced latent features as conditions into the U-Net prediction network for the next layer of latent feature enhancement.

2. The method according to claim 1, characterized in that, The process of extracting multi-scale features from the image to be processed to obtain a multi-scale feature map includes: Based on a pre-defined lightweight network model, U is calculated. 2 The target weight values ​​of each convolutional layer in the Net network model; Based on the weight values ​​of each objective, and using U 2 The Net network model calculates the dynamic convolution kernels of each convolutional layer and extracts multi-scale features from the image to be processed based on these dynamic convolution kernels, resulting in a multi-scale feature map. The dynamic convolution kernels are calculated using the following formula: ;in, For the dynamic convolution kernel of the i-th convolutional layer, Let be the target weight values ​​for the i-th convolutional layer. The image to be processed. For the first One basic convolutional kernel, The number of basic convolutional kernels.

3. The method according to claim 1, characterized in that, The process of performing pyramid feature fusion on multi-scale feature maps to obtain a fused feature map includes: The multi-scale feature maps are sorted from high to low spatial resolution to obtain the multi-scale feature map sequence [F1, F2, ..., F]. L ], where F1 is the shallowest feature map with the highest resolution, F L This is the deep feature map with the lowest resolution; From the deepest feature map F L Begin by following these steps sequentially from top to bottom: Upsample the feature map of the current level so that the spatial size of the feature map of the current level is the same as that of the feature map of the next level. Adjust the channel dimension of the feature map at the current level using a 1×1 convolution kernel to ensure that it has the same number of channels as the feature map at the next level. The feature map of the current level, after upsampling and channel adjustment, is added element by element to the feature map of the next level to obtain the fused feature map of the current level. Repeat the above steps until all levels of feature maps have been processed to obtain a multi-scale fused feature pyramid. Spatial attention weights are generated for each feature map in the multi-scale fused feature pyramid using a small attention network; Based on spatial attention weights, the multi-scale fusion feature pyramid is weighted and summed pixel by pixel to obtain the final fusion feature map.

4. The method according to claim 1, characterized in that, The process of retrieving similar reference images of the image to be processed from a natural image reference database based on a preset retrieval algorithm, and extracting reference features from the similar reference images, includes: Based on the attribute conditions of the image to be processed, construction attribute conditions are selected, and a natural image reference library is constructed according to the construction attribute conditions. Based on a preset image retrieval algorithm, similar reference images that are similar to the image to be processed are retrieved from a natural image reference library; A pre-trained image recognition and classification model is used to extract reference features from similar reference images.

5. The method according to claim 1, characterized in that, The method further includes: A training sample library is constructed by selecting a preset number of unpaired first low-quality images and first high-quality images, or by selecting a preset number of paired second low-quality images and second high-quality images. Multi-scale feature extraction is performed on the first or second low-quality image in the training sample library to obtain the first multi-scale feature map; Pyramid feature fusion is performed on the first multi-scale feature map to obtain the first fused feature map; Perform compressed block detection on the first low-quality image or the second low-quality image, and generate the first block boundary mask; Based on a preset retrieval algorithm, a first similar reference image of a first low-quality image or a second low-quality image is retrieved from a natural image reference library, and a first reference feature of the first similar reference image is extracted. Using the first fused feature map, the first boundary mask, and the first reference feature as conditions, the conditional latent space diffusion model is used to enhance the image to be processed, and the enhanced training image is output. Based on the enhanced training images, a hierarchical loss function is calculated, which includes one or more of the following: conditional latent space diffusion loss, pixel loss, perceptual loss, frequency domain loss, edge loss, block artifact suppression loss, and perceptual style alignment loss. The hierarchical loss function is input into the preset model optimizer to calculate the optimization parameters of the conditional latent space diffusion model. Based on the optimization parameters, the parameters of the conditional latent space diffusion model are adjusted to obtain the optimized conditional latent space diffusion model.

6. The method according to claim 5, characterized in that, The method further includes: Based on preset attribute conditions, select the second highest quality image; The second high-quality image is sequentially subjected to low-light, noise, blur, and compression processing to obtain the second low-quality image. Pair the second low-quality image with the second high-quality image to obtain a paired second low-quality image and a paired second high-quality image.

7. An image diffusion enhancement device based on multi-scale feature extraction and fusion, characterized in that, include: The image acquisition module is used to acquire the image to be processed. The first extraction module, connected to the image acquisition module, is used to extract multi-scale features from the image to be processed, and obtain a multi-scale feature map. The fusion module, connected to the first extraction module, is used to perform pyramid feature fusion on multi-scale feature maps to obtain a fused feature map. The compression detection and mask generation module is connected to the image acquisition module and is used to perform compression detection on the image to be processed. If the image to be processed conforms to the preset compression mode, the image to be processed is converted to grayscale, and the processed image to be processed is divided into pixel blocks of a preset size; If the image to be processed does not conform to the preset compression mode, the image to be processed is compressed and grayscaled sequentially according to the preset compression mode, and the processed image to be processed is divided into pixel blocks of a preset size. Perform a discrete cosine transform on each pixel block to obtain the corresponding discrete cosine coefficient matrix; based on each discrete cosine coefficient matrix, calculate the sum of the absolute values ​​of the high-frequency regions in each pixel block as the high-frequency energy value of the corresponding pixel block; Traverse the adjacent boundaries of all adjacent pixel blocks in the image to be processed, calculate the absolute difference of the high-frequency energy values ​​of all two pixel blocks with adjacent boundaries, and map each absolute difference to a preset interval through a preset function as the corresponding block boundary mask. The second extraction module, connected to the image acquisition module, is used to retrieve similar reference images of the image to be processed from the natural image reference library based on a preset retrieval algorithm, and extract reference features of the similar reference images. The conditional latent space diffusion model is connected to the image acquisition module, the fusion module, the compression detection and mask generation module, and the second extraction module, respectively. It is used to encode the image to be processed into the latent space using a pre-trained encoder to obtain the initial latent space representation. Based on the conditional fusion mechanism, the fusion feature map, block boundary mask and reference features are fused to obtain the fusion condition; the initial latent space representation and the fusion condition are input together into the U-Net prediction network in the conditional latent space diffusion model for reverse denoising, and the denoised latent space representation is predicted; the denoised latent space representation is decoded using a pre-trained decoder to output the enhanced image; The cross-attention conditional injection module is connected to the second extraction module and the conditional latent space diffusion model, respectively. It is used to input the reference features into the cross-attention conditional injection module, calculate the attention weight between the current layer latent features and the reference features in the U-Net prediction network in the conditional latent space diffusion model, and generate latent features enhanced by the reference features. The enhanced latent features are then used as conditional inputs into the U-Net prediction network in the conditional latent space diffusion model for the next layer of latent feature enhancement.

Images