Diffusion image super-division method and device driven by implicit material library

Abstract

The invention provides an implicit material library-driven diffusion image super-division method, which comprises the following steps of: constructing an image super-division model consisting of an encoder and a decoder, carrying out compressed encoding on a low-resolution picture to obtain a compressed feature, and carrying out nearest neighbor search on a dimensional feature in each range in a codebook to find a nearest discrete feature; decoding is carried out through a decoder to obtain a reconstructed image, the picture content of the original image is taken as a target, a loss function is constructed according to the content difference between the reconstructed image and the original image, and the encoder, the decoder and a codebook are trained; and inputting a to-be-super-divided image into the trained image super-division model to obtain a super-division result image.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution technology in computer vision, and particularly to an implicit material library-driven diffusion super-resolution method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Image super-resolution refers to the process of recovering a high-resolution image from a low-resolution image or image sequence. Early image super-resolution methods, such as bilinear interpolation, employed reconstruction-based approaches with limited effectiveness. With the rapid development of deep learning technology, numerous deep learning-based super-resolution methods have emerged. Representative works, such as SRCNN and EDSR based on convolutional neural networks and SwinIR and HAT based on Transformers, focus on building more efficient backbone network architectures. In pursuing higher super-resolution results, these methods often overemphasize fidelity as a single dimension. They commonly use pixel-level L1 or MSE loss functions for training. While this ensures a certain degree of similarity in pixel values ​​between the generated image and the original high-resolution image, it results in overly smoothed details in the model's output. Subsequent methods have gradually recognized the importance of generating more perceptually realistic details and have actively explored this area. Some works have attempted to introduce SSIM loss and perceptual loss into the training process. The introduction of these two loss functions has effectively improved the perceptual quality of the super-resolution results to a certain extent, making the generated image visually closer to human subjective perception. With the rise of Generative Adversarial Networks (GANs), numerous subsequent studies, such as SRGAN, ESRGAN, CodeFormer, and FeMaSR, have actively incorporated adversarial training strategies. These strategies introduce adversarial loss into the training process, constructing a game-theoretic architecture where the generator and discriminator compete against each other. This approach can deeply mine the inherent distributional characteristics of image data, powerfully driving the model to generate more nuanced and richer details, thereby significantly improving the perceptual quality of the generated images and making them more visually consistent with human subjective visual perception. However, the training process of GAN-based methods is extremely unstable and prone to pattern collapse.

[0003] Recently, the Denoising Diffusion Model (DDPM) and its variant DDIM have demonstrated superior ability to fit more complex and diverse distributions, making them an ideal alternative to GANs. The diffusion model is primarily defined based on a diffusion process, comprising forward diffusion and backward diffusion. First, let's consider the forward diffusion process, assuming the original image data is... In time step At that time, the image The process can be gradually transformed into pure noise in the following ways: ;

[0004] in It is a series of pre-defined variance scheduling parameters. It is standard normally distributed noise. The reverse diffusion process, on the other hand, involves noise originating from pure noise. Gradually restore the original image The process involves learning a denoising model. To predict at time step Add to image Noise in Thus, through iterative calculation: ;

[0005] Gradually restore the original image The diffusion model, through its forward noise addition and backward denoising process, effectively models and generates complex distributions of image data, demonstrating unique advantages and potential in tasks such as image super-resolution. Compared to other super-resolution algorithms, the diffusion super-resolution model has significant advantages in terms of strong detail recovery capability, high training stability, better generation quality, and better integration of multimodal conditions.

[0006] Although diffusion super-resolution models have shown significant advantages over other super-resolution models in the field of image super-resolution, their development still faces many key problems that urgently need to be overcome.

[0007] On the one hand, the problem of missing effective texture features remains unresolved. Current diffusion super-resolution methods such as SRDiff, SR3, and SR3+ simply use the original low-resolution image as a condition, and the reconstruction process is highly dependent on the information of the low-resolution image itself. However, due to image degradation, a large amount of high-frequency detail information is lost, resulting in a severe lack of effective texture features that can be used to guide super-resolution. Image degradation refers to the phenomenon that an image becomes blurrier, has lower resolution, and exhibits noise, color distortion, or compression artifacts during the shooting, transmission, or processing process due to various reasons. Common degradations include: blurring (due to lens blur, atmospheric effects, or motion), downsampling / lower resolution, and increased noise (random points generated by the sensor). Furthermore, image degradation not only causes the loss of effective texture features but also introduces additional noise information. This noise is easily misidentified as the true features of the image during model reconstruction, thus interfering with the super-resolution process and leading to serious problems such as texture distortion, blurred edges, and detail distortion in the reconstructed high-resolution image. Some diffusion super-resolution methods, such as StableSR, DiffBir, and SUPIR, use efficient networks to extract features from low-resolution images as conditional super-resolution, but they still cannot overcome the limitations of low-resolution images. Other methods, such as PASD and SeeSR, introduce semantic information into the super-resolution process, providing more clues through multimodal fusion. However, none of the above methods can explicitly provide more effective texture features.

[0008] On the other hand, many existing methods commonly employ ControlNet's conditional introduction mechanism. This approach preserves the parameters of the pre-trained model while simultaneously replicating the parameters for feature extraction, achieving feature fusion through zero-convolution. While this structure is effective in some tasks, it has significant drawbacks. First, it requires a large number of additional trainable parameters, significantly increasing computational resource consumption and training complexity, leading to inefficient model training. More critically, during feature fusion, ControlNet incorporates input conditional information into the diffusion model through a simple feature summation operation. Although this method performs well when processing coarse-grained information (such as keypoint maps), it is significantly inadequate when integrating fine-grained material information such as texture. Its feature summation operation fails to fully consider the complex relationships and interactions between different features, resulting in inefficient feature fusion and an inability to fully utilize complementary information between features. Therefore, the model struggles to accurately recover image details in super-resolution tasks, and the generated high-resolution images often lack fine texture information. Summary of the Invention

[0009] The purpose of this invention is to overcome the problems of insufficient effective texture features (relying only on low-resolution images with severe information degradation) and inefficient conditional introduction mechanisms in existing diffusion super-resolution algorithms. This invention proposes an implicit material library-driven diffusion super-resolution algorithm and proposes an efficient texture feature self-attention module to integrate texture features into the diffusion model.

[0010] To address the shortcomings of existing technologies, such as Figure 8 As shown, this invention proposes an implicit material library-driven diffusion image super-resolution method, which includes:

[0011] The initial step involves constructing an image super-resolution model consisting of an encoder and a decoder to compress high-resolution color images. A low-resolution image is obtained, and the encoder compresses and encodes the low-resolution image to obtain compressed features. , and These are the spatial dimensions of the feature map. It is the dimension of the feature vector;

[0012] Search steps, for Each of them Within range Dimensional features in the codebook Perform a nearest neighbor search to find the closest discrete feature. The codebook ,in Indicates the codebook size; where each compression feature... Mapped to the corresponding codebook number , representing compression features In the codebook The closest discrete feature ;

[0013] The training steps involve using this decoder to... Decode the image to obtain the reconstructed image. With the original image The image content is the target, and the reconstructed image is based on this. With the original image Based on the differences in content, a loss function is constructed, and the trained image super-resolution model is obtained by training the encoder, the decoder, and the codebook;

[0014] The super-resolution step involves inputting the image to be super-resolution into the trained image super-resolution model to obtain a super-resolution result image of the image to be super-resolution, the resolution of which is higher than that of the image to be super-resolution.

[0015] The implicit material library-driven diffusion image super-resolution method, wherein the process of constructing the loss function in the training step includes:

[0016] Construct a pixel loss to measure the difference between the reconstructed image and the original image at the pixel level. : ;

[0017] Constructing perceived loss : ,in, This represents the features extracted through a convolutional neural network;

[0018] Building against loss , ;

[0019] The codebook loss is constructed by measuring the distance between the encoder's output feature vector and the corresponding discrete features in the codebook. : ,in For loss weights;

[0020] Through this pixel loss This perceived loss The losses incurred in the fight and the codebook loss The encoder, decoder, and codebook are jointly optimized.

[0021] The implicit material library-driven diffusion image super-resolution method, wherein the search step includes:

[0022] The compressed feature is obtained through a classification network. The matching probabilities of N vectors in the codebook are used, and the vector with the highest probability is taken as the discrete feature. Based on this discrete characteristic As labels, calculate classification loss. Consistency loss of intermediate features with encoder , to train the codebook;

[0023] Classification loss ,in, It is a high-resolution color image The codebook index obtained through nearest neighbor search This is the matching probability; The low-resolution image is in the encoder's... The intermediate features of the layer are This high-resolution image The intermediate features in the corresponding layer are , This indicates the number of intermediate feature layers in the encoder.

[0024] The implicit material library-driven diffusion image super-resolution method further includes: the self-attention module in the encoder and decoder is a texture feature fusion self-attention module TFSA, and the training process of the self-attention module TFSA includes:

[0025] Input features of the self-attention module TFSA The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix By calculating attention scores Then, the calculated attention score is used. Log-value matrix Perform a weighted summation to obtain the original self-attention result. ;

[0026] The low-resolution image was used to extract multi-scale texture feature priors from a texture library. ,in The scale of the feature is represented; for each scale of texture feature, a weight matrix is ​​used. and Transform it to obtain the corresponding keys. Sum : ;

[0027] Query Matrix With the key matrix at every scale Perform attention calculations to obtain attention results. : , Attention output Perform group normalization ,get Number of sampling steps Encode to obtain control coefficients and and the normalized features Perform scaling and translation operations: Through a zero convolutional layer The results after scaling and translation The process is performed to obtain the final output features. The output feature Compared with the original self-attention results Add them together to get the final output of the TFSA module. .

[0028] like Figure 9 As shown, this invention also proposes an implicit material library-driven diffusion image super-resolution device, which includes:

[0029] The initial module constructs an image super-resolution model consisting of an encoder and a decoder to compress high-resolution color images. A low-resolution image is obtained, and the encoder compresses and encodes the low-resolution image to obtain compressed features. , and These are the spatial dimensions of the feature map. It is the dimension of the feature vector;

[0030] The search module, for Each of them Within range Dimensional features in the codebook Perform a nearest neighbor search to find the closest discrete feature. The codebook ,in Indicates the codebook size; where each compression feature... Mapped to the corresponding codebook number , representing compression features In the codebook The closest discrete feature ;

[0031] The training module, through this decoder, Decode the image to obtain the reconstructed image. With the original image The image content is the target, and the reconstructed image is based on this. With the original image Based on the differences in content, a loss function is constructed, and the trained image super-resolution model is obtained by training the encoder, the decoder, and the codebook;

[0032] The super-resolution module inputs the image to be super-resolution into the trained image super-resolution model to obtain a super-resolution result image of the image to be super-resolution, the resolution of which is higher than that of the image to be super-resolution.

[0033] The implicit material library-driven diffusion image super-resolution device, wherein the process of constructing the loss function in the training module includes:

[0034] Construct a pixel loss to measure the difference between the reconstructed image and the original image at the pixel level. : ;

[0035] Constructing perceived loss : ,in, This represents the features extracted through a convolutional neural network;

[0036] Building against loss , ;

[0037] The codebook loss is constructed by measuring the distance between the encoder's output feature vector and the corresponding discrete features in the codebook. : ,in For loss weights;

[0038] Through this pixel loss This perceived loss The losses incurred in the fight and the codebook loss The encoder, decoder, and codebook are jointly optimized.

[0039] The implicit material library-driven diffusion image super-resolution device, wherein the search module includes:

[0040] The compressed feature is obtained through a classification network. The matching probabilities of N vectors in the codebook are used, and the vector with the highest probability is taken as the discrete feature. Based on this discrete characteristic As labels, calculate classification loss. Consistency loss of intermediate features with encoder , to train the codebook;

[0041] Classification loss ,in, It is a high-resolution color image The codebook index obtained through nearest neighbor search This is the matching probability; The low-resolution image is in the encoder's... The intermediate features of the layer are This high-resolution image The intermediate features in the corresponding layer are , Indicates the number of intermediate feature layers in the encoder;

[0042] The self-attention module in the encoder and decoder is a texture feature fusion self-attention module (TFSA). The training process of the TFSA self-attention module includes:

[0043] Input features of the self-attention module TFSA The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix By calculating attention scores Then, the calculated attention score is used. Log-value matrix Perform a weighted summation to obtain the original self-attention result. ;

[0044] The low-resolution image was used to extract multi-scale texture feature priors from a texture library. ,in The scale of the feature is represented; for each scale of texture feature, a weight matrix is ​​used. and Transform it to obtain the corresponding keys. Sum : ;

[0045] Query Matrix With the key matrix at every scale Perform attention calculations to obtain attention results. : , Attention output Perform group normalization ,get Number of sampling steps Encode to obtain control coefficients and and the normalized features Perform scaling and translation operations: Through a zero convolutional layer The results after scaling and translation The process is performed to obtain the final output features. The output feature Compared with the original self-attention results Add them together to get the final output of the TFSA module. .

[0046] The present invention also proposes an electronic device, including the aforementioned implicit material library-driven diffusion image super-resolution device, which may be connected to an information display device for displaying the super-resolution result image with user-set display parameters, attributes, or through an artificial intelligence model.

[0047] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the implicit material library-driven diffusion image super-resolution method.

[0048] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the implicit material library-driven diffusion image super-resolution method.

[0049] As can be seen from the above solutions, the advantages of the present invention are:

[0050] Overall, our proposed implicit material library-driven diffusion super-resolution algorithm brings numerous conveniences and improvements to image super-resolution. Using the algorithm presented in this paper, the problems of insufficient effective texture prior features and inefficient conditional introduction mechanisms in existing diffusion super-resolution models are largely resolved, and the performance of image super-resolution is significantly improved.

[0051] This paper trains the model on the first 10K images from LSDIR, DIV2K, Flickr2K, and FFHQ. A degradation process using Real-ESRGAN is used to synthesize LR-HR image pairs for training. For a comprehensive evaluation of ISR, both synthetic and real-world datasets are used for testing. The synthetic dataset is generated from the DIV2K validation set by randomly cropping 3K blocks from the original images and then synthesizing the validation set using the same degradation process as training. For the real-world datasets, we evaluate using the DRealSR and RealSR datasets, with each image centrally cropped. Each HR image in both the training and test sets has a resolution of 512 × 512.

[0052] A comprehensive quantitative evaluation of the ISR method was conducted using a range of widely used reference and non-reference metrics. Among the reference-based metrics, PSNR and SSIM (calculated on the Y channel in the YCbCr space) are fidelity metrics, while LPIPS and DISTS are quality assessment metrics. FID calculates the distance between the distributions of the generated image and the reference image. MANIQA, MUSIQ, and CLIPIQA are non-reference image quality assessment (IQA) metrics.

[0053] To verify the effectiveness of the method, it was compared with GAN-based methods Real ESRGAN, BSRGAN, and HAT-GAN, and diffusion model-based methods LDM, StableSR, and DiffBIR. The performance on the DIV2K validation set is as follows: Figure 5 As shown, the performance of RealSR and DRealSR on the real dataset is as follows: Figure 6 and Figure 7As shown, experimental results demonstrate that our proposed method significantly outperforms other GAN- or diffusion-based methods on subjective quality assessment metrics such as MANIQA, MUSIQ, and CLIPIQA. This is attributed to the implicit material library-driven texture prior guidance mechanism, which provides high-quality texture priors through the VQGAN encoding and decoding process and optimizes the retrieval effect through the feature alignment module, making the texture features more closely match the target image. Simultaneously, the TFSA module efficiently integrates multi-scale texture priors and contextual information, optimizing detail restoration and structural consistency. These improvements effectively compensate for the lack of texture information in low-resolution images, resulting in generated images with excellent realism and detail reproduction, fully validating the superiority of our proposed method. Attached Figure Description

[0054] Figure 1 Material library construction diagram;

[0055] Figure 2 Feature alignment optimizes the feature prior retrieval graph for low-scoring images;

[0056] Figure 3 Texture feature fusion self-attention module diagram;

[0057] Figure 4 Graph of texture prior-assisted diffusion super-resolution algorithm driven by implicit material library;

[0058] Figure 5 Performance comparison of different super-resolution methods on the DIV2K-Val dataset;

[0059] Figure 6 Performance comparison of different super-resolution methods on the RealSR dataset;

[0060] Figure 7 Performance comparison of different super-resolution methods on the DRealSR dataset;

[0061] Figure 8 This is a flowchart of the method of the present invention;

[0062] Figure 9 This is a block diagram of the device of the present invention;

[0063] Figure 10 This is a schematic diagram of the structure of the first electronic device of the present invention;

[0064] Figure 11 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0065] Figure 12 This is a schematic diagram of the structure of the second electronic device of the present invention.

[0066] Figure label:

[0067] A - First electronic device;

[0068] B-implicit material library driven image diffusion super-resolution device;

[0069] C-Data acquisition equipment;

[0070] D-Information display device;

[0071] 1000 - Second electronic device;

[0072] Ⅰ-Computational Unit;

[0073] II-ROM;

[0074] III-RAM;

[0075] N-bus;

[0076] V-Interface;

[0077] VI - Input Unit;

[0078] VII - Output Unit;

[0079] VIII - Storage medium;

[0080] IX - Communication Unit. Detailed Implementation

[0081] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0082] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0083] The processor described in this invention is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of this invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0084] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0085] In a specific implementation, as one example, the processor may include one or more CPUs. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include servers, desktop computers, laptops, smartphones, tablets, embedded computers, etc., where the embedded computer includes vehicles and robots, etc.

[0086] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0087] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0088] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0089] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0090] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0091] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0095] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Considering the shortcomings of existing technologies, we found that the main technical bottlenecks lie in two aspects: the lack of effective texture features and the inefficiency of the conditional introduction mechanism during the training of the diffusion super-resolution model. Regarding the former, inspired by super-resolution algorithms based on reference images, these methods provide several additional high-resolution images with highly correlated textures during the super-resolution process. The texture features of these reference images are used to supplement the super-resolution process and achieve better results. However, this type of method involves matching low-resolution features with reference images, and its complexity is proportional to the number of reference images. Furthermore, it requires high texture similarity with the reference images, and in most cases, sufficiently similar reference images cannot be provided. Considering both the advantages and limitations of this type of algorithm, we propose constructing an implicit texture library to provide more effective texture features for the super-resolution process without the need for additional reference images. Regarding the inefficiency of the super-resolution conditional introduction mechanism, existing ControlNet-based methods can retain as much image knowledge as possible from the pre-trained model, but simply introducing conditions through feature summation is inefficient. Therefore, we improve upon ControlNet by proposing a texture feature fusion self-attention module to achieve efficient conditional introduction. In summary, to achieve the above-mentioned technical effects, the present invention proposes the following key technical points:

[0097] Key point 1: Our algorithm proposes to provide more effective texture prior features for the diffusion super-resolution model by constructing an implicit material library, which effectively makes up for the shortcomings of the current algorithm that only relies on low-resolution images with severe information degradation for super-resolution.

[0098] Key point 2: Considering that current algorithms all introduce texture features conditionally through a structure similar to ControlNet, which is relatively inefficient, our proposed texture feature fusion self-attention module can effectively make up for this and introduce conditions efficiently.

[0099] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0100] The key to this method is to provide effective texture prior information for low-resolution images by constructing an implicit texture material library, and to combine it with optimized feature retrieval and efficient feature integration modules to achieve high-quality super-resolution image reconstruction.

[0101] When building an implicit asset library, such as Figure 1As shown, we first examine high-resolution color images. Encode it and map it to a compressed feature representation through the encoder. ,in and These are the spatial dimensions of the feature map. This refers to the dimension of the feature vector, the value of which is determined by the network design. Because in the feature space, the dimension no longer represents color, but rather the feature dimension itself. For example, the first channel (dimension) represents "edge," the second channel represents "texture," the third channel represents "corner," etc. Therefore, the model will learn dozens to hundreds of different feature dimensions. Next, following the methods of Vector Quantized Variational Autoencoder (VQVAE) and Vector Quantized Generative Adversarial Network (VQGAN), we will... Each of them Within range The feature is used to perform a nearest neighbor search to find the closest discrete feature. This process is assisted by a learnable codebook. ,in Representing the size of the codebook, we obtain discrete features. Each feature vector It will be mapped to the corresponding codebook number. This represents the closest discrete feature in the codebook, where i and j represent the row and column indices of the matrix, respectively.

[0102] The compressed features mentioned above refer to the feature representations formed after an image is encoded by a deep neural network (such as a convolutional network or a variational autoencoder), which have a lower dimension than the original image. These features contain important structural and texture information of the image, but no longer directly correspond to the visualized pixel data. For example, a 256×256×3 image input to an encoder can yield a compressed feature of 32×32×16.

[0103] A codebook is a set of learnable vectors used to represent the discrete latent features of an image. The network "quantizes" continuous feature vectors into a single vector from the codebook, thus achieving a "discrete" latent space. The model selects these vectors to represent the image content. This represents the entire codebook, which is a set containing N vectors. This represents the k-th vector in the codebook, each... Both are d-dimensional feature vectors. During model quantization, the input feature vector is "replaced" with the closest one. . It represents the d-dimensional real vector space.

[0104] Then, through the decoder Decode the image to obtain the reconstructed image. The image is the same as the original image. As close as possible. The reconstructed image has a higher resolution than the original.

[0105] It's important to note that the reconstructed images here are not from the implicit resource library. The implicit resource library is equivalent to the trained neural network itself. The network "absorbs" a large number of training images into its parameters, forming an implicit, compressed, latent resource representation. These resources are not explicit images, but rather visual patterns implied in the network parameters. There is no corresponding original image available during inference (generating new images). Therefore, the model cannot rely on the "original image" but must rely on the "reconstructed image" generated by the model itself through the encoder, latent space, and decoder.

[0106] Both the encoder and decoder are built based on ResNet modules and self-attention modules. To train the encoder, decoder, and codebook, we use a weighted sum of the following loss functions:

[0107] Pixel loss : Used to measure the difference between the reconstructed image and the original image at the pixel level, the formula is: .

[0108] Perceived loss The perceptual differences between images are measured based on features extracted by the VGG convolutional neural network. The formula is as follows: ,in, This represents the features extracted through the VGG network.

[0109] Combating losses Generative Adversarial Networks (GANs) are used to further optimize image quality and enhance the naturalness of generated images. The adversarial loss can be measured by a discriminator to differentiate the generated image from the real image. .

[0110] In the formula Represents true high-resolution images, derived from real data distributions. ; The reconstructed image generated by the model comes from the generation distribution. ; It's the discriminator, used to determine whether an image is real or generated. The formula embodies typical GAN ​​adversarial training: the first term instructs the discriminator to classify real images as real, and the second term instructs the discriminator to classify generated images as fake. During training, the generator will inversely minimize this loss, making the generated images "fool" the discriminator as much as possible, thereby improving the realism of the generated images. This is the embodiment of adversarial training.

[0111] Codebook loss Regularization is applied to codebook updates to prevent the model from falling into pattern collapse and to ensure the diversity of generated features. This loss is calculated by measuring the distance between the encoder's output feature vector and the corresponding discrete features in the codebook. ,in To lose weight, Stop gradient refers to the gradient stopping during training, which is a training technique.

[0112] The entire training process uses the aforementioned loss function to jointly optimize the encoder, decoder, and codebook, thereby obtaining a material library that can effectively extract image texture features, and thus providing texture prior information for subsequent super-resolution tasks.

[0113] In addition, feature matching training is needed in the material library to make feature retrieval for low-scoring images in the material library more accurate. The flowchart is as follows. Figure 2 As shown, in cases of low-resolution image degradation, features obtained directly from the image library using a simple nearest neighbor matching method are often inaccurate. This is because low-resolution images typically suffer from detail loss and blurring, leading to significant matching errors between features extracted from low-resolution images and features from high-resolution images in the image library. Therefore, to optimize the feature retrieval process for low-resolution images, we introduce a codebook matching module between the encoder and decoder. This module, employing a classification network, performs nearest neighbor search to improve the accuracy of feature matching.

[0114] This codebook matching module is essentially a Classification networks aim to provide features for each low-scoring image. Assign a best-matching codebook index. This module uses the features obtained from the encoder on the input low-resolution image. It will pass through the codebook matching module and output This corresponds to the matching probability of N vectors in the codebook, and yields the corresponding matching result. ,in It represents the size of the codebook in the resource library, indicating the number of discrete vectors that can be matched for each feature dimension. This represents the result calculated / predicted by the network (the estimated value given by the model). It is an eigenvalue matrix. These are index tags, which can only be obtained by indexing the quantization features in the codebook C. .

[0115] Next, we use the codebook index obtained from the high-resolution image through nearest neighbor search. As the Ground Truth (GT), and calculating the loss, the high-resolution image and the previously mentioned low-resolution image, having the same content, indicate different resolutions. This is the second stage of training, used to train the encoder and codebook matching module, and to fine-tune the encoder and train the codebook matching module. The specific loss consists of two parts: one is the classification loss. One is used to optimize the predicted class labels; the other is the encoder intermediate feature consistency loss, which is used to optimize the feature matching accuracy.

[0116] Classification loss The expression is: ,in, It is the codebook index obtained from the high-resolution image through nearest neighbor search. It represents the probability of the predicted class label output by the classification network after the low-resolution image is processed.

[0117] For the intermediate feature consistency loss, let the low-resolution image be in the encoder's... The intermediate features of the layer are The intermediate features of a high-resolution image at the corresponding layer are The element-wise loss can then be expressed as: .in, This indicates the number of intermediate feature layers in the encoder. Low-resolution features are optimized layer by layer. With high-resolution features The differences between these elements can significantly improve the encoder's ability to accurately model features of low-resolution images, thereby enhancing the accuracy of feature alignment. This element-wise feature map loss provides a more refined supervisory signal for the feature representation of low-resolution images.

[0118] Through the training described above, we can effectively optimize the feature matching between low-resolution images and the material library, improve the accuracy of low-resolution image features, and thus provide more accurate texture information for the super-resolution model, thereby enhancing the super-resolution effect.

[0119] To more efficiently integrate texture features obtained from the texture library into the diffusion model, we designed an improved self-attention layer, the Texture Feature Fusion SelfAttention (TFSA) module. Specific implementation details are as follows... Figure 3As shown. The core objective of this module is to enhance feature fusion efficiency while retaining key feature information, thereby reducing model parameters and computational cost. The self-attention layer is a fundamental layer in the neural network. The encoder and decoder are large modules in the entire network pipeline, and they also contain self-attention layers, linear layers, residual connections, convolutional layers, and so on. Here, a self-attention layer is used to inject texture information when integrating a texture library into the diffusion model.

[0120] In traditional self-attention mechanisms, for input features H, W, and D represent the height, width, and number of channels of the input features, respectively. The self-attention layer calculates the correlation between each location to obtain global feature information fusion. Specifically, the input features are first transformed linearly to obtain a query, key, and value matrix, denoted as [Matrices to be inserted here]. The attention score is calculated using the following formula: Then, the calculated attention scores are used to perform a weighted summation of the value matrix V to obtain the final output features: .

[0121] In the TFSA module, low-resolution images are used to extract multi-scale texture feature priors from a texture library. ,in The texture library represents the scale of the features and contains texture mapping information from low-resolution images to high-resolution images. The dataset used is a publicly available internet dataset. This texture library refers to a neural network model trained on this dataset that learns the texture mapping relationship from low-resolution images to high-resolution images. For each scale of texture feature, we use a trainable weight matrix. and Transform it to obtain the corresponding keys. Sum :

[0122] Next, the query matrix obtained from the backbone network is used. We perform attention calculations with the key at each scale to obtain the attention results. : , Then attention output Perform Group Normalization The degree of control over texture features should vary depending on the number of sampling steps. Therefore, we will determine the number of sampling steps. Encode to obtain control coefficients and And perform scaling and shifting operations on the normalized features: Through a zero convolutional layer The results after scaling and translation are processed to obtain the final output features. The output is then added to the original self-attention result to obtain the final output of the TFSA module. : .

[0123] The sampling steps refer to the number of times the diffusion model samples from standard Gaussian noise during image generation. The inverse diffusion process (predicting noise at each step and adding it to the final result) eventually generates an image. This refers to the number of sampling steps in the reverse diffusion process. It can take any positive integer value greater than 1.

[0124] In this way, the TFSA module can effectively fuse multi-scale texture features from the texture library and precisely control the influence of textures in each sampling step, optimizing the feature fusion process and improving the detail recovery capability in super-resolution tasks.

[0125] By replacing the self-attention module in the SD model with the TFSA module, super-resolution can be achieved under the condition of efficiently fusing texture feature priors, thereby significantly improving the model's performance.

[0126] The overall architecture of the method is as follows Figure 4 As shown, where Figure 4 The content of the top left corner screen is Figure 1 The content in the upper right corner of the screen is Figure 2 , Figure 4 The content of the lower right screen is Figure 3 .

[0127] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0128] like Figure 9 As shown, this invention also proposes an implicit material library-driven diffusion image super-resolution device, which includes:

[0129] The initial module constructs an image super-resolution model consisting of an encoder and a decoder to compress high-resolution color images. A low-resolution image is obtained, and the encoder compresses and encodes the low-resolution image to obtain compressed features. , and These are the spatial dimensions of the feature map. It is the dimension of the feature vector;

[0130] The search module, for Each of them Within range Dimensional features in the codebook Perform a nearest neighbor search to find the closest discrete feature. The codebook ,in Indicates the codebook size; where each compression feature... Mapped to the corresponding codebook number , representing compression features In the codebook The closest discrete feature ;

[0131] The training module, through this decoder, Decode the image to obtain the reconstructed image. With the original image The image content is the target, and the reconstructed image is based on this. With the original image Based on the differences in content, a loss function is constructed, and the trained image super-resolution model is obtained by training the encoder, the decoder, and the codebook;

[0132] The super-resolution module inputs the image to be super-resolution into the trained image super-resolution model to obtain a super-resolution result image of the image to be super-resolution, the resolution of which is higher than that of the image to be super-resolution.

[0133] The implicit material library-driven diffusion image super-resolution device, wherein the process of constructing the loss function in the training module includes:

[0134] Construct a pixel loss to measure the difference between the reconstructed image and the original image at the pixel level. : ;

[0135] Constructing perceived loss : ,in, This represents the features extracted through a convolutional neural network;

[0136] Building against loss , ;

[0137] The codebook loss is constructed by measuring the distance between the encoder's output feature vector and the corresponding discrete features in the codebook. : ,in For loss weights;

[0138] Through this pixel loss This perceived loss The losses incurred in the fight and the codebook loss The encoder, decoder, and codebook are jointly optimized.

[0139] The implicit material library-driven diffusion image super-resolution device, wherein the search module includes:

[0140] The compressed feature is obtained through a classification network. The matching probabilities of N vectors in the codebook are used, and the vector with the highest probability is taken as the discrete feature. Based on this discrete characteristic As labels, calculate classification loss. Consistency loss of intermediate features with encoder , to train the codebook;

[0141] Classification loss ,in, It is a high-resolution color image The codebook index obtained through nearest neighbor search This is the matching probability; The low-resolution image is in the encoder's... The intermediate features of the layer are This high-resolution image The intermediate features in the corresponding layer are , Indicates the number of intermediate feature layers in the encoder

[0142] The self-attention module in the encoder and decoder is a texture feature fusion self-attention module (TFSA). The training process of the TFSA self-attention module includes:

[0143] Input features of the self-attention module TFSA The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix By calculating attention scores Then, the calculated attention score is used. Log-value matrix Perform a weighted summation to obtain the original self-attention result. ;

[0144] The low-resolution image was used to extract multi-scale texture feature priors from a texture library. ,in The scale of the feature is represented; for each scale of texture feature, a weight matrix is ​​used. and Transform it to obtain the corresponding keys. Sum :

[0145] Query Matrix With the key matrix at every scale Perform attention calculations to obtain attention results. : , Attention output Perform group normalization ,get Number of sampling steps Encode to obtain control coefficients and and the normalized features Perform scaling and translation operations: Through a zero convolutional layer The results after scaling and translation The process is performed to obtain the final output features. The output feature Compared with the original self-attention results Add them together to get the final output of the TFSA module. .

[0146] like Figure 10 As shown, in another embodiment of the present invention, a first electronic device A is also proposed, which includes the aforementioned implicit material library driven diffusion image super-resolution device B.

[0147] like Figure 11 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to acquire the image to be super-resolved, and the information display device D is used to display the super-resolved result image obtained by the present invention.

[0148] The information display device D can process and organize the data output by the first electronic device A based on an information display mechanism to improve the readability of the data. This information display mechanism can be manually preset, for example, visualizing the data output by the first electronic device A. It can present the user with the specified key information based on user-defined display parameters and / or attributes, such as the data range and font, color, and scrolling options. Users can access this information more quickly without needing to navigate to secondary pages or scroll through pages, saving them time and effort. Alternatively, the information display mechanism can be an artificial intelligence (AI) display model that learns the user's key information interests based on past usage habits, such as viewing time, click count, and edit count, and automatically presents rich and necessary key information.

[0149] The present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the implicit material library-driven diffusion super-resolution method provided by the above methods.

[0150] In another embodiment, the present invention also proposes a storage medium VIII for storing a computer program that executes the implicit material library-driven diffusion super-resolution method. It should be understood that the storage medium in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0151] Figure 12 A schematic block diagram of a second electronic device 1000 that can be used to implement embodiments of the present invention is shown. The second electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein. The second electronic device 1000 may be the same as or different from the first electronic device A.

[0152] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from storage medium VIII into random access memory (RAM) III. The RAM III may also store various programs and data required for the operation of the device 1000. The computing unit I, ROM II, and RAM III are interconnected via bus IV. An input / output (I / O) interface V is also connected to bus IV.

[0153] Multiple components in the second electronic device 1000 are connected to I / O interface V, including: input unit VI, such as a keyboard, mouse, etc.; output unit VII, such as various types of displays, speakers, etc.; storage medium VIII, such as a disk, optical disk, etc.; and communication unit IX, such as a network card, modem, wireless transceiver, etc. Communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0154] The computing unit I can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to perform methods by any other suitable means (e.g., by means of firmware).

[0155] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. An implicit material library driven diffusion image super-resolution method, characterized in that, The method comprises: Initial step, construct an image super-resolution model composed of an encoder and a decoder, compress high-resolution color images Get a low-resolution picture, compress and encode the low-resolution picture through the encoder to get compressed features , are the spatial dimensions of the feature map respectively, is the dimension of the feature vector;​ Search steps, for Each of them Within range Dimensional features in the codebook Perform a nearest neighbor search to find the closest discrete feature. The codebook ,in Indicates the codebook size; where each compression feature... Mapped to the corresponding codebook number , representing compression features In the codebook The closest discrete feature ; a training step, decoding, by the decoder, to obtain a reconstructed image a training step, decoding, by the decoder, to obtain a reconstructed image a training step, decoding, by the decoder, to obtain a reconstructed image a training step, decoding, by the decoder, to obtain a reconstructed image a training step, decoding, by the decoder, to obtain a reconstructed image a training step, decoding, by the decoder, to obtain a reconstructed image The process of constructing the loss function in the training step comprises:

2. The implicit footage library driven diffusion image super-resolution method of claim 1, wherein, The search step comprises: constructing a pixel loss for measuring the difference between the reconstructed image and the original image at the pixel level : ; Constructing perceptual loss : , wherein, denotes features extracted by a convolutional neural network; Constructing adversarial loss , ; The codebook loss is constructed by measuring the distance between the encoder output feature vector and the corresponding discrete feature in the codebook : wherein is a loss weight; The encoder, the decoder and the codebook are jointly optimized by the pixel loss , the perceptual loss , the adversarial loss and the codebook loss .

3. The implicit stock library driven diffusion image super-resolution method of claim 1, wherein, The method further comprises: The compressed feature is obtained through the classification network The matching probability of N vectors in the corresponding codebook is obtained, and the vector with the maximum probability is taken as the discrete feature ; According to the discrete feature As a label, the classification loss is calculated And the consistency loss of the intermediate feature in the encoder To train the codebook; Classification loss wherein, is a high resolution color image is the codebook index obtained by the nearest neighbor search, is the matching probability; wherein the low resolution picture is at the input of the encoder the intermediate features of the layers are the high resolution picture is the intermediate features of the corresponding layers are , denotes the number of intermediate feature layers in the encoder.

4. The implicit stock library driven diffusion image super-resolution method of claim 1, wherein, The self-attention module in the encoder and the decoder is a texture feature fusion self-attention module TFSA, and a training process of the texture feature fusion self-attention module TFSA comprises: The method comprises: Input features of the self-attention module TFSA The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix By calculating attention scores Then, the calculated attention score is used. Log-value matrix Perform a weighted summation to obtain the original self-attention result. ; The low-resolution picture extracts multi-scale texture feature priors through a texture material library wherein denotes the scale of the feature; For each scale of texture feature, a weight matrix is used to transform it, respectively obtaining the corresponding key and value : and value : Query Matrix With the key matrix at every scale Perform attention calculations to obtain attention results. : , Attention output Perform group normalization ,get Number of sampling steps Encode to obtain control coefficients and and the normalized features Perform scaling and translation operations: Through a zero convolutional layer The results after scaling and translation The process is performed to obtain the final output features. The output feature Compared with the original self-attention results Add them together to get the final output of the TFSA module. .

5. An implicit material library driven diffusion image super-resolution device, characterized in that, The super-resolution module inputs the image to be super-resolved into the trained image super-resolution model to obtain a super-resolution result image of the image to be super-resolved, the resolution of the super-resolution result image being higher than that of the image to be super-resolved. An initial module constructs an image super-resolution model composed of an encoder and a decoder, compressing a high-resolution color image A low-resolution picture is obtained, and the low-resolution picture is compressed and encoded by the encoder to obtain compressed features , are the spatial dimensions of the feature map, is the dimension of the feature vector;​ searching module, for each of the ranges nearest neighbor search is performed in the codebook to find the closest discrete feature ; the codebook where denotes the codebook size; where each compressed feature is mapped to a corresponding codebook index , representing the closest discrete feature in the codebook to the compressed feature ; The training module, through this decoder, Decode the image to obtain the reconstructed image. With the original image The image content is the target, and the reconstructed image is based on this. With the original image Based on the differences in content, a loss function is constructed, and the trained image super-resolution model is obtained by training the encoder, the decoder, and the codebook; The process of constructing the loss function in the training module comprises:

6. The implicit footage library driven diffusion image super-resolution apparatus of claim 1, wherein, The search module comprises: constructing a pixel loss for measuring the difference between the reconstructed image and the original image at the pixel level : ; Constructing perceptual loss : wherein, represents features extracted by a convolutional neural network; constructing adversarial loss , ; The codebook loss is constructed by measuring the distance between the encoder output feature vector and the corresponding discrete feature in the codebook : wherein is a loss weight; The encoder, the decoder and the codebook are jointly optimized by the pixel loss , the perceptual loss , the adversarial loss and the codebook loss .

7. The implicit footage library driven diffusion image super-resolution apparatus of claim 1, wherein, The self-attention module in the encoder and the decoder is a texture feature fusion self-attention module TFSA, and a training process of the texture feature fusion self-attention module TFSA comprises: The compressed feature is obtained through the classification network The matching probability of N vectors in the corresponding codebook is obtained, and the vector with the maximum probability is taken as the discrete feature ; According to the discrete feature As a label, the classification loss is calculated And the consistency loss of the intermediate feature in the encoder To train the codebook; classification loss wherein, is a high resolution color image is the codebook index obtained by the nearest neighbor search, is the matching probability; wherein the low resolution picture is at the input of the encoder the intermediate features of the layer are the high resolution picture is the intermediate features of the corresponding layer are , denotes the number of intermediate feature layers in the encoder The electronic device or the information display device connected thereto is configured to display the super-resolution result image in a display parameter, an attribute, or through an artificial intelligence model set by a user. Input features of the self-attention module TFSA The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix By calculating attention scores Then, the calculated attention score is used. Log-value matrix Perform a weighted summation to obtain the original self-attention result. ; The low-resolution picture extracts multi-scale texture feature priors through a texture material library wherein denotes the scale of the feature; for each scale of the texture feature, a weight matrix is used is transformed, to obtain a corresponding key and value respectively: query matrix key matrix attention calculation : , attention output group normalization : sampling steps encoding control coefficients and scaling and translation zero convolution layer scaled and translated result final output feature output feature original self-attention result final output of the TFSA module .

8. An electronic device, comprising: 9.A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the implicit material library driven diffusion image super-resolution method according to any one of claims 1-4. The computer program is executed by a processor to implement the steps of the implicit material library driven diffusion image super-resolution method according to any one of claims 1-4.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the implicit material library driven diffusion image super-resolution method according to any one of claims 1-4.

Images