Image super-resolution method and system based on standard contrastive learning

Abstract

The present invention provides an image super-resolution method and system based on standard contrast learning, which relates to the field of image processing technology. The method first extracts features from the original real-world image, then establishes a positive standard loss function and a negative standard loss function based on the extracted feature image and its corresponding spatial mask, and finally directly contrasts the positive standard loss and the negative standard loss, and iteratively optimizes the initial model. By constructing a new framework for standard contrast learning, the method directly contrasts the positive and negative standard losses instead of the image blocks used in traditional methods. In terms of the visual effect of the restored image, the method can not only significantly improve the visual quality of the restored image, but also restore more correct details with fewer artifacts.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more particularly to an image super-resolution method and system based on standard contrast learning. Background Art

[0002] Image super-resolution mainly refers to the process of restoring low-resolution (LR, Low resolution) images to high-resolution (HR, High resolution) images. It is a classic application in computer vision and has important application value in monitoring equipment, digital high-definition, satellite image remote sensing, and medical imaging.

[0003] Unlike traditional image super-resolution methods that use downsampling techniques, real-world image super-resolution typically uses two different optical sensors to capture images of different resolutions in real-world settings. However, this also results in significant pixel shifts in real-world image super-resolution. Contrastive learning is a promising approach that primarily learns good feature representations by constructing positive and negative training pairs.

[0004] Although contrastive learning has been successfully applied in many computer vision tasks, it is not straightforward to introduce contrastive learning into real-world image super-resolution tasks due to the difficulty in defining effective positive samples in the real-world image super-resolution setting.

[0005] The current prior art discloses a fine-grained scale image super-resolution method based on a non-local enhancement network, comprising the following steps: Step A: Preprocessing the original high-resolution training image to obtain a dataset of image block pairs consisting of low-quality high-resolution image blocks of different scales and the original high-resolution training image blocks; Step B: Using the image block pair dataset to train a non-local enhancement deep network; Step C: Inputting the high-resolution image of the low-quality test image into the deep network for reconstruction to obtain a super-resolution result. Although the prior art can achieve image super-resolution by combining non-local operations with ordinary convolution, this technology essentially restores low-quality images through downsampling techniques and contrastive learning methods. It does not achieve good results when faced with complex real-world image super-resolution tasks. The restored real-world images are of low quality, and some detailed textures are not clearly restored, which cannot meet the needs of widespread application. Summary of the Invention

[0006] In order to overcome the above-mentioned defect of low quality of the existing technology when restoring real-world images, the present invention provides an image super-resolution method and system based on standard contrast learning, which can significantly improve the visual quality of the restored image.

[0007] The present invention aims to solve the above technical problems at least to a certain extent.

[0008] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0009] An image super-resolution method based on standard contrastive learning, comprising the following steps:

[0010] S1: Get the original real-world image;

[0011] S2: Input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain the corresponding first feature image and second feature image;

[0012] S3: using a preset spatial mask to perform convolution operations on the first feature image and the second feature image respectively to obtain a first feature convolution image and a second feature convolution image;

[0013] S4: Select an anchoring standard from the existing image restoration standards, and use the first feature convolution image and the second feature convolution image to calculate the standard loss relative to the anchoring standard;

[0014] S5: Establish a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and divide the standard contrast loss function into a positive term and a negative term;

[0015] S6: Establishing a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard contrast loss function respectively;

[0016] S7: Optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model;

[0017] S8: Obtain a low-resolution real-world image and input it into the optimized contrastive learning model to obtain a real-world super-resolution image.

[0018] Preferably, before step S2, the method further includes performing a data augmentation operation on the original real-world image, wherein the data augmentation operation includes:

[0019] Horizontal flip, vertical flip, and random rotation of 90°, 180°, and 270°.

[0020] Preferably, in step S2, the specific method of performing feature extraction on each original real-world image to obtain the corresponding first feature image and second feature image is:

[0021] With Extractor F α (·) For the i-th original real-world image I i,lfFeature extraction can be expressed as follows:

[0022] [I i,align ,I i,unif ]=F α (I i,lf |θ α ).

[0023] Among them, I i,lf represents the i-th original real-world image, I i,align Represents the first feature image of the i-th image, I i,unif represents the i-th second feature image, F α (·) represents the extractor, θ α Represents an extractor parameter.

[0024] Preferably, in step S3, a convolution operation is performed on the first feature image and the second feature image respectively using a preset spatial mask to obtain the first feature convolution image and the second feature convolution image, and the specific method is:

[0025]

[0026]

[0027] in, is a spatial mask corresponding to the first feature image and the second feature image; and They are respectively the first feature convolution image corresponding to the i-th original real-world image and the second feature convolution image corresponding to the i-th original real-world image.

[0028] Preferably, in step S4, existing image restoration standards include: adversarial criterion, perceptual criterion and Euclidean criterion.

[0029] Preferably, the anchoring criterion is the Euclidean criterion.

[0030] Preferably, in step S5, the first feature convolution image and the second feature convolution image are used to calculate the standard loss relative to the anchor standard, and the specific method is:

[0031] For each of the first feature convolution image and the second feature convolution image, the positive standard loss and negative standard loss relative to the anchor standard are calculated:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] Among them, C i,a is the first feature convolution image of the i-th image Anchor standard loss; C k,a is the kth second feature convolution image Anchor standard loss of is the first feature convolution image corresponding to the i-th original real-world image Positive standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Positive standard loss relative to the anchor standard; is the first feature convolution image corresponding to the i-th original real-world image negative standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Negative standard loss with respect to the anchor standard; i and k both take values ​​from 1 to N, where N represents the number of original real-world images.

[0039] Preferably, in step S5, a standard contrast loss function is established based on the standard loss of the first feature convolution image and the second feature convolution image relative to the anchoring standard, and a specific method for dividing the standard contrast loss function into positive and negative terms is:

[0040]

[0041] in, is the standard contrastive learning loss function, φ + is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard positive standard loss, φ - is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard negative standard loss; N is the number of original real-world images.

[0042] Preferably, in step S6, the specific method of establishing a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard contrast loss function is:

[0043] The positive standard loss function is:

[0044]

[0045] in, represents the positive standard loss function corresponding to the i-th first feature convolution image, and η is the first empirical parameter.

[0046] The negative standard loss function is:

[0047]

[0048] in, represents the negative standard loss function corresponding to the k-th second feature convolution image, and t represents the second empirical parameter.

[0049] An image super-resolution system, applying the above-mentioned image super-resolution method based on standard contrastive learning, comprises:

[0050] A data acquisition module, used to acquire original real-world images;

[0051] A feature extraction module is used to input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain a corresponding first feature image and a second feature image;

[0052] a mask convolution module that uses a preset spatial mask to perform convolution operations with the first feature image and the second feature image to obtain the first feature convolution image and the second feature convolution image;

[0053] a loss calculation module, configured to select an anchoring standard from existing image restoration standards and calculate a standard loss relative to the anchoring standard using the first feature convolution image and the second feature convolution image;

[0054] a standard contrast loss function division module, configured to establish a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchoring standard, and to divide the standard contrast loss function into a positive term and a negative term;

[0055] A positive and negative standard loss function construction module is used to establish a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard comparison loss function respectively;

[0056] A model optimization module is used to optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model;

[0057] The super-resolution module is used to obtain low-resolution real-world images and input them into the optimized contrastive learning model to obtain real-world super-resolution images.

[0058] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0059] The present invention provides an image super-resolution method and system based on standard contrast learning. This method extracts features from original real-world images and establishes a positive standard loss function and a negative standard loss function based on the extracted feature images and their corresponding spatial masks. This method uses direct contrast learning of the positive standard loss and the negative standard loss, rather than the image blocks used in traditional methods, to significantly improve the visual quality of the restored image and restore more accurate details with fewer artifacts.

[0060] In addition, unlike the existing contrastive learning that achieves image super-resolution by simply excluding negative data pairs, the present invention establishes different positive standard loss functions and negative standard loss functions based on the standard contrastive loss functions of different feature images, which can provide new asymmetric context during the iterative optimization of the contrastive learning model, further improving the restoration effect of real-world images. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of an image super-resolution method based on standard contrast learning provided by the present invention.

[0062] Figure 2 This is a schematic diagram of an image super-resolution method based on standard contrast learning provided by the present invention.

[0063] Figure 3 This is the test result diagram of Image No. 45 of the RealSR dataset provided by this invention.

[0064] Figure 4 This is the test result diagram of image No. 867 of the NTIRE20 dataset provided by the present invention.

[0065] Figure 5 This is the test result diagram of image No. 891 of the NTIRE20 dataset provided by the present invention.

[0066] Figure 6 This is a connection diagram of an image super-resolution system provided by the present invention.

[0067] 301-Data acquisition module, 302-Feature extraction module, 303-Mask convolution module, 304 Loss calculation module, 305-Standard contrast loss function division module, 306-Positive and negative standard loss function construction module, 307-Model optimization module, 308-Super resolution module. DETAILED DESCRIPTION

[0068] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0069] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0070] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0071] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0072] Example 1

[0073] like Figure 1 As shown, this embodiment provides an image super-resolution method based on standard contrast learning, comprising the following steps:

[0074] S1: Get the original real-world image;

[0075] S2: Input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain the corresponding first feature image and second feature image;

[0076] S3: using a preset spatial mask to perform convolution operations on the first feature image and the second feature image respectively to obtain a first feature convolution image and a second feature convolution image;

[0077] S4: Select an anchoring standard from the existing image restoration standards, and use the first feature convolution image and the second feature convolution image to calculate the standard loss relative to the anchoring standard;

[0078] S5: Establish a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and divide the standard contrast loss function into a positive term and a negative term;

[0079] S6: Establishing a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard contrast loss function respectively;

[0080] S7: Optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model;

[0081] S8: Obtain a low-resolution real-world image and input it into the optimized contrastive learning model to obtain a real-world super-resolution image.

[0082] In the specific implementation process, first, the original real-world image is obtained from the RealSR (Real-World Super-Resolution, real-world image super-resolution) dataset; then, the obtained original real-world image is input into the constructed initial contrast learning model, and the original real-world image is feature extracted to obtain a first feature image and a second feature image; then, a preset spatial mask is used to perform convolution operations with the first feature image and the second feature image respectively to obtain a first feature convolution image and a second feature convolution image, so as to more accurately calculate the loss of the feature image; then, a fixed anchor standard is selected from the existing image restoration standards, and the standard loss relative to the anchor standard is calculated using the extracted first feature convolution images and second feature convolution images; then, a standard contrast loss function is established based on the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and the standard contrast loss function is divided into positive and negative terms; then, a positive standard loss function and a negative standard loss function are established based on the positive and negative terms of the standard contrast loss function respectively; then, the initial contrast learning model is optimized based on the calculated positive standard loss and negative standard loss to obtain an optimized contrast learning model;

[0083] Finally, a low-resolution real-world image is obtained and input into the optimized contrastive learning model to obtain a real-world super-resolution image. This method significantly improves the quality of real-world image restoration by directly contrasting the positive standard loss and the negative standard loss instead of the contrast image blocks used in traditional methods.

[0084] Example 2

[0085] like Figure 2 As shown, this embodiment provides an image super-resolution method based on standard contrast learning, comprising the following steps:

[0086] S1: Get the original real-world image;

[0087] S2: Input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain the corresponding first feature image and second feature image;

[0088] S3: using a preset spatial mask to perform convolution operations on the first feature image and the second feature image respectively to obtain a first feature convolution image and a second feature convolution image;

[0089] S4: Select an anchoring standard from the existing image restoration standards, and use the first feature convolution image and the second feature convolution image to calculate the standard loss relative to the anchoring standard;

[0090] S5: Establish a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and divide the standard contrast loss function into a positive term and a negative term;

[0091] S6: Establishing a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard contrast loss function respectively;

[0092] S7: Optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model;

[0093] S8: Obtain a low-resolution real-world image and input it into the optimized contrastive learning model to obtain a real-world super-resolution image.

[0094] Before step S2, a data augmentation operation is performed on the original real-world image, and the data augmentation operation includes:

[0095] Horizontal flip, vertical flip, and random rotation of 90°, 180°, and 270°.

[0096] In step S2, the specific method of performing feature extraction on each original real-world image to obtain the corresponding first feature image and second feature image is:

[0097] With Extractor F α (·) For the i-th original real-world image I i,lf Feature extraction can be expressed as follows:

[0098] [I i,align ,I i,unif ]=F α (I i,lf |θ α ).

[0099] Among them, I i,lf represents the i-th original real-world image, I i,align Represents the first feature image of the i-th image, I i,unif represents the i-th second feature image, F α (·) represents the extractor, θ α Represents an extractor parameter.

[0100] In step S3, a convolution operation is performed on the first feature image and the second feature image respectively using a preset spatial mask to obtain the first feature convolution image and the second feature convolution image. The specific method is:

[0101]

[0102]

[0103] in, is a spatial mask corresponding to the first feature image and the second feature image; and They are respectively the first feature convolution image corresponding to the i-th original real-world image and the second feature convolution image corresponding to the i-th original real-world image.

[0104] In step S4, existing image restoration standards include: adversarial criterion, perceptual criterion and Euclidean criterion.

[0105] The anchoring criterion is the Euclidean criterion.

[0106] In step S5, the first feature convolution image and the second feature convolution image are used to calculate the standard loss relative to the anchor standard. The specific method is:

[0107] For each of the first feature convolution image and the second feature convolution image, the positive standard loss and negative standard loss relative to the anchor standard are calculated:

[0108]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114] Among them, C i,a is the first feature convolution image of the i-th image Anchor standard loss; C k,a is the kth second feature convolution image Anchor standard loss of is the first feature convolution image corresponding to the i-th original real-world image Positive standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Positive standard loss relative to the anchor standard; is the first feature convolution image corresponding to the i-th original real-world image negative standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Negative standard loss with respect to the anchor standard; i and k both take values ​​from 1 to N, where N represents the number of original real-world images.

[0115] In step S5, a standard contrast loss function is established based on the standard loss of the first feature convolution image and the second feature convolution image relative to the anchoring standard, and a specific method for dividing the standard contrast loss function into positive and negative terms is as follows:

[0116]

[0117] in, is the standard contrastive learning loss function, φ + is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard positive standard loss, φ - is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard negative standard loss; N is the number of original real-world images.

[0118] In step S6, the specific method of establishing the positive standard loss function and the negative standard loss function according to the positive term and the negative term of the standard contrast loss function is as follows:

[0119] The positive standard loss function is:

[0120]

[0121] in, represents the positive standard loss function corresponding to the i-th first feature convolution image, and η is the first empirical parameter.

[0122] The negative standard loss function is:

[0123]

[0124] in, represents the negative standard loss function corresponding to the k-th second feature convolution image, and t represents the second empirical parameter.

[0125] In the specific implementation process, the original real-world images are first obtained from the RealSR dataset. The real-world image dataset RealSR consists of 595 pairs of LR-HR image pairs. This embodiment uses 200 pairs of image pairs and another 800 images collected by DIV2K for training; then the original real-world images are augmented by horizontal flipping, vertical flipping and random rotation of 90°, 180° and 270°; then the original real-world image dataset after data augmentation is input into the preset contrastive learning model for feature extraction. This embodiment uses a simplified RRDB network and feature extractor F α(·) performs feature extraction on the i-th original real-world image I i,lf After feature extraction, the first feature image I is obtained i,align and the i-th second feature image I i,unif ;

[0126] Then use the preset spatial mask Respectively with the first feature image I i,align and the second feature image I i,unif Perform convolution operation to obtain the first feature convolution image and the second feature convolution image

[0127] Then, an anchoring criterion is selected from existing image restoration standards, including adversarial criterion, perceptual criterion, and Euclidean criterion. Since the Euclidean criterion can find obvious positive and negative examples and shows reliable results, this embodiment uses the Euclidean criterion as the anchoring criterion.

[0128] Afterwards, the first feature extracted is used to convolve the image and the second feature convolution image Calculate the standard loss relative to the anchor standard as follows:

[0129] For each of the first feature convolution image and the second feature convolution image, the positive standard loss and negative standard loss relative to the anchor standard are calculated:

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136] Among them, C i,a is the first feature convolution image of the i-th image Anchor standard loss; C k,a is the kth second feature convolution image Anchor standard loss of is the first feature convolution image corresponding to the i-th original real-world image Positive standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Positive standard loss relative to the anchor standard; is the first feature convolution image corresponding to the i-th original real-world image negative standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Negative standard loss relative to the anchor standard; i and k both range from 1 to N, where N represents the number of original real-world images;

[0137] Afterwards, a standard contrast loss function is established and divided into positive and negative terms. The specific method is as follows:

[0138]

[0139] in, is the standard contrastive learning loss function, φ + is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard positive standard loss, φ - is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard negative standard loss; N is the number of original real-world images;

[0140] Then, a positive standard loss function and a negative standard loss function are established according to the positive and negative terms of the standard comparison loss function, respectively. The specific method is:

[0141] The following positive standard loss function is established based on the positive term:

[0142]

[0143] in, represents the positive standard loss function corresponding to the i-th first feature convolution image, and η is the first empirical parameter.

[0144] In traditional contrastive learning methods, negative data pairs are usually simply rejected. However, the standard contrast method in this embodiment does not reject negative standards irregularly, but is enforced to achieve a uniform distribution on the hypersphere. In order to ensure rotational symmetry, the similarity of a given vector (x, y) is calculated using the Gaussian potential kernel as follows:

[0145]

[0146] The following negative standard loss function is established based on the negative term and Gaussian potential kernel:

[0147]

[0148] in, It represents the negative standard loss function corresponding to the k-th second feature convolution image, t>0, and represents the second empirical parameter.

[0149] The initial contrastive learning model is then optimized based on the calculated positive standard loss and negative standard loss. In this embodiment, Adam is used as the optimizer, the batch size is 16, and the number of training iterations is 106 epochs. After training, the optimized contrastive learning model is obtained.

[0150] Finally, a low-resolution real-world image is obtained. In this embodiment, original real-world images are selected from the RealSR and NTIRE20 datasets and processed into low-resolution images as a test set. The low-resolution real-world images are input into the optimized contrastive learning model to obtain real-world super-resolution images.

[0151] like Figure 3 、 Figure 4 and Figure 5 The following figures show the restoration effects of three different real-world images. As can be seen from these three figures, compared with the traditional ESRGAN, FSSR, and DASR methods, the method provided by this embodiment has the highest peak signal-to-noise ratio (PSNR) and structural similarity score (SSIM), indicating that the image restored by the method provided by this embodiment has better quality and is closer to the real image in terms of texture details. In addition, the learned perceptual patch similarity (LPIPS) of the restored image in this embodiment is the smallest compared with the other methods, indicating that the real-world image restored by this method has the smallest difference from the original image and has better effect.

[0152] Qualitative and quantitative analysis of the results show that this method constructs a new framework for standard contrastive learning by directly contrasting positive and negative standard losses. This not only significantly improves the quality of real-world image restoration, but also recovers more correct details with fewer artifacts.

[0153] In addition, unlike the existing contrastive learning that achieves image super-resolution by simply excluding negative data pairs, the present invention establishes different positive standard loss functions and negative standard loss functions based on the standard contrastive loss functions of different feature images, which can provide new asymmetric context during the iterative optimization of the contrastive learning model, further improving the restoration effect of real-world images.

[0154] Example 3

[0155] like Figure 6 As shown, this embodiment provides an image super-resolution system, applying the image super-resolution method based on standard contrast learning in any one of Embodiments 1 or 2, including:

[0156] A data acquisition module 301 is used to acquire original real-world images;

[0157] A feature extraction module 302 is configured to input the original real-world images into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain a corresponding first feature image and a second feature image;

[0158] The mask convolution module 303 performs convolution operations on the first feature image and the second feature image using a preset spatial mask to obtain a first feature convolution image and a second feature convolution image;

[0159] a loss calculation module 304 for selecting an anchoring standard from existing image restoration standards and calculating a standard loss relative to the anchoring standard using the first feature convolution image and the second feature convolution image;

[0160] a standard contrast loss function division module 305 for establishing a standard contrast loss function based on the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and dividing the standard contrast loss function into a positive term and a negative term;

[0161] A positive and negative standard loss function construction module 306 is used to respectively establish a positive standard loss function and a negative standard loss function according to the positive term and the negative term of the standard comparison loss function;

[0162] A model optimization module 307 is used to optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model;

[0163] The super-resolution module 308 is used to obtain a low-resolution real-world image and input it into the optimized contrast learning model to obtain a real-world super-resolution image.

[0164] In the specific implementation process, first, the data acquisition module 301 acquires the original real-world image; then the feature extraction module 302 inputs the original real-world image into the constructed initial contrast learning model, performs feature extraction on each original real-world image, and obtains the corresponding first feature image and second feature image; then the mask convolution module 303 uses the preset spatial mask to perform convolution operations with the first feature image and the second feature image respectively to obtain the first feature convolution image and the second feature convolution image; then the loss calculation module 304 selects the anchor standard from the existing image restoration standards, and uses the first feature convolution image and the second feature convolution image to calculate the standard loss relative to the anchor standard; then the standard contrast loss function division module 305 establishes a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and divides the standard contrast loss function into positive and negative terms; then the positive and negative standard loss function construction module 306 establishes a positive standard loss function and a negative standard loss function according to the positive and negative terms of the standard contrast loss function respectively; then the model optimization module 307 optimizes the initial contrast learning model according to the positive standard loss and the negative standard loss to obtain the optimized contrast learning model;

[0165] Finally, the super-resolution module 308 obtains a low-resolution real-world image and inputs the low-resolution real-world image into the optimized contrastive learning model to obtain a real-world super-resolution image. The device constructs a new framework for standard contrastive learning by directly contrasting positive standard loss and negative standard loss, which significantly improves the quality of real-world image restoration.

[0166] The same or similar reference numerals correspond to the same or similar components;

[0167] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0168] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. An image super-resolution method based on standard contrastive learning, characterized in that: The following steps are involved: S1: Get the original real-world image; S2: Input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain the corresponding first feature image and second feature image; S3: using a preset spatial mask to perform convolution operations on the first feature image and the second feature image respectively to obtain a first feature convolution image and a second feature convolution image; S4: Select an anchoring standard from the existing image restoration standards, and use the first feature convolution image and the second feature convolution image to calculate the standard loss relative to the anchoring standard; S5: Establish a standard contrast loss function based on the standard loss of the first feature convolution image and the second feature convolution image relative to the anchor standard, and divide the standard contrast loss function into positive and negative terms; specifically: For each of the first feature convolution image and the second feature convolution image, the positive standard loss and negative standard loss relative to the anchor standard are calculated: in, is the first feature convolution image of the i-th image Anchor standard loss of is the kth second feature convolution image Anchor standard loss of is the first feature convolution image corresponding to the i-th original real-world image Positive standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Positive standard loss relative to the anchor standard; is the first feature convolution image corresponding to the i-th original real-world image negative standard loss relative to the anchor standard; is the second feature convolution image corresponding to the kth original real-world image Negative standard loss relative to the anchor standard; i and k both range from 1 to N, where N represents the number of original real-world images; represents the i-th original real-world image; The standard contrast loss function is specifically: in, is the standard contrastive learning loss function, is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard positive standard loss, is the similarity measurement function of the first feature convolution image and the second feature convolution image relative to the anchor standard negative standard loss; N is the number of original real-world images; S6: Establishing a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard contrast loss function respectively; S7: Optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model; S8: Obtain a low-resolution real-world image and input it into the optimized contrastive learning model to obtain a real-world super-resolution image.

2. The image super-resolution method based on standard contrast learning according to claim 1, characterized in that: Before step S2, a data augmentation operation is performed on the original real-world image, and the data augmentation operation includes: Horizontal flip, vertical flip, and random rotation of 90°, 180°, and 270°.

3. The image super-resolution method based on standard contrast learning according to claim 1, characterized in that: In step S2, the specific method of performing feature extraction on each original real-world image to obtain the corresponding first feature image and second feature image is: Use extractor For the i-th original real-world image Feature extraction can be expressed as follows: in, represents the i-th original real-world image, represents the first feature image of the i-th image, represents the i-th second feature image, represents an extractor, Represents an extractor parameter.

4. The image super-resolution method based on standard contrast learning according to claim 3, characterized in that: In step S3, a convolution operation is performed on the first feature image and the second feature image respectively using a preset spatial mask to obtain the first feature convolution image and the second feature convolution image. The specific method is: in, is a spatial mask corresponding to the first feature image and the second feature image; and They are respectively the first feature convolution image corresponding to the i-th original real-world image and the second feature convolution image corresponding to the i-th original real-world image.

5. The image super-resolution method based on standard contrast learning according to claim 4, characterized in that: In step S4, existing image restoration standards include: adversarial criterion, perceptual criterion and Euclidean criterion.

6. The image super-resolution method based on standard contrast learning according to claim 5, characterized in that: The anchoring criterion is the Euclidean criterion.

7. The image super-resolution method based on standard contrast learning according to claim 6, characterized in that: In step S6, the specific method of establishing the positive standard loss function and the negative standard loss function according to the positive term and the negative term of the standard contrast loss function is as follows: The positive standard loss function is: in, represents the positive standard loss function corresponding to the i-th first feature convolution image, and η is the first empirical parameter; The negative standard loss function is: in, represents the negative standard loss function corresponding to the k-th second feature convolution image, and t represents the second empirical parameter.

8. An image super-resolution system, characterized in that: Applying the image super-resolution method based on standard contrast learning in any one of claims 1 to 7, comprising: A data acquisition module, used to acquire original real-world images; A feature extraction module is used to input the original real-world image into the constructed initial contrastive learning model, perform feature extraction on each original real-world image, and obtain a corresponding first feature image and a second feature image; a mask convolution module that uses a preset spatial mask to perform convolution operations with the first feature image and the second feature image to obtain the first feature convolution image and the second feature convolution image; a loss calculation module, configured to select an anchoring standard from existing image restoration standards and calculate a standard loss relative to the anchoring standard using the first feature convolution image and the second feature convolution image; a standard contrast loss function division module, configured to establish a standard contrast loss function according to the standard loss of the first feature convolution image and the second feature convolution image relative to the anchoring standard, and to divide the standard contrast loss function into a positive term and a negative term; A positive and negative standard loss function construction module is used to establish a positive standard loss function and a negative standard loss function according to the positive term and negative term of the standard comparison loss function respectively; A model optimization module is used to optimize the initial contrastive learning model according to the positive standard loss and the negative standard loss to obtain an optimized contrastive learning model; The super-resolution module is used to obtain low-resolution real-world images and input them into the optimized contrastive learning model to obtain real-world super-resolution images.

Images