A relational reinforcement learning system and method based on graph structure data

Abstract

The present invention provides a relational reinforcement learning system and method based on graph-structured data. The system encodes real-world images and mismatched high-resolution images into degraded representations, and then uses contrastive learning to process the relationship between positive and negative samples between the representations to obtain an encoder that can correctly extract image degradation features for use in subsequent downsampling networks. The low-resolution image obtained by the linear downsampling network of the high-resolution image will have the degradation features of the real world. The data set finally generated is input into the reconstruction network for training to complete the image super-resolution task. Compared with the current super-resolution algorithm based on blur kernel estimation, the present invention can learn the implicit expression between images without introducing additional degradation models and prior knowledge, and has better generalization while achieving the same performance as supervised methods.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and image super-resolution technology, and more specifically, to a relational reinforcement learning system and method based on graph-structured data. Background Art

[0002] Image super-resolution has been a key research topic in computer vision in recent years. Single-image super-resolution aims to produce a clearer, higher-resolution image through end-to-end training of low-resolution images. Sampling-based methods such as linear interpolation, nearest neighbor interpolation, and bicubic linear interpolation have emerged as simple and convenient super-resolution techniques. These operators can quickly generate a high-resolution image without requiring additional space. Recent research has demonstrated that methods based on convolutional neural networks (CNNs) have achieved significantly better results than traditional methods. These methods use deep learning to map low-resolution images to high-resolution images end-to-end. These deep networks can extract high-level features from the input image and thus learn complex nonlinear mappings between images. However, these single-image super-resolution methods have not achieved satisfactory results when applied to real-world image super-resolution. This is because the low-resolution images used for training are typically downsampled from high-resolution images and lack the degraded features found in the real world. To address this issue, generating real-world low-resolution-high-resolution image pairs is a key research topic. One approach uses generative adversarial networks to generate low-resolution images with degraded features, but the resulting images lack detail and realism. Another approach models image degradation, assuming that image degradation is caused by a blur kernel and noise. The blur kernel is then estimated to generate the corresponding image. The drawback of this approach is that it is difficult to predict the blur kernel in the real world and can result in large errors. Compared to single-image super-resolution, the aforementioned super-resolution method, which optimizes the training image, achieves significantly better super-resolution results on real-world images.

[0003] Unsupervised learning is a learning approach in machine learning that seeks to discover previously undetected patterns in a dataset without pre-existing labels and with minimal human supervision. In computer vision, unsupervised learning typically refers to the absence of labeled images in the training set that match the training images. A key characteristic of unsupervised learning is that the data fed to the algorithm is rich in internal structure, while the training objectives and rewards are sparse. Therefore, unsupervised learning aims to learn common features across similar data from a large amount of data and encode them into high-level representations. The learned model is then fine-tuned for specific tasks to achieve superior results. Contrastive learning is a type of unsupervised learning. Its core idea is to reduce the distance between positive samples and increase the distance between positive and negative samples. In this paper's real-world image super-resolution method, degraded features extracted from natural images are positive samples, and all others are negative samples.

[0004] Degradation Learning Real-World Super-Resolution (DLSR). Current super-resolution algorithms based on convolutional neural networks (CNNs) are typically trained on large, annotated datasets of high-quality images. However, the resulting networks often have poor generalization capabilities for real-world, low-resolution images containing blur kernels and noise, resulting in poor quality high-resolution images. Because image degradation destroys the structure and statistical properties of pixels within a neighborhood, it is impossible to use the common bicubic linear interpolation downsampling method to obtain a low-resolution degraded image. The image degradation model can generally be viewed as a high-resolution image that is downsampled with a blur kernel and then noise is added to obtain a degraded image. Based on this model, some existing methods propose super-resolution algorithms based on blur kernel estimation. However, this degradation estimation displayed in pixel space not only depends on the effectiveness of the degradation model, but also requires a strong prior auxiliary model to learn the blur kernel. Therefore, this patent proposes a real-world image algorithm based on unsupervised learning, which aims to use contrastive learning to extract implicit degradation feature representations in real-world images, thereby generating corresponding low-resolution images.

[0005] In general, DLSR has two main tasks: generating paired low-resolution and high-resolution images, and super-resolution of images. For the first task, contrastive learning can be used to reduce the distance between low-resolution image features (positive samples) and reduce the distance between them and negative samples (high-resolution image features), thereby obtaining an implicit representation of the degradation features. For the second task, the super-resolution module is primarily based on a residual feedforward neural network, while introducing different loss functions to ensure image super-resolution quality and restore texture details. The main steps of DLSR can be summarized as follows: first, a degradation feature encoder is trained through contrastive learning. This encoder extracts implicit degradation features from the image. Using these extracted degradation features, the model ensures that the low-resolution image obtained after the downsampling network also has the same degradation features, thus generating paired training data for the super-resolution network. It is worth noting that, unlike common unsupervised real-world image super-resolution algorithms, DLSR learns implicit representations based on the degradation differences between high-resolution and low-resolution images. It does not require any special prior knowledge for explicit estimation, making it well-suited for super-resolution of diverse datasets. In addition, the super-resolution module used by DLSR has a small number of parameters and a moderate network depth, which ensures good super-resolution effects while reducing computational overhead and time consumption. Moreover, this module can be arbitrarily replaced by other excellent single-image super-resolution networks and has strong coupling.

[0006] Image quality can degrade during image formation, recording, and processing due to imperfections in imaging systems, recording equipment, and processing methods. Therefore, low-resolution images in the real world can generally be considered degraded. Regarding the poor performance of traditional single-image super-resolution methods on real-world images, analysis reveals that the low-resolution images used in traditional single-image super-resolution training are obtained through simple downsampling operators, such as bicubic linear interpolation. This simple downsampling strategy eliminates the degradation features present in real-world low-resolution images in the training data, meaning the model cannot learn how to super-resolve images with these features. In real-world image super-resolution, the low-resolution images in our training data all have degradation features and lack corresponding high-resolution images, making direct supervised training impractical. While paired high-resolution images are unavailable, it is possible to start with unpaired high-resolution images and downsample them to low-resolution images with degradation features, thereby forming training pairs for the model. For image degradation feature extraction, the lack of labeled images prevents the use of traditional variational encoders, necessitating unsupervised approaches to learn these features. For the above problems, contrastive learning and perceptual loss optimization algorithms can be used to solve them one by one.

[0007] The prior art discloses a patent for a sampling-based undirected graph structure data comparison method. This patent is based on a sampling-based undirected graph structure data comparison method and uses random sampling to compare node connection information to determine whether two undirected graphs are the same. Compared with existing graph similarity algorithms, the method designed in this patent has better efficiency and can be well applied to massive data processing; when the scale of the graph is large, it can also achieve good operating results; and when it is determined that two graphs are different, it can ensure that they are definitely different, and when it is determined that they are different, the correctness of the result can also be guaranteed with a high probability; at the same time, the sampling-based undirected graph structure data comparison method provided by the patent has good versatility and can be extended to the comparison of weighted undirected graphs; however, the patent does not involve any technical solutions for extracting degraded features by training an encoder through contrastive learning, and then using the encoder and downsampling network to generate low-resolution images with degraded features to construct paired training data, and finally use them to obtain high-resolution images with complete details and good visual effects. Summary of the Invention

[0008] The present invention provides a relational reinforcement learning system based on graph-structured data that can extract degradation features of real-world images.

[0009] Another object of the present invention is to provide the above-mentioned relational reinforcement learning system learning method based on graph structure data.

[0010] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:

[0011] A relational reinforcement learning system based on graph-structured data, comprising:

[0012] The degradation feature extraction module inputs the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain the image degradation feature encoder;

[0013] The downsampling module inputs the high-resolution image into the linear downsampling network. The generated low-resolution image is then input into the image degradation feature encoder together with the real-world image, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0014] The super-resolution reconstruction module inputs the low-resolution-high-resolution image data generated by the downsampling module into the super-resolution reconstruction network training to obtain a real-world image super-resolution model.

[0015] Furthermore, the degradation feature extraction module classifies degradation features extracted from real-world images as positive samples and degradation features extracted from high-resolution images as negative samples. Based on contrastive learning, the model can minimize the distance between positive samples while keeping them away from negative samples, thereby achieving the effect of extracting degradation features.

[0016] Furthermore, the degradation feature extraction module includes two networks:

[0017] Feature encoder, used to extract degradation features from images, whose input is the picture;

[0018] Multi-layer perceptron, the user graph converts the degraded representation into positive and negative samples.

[0019] Furthermore, the output of the image degradation feature encoder is a vector representation of a feature. These features are then grouped in pairs and passed through a multi-layer perceptron. One feature in the real-world image will serve as the query sample, the other as its positive sample, and the features of the two high-resolution images will serve as negative samples. The contrastive learning process is to shorten the distance between the query sample and the positive sample and increase the distance between it and the negative sample, that is, to make the positive samples as similar as possible and the positive and negative samples as dissimilar as possible.

[0020] Furthermore, the specific process of degradation feature extraction based on contrastive learning is:

[0021] 1): First, N real-world images and N high-resolution images are passed through the image degradation feature encoder to obtain the image feature F q and F k , the dimension is N×C;

[0022] 2): The image feature F q and F k Send it to the multi-layer perceptron to get the query sample q and positive sample k + and negative samples k + ;

[0023] 3): Use InfoNCE to measure the similarity between positive and negative samples. During training, InfoNCELoss should be minimized, i.e. Where N is the number of images, is the i-th query sample and positive sample, is the jth negative sample, S is the queue size, and τ is the hyperparameter;

[0024] 4): Gradient backpropagation is performed based on the Loss to update the encoder parameters. The parameters of the image degradation feature encoder are updated using momentum.

[0025] 5): Update the queue, delete the earliest batch, and add a new batch.

[0026] Furthermore, the specific processing process of the downsampling module is:

[0027] Build an image degradation model:

[0028]

[0029] Among them, y and x represent the low-resolution image and high-resolution image respectively, Represents the convolution operation between the image and the blur kernel,↓ s Represents the downsampling scaling factor, n represents noise. Starting from this model, because the convolution operation is a linear operation, the use of a nonlinear downsampling network is obviously inconsistent with the model theory, so a linear downsampling network is used here, that is, without any activation function; the convolution kernels of the first three layers of the network are 7×7, 5×5 and 3×3, and the convolution kernels of the next three layers are 1×1. The last layer is composed of a hidden layer with a stride of 2. The number of layers here depends on the downsampling scaling factor; the entire network is equivalent to the image convolving with a 13×13 convolution kernel and downsampling, which can be regarded as a 13×13 single-layer linear network, which meets the definition of the model.

[0030] Furthermore, due to the difficulty in optimizing a single-layer network and its slow convergence speed, a multi-layer network is used for training. During the training process, a weighted combination of color loss, adversarial loss, and feature matching loss is used as the total loss function. The feature matching loss function is used to ensure that the features of the generated image obtained by the encoder are similar to the features of the real-world image obtained by the encoder. The role of color loss and adversarial loss is to ensure that the basic structural information of the downsampled image does not change too much.

[0031] Furthermore, the super-resolution reconstruction module super-resolves the generated low-resolution images and uses the corresponding high-resolution images for training. This module is a deep convolutional neural network that uses a residual network to reduce the network's overfitting and gradient vanishing problems. Each residual block first consists of two convolutional layers with a convolution kernel size of 3×3 and 64 channels, followed by a batch normalization layer and ReLU as an activation function, and then two Pixelshuffler layers are used to enlarge the size of the features. Finally, a 3×3 convolution outputs a 3-channel image, and l1 and l are used in training. per As the loss function, l1loss is defined as:

[0032]

[0033] Among them, s is the scaling factor, W and H represent the width and height of the scaled image, Represents the pixel value of the image, and F is the super-resolution reconstruction network. This loss function is the most common super-resolution loss function, which measures the distance between pixels as the L1 norm. If only a single L1 loss is used, the super-resolved image often lacks high-frequency parts, and the details and texture of the image cannot be well displayed. Therefore, another perceptual loss is introduced, L per The loss is defined as:

[0034]

[0035] Among them, φ i, W represents the feature map of the jth convolutional layer before the i-th maximum pooling layer of the VGG network, i,j , H i,j Represent the length and width in the feature map, I HR Represents the pixel value of the image, F is the super-resolution reconstruction network, and the perceptual loss can compare the features of the high-resolution image after super-resolution with the features of the label image, so that the content and global structure between the images are close. The super-resolution image has more edge, color and detail information and has a good visual effect.

[0036] A learning method for a relational reinforcement learning system based on graph-structured data, comprising the following steps:

[0037] S1: Input the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain an image degradation feature encoder;

[0038] S2: Input the high-resolution image into the linear downsampling network. The generated low-resolution image and the real-world image are then input into the encoder trained in step S1 together, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0039] S3: The low-resolution-high-resolution image data generated in step S2 is input into the super-resolution reconstruction network for training to obtain a real-world image super-resolution model.

[0040] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0041] The present invention can better complete the super-resolution task of real-world images. It trains the encoder through contrastive learning to extract degradation features, and then uses the encoder and downsampling network to generate low-resolution images with degradation features, constructs paired training data, and finally uses them to obtain high-resolution images with perfect details and good visual effects. The various indicators are close to the super-resolution method of supervised learning. It solves the problems of how to extract degradation features of real-world images and how to generate low-resolution images close to real-world images for model training. At the same time, it is necessary to design a downsampling network and a super-resolution reconstruction network to better achieve image super-resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the system structure of the present invention;

[0043] Figure 2 Flow chart of the method of the present invention;

[0044] Figure 3 Graph showing the experimental results of the method of the present invention. DETAILED DESCRIPTION

[0045] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0046] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0047] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0048] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0049] Example 1

[0050] like Figure 1 As shown, a relational reinforcement learning system based on graph structured data includes:

[0051] The degradation feature extraction module inputs the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain the image degradation feature encoder;

[0052] The downsampling module inputs the high-resolution image into the linear downsampling network. The generated low-resolution image is then input into the image degradation feature encoder together with the real-world image, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0053] The super-resolution reconstruction module inputs the low-resolution-high-resolution image data generated by the downsampling module into the super-resolution reconstruction network training to obtain a real-world image super-resolution model.

[0054] The degradation feature extraction module classifies degradation features extracted from real-world images as positive samples and degradation features extracted from high-resolution images as negative samples. Based on contrastive learning, the model can minimize the distance between positive samples while keeping them away from negative samples, thereby achieving the effect of extracting degradation features.

[0055] The degradation feature extraction module consists of two networks:

[0056] Feature encoder, used to extract degradation features from images, whose input is the picture;

[0057] Multi-layer perceptron, the user graph converts the degraded representation into positive and negative samples.

[0058] The output of the image degradation feature encoder is a vector representation of a feature. These features are then paired and passed through a multi-layer perceptron. One feature from the real-world image serves as the query sample, the other as its positive sample, and the features of the two high-resolution images serve as negative samples. The contrastive learning process aims to narrow the distance between the query sample and the positive sample and increase the distance between them and the negative sample. In other words, the goal is to make the positive samples as similar as possible and the positive and negative samples as dissimilar as possible.

[0059] The specific process of degradation feature extraction based on contrastive learning is:

[0060] 1): First, N real-world images and N high-resolution images are passed through the image degradation feature encoder to obtain the image feature F q and F k , the dimension is N×C;

[0061] 2): The image feature F q and F k Send it to the multi-layer perceptron to get the query sample q and positive sample k + and negative samples k + ;

[0062] 3): Use InfoNCE to measure the similarity between positive and negative samples, and minimize InfoNCE Loss during training, i.e. Where N is the number of images, is the i-th query sample and positive sample, is the jth negative sample, S is the queue size, and τ is the hyperparameter;

[0063] 4): Gradient backpropagation is performed based on the Loss to update the encoder parameters. The parameters of the image degradation feature encoder are updated using momentum.

[0064] 5): Update the queue, delete the earliest batch, and add a new batch.

[0065] The specific processing process of the downsampling module is:

[0066] Build an image degradation model:

[0067]

[0068] Among them, y and x represent the low-resolution image and high-resolution image respectively, Represents the convolution operation between the image and the blur kernel,↓ s Represents the downsampling scaling factor, n represents noise. Starting from this model, because the convolution operation is a linear operation, the use of a nonlinear downsampling network is obviously inconsistent with the model theory, so a linear downsampling network is used here, that is, without any activation function; the convolution kernels of the first three layers of the network are 7×7, 5×5 and 3×3, and the convolution kernels of the next three layers are 1×1. The last layer is composed of a hidden layer with a stride of 2. The number of layers here depends on the downsampling scaling factor; the entire network is equivalent to the image convolving with a 13×13 convolution kernel and downsampling, which can be regarded as a 13×13 single-layer linear network, which meets the definition of the model.

[0069] Since single-layer networks are difficult to optimize and converge slowly, multi-layer networks are used for training. During the training process, a weighted combination of color loss, adversarial loss, and feature matching loss is used as the total loss function. The feature matching loss function is used to ensure that the features of the generated image obtained through the encoder are similar to the features of the real-world image obtained through the encoder. The role of color loss and adversarial loss is to ensure that the basic structural information of the downsampled image does not change too much.

[0070] The super-resolution reconstruction module performs super-resolution on the generated low-resolution images and uses the corresponding high-resolution images for training. This module is a deep convolutional neural network that uses a residual network to reduce the network's overfitting and gradient vanishing problems. Each residual block first consists of two convolutional layers with a convolution kernel size of 3×3 and 64 channels, followed by a batch normalization layer and ReLU as an activation function, and then two Pixelshuffler layers are used to enlarge the size of the features. Finally, a 3×3 convolution outputs a 3-channel image. The l1 and l per As the loss function, l1loss is defined as:

[0071]

[0072] Among them, s is the scaling factor, W and H represent the width and height of the scaled image, Represents the pixel value of the image, and F is the super-resolution reconstruction network. This loss function is the most common super-resolution loss function, which measures the distance between pixels as the L1 norm. If only a single L1 loss is used, the super-resolved image often lacks high-frequency parts, and the details and texture of the image cannot be well displayed. Therefore, another perceptual loss is introduced, L per The loss is defined as:

[0073]

[0074] Among them, φ i, W represents the feature map of the jth convolutional layer before the i-th maximum pooling layer of the VGG network, i,j , H i,j Represent the length and width in the feature map, I HR Represents the pixel value of the image, F is the super-resolution reconstruction network, and the perceptual loss can compare the features of the high-resolution image after super-resolution with the features of the label image, so that the content and global structure between the images are close. The super-resolution image has more edge, color and detail information and has a good visual effect.

[0075] Example 2

[0076] like Figure 2 As shown, a learning method for a relational reinforcement learning system based on graph structured data includes the following steps:

[0077] S1: Input the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain an image degradation feature encoder;

[0078] S2: Input the high-resolution image into the linear downsampling network. The generated low-resolution image and the real-world image are then input into the encoder trained in step S1 together, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0079] S3: The low-resolution-high-resolution image data generated in step S2 is input into the super-resolution reconstruction network for training to obtain a real-world image super-resolution model.

[0080] This method encodes the real-world image and the mismatched high-resolution image into degraded representations, and then uses contrastive learning to process the relationship between positive and negative samples between the representations to obtain an encoder that can correctly extract the image degradation features for use in the subsequent downsampling network. The low-resolution image obtained by the high-resolution image through the linear downsampling network will have the degradation features of the real world. The data set finally generated is input into the reconstruction network for training to complete the image super-resolution task. Compared with the current super-resolution algorithm based on blur kernel estimation, the present invention can learn the implicit expression between images without introducing additional degradation models and prior knowledge, and has better generalization when achieving the same performance as the supervised method. This method can better complete the super-resolution task of real-world images. It trains the encoder through contrastive learning to extract degradation features, and then uses the encoder and downsampling network to generate low-resolution images with degradation features to construct paired training data. Finally, it is used to obtain high-resolution images with perfect details and good visual effects. The various indicators are close to the super-resolution method of supervised learning, and are better than the current real-world image super-resolution algorithm.

[0081] Example 3

[0082] like Figure 1-2 As shown, a learning method for a relational reinforcement learning system based on graph structured data includes the following steps:

[0083] S1: Input the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain an image degradation feature encoder;

[0084] S2: Input the high-resolution image into the linear downsampling network. The generated low-resolution image and the real-world image are then input into the encoder trained in step S1 together, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0085] S3: The low-resolution-high-resolution image data generated in step S2 is input into the super-resolution reconstruction network for training to obtain a real-world image super-resolution model.

[0086] This method is applied to a relational reinforcement learning system based on graph-structured data, which includes:

[0087] The degradation feature extraction module inputs the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain the image degradation feature encoder;

[0088] The downsampling module inputs the high-resolution image into the linear downsampling network. The generated low-resolution image is then input into the image degradation feature encoder together with the real-world image, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged.

[0089] The super-resolution reconstruction module inputs the low-resolution-high-resolution image data generated by the downsampling module into the super-resolution reconstruction network training to obtain a real-world image super-resolution model.

[0090] The degradation feature extraction module classifies degradation features extracted from real-world images as positive samples and degradation features extracted from high-resolution images as negative samples. Based on contrastive learning, the model can minimize the distance between positive samples while keeping them away from negative samples, thereby achieving the effect of extracting degradation features.

[0091] The degradation feature extraction module consists of two networks:

[0092] Feature encoder, used to extract degradation features from images, whose input is the picture;

[0093] Multi-layer perceptron, the user graph converts the degraded representation into positive and negative samples.

[0094] The output of the image degradation feature encoder is a vector representation of a feature. These features are then paired and passed through a multi-layer perceptron. One feature from the real-world image serves as the query sample, the other as its positive sample, and the features of the two high-resolution images serve as negative samples. The contrastive learning process aims to narrow the distance between the query sample and the positive sample and increase the distance between them and the negative sample. In other words, the goal is to make the positive samples as similar as possible and the positive and negative samples as dissimilar as possible.

[0095] The specific process of degradation feature extraction based on contrastive learning is:

[0096] 1): First, N real-world images and N high-resolution images are passed through the image degradation feature encoder to obtain the image feature F q and F k , the dimension is N×C;

[0097] 2): The image feature F q and F k Send it to the multi-layer perceptron to get the query sample q and positive sample k + and negative samples K + ;

[0098] 3): Use InfoNCE to measure the similarity between positive and negative samples, and minimize InfoNCE Loss during training, i.e. Where N is the number of images, is the i-th query sample and positive sample, is the jth negative sample, S is the queue size, and τ is the hyperparameter;

[0099] 4): Gradient backpropagation is performed based on the Loss to update the encoder parameters. The parameters of the image degradation feature encoder are updated using momentum.

[0100] 5): Update the queue, delete the earliest batch, and add a new batch.

[0101] The specific processing process of the downsampling module is:

[0102] Build an image degradation model:

[0103]

[0104] Among them, y and x represent the low-resolution image and high-resolution image respectively, Represents the convolution operation between the image and the blur kernel,↓ s Represents the downsampling scaling factor, n represents noise. Starting from this model, because the convolution operation is a linear operation, the use of a nonlinear downsampling network is obviously inconsistent with the model theory, so a linear downsampling network is used here, that is, without any activation function; the convolution kernels of the first three layers of the network are 7×7, 5×5 and 3×3, and the convolution kernels of the next three layers are 1×1. The last layer is composed of a hidden layer with a stride of 2. The number of layers here depends on the downsampling scaling factor; the entire network is equivalent to the image convolving with a 13×13 convolution kernel and downsampling, which can be regarded as a 13×13 single-layer linear network, which meets the definition of the model.

[0105] Since single-layer networks are difficult to optimize and converge slowly, multi-layer networks are used for training. During the training process, a weighted combination of color loss, adversarial loss, and feature matching loss is used as the total loss function. The feature matching loss function is used to ensure that the features of the generated image obtained through the encoder are similar to the features of the real-world image obtained through the encoder. The role of color loss and adversarial loss is to ensure that the basic structural information of the downsampled image does not change too much.

[0106] The super-resolution reconstruction module performs super-resolution on the generated low-resolution images and uses the corresponding high-resolution images for training. This module is a deep convolutional neural network that uses a residual network to reduce the network's overfitting and gradient vanishing problems. Each residual block first consists of two convolutional layers with a convolution kernel size of 3×3 and 64 channels, followed by a batch normalization layer and ReLU as an activation function, and then two Pixelshuffler layers are used to enlarge the size of the features. Finally, a 3×3 convolution outputs a 3-channel image. The l1 and l per As the loss function, l1loss is defined as:

[0107]

[0108] Among them, s is the scaling factor, W and H represent the width and height of the scaled image, Represents the pixel value of the image, and F is the super-resolution reconstruction network. This loss function is the most common super-resolution loss function, which measures the distance between pixels as the L1 norm. If only a single L1 loss is used, the super-resolved image often lacks high-frequency parts, and the details and texture of the image cannot be well displayed. Therefore, another perceptual loss is introduced, L per The loss is defined as:

[0109]

[0110] Among them, φ i, W represents the feature map of the jth convolutional layer before the i-th maximum pooling layer of the VGG network, i,j , H i,j Represent the length and width in the feature map, I HR Represents the pixel value of the image, F is the super-resolution reconstruction network, and the perceptual loss can compare the features of the high-resolution image after super-resolution with the features of the label image, so that the content and global structure between the images are close. The super-resolution image has more edge, color and detail information and has a good visual effect.

[0111] Table 1 shows the test results of the proposed algorithm using two evaluation methods: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). PSNR is based on the error between corresponding pixels, i.e., it is an error-sensitive image quality assessment. Structural Similarity is a full-reference image quality assessment metric that measures image similarity in terms of brightness, contrast, and structure. As can be seen in the table above, the proposed algorithm outperforms existing super-resolution algorithms in both metrics, demonstrating that this invention can effectively perform super-resolution of real-world images.

[0112] Table 1 Comparison of experimental data

[0113] Methods PSNR SSIM bicubic 25.48 0.680 EDSR 25.36 0.640 SRFBN 25.37 0.642 ESRGAN (Unsupervised) 19.04 0.242 Impressionism 24.82 0.662 DBPN 24.51 0.701 OURS 25.50 0.738

[0114] Figure 3 This paper demonstrates the experimental results of our algorithm. Using the validation dataset from Track 1 of the NTIRE 2020 competition, we successfully super-resolved real-world images and compared them with existing algorithms. As can be seen, the super-resolved images generated by our algorithm are relatively clear, with details approaching those of true high-resolution images, featuring sharp edges and minimal blurring artifacts, resulting in excellent visual quality.

[0115] The same or similar reference numerals correspond to the same or similar components;

[0116] The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0117] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A relational reinforcement learning system based on graph structured data, characterized in that: include: The degradation feature extraction module inputs the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain the image degradation feature encoder; The degradation feature extraction module consists of two networks: a feature encoder, which is used to extract degradation features from images and takes images as input; and a multi-layer perceptron, which converts degradation representations into positive and negative samples. The downsampling module inputs the high-resolution image into the linear downsampling network. The generated low-resolution image is then input into the image degradation feature encoder together with the real-world image, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged. The super-resolution reconstruction module inputs the low-resolution-high-resolution image data generated by the downsampling module into the super-resolution reconstruction network training to obtain a real-world image super-resolution model; The super-resolution reconstruction module performs super-resolution on the generated low-resolution images and uses the corresponding high-resolution images for training; This module is a deep convolutional neural network that uses a residual network to reduce the problems of network overfitting and gradient vanishing. Each residual block first consists of two convolutional layers with a convolution kernel size of 3×3 and 64 channels, followed by a batch normalization layer and ReLU as the activation function, and then two Pixelshuffler layers are used to enlarge the size of the features. Finally, a 3×3 convolution outputs a 3-channel image.

2. The relational reinforcement learning system based on graph structure data according to claim 1, characterized in that: The degradation feature extraction module classifies the degradation features extracted from real-world images as positive samples and the degradation features extracted from high-resolution images as negative samples. Based on contrastive learning, the model can shorten the distance between positive samples as much as possible while keeping them away from negative samples, thereby achieving the effect of extracting degradation features.

3. The relational reinforcement learning system based on graph structure data according to claim 1, characterized in that: The output of the image degradation feature encoder is a vector representation of a feature. These features are then grouped in pairs and passed through a multi-layer perceptron. One feature in the real-world image will serve as the query sample, the other as its positive sample, and the features of the two high-resolution images will serve as negative samples. The contrastive learning process is to shorten the distance between the query sample and the positive sample and increase the distance between it and the negative sample, that is, to make the positive samples as similar as possible and the positive and negative samples as dissimilar as possible.

4. The relational reinforcement learning system based on graph structure data according to claim 3, characterized in that: The specific process of degradation feature extraction based on contrastive learning is: 1): First, N real-world images and N high-resolution images are passed through the image degradation feature encoder to obtain the image feature F q and F k , the dimension is N×C; 2): The image feature F q and F k Send it to the multi-layer perceptron to get the query sample q and positive sample k + and negative samples k + ; 3) Use InfoNCE to measure the similarity between positive and negative samples. During training, the InfoNCE Loss should be minimized, i.e. Where N is the number of images, is the i-th query sample and positive sample, is the jth negative sample, S is the queue size, and τ is the hyperparameter; 4): Gradient backpropagation is performed based on the Loss to update the encoder parameters. The parameters of the image degradation feature encoder are updated using momentum. 5): Update the queue, delete the earliest batch, and add a new batch.

5. The relational reinforcement learning system based on graph structure data according to claim 4, characterized in that: The specific processing process of the downsampling module is: Build an image degradation model: Among them, y and x represent the low-resolution image and high-resolution image respectively, Represents the convolution operation between the image and the blur kernel,↓ s Represents the downsampling scaling factor, n represents noise. Starting from this model, because the convolution operation is a linear operation, the use of a nonlinear downsampling network is obviously inconsistent with the model theory, so a linear downsampling network is used here, that is, there is no activation function; the convolution kernels of the first three layers of the network are 7×7, 5×5 and 3×3, and the convolution kernels of the next three layers are 1×1. The last layer is composed of a hidden layer with a stride of 2. The number of layers here depends on the downsampling scaling factor; the entire network is a 13×13 single-layer linear network, which uses a 13×13 convolution kernel to convolve and downsample the image, which is in line with the definition of the model.

6. The relational reinforcement learning system based on graph structure data according to claim 5, characterized in that: Since single-layer networks are difficult to optimize and converge slowly, multi-layer networks are used for training. During the training process, a weighted combination of color loss, adversarial loss, and feature matching loss is used as the total loss function. The feature matching loss function is used to ensure that the features of the generated image obtained through the encoder are similar to the features of the real-world image obtained through the encoder. The role of color loss and adversarial loss is to ensure that the basic structural information of the downsampled image does not change too much.

7. The relational reinforcement learning system based on graph structure data according to claim 6, characterized in that: The super-resolution reconstruction module uses l1 and l in training per As the loss function, l1loss is defined as: Among them, s is the scaling factor, W and H represent the width and height of the scaled image, Represents the pixel value of the image, F is the super-resolution reconstruction network, this loss function is the super-resolution loss function, and the distance between pixels is measured by the L1 norm.

8. The relational reinforcement learning system based on graph structure data according to claim 7, characterized in that: If only a single l1 loss is used, the super-resolved image often lacks high-frequency parts, and the details and texture of the image cannot be well displayed. In this way, another perceptual loss is introduced, l1 loss. per The loss is defined as: Among them, φ i,j Represents the feature map of the jth convolutional layer before the i-th maximum pooling layer of the VGG network, W i,j , H i,j Represent the length and width in the feature map, I HR Represents the pixel value of the image, F is the super-resolution reconstruction network, and the perceptual loss can compare the features of the high-resolution image after super-resolution with the features of the label image, so that the content and global structure between the images are close. The super-resolution image has more edge, color and detail information and has a good visual effect.

9. A learning method for a relational reinforcement learning system based on graph structured data according to claim 8, characterized in that: The following steps are involved: S1: Input the low-resolution image and the high-resolution image into the degradation feature extraction module for training to obtain an image degradation feature encoder; S2: Input the high-resolution image into the linear downsampling network. The generated low-resolution image and the real-world image are then input into the encoder trained in step S1 together, so that the two can obtain the same degradation features. At the same time, the color loss and pixel loss functions are used to ensure that the original basic structural information of the downsampled image remains unchanged. S3: The low-resolution-high-resolution image data generated in step S2 is input into the super-resolution reconstruction network training to obtain a real-world image super-resolution model.

Images