Image super-resolution model training method, super-resolution method and processing terminal

Abstract

The application discloses a kind of image super-resolution model training methods, comprising: step 1: obtain training image sample group set, training image sample group set includes first resolution image sample and second resolution image sample;Step 2: first resolution image sample and second resolution image sample are divided into several pixel sub-blocks, step 3: calculate the gray standard deviation, gray entropy value, local texture complexity of each pixel sub-block;Step 4: according to local texture complexity, two control parameters of the loss function of super-resolution model are calculated;Step 5: calculate the total loss value of pixel: step 6: the total loss value of pixel is input to super-resolution model and is propagated in reverse, obtains the super-resolution model after training.The application can realize high fidelity in the smooth area of image, and keep detail and perceptual quality in the complex texture area of image.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to an image super-resolution model training method, a super-resolution method, and a processing terminal. Background Technology

[0002] Image super-resolution refers to the process of reconstructing a high-resolution image from a low-resolution image. It is widely used in applications such as medicine, remote sensing and mapping, and video enhancement.

[0003] Existing image super-resolution methods often rely on models such as deep convolutional neural networks and generative adversarial networks for image reconstruction. The quality of the reconstructed image in terms of resolution largely depends on the loss function chosen by the model. Different loss functions exhibit significant differences in fidelity and perceptual quality when processing balanced and multi-textured regions of the image.

[0004] The commonly used loss functions mainly include the following categories: The L1 loss function is trained by minimizing the mean squared error between the predicted and ground truth images, achieving good reconstruction results in smooth regions. However, its drawback is that it is overly biased towards the mean solution, which can lead to overly smoothed results and significant loss of texture and detail.

[0005] L2 loss function: By optimizing the absolute error, it can alleviate the problem of over-smoothing to some extent. However, because its gradient is not smooth enough near zero, it may lead to instability in the training process.

[0006] Charbonnier loss function: The Charbonnier loss function combines the characteristics of both L1 and L2 loss functions, which can improve gradient stability. However, existing methods generally use a fixed form of the Charbonnier loss function, which does not consider the differences in texture complexity of image regions, resulting in an inability to simultaneously achieve both fidelity and perceptual quality in smooth and multi-textured regions.

[0007] Therefore, we hope to find a method that can employ the Charbonnier loss function, achieve high fidelity in smooth regions of an image, and preserve detail and perceptual quality in complex texture regions, thereby comprehensively improving the reconstruction effect of image super-resolution. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide an image super-resolution model training method, a super-resolution method, and a processing terminal, which can solve the problems described in the background art.

[0009] The technical solution to achieve the objective of this invention is: an image super-resolution model training method, comprising the following steps: Step 1: Obtain a set of training image sample groups. The set of training image sample groups includes several training image sample groups. Each training image sample group includes a first-resolution image sample and a second-resolution image sample obtained from the same image but with different resolutions. The first-resolution image sample represents the low-resolution image, and the second-resolution image sample represents the high-resolution image. The first-resolution image sample is input into the super-resolution model for training. The super-resolution model outputs the training image, and the resolution of the training image is made as similar as possible to that of the second-resolution image sample. Step 2: Divide the first-resolution image sample into several blocks with width and height as follows: The pixel sub-blocks divide the second-resolution image sample into several blocks with widths and heights of [missing information]. pixel sub-blocks, in, A is the width multiple of the second resolution image sample relative to the first resolution image sample, and B is the height multiple of the second resolution image sample relative to the first resolution image sample. Step 3: Convert all pixel sub-blocks of the first-resolution image sample to grayscale, calculate the grayscale standard deviation of each pixel sub-block of the first-resolution image sample, and calculate the i-th pixel sub-block of the first-resolution image sample. gray standard deviation The calculation formula is as follows:

[0010] In the formula, Represents the i-th pixel sub-block of the first resolution image sample. The number of pixels, This represents the pixel value within a pixel sub-block. The average grayscale value of a pixel sub-block is calculated using the following formula:

[0011] Calculate the grayscale entropy value of the pixel sub-blocks of the first resolution image sample, where the i-th pixel sub-block of the first resolution image sample... gray entropy value The calculation formula is as follows:

[0012] Calculate the local texture complexity of a pixel sub-block of a first-resolution image sample, where the i-th pixel sub-block of the first-resolution image sample... Local texture complexity The calculation formula is as follows:

[0013] In the formula, It is the weight of the grayscale entropy value of the pixel sub-block; Step 4: Calculate the two control parameters of the loss function of the super-resolution model based on the local texture complexity. and Wherein, the control parameters corresponding to the i-th pixel sub-block of the first resolution image sample and Calculated using the following formula:

[0014]

[0015] In the formula, and These are the control parameters. The maximum and minimum values, and These are the control parameters. The maximum and minimum values, Indicates control parameters scaling factor, Indicates control parameters scaling factor, The function is the sigmoid function, therefore Indicates that the independent variable is The sigmoid function, similarly, Indicates that the independent variable is The sigmoid function; Step 5: Calculate the total pixel loss value representing the training image and the second-resolution image sample using the following formula. :

[0016] In the formula, Let represent the pixel loss value of the i-th pixel sub-block between the training image and the second-resolution image samples, where N represents the total number of pixel sub-blocks in the second-resolution image. This represents the smoothing term, which is a constant. Indicates that the independent variable is control parameters ; Step 6: Calculate the total pixel loss value The input is fed into the super-resolution model for backpropagation, the parameters of the super-resolution model are updated, and the trained super-resolution model is obtained.

[0017] Furthermore, the super-resolution model adopts the CodeFormer model, or the GFPGAN model, or the BasicSR model.

[0018] Furthermore, the loss function of the super-resolution model adopts the Charbonnier loss function.

[0019] Furthermore, step 3 also includes normalizing the grayscale values ​​of the grayscale image to [0,1].

[0020] Furthermore, , , , .

[0021] A super-resolution method includes the following steps: A low-resolution image is input into a super-resolution model, wherein the super-resolution model is trained using the super-resolution model training method described above. The high-resolution image corresponding to the low-resolution image is obtained based on the super-resolution model, thereby completing the super-resolution of the image.

[0022] A processing terminal, comprising: Memory, used to store program instructions; A processor for running the program instructions to execute the steps of the image super-resolution model training method.

[0023] The beneficial effects of this invention are: This invention can achieve high fidelity in smooth areas of an image and maintain detail and perceptual quality in complex texture areas. Furthermore, by introducing two control parameters into existing loss functions, the loss value can reflect the texture of each pixel sub-block. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a preferred embodiment of an image super-resolution model training method; Figure 2 This is a schematic diagram of the processing terminal. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments: like Figure 1 As shown, an image super-resolution model training method includes the following steps: Step 1: Obtain a set of training image samples. This set includes several training image sample groups. Each training image sample group includes a first-resolution image sample and a second-resolution image sample obtained from the same image but with different resolutions. The first-resolution image sample represents a low-resolution image, and the second-resolution image sample represents a high-resolution image. Input the first-resolution image samples into the super-resolution model for training. The super-resolution model outputs training images, aiming to make the resolution of the training images as similar as possible to that of the second-resolution image samples.

[0026] The training image is the reconstructed image obtained after training the super-resolution model.

[0027] It is understandable that the first-resolution image sample and the second-resolution image sample are two images generated from the same image, differing only in resolution. Therefore, there is a one-to-one correspondence between the first-resolution image sample and the second-resolution image sample in each group of training image samples. For example, the image may consist of an image containing only a cup with a pure white background, from which a low-resolution image and a high-resolution image are generated, serving as the first-resolution image sample and the second-resolution image sample, respectively. The second-resolution image sample can be this image, i.e., the original image.

[0028] The terms "low resolution" and "high resolution" refer to relative resolution, not absolute resolution values. In other words, a specific resolution level doesn't determine whether an image is low or high. For example, a first-resolution image is an image whose resolution has been reduced by compression, while a second-resolution image is the original image with no loss of resolution.

[0029] The purpose of this embodiment is to enable the trained super-resolution model to restore a low-resolution image to a higher-resolution image, that is, to obtain a super-resolution image.

[0030] It is understandable that the purpose of inputting the first-resolution image sample into the super-resolution model for training is to make the image output by the trained resolution model as similar as possible to the second-resolution image sample, that is, to make the resolution of the trained image the same as that of the second-resolution image sample.

[0031] For example, the super-resolution model may use the CodeFormer model, GFPGAN model, BasicSR model, etc., and the loss function of the super-resolution model may use the Charbonnier loss function.

[0032] Step 2: Divide the first-resolution image sample into several blocks with width and height as follows: The pixel sub-blocks divide the second-resolution image sample into several blocks with widths and heights of [missing information]. Pixel sub-blocks. Among them, This means that the number of pixel sub-blocks in the second-resolution image is AB times the number of pixel sub-blocks in the first-resolution image. A is the multiple of the width of the second-resolution image sample relative to the first-resolution image sample, and B is the multiple of the height of the second-resolution image sample relative to the first-resolution image sample. In other words, AB represents the multiple of the product of width and height. For example, if the width and height of the second-resolution image sample are 4 times the size of the first-resolution image sample, then A=4 and B=4.

[0033] It is understandable that, assuming A=4 and B=4, if the first-resolution image sample is divided into several blocks with widths and heights as follows... The pixel sub-blocks are defined as m=11 and n=11. Therefore, the second-resolution image sample is divided into several blocks with width and height of... The pixel sub-blocks, that is, p=4, q=4.

[0034] It is understandable that dividing an image into several pixel sub-blocks can be achieved using existing image cropping methods, such as partial overlap cropping, adjacent cropping, and interval cropping. The specific cropping process is existing technology and will not be elaborated upon here.

[0035] Step 3: Convert all pixel sub-blocks of the first resolution image sample into grayscale images, and normalize the grayscale values ​​of the grayscale images to [0,1].

[0036] Calculate the grayscale standard deviation of each pixel sub-block of the first resolution image sample, and the i-th pixel sub-block of the first resolution image sample. gray standard deviation The calculation formula is as follows:

[0037] In the formula, Represents the i-th pixel sub-block of the first resolution image sample. The number of pixels, for example, dividing the first resolution image sample into several blocks with width and height respectively. The pixel sub-blocks, then , This represents the pixel value within a pixel sub-block. The average grayscale value of a pixel sub-block is calculated using the following formula:

[0038] The grayscale standard deviation of a pixel sub-block reflects the magnitude of grayscale fluctuation in the pixel block. In other words, the larger the grayscale standard deviation, the stronger the texture and the richer the details of the pixel sub-block.

[0039] Calculate the grayscale entropy value of the pixel sub-block of the first resolution image sample, where the i-th pixel sub-block of the first resolution image sample... gray entropy value The calculation formula is as follows:

[0040] Calculate the local texture complexity of a pixel sub-block of a first-resolution image sample, where the i-th pixel sub-block of the first-resolution image sample... Local texture complexity The calculation formula is as follows:

[0041] In the formula, This is the weight of the grayscale entropy value of the pixel sub-block, used to balance the importance of the standard deviation and the entropy value, and its value ranges from [0.1, 1]. The default value is 0.5.

[0042] Among them, the gray-scale entropy value reflects the structural complexity of the image region where the pixel sub-block is located from another perspective. The higher the gray-scale entropy value, the more complex the structure of the image region where the pixel sub-block is located. Conversely, the lower the gray-scale entropy value, the smoother the image region where the pixel sub-block is located.

[0043] Local texture complexity is obtained by weighted sum of the grayscale standard deviation and grayscale entropy value of a pixel sub-block.

[0044] Step 4: Since local texture complexity cannot be directly applied to the loss function of the super-resolution model, parameter mapping is needed to incorporate it into the loss function. Two control parameters for calculating the loss function of the super-resolution model are then used based on the local texture complexity. and Wherein, the control parameters corresponding to the i-th pixel sub-block and Calculated using the following formula:

[0045]

[0046] In the formula, and These are the control parameters. The maximum and minimum values, that is, the two, constitute the control parameters. The range of values, usually , Similarly, and These are the control parameters. The maximum and minimum values, that is, the two, constitute the control parameters. The range of values, usually , . Indicates control parameters scaling factor, Indicates control parameters The scaling factor, with a default value of 1 for both, controls the local texture complexity. Sensitivity after function The function is the sigmoid function, therefore Indicates that the independent variable is The sigmoid function. Similarly, Indicates that the independent variable is The sigmoid function.

[0047] Control parameters can be dynamically adjusted based on local texture complexity. and .

[0048] Step 5: Calculate the total pixel loss value representing the training image and the second-resolution image sample using the following formula. :

[0049] In the formula, Let represent the pixel loss value of the i-th pixel sub-block between the training image and the second-resolution image samples, where N represents the total number of pixel sub-blocks in the second-resolution image. This represents the smoothing term, which is a constant, typically set to 0.001, used to prevent gradient explosion during model training. Indicates that the independent variable is control parameters That is to say .

[0050] Step 6: Calculate the total pixel loss value The input is fed into the super-resolution model for backpropagation to update the parameters of the super-resolution model, thus obtaining the updated super-resolution model and completing the training of the super-resolution model, i.e., obtaining the trained super-resolution model.

[0051] Figure 1 Updating network parameters in this context refers to updating the parameters of the super-resolution model. Figure 1 The pixel block in the text refers to the sub-block of pixels. It refers to the pixel sub-blocks of a second-resolution image sample.

[0052] Through the above steps, the trained super-resolution model can be obtained. During the training process, compared to existing super-resolution models, this method introduces two control parameters. and These two parameters take into account local texture complexity, thus enabling high fidelity in smooth areas of the image while preserving detail and perceptual quality in complex texture areas. Furthermore, by introducing these two control parameters into the existing loss function, the loss value can reflect the texture of each individual pixel sub-block.

[0053] Furthermore, by using adaptive parameter mapping, it is possible to better... and By limiting the loss value to a safe range to prevent outliers from affecting model training, and by dividing the image into multiple small pixel sub-blocks and calculating the loss value for each, the local texture of the image can be better reflected.

[0054] This invention also discloses a super-resolution method, comprising the following steps: A low-resolution image is input into a super-resolution model, wherein the super-resolution model is trained using the super-resolution model training method described above. The high-resolution image corresponding to the low-resolution image is obtained based on the super-resolution model, thereby completing the super-resolution of the image.

[0055] like Figure 2 As shown, the present invention also provides a processing terminal 100, which includes: Memory 101 is used to store program instructions; Processor 102 is configured to run the program instructions to perform the steps of the super-resolution model training method.

[0056] The embodiments disclosed in this specification are merely illustrative of one aspect of the invention, and the scope of protection of the invention is not limited to these embodiments. Any other functionally equivalent embodiments fall within the scope of protection of the invention. Those skilled in the art can make various other corresponding changes and modifications based on the technical solutions and concepts described above, and all such changes and modifications should fall within the scope of protection of the claims of this invention.

Claims

1. A method for training an image super-resolution model, characterized in that, Includes the following steps: Step 1: Obtain a set of training image sample groups. The set of training image sample groups includes several training image sample groups. Each training image sample group includes a first-resolution image sample and a second-resolution image sample obtained from the same image but with different resolutions. The first-resolution image sample represents the low-resolution image, and the second-resolution image sample represents the high-resolution image. The first-resolution image sample is input into the super-resolution model for training. The super-resolution model outputs the training image, and the resolution of the training image is made as similar as possible to that of the second-resolution image sample. Step 2: Divide the first-resolution image sample into several blocks with width and height as follows: The pixel sub-blocks divide the second-resolution image sample into several blocks with widths and heights of [missing information]. pixel sub-blocks, in, A is the width multiple of the second resolution image sample relative to the first resolution image sample, and B is the height multiple of the second resolution image sample relative to the first resolution image sample. Step 3: Convert all pixel sub-blocks of the first resolution image sample to grayscale, calculate the grayscale standard deviation of each pixel sub-block of the first resolution image sample, and calculate the i-th pixel sub-block of the first resolution image sample. gray standard deviation The calculation formula is as follows: In the formula, Represents the i-th pixel sub-block of the first resolution image sample. The number of pixels, This represents the pixel value within a pixel sub-block. The average grayscale value of a pixel sub-block is calculated using the following formula: Calculate the grayscale entropy value of the pixel sub-blocks of the first resolution image sample, where the i-th pixel sub-block of the first resolution image sample... grayscale entropy value The calculation formula is as follows: Calculate the local texture complexity of a pixel sub-block of a first-resolution image sample, where the i-th pixel sub-block of the first-resolution image sample... Local texture complexity The calculation formula is as follows: In the formula, These are the weights of the grayscale entropy values ​​of pixel sub-blocks; Step 4: Calculate the two control parameters of the loss function of the super-resolution model based on the local texture complexity. and Wherein, the control parameters corresponding to the i-th pixel sub-block of the first resolution image sample and Calculated using the following formula: In the formula, and These are the control parameters. The maximum and minimum values, and These are the control parameters. The maximum and minimum values, Indicates control parameters scaling factor, Indicates control parameters scaling factor, The function is the sigmoid function, therefore Indicates that the independent variable is The sigmoid function, similarly, Indicates that the independent variable is The sigmoid function; Step 5: Calculate the total pixel loss value between the training image and the second-resolution image sample using the following formula. : In the formula, Let represent the pixel loss value of the i-th pixel sub-block between the training image and the second-resolution image samples, where N represents the total number of pixel sub-blocks in the second-resolution image. This represents the smoothing term, which is a constant. Indicates that the independent variable is control parameters ; Step 6: Calculate the total pixel loss value The input is fed into the super-resolution model for backpropagation, the parameters of the super-resolution model are updated, and the trained super-resolution model is obtained.

2. The image super-resolution model training method according to claim 1, characterized in that, The super-resolution model can be either the CodeFormer model, the GFPGAN model, or the BasicSR model.

3. The image super-resolution model training method according to claim 1 or 2, characterized in that, The loss function used in the super-resolution model is the Charbonnier loss function.

4. The image super-resolution model training method according to claim 1, characterized in that, Step 3 also includes normalizing the grayscale values ​​of the grayscale image to [0,1].

5. The image super-resolution model training method according to claim 1, characterized in that, ， ， ， 。 6. A super-resolution method, characterized in that, Includes the following steps: A low-resolution image is input into a super-resolution model, wherein the super-resolution model is trained using the image super-resolution model training method as described in any one of claims 1-5; The high-resolution image corresponding to the low-resolution image is obtained based on the super-resolution model, thereby completing the super-resolution of the image.

7. A processing terminal, characterized in that, It includes: Memory, used to store program instructions; A processor for running the program instructions to perform the steps of the image super-resolution model training method as described in any one of claims 1-5.

Images