An image super-resolution enhancement method and system

By processing the texture and color features of an image separately using a deep learning network, and optimizing the training with various scaling methods and loss functions, the problem of inconsistent model training results in image super-resolution processing is solved, improving the image detail restoration effect and computation speed.

CN115409704BActive Publication Date: 2026-03-17FUJIAN JOYUSING TECHNOLOGY CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-24
Publication Date
2026-03-17

AI Technical Summary

Technical Problem

In existing image super-resolution processing, differences in model training methods lead to inconsistent results and poor image detail restoration.

Method used

A deep learning network is used to process the texture and color features of the image separately. Through iterative training of the texture feature network and the color feature network, a high-resolution image is finally generated. Various scaling methods and loss functions are used to optimize the training effect.

Benefits of technology

It improves the clarity of image details and adaptability to diverse data, avoids the errors caused by a single scaling method, and achieves faster computing speed and better image restoration results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115409704B_ABST
    Figure CN115409704B_ABST
Patent Text Reader

Abstract

The present application relates to a kind of image super-resolution enhancement algorithm and system, comprising: collection and processing high-definition image;Obtain training sample set;Build depth learning network, the depth learning network includes texture feature network NetV and color feature network NetC, first scale image SrcscaleIMG input texture feature network NetV, output first predicted image, with the difference between first predicted image and gray processing image GrayIMG minimum as goal iterative training, end iteration output optimal texture feature image prediction model, image result after being processed by optimal texture feature image prediction model and scale image SrcscaleIMG new superimposed image is generated by the way of pixel point superposition;Second, superimposed image is input to color feature network NetC, output second predicted image, with the difference between second predicted image and high-definition image SrcIMG minimum as goal iterative training, end iteration output optimal image super-resolution enhancement model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an image super-resolution enhancement method and system. Background Technology

[0002] Image super-resolution has a wide range of applications, such as upscaling old, small-sized photos and videos. However, super-resolution is an ill-posed problem; when image size is reduced, some details are lost. Deep learning can effectively mitigate this issue. This paper employs improved training methods to enhance training performance. The existing deep learning-based image super-resolution method (CN110766612A) constructs an automatic image resolution discrimination model for low-resolution images. This algorithm supports input of original-size low-resolution images, saving computational resources and increasing computation speed. While existing deep learning-based solutions to image super-resolution problems are largely consistent, differences in model training methods lead to varying results. This invention improves the training method using improved models, effectively enhancing training performance and resulting in clearer image details. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention proposes an image super-resolution enhancement method and system.

[0004] The technical solution of the present invention is as follows:

[0005] On the one hand, this invention proposes an image super-resolution enhancement method, the specific steps of which are as follows:

[0006] S1. Randomly collect several high-definition images SrcIMG, and scale and grayscale the high-definition images SrcIMG.

[0007] S2. Obtain several sets of feature images. Each set of feature images includes a high-resolution image SrcIMG, a grayscale processed image GrayIMG corresponding to the high-resolution image SrcIMG, and a scaled image SrcscaleIMG corresponding to the high-resolution image SrcIMG. Add original or processed image labels to each set of feature images to form a training sample set.

[0008] S3. Construct a deep learning network, which includes a texture feature network NetV and a color feature network NetC. First, input the scaled image SrcscaleIMG into the texture feature network NetV to output a first predicted image. Iterate through training with the goal of minimizing the difference between the first predicted image and the grayscale image GrayIMG. After the iteration ends, output the optimal texture feature image prediction model. Generate a new superimposed image by overlaying the image result output by the optimal texture feature image prediction model and the scaled image SrcscaleIMG with pixels. Then, input the superimposed image into the color feature network NetC to output a second predicted image. Iterate through training with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG. After the iteration ends, output the optimal image super-resolution enhancement model.

[0009] S4. Use the image super-resolution enhancement model to upgrade the scaled image after scaling the original image to high definition.

[0010] As a preferred embodiment, the scaling method for the high-definition image SrcIMG includes:

[0011] Quadratic linear interpolation, cubic convolution interpolation, local mean interpolation, and nearest neighbor interpolation.

[0012] In a preferred embodiment, in the two steps of inputting the scaled image SrcscaleIMG into the texture feature network NetV to output a first predicted image and performing iterative training with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG, and inputting the overlaid image into the color feature network NetC to output a second predicted image and performing iterative training with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG, two loss functions are constructed to measure the difference between the first predicted image and the grayscale processed image GrayIMG, and the difference between the second predicted image and the high-resolution image SrcIMG. The two loss functions are as follows:

[0013] The formula for calculating the loss function LossV of the texture feature network NetV is:

[0014] LossV = abs(OutputV - GrayIMG)

[0015] In the formula, OutputV is the output of the texture feature network, GrayIMG is the grayscale image corresponding to the high-resolution image SrcIMG, and abs is the absolute value function.

[0016] The formula for calculating the loss function LossC of the color feature network NetC is as follows:

[0017] LossC = abs(OutputC - SrcIMG)

[0018] In the formula, OutputC is the output of the color feature network, SrcIMG is the high-resolution image, and abs is the absolute value function.

[0019] In a preferred embodiment, the texture feature network specifically includes an input layer, a convolutional layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer is used to input an image, and the convolutional layer performs convolution operations on the input image to output a first feature map. The feature extraction layer includes a convolutional module and a fully connected module, used to perform convolutional and fully connected operations on the first feature map, respectively. The feature fusion layer performs superposition processing on the data after the convolutional and fully connected operations on the first feature map to generate a final feature map. The superposition formula is as follows:

[0020] Add layer = a * Conv + (1 - a) * Dense

[0021] In the formula, a is a trainable variable, conv is a convolutional layer function, and Dense is a fully connected layer function;

[0022] The output layer is used to output the final feature map.

[0023] On the other hand, the present invention proposes an image super-resolution enhancement system, comprising:

[0024] Image processing module: randomly collects several high-definition images SrcIMG, and performs scaling and grayscale processing on the high-definition images SrcIMG;

[0025] Feature image extraction module: acquires several sets of feature images, each set of feature images includes a high-resolution image SrcIMG, a grayscale processed image GrayIMG corresponding to the high-resolution image SrcIMG, and a scaled image SrcscaleIMG corresponding to the high-resolution image SrcIMG. Add original or processed image labels to each set of feature images to form a training sample set.

[0026] Deep Learning Network Module: A deep learning network is constructed, comprising a texture feature network NetV and a color feature network NetC. First, the scaled image SrcscaleIMG is input into the texture feature network NetV, which outputs a first predicted image. Iterative training is then performed with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG. The iteration ends, and the optimal texture feature image prediction model is output. The image result processed by the optimal texture feature image prediction model and the scaled image SrcscaleIMG are then combined using pixel-by-pixel stacking to generate a new overlay image. Next, the overlay image is input into the color feature network NetC, which outputs a second predicted image. Iterative training is then performed with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG. The iteration ends, and the optimal image super-resolution enhancement model is output.

[0027] As a preferred embodiment, the scaling method for the high-definition image SrcIMG includes:

[0028] Quadratic linear interpolation, cubic convolution interpolation, local mean interpolation, and nearest neighbor interpolation.

[0029] In a preferred embodiment, in the two steps of inputting the scaled image SrcscaleIMG into the texture feature network NetV to output a first predicted image and performing iterative training with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG, and inputting the overlaid image into the color feature network NetC to output a second predicted image and performing iterative training with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG, two loss functions are constructed to measure the difference between the first predicted image and the grayscale processed image GrayIMG, and the difference between the second predicted image and the high-resolution image SrcIMG. The two loss functions are as follows:

[0030] The formula for calculating the loss function LossV of the texture feature network NetV is:

[0031] LossV = abs(OutputV - GrayIMG)

[0032] In the formula, OutputV is the output of the texture feature network, GrayIMG is the grayscale image corresponding to the high-resolution image SrcIMG, and abs is the absolute value function.

[0033] The formula for calculating the loss function LossC of the color feature network NetC is as follows:

[0034] LossC = abs(OutputC - SrcIMG)

[0035] In the formula, OutputC is the output of the color feature network, SrcIMG is the high-resolution image, and abs is the absolute value function.

[0036] In a preferred embodiment, the texture feature network specifically includes an input layer, a convolutional layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer is used to input an image, and the convolutional layer performs convolution operations on the input image to output a first feature map. The feature extraction layer includes a convolutional module and a fully connected module, used to perform convolutional and fully connected operations on the first feature map, respectively. The feature fusion layer performs superposition processing on the data after the convolutional and fully connected operations on the first feature map to generate a final feature map. The superposition formula is as follows:

[0037] Add layer = a * Conv + (1 - a) * Dense

[0038] In the formula, a is a trainable variable, conv is a convolutional layer function, and Dense is a fully connected layer function;

[0039] The output layer is used to output the final feature map.

[0040] On the other hand, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image super-resolution enhancement method described in any embodiment of the present invention.

[0041] On the other hand, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image super-resolution enhancement method described in any embodiment of the present invention.

[0042] The present invention has the following beneficial effects:

[0043] 1. This invention proposes an image super-resolution enhancement method that improves the training method by processing the texture and color features of the image separately, thereby more finely restoring the details of the image and obtaining a clearer image.

[0044] 2. This invention employs multiple scaling methods, resulting in more diverse data. This diverse data can better adapt to real-world scenarios while avoiding errors that may result from repeatedly using a single scaling method.

[0045] 3. The training model uses a combination of fully connected layers and convolutional layers, which allows the size of the input image to be determined according to the actual situation rather than being fixed to a single size. Attached Figure Description

[0046] Figure 1 This is a flowchart of the present invention;

[0047] Figure 2 For training models in deep learning networks;

[0048] Figure 3 This is the network unfolded diagram of the NetV texture feature network. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0051] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0053] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0054] Example 1:

[0055] See Figure 1 An image super-resolution enhancement method, the specific steps of which are as follows:

[0056] S1. Randomly collect several high-definition images SrcIMG, and scale and grayscale the high-definition images SrcIMG.

[0057] In practice, a large number of high-definition images SrcIMG containing random content are collected, and the high-definition images SrcIMG are scaled and grayscaled before being grouped and placed.

[0058] S2. Obtain several sets of feature images. Each set of feature images includes a high-resolution image SrcIMG, a grayscale processed image GrayIMG corresponding to the high-resolution image SrcIMG, and a scaled image SrcscaleIMG corresponding to the high-resolution image SrcIMG. Add original or processed image labels to each set of feature images to form a training sample set.

[0059] In practice, several groups of high-resolution SrcIMG images that have been grouped and classified as described above are selected, and the original images and the images that have been scaled or grayscaled in each group are labeled to form a training sample set.

[0060] S3, see also Figure 2 A deep learning network is constructed, comprising a texture feature network NetV and a color feature network NetC. First, a scaled image SrcscaleIMG is input into the texture feature network NetV, which outputs a first predicted image. Iterative training is then performed with the goal of minimizing the difference between the first predicted image and the grayscale image GrayIMG. The iteration ends, and the optimal texture feature image prediction model is output. The image result processed by the optimal texture feature image prediction model and the scaled image SrcscaleIMG are then combined to generate a new overlay image. Next, the overlay image is input into the color feature network NetC, which outputs a second predicted image. Iterative training is then performed with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG. The iteration ends, and the optimal image super-resolution enhancement model is output.

[0061] In practice, the texture feature network NetV is iteratively trained by taking the scaled image SrcscaleIMG from each training sample set as input and the grayscale processed image GrayIMG as output; the superimposed image obtained by superimposing the scaled image SrcscaleIMG and the output image of the texture feature network NetV is iteratively trained by taking the high-resolution image SrcIMG as input and the color feature network NetC as output.

[0062] S4. Use the image super-resolution enhancement model to upgrade the scaled image after scaling the original image to high definition.

[0063] As a preferred embodiment of this invention, the method for scaling the high-definition image includes:

[0064] Quadratic linear interpolation, cubic convolution interpolation, local mean interpolation, and nearest neighbor interpolation.

[0065] The nearest-neighbor interpolation method is the fastest to compute, but it has the worst scaling performance.

[0066] The quadratic linear interpolation method is slower than the nearest neighbor interpolation method but faster than the cubic convolution interpolation method, making it a relatively balanced method.

[0067] The cubic convolution interpolation method achieves better scaling results than the quadratic linear interpolation method and the nearest neighbor interpolation method, but it is slower in computation.

[0068] The advantage of the local mean method is that it is simple to calculate and fast; however, while reducing noise, it blurs the image, especially at edges and details.

[0069] Four methods are used to scale images, resulting in more diverse data. This diverse data can better adapt to real-world scenes and avoid errors that may result from repeatedly using a single scaling method.

[0070] In a preferred embodiment of this invention, in the two steps of inputting the scaled image SrcscaleIMG into the texture feature network NetV to output a first predicted image and performing iterative training with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG, and inputting the overlaid image into the color feature network NetC to output a second predicted image and performing iterative training with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG, two loss functions are constructed to measure the difference between the first predicted image and the grayscale processed image GrayIMG and the difference between the second predicted image and the high-resolution image SrcIMG. The two loss functions are specifically as follows:

[0071] The formula for calculating the loss function LossV of the texture feature network NetV is:

[0072] LossV = abs(OutputV - GrayIMG)

[0073] In the formula, OutputV is the output of the texture feature network, GrayIMG is the grayscale image corresponding to the high-resolution image SrcIMG, and abs is the absolute value function.

[0074] The formula for calculating the loss function LossC of the color feature network NetC is as follows:

[0075] LossC = abs(OutputC - SrcIMG)

[0076] In the formula, OutputC is the output of the color feature network, SrcIMG is the high-resolution image, and abs is the absolute value function.

[0077] In this step, color and texture, as the two most important features of an image, are processed separately and then compared with the original image and the image after grayscale processing. This allows for a more accurate comparison of the differences between the scaled image and the original image, and a more refined use of the model to restore the blurred areas of the scaled image.

[0078] In a preferred embodiment of this invention, the texture feature network specifically includes an input layer, a convolutional layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer is used to input an image, and the convolutional layer performs convolution operations on the input image to output a first feature map. The feature extraction layer includes a convolutional module and a fully connected module, used to perform convolutional and fully connected operations on the first feature map, respectively. The feature fusion layer performs superposition processing on the data after the convolutional and fully connected operations on the first feature map to generate a final feature map. The superposition formula is as follows:

[0079] Add layer = a * Conv + (1 - a) * Dense

[0080] In the formula, a is a trainable variable, conv is a convolutional layer function, and Dense is a fully connected layer function;

[0081] The output layer is used to output the final feature map.

[0082] Participate specifically Figure 3 , is the unfolded diagram of the NetV texture feature network.

[0083] Fully connected layers can extract more detailed features, and when stacked with convolutional layers, they can combine coarse and fine features, resulting in more detailed and accurate images.

[0084] When adding layers, the corresponding positions of the two arrays are added together to achieve the overlay function.

[0085] Example 2:

[0086] An image super-resolution enhancement system, comprising:

[0087] Image processing module: randomly collects several high-definition images SrcIMG, and performs scaling and grayscale processing on the high-definition images SrcIMG;

[0088] Feature image extraction module: acquires several sets of feature images, each set of feature images includes a high-resolution image SrcIMG, a grayscale processed image GrayIMG corresponding to the high-resolution image SrcIMG, and a scaled image SrcscaleIMG corresponding to the high-resolution image SrcIMG. Add original or processed image labels to each set of feature images to form a training sample set.

[0089] Deep Learning Network Module: A deep learning network is constructed, comprising a texture feature network NetV and a color feature network NetC. First, the scaled image SrcscaleIMG is input into the texture feature network NetV, which outputs a first predicted image. Iterative training is then performed with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG. The iteration ends, and the optimal texture feature image prediction model is output. The image result processed by the optimal texture feature image prediction model and the scaled image SrcscaleIMG are then combined using pixel-by-pixel stacking to generate a new overlay image. Next, the overlay image is input into the color feature network NetC, which outputs a second predicted image. Iterative training is then performed with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG. The iteration ends, and the optimal image super-resolution enhancement model is output.

[0090] As a preferred embodiment of this invention, the method for scaling the high-definition image SrcIMG includes:

[0091] Quadratic linear interpolation, cubic convolution interpolation, local mean interpolation, and nearest neighbor interpolation.

[0092] In a preferred embodiment of this invention, in the two steps of inputting the scaled image SrcscaleIMG into the texture feature network NetV to output a first predicted image and performing iterative training with the goal of minimizing the difference between the first predicted image and the grayscale processed image GrayIMG, and inputting the overlaid image into the color feature network NetC to output a second predicted image and performing iterative training with the goal of minimizing the difference between the second predicted image and the high-resolution image SrcIMG, two loss functions are constructed to measure the difference between the first predicted image and the grayscale processed image GrayIMG and the difference between the second predicted image and the high-resolution image SrcIMG. The two loss functions are specifically as follows:

[0093] The formula for calculating the loss function LossV of the texture feature network NetV is:

[0094] LossV = abs(OutputV - GrayIMG)

[0095] In the formula, OutputV is the output of the texture feature network, GrayIMG is the grayscale image corresponding to the high-resolution image SrcIMG, and abs is the absolute value function.

[0096] The formula for calculating the loss function LossC of the color feature network NetC is as follows:

[0097] LossC = abs(OutputC - SrcIMG)

[0098] In the formula, OutputC is the output of the color feature network, SrcIMG is the high-resolution image, and abs is the absolute value function.

[0099] In a preferred embodiment of this invention, the texture feature network specifically includes an input layer, a convolutional layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer is used to input an image, and the convolutional layer performs convolution operations on the input image to output a first feature map. The feature extraction layer includes a convolutional module and a fully connected module, used to perform convolutional and fully connected operations on the first feature map, respectively. The feature fusion layer performs superposition processing on the data after the convolutional and fully connected operations on the first feature map to generate a final feature map. The superposition formula is as follows:

[0100] Add layer = a * Conv + (1 - a) * Dense

[0101] In the formula, a is a trainable variable, conv is a convolutional layer function, and Dense is a fully connected layer function;

[0102] The output layer is used to output the final feature map.

[0103] Example 3:

[0104] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image super-resolution enhancement method described in any embodiment of the present invention.

[0105] Example 4:

[0106] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image super-resolution enhancement method according to any embodiment of the present invention.

[0107] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An image super-resolution enhancement method, characterized in that, The specific steps are as follows: S1, a plurality of high-definition images SrcIMG are randomly collected, and the high-definition images SrcIMG are scaled and grayed; S2, a plurality of groups of feature images are obtained, each group of feature images includes a high-definition image SrcIMG, a grayed image GrayIMG corresponding to the high-definition image SrcIMG after gray processing, and a scaled image SrcscaleIMG corresponding to the high-definition image SrcIMG after scaling processing, and an original image or a processed image label is added to each group of feature images to form a training sample set; S3, a deep learning network is built, the deep learning network includes a texture feature network NetV and a color feature network NetC, the scaled image SrcscaleIMG is first input into the texture feature network NetV, a first prediction image is output, and the minimum difference between the first prediction image and the grayed image GrayIMG is taken as the target for iterative training, the optimal texture feature image prediction model is output after the iteration is ended, and the image result output after the optimal texture feature image prediction model is processed and the scaled image SrcscaleIMG are superimposed by pixel points to generate a new superimposed image; then the superimposed image is input into the color feature network NetC, a second prediction image is output, and the minimum difference between the second prediction image and the high-definition image SrcIMG is taken as the target for iterative training, and the optimal image super-resolution enhancement model is output after the iteration is ended; S4, the image super-resolution enhancement model is used to complete high-definition of the scaled image after the original image is scaled.

2. The image super-resolution enhancement method of claim 1, wherein, The scaling method for the high-definition image SrcIMG includes: Quadratic linear interpolation method, cubic convolution interpolation method, local mean method and nearest neighbor interpolation method.

3. The image super-resolution enhancement method of claim 1, wherein, In the two steps of inputting the scaled image SrcscaleIMG into the texture feature network NetV to output the first prediction image, taking the minimum difference between the first prediction image and the grayed image GrayIMG as the target for iterative training, and inputting the superimposed image into the color feature network NetC to output the second prediction image, taking the minimum difference between the second prediction image and the high-definition image SrcIMG as the target for iterative training, two loss functions are constructed to measure the difference between the first prediction image and the grayed image GrayIMG and the difference between the second prediction image and the high-definition image SrcIMG, and the two loss functions are specifically: The calculation formula of the loss function LossV of the texture feature network NetV is: LossV=abs(OutputV-GrayIMG) In the formula, OutputV is the output of the texture feature network, GrayIMG is the image processed by the high-definition image SrcIMG, and abs is an absolute value function; The calculation formula of the loss function LossC of the color feature network NetC is: LossC=abs(OutputC-SrcIMG) In the formula, OutputC is the output of the color feature network, SrcIMG is the high-definition image, and abs is an absolute value function.

4. The image super-resolution enhancement method of claim 1, wherein, The texture feature network specifically comprises an input layer, a convolution layer, a feature extraction layer, a feature fusion layer and an output layer; the input layer is used for inputting an image, the convolution layer is used for performing a convolution operation on the input image to output a first feature map; the feature extraction layer comprises a convolution module and a full connection module, and is used for performing convolution operation and full connection operation on the first feature map respectively; the feature fusion layer is used for performing superposition processing on the data after the convolution operation and the full connection operation on the first feature map to generate a final feature map, and a superposition formula is specifically: Add layer = a * Conv + (1-a) * Dense In the formula, a is a trainable variable, Conv is a convolution layer function, and Dense is a full connection layer function; The output layer is used for outputting the final feature map.

5. An image super-resolution enhancement system characterized by, It comprises: An image processing module: a plurality of high-definition images SrcIMG are randomly collected, and scaling and grayscale processing are performed on the high-definition images SrcIMG; A feature image extraction module: a plurality of groups of feature images are acquired, each group of feature images comprises a high-definition image SrcIMG, a grayscale processing image GrayIMG corresponding to the high-definition image SrcIMG after grayscale processing and a scaling image SrcscaleIMG corresponding to the high-definition image SrcIMG after scaling processing, and an original image or a processed image label is added to each group of feature images to form a training sample set; A deep learning network module: a deep learning network is built, the deep learning network comprises a texture feature network NetV and a color feature network NetC, the scaling image SrcscaleIMG is input into the texture feature network NetV first, a first prediction image is output, iterative training is performed with the minimum difference between the first prediction image and the grayscale processing image GrayIMG as the target, an optimal texture feature image prediction model is output after the iterative training is ended, and an image result output after the optimal texture feature image prediction model processing and the scaling image SrcscaleIMG are superimposed by pixel points to generate a new superimposed image; then the superimposed image is input into the color feature network NetC, a second prediction image is output, and iterative training is performed with the minimum difference between the second prediction image and the high-definition image SrcIMG as the target, and an optimal image super-resolution enhancement model is output after the iterative training is ended.

6. The image super-resolution enhancement system of claim 5, wherein, The scaling method for the high-definition image SrcIMG comprises: A quadratic linear interpolation method, a cubic convolution interpolation method, a local mean method and a nearest neighbor interpolation method.

7. The image super-resolution enhancement system of claim 5, wherein, In the two steps of inputting the scaling image SrcscaleIMG into the texture feature network NetV to output a first prediction image, performing iterative training with the minimum difference between the first prediction image and the grayscale processing image GrayIMG as the target, and inputting the superimposed image into the color feature network NetC to output a second prediction image, performing iterative training with the minimum difference between the second prediction image and the high-definition image SrcIMG as the target, two loss functions are constructed to measure the difference between the first prediction image and the grayscale processing image GrayIMG and the difference between the second prediction image and the high-definition image SrcIMG, and the two loss functions are specifically: The calculation formula of the loss function LossV of the texture feature network NetV is: LossV = abs (OutputV-GrayIMG) In the formula, OutputV is the output of the texture feature network, GrayIMG is the image processed by gray scale corresponding to the high-definition image SrcIMG, and abs is an absolute value function. The calculation formula of the loss function LossC of the color feature network NetC is: LossC = abs (OutputC-SrcIMG) In the formula, OutputC is the output of the color feature network, SrcIMG is the high-definition image, and abs is an absolute value function.

8. The image super-resolution enhancement system of claim 5, wherein, The texture feature network specifically includes an input layer, a convolution layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer is used to input an image, the convolution layer is used to perform a convolution operation on the input image to output a first feature map, the feature extraction layer includes a convolution module and a full connection module, and is used to perform convolution and full connection operations on the first feature map, respectively. The feature fusion layer is used to perform superposition processing on the data after the convolution and full connection operations on the first feature map to generate a final feature map, and the superposition formula is specifically: Add layer = a * Conv + (1-a) * Dense In the formula, a is a trainable variable, conv is a convolution layer function, and Dense is a full connection layer function. The output layer is used to output the final feature map.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the image super-resolution enhancement method according to any one of claims 1 to 4.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the image super-resolution enhancement method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Image super-resolution method based on deep learning

    CN110766612A

  • Image super-resolution reconstruction method and system based on edge detection

    CN111062872A

  • Image processing method and apparatus, portrait super-resolution reconstruction method and apparatus, and portrait super-resolution reconstruction model training method and apparatus, electronic device, and storage medium

    WO2022057837A1