A controllable image super-resolution method and system based on neural network

Abstract

The present invention provides a method and system for adjustable image super-resolution based on a neural network, which specifically relates to the field of image processing technology. The scheme includes: collecting a number of images to be processed; based on preset super-resolution parameters, using a trained adjustable super-resolution network to process each image to be processed respectively, and obtaining a super-resolution image corresponding to each image to be processed. The scheme allows flexible adjustment of upsampling multiples, deblurring strength, and denoising strength according to actual application requirements. The flexible control mechanism of the externally controlled super-resolution parameters can effectively respond to image processing requirements in different scenarios, which is conducive to improving the generalization ability of the model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a neural network-based adjustable image super-resolution method and system. Background Art

[0002] With the rapid development of medical imaging technology, medical images are becoming increasingly important in disease diagnosis and treatment planning. However, many medical images often suffer from problems such as insufficient clarity and loss of details due to equipment resolution limitations, unsatisfactory acquisition conditions, or imaging noise. These problems seriously affect doctors' interpretation of images and the accuracy of diagnosis. Therefore, improving the resolution of medical images has become a research hotspot.

[0003] Super-resolution technology (SR) aims to reconstruct low-resolution images into high-resolution images through algorithms to enhance the details and clarity of the images. Traditional medical image super-resolution algorithms (such as linear interpolation, dictionary learning, and local embedding, etc.), although they can improve the resolution of images, often lead to loss of details or artifacts. Although deep learning models (such as VDSR (Very Deep Super Resolution), SRGAN (Super-Resolution Generative Adversarial Network), etc.) can enhance image details and generation quality, the fixed network structure and parameters make it impossible for the network to be personalized according to different scenarios, resulting in the super-resolution effect of the reconstructed super-resolution images being unstable in different scenarios. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide an adjustable image super-resolution method and system based on a neural network, aiming to solve the problem in the prior art that the super-resolution effect of the reconstructed super-resolution image is unstable in different scenarios due to the inability to perform personalized adjustments according to different scenarios.

[0005] In order to achieve the above object, the present invention provides a first aspect of a controllable image super-resolution method based on a neural network, comprising:

[0006] Collecting a number of images to be processed;

[0007] Based on preset super-resolution parameters, each of the images to be processed is processed respectively using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

[0008] Optionally, the adjustable super-resolution network includes a deblurring module and a super-resolution network, and based on preset super-resolution parameters, each of the images to be processed is processed using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed, including:

[0009] Based on a preset deblurring strength parameter, using the deblurring module to perform deblurring processing on each of the images to be processed, to obtain a deblurred image corresponding to each of the images to be processed;

[0010] Based on the preset upsampling multiple and denoising strength parameters, the super-resolution network is used to infer each of the deblurred images respectively to obtain a super-resolution image corresponding to each of the images to be processed.

[0011] Optionally, based on a preset upsampling multiple and denoising strength parameter, using the super-resolution network to infer each of the deblurred images respectively to obtain a super-resolution image corresponding to each of the images to be processed includes:

[0012] Determining a consistency weight parameter and a super-resolution reconstructed image based on the deblurred image, the denoising intensity parameter and the upsampling multiple;

[0013] The consistency weight parameter is used to adjust the weights of the super-resolution reconstructed image and the image to be processed, so as to obtain a super-resolution image corresponding to each image to be processed.

[0014] Optionally, the training process of the adjustable super-resolution network includes:

[0015] Collect several high-resolution image samples and paired low-resolution image samples to construct a training set;

[0016] Based on deep neural network, construct an initial adjustable super-resolution network;

[0017] Based on a preset deblurring strength parameter, a preset denoising strength parameter and a preset upsampling multiple, the initial adjustable super-resolution network is iteratively trained using the training set until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

[0018] Optionally, the iterative training of the initial adjustable super-resolution network using the training set based on a preset deblurring strength parameter, a preset denoising strength parameter and a preset upsampling multiple until the network converges or the loss value satisfies a preset loss threshold, to obtain a trained adjustable super-resolution network, comprises:

[0019] Inputting the low-resolution image samples and the deblurring strength parameters into the initial adjustable super-resolution network, and outputting a deblurred image corresponding to each of the low-resolution images;

[0020] Based on the denoising intensity parameter and the upsampling multiple, the deblurred image is processed using the initial adjustable super-resolution network to obtain a super-resolution image;

[0021] The initial adjustable super-resolution network is iteratively trained using the super-resolution image and the high-resolution image samples until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

[0022] Optionally, the loss value calculation process includes:

[0023] The difference between the super-resolution image and the corresponding high-resolution image sample is calculated using a preset image difference measurement loss function to obtain a loss value.

[0024] A second aspect of the present invention provides a controllable image super-resolution system based on a neural network, the system comprising:

[0025] An image acquisition module, used for acquiring a number of images to be processed;

[0026] The super-resolution processing module is used to process each of the images to be processed based on preset super-resolution parameters using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

[0027] A third aspect of the present invention provides an intelligent terminal, comprising a memory, a processor, and a neural network-based adjustable image super-resolution program stored in the memory and executable on the processor, wherein the neural network-based adjustable image super-resolution program, when executed by the processor, implements any step of the above-mentioned neural network-based adjustable image super-resolution method.

[0028] A fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a neural network-based adjustable image super-resolution program, which, when executed by a processor, implements any step of the above-mentioned neural network-based adjustable image super-resolution method.

[0029] Compared with the prior art, the beneficial effects of this solution are as follows:

[0030] The present invention constructs a new adjustable image super-resolution network framework, including a deblurring module and a super-resolution network. The deblurring module deblurs the low-resolution image according to the deblurring strength, which can effectively reduce the blurring effect of the image and improve the clarity of the image; the super-resolution network maps the deblurred image to a super-resolution image according to the denoising strength parameter and the upsampling multiple, which can significantly improve the image signal-to-noise and specific spatial resolution, making the image details clearer and smoother. In addition, the consistency weight subnetwork can adaptively output the optimal consistency weight coefficient, perform consistency calibration on the super-resolution image, and prevent the image from being distorted during the super-resolution process.

[0031] The trained adjustable super-resolution network of the present invention allows the user to flexibly adjust key parameters such as upsampling multiple, deblurring strength and denoising strength according to actual application requirements. This flexible control mechanism of externally controlled super-resolution parameters can effectively respond to image processing requirements in different scenarios, which is conducive to improving the generalization ability of the model and improving the usability and adaptability of the algorithm in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 A brief flow chart of the neural network-based adjustable image super-resolution method of the present invention;

[0034] Figure 2 It is a processing block diagram of the adjustable image super-resolution method based on a neural network of the present invention;

[0035] Figure 3 This is a block diagram of the adjustable super-resolution network training of the present invention;

[0036] Figure 4 This is a block diagram of image degradation processing of the present invention;

[0037] Figure 5 It is a schematic diagram of the module of the neural network-based adjustable image super-resolution system of the present invention;

[0038] Figure 6 It is a schematic diagram of the structure of the intelligent terminal of the present invention. DETAILED DESCRIPTION

[0039] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0040] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0041] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0042] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0043] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0045] In view of the problem in the prior art that the super-resolution effect of the reconstructed super-resolution image cannot be adjusted according to different scenes, resulting in unstable super-resolution effect in different scenes, the present invention proposes an adjustable image super-resolution method based on a neural network, which is deployed on electronic devices such as computers and servers, and is applied to the scene of super-resolution enhancement of images, aiming at the situation where the super-resolution of images can be flexibly adjusted. The types of the above images are not limited, including but not limited to medical images such as CT images, magnetic resonance images, ultrasound images, X-ray images, and images in various other fields. Specifically, such as Figure 1 and Figure 2 As shown, the steps of the method in this embodiment include:

[0046] Step S100: Acquire a number of images to be processed.

[0047] Specifically, a number of low-resolution images to be processed are collected. These images to be processed may be images with the same or different resolutions taken in the same or different scenes, and this embodiment does not impose any specific limitation on this.

[0048] Step S200: Based on preset super-resolution parameters, each of the images to be processed is processed using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

[0049] Specifically, according to the super-resolution control requirements of the image, a super-resolution parameter is set for each image to be processed. Among them, the super-resolution parameter is used to adjust the ratio of the size enlargement of the final generated super-resolution image relative to the original image to be processed. It should be noted that the super-resolution parameter in this embodiment may refer to a parameter that affects the super-resolution of the image, or refer to multiple parameters related to the super-resolution of the image. When referring to multiple parameters related to the super-resolution of the image, the enlargement ratio of the image size is the result of the joint action of these parameters. For example, if the size enlargement ratio corresponding to the super-resolution parameter is set to 2, it means that the image to be processed is enlarged twice to obtain a high-resolution image, and this multiple determines the resolution and detail of the final image. Then, each image to be processed is processed separately using an adjustable super-resolution network built and trained based on a deep learning model to obtain a super-resolution image corresponding to each image to be processed. Among them, the deep learning model can be a convolutional neural network (CNN), a generative adversarial network (GAN), etc., and this embodiment does not make specific restrictions on this; the processing steps of the adjustable super-resolution network include but are not limited to feature extraction, upsampling, cropping or filling steps to ensure that the obtained super-resolution image meets the expected resolution and size requirements. It is easy to understand that this embodiment describes the process of processing a single image to be processed or a group of images to be processed with the same super-resolution multiple. Those skilled in the art can process the images to be processed corresponding to different super-resolution multiples in batches according to the processing logic, which will not be repeated here.

[0050] Furthermore, before each image to be processed is input into the trained adjustable super-resolution network for processing, the image to be processed is also pre-processed by adjusting the image size, normalizing, format conversion, etc., so that the image to be processed meets the processing requirements of the trained adjustable super-resolution network, which helps to improve the super-resolution stability of the image to be processed. The generated super-resolution image can also be post-processed, such as denoising, sharpening, etc., to further improve the image quality.

[0051] In this embodiment, the trained adjustable super-resolution network allows the user to adjust the super-resolution parameters related to image super-resolution in real time according to specific needs. This flexible control mechanism of externally controlled super-resolution parameters can effectively respond to image processing requirements in different scenarios, which is conducive to improving the usability and adaptability of the algorithm in practical applications.

[0052] In a preferred embodiment, the adjustable super-resolution network includes a deblurring module and a super-resolution network. In step S200, based on preset super-resolution parameters, each of the images to be processed is processed using the trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed, including:

[0053] Step S210: Based on a preset deblurring strength parameter, the deblurring module is used to perform deblurring processing on each of the images to be processed, so as to obtain a deblurred image corresponding to each of the images to be processed.

[0054] Specifically, in this embodiment, the adjustable super-resolution network includes a deblurring module and a super-resolution network. The deblurring module is used to deblur the image to be processed, and the super-resolution network is used to perform super-resolution mapping on the deblurred image to obtain a super-resolution image.

[0055] Considering that image blur may be caused by a variety of factors, such as misfocus, aberration of the optical system, relative motion during imaging, atmospheric turbulence effects, low light, etc., the blurring conditions of different images often vary greatly. This embodiment determines the deblurring intensity by setting the value of the deblurring intensity parameter in the deblurring module to effectively remove image blur and improve image quality, avoiding image distortion or artifacts. Among them, the deblurring intensity parameter is used to control the processing strength of the deblurring module. The higher the value, the stronger the deblurring effect and the clearer and sharper the image.

[0056] Step S220: Based on the preset upsampling multiple and denoising strength parameters, the super-resolution network is used to infer each of the deblurred images respectively to obtain a super-resolution image corresponding to each of the images to be processed.

[0057] Specifically, according to the super-resolution control requirements of the image, the upsampling multiple and denoising strength parameters are preset for each image to be processed. Among them, the upsampling multiple is used to indicate the proportional factor of the size enlargement of the image to be processed. The larger the multiple, the higher the multiple of the image size is enlarged. The denoising strength indicates the strength of removing image noise. The higher the value, the greater the denoising strength, which can effectively remove image noise, but may also cause loss of image details or over-smoothing; the lower the value, the smaller the denoising strength, retaining more image details, but more noise may also remain. The appropriate denoising strength needs to be adjusted according to the noise level and detail richness of the image to achieve the best effect. Then, the upsampling multiple, denoising strength parameter and deblurred image are input into the super-resolution network together, and the upsampling multiple and denoising strength parameter are used to guide and regulate the reasoning process of the super-resolution network for the deblurred image, so as to obtain the super-resolution image corresponding to each image to be processed.

[0058] In this embodiment, the low-resolution image is deblurred according to the deblurring strength by the deblurring module, which can effectively reduce the blurring effect of the image and thus improve the clarity of the image; the deblurred image is mapped into a super-resolution image according to the denoising strength parameter and the upsampling multiple by the super-resolution network, which can significantly improve the image signal-to-noise and specific spatial resolution, making the image details clearer and smoother.

[0059] The super-resolution network in this embodiment includes a consistency weight sub-network and a super-resolution sub-network to improve the resolution of the image and restore the image details while ensuring the consistency and accuracy of the image. The process of using the consistency weight sub-network and the super-resolution sub-network to process each of the deblurred images respectively specifically includes:

[0060] First, the denoising strength parameter, upsampling factor and deblurred image are input into the consistency weight sub-network together, and the consistency weight parameter is output.

[0061] Then, based on the deblurred image, denoising strength parameters and upsampling multiples, the super-resolution sub-network is used to generate a super-resolution reconstructed image to improve the resolution of the image and restore the image details. The super-resolution sub-network is a deep convolutional neural network that contains multiple convolutional layers, nonlinear activation functions, residual connections and upsampling modules (such as transposed convolution, bilinear interpolation or nearest neighbor interpolation, etc.). In the process of reconstructing image details by the super-resolution sub-network, the low-level and high-level features of the image are first extracted through the convolutional layer, and then these features are mapped to a higher resolution space through the upsampling module, thereby obtaining a super-resolution reconstructed image.

[0062] Finally, through fusion strategies such as weighted summation and weighted average, the super-resolution reconstructed image and the image to be processed are fused using the consistency weight parameter, so as to balance the fusion weight between the super-resolution reconstructed image and the image to be processed using the consistency weight parameter, and obtain the super-resolution image corresponding to each image to be processed, so that these images have higher resolution, more details and better visual effects.

[0063] In this embodiment, the super-resolution network combines the consistency weight sub-network and the super-resolution sub-network, and uses the denoising intensity parameter, upsampling factor and deblurred image to adaptively estimate the consistency weight parameter and generate a super-resolution reconstructed image, and then uses the consistency weight parameter to regulate the fusion weight of the super-resolution reconstructed image and the image to be processed, thereby realizing the calibration and high-resolution reconstruction of the deblurred image, which is beneficial to improving the performance and consistency of image super-resolution.

[0064] like Figure 3 As shown, in a preferred embodiment, the training process of the adjustable super-resolution network in step S200 includes:

[0065] Step M100: Collect a number of high-resolution image samples and paired low-resolution image samples to construct a training set.

[0066] Specifically, sufficient paired training image data is collected, each pair of training data includes a high-resolution image sample and a low-resolution image sample, and a training set is constructed. The high-resolution image samples are directly collected, and the low-resolution image samples are obtained by degradation processing of the high-resolution image samples, such as Figure 4 As shown, it specifically includes: randomly generating one or more groups of blur intensity parameters, noise intensity parameters and downsampling multiples within a preset range, using the blur intensity parameters, noise intensity parameters, downsampling multiples and high-resolution image samples as inputs of the image degradation module, and outputting a low-resolution degraded image as a low-resolution image sample. It is easy to understand that in the process of performing degradation processing on the super-resolution image in this embodiment, the high-resolution image samples can be grouped, and a group of blur intensity parameters, noise intensity parameters and downsampling multiples can be selected for each group of high-resolution image samples for degradation processing to enrich the diversity of samples.

[0067] The calculation process included in the image degradation module can be expressed as:

[0068]

[0069] Among them, y is the low-resolution degraded image, x is the high-resolution image, ↓ s It means that the image is downsampled by the sampling multiple s, k is the Gaussian blur kernel, and the Gaussian kernel variance is equal to the blur intensity parameter. Represents a convolution operation, n is a random Gaussian noise, and the noise variance is equal to the noise intensity parameter. By executing the training image generation process multiple times, a large number of training data pairs for training the initial adjustable super-resolution network can be obtained, and each data pair includes: a high-resolution image, a low-resolution degraded image, a blur intensity parameter, a noise intensity parameter, and a downsampling multiple. It can be seen that this embodiment uses the blur intensity parameter to blur the high-resolution image samples to reduce the clarity and detail information of the high-resolution image samples; uses the noise intensity parameter to add noise to the high-resolution image samples to reduce the signal-to-noise ratio of the high-resolution image samples; and uses the downsampling multiple to reduce the dimensionality of the high-resolution image samples to reduce the spatial resolution of the high-resolution image samples.

[0070] Furthermore, before using the training set to train the initial adjustable super-resolution network, the training set can be preprocessed, such as removing useless information, enhancing data diversity, resizing, normalizing, etc., to ensure the consistency of the input data.

[0071] Step M200: Based on the deep neural network, construct an initial adjustable super-resolution network.

[0072] Specifically, a deblurring module is first constructed, and then a super-resolution network is constructed based on a deep neural network (such as a convolutional neural network (CNN), a generative adversarial network (GAN), etc.), and an initial adjustable super-resolution network is constructed using the deblurring module and the super-resolution network. The deblurring module is used to deblur the image to be processed, and the super-resolution network is used to perform super-resolution mapping on the deblurred image to obtain a super-resolution image.

[0073] Step M300: Based on a preset deblurring strength parameter, a preset denoising strength parameter and a preset upsampling multiple, the initial adjustable super-resolution network is iteratively trained using the training set until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

[0074] Specifically, in order to simulate the pseudo conversion process of an image from low resolution to high resolution, the present embodiment presets a deblurring intensity parameter of the same intensity based on the blur intensity parameter used in the image degradation module, presets a denoising intensity parameter of the same intensity based on the noise intensity parameter used in the image degradation module, and presets an upsampling multiple of the same multiple based on the downsampling multiple used in the image degradation module.

[0075] Then, the low-resolution image samples and deblurring strength parameters are input into the deblurring module in the initial adjustable super-resolution network, and the deblurred image corresponding to each low-resolution image is output; based on the denoising strength parameter and the upsampling multiple, the deblurred image is inferred using the super-resolution network in the initial adjustable super-resolution network to obtain a super-resolution image; the initial adjustable super-resolution network is iteratively trained end-to-end using the super-resolution image and the corresponding high-resolution image samples using a back-propagation algorithm to optimize the network parameters until the network converges or the loss value meets the preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

[0076] For example, firstly, the low-resolution degraded image and the blur intensity parameter are input into the deblurring module to obtain the deblurred image. The deblurring module can be realized by solving the inverse problem of equation (1), that is:

[0077]

[0078] Where z is the deblurred image, and are Fourier transform and inverse Fourier transform, respectively. represents the complex conjugate of Fourier transform, y is the low-resolution degraded image, k is the Gaussian blur kernel, the Gaussian kernel variance is equal to the blur strength parameter, and α is a learnable calibration parameter that is optimized during the model training process.

[0079] Then, the deblurred image z, the noise intensity parameter and the downsampling multiple are input into the super-resolution network, whose function is to map the deblurred image into a super-resolution image according to the noise intensity parameter and the downsampling multiple. The specific calculation process can be expressed as:

[0080]

[0081] Among them, the image is the super-resolution image; z, s and μ are the deblurred image, downsampling multiple and denoising intensity parameters respectively; g w (z, s, μ) is the consistency weight subnetwork, with parameter w, input including z, s and μ, and output consistency weight parameter, which is used to adaptively constrain the consistency of images before and after super-resolution to prevent image distortion. θ (z,s,μ) is a super-resolution sub-network with parameters θ, inputs including z, s and μ, and outputs the super-resolution image to be calibrated. w and f θ It can be any trainable neural network model, such as U-Net, residual network, etc. It can be seen that formula (3) is obtained by transforming the low-resolution degraded image y and the super-resolution image to be calibrated f θ (z, s, μ) is weightedly fused to achieve the consistency calibration process and finally obtain a super-resolution image. w The larger (z,s,μ) is, the stronger the consistency constraint is. The final super-resolution image obtained by calibration is closer to the original low-resolution degraded image y, and the super-resolution effect will be weakened. w The smaller (z,s,μ) is, the weaker the consistency constraint is, and the final super-resolution image obtained by calibration is closer to the super-resolution image to be calibrated f. θ (z,s,μ), the super-resolution effect will be enhanced.

[0082] A new image-adjustable super-resolution network framework constructed in this embodiment includes a deblurring module and a super-resolution network. The deblurring module deblurs the low-resolution image according to the deblurring strength, which can effectively reduce the blurring effect of the image, thereby restoring the clarity and detail information of the image; the super-resolution network maps the deblurred image to a super-resolution image according to the denoising strength parameter and the upsampling multiple, which can significantly improve the image signal-to-noise and spatial resolution, making the image details clearer and smoother. At the same time, by randomly generating several groups of blur strength parameters, noise intensity parameters and downsampling multiples to degrade the high-resolution image samples, a variety of low-resolution image samples can be obtained, which is conducive to improving the accuracy and stability of the image-adjustable super-resolution network training.

[0083] In a preferred embodiment, the calculation process of the loss value in step M300 includes:

[0084] The preset image difference measurement loss function is used to calculate the difference between the super-resolution image to be calibrated and the corresponding high-resolution image sample to obtain the loss value.

[0085] For example, the super-resolution image and the original high-resolution image are input into the loss calculation module to obtain the training loss. The loss calculation module can be expressed as:

[0086]

[0087] Among them, l is the loss value, and the loss function It can be any image difference measurement loss function, such as mean square error loss function, root mean square error loss function, mean absolute error loss function, cross entropy cost function or structure similarity index measure (SSIM) loss function.

[0088] like Figure 5 As shown, corresponding to the above-mentioned adjustable image super-resolution method based on a neural network, an embodiment of the present invention further provides an adjustable image super-resolution system based on a neural network, and the above-mentioned adjustable image super-resolution system based on a neural network includes:

[0089] An image acquisition module 510 is used to acquire a number of images to be processed;

[0090] The super-resolution processing module 520 is used to process each of the images to be processed based on preset super-resolution parameters using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

[0091] Specifically, in this embodiment, the specific functions of the above-mentioned neural network-based adjustable image super-resolution system can also refer to the corresponding description in the above-mentioned neural network-based adjustable image super-resolution method, which will not be repeated here.

[0092] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 6As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and an adjustable image super-resolution program based on a neural network. The internal memory provides an environment for the operation of the operating system and the adjustable image super-resolution program based on a neural network in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the adjustable image super-resolution program based on a neural network is executed by the processor, the steps of any one of the above-mentioned adjustable image super-resolution methods based on a neural network are implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.

[0093] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] In one embodiment, a smart terminal is provided, which includes a memory, a processor, and a neural network-based adjustable image super-resolution program stored in the memory and executable on the processor. When the neural network-based adjustable image super-resolution program is executed by the processor, the steps of any one of the neural network-based adjustable image super-resolution methods provided in the embodiments of the present invention are implemented.

[0095] An embodiment of the present invention further provides a computer-readable storage medium, on which is stored a neural network-based adjustable image super-resolution program. When the neural network-based adjustable image super-resolution program is executed by a processor, the steps of any one of the neural network-based adjustable image super-resolution methods provided in the embodiments of the present invention are implemented.

[0096] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0097] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0098] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0100] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0101] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A neural network-based adjustable image super-resolution method, characterized in that: The following steps are involved: Collecting a number of images to be processed; Based on preset super-resolution parameters, each of the images to be processed is processed respectively using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

2. The neural network-based adjustable image super-resolution method according to claim 1, characterized in that: The adjustable super-resolution network includes a deblurring module and a super-resolution network. Based on preset super-resolution parameters, each of the images to be processed is processed using the trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed, including: Based on a preset deblurring strength parameter, using the deblurring module to perform deblurring processing on each of the images to be processed, to obtain a deblurred image corresponding to each of the images to be processed; Based on the preset upsampling multiple and denoising strength parameters, the super-resolution network is used to infer each of the deblurred images respectively to obtain a super-resolution image corresponding to each of the images to be processed.

3. The neural network-based adjustable image super-resolution method according to claim 2, characterized in that: The method of using the super-resolution network to infer each of the deblurred images based on the preset upsampling multiple and denoising strength parameters to obtain a super-resolution image corresponding to each of the images to be processed includes: Determining a consistency weight parameter and a super-resolution reconstructed image based on the deblurred image, the denoising intensity parameter and the upsampling multiple; The consistency weight parameter is used to adjust the weights of the super-resolution reconstructed image and the image to be processed, so as to obtain a super-resolution image corresponding to each image to be processed.

4. The neural network-based adjustable image super-resolution method according to claim 1, characterized in that: The training process of the adjustable super-resolution network includes: Collect several high-resolution image samples and paired low-resolution image samples to construct a training set; Based on deep neural network, construct an initial adjustable super-resolution network; Based on a preset deblurring strength parameter, a preset denoising strength parameter and a preset upsampling multiple, the initial adjustable super-resolution network is iteratively trained using the training set until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

5. The neural network-based adjustable image super-resolution method according to claim 4, characterized in that: The method of iteratively training the initial adjustable super-resolution network based on a preset deblurring strength parameter, a preset denoising strength parameter and a preset upsampling multiple using the training set until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network, comprises: Inputting the low-resolution image samples and the deblurring strength parameters into the initial adjustable super-resolution network, and outputting a deblurred image corresponding to each of the low-resolution images; Based on the denoising intensity parameter and the upsampling multiple, the deblurred image is processed using the initial adjustable super-resolution network to obtain a super-resolution image; The initial adjustable super-resolution network is iteratively trained using the super-resolution image and the high-resolution image samples until the network converges or the loss value meets a preset loss threshold, thereby obtaining a trained adjustable super-resolution network.

6. The neural network-based adjustable image super-resolution method according to claim 5, characterized in that: The calculation process of the loss value includes: The difference between the super-resolution image and the corresponding high-resolution image sample is calculated using a preset image difference measurement loss function to obtain a loss value.

7. A controllable image super-resolution system based on a neural network, characterized in that: The system comprises: An image acquisition module, used for acquiring a number of images to be processed; The super-resolution processing module is used to process each of the images to be processed based on preset super-resolution parameters using a trained adjustable super-resolution network to obtain a super-resolution image corresponding to each of the images to be processed.

8. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a neural network-based adjustable image super-resolution program stored in the memory and executable on the processor. When the neural network-based adjustable image super-resolution program is executed by the processor, the steps of the neural network-based adjustable image super-resolution method as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a neural network-based adjustable image super-resolution program, and when the neural network-based adjustable image super-resolution program is executed by a processor, the steps of the neural network-based adjustable image super-resolution method as described in any one of claims 1 to 6 are implemented.

Images