A super-resolution image reconstruction method and device with adjustable sharpness

Abstract

The present invention discloses a super-resolution image reconstruction method and device with adjustable sharpening degree, which relates to the field of deep learning technology, and its main purpose is to adjust the results of the model output according to actual needs in the application link. The main technical scheme of the present invention is: obtaining the required sharpening coefficient and the low-resolution remote sensing image to be input; using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to process the required sharpening coefficient, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model, and obtain the target super-resolution image reconstruction deep learning model; using the target super-resolution image reconstruction deep learning model to perform super-resolution reconstruction processing on the low-resolution remote sensing image, and output the corresponding high-resolution remote sensing image. The present invention is used for super-resolution image reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a super-resolution image reconstruction method and device with adjustable sharpness. Background Art

[0002] Remote sensing high-resolution remote sensing images are widely used because of the rich texture, spatial and semantic information they contain. However, during the imaging process, remote sensing high-resolution remote sensing images are often interfered by various factors, such as airflow disturbance, imaging diffraction, transmission interference, relative attitude change, downsampling, additional noise, etc., resulting in a decrease in image quality.

[0003] To solve this problem, the existing technical means is to use super-resolution image reconstruction methods. However, the existing super-resolution image reconstruction methods cannot change the model configuration during the application stage, so the model output results cannot be adjusted according to actual needs, resulting in poor flexibility of the method. Summary of the invention

[0004] In view of the above problems, the present invention provides a super-resolution image reconstruction method and device with adjustable sharpening degree, the main purpose of which is to adjust the results of the model output according to actual needs in the application link.

[0005] In order to solve the above technical problems, the present invention proposes the following solutions:

[0006] In a first aspect, the present invention provides a super-resolution image reconstruction method with adjustable sharpening degree, the method comprising:

[0007] Obtain the required sharpening factor and the low-resolution remote sensing image to be input;

[0008] The required sharpening coefficient is processed using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model;

[0009] The low-resolution remote sensing image is reconstructed with super-resolution using a target super-resolution image reconstruction deep learning model to output a corresponding high-resolution remote sensing image.

[0010] In a second aspect, the present invention provides a super-resolution image reconstruction device with adjustable sharpness, the device comprising:

[0011] An acquisition unit, used for acquiring a required sharpening coefficient and a low-resolution remote sensing image to be input;

[0012] a parameter processing unit, configured to process the required sharpening coefficient acquired by the acquisition unit using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model, and obtain a target super-resolution image reconstruction deep learning model;

[0013] The image processing unit is used to perform super-resolution reconstruction processing on the low-resolution remote sensing image acquired by the acquisition unit using the target super-resolution image reconstruction deep learning model obtained by the parameter processing unit, and output a corresponding high-resolution remote sensing image.

[0014] In order to achieve the above-mentioned purpose, according to a third aspect of the present invention, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the super-resolution image reconstruction method with adjustable sharpening degree according to the above-mentioned first aspect.

[0015] In order to achieve the above object, according to a fourth aspect of the present invention, a processor is provided, wherein the processor is used to run a program, wherein when the program is run, the super-resolution image reconstruction method with adjustable sharpening degree according to the first aspect is executed.

[0016] Through the above technical scheme, the present invention provides a super-resolution image reconstruction method and device with adjustable sharpening degree, introduces a required sharpening coefficient, and can process the required sharpening coefficient through the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model, thereby obtaining the target super-resolution image reconstruction deep learning model, and finally using the target super-resolution image reconstruction deep learning model to process the low-resolution remote sensing image, and can output a high-resolution remote sensing image corresponding to the required sharpening coefficient. That is, if the required sharpening coefficient is changed in the application stage of the present invention, the output high-resolution remote sensing image effect is also adjusted accordingly, thereby achieving the result of adjusting the model output according to actual needs, with strong flexibility.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0019] Figure 1 A flow chart of a super-resolution image reconstruction method with adjustable sharpness provided by an embodiment of the present invention is shown;

[0020] Figure 2 A flow chart of another super-resolution image reconstruction method with adjustable sharpness provided by an embodiment of the present invention is shown;

[0021] Figure 3 A block diagram showing a composition of a super-resolution image reconstruction device with adjustable sharpness provided by an embodiment of the present invention;

[0022] Figure 4 A block diagram showing the composition of another super-resolution image reconstruction device with adjustable sharpness provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0024] In view of the problem that the current super-resolution image reconstruction method cannot change the model configuration during the application stage, and therefore cannot adjust the model output results according to actual needs during the application stage, resulting in poor flexibility of the method, the inventors have discovered through research that when constructing the model, the model can be constructed as a model that can be dynamically changed according to the sharpening coefficient. In this way, in the actual application stage, the trained super-resolution image reconstruction deep learning model can be adjusted according to the actual required sharpening coefficient to obtain the target super-resolution image reconstruction deep learning model. Finally, the target super-resolution image reconstruction deep learning model is used to perform super-resolution reconstruction processing on the low-resolution remote sensing image, and the corresponding high-resolution remote sensing image is output, which solves the problem of poor flexibility in the prior art. The specific execution steps are as follows: Figure 1 As shown, including:

[0025] 101. Obtain the required sharpening factor and the low-resolution remote sensing image to be input.

[0026] 102. The required sharpening coefficient is processed using the blur parameter input part of the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part of the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model.

[0027] In the present invention, a blind image super-resolution reconstruction model part and a fuzzy parameter input part can be constructed separately, and then a super-resolution image reconstruction deep learning model can be constructed according to the blind image super-resolution reconstruction model part and the fuzzy parameter input part. Then, the constructed super-resolution image reconstruction deep learning model can be trained, evaluated and tested to obtain a trained super-resolution image reconstruction deep learning model.

[0028] In this way, in actual applications, if there is a need to sharpen the output effect of the low-resolution remote sensing image for the trained super-resolution image reconstruction deep learning model, the required sharpening coefficient and the low-resolution remote sensing image to be input can be obtained, and then the trained super-resolution image reconstruction deep learning model can be adjusted according to the required sharpening coefficient, so that the adjusted model can output the low-resolution remote sensing image and the high-resolution remote sensing image that meets the required sharpening coefficient, thereby meeting the demand for image sharpening effect.

[0029] Specifically, the required sharpening coefficient can be processed using the fuzzy parameter input part of the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part to obtain the target super-resolution image reconstruction deep learning model, and the target super-resolution image reconstruction deep learning model can be used to perform super-resolution reconstruction on the low-resolution remote sensing image, and output a high-resolution remote sensing image that meets the required sharpening coefficient.

[0030] Among them, considering that different applications have different sharpening effects on images, the required sharpening coefficient can depend on the application source corresponding to the low-resolution remote sensing image to be input; in addition, a custom demand configuration function can be set on the client display page, and the user can specify the required sharpening coefficient.

[0031] 103. Use the target super-resolution image reconstruction deep learning model to perform super-resolution reconstruction on the low-resolution remote sensing image and output the corresponding high-resolution remote sensing image.

[0032] In this step, the low-resolution remote sensing image can be processed in blocks according to the hardware conditions of the image processor to obtain multiple corresponding low-resolution remote sensing image blocks. Then, the multiple low-resolution remote sensing image blocks can be input into the target super-resolution image reconstruction deep learning model to output the high-resolution remote sensing image blocks corresponding to each low-resolution remote sensing image block that meets the required sharpening coefficient. Then, the multiple high-resolution remote sensing image blocks can be spliced ​​to obtain a high-resolution remote sensing image corresponding to the low-resolution remote sensing image.

[0033] Specifically, when multiple low-resolution remote sensing image blocks are input into the target super-resolution image reconstruction deep learning model, and the high-resolution remote sensing image blocks corresponding to each low-resolution remote sensing image block that meets the required sharpening coefficient are output, the multiple low-resolution remote sensing image blocks can be processed according to the changed blind image super-resolution reconstruction model part.

[0034] Specifically, the low-resolution remote sensing image block can be first passed through an n×n×c convolution layer for preliminary feature extraction to obtain the extracted features, and then the extracted features are passed through g stacked Residual in Residual DenseBlocks for deep feature extraction and processing to obtain a feature map, and then the feature map is passed through k consecutive upsampling layers to expand the feature map size to r times the original size. The upsampling layer uses an interpolation algorithm, and then is decoded through z convolution layers to output high-resolution remote sensing image blocks corresponding to each low-resolution remote sensing image block that meets the required sharpening coefficient.

[0035] Based on the above Figure 1 It can be seen from the implementation method that a super-resolution image reconstruction method with adjustable sharpening degree provided by the present invention introduces a required sharpening coefficient. The required sharpening coefficient can be processed by the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model, thereby obtaining the target super-resolution image reconstruction deep learning model. Finally, the low-resolution remote sensing image is processed by the target super-resolution image reconstruction deep learning model, and a high-resolution remote sensing image corresponding to the required sharpening coefficient can be output. That is, if the required sharpening coefficient is changed in the application stage of the present invention, the output high-resolution remote sensing image effect is also adjusted accordingly, thereby achieving the result of adjusting the model output according to actual needs, and having strong flexibility.

[0036] Furthermore, as a Figure 1 The embodiment shown in the figure is refined and extended. The embodiment of the present invention also provides another super-resolution image reconstruction method with adjustable sharpness, such as Figure 2 As shown, the specific steps are as follows:

[0037] 201. Construct a super-resolution image reconstruction deep learning model, train the super-resolution image reconstruction deep learning model, and obtain a trained super-resolution image reconstruction deep learning model.

[0038] In this step, the blind image super-resolution reconstruction model part can be constructed first, and then the fuzzy parameter input part can be constructed. After that, the blind image super-resolution reconstruction model part and the fuzzy parameter input part can be combined into a super-resolution image reconstruction deep learning model.

[0039] The fuzzy parameter input part includes an independent multi-layer perceptron, which can process the fuzzy parameters in the form of one-hot codes to generate convolution kernels of all convolution layers in the blind image super-resolution reconstruction model. The blind image super-resolution reconstruction model includes an n×n×c convolution layer, which is responsible for preliminary feature extraction, where n is the convolution kernel size, which is set to an odd number, and c is the number of features, which is determined according to the hardware conditions of the graphics processing unit (GPU); it also includes g stacked Residual in Residual Dense Blocks, which are responsible for deep feature extraction and processing; it also includes k continuous upsampling layers, which are responsible for expanding the feature map size to r times the original size. The upsampling layer uses an interpolation algorithm, and then decodes through z convolution layers to output high-resolution remote sensing images.

[0040] Afterwards, the composed super-resolution image reconstruction deep learning model can be trained: the dynamic super-pixel reconstruction dataset can be obtained first, and then the super-resolution image reconstruction deep learning model can be trained according to the dynamic super-pixel reconstruction dataset. Specifically:

[0041] The dynamic superpixel reconstruction dataset can be: first obtain sample images in various formats, including natural images and high-resolution remote sensing images, etc. Among them, natural images can be selected as high-definition as possible, in order to enrich the sample volume when high-resolution remote sensing image data is scarce.

[0042] The purpose of selecting sample images in various formats is to increase diversity so as to enrich the training data and benefit the training effect of the model.

[0043] Then, the sample images in various formats are normalized to between 0 and 1, and then a Px×Py area is randomly cut out from each sample image, and the cut area is used as the target sample image corresponding to each sample image. The target sample image is then subjected to data enhancement processing such as random flipping and rotation to obtain the original high-resolution remote sensing image corresponding to each target sample image.

[0044] After that, the image degradation method can be used to degrade multiple original high-resolution remote sensing images to obtain corresponding low-resolution remote sensing images. Specifically:

[0045] Image degradation methods include Gaussian blur, random Gaussian noise, random multiple downsampling followed by upsampling for blurring, and random JPEG compression noise.

[0046] Therefore, a target image degradation method can be obtained according to the above-mentioned image degradation methods, and the original high-resolution remote sensing image can be degraded using the target image degradation method, wherein the target image degradation method can be a random combination of the above-mentioned image degradation methods; or an image degradation method randomly selected from the above-mentioned image degradation methods.

[0047] It should be noted that if there are multiple target image degradation methods, the original high-resolution remote sensing image can be degraded using multiple target image degradation methods respectively, but ultimately only the output of one target image degradation method is randomly selected for subsequent use.

[0048] Among them, the reason for degrading the original high-resolution remote sensing image is: in order to train the model accurately enough, it is best to use low-resolution remote sensing images that are sufficiently matched with the high-resolution remote sensing images to train the model, so that the model can learn the mapping relationship between low-resolution remote sensing images and high-resolution remote sensing images that is sufficiently accurate. However, in fact, low-resolution remote sensing images that are sufficiently matched with high-resolution remote sensing images are difficult to obtain. Therefore, the degradation algorithm can be executed on the original high-resolution remote sensing image to obtain a matching low-resolution remote sensing image.

[0049] Then, the Gaussian blur method can be used to obtain the blur parameters in the form of one-hot code: Since the sigma value in the Gaussian blur is the largest for the image degradation method, the generation of blur parameters is mainly based on the sigma value in the Gaussian blur. The blur degree can be random during the training stage.

[0050] 202. Obtain the required sharpening factor and the low-resolution remote sensing image to be input.

[0051] Among them, the implementation method of step 202 is the same as that of step 101, and can achieve the same technical effect and solve the same technical problem, and will not be repeated here.

[0052] 203. The required sharpening coefficient is processed using the blur parameter input part of the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part of the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model.

[0053] Among them, step 203 proposes a more detailed implementation method compared to step 102.

[0054] The required sharpening coefficient can be used to characterize the target sharpening degree of the final output high-resolution remote sensing image. In the model application stage, the required fuzzy parameters can be obtained according to the required sharpening coefficient, and then the required fuzzy parameters are processed through an independent multi-layer perceptron containing multiple hidden layers of neurons using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the convolution kernel of the convolution layer in the blind image super-resolution reconstruction model part of the super-resolution image reconstruction deep learning model, and obtain the target super-resolution image reconstruction deep learning model.

[0055] 204. Use the target super-resolution image reconstruction deep learning model to perform super-resolution reconstruction on the low-resolution remote sensing image and output the corresponding high-resolution remote sensing image.

[0056] In this step, the low-resolution remote sensing image can be processed in blocks according to the hardware conditions of the image processor to obtain a corresponding plurality of low-resolution remote sensing image blocks. The plurality of low-resolution remote sensing image blocks can then be input into the target super-resolution image reconstruction deep learning model to output high-resolution remote sensing image blocks corresponding to each low-resolution remote sensing image block that meets the required sharpening coefficient. The plurality of high-resolution remote sensing image blocks can then be spliced ​​to obtain a high-resolution remote sensing image corresponding to the low-resolution remote sensing image.

[0057] Among them, when the low-resolution remote sensing image is divided into blocks to obtain corresponding multiple low-resolution remote sensing image blocks, the overlap between the image blocks can be set to 2B to avoid the super-resolution effect being not good enough at the boundary of the image block. Therefore, the size of the low-resolution remote sensing image block can be (px+2B)×(py+2B).

[0058] Therefore, when the corresponding high-resolution remote sensing image block is obtained, the size of the high-resolution remote sensing image block is also (px+2B)×(py+2B). In this way, when multiple high-resolution remote sensing image blocks are stitched together to obtain a high-resolution remote sensing image corresponding to the low-resolution remote sensing image, the overlapping parts can be removed, that is, the size of the stitched high-resolution remote sensing image block is Px and Py.

[0059] In addition, before stitching the high-resolution remote sensing image blocks to obtain the high-resolution remote sensing image corresponding to the low-resolution remote sensing image, the value of each pixel point can be restored to the corresponding pixel point in the original input low-resolution remote sensing image block according to the maximum and minimum pixel values ​​in the high-resolution remote sensing image block, so as to prevent the pixel values ​​of the result image from being too different from those in the input low-resolution remote sensing image, and also to prevent errors from being caused in the subsequent quantitative remote sensing of the image. In addition, since the difference values ​​of each image block are different, for example, some are brighter than the original, and some are darker than the original, the stitching process will also cause block effects. Therefore, it is necessary to perform pixel value restoration processing on each high-resolution remote sensing image block obtained.

[0060] Furthermore, as a response to the above Figure 1 In order to realize the method shown in the figure, the embodiment of the present invention also provides a super-resolution image reconstruction device with adjustable sharpness, which is used for Figure 1 The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. Figure 3 As shown, the device comprises:

[0061] An acquisition unit 301 is used to acquire a required sharpening coefficient and a low-resolution remote sensing image to be input;

[0062] A parameter processing unit 302 is used to process the required sharpening coefficient obtained by the obtaining unit 301 using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain a target super-resolution image reconstruction deep learning model;

[0063] The image processing unit 303 is used to perform super-resolution reconstruction processing on the low-resolution remote sensing image acquired by the acquisition unit 301 using the target super-resolution image reconstruction deep learning model obtained by the parameter processing unit 302, and output a corresponding high-resolution remote sensing image.

[0064] Furthermore, as a response to the above Figure 2 In order to realize the method shown in the figure, the embodiment of the present invention also provides another super-resolution image reconstruction device with adjustable sharpness, which is used for Figure 2 The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. Figure 4 As shown, the device comprises:

[0065] An acquisition unit 301 is used to acquire a required sharpening coefficient and a low-resolution remote sensing image to be input;

[0066] A parameter processing unit 302 is used to process the required sharpening coefficient obtained by the obtaining unit 301 using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain a target super-resolution image reconstruction deep learning model;

[0067] The image processing unit 303 is used to perform super-resolution reconstruction processing on the low-resolution remote sensing image acquired by the acquisition unit 301 using the target super-resolution image reconstruction deep learning model obtained by the parameter processing unit 302, and output a corresponding high-resolution remote sensing image.

[0068] In an optional implementation manner, the parameter processing unit 302 includes:

[0069] A parameter generation module 3021, used to generate a demand fuzzy parameter based on the demand sharpening coefficient;

[0070] The model adjustment module 3022 is used to process the required fuzzy parameters generated by the parameter generation module 3021 using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model.

[0071] In an optional implementation manner, the model adjustment module 3022 is specifically used for:

[0072] The required fuzzy parameters are processed by an independent multilayer perceptron containing multiple neuron hidden layers using the fuzzy parameter input part to change the convolution kernel of the convolution layer in the blind image super-resolution reconstruction model part of the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model.

[0073] In an optional implementation, before the acquisition unit 301 acquires the required sharpening coefficient and the low-resolution remote sensing image to be input, the device further includes a model building unit 304, and the model building unit 304 includes:

[0074] The first model building module 3041 is used to build a blind image super-resolution reconstruction model part;

[0075] The second model building module 3042 is used to build a fuzzy parameter input part;

[0076] The model forming module 3043 is used to form a super-resolution image reconstruction deep learning model based on the blind image super-resolution reconstruction model part constructed by the first model building module 3041 and the fuzzy parameter input part constructed by the second model building module 3042.

[0077] In an optional implementation manner, after the model forming module 3043 forms the super-resolution image reconstruction deep learning model based on the blind image super-resolution reconstruction model part and the fuzzy parameter input part, the device further includes a model training unit 305, and the model training unit 305 includes:

[0078] The data set acquisition module 3051 is used to acquire a dynamic superpixel reconstruction data set;

[0079] The model training module 3052 is used to train the super-resolution image reconstruction deep learning model based on the dynamic super-pixel reconstruction data set obtained by the data set acquisition module 3051 to obtain a trained super-resolution image reconstruction deep learning model.

[0080] In an optional implementation manner, the data set acquisition module 3051 is specifically used for:

[0081] Obtaining a plurality of original high-resolution remote sensing images, wherein the plurality of original high-resolution remote sensing images have the same size and are normalized;

[0082] Using image degradation methods to degrade multiple original high-resolution remote sensing images, we can obtain corresponding low-resolution remote sensing images.

[0083] The fuzzy parameters in the form of one-hot encoding are obtained using Gaussian fuzzy method.

[0084] In an optional implementation manner, when the data set acquisition module 3051 uses an image degradation method to perform degradation processing on a plurality of original high-resolution remote sensing images to obtain corresponding low-resolution remote sensing images, it is specifically used to:

[0085] Determine multiple image degradation methods;

[0086] Processing the multiple image degradation methods to obtain a target image degradation method, wherein the target image degradation method is a combination of several image degradation methods or a random image degradation method, wherein the image degradation method includes Gaussian blur, random Gaussian noise, random multiple downsampling followed by upsampling for blurring, etc.;

[0087] The target image degradation method is used to perform degradation processing on a plurality of original high-resolution remote sensing images to obtain corresponding low-resolution remote sensing images.

[0088] Furthermore, an embodiment of the present invention further provides a storage medium, wherein the storage medium is used to store a computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the above Figure 1-2 A super-resolution image reconstruction method with adjustable sharpening degree as described in .

[0089] Furthermore, an embodiment of the present invention further provides a processor, the processor is used to run a program, wherein the program executes the above Figure 1-2 A super-resolution image reconstruction method with adjustable sharpening degree as described in .

[0090] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] It is understandable that the related features in the above methods and devices can be referenced to each other. In addition, the "first", "second" and the like in the above embodiments are used to distinguish the embodiments, but do not represent the advantages and disadvantages of the embodiments.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0093] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0094] In addition, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0100] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0101] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0104] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A super-resolution image reconstruction method with adjustable sharpness, characterized in that: The method comprises: Obtain the required sharpening factor and the low-resolution remote sensing image to be input; The required sharpening coefficient is processed by using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model, including: generating a demand fuzzy parameter based on the demand sharpening coefficient; The required fuzzy parameter is processed by using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain the target super-resolution image reconstruction deep learning model, including: The fuzzy parameter input part is used to process the required fuzzy parameters through an independent multilayer perceptron containing multiple neuron hidden layers to change the convolution kernel of the convolution layer in the blind image super-resolution reconstruction model part of the trained super-resolution image reconstruction deep learning model to obtain a target super-resolution image reconstruction deep learning model; the low-resolution remote sensing image is super-resolution reconstructed using the target super-resolution image reconstruction deep learning model to output a corresponding high-resolution remote sensing image, wherein the trained super-resolution image reconstruction deep learning model is obtained by training a dynamic super-pixel reconstruction data set, the dynamic super-pixel reconstruction data set includes fuzzy parameters in the form of a one-hot code obtained by using a Gaussian blur method and low-resolution remote sensing images obtained by degrading multiple original high-resolution remote sensing images by a target image degradation method, the target image degradation method can be a random combination of multiple image degradation methods, and the image degradation methods include Gaussian blur, random Gaussian noise, random multiple downsampling and then upsampling for blurring, and random jpeg compression noise.

2. The method according to claim 1, characterized in that Before obtaining the required sharpening coefficient and the low-resolution remote sensing image to be input, the method includes: Construct the blind image super-resolution reconstruction model part; Construct the fuzzy parameter input part; A super-resolution image reconstruction deep learning model is formed based on the blind image super-resolution reconstruction model part and the fuzzy parameter input part.

3. The method according to claim 2, characterized in that After forming the super-resolution image reconstruction deep learning model based on the blind image super-resolution reconstruction model part and the fuzzy parameter input part, the method further includes: Get dynamic superpixel reconstruction dataset; The super-resolution image reconstruction deep learning model is trained based on the dynamic super-pixel reconstruction dataset to obtain a trained super-resolution image reconstruction deep learning model.

4. The method according to claim 3, characterized in that Get dynamic superpixel reconstruction datasets, including: Obtaining a plurality of original high-resolution remote sensing images, wherein the plurality of original high-resolution remote sensing images have the same size and are normalized; Using image degradation methods to degrade multiple original high-resolution remote sensing images, we can obtain corresponding low-resolution remote sensing images. The fuzzy parameters in the form of one-hot encoding are obtained using Gaussian fuzzy method.

5. The method according to claim 4, characterized in that The image degradation method is used to degrade multiple original high-resolution remote sensing images to obtain corresponding low-resolution remote sensing images, including: Determine multiple image degradation methods; Processing the multiple image degradation methods to obtain a target image degradation method, wherein the target image degradation method is a combination of several image degradation methods or a random image degradation method, wherein the image degradation method includes Gaussian blur, random Gaussian noise, random multiple downsampling followed by upsampling for blurring, etc.; The target image degradation method is used to perform degradation processing on a plurality of original high-resolution remote sensing images to obtain corresponding low-resolution remote sensing images.

6. A super-resolution image reconstruction device with adjustable sharpness, characterized in that: The device comprises: An acquisition unit, used for acquiring a required sharpening coefficient and a low-resolution remote sensing image to be input; a parameter processing unit, configured to process the required sharpening coefficient acquired by the acquisition unit using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model, and obtain a target super-resolution image reconstruction deep learning model; The parameter processing unit comprises: A parameter generation module, used for generating a demand fuzzy parameter based on the demand sharpening coefficient; A model adjustment module is used to process the required fuzzy parameters generated by the parameter generation module using the fuzzy parameter input part in the trained super-resolution image reconstruction deep learning model, so as to change the blind image super-resolution reconstruction model part in the trained super-resolution image reconstruction deep learning model to obtain a target super-resolution image reconstruction deep learning model; The model adjustment module is specifically used for: The required fuzzy parameters are processed by an independent multilayer perceptron containing multiple hidden layers of neurons using the fuzzy parameter input part to change the convolution kernel of the convolution layer in the blind image super-resolution reconstruction model part of the trained super-resolution image reconstruction deep learning model, thereby obtaining a target super-resolution image reconstruction deep learning model; The image processing unit is used to perform super-resolution reconstruction processing on the low-resolution remote sensing image acquired by the acquisition unit using the target super-resolution image reconstruction deep learning model obtained by the parameter processing unit, and output the corresponding high-resolution remote sensing image, wherein the trained super-resolution image reconstruction deep learning model is obtained by training a dynamic super-pixel reconstruction data set, and the dynamic super-pixel reconstruction data set includes a low-resolution remote sensing image obtained by degrading multiple original high-resolution remote sensing images using a Gaussian blur method and a target image degradation method. The target image degradation method can be a random combination of multiple image degradation methods, and the image degradation methods include Gaussian blur, random Gaussian noise, random multiple downsampling and then upsampling for blurring, and random jpeg compression noise.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the super-resolution image reconstruction method with adjustable sharpening degree as described in any one of claims 1 to 5.

Images