Method, apparatus, device and computer storage medium for acquiring high resolution images

Abstract

The embodiment of the present disclosure provides a method, device, equipment and computer storage medium for obtaining a high-resolution image. The method for obtaining a high-resolution image comprises: reconstructing input Gaussian white noise through an image depth prior network to obtain a transient output result; under the constraint of a loss function, performing convolution on the transient output result and input known blur kernel to obtain first low-frequency channel brightness temperature information, which is matched with the input degraded image; through a gradient loss function, extracting gradient information of a high-frequency channel, and making the gradient information of the first low-frequency channel brightness temperature information and the gradient information of the high-frequency channel matched, thereby obtaining an enhanced high-resolution image. Through the processing scheme of the present disclosure, under the zero sample method, the training set is not restricted by the scarcity and type mismatch, the extremely high detail satellite radiometer undegraded low-frequency channel brightness temperature data is recovered, and the enhanced high-resolution image is obtained.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of microwave remote sensing, and particularly relates to a method, device and equipment for obtaining high-resolution images and a computer storage medium. BACKGROUND

[0002] As a remote sensing observation device, the terahertz radiometer plays an important role in meteorological monitoring, resource exploration, environmental monitoring and other fields. However, due to the physical limitations of satellite radiometers and the limitations of imaging technology, the brightness temperature data obtained is often limited by spatial resolution, which leads to the inability to effectively interpret accurate remote sensing data. The current method for improving the spatial resolution of the radiometer mainly relies on the learning process of sample supervision, and requires a large amount of high-quality and labeled training data. However, such data acquisition is costly and difficult to cover all use cases, especially for new devices or data collection in special environments, which has the problems of training sample scarcity and type mismatch. In addition, the previous method often needs to use complex algorithms and expensive computing resources when processing cross-channel resolution matching, which has limitations in real-world applications. Therefore, there is an urgent need for an effective method to achieve high-fidelity restoration of terahertz radiometer data in a zero-sample environment, improve the spatial resolution matching degree between different channels, enhance the usability of data, and reduce the complexity of data processing and the consumption of computing resources. SUMMARY

[0003] In view of this, the embodiments of the present disclosure provide a method, device, equipment and computer storage medium for obtaining high-resolution images, which at least partially solve the problems in the prior art.

[0004] The first aspect of the present disclosure provides a method for obtaining high-resolution images, comprising:

[0005] S1, reconstructing the input Gaussian white noise through an image depth prior network to obtain a transient output result;

[0006] S2, under the constraint of a loss function, convolving the transient output result and the input known blur kernel to obtain first low-frequency channel brightness temperature information, the first low-frequency channel brightness temperature information being matched with the input degraded image;

[0007] S3, extracting gradient information of a high-frequency channel through a gradient loss function, and matching the gradient information of the first low-frequency channel brightness temperature information with the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image.

[0008] According to a specific implementation of the first aspect of the present disclosure, S2 comprises generating the first low-frequency channel brightness temperature information using the following formula:

[0009]

[0010]

[0011] where z k and z x are one-dimensional and two-dimensional Gaussian white noise respectively, T k (z k ) is the two-dimensional PSF after reshaping of the simple neural network, T x (z x ) is the bright temperature image with higher spatial resolution after the DIP network, t A (x,y,ν) is the transient output result.

[0012] According to a specific implementation manner of the first aspect of the present disclosure, before S2, further comprising:

[0013] S4, pre-processing the transient output result.

[0014] According to a specific implementation manner of the first aspect of the present disclosure, the pre-processing of the transient output result comprises denoising and contrast enhancement.

[0015] According to a specific implementation manner of the first aspect of the present disclosure, the gradient loss function uses the 36.5GHz channel information as gradient constraint information.

[0016] According to a specific implementation manner of the first aspect of the present disclosure, the expression of the gradient loss function is as follows:

[0017]

[0018] where represents the gradient of the corresponding bright temperature information using the Sobel operator, and the calculation formula is as follows:

[0019]

[0020]

[0021] represents the gradient information obtained from the network output by the Sobel gradient operator, and represents the gradient information extracted from the 36.5GHz channel information by the same operator.

[0022] The second aspect embodiment of the present disclosure provides a device for obtaining a high-resolution image, comprising a first processing module, a second processing module and a third processing module, the first processing module is configured to reconstruct input Gaussian white noise through an image depth prior network to obtain a transient output result; the second processing module is configured to convolve the transient output result and input known blur kernels under the constraint of a loss function to obtain first low-frequency channel brightness temperature information, the first low-frequency channel brightness temperature information is matched with the input degraded image; and the third processing module is configured to extract gradient information of a high-frequency channel through a gradient loss function, and make the gradient information of the first low-frequency channel brightness temperature information and the gradient information of the high-frequency channel matched, thereby obtaining an enhanced high-resolution image.

[0023] The third aspect embodiment of the present disclosure provides a device for obtaining a high-resolution image, comprising a processor and a memory storing computer program instructions; the processor implements the method for obtaining a high-resolution image according to any one of the preceding first aspect embodiments when executing the computer program instructions.

[0024] The fourth aspect embodiment of the present disclosure provides a computer storage medium, the computer storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the method for obtaining a high-resolution image according to any one of the preceding first aspect embodiments.

[0025] The method for obtaining a high-resolution image provided by the embodiments of the present disclosure comprises reconstructing input Gaussian white noise through an image depth prior network to obtain a transient output result; convolving the transient output result and input known blur kernels under the constraint of a loss function to obtain first low-frequency channel brightness temperature information, the first low-frequency channel brightness temperature information is matched with the input degraded image; extracting gradient information of a high-frequency channel through a gradient loss function, and making the gradient information of the first low-frequency channel brightness temperature information and the gradient information of the high-frequency channel matched, thereby obtaining an enhanced high-resolution image. Through the method provided by the present disclosure, under the zero-sample unsupervised method, the recovery of extremely high-detail satellite radiometer un-degraded low-frequency channel brightness temperature data is not subject to the scarcity and type mismatch of the training set, and an enhanced high-resolution image is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1A flowchart of a method for obtaining a high-resolution image is provided for a first embodiment of the present disclosure.

[0028] Figure 2 A structural diagram of a device for obtaining a high-resolution image is provided for a third embodiment. DETAILED DESCRIPTION

[0029] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0030] The above and other aspects of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0031] It should be apparent that the foregoing description is merely illustrative of the embodiments of the present disclosure and that the true scope of the present disclosure should be determined by reference to the appended claims.

[0032] It should be apparent that the foregoing description is merely illustrative of the embodiments of the present disclosure and that the true scope of the present disclosure should be determined by reference to the appended claims.

[0033] In addition, in the following description, specific details are provided to thoroughly understand examples. However, it will be understood by one of ordinary skill in the art that the described aspects can be practiced without these specific details.

[0034] As a remote sensing observation device, terahertz radiometer plays an important role in meteorological monitoring, resource exploration, environmental monitoring and other fields. However, due to the physical limitations of satellite radiometer and the limitations of imaging technology, the brightness temperature data obtained is often limited by spatial resolution, which leads to the inability to effectively interpret the precise remote sensing data. The current method to improve the spatial resolution of radiometer mainly relies on the learning process of sample supervision, and requires a large amount of high-quality and labeled training data. However, such data acquisition is costly and difficult to cover all use cases, especially for new devices or data collection in special environments, there are problems of training sample scarcity and type mismatch. In addition, the previous method often needs to use complex algorithms and expensive computing resources when dealing with cross-channel resolution matching, which has limitations in real-world applications. Therefore, an effective method is needed to realize high-fidelity restoration of terahertz radiometer data in an unsupervised environment, improve the spatial resolution matching degree between different channels, enhance the usability of data, and reduce the complexity of data processing and the consumption of computing resources.

[0035] To solve the above problems, the embodiments of the present disclosure provide a method, device, equipment and computer storage medium for obtaining a high-resolution image, which will be described below in conjunction with the accompanying drawings.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a method for obtaining a high-resolution image is provided for the first aspect of the present disclosure. The method for obtaining a high-resolution image includes the following steps:

[0037] S1, reconstructing the input Gaussian white noise by an image depth prior network to obtain a transient output result;

[0038] S2, under the constraint of a loss function, convolving the transient output result and the input known blur kernel to obtain first low-frequency channel brightness temperature information, which is matched with the input degraded image. Specifically, the loss function can include content loss (such as mean square error MSE) and structure loss.

[0039] S3, extracting gradient information of a high-frequency channel by a gradient loss function, and making the gradient information of the first low-frequency channel brightness temperature information and the gradient information of the high-frequency channel matched, thereby obtaining an enhanced high-resolution image. The introduction of the gradient loss function can guide the model to learn the high-frequency information of the low-frequency channel. This function focuses on the edges and texture details of the radiometer image, which helps to restore the details of the high-resolution image from the low-resolution image.

[0040] Through the method provided by the present disclosure, in the environment of zero samples, the high-detail satellite radiometer non-degraded low-frequency channel brightness temperature data can be recovered, and the enhanced high-resolution image can be obtained, without being subject to the scarcity and type mismatch of the training set.

[0041] In some optional embodiments, S2 comprises generating the first low-frequency channel brightness temperature information by using the following formula:

[0042]

[0043]

[0044] wherein z k and z x are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise respectively, T k (z k ) is the two-dimensional PSF after remodeling of the simple neural network, T x (z x ) is the high-spatial-resolution brightness temperature image after the DIP network, t A (x, y, v) is the transient output result.

[0045] In this embodiment, by using the powerful fitting function of the neural network, the first low-frequency channel brightness temperature information that matches the degraded image is generated by using the neural blind deconvolution model of the DIP-Deep Image Prior network, and some inherent problems of the initial DIP asymmetric codec, such as the insufficient ability to capture the prior information of the blur kernel, are improved and solved.

[0046] In some optional embodiments, before S2, the method further comprises:

[0047] S4, preprocessing the transient output result. Specifically, the preprocessing can include denoising and enhancing contrast, so as to eliminate some interference, improve the quality of the obtained image, and reduce the training time and improve the processing efficiency.

[0048] In some optional embodiments, the gradient loss function uses the 36.5GHz channel information as the gradient constraint information. The 36.5GHz channel information is used as the gradient constraint information not only because the 36.5GHz channel contains more high-frequency information, but also because the extracted information and the gradient information of the 18GHz low-frequency channel have higher similarity, so as to avoid introducing too much error and improve the quality of the obtained high-resolution signal.

[0049] Specifically, the 36.5GHz channel information is used as the gradient constraint information, and the expression of the gradient loss function is as follows:

[0050]

[0051] where the gradient of the corresponding brightness temperature information using the Sobel operator is represented, and the calculation formula is as follows:

[0052]

[0053]

[0054] The gradient information obtained from the network output by the Sobel gradient operator is represented, and the gradient information extracted from the 36.5GHz channel information by the same operator is represented.

[0055] The second aspect embodiment of the present disclosure provides a device for obtaining a high-resolution image, comprising a first processing module, a second processing module and a third processing module, the first processing module is configured to reconstruct the input Gaussian white noise through an image depth prior network to obtain a transient output result; the second processing module is configured to, under the constraint of a loss function, convolve the transient output result and the input known blur kernel to obtain first low-frequency channel brightness temperature information, the first low-frequency channel brightness temperature information is matched with the input degraded image; the third processing module is configured to extract gradient information of a high-frequency channel through a gradient loss function, and make the gradient information of the first low-frequency channel brightness temperature information and the gradient information of the high-frequency channel remain matched, thereby obtaining an enhanced high-resolution image.

[0056] It should be noted that the information interaction, execution process and the like between the above devices / units are based on the same concept as the method embodiments of the present application, and are corresponding devices for the above method of obtaining a high-resolution image. All implementation manners in the above method embodiments are applicable to the embodiments of the device, and the specific functions and technical effects brought by the implementation manners can be referred to the method embodiments part. Therefore, no further description is given here.

[0057] ​Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0058] Please refer to Figure 2 The third aspect of the present disclosure provides a device for obtaining a high-resolution image, comprising a processor 301 and a memory 302 storing computer program instructions; the processor 301 implements the control method of the first aspect when executing the computer program instructions.

[0059] For example, the program can be divided into one or more modules / units, one or more modules / units are stored in the memory 302 and executed by the processor 301 to complete the present application. One or more modules / units can be a series of program instruction segments that can complete a specific function, which is used to describe the execution process of the program in the device.

[0060] Specifically, the above-mentioned processor 301 can include a central processing unit (CPU), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0061] The memory 302 can include a mass storage for data or instructions. For example, but not limited to, the memory 302 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of the above. In appropriate cases, the memory 302 can include removable or non-removable (or fixed) media. In appropriate cases, the memory 302 can be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0062] The memory can include read-only memory (ROM), random access memory (RAM), magnetic disk storage mediums devices, optical storage mediums devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage mediums (e.g., memory devices) encoded with software that, when executed (e.g., by one or more processors), is operable to perform the operations described with reference to the methods according to an aspect of the present disclosure.

[0063] The processor 301 implements any one of the methods in the above-described embodiments by reading and executing program instructions stored in the memory 302.

[0064] In one example, the device for acquiring a high-resolution image can further include a communication interface 303 and a bus 310. The processor 301, the memory 302, and the communication interface 303 are connected through the bus 310 and complete communication therebetween.

[0065] The communication interface 303 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0066] The bus 310 includes hardware, software, or both, that couples components of the online data traffic billing device to each other in a known manner. By way of example, and not limitation, the bus can include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a HyperTransport (HT) interconnect, an industry standard architecture (ISA) bus, an infiniband interconnect, a low pin count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standards association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, the bus 310 can include one or more buses. Although the present application describes and illustrates a particular bus, the present application contemplates any suitable bus or interconnect.

[0067] In addition, in combination with the method in the above-described embodiments, the embodiments of the present application can provide a computer storage medium to implement. The computer storage medium stores program instructions; the program instructions are executed by the processor to implement any one of the methods in the above-described embodiments.

[0068] It is to be understood that the application is not limited to particular configurations and processes described herein and shown in the drawings. The detailed description is not to be taken in a limiting sense, and the scope of the present application is defined by the appended claims. In the above embodiments, several specific steps are described and illustrated in order to provide a thorough understanding of the present application. However, the process of the present application can be carried out in some orders of the steps, under some conditions, and using some alternatives, all without departing from the spirit and scope of the application.

[0069] The functional modules shown in the structural block diagram above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The "machine-readable medium" can include any medium that can store or transfer information. Examples of the machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via a computer network such as the Internet, an intranet, and the like.

[0070] It is also to be understood that the example embodiments described in this application are based on a series of steps or apparatuses to describe some methods or systems. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0071] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer program instructions can also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other processing devices to operate in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0072] The above merely provides the specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present disclosure, which should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for acquiring high-resolution images, characterized in that, include: The transient output result is obtained by reconstructing the input Gaussian white noise using an image depth prior network. Under the constraint of the loss function, the transient output result and the known blur kernel of the input are convolved to obtain the brightness temperature information of the first low-frequency channel, and the brightness temperature information of the first low-frequency channel is matched with the input degraded image. The gradient information of the high-frequency channel is extracted by using the gradient loss function, and the gradient information of the brightness temperature information of the first low-frequency channel is matched with the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image. Under the constraint of the loss function, the transient output result and the known blur kernel are convolved to obtain the brightness temperature information of the first low-frequency channel, including generating the brightness temperature information of the first low-frequency channel using the following formula: ； ； Among them, z k and z x These are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise, respectively, T k (z k T is the two-dimensional PSF after reshaping a simple neural network. x (z x (This is a bright temperature image with higher spatial resolution after DIP networking;) The gradient loss function uses 36.5GHz channel information as gradient constraint information; The expression for the gradient loss function is as follows: ;in The gradient representing the corresponding brightness temperature information using the Sobel operator is calculated using the following formula: ； ； This represents the gradient information obtained from the network output through the Sobel gradient operator, while This represents the gradient information extracted from the 36.5GHz channel information using the same operator.

2. The method for acquiring high-resolution images according to claim 1, characterized in that, Before convolving the transient output with a known blur kernel under the constraint of the loss function to obtain the brightness temperature information of the first low-frequency channel, the method further includes: The transient output results are preprocessed.

3. The method for acquiring high-resolution images according to claim 2, characterized in that, The preprocessing of the transient output includes: noise reduction and contrast enhancement.

4. An apparatus for acquiring high-resolution images, characterized in that, The high-resolution image acquisition device includes: The first processing module is used to reconstruct the input Gaussian white noise through an image depth prior network to obtain a transient output result; The second processing module is used to convolve the transient output result and the known input blur kernel under the constraint of the loss function to obtain the brightness temperature information of the first low-frequency channel, and the brightness temperature information of the first low-frequency channel is matched with the input degraded image. The third processing module is used to extract the gradient information of the high-frequency channel through the gradient loss function, and to keep the gradient information of the brightness temperature information of the first low-frequency channel and the gradient information of the high-frequency channel matched, thereby obtaining the enhanced high-resolution image. Under the constraint of the loss function, the transient output result and the known blur kernel are convolved to obtain the brightness temperature information of the first low-frequency channel, including generating the brightness temperature information of the first low-frequency channel using the following formula: ； ； Among them, z k and z x These are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise, respectively, T k (z k T is the two-dimensional PSF after reshaping a simple neural network. x (z x (This is a bright temperature image with higher spatial resolution after DIP networking;) The gradient loss function uses 36.5GHz channel information as gradient constraint information; The expression for the gradient loss function is as follows: ;in The gradient representing the corresponding brightness temperature information using the Sobel operator is calculated using the following formula: ； ； This represents the gradient information obtained from the network output through the Sobel gradient operator, while This represents the gradient information extracted from the 36.5GHz channel information using the same operator.

5. An apparatus for acquiring high-resolution images, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the method for acquiring a high-resolution image as described in any one of claims 1-3.

6. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the method for acquiring a high-resolution image as described in any one of claims 1-3.

Images