Method and device for acquiring high-resolution image, equipment and computer storage medium

Abstract

The embodiment of the invention provides a method and device for obtaining a high-resolution image, equipment and a computer storage medium. The method for obtaining the high-resolution image comprises the following steps: reconstructing input Gaussian white noise through an image depth prior network to obtain a transient output result; under the constraint of a loss function, convolution is carried out on the transient output result and an input known fuzzy kernel to obtain first low-frequency channel brightness temperature information, and the first low-frequency channel brightness temperature information is matched with the input degraded image; and extracting gradient information of a high-frequency channel through a gradient loss function, and enabling the gradient information of the brightness temperature information of the first low-frequency channel to be matched with the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image. Through the processing scheme disclosed by the invention, the undegraded low-frequency channel brightness temperature data of the satellite radiometer with extremely high details can be recovered without limitation of scarcity and type mismatching of a training set under a zero-sample method, and an enhanced high-resolution image is obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of microwave remote sensing technology, and in particular to a method, device, equipment and computer storage medium for acquiring high-resolution images. Background Art

[0002] As a remote sensing observation device, terahertz radiometer plays an important role in meteorological monitoring, resource exploration, environmental monitoring and other fields. However, due to the physical limitations of satellite radiometers and the limitations of imaging technology, the brightness temperature data obtained is often limited by spatial resolution, which makes it impossible to effectively interpret remote sensing data accurately. The current method of improving the spatial resolution of radiometers mainly relies on the learning process of sample supervision and requires a large amount of high-quality, labeled training data. However, such data acquisition is expensive and difficult to cover all usage scenarios, especially for new equipment or data collection in special environments, there are problems of scarce training samples and type mismatch. In addition, previous methods often require the use of complex algorithms and expensive computing resources when dealing with cross-channel resolution matching, which is limited in real applications. Therefore, there is an urgent need for an effective method that can achieve high-fidelity recovery of terahertz radiometer data in a zero-sample environment, improve the spatial resolution matching between different channels, enhance data availability, and reduce the complexity of data processing and the consumption of computing resources. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a method, apparatus, device and computer storage medium for acquiring high-resolution images, which at least partially solve the problems existing in the prior art.

[0004] The first aspect of the present disclosure provides a method for acquiring a high-resolution image, comprising:

[0005] S1, reconstruct the input Gaussian white noise through the image deep prior network to obtain the transient output result;

[0006] S2. Under the constraint of the loss function, convolve the transient output result and the input known blur kernel to obtain the brightness temperature information of the first low-frequency channel, and the brightness temperature information of the first low-frequency channel keeps matching with the input degraded image;

[0007] S3. Extracting the gradient information of the high-frequency channel through a gradient loss function, and making the gradient information of the brightness temperature information of the first low-frequency channel match the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image.

[0008] According to a specific implementation manner of the embodiment of the first aspect of the present disclosure, S2 includes generating the brightness temperature information of the first low-frequency channel by using the following formula:

[0009]

[0010]

[0011] Among them, z k and z x are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise respectively, T k (z k ) is the two-dimensional PSF after simple neural network reshaping, T x (z x ) is the bright temperature image with higher spatial resolution after the DIP network, t A (x, y, ν) is the transient output result.

[0012] According to a specific implementation of the embodiment of the first aspect of the present disclosure, before S2, it also includes:

[0013] S4. Preprocess the transient output results.

[0014] According to a specific implementation manner of the embodiment of the first aspect of the present disclosure, preprocessing the transient output result includes: denoising and contrast enhancement.

[0015] According to a specific implementation method of the embodiment of the first aspect of the present disclosure, the gradient loss function uses 36.5 GHz channel information as gradient constraint information.

[0016] According to a specific implementation of the first aspect of the present disclosure, the expression of the gradient loss function is as follows:

[0017]

[0018] Where represents the gradient of the corresponding brightness temperature information using the Sobel operator, and the calculation formula is as follows:

[0019]

[0020]

[0021] represents the gradient information obtained from the network output through the Sobel gradient operator, and Represents the gradient information extracted from the 36.5 GHz channel information by the same operator.

[0022] An embodiment of the second aspect of the present disclosure provides a device for acquiring a high-resolution image, comprising a first processing module, a second processing module and a third processing module, wherein the first processing module is used to reconstruct an input Gaussian white noise through an image depth prior network to obtain a transient output result; the second processing module is used to convolve the transient output result and an input known blur kernel under the constraint of a loss function to obtain brightness temperature information of a first low-frequency channel, and the brightness temperature information of the first low-frequency channel is kept matching with the input degraded image; the third processing module is used to extract gradient information of a high-frequency channel through a gradient loss function, and make the gradient information of the brightness temperature information of the first low-frequency channel match the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image.

[0023] The third aspect of the present disclosure provides a device for acquiring a high-resolution image, comprising a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements a method for acquiring a high-resolution image as described in any one of the first aspect embodiments described above.

[0024] The fourth aspect embodiment of the present disclosure provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, a method for acquiring a high-resolution image as described in any one of the aforementioned first aspect embodiments is implemented.

[0025] The method for obtaining a high-resolution image provided by the embodiment of the present disclosure includes reconstructing the input Gaussian white noise through an image deep prior network to obtain a transient output result; under the constraint of a loss function, convolving the transient output result and the input known blur kernel to obtain the brightness temperature information of the first low-frequency channel, and the brightness temperature information of the first low-frequency channel keeps matching with the input degraded image; through a gradient loss function, extracting the gradient information of the high-frequency channel, and making the gradient information of the brightness temperature information of the first low-frequency channel keep matching with the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image. Through the method provided by the present disclosure, it is possible to restore the extremely detailed low-frequency channel brightness temperature data of the satellite radiometer that has not been degraded under the zero-sample unsupervised method, without being constrained by the scarcity and type mismatch of the training set, to obtain an enhanced high-resolution image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1A schematic flow chart of a method for acquiring a high-resolution image provided in an embodiment of the first aspect of the present disclosure;

[0028] Figure 2 A schematic structural diagram of a device for acquiring high-resolution images is provided for an embodiment of the third aspect. DETAILED DESCRIPTION

[0029] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0030] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0031] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0032] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0033] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0034] As a remote sensing observation device, terahertz radiometer plays an important role in meteorological monitoring, resource exploration, environmental monitoring and other fields. However, due to the physical limitations of satellite radiometers and the limitations of imaging technology, the brightness temperature data obtained is often limited by spatial resolution, which makes it impossible to effectively interpret remote sensing data accurately. The current method of improving the spatial resolution of radiometers mainly relies on the learning process of sample supervision and requires a large amount of high-quality, labeled training data. However, such data acquisition is expensive and difficult to cover all usage scenarios, especially for data collection of new equipment or special environments, there are problems of scarce training samples and type mismatch. In addition, previous methods often require the use of complex algorithms and expensive computing resources when dealing with cross-channel resolution matching, which has limitations in real applications. Therefore, there is an urgent need for an effective method that can achieve high-fidelity restoration of terahertz radiometer data in an unsupervised environment, improve the spatial resolution matching between different channels, enhance data availability, and reduce the complexity of data processing and the consumption of computing resources.

[0035] To solve the above problems, the embodiments of the present disclosure provide a method, apparatus, device and computer storage medium for obtaining high-resolution images. The method, apparatus, device and computer storage medium of the embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a method for obtaining a high-resolution image provided by an embodiment of the first aspect of the present disclosure. The method for obtaining a high-resolution image comprises the following steps:

[0037] S1, reconstruct the input Gaussian white noise through the image deep prior network to obtain the transient output result;

[0038] S2. Under the constraint of the loss function, the transient output result and the input known blur kernel are convolved to obtain the brightness temperature information of the first low-frequency channel, and the brightness temperature information of the first low-frequency channel is matched with the input degraded image. Specifically, the loss function may include content loss (such as mean square error MSE), structural loss, etc.

[0039] S3. By using the gradient loss function, the gradient information of the high-frequency channel is extracted, and the gradient information of the brightness temperature information of the first low-frequency channel is matched with the gradient information of the high-frequency channel, so as to obtain an enhanced high-resolution image. The introduction of the gradient loss function can guide the model to learn the high-frequency information of the low-frequency channel. The function focuses on the edge and texture details of the radiometer image, which helps to recover the details of the high-resolution image from the low-resolution image.

[0040] Through the method provided by the present invention, it is possible to restore extremely detailed, non-degraded low-frequency channel brightness temperature data of a satellite radiometer in a zero-sample environment without being constrained by the scarcity and type mismatch of a training set, and obtain an enhanced high-resolution image.

[0041] In some optional embodiments, S2 includes generating the brightness temperature information of the first low-frequency channel using the following formula:

[0042]

[0043]

[0044] Among them, z k and z x are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise respectively, T k (z k ) is the two-dimensional PSF after simple neural network reshaping, T x (z x ) is the bright temperature image with higher spatial resolution after the DIP network, t A (x, y, ν) is the transient output result.

[0045] In this embodiment, the powerful fitting function of the neural network is utilized, and a neural blind deconvolution model of the DIP-Deep Image Prior network is used to generate the brightness temperature information of the first low-frequency channel that matches the degraded image. Some inherent problems of the initial DIP asymmetric codec, such as insufficient ability to capture prior information of the blur kernel, are improved and solved.

[0046] In some optional embodiments, before S2, the method further includes:

[0047] S4. Preprocess the transient output results. Specifically, the preprocessing may include denoising and contrast enhancement, thereby eliminating some interference, improving the quality of the acquired image, and reducing training time and improving processing efficiency.

[0048] In some optional embodiments, the gradient loss function uses 36.5 GHz channel information as gradient constraint information. The 36.5 GHz channel information is used as gradient constraint information not only because the 36.5 GHz channel contains more high-frequency information, but also because its extracted information has a higher similarity with the gradient information of the 18 GHz low-frequency channel, thereby avoiding the introduction of excessive errors and improving the quality of obtaining high-resolution signals.

[0049] Specifically, the 36.5 GHz channel information is used as the gradient constraint information, and the expression of the gradient loss function is as follows:

[0050]

[0051] Where represents the gradient of the corresponding brightness temperature information using the Sobel operator, and the calculation formula is as follows:

[0052]

[0053]

[0054] represents the gradient information obtained from the network output through the Sobel gradient operator, and Represents the gradient information extracted from the 36.5 GHz channel information by the same operator.

[0055] An embodiment of the second aspect of the present disclosure provides a device for acquiring a high-resolution image, comprising a first processing module, a second processing module and a third processing module, wherein the first processing module is used to reconstruct an input Gaussian white noise through an image depth prior network to obtain a transient output result; the second processing module is used to convolve the transient output result and an input known blur kernel under the constraint of a loss function to obtain brightness temperature information of a first low-frequency channel, and the brightness temperature information of the first low-frequency channel is kept matching with the input degraded image; the third processing module is used to extract gradient information of a high-frequency channel through a gradient loss function, and make the gradient information of the brightness temperature information of the first low-frequency channel match the gradient information of the high-frequency channel, thereby obtaining an enhanced high-resolution image.

[0056] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned method of obtaining high resolution. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and technical effects brought about can be found in the method embodiment part, which will not be repeated here.

[0057] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0058] Please refer to Figure 2 The third aspect of the present disclosure provides a device for acquiring high-resolution images, including a processor 301 and a memory 302 storing computer program instructions; when the processor 301 executes the computer program instructions, the control method of the first aspect is implemented.

[0059] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 302, and executed by the processor 301 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.

[0060] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0061] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 302 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0062] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0063] The processor 301 implements any one of the methods in the above embodiments by reading and executing program instructions stored in the memory 302 .

[0064] In one example, the device for acquiring high-resolution images may further include a communication interface 303 and a bus 310. The processor 301, the memory 302, and the communication interface 303 are connected via the bus 310 and communicate with each other.

[0065] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0066] Bus 310 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0067] In addition, in combination with the method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores program instructions; when the program instructions are executed by a processor, any one of the methods in the above embodiment is implemented.

[0068] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0069] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0070] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0071] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0072] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for obtaining a high-resolution image, characterized in that: include: The input Gaussian white noise is reconstructed through the image deep prior network to obtain the transient output result; Under the constraint of the loss function, convolve the transient output result and the input known blur kernel to obtain brightness temperature information of the first low-frequency channel, where the brightness temperature information of the first low-frequency channel keeps matching with the input degraded image; The gradient information of the high-frequency channel is extracted through the gradient loss function, and the gradient information of the brightness temperature information of the first low-frequency channel is matched with the gradient information of the high-frequency channel, so as to obtain an enhanced high-resolution image.

2. The method for obtaining a high-resolution image according to claim 1, characterized in that: Under the constraint of the loss function, convolving the transient output result with the known blur kernel to obtain the brightness temperature information of the first low-frequency channel includes using the following formula to generate the brightness temperature information of the first low-frequency channel: ； ； Among them, z k and z x are one-dimensional Gaussian white noise and two-dimensional Gaussian white noise respectively, T k (z k ) is the two-dimensional PSF after simple neural network reshaping, T x (z x ) is the bright temperature image with higher spatial resolution after the DIP network, t A (x, y, ν) is the transient output result.

3. The method for obtaining a high-resolution image according to claim 1, characterized in that: Before convolving the transient output result and the known blur kernel under the constraint of the loss function to obtain the brightness temperature information of the first low-frequency channel, the method further includes: The transient output result is preprocessed.

4. The method for obtaining a high-resolution image according to claim 3, characterized in that: The preprocessing of the transient output result includes: denoising and contrast enhancement.

5. The method for obtaining a high-resolution image according to claim 1, characterized in that: The gradient loss function uses 36.5 GHz channel information as gradient constraint information.

6. The method for obtaining a high-resolution image according to claim 5, characterized in that: The expression of the gradient loss function is as follows: ； Where represents the gradient of the corresponding brightness temperature information using the Sobel operator, and the calculation formula is as follows: ； ； represents the gradient information obtained from the network output through the Sobel gradient operator, and Represents the gradient information extracted from the 36.5 GHz channel information by the same operator.

7. A device for obtaining high-resolution images, characterized in that: The device for obtaining high-resolution images comprises: The first processing module is used to reconstruct the input Gaussian white noise through the image depth prior network to obtain a transient output result; A second processing module is used to convolve the transient output result and the input known blur kernel under the constraint of the loss function to obtain brightness temperature information of a first low-frequency channel, where the brightness temperature information of the first low-frequency channel keeps matching with the input degraded image; The third processing module is used to extract the gradient information of the high-frequency channel through the gradient loss function, and make the gradient information of the brightness temperature information of the first low-frequency channel match the gradient information of the high-frequency channel, so as to obtain an enhanced high-resolution image.

8. A device for acquiring high-resolution images, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for acquiring a high-resolution image according to any one of claims 1 to 5 is implemented.

9. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, which, when executed by a processor, implement the method for acquiring a high-resolution image according to any one of claims 1 to 5.

Images