Apparatus and method for generating extended reference image of low resolution satellite SAR image
By generating a reference image through geometric correction and difference image techniques, the method enhances the resolution and quality assessment of low-resolution satellite SAR images, addressing the limitations of existing methods and enabling improved super-resolution and classification capabilities.
Patent Information
- Application Number
- KR1020230184887
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-12-18
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2043-12-18
AI Technical Summary
Low-resolution satellite SAR images acquired by micro-satellites suffer from poor object-distinguishing resolution, and existing methods for improving resolution or measuring super-resolved image quality are limited by the lack of high-quality reference images for the same regions and time periods.
A method involving geometric correction, contour extraction, and difference image generation using Electro-Optical (EO) images to create a reference image for Synthetic Aperture Radar (SAR) images, enhancing resolution and enabling accurate quality measurement of super-resolved images.
The method improves the resolution of low-resolution satellite SAR images and allows for more accurate quality assessment of super-resolved images, facilitating applications in super-resolution technology and artificial intelligence-based small object classification.
Smart Images

Figure R1020230184887_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to super-resolution of Synthetic Aperture Radar (SAR) images, and more specifically, to an apparatus and method for generating an expanded reference image for a low-resolution satellite SAR image to improve identification performance for small targets. Background Technology
[0002] Synthetic Aperture Radar (SAR) is a radar system that creates topographic maps or observes the Earth's surface based on the time difference between the reflections of radar waves transmitted sequentially from satellites. SAR satellites have the advantage of being able to acquire images even in adverse weather conditions, unaffected by weather elements such as clouds. However, since SAR images acquired by ultra-small SAR satellites are generally low-resolution images of the 1m level, they have the disadvantage of poor object-distinguishing resolution. To overcome this drawback, a satellite SAR sensor system capable of acquiring high-resolution SAR images of the 0.5m level is required, but current technology faces limitations.
[0003] Accordingly, to improve the resolution of low-resolution satellite SAR images, various image interpolation methods are applied, or methods for generating super-resolution SAR images based on artificial neural networks are used. Additionally, to compare the quality of super-resolution SAR images, high-resolution (or high-quality) original satellite SAR images are artificially converted into low-resolution SAR images using artificial neural networks. However, in this case, since minute high-frequency components contained in the high-quality original satellite SAR images are removed, measurement deviation occurs when evaluating similarity with super-resolution satellite SAR images. Meanwhile, although there is a method of training artificial neural networks with pairs of low-resolution satellite SAR images and high-resolution original satellite SAR images, there are limitations in obtaining high-quality and low-resolution satellite SAR images for the actual same region and / or similar time periods. Therefore, in situations where it is difficult to obtain high-resolution satellite SAR images, there is a need for a method that can improve the resolution of low-resolution satellite SAR images acquired by a micro-satellite SAR sensor system and more accurately measure the quality of super-resolved satellite SAR images. Prior art literature
[0004] Korean Patent Publication No. 10-2185307 (Registration Date: November 25, 2020) The problem to be solved
[0005] The technical problem of the present invention is to provide an apparatus and method capable of generating a reference image for a Synthetic Aperture Radar (SAR) image.
[0006] Another technical objective of the present invention is to provide an apparatus and method capable of improving the resolution of low-resolution satellite SAR images acquired by a micro-satellite SAR sensor system.
[0007] The problems that the present invention aims to solve are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem
[0008] According to one embodiment of the present invention, a device for generating a reference image for a Synthetic Aperture Radar (SAR) image may include a correction unit configured to perform geometric correction on the SAR image based on an Electro-Optical (EO) image for the SAR image, an extraction unit configured to extract contours from the EO image and the geometrically corrected SAR image, respectively, and a generation unit configured to generate a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image, and to generate a reference image for the SAR image based on the geometrically corrected SAR image and the difference image.
[0009] Here, the SAR image and the EO image may have different resolutions.
[0010] In addition, the correction unit can reduce the size of the EO image to be the same as the SAR image and geometrically correct the SAR image based on the reduced EO image to match the coordinate system of the SAR image with the coordinate system of the reduced EO image.
[0011] In addition, the extraction unit can extract contours from the reduced EO image and the geometrically corrected SAR image, respectively, using at least one contour detection filter.
[0012] In addition, the generation unit can enlarge the size of the image formed by combining the geometrically corrected SAR image and the difference image to be the same as the EO image, and compare the similarity between the feature points extracted from the enlarged image and the feature points extracted from the EO image.
[0013] According to another embodiment of the present invention, a method for a reference image generating device to generate a reference image for a SAR image may include the steps of: geometrically correcting the SAR image based on an EO image for the SAR image; extracting contours from the EO image and the geometrically corrected SAR image, respectively; generating a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image; and generating a reference image for the SAR image based on the geometrically corrected SAR image and the difference image.
[0014] According to another embodiment of the present invention, a computer program stored in a computer-readable recording medium may include instructions for the processor to perform a reference image generation method, which, when executed by a processor, comprises the steps of: geometrically correcting a SAR image based on an EO image for the SAR image; extracting contours from the EO image and the geometrically corrected SAR image, respectively; generating a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image; and generating a reference image for the SAR image based on the geometrically corrected SAR image and the difference image. Effects of the invention
[0015] According to one embodiment of the present invention, since a reference image for a low-resolution satellite SAR (Synthetic Aperture Radar) can be generated, the quality of a super-resolved satellite SAR image can be measured more accurately based on this, and it can be utilized in fields related to super-resolution technology and in fields of artificial intelligence-based small object classification / detection.
[0016] According to one embodiment of the present invention, the resolution of low-resolution satellite SAR images acquired by a micro-satellite SAR sensor system can be improved. Brief explanation of the drawing
[0017] FIG. 1 is a drawing showing a reference image generating device according to one embodiment of the present invention. FIG. 2 is a drawing for explaining the process of generating a reference image according to an embodiment of the present invention. FIG. 3 is a drawing showing a contour image according to one embodiment of the present invention. FIG. 4 is a drawing showing a difference image according to one embodiment of the present invention. FIG. 5 is a drawing illustrating a method for generating a reference image according to an embodiment of the present invention. Specific details for implementing the invention
[0018] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Accordingly, in some embodiments, well-known processing steps, well-known device structures, and well-known techniques are not specifically described to avoid the present invention being interpreted ambiguously.
[0019] The terms used in this specification have been selected from among currently widely used general terms while considering their functions in the present invention; however, these may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this specification should be interpreted not merely by their names, but based on their meanings and the overall content of the present invention.
[0020] Throughout this specification, when a part is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.
[0021] Additionally, in this specification, the components of the reference image generation device may refer to software or hardware components such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and perform at least one function or operation. However, the components are not limited to software or hardware. The components may be configured to reside in an addressable storage medium or configured to run one or more processors. Accordingly, by example, the components include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided by the components of the present invention may be combined into a smaller number of components or further separated into additional components.
[0022] Additionally, while terms such as first, second, third, etc. may be used in this specification to describe various components, these components are not limited by said terms. These terms are used for the purpose of distinguishing one component from other components.
[0023] Hereinafter, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. In addition, parts unrelated to the description are omitted in the drawings to clearly explain the invention. Throughout the specification, the same reference numerals refer to the same components.
[0024] FIG. 1 is a drawing showing a reference image generating device according to one embodiment of the present invention.
[0025] Referring to FIG. 1, a reference image generating device (100) according to one embodiment of the present invention may include a correction unit (110), an extraction unit (120), and a generating unit (130).
[0026] The correction unit (110) may be configured to perform geometric correction on a SAR (Synthetic Aperture Radar) image based on an EO (Electro-Optical) image. Here, the SAR image and the EO image may be images acquired by a SAR satellite and an EO satellite, respectively, at the same or similar time period for the same region or area, and may have different resolutions.
[0027] For example, if the SAR image is a low-resolution image with a resolution of 1m and the EO image is a high-resolution image with a resolution of 0.5m, the correction unit (110) can reduce the size of the EO image to be the same as the SAR image and geometrically correct the SAR image based on the reduced EO image to match the coordinate system of the SAR image with the coordinate system of the reduced EO image.
[0028] The extraction unit (120) may be configured to extract contours from each of the geometrically corrected SAR image and the EO image in the correction unit (110). For example, the extraction unit (120) may extract contours from each of the reduced EO image and the geometrically corrected SAR image using at least one contour detection filter.
[0029] The generation unit (130) may be configured to generate a difference image between a contour image extracted from the geometrically corrected SAR image and a contour image extracted from the EO image, and to generate a reference image for the SAR image based on the geometrically corrected SAR image and the difference image. At this time, the generation unit (130) may enlarge the size of the image formed by combining the geometrically corrected SAR image and the difference image to be equal to that of the EO image, and compare the similarity between the feature points extracted from the enlarged image and the feature points extracted from the EO image. The generation unit (130) may determine an image among the enlarged images in which the similarity with the feature points extracted from the EO image is greater than or equal to a threshold value as a reference image for the corresponding SAR image.
[0030] FIG. 2 is a drawing for explaining the process of generating a reference image according to an embodiment of the present invention, FIG. 3 is a drawing showing a contour image according to an embodiment of the present invention, and FIG. 4 is a drawing showing a difference image according to an embodiment of the present invention.
[0031] Hereinafter, with reference to FIGS. 2 to 4, an example of a process in which a reference image generating device according to an embodiment of the present invention generates an extended reference image for a low-resolution satellite SAR image acquired by a micro-SAR satellite based on a high-resolution satellite EO image acquired by an EO satellite will be described.
[0032] First, referring to FIG. 2, a reference image generating device according to one embodiment of the present invention can reduce a high-resolution satellite EO image to the same size as a low-resolution satellite SAR image. For example, if the low-resolution satellite SAR image is an image with a resolution of 1 m and the high-resolution satellite EO image is an image with a resolution of 0.5 m, the reference image generating device can reduce the high-resolution satellite EO image by a factor of 2. Then, the reference image generating device can generate a geometrically corrected SAR image by geometrically correcting the low-resolution satellite SAR image based on the reduced EO image, and can generate a contour image as shown in FIG. 3 by applying a contour detection filter to each of the geometrically corrected SAR image and the reduced EO image.
[0033] Figure 3 illustrates, as an example, the result of binarizing a low-resolution satellite SAR image and a high-resolution satellite EO image by applying a Canny edge filter and a Prewitt filter, respectively. Referring to Figure 3, it can be seen that temporal object movement or spatial contour information is distinguished depending on the type of filter. Therefore, in one embodiment of the present invention, various contour detection filters may be selectively applied as needed.
[0034] When contour images are extracted from a low-resolution SAR image and a reduced EO image, respectively, a reference image generating device can generate a difference image as shown in FIG. 4 by calculating the difference values between pixels at the same location within each contour image. The difference image can be combined with a geometrically corrected SAR image to form an expanded image. Here, the expanded image may refer to a SAR image to which information (objects, contours, etc.) that exists in the EO image but not in the SAR image has been added.
[0035] Subsequently, when the SAR image is an image with a resolution of 1m and the EO image is an image with a resolution of 0.5m, the reference image generating device may generate an enlarged image by enlarging the enlarged image by a factor of 2 in order to compare the similarity between the enlarged image and the EO image. Then, the reference image generating device may extract feature points (or contours) from each of the enlarged image and the EO image and determine the similarity between the images by comparing them. For example, the reference image generating device may evaluate the similarity between the images by measuring the quality of each image based on PSNR (Peak Signal to Noise Rate), SSIM (Structural Similarity Index Measure), etc.
[0036] If the similarity between the enlarged image and the EO image is greater than or equal to a threshold (e.g., 95%), the reference image generating device may determine the enlarged image as a reference image for the low-resolution SAR image.
[0037] FIG. 5 is a drawing illustrating a method for generating a reference image according to an embodiment of the present invention.
[0038] Referring to FIG. 5, a reference image generating device according to one embodiment of the present invention can geometrically correct a SAR image based on a high-resolution EO image for a low-resolution SAR image (S500). Here, the SAR image and the EO image may be images acquired by a SAR satellite and an EO satellite, respectively, for the same region or area at the same or similar time. For example, if the SAR image is an image with a resolution of 1m and the EO image is an image with a resolution of 0.5m, the reference image generating device can reduce the size (or resolution) of the EO image to be the same as the SAR image and geometrically correct the SAR image based on the reduced EO image to match the coordinate system of the SAR image with the coordinate system of the reduced EO image.
[0039] Subsequently, the reference image generating device can generate a contour image by extracting contours from the geometrically corrected SAR image and the EO image, respectively, using a pre-selected contour detection filter (S510), and can generate a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image (S520). Then, the reference image generating device can generate a reference image for the SAR image based on the geometrically corrected SAR image and the difference image (S530). To this end, the reference image generating device can generate an expanded SAR image by combining the geometrically corrected SAR image and the difference image, enlarge the size of the expanded SAR image to be the same as the EO image, and then compare the similarity between the feature points extracted from the enlarged image and the feature points extracted from the EO image. If the similarity between the feature points extracted from the enlarged image and the feature points extracted from the EO image is greater than or equal to a threshold value, the reference image generating device can determine the enlarged image as the reference image for the corresponding SAR image.
[0040] Meanwhile, each step included in the reference image generation method performed by the reference image generation device according to the above-described embodiment may be implemented as a computer program including instructions for a processor to perform such steps.
[0041] In addition, each step included in the reference image generation method performed by the reference image generation device according to the above-described embodiment may be implemented on a computer-readable recording medium on which a computer program including instructions for a processor to perform such steps is recorded.
[0042] Combinations of each step of the flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions may be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the processor of the computer or other programmable data processing equipment create means for performing the functions described in each step of the flowchart. Since these computer program instructions may also be stored in a computer-available or computer-readable recording medium that can be oriented toward the computer or other programmable data processing equipment to implement the function in a specific way, the instructions stored in the computer-available or computer-readable recording medium may also produce a manufactured item containing instruction means for performing the function described in each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each step of the flowchart.
[0043] Additionally, each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Furthermore, in some alternative embodiments, the functions mentioned in the steps may occur out of order. For example, two steps described in succession may be performed substantially simultaneously, or the steps may be performed in reverse order depending on the corresponding function.
[0044] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention.
Claims
Claim 1 An apparatus for generating a reference image for a Synthetic Aperture Radar (SAR) image, comprising: a correction unit configured to perform geometric correction on the SAR image based on an Electro-Optical (EO) image for the SAR image; an extraction unit configured to extract contours from the EO image and the geometrically corrected SAR image, respectively; and a generation unit configured to generate a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image, and to generate a reference image for the SAR image based on the geometrically corrected SAR image and the difference image, wherein the SAR image and the EO image have different resolutions. Claim 2 delete Claim 3 An apparatus according to claim 1, wherein the correction unit reduces the size of the EO image to be the same as the SAR image, and geometrically corrects the SAR image based on the reduced EO image to align the coordinate system of the SAR image with the coordinate system of the reduced EO image. Claim 4 An apparatus according to claim 3, wherein the extraction unit extracts contours from the reduced EO image and the geometrically corrected SAR image, respectively, using at least one contour detection filter. Claim 5 An apparatus according to claim 1, wherein the generating unit enlarges the size of the image formed by combining the geometrically corrected SAR image and the difference image to be equal to the size of the EO image, and compares the similarity between feature points extracted from the enlarged image and feature points extracted from the EO image. Claim 6 A method for a reference image generating device to generate a reference image for a Synthetic Aperture Radar (SAR) image, comprising: a step of geometrically correcting the SAR image based on an Electro-Optical (EO) image for the SAR image; a step of extracting contours from the EO image and the geometrically corrected SAR image, respectively; a step of generating a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image; and a step of generating a reference image for the SAR image based on the geometrically corrected SAR image and the difference image. Claim 7 A method according to claim 6, wherein the SAR image and the EO image have different resolutions, and the step of geometrically correcting the SAR image comprises: a step of reducing the size of the EO image to be equal to the size of the SAR image; and a step of geometrically correcting such that the coordinate system of the SAR image matches the coordinate system of the reduced EO image based on the reduced EO image. Claim 8 A method according to claim 7, wherein the step of extracting the contour includes the step of extracting contours from the reduced EO image and the geometrically corrected SAR image, respectively, using at least one contour detection filter. Claim 9 A method according to claim 6, wherein the step of generating the reference image comprises: a step of enlarging the size of the image formed by combining the geometrically corrected SAR image and the difference image to be equal to the size of the EO image; and a step of comparing the similarity between feature points extracted from the enlarged image and feature points extracted from the EO image. Claim 10 A computer program stored on a computer-readable recording medium, the computer program comprising instructions for the processor to perform a reference image generation method, the method comprising: a step of geometrically correcting a Synthetic Aperture Radar (SAR) image based on an Electro-Optical (EO) image for the SAR image when executed by a processor; a step of extracting contours from the EO image and the geometrically corrected SAR image, respectively; a step of generating a difference image between the contour image extracted from the geometrically corrected SAR image and the contour image extracted from the EO image; and a step of generating a reference image for the SAR image based on the geometrically corrected SAR image and the difference image.
Citation Information
Patent Citations
Accurate extraction method of boundary line in high-resolution SAR amplitude images
KR101207220B1
Apparatus and method clearing image
KR1020120035481A
Method and Apparatus for Detecting Camera Tampering Using Edge Images
KR1020140049411A
Method and system for analyzing jamming effect
KR102546292B1
