Method and system for generating low-resolution satellite image usable in super-resolution model training, and method and system for training blur kernel generator therefor

Abstract

The present invention provides a method for training a blur kernel generator, the method including the steps of: receiving a gray-scale first satellite image and a color-scale second satellite image having a lower resolution than the first satellite image; lowering the resolution of the first satellite image through a predetermined interpolation method; estimating a first blur kernel; converting the second satellite image to gray-scale; estimating a second blur kernel; and using the first blur kernel as training data and the second blur kernel as target data to train the blur kernel generator to simulate a blur kernel of the second satellite image.

Description

Method and system for generating low-resolution satellite images that can be used for super-resolution model learning, and method and system for learning a blur kernel generator therefor

[0001] The present invention relates to a method and system for generating low-resolution satellite images that can be used for super-resolution model learning, and a method and system for learning a blur kernel generator therefor.

[0002] Recently, interest in techniques for super-resolving image resolution through various artificial neural networks, which are actively being researched, is increasing.

[0003] This image super-resolution is performed by utilizing training data consisting of high-resolution and low-resolution images, and when a low-resolution image is input to an artificial neural network, the image is upscaled and converted into a high-resolution image.

[0004] To this end, a method of downscaling high-resolution images through various interpolations has been used to build a large amount of learning data for learning super-resolution models.

[0005] The present invention relates to a method and system for generating low-resolution satellite images usable for super-resolution model learning.

[0006] In addition, the present invention relates to a method and system for generating low-resolution satellite images, which generate low-resolution images that reflect the characteristics and blur of actual low-resolution images in response to high-resolution images.

[0007] In addition, the present invention relates to a method and system for learning a blur kernel generator for generating low-resolution satellite images.

[0008] In addition, the present invention relates to a method and system for learning a blur kernel generator that generates a blur kernel reflecting the characteristics and blur of an actual low-resolution image from a blur kernel of a low-resolution image.

[0009] In order to solve the problem discussed above, a method for learning a blur kernel generator according to the present invention may include the steps of: receiving a first satellite image in gray scale and a second satellite image in color scale, the second satellite image having a lower resolution than the first satellite image; down-scaling the resolution of the first satellite image through a predetermined interpolation method and estimating a first blur kernel from the down-scaled first satellite image; converting the second satellite image to gray scale and estimating a second blur kernel from the second satellite image converted to gray scale; and training a blur kernel generator to imitate a blur kernel of the second satellite image by using the first blur kernel as learning data and the second blur kernel as correct answer data.

[0010] In addition, a blur kernel generator learning system according to the present invention includes an input unit that receives a first satellite image in gray scale and a second satellite image in color scale, the second satellite image having a lower resolution than the first satellite image; and a control unit that learns a blur kernel generator based on the first satellite image and the second satellite image, wherein the control unit lowers the resolution of the first satellite image through a predetermined interpolation method, estimates a first blur kernel from the lowered first satellite image, converts the second satellite image to gray scale, estimates a second blur kernel from the second satellite image converted to gray scale, and trains the blur kernel generator to imitate the blur kernel of the second satellite image by using the first blur kernel as learning data and the second blur kernel as correct answer data.

[0011] In addition, a program stored in a computer-readable recording medium according to the present invention is a program stored in a computer-readable recording medium, which is executed by one or more processes in an electronic device, and which is a program, the program may include instructions for performing the steps of: receiving a first satellite image in gray scale and a second satellite image in color scale, the second satellite image having a lower resolution than the first satellite image; down-scaling the resolution of the first satellite image through a predetermined interpolation method and estimating a first blur kernel from the down-scaled first satellite image; converting the second satellite image to gray scale and estimating a second blur kernel from the second satellite image converted to gray scale; and training a blur kernel generator to imitate the blur kernel of the second satellite image by using the first blur kernel as training data and the second blur kernel as correct data.

[0012] In addition, a method for generating a low-resolution satellite image according to the present invention may include a step of learning a blur kernel generator using a blur kernel generator learning method according to any one of claims 1 to 6; a step of receiving a high-resolution image; and a step of converting the high-resolution image into a low-resolution image using the learned blur kernel generator to generate learning data for a super-resolution model.

[0013] According to various embodiments of the present invention, a method and system for learning a blur kernel generator trains a generator implemented to generate a predetermined blur kernel using a blur kernel of an image obtained by downscaling a high-resolution image and a blur kernel estimated from an actual low-resolution image, thereby generating a blur kernel that reflects the characteristics and blur of an actual low-resolution image from the blur kernel of the downscaling image.

[0014] In addition, according to various embodiments of the present invention, a method and system for generating a low-resolution satellite image can generate a low-resolution image that reflects the characteristics and blur of an actual low-resolution image in response to a high-resolution image by converting the blur kernel of an image that has been down-scaled from a high-resolution image into a new blur kernel using a blur kernel generator that has been pre-learned based on an adversarial generative neural network.

[0015] Figure 1 illustrates a blur kernel generator learning system according to the present invention.

[0016] Figure 2 illustrates a low-resolution satellite image generation system according to the present invention.

[0017] Figure 3 is a flowchart showing a blur kernel generator learning method according to the present invention.

[0018] Figure 4 illustrates an example of downscaling a first satellite image.

[0019] Figure 5 illustrates an embodiment of gray-scaling a second satellite image.

[0020] Figure 6 illustrates an embodiment of training a blur kernel generator.

[0021] Figure 7 is a flowchart showing a method for generating low-resolution satellite images according to the present invention.

[0022] Figure 8 illustrates an embodiment of generating a new blur kernel using a pre-trained blur kernel generator.

[0023] Figure 9 illustrates an embodiment of performing downscaling using a new blur kernel.

[0024] Figure 10 illustrates an example of training a super-resolution model using low-resolution satellite images.

[0025] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0026] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0027] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0028] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0029] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0030] Fig. 1 illustrates a blur kernel generator learning system according to the present invention. Fig. 2 illustrates a low-resolution satellite image generation system according to the present invention.

[0031] Referring to FIG. 1, a blur kernel generator learning system (100) according to the present invention can train a blur kernel generator (21) using a first satellite image (11) and a second satellite image (12) having different resolutions so that, when an arbitrary blur kernel is input, a blur kernel corresponding to the resolution of the second satellite image (12) is output based on the input blur kernel.

[0032] Here, the first satellite image (11) is a satellite image having a first resolution, and may be a satellite image having a higher resolution than the resolution of the second satellite image (12). For example, the first satellite image (11) may be a panchromatic image having a high spatial resolution and captured in black and white.

[0033] The second satellite image (12) is a satellite image having a second resolution lower than the first resolution, and may be a satellite image having a lower resolution than the resolution of the first satellite image (11). For example, the second satellite image (12) may be a multispectral image captured in color, although it has a relatively lower spatial resolution than the first satellite image (11).

[0034] In this regard, in one embodiment, the first satellite image (11) and the second satellite image (12) may have a first resolution and a second resolution, respectively, determined so that a blur kernel generated from a learning-completed blur kernel generator can be used to generate learning data used for learning a predetermined super-resolution model.

[0035] Here, the super-resolution model may be a model trained to generate high-resolution images based on input low-resolution images. In other words, the super-resolution model can be trained using training data in which high-resolution images are labeled on low-resolution images.

[0036] Accordingly, the first resolution of the first satellite image (11) can be determined to correspond to the resolution of the high-resolution image to be output by the super-resolution model, and the second resolution of the second satellite image (12) can be determined to correspond to the resolution of the low-resolution image to be input to the super-resolution model.

[0037] Meanwhile, a blur kernel may contain information related to the blur phenomenon appearing in an image. For example, a blur kernel can be estimated by analyzing the edge region of a blurred image, estimating a sharp image corresponding to the blurred image, and comparing the estimated sharp image with the blurred image.

[0038] Accordingly, the blur kernel generator learning system (100) can estimate a blur kernel for each of the first satellite image (11) and the second satellite image (12) through a predetermined blur kernel estimation algorithm. At this time, the blur kernel generator learning system (100) can perform predetermined preprocessing on each of the first satellite image (11) and the second satellite image (12), and estimate a blur kernel for each of the preprocessed first satellite image (11) and the second satellite image (12).

[0039] The blur kernel generator (21) may be trained to convert an input blur kernel into a blur kernel corresponding to a second resolution when an arbitrary blur kernel is input.

[0040] To this end, when a given blur kernel is input, the blur kernel generator (21) can generate another blur kernel that is a conversion of the input blur kernel.

[0041] Accordingly, a pre-implemented discriminator (22) corresponding to a blur kernel generator (21) can compare another previously generated blur kernel with a predetermined correct blur kernel, and the blur kernel generator (21) can learn the other previously generated blur kernel to imitate the correct blur kernel based on the comparison result from the discriminator (22).

[0042] For example, the blur kernel generator learning system (100) can train the blur kernel generator (21) to simulate a blur kernel corresponding to the second resolution of the second satellite image (12) when an arbitrary blur kernel is input, using a first blur kernel generated based on a first satellite image (11) and a second blur kernel generated based on a second satellite image (12).

[0043] That is, the blur kernel generator learning system (100) inputs a first blur kernel into a pre-implemented blur kernel generator (21), generates a new blur kernel corresponding to a second resolution, which is the resolution of a second satellite image (12), compares the previously generated new blur kernel with the second blur kernel using a pre-implemented discriminator (22), and trains the blur kernel generator (21) so that the previously generated new blur kernel imitates the second blur kernel based on the comparison result.

[0044] In one embodiment, the blur kernel generator (21) and the discriminator (22) may be Generative Adversarial Networks (GANs) that repeat learning based on generation and discrimination.

[0045] Meanwhile, the blur kernel generator learning system (100) according to the present invention may include an input unit (110), a storage unit (120), and a control unit (130).

[0046] The input unit (110) can be connected to at least one server or another device storing at least one of the first satellite image (11) and the second satellite image (12) via a wireless or wired network. Accordingly, the input unit (110) can receive at least one of the first satellite image (11) and the second satellite image (12) from at least one server or another device.

[0047] At this time, the input unit (110) may be connected to at least one of a satellite equipped to capture a first satellite image (11) and another satellite equipped to capture a second satellite image (12) via a wireless network. In this case, the input unit (110) may receive at least one of the first satellite image (11) and the second satellite image (12) from each satellite.

[0048] The storage unit (120) can store data and commands necessary for the operation of the blur kernel generator learning system (100) according to the present invention.

[0049] For example, the storage unit (120) can store a blur kernel generator (21) and a discriminator (22).

[0050] Additionally, the storage unit (120) can store a first satellite image (11), a second satellite image (12), and a first blur kernel estimated from the first satellite image (11) and a second blur kernel estimated from the second satellite image (12).

[0051] The control unit (130) can control the overall operation of the blur kernel generator learning system (100) according to the present invention. For example, the control unit (130) can perform preprocessing on a first satellite image (11), estimate a first blur kernel from the preprocessed first satellite image (11), perform preprocessing on a second satellite image (12), and estimate a second blur kernel from the preprocessed second satellite image (12).

[0052] Additionally, the control unit (130) can train the blur kernel generator (21) using the discriminator (22) together with the first blur kernel and the second blur kernel.

[0053] Meanwhile, referring to FIG. 2, the low-resolution satellite image generation system (200) according to the present invention can generate a low-resolution image (15) from a high-resolution image (13) by simulating a blur kernel (14) corresponding to a second resolution using a pre-learned blur kernel generator (23) based on a high-resolution image (13) and applying the previously generated blur kernel (14) to the high-resolution image (13).

[0054] Here, the high-resolution image (13) is a high-resolution image, for example, a satellite image having the same resolution as the first satellite image (11) used for training the blur kernel generator (21) by the blur kernel generator training system (100), and may be a different satellite image depending on the embodiment.

[0055] That is, the resolution of the high-resolution image (13) may be the same as the first resolution of the first satellite image (11).

[0056] In addition, the low-resolution image (15) is a low-resolution image from the high-resolution image (13), and may be, for example, a satellite image having the same resolution as the second resolution of the second satellite image (12) used for learning the blur kernel generator (21).

[0057] Accordingly, the high-resolution image (13) and the low-resolution image (15) can be used as learning data for a super-resolution model, and for example, by labeling the high-resolution image (13) as correct data in the low-resolution image (15) generated from the high-resolution image (13) using a blur kernel generator (23), the super-resolution model can be trained so that when a low-resolution image is input, an image converted to a high-resolution is output.

[0058] It goes without saying that this low-resolution satellite image generation system (200) may, depending on the embodiment, be integrated with the blur kernel generator learning system (100) according to the present invention to operate as a single system.

[0059] Meanwhile, the low-resolution satellite image generation system (200) may include an input unit (210), a storage unit (220), a control unit (230), and an output unit (240).

[0060] The input unit (210) can be connected to a server or other device storing a high-resolution image (13) via a wireless or wired network. Accordingly, the input unit (210) can receive a high-resolution image (13) from a server or other device.

[0061] In some embodiments, the input unit (210) may be connected to a satellite equipped to capture high-resolution images (13) via a wireless network. In this case, the input unit (210) may receive high-resolution images (13) from the satellite.

[0062] The storage unit (220) can store data and commands necessary for the operation of the low-resolution satellite image generation system (200) according to the present invention.

[0063] For example, the storage unit (220) may store a high-resolution image (13) and a pre-learned blur kernel generator (23). Additionally, the storage unit (220) may store a blur kernel estimated based on the high-resolution image (13).

[0064] Additionally, the storage unit (220) can store a blur kernel (14) generated from a pre-learned blur kernel generator (23), and can store a low-resolution image (15) converted from a high-resolution image (13) through a previously generated blur kernel (14).

[0065] The control unit (230) can control the overall operation of the low-resolution satellite image generation system (100) according to the present invention. For example, the control unit (230) can perform preprocessing on a high-resolution image (13) and estimate a blur kernel from the preprocessed high-resolution image (13).

[0066] At this time, the control unit (230) can estimate the blur kernel through a predetermined blur kernel estimation algorithm for the high-resolution image (13).

[0067] In addition, the control unit (230) can input the estimated blur kernel to the pre-learned blur kernel generator (23) to generate a blur kernel (14), and apply the blur kernel (14) to the high-resolution image (13) to generate a low-resolution image (15) that has been down-scaled from the high-resolution image (13).

[0068] The output unit (240) can be connected to a predetermined display device via a wireless or wired network. Accordingly, the output unit (240) can output information related to a high-resolution image (13), a blur kernel estimated from the high-resolution image (13), a blur kernel (14) generated from a blur kernel generator (23) based on the high-resolution image (13), and a low-resolution image (15) so that a user can visually confirm the information.

[0069] Based on the configuration of the blur kernel generator learning system (100) and the low-resolution satellite image generation system (200) discussed above, the blur kernel generator learning method and the low-resolution satellite image generation method will be described in more detail below.

[0070] FIG. 3 is a flowchart illustrating a method for training a blur kernel generator according to the present invention. FIG. 4 illustrates an embodiment of downscaling a first satellite image. FIG. 5 illustrates an embodiment of gray-scaling a second satellite image. FIG. 6 illustrates an embodiment of training a blur kernel generator. FIG. 7 is a flowchart illustrating a method for generating a low-resolution satellite image according to the present invention. FIG. 8 illustrates an embodiment of generating a new blur kernel using a pre-trained blur kernel generator. FIG. 9 illustrates an embodiment of performing downscaling using a new blur kernel. FIG. 10 illustrates an embodiment of training a super-resolution model using a down-resolution satellite image.

[0071] Referring to FIG. 3, the blur kernel generator learning system (100) according to the present invention can receive a first satellite image in gray scale and a second satellite image in color scale that has a lower resolution than the first satellite image (S100).

[0072] Specifically, the blur kernel generator learning system (100) can receive a first satellite image having a first resolution and a second satellite image having a second resolution lower than the first resolution.

[0073] For example, the blur kernel generator learning system (100) can receive a first satellite image, which is a panchromatic image that has a high spatial resolution and is photographed in black and white, and a second satellite image, which is a multispectral image that has a relatively low spatial resolution compared to the first satellite image but is photographed in color.

[0074] As another example, the blur kernel generator learning system (100) can receive high-resolution satellite images such as KOMPSAT (Korea Multi-Purpose SATellite) and Sentinel-2 as first satellite images, and can receive medium-to-low-resolution satellite images such as Landsat as second satellite images.

[0075] The blur kernel generator learning system (100) according to the present invention can lower the resolution of a first satellite image through a predetermined interpolation method and estimate a first blur kernel from the lowered resolution first satellite image (S200).

[0076] Specifically, the blur kernel generator learning system (100) can convert the first resolution of the first satellite image into the second resolution of the second satellite image by lowering the resolution of the first satellite image so that the first satellite image and the second satellite image correspond to each other when the first resolution of the first satellite image is higher than the second resolution of the second satellite image.

[0077] For example, referring to FIG. 4, if the first resolution of the first satellite image (31) is higher than the second resolution of the second satellite image (41), the blur kernel generator learning system (100) can perform bicubic interpolation on the first satellite image (31) to downscale the first resolution of the first satellite image (31) to the second resolution of the second satellite image.

[0078] Furthermore, the blur kernel generator learning system (100) can estimate a first blur kernel from a first satellite image down-scaled to a second resolution corresponding to a second satellite image.

[0079] For example, the blur kernel generator learning system (100) can compare a first satellite image with a first satellite image down-resolution to a second resolution, and estimate a first blur kernel based on the comparison result.

[0080] As another example, the blur kernel generator learning system (100) may estimate the first blur kernel by performing a predetermined blur kernel estimation algorithm on the first low-resolution satellite image.

[0081] The blur kernel generator learning system (100) according to the present invention can convert a second satellite image into gray scale and estimate a second blur kernel from the second satellite image converted into gray scale.

[0082] For example, referring to FIG. 5, the blur kernel generator learning system (100) can convert the second satellite image (41) to a gray scale (43) so that the color distributions of the first satellite image (31) and the second satellite image (41) match when the first satellite image (31) is a gray scale (34) image and the second satellite image (41) is a color scale (42) (e.g., RGB) image.

[0083] In this regard, the blur kernel generator learning system (100) may utilize various color conversion techniques to make the color distribution of the second satellite image (41) and the low-resolution first satellite image (31) the same, and in the case where the color distribution of the second satellite image (41) and the low-resolution first satellite image (31) are the same, the preprocessing process for the second satellite image (41) may be omitted.

[0084] Furthermore, the blur kernel generator learning system (100) can estimate a second blur kernel from a second satellite image converted to grayscale corresponding to the first satellite image.

[0085] For example, the blur kernel generator learning system (100) can estimate a second blur kernel by performing a predetermined blur kernel estimation algorithm on a second satellite image converted to gray scale.

[0086] The blur kernel generator learning system (100) according to the present invention can train the blur kernel generator to imitate the blur kernel of the second satellite image by using the previously estimated first blur kernel as learning data and the previously estimated second blur kernel as correct data.

[0087] Specifically, the blur kernel generator learning system (100) can input a first blur kernel into a pre-implemented blur kernel generator and generate a new blur kernel corresponding to the first blur kernel.

[0088] Accordingly, the blur kernel generator learning system (100) can compare a previously generated new blur kernel with a second blur kernel using a discriminator implemented in advance to correspond to the blur kernel generator, and train the blur kernel generator so that the new blur kernel imitates the second blur kernel based on the comparison result.

[0089] Referring to FIG. 6, for example, when a new blur kernel (e.g., a fourth blur kernel (54)) corresponding to a first blur kernel (37) is generated through a pre-implemented blur kernel generator (51), the blur kernel generator learning system (100) can compare the previously generated fourth blur kernel (54) with the second blur kernel (47) using a pre-implemented discriminator and generate a result value corresponding to the comparison result.

[0090] At this time, the first blur kernel (37) may be estimated from a low-resolution first satellite image (36), and the second blur kernel (47) may be estimated from a gray-scale second satellite image (46).

[0091] Accordingly, the blur kernel generator learning system (100) can input a result value generated in advance into the blur kernel generator (51) to correct at least one parameter implemented in the blur kernel generator (51).

[0092] At this time, the blur kernel generator learning system (100) can determine that the learning of the blur kernel generator (51) is completed when the result value generated based on the comparison result between the fourth blur kernel (54) and the second blur kernel (47) reaches a predetermined threshold.

[0093] Alternatively, the blur kernel generator learning system (100) may compare a plurality of test blur kernels output from the blur kernel generator (51) with a plurality of second blur kernels corresponding to each of the plurality of test first blur kernels to generate a plurality of result values, and may determine that learning of the blur kernel generator (51) is completed when the number of times a result value higher than a predetermined threshold is output reaches a predetermined ratio with respect to the number of the plurality of test first blur kernels.

[0094] Through the above configurations, the blur kernel generator learning system (100) according to the present invention trains a generator implemented to generate a predetermined blur kernel using the blur kernel of an image that has been down-scaled from a high-resolution image and the blur kernel estimated from an actual low-resolution image, thereby generating a blur kernel that reflects the characteristics and blur of an actual low-resolution image from the blur kernel of the down-scaled image.

[0095] Referring to FIG. 7, the low-resolution satellite image generation system (200) according to the present invention receives a high-resolution image (S600), and converts the high-resolution image into a low-resolution image through a pre-trained blur kernel generator, thereby generating learning data for a super-resolution model.

[0096] At this time, the low-resolution satellite image generation system (200) can learn a blur kernel generator and receive a high-resolution image by using a blur kernel generator learning method.

[0097] For example, a low-resolution satellite image generation system (200) can receive a high-resolution Panchromatic image as a high-resolution image.

[0098] As another example, the low-resolution satellite image generation system (200) can also receive satellite images such as KOMPSAT (Korea Multi-Purpose SATellite) and Sentinel-2 as high-resolution images.

[0099] Furthermore, the low-resolution satellite image generation system (200) can downscale a high-resolution image through a predetermined interpolation method and estimate a blur kernel from the downscaled high-resolution image (S710).

[0100] For example, a low-resolution satellite image generation system (200) can perform bicubic interpolation on a high-resolution image to downscale the resolution of the high-resolution image to a second resolution, and estimate a blur kernel from the high-resolution image downscaled to the second resolution.

[0101] As another example, a low-resolution satellite image generation system (200) can downscale a high-resolution image through a predetermined interpolation method, compare the high-resolution image with the downscaled high-resolution image, and estimate a blur kernel based on the comparison result.

[0102] As another example, the low-resolution satellite image generation system (200) may estimate a blur kernel by performing a predetermined blur kernel estimation algorithm on a low-resolution high-resolution image.

[0103] Furthermore, the low-resolution satellite image generation system (200) can input a blur kernel estimated from a high-resolution image into a pre-trained blur kernel generator to obtain a new blur kernel (S720), and apply the new blur kernel obtained from the trained blur kernel generator to the high-resolution image to generate a low-resolution image (S730).

[0104] Specifically, the low-resolution satellite image generation system (200) can input a previously estimated blur kernel into a pre-learned blur kernel generator to obtain a new blur kernel that mimics the second blur kernel of the second satellite image.

[0105] Referring to FIG. 8, for example, a low-resolution satellite image generation system (200) can input a previously estimated blur kernel (e.g., a fifth blur kernel (63)) into a pre-learned blur kernel generator (56) based on a first satellite image having a first resolution and a second satellite image having a second resolution, thereby obtaining a new blur kernel (e.g., a third blur kernel (64)).

[0106] At this time, the fifth blur kernel (63) may be estimated from an image (62) that is a low-resolution image (61), and the third blur kernel (64) may be generated from a pre-learned blur kernel generator.

[0107] As another example, a low-resolution satellite image generation system (200) can input a fifth blur kernel to a pre-learned blur kernel generator based on a panchromatic image and a multispectral image having a lower resolution than the panchromatic image, thereby obtaining a third blur kernel corresponding to the resolution of the multispectral image.

[0108] As another example, the low-resolution satellite image generation system (200) can input the fifth blur kernel to a blur kernel generator that has been previously learned based on high-resolution satellite images such as KOMPSAT and Sentinel-2 and low-to-medium resolution satellite images such as Landsat, and obtain a third blur kernel corresponding to the resolution of the low-to-medium resolution satellite image.

[0109] Through this, as shown in FIG. 9, the low-resolution satellite image generation system (200) can generate a low-resolution image (66) that is down-scaled to correspond to the second resolution from the high-resolution image by applying a blur kernel (e.g., a third blur kernel (64)) previously acquired (or generated) to the high-resolution image (61).

[0110] Furthermore, the low-resolution satellite image generation system (200) can generate learning data using high-resolution images and low-resolution images so that when a low-resolution image is input, a super-resolution model can be trained to generate a high-resolution image corresponding to the low-resolution image.

[0111] Referring to FIG. 10, for example, a low-resolution satellite image generation system (200) can generate a high-resolution image (61) and a low-resolution image (66) as learning data (71) by labeling a high-resolution image (61) as correct answer data for the low-resolution image (66) in the low-resolution image (66).

[0112] Through this, the learning data (71) generated based on the high-resolution image (61) and the low-resolution image (66) can be used as learning data (71) for training a super-resolution model (70) implemented to output a high-resolution image (77) corresponding to the input low-resolution image (76) when a low-resolution image (76) is input.

[0113] Through the above configurations, the low-resolution satellite image generation system (200) according to the present invention can generate a low-resolution image that reflects the characteristics and blur of an actual low-resolution image in response to a high-resolution image by converting the blur kernel of an image that has been low-resolutioned from a high-resolution image into a new blur kernel using a blur kernel generator that has been learned in advance based on an adversarial generative neural network.

[0114] Furthermore, the present invention discussed above can be implemented as a program executed by one or more processes in an electronic device and stored in a computer-readable recording medium.

[0115] Accordingly, the present invention can be implemented as computer-readable code or instructions on a program-recorded medium. That is, the various control methods according to the present invention can be provided in the form of integrated or individual programs.

[0116] Meanwhile, computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0117] Furthermore, the computer-readable medium may include a storage device and may be a server or cloud storage device accessible via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage device via wired or wireless communication.

[0118] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, i.e., a CPU (Central Processing Unit), and there is no particular limitation on its type.

[0119] Meanwhile, the above detailed description should not be construed as limiting in any respect and should be considered illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A step of receiving a first satellite image in gray scale and a second satellite image in color scale, the second satellite image having a lower resolution than the first satellite image; A step of lowering the resolution of the first satellite image through a predetermined interpolation method and estimating a first blur kernel from the lowered resolution first satellite image; A step of converting the second satellite image into grayscale and estimating a second blur kernel from the second satellite image converted into grayscale; and A method for training a blur kernel generator, comprising the step of training a blur kernel generator to imitate a blur kernel of the second satellite image by using the first blur kernel as training data and the second blur kernel as correct data.

2. In the first paragraph, the step of matching the resolution is: A blur kernel generator learning method for performing bicubic interpolation on the first satellite image to downscale the first resolution of the first satellite image to the second resolution of the second satellite image.

3. In paragraph 1, The above first satellite image is a panchromatic image that has a higher spatial resolution than the above second satellite image and is photographed in black and white. A method for learning a blur kernel generator, wherein the second satellite image is a multispectral image captured in color and has a lower spatial resolution than the first satellite image.

4. In the first paragraph, the step of training the blur kernel generator comprises: A step of inputting the first blur kernel into the blur kernel generator to generate a new blur kernel corresponding to the first blur kernel; A step of comparing the newly generated blur kernel with the second blur kernel using a pre-implemented discriminator corresponding to the blur kernel generator; and A method for training a blur kernel generator, comprising a step of training the blur kernel generator so that the new blur kernel mimics the second blur kernel based on the comparison results.

5. In the first paragraph, the step of estimating the first blur kernel comprises: A method for learning a blur kernel generator, comprising the step of converting a first resolution of the first satellite image into a second resolution of the second satellite image by lowering the resolution of the first satellite image so that the first satellite image and the second satellite image correspond to each other.

6. A step of learning a blur kernel generator using a blur kernel generator learning method of any one of clauses 1 to 5; Step of receiving a high-resolution image; and A method for generating low-resolution satellite images, comprising the step of generating training data for a super-resolution model by converting the high-resolution image into a low-resolution image through the learned blur kernel generator.

7. In the 6th paragraph, the step of generating learning data of the super-resolution network is as follows: A step of downscaling the high-resolution image through a predetermined interpolation method and estimating a blur kernel from the downscaled high-resolution image; A step of obtaining a new blur kernel by inputting a blur kernel estimated from the high-resolution image into the learned blur kernel generator; and A method for generating a low-resolution satellite image, comprising the step of generating a low-resolution image by applying the new blur kernel obtained from the learned blur kernel generator to the high-resolution image.

8. In the 6th paragraph, the super-resolution model is A method for generating low-resolution satellite images, wherein when a low-resolution image is input, a model is trained to generate a high-resolution image based on the input low-resolution image, and the model is trained using the low-resolution image as training data and the high-resolution image as correct data.

9. An input unit for receiving a first satellite image in gray scale and a second satellite image in color scale, having a lower resolution than the first satellite image; and A control unit for training a blur kernel generator based on the first satellite image and the second satellite image, The above control unit, A blur kernel generator learning system, which lowers the resolution of the first satellite image through a predetermined interpolation method, estimates a first blur kernel from the lowered resolution first satellite image, converts the second satellite image to gray scale, estimates a second blur kernel from the second satellite image converted to gray scale, and trains the blur kernel generator to imitate the blur kernel of the second satellite image by using the first blur kernel as learning data and the second blur kernel as correct data.

10. A program that is executed by one or more processes in an electronic device and stored in a computer-readable recording medium, The above program is, A step of receiving a first satellite image in gray scale and a second satellite image in color scale, the second satellite image having a lower resolution than the first satellite image; A step of lowering the resolution of the first satellite image through a predetermined interpolation method and estimating a first blur kernel from the lowered resolution first satellite image; A step of converting the second satellite image into grayscale and estimating a second blur kernel from the second satellite image converted into grayscale; and A program stored on a computer-readable recording medium, characterized in that it includes commands for performing a step of training a blur kernel generator to imitate the blur kernel of the second satellite image by using the first blur kernel as learning data and the second blur kernel as correct data.

Images