High resolution system, high resolution method, and high resolution program

Abstract

To provide a learning method which relatively increases accuracy of a high-resolution image generated by inputting an actual low-resolution image to a learned model in the learning method of a resolution improvement model for an SAR image.SOLUTION: A resolution improvement system includes a learning device 20. The learning device 20 learns a learned model for executing super resolution processing by using data for learning consisting of a high-resolution full-aperture image DIN2 generated by executing image regeneration processing using a pulse reflected in each irradiation region and a low-resolution sub-aperture image DIN1 generated by executing image regeneration processing using a portion of the pulse used when generating the high-resolution full-aperture image DIN2.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a high resolution system, a high resolution method, and a high resolution program. [Background technology]

[0002] Conventionally, ground sensors have been used to grasp the damage caused by disasters such as floods, heavy rains, and earthquakes. The method of grasping the damage caused by disasters using ground sensors is useful for grasping the damage situation at spots such as houses and power plants. However, when the damage occurs over a wide area, it is difficult to obtain surface information such as which areas have suffered the most damage.

[0003] In recent years, analytical methods using remote sensing data have been proposed as a way to grasp the extent of disaster damage occurring over a wide area.Specific examples of remote sensing include aircraft and UAVs (drones), optical satellites, and synthetic aperture radar (SAR) satellites. The means for acquiring SAR images is a means for observing the earth's surface regardless of weather and time of day, and is used as a means for monitoring various objects on the earth's surface.

[0004] There is a technology that improves the ground resolution of an image by increasing the resolution of the image using a high-resolution model trained by machine learning. Here, the high-resolution model is also called a super-resolution model. Patent Document 1 discloses a technology for increasing the resolution of an image using a high-resolution model, in which a high-resolution image is prepared as a teacher image, a low-resolution image generated by downsampling the high-resolution image is prepared as an input image, and a high-resolution model is generated by training a machine learning model using the data pair of the prepared high-resolution image and low-resolution image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2020 / 175446 Brochure Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology disclosed in Patent Document 1, a high-resolution technology using machine learning generates a low-resolution image by using downsampling as a degradation process. However, the degradation process corresponding to an actual low-resolution image is often different from simple downsampling. Therefore, this technology has a problem that the accuracy of a high-resolution image generated by inputting an image actually acquired at low resolution to a trained model generated by this technology is reduced.

[0007] The present disclosure aims to provide a learning method for a high-resolution model for SAR images, in which the accuracy of a high-resolution image generated by inputting an actual low-resolution image into a trained model is relatively high. [Means for solving the problem]

[0008] The resolution enhancing system according to the present disclosure comprises: A model generating unit that uses learning data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating a pulse reflected in each of a plurality of irradiation regions, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using a portion of the pulses used in generating each of the plurality of high-resolution full aperture images, to learn a learned model that is a model for inferring an estimated high-resolution full aperture image, which is an image showing the target irradiation region and has a resolution higher than the resolution of the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to a pulse reflected in the target irradiation region and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images. A learning device comprising: Equipped with. Effect of the Invention

[0009] In the present disclosure, the model generation unit learns the trained model using training data consisting of a high-resolution full aperture image generated by performing an image reproduction process using a pulse reflected in an irradiation area, and a low-resolution sub-aperture image generated by performing an image reproduction process using a part of the pulse used to generate the high-resolution full aperture image. Here, the low-resolution sub-aperture image can be said to be an image based on an actual degradation process. Therefore, according to the present disclosure, in a training method for a high-resolution model for SAR images, a training method can be provided in which the accuracy of a high-resolution image generated by inputting an actual low-resolution image into the trained model is relatively high. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a high resolution system 1 according to a first embodiment. [Diagram 2] 1 is a block diagram illustrating a schematic configuration of an input image generating device 10 according to a first embodiment. [Diagram 3] 4A and 4B are diagrams for explaining image synthesis according to the first embodiment, where FIG. 4A illustrates a full aperture image and FIG. 4B illustrates a sub-aperture image. [Figure 4] 1 is a block diagram showing a schematic configuration of a learning device 20 according to a first embodiment. [Diagram 5] FIG. 1 is a diagram explaining a neural network. [Figure 6] 1 is a block diagram showing a schematic configuration of an inference device 30 according to a first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of the hardware configuration of a learning device 20 according to the first embodiment. [Figure 8] 4 is a flowchart showing the operation of the learning device 20 according to the first embodiment. [Figure 9]4 is a flowchart showing the operation of the inference device 30 according to the first embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a hardware configuration of a learning device 20 according to a modified example of the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. Descriptions of elements given the same reference numerals are omitted or simplified as appropriate. Arrows in the drawings primarily indicate data flow or processing flow. In addition, each of "device" and "part" may be read as "circuit", "process", "procedure", "processing" or "circuitry" as appropriate.

[0012] Embodiment 1 Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0013] ***Configuration Description*** Fig. 1 is a block diagram showing a schematic configuration example of a high resolution system 1 according to embodiment 1. As shown in Fig. 1, the high resolution system 1 includes an input image generating device 10 that generates an input image for learning, a learning device 20 that functions as a model generating device, and an inference device 30 that functions as an utilization device. The multiple devices that make up the high resolution system 1 may be configured integrally as appropriate. In the resolution improvement system 1, the inference device 30 uses a trained model trained by the learning device 20 to improve the resolution of the low-resolution full aperture image DIN3.

[0014] FIG. 2 is a block diagram illustrating an example of the configuration of the input image generating device 10. As shown in FIG. The input image generating device 10 includes an image reproducing unit 101 . The image reconstructing unit 101 receives a complex radar image DIN0 as an input, and generates two types of amplitude images, a low-resolution sub-aperture image DIN1 and a high-resolution full aperture image DIN2, by performing image reconstruction on the input complex radar image DIN0. The image reconstructing unit 101 typically generates a plurality of high-resolution full aperture images DIN2 by performing image reconstruction processing using pulses indicated by the complex radar images DIN0 indicating pulses reflected in each of a plurality of irradiation regions. Also, the image reconstructing unit 101 typically generates a plurality of low-resolution sub-aperture images DIN1 by performing image reconstruction processing using some of the pulses used when generating each of the plurality of high-resolution full aperture images DIN2. The complex radar image DIN0 is an image showing the pulse reflected in the illuminated area and an image showing the phase. The complex radar image may be an image captured by using either a synthetic aperture radar satellite or an aircraft.

[0015] SAR is a technology in which a flying object continuously emits microwaves while moving, and by appropriately combining the reflected waves that return from the same target, it is possible to obtain a resolution equivalent to that obtained by transmitting and receiving radio waves using a single huge antenna. In SAR, a single point on the ground receives multiple pulses from a moving flying object.

[0016] Here, as shown in (a) of Fig. 3, a full aperture image is defined as an image generated by synthesizing all of the pulses reflected from the radar irradiation area and received by the antenna. In Fig. 3, an aircraft equipped with an antenna is moving in the direction of travel, and the antenna receives a pulse at the position where the antenna is depicted. A full aperture image does not necessarily have to be an image generated using all of the pulses strictly speaking. Also, as shown in Fig. 3(b), an image generated by synthesizing an image using a portion of the pulses reflected from the radar irradiation area and received by the antenna is defined as a subaperture image. In Fig. 3(b), of the three antenna positions shown, the image is shown to be synthesized using only the pulses received by the antenna at the antenna position shaded in black. As a specific example, the portion of the pulse indicates about 1 / 10 to 1 / 2 of the entire band of the pulses received by the antenna. The irradiation area is also called the irradiation surface.

[0017] Unlike low-resolution images generated by downsampling high-resolution SAR images, subaperture images generated by image reconstruction processing can be said to be SAR images that are actually acquired. Therefore, generating subaperture images is considered appropriate as a degradation process of SAR images. As a specific example, the shape of the point image of the SAR image is maintained in the subaperture image, so the subaperture image can be said to be an actual SAR image. On the other hand, the point image of the SAR image is crushed in the downsampled image, so the downsampled image is different from the actual SAR image.

[0018] Fig. 4 is a block diagram showing a schematic configuration example of the learning device 20. As shown in Fig. 4, the learning device 20 includes a data acquisition unit 201, a model generation unit 202, and a learned model storage unit 203. The learning method executed by the learning device 20 corresponds in particular to a learning method for high resolution processing of remote sensing data.

[0019] The data acquisition unit 201 acquires learning data, and provides the acquired learning data to the model generation unit 202 . The learning data is teacher data indicating each of a low-resolution sub-aperture image DIN1 and a high-resolution full aperture image DIN2. The high-resolution full aperture image DIN2 corresponds to a correct answer to be inferred from the low-resolution sub-aperture image DIN1. The learning data typically consists of a plurality of low-resolution sub-aperture images DIN1 and a plurality of high-resolution full aperture images DIN2.

[0020] The model generation unit 202 learns a high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 by using the learning data provided by the data acquisition unit 201. That is, the model generation unit 202 executes machine learning by using a combination of the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 indicated by the learning data, thereby generating a learned model that infers an optimal high-resolution image corresponding to the low-resolution full aperture image DIN3. The model generation unit 202 stores the generated trained model in the trained model storage unit 203.

[0021] The model generation unit 202 may use a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning as a learning algorithm. Here, supervised learning is a method of providing a learning device with a set of data indicating an input and data indicating a result (label) as learning data, thereby learning features contained in the learning data and inferring a result from the input. As an example, a case where the model generation unit 202 uses a neural network will be described here.

[0022] Here, when the model generation unit 202 uses supervised learning, the learning data must include a combination of data consisting of a low-resolution sub-aperture image DIN1 and a high-resolution full-aperture image DIN2, which contains the same subject. Note that when the model generation unit 202 uses unsupervised learning, the low-resolution sub-aperture image DIN1 and the high-resolution full-aperture image DIN2 do not need to contain the same subject.

[0023] As a specific example, the model generating unit 202 learns a high-resolution full-aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 by so-called supervised learning according to a neural network model. A neural network is composed of an input layer consisting of multiple neurons, an intermediate layer (hidden layer) consisting of multiple neurons, and an output layer consisting of multiple neurons. The number of intermediate layers may be one or more.

[0024] FIG. 5 shows a schematic diagram illustrating an example of a three-layer neural network. In FIG. 5, when a plurality of input values ​​are input from the input layer X1 to the input layer X3, each of the plurality of input values ​​is multiplied by a first weight w11 to a first weight w16 (hereinafter also referred to as a first weight W1). Each calculated value, which is a value obtained by multiplying the input value by the first weight w11 to the first weight w16, is input to one of the intermediate layers Y1 and Y2. Each calculated value is multiplied by a second weight w21 to a second weight w26 (hereinafter also referred to as a second weight W2). Each output value, which is a value obtained by multiplying the calculated value by the second weight w21 to the second weight w26, is output from one of the output layers Z1 to Z3. Each output value is determined according to the value of the first weight W1 and the value of the second weight W2.

[0025] In this embodiment, the neural network learns a learned model for inferring an optimal high-resolution image corresponding to the low-resolution full-aperture image DIN3 by so-called supervised learning, according to learning data created based on a combination of a low-resolution sub-aperture image DIN1 and a high-resolution full-aperture image DIN2 indicated by the learning data acquired by the data acquisition unit 201. That is, the model generation unit 202 uses a neural network to learn a learned model by adjusting the first weight W1 and the second weight W2 so that when a low-resolution sub-aperture image DIN1 is input to the input layer, the result output from the output layer approaches the correct high-resolution full-aperture image DIN2.

[0026] As shown in FIG. 4, the trained model storage unit 203 stores the trained model provided from the model generation unit 202.

[0027] Fig. 6 is a block diagram showing a schematic configuration example of the inference device 30. As shown in Fig. 6, the inference device 30 includes a trained model storage unit 203, a data acquisition unit 301, and an inference unit 302. The trained model storage unit 203 included in the inference device 30 may be the same as the trained model storage unit 203 included in the learning device 20. The inference device 30 executes the utilization method. The inference device 30 uses the learned model provided by the learning device 20 to infer an estimated high-resolution full-aperture image DOUT from the low-resolution full-aperture image DIN3. The low-resolution full aperture image DIN3 is an image generated based on an actually observed pulse, corresponds to a pulse reflected in a target irradiation area, and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images DIN1. The resolution corresponding to the plurality of low-resolution sub-aperture images DIN1 is, as a specific example, a resolution equal to or greater than the lowest resolution and equal to or less than the highest resolution among the resolutions possessed by each of the plurality of low-resolution sub-aperture images DIN1. The resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images DIN1 is, as a specific example, a resolution indicated by the resolution corresponding to the plurality of low-resolution sub-aperture images DIN1, or a resolution close to the resolution corresponding to the plurality of low-resolution sub-aperture images DIN1. The estimated high-resolution full aperture image DOUT is an image showing the target illumination area corresponding to the low-resolution full aperture image DIN3, is an image inferred from the low-resolution full aperture image DIN3 by the learned model, and has a higher resolution than the resolution of the low-resolution full aperture image DIN3.

[0028] The data acquisition unit 301 acquires a low-resolution full-aperture image DIN3, and provides the acquired low-resolution full-aperture image DIN3 to the inference unit 302. Here, it is assumed that the low-resolution full-aperture image DIN3 represents a target subject.

[0029] The inference unit 302 infers an estimated high-resolution full-aperture image DOUT showing a target subject, using the learned model stored in the learned model storage unit 203 and the low-resolution full-aperture image DIN3. That is, the inference unit 302 acquires an estimated high-resolution full-aperture image DOUT by inputting the low-resolution full-aperture image DIN3 to the learned model.

[0030] 7 shows an example of the hardware configuration of learning device 20 according to this embodiment. Learning device 20 is made up of a computer. Learning device 20 may be made up of multiple computers.

[0031] As shown in the figure, the learning device 20 is a computer including hardware such as a processor 51, a memory 52, an auxiliary storage device 53, an input / output IF (Interface) 54, and a communication device 55. These pieces of hardware are appropriately connected via signal lines 59.

[0032] The processor 51 is an integrated circuit (IC) that performs arithmetic processing and controls hardware included in a computer. Specific examples of the processor 51 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The learning device 20 may include a plurality of processors that replace the processor 51. The plurality of processors share the role of the processor 51.

[0033] The memory 52 is typically a volatile storage device, and a specific example is a RAM (Random Access Memory). The memory 52 is also called a primary storage device or a main memory. Data stored in the memory 52 is saved in the auxiliary storage device 53 as necessary.

[0034] The auxiliary storage device 53 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 53 is loaded into the memory 52 as necessary. The memory 52 and the auxiliary storage device 53 may be integrated into one unit.

[0035] The input / output IF 54 is a port to which an input device and an output device are connected. A specific example of the input / output IF 54 is a Universal Serial Bus (USB) terminal. Specific examples of the input device are a keyboard and a mouse. A specific example of the output device is a display.

[0036] The communication device 55 is a receiver and a transmitter, and is, for example, a communication chip or a network interface card (NIC).

[0037] Each part of the learning device 20 may use the input / output IF 54 and the communication device 55 as appropriate when communicating with other devices.

[0038] The auxiliary storage device 53 stores a high resolution program. The high resolution program is a program that causes a computer to realize the functions of each unit of the learning device 20. The high resolution program is loaded into the memory 52 and executed by the processor 51. The functions of each unit of the learning device 20 are realized by software.

[0039] Data used when executing the high resolution program and data obtained by executing the high resolution program are appropriately stored in a storage device. Each part of the learning device 20 uses a storage device as appropriate. As a specific example, the storage device is composed of at least one of the memory 52, the auxiliary storage device 53, a register in the processor 51, and a cache memory in the processor 51. The storage device may be independent of the computer. The functions of the memory 52 and the auxiliary storage device 53 may be realized by other storage devices.

[0040] The high resolution program may be recorded in a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The high resolution program may be provided as a program product. The hardware configuration of each device included in the high resolution system 1 may be similar to the hardware configuration of the learning device 20.

[0041] ***Explanation of Operation*** The operation procedure of the high resolution system 1 corresponds to a high resolution method. Moreover, a program that realizes the operation of the high resolution system 1 corresponds to a high resolution program. Moreover, the high resolution method is also a general term for methods executed in each device included in the high resolution system 1. The high resolution program is also a general term for programs executed in each device included in the high resolution system 1.

[0042] 8 is a flowchart showing an example of the learning process executed by the learning device 20. The learning process will be described with reference to this figure.

[0043] (Step S11) The data acquisition unit 201 acquires learning data via the input image generation device 10 , and provides the acquired learning data to the model generation unit 202 .

[0044] (Step S12) The model generation unit 202 generates a learned model by learning the high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 using so-called supervised learning based on a combination of a low-resolution sub-aperture image DIN1 and a high-resolution full aperture image DIN2 indicated by the learning data.

[0045] (Step S13) The trained model storage unit 203 stores the generated trained model.

[0046] 9 is a flowchart showing an example of the inference process executed by the inference device 30. The inference process will be described with reference to this figure. Hereinafter, it is assumed that the inference device 30 infers an estimated high-resolution full-aperture image DOUT corresponding to the low-resolution full-aperture image DIN3.

[0047] (Step S21) The data acquisition unit 301 acquires a low-resolution full-aperture image DIN3, and provides the acquired low-resolution full-aperture image DIN3 to the inference unit 302.

[0048] (Step S22) The inference unit 302 inputs the low-resolution full aperture image DIN3 to the learned model stored in the learned model memory unit 203, and obtains an estimated high-resolution full aperture image DOUT corresponding to the low-resolution full aperture image DIN3.

[0049] (Step S23) The inference unit 30 outputs the generated estimated high resolution full aperture image DOUT.

[0050] Although the case where supervised learning is applied as the learning algorithm used by the model generation unit 202 has been described, the first embodiment is not limited to this case. As a specific example, the model generation unit 202 may use reinforcement learning, unsupervised learning, semi-supervised learning, or the like as the learning algorithm instead of supervised learning.

[0051] Furthermore, the model generating unit 202 may learn a high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 according to learning data created for a plurality of high resolution systems 1 including the high resolution system 1 having the model generating unit 202. Note that the model generating unit 202 may learn using learning data acquired from a plurality of high resolution systems 1 used in the same area, or may learn using learning data acquired from a plurality of high resolution systems 1 operating independently in different areas.

[0052] Furthermore, the model generating unit 202 may add the high resolution system 1 that collects learning data to the targets from which learning data is acquired along the way, or may remove it from the targets from which learning data is acquired along the way. Furthermore, the model generation unit 202 may apply a trained model that has been trained on a high-resolution full-aperture image DIN2 corresponding to a low-resolution sub-aperture image DIN1 for a certain high-resolution system 1 to a high-resolution system 1 other than the certain high-resolution system 1, and update the trained model by re-training the high-resolution full-aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1 for the other high-resolution system 1.

[0053] The model generating unit 202 may use, as a learning algorithm, deep learning, which is a method for learning to extract features themselves. The model generating unit 202 may also execute machine learning according to other known methods, such as genetic programming, functional logic programming, or support vector machines, as specific examples.

[0054] Here, each of the multiple high-resolution full aperture images DIN2 is a target high-resolution full aperture image, and the low-resolution sub-aperture image DIN1 corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image. At this time, it is preferable that the ratio between the bandwidth in the image reproduction process corresponding to the target high-resolution full aperture image and the bandwidth in the image reproduction process corresponding to the target low-resolution sub-aperture image is determined based on the ratio between the spatial resolution of the target high-resolution full aperture image and the spatial resolution of the low-resolution full aperture image DIN3. In general, the ratio of the spatial resolution is equal to the ratio of the bandwidth. As a specific example, when the ratio between the spatial resolution of the target high-resolution full aperture image and the spatial resolution of the low-resolution full aperture image DIN3 is 1:4, it is preferable to generate the target low-resolution sub-aperture image corresponding to the target high-resolution full aperture image using a band 1 / 4. However, the relationship between the bandwidth and the spatial resolution may differ depending on the satellite.

[0055] In addition, noise characteristics such as side lobes may differ between the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 corresponding to the low-resolution sub-aperture image DIN1. Therefore, each of the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 may be decomposed into images of finer size, and when a certain or greater difference occurs in the signal components between the decomposed images, the low-resolution sub-aperture image DIN1 and the high-resolution full aperture image DIN2 may not be used when generating a trained model, thereby improving the learning efficiency. Side lobes are noises specific to radar images. That is, in order to improve the learning efficiency, when the difference between the signal components in the target area of ​​the target high-resolution full aperture image and the signal components in the target area of ​​the target low-resolution sub-aperture image is equal to or greater than a threshold value, the target high-resolution full aperture image and the target low-resolution sub-aperture image are not used when training the trained model. The target area is at least a part of each of the target high-resolution full aperture image and the target low-resolution sub-aperture image. Moreover, the target area of ​​the target high-resolution full aperture image and the target area of ​​the target low-resolution sub-aperture image indicate the same area. In this case, as a specific example, the user sets a tolerance parameter indicating an allowable difference in signal components while visually checking the images. Specifically, the user checks whether or not a side lobe occurs by visually comparing the two images, and sets the tolerance parameter according to the difference in signal components when a side lobe occurs.

[0056] ***Explanation of the Effects of the First Embodiment*** As described above, in the high resolution system 1 according to the present embodiment, a trained model is generated using learning data based on the actual degradation process of SAR images, and the trained model is used to execute super-resolution processing. Therefore, according to the present embodiment, in the method for training a high resolution model for SAR images, when an actual low resolution image is input to the trained model, the super-resolution processing can be executed with relatively high accuracy.

[0057] ***Other configurations*** <Variation 1> FIG. 10 shows an example of the hardware configuration of a learning device 20 according to this modified example. The learning device 20 includes a processing circuit 58 in place of the processor 51 , the processor 51 and a memory 52 , the processor 51 and an auxiliary storage device 53 , or the processor 51 , the memory 52 , and the auxiliary storage device 53 . The processing circuitry 58 is hardware that realizes at least a portion of each unit of the learning device 20. The processing circuitry 58 may be dedicated hardware, or may be a processor that executes programs stored in the memory 52 .

[0058] When processing circuitry 58 is dedicated hardware, processing circuitry 58 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The learning device 20 may include a plurality of processing circuits that replace the processing circuit 58. The plurality of processing circuits share the role of the processing circuit 58.

[0059] In the learning device 20, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0060] Processing circuitry 58 is illustratively implemented in hardware, software, firmware, or a combination thereof. The processor 51, memory 52, auxiliary storage device 53, and processing circuit 58 are collectively referred to as the “processing circuitry.” In other words, the functions of the functional components of learning device 20 are realized by the processing circuitry. The hardware configuration of each device included in the high resolution system 1 may be similar to the hardware configuration of the learning device 20 according to this modified example.

[0061] ***Other embodiments*** Although the first embodiment has been described, it is possible to combine a plurality of parts of the present embodiment. Alternatively, it is possible to partially implement the present embodiment. In addition, the present embodiment may be modified in various ways as necessary, and may be implemented in any combination, either as a whole or in part. The above-described embodiment is essentially a preferred example, and is not intended to limit the scope of the present disclosure, its applications, and uses. The procedures described using the flowcharts and the like may be modified as appropriate. [Explanation of symbols]

[0062] 1 High resolution system, 10 Input image generation device, 101 Image reproduction unit, 20 Learning device, 201 Data acquisition unit, 202 Model generation unit, 203 Learned model storage unit, 30 Inference device, 301 Data acquisition unit, 302 Inference unit, 51 Processor, 52 Memory, 53 Auxiliary storage device, 54 Input / output IF, 55 Communication device, 58 Processing circuit, 59 Signal line, DIN0 Complex radar image, DIN1 Low resolution sub-aperture image, DIN2 High resolution full aperture image, DIN3 Low resolution full aperture image, DOUT Estimated high resolution full aperture image, W1 First weight, W2 Second weight.

Claims

1. A model generating unit that uses learning data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating a pulse reflected in each of a plurality of irradiation regions, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using a portion of the pulses used in generating each of the plurality of high-resolution full aperture images, to learn a learned model that is a model for inferring an estimated high-resolution full aperture image, which is an image showing the target irradiation region and has a resolution higher than the resolution of the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to a pulse reflected in the target irradiation region and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images. A learning device comprising: A high resolution system comprising:

2. an inference unit that infers an estimated high-resolution full aperture image, which is an image showing a target illumination area and has a higher resolution than the low-resolution full aperture image, by using a trained model trained using training data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination areas, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, and a low-resolution full aperture image, which is an image having a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images and corresponds to the pulses reflected in the target illumination area; An inference device comprising: Equipped with A resolution improvement system, wherein the learned model is a model that infers the estimated high-resolution full-aperture image from the low-resolution full-aperture image.

3. 3. The high resolution system according to claim 1, wherein the complex radar image is an image captured by using either a synthetic aperture radar satellite or an aircraft.

4. When each of the plurality of high-resolution full aperture images is a target high-resolution full aperture image, and a low-resolution sub-aperture image corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image, A high resolution system described in any one of claims 1 to 3, wherein the ratio between the bandwidth in the image reproduction process corresponding to the target high resolution full aperture image and the bandwidth in the image reproduction process corresponding to the target low resolution sub-aperture image is determined based on the ratio between the spatial resolution of the target high resolution full aperture image and the spatial resolution of the low resolution full aperture image.

5. When each of the plurality of high-resolution full aperture images is a target high-resolution full aperture image, and a low-resolution sub-aperture image corresponding to the target high-resolution full aperture image is a target low-resolution sub-aperture image, A high-resolution system as described in any one of claims 1 to 4, wherein when a difference between a signal component in a target region of the target high-resolution full aperture image and a signal component in the target region of the target low-resolution sub-aperture image is equal to or greater than a threshold, the target high-resolution full aperture image and the target low-resolution sub-aperture image are not used when training the trained model.

6. A method for increasing resolution, in which a computer uses learning data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of irradiation areas, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, to learn a learned model that is a model for inferring an estimated high-resolution full aperture image, which is an image showing a target irradiation area and has a higher resolution than the resolution of the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to pulses reflected in a target irradiation area and has a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images.

7. A high resolution method for inferring an estimated high resolution full aperture image, which is an image showing a target illumination area and has a higher resolution than the low resolution full aperture image, using a trained model trained using training data consisting of a plurality of high resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination areas, and a plurality of low resolution sub aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high resolution full aperture images, and a low resolution full aperture image, which is an image having a resolution equivalent to a resolution corresponding to the plurality of low resolution sub aperture images and corresponds to pulses reflected in a target illumination area, A high resolution method, in which the learned model is a model that infers the estimated high resolution full aperture image from the low resolution full aperture image.

8. A model generation process for learning a learned model that is a model for inferring an estimated high-resolution full aperture image, which is an image showing a target illumination region and has a higher resolution than the low-resolution full aperture image, from a low-resolution full aperture image, which is an image corresponding to a pulse reflected in a target illumination region and has a resolution equivalent to the resolution corresponding to the low-resolution sub-aperture images, using learning data consisting of a plurality of high-resolution full aperture images generated by executing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating a pulse reflected in each of a plurality of illumination regions, and a plurality of low-resolution sub-aperture images generated by executing an image reproduction process using a portion of the pulses used in generating each of the plurality of high-resolution full aperture images. A high-resolution program that causes a learning device, which is a computer, to execute the above.

9. An inference process for inferring an estimated high-resolution full aperture image, which is an image showing a target illumination region and has a higher resolution than the low-resolution full aperture image, using a trained model trained using training data consisting of a plurality of high-resolution full aperture images generated by performing an image reproduction process using pulses indicated by each of a plurality of complex radar images indicating pulses reflected in each of a plurality of illumination regions, and a plurality of low-resolution sub-aperture images generated by performing an image reproduction process using some of the pulses used in generating each of the plurality of high-resolution full aperture images, and a low-resolution full aperture image, which is an image having a resolution equivalent to the resolution corresponding to the plurality of low-resolution sub-aperture images and corresponds to pulses reflected in a target illumination region, A high resolution program for causing an inference device, which is a computer, to execute the above. The learned model is a resolution improvement program that is a model that infers the estimated high-resolution full-aperture image from the low-resolution full-aperture image.

Images