Land use classification system, land use classification method, and land use classification program

Abstract

This land use classification system is provided with a learning device (20) provided with a model generation unit (202). The model generation unit (202) learns, on the basis of machine learning, the correspondence relationships between simulated satellite images (D102), each being an image which is generated by applying deterioration processing to a first reference image being a reference high-resolution image having a resolution higher than the resolution of a first satellite image being a target satellite image captured by a target satellite and which has a resolution corresponding to the resolution of the first satellite image, and land use maps each indicating a land use state in an observation range of the first reference image, by using training data comprising combinations of the simulated satellite images (D102) and the land use maps, to thereby generate a trained model for inferring a land use map from a simulated satellite image (D102). The first satellite image is used in inference processing using the trained model.

Description

Land use classification system, land use classification method, and land use classification program

[0001] The present disclosure relates to a land use classification system, a land use classification method, and a land use classification program. In particular, the present disclosure relates to a method for training a land use classification technique for remote sensing data.

[0002] Administrative agencies and designated public institutions have the task of grasping the land use situation over a wide area, such as urban and mountainous areas. This task has traditionally been accomplished by conducting on-site surveys. However, this task has been limited in terms of work time and scope.

[0003] In recent years, analytical methods using remote sensing data have been proposed as an alternative means of understanding land use over a wide area. Specific examples of remote sensing include methods using aircraft or UAVs (drones), as well as methods using optical satellites or synthetic aperture radar satellites. Optical satellite images are used as a means of monitoring various objects on the Earth's surface.

[0004] There is a technology for understanding land use that automatically interprets land use from remote sensing images using a classifier generated in advance through machine learning. One example of a technology for understanding land use is semantic segmentation.

[0005] In land use status assessment technology, data showing any remote sensing image is prepared as an input image, data showing a labeled image in which features on the ground surface that appear in the remote sensing image are labeled is generated and prepared as a teacher image, and a land use status assessment model is generated by performing machine learning using the prepared data pair (see, for example, Patent Document 1).

[0006] Japanese Patent Application Laid-Open No. 2019-046416

[0007] Patent Literature 1 discloses a method for estimating the position and shape of buildings from an input image. Furthermore, a pre-trained model with high generalized performance has been published for the task of classifying land use status, including conventional techniques.

[0008] When applying a pre-trained model to proprietary remote sensing data and tasks, there is a problem in that sufficient performance cannot be achieved, mainly due to differences in the domain (or features) between the training dataset of the publicly available pre-trained model and the proprietary data. For example, such differences arise when the data used as the training dataset is often high-resolution aerial imagery, while the proprietary data is low-resolution optical satellite imagery. Another method involves manually labeling proprietary optical satellite imagery. However, this method has the problem of being unable to ensure a sufficient volume and variety of training datasets due to the amount of work required.

[0009] Therefore, one or more aspects of the present disclosure relate to a training method for a trained model that classifies land use conditions for optical satellite images, and aim to provide a training method for generating a trained model that classifies land use conditions with relatively high accuracy when actual optical satellite images are input into the trained model.

[0010] The land use classification system according to the present disclosure is a land use classification system comprising: a learning device having a model generation unit that uses learning data consisting of a combination of a simulated satellite image, which is an image generated by applying degradation processing to a first reference image, which is a reference high-resolution image having a resolution higher than the resolution of a first satellite image, which is a target satellite image captured by a target satellite, and a land use map showing the land use status in the observation range of the first reference image, to learn the correspondence between the simulated satellite image and the land use map based on machine learning, thereby generating a trained model that infers the land use map from the simulated satellite image; and the first satellite image is used in the inference process using the trained model.

[0011] According to the present disclosure, a trained model is generated that generates a trained model that infers a land-use map from simulated satellite imagery. Here, the simulated satellite imagery is a simulated optical satellite imagery generated by applying a degradation process to a reference high-resolution image. Therefore, according to one or more aspects of the present disclosure, a training method can be provided for generating a trained model that can classify land use conditions on the Earth's surface from optical satellite imagery with relatively high accuracy when the optical satellite imagery is input to the trained model.

[0012] FIG. 1 is a diagram showing an example of the configuration of a land use classification system 1 according to the first embodiment. FIG. 2 is a diagram showing an example of the configuration of an input image generation device 10 according to the first embodiment. FIG. 3 is a diagram showing an example of the configuration of a degradation processing unit 102 according to the first embodiment. FIG. 4 is a diagram showing an example of the configuration of a learning device 20 according to the first embodiment. Schematic diagram of a neural network. FIG. 5 is a diagram showing an example of the configuration of an inference device 30 according to the first embodiment. FIG. 6 is a diagram showing an example of the hardware configuration of the input image generation device 10 according to the first embodiment. A flowchart showing the operation of the learning device 20 according to the first embodiment. A flowchart showing the operation of the inference device 30 according to the first embodiment. A diagram showing an example of the configuration of an input image generation device 10b according to a modification of the first embodiment. A diagram showing an example of the hardware configuration of the input image generation device 10 according to the modification of the first embodiment.

[0013] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. The description of elements given the same reference numerals will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, "unit" may be read as "circuit," "step," "procedure," "process," or "circuitry" as appropriate.

[0014] First Embodiment Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0015] ***Description of Configuration*** FIG. 1 is a block diagram illustrating an example of the configuration of a land use classification system 1 according to the first embodiment. The land use classification system 1 is a system that realizes a land use classification technology that estimates the land use status of the earth's surface from satellite images based on machine learning. The land use classification system 1 includes an input image generation device 10 that generates input images for learning, a learning device 20 that functions as a model generation device, an inference device 30 that functions as an utilization device, and a trained model storage unit 90. Note that the processing method performed by the inference device 30 corresponds to the utilization method. The multiple devices included in the land use classification system 1 may be integrated as appropriate. The land use classification system 1 uses a trained model to classify a target satellite image DIN3 according to land use. At this time, a land use label may be assigned to the target satellite image DIN3. The land use label is a label that indicates the land use status. Specific examples of land use labels include labels that indicate any of "building," "road," "water body," "forest," "bare land," and "field." The land use classification system 1 may assign one land use label to the entire target satellite image DIN3, or may assign a land use label to each feature appearing in the target satellite image DIN3. The process of assigning labels to an image is also called segmentation. The trained model is an inference model generated by the learning device 20 performing machine learning. As a specific example of generating the trained model, an open source pre-trained model is utilized. The pre-trained model is an inference model that infers a land use map from a satellite image. Typically, the pre-trained model uses an image with a higher resolution than the resolution of the target satellite image DIN3. The trained model storage unit 90 stores the pre-trained model and the trained model.

[0016] FIG. 2 is a block diagram illustrating an example of the configuration of the input image generation device 10. The input image generation device 10 includes an inference unit 101 and a degradation processing unit 102. The inference unit 101 acquires a pre-trained model from the trained model storage unit 90 and generates a high-precision land use map D101 by inputting a reference high-resolution image DIN1 to the acquired pre-trained model. By using the pre-trained model, the inference unit 101 can classify land use conditions with relatively high accuracy. The degradation processing unit 102 performs degradation processing. Specifically, the degradation processing unit 102 receives the reference high-resolution image DIN1 and target satellite information DIN2 as input and applies degradation processing to the reference high-resolution image DIN1 to generate a simulated satellite image D102. The satellite image is an image captured by an artificial satellite, and a specific example is an optical satellite image. The degradation processing is processing for generating a simulated satellite image of a target satellite by degrading the reference high-resolution image DIN1. The degradation processing may be processing corresponding to the target satellite information DIN2. If the degradation process is not a process corresponding to the target satellite information DIN2, the target satellite information DIN2 does not need to be input to the degradation process unit 102.

[0017] A specific example of the reference high-resolution image DIN1 is an image acquired by a remote sensing method capable of acquiring an image having a higher resolution than the target satellite image DIN3. Specific examples of the remote sensing method include aerial photography methods using a helicopter, a UAV (Unmanned Aerial Vehicle), or an aircraft, as well as methods using a satellite that can capture an image having a higher resolution than the target satellite image DIN3. The target satellite information DIN2 is information about the target satellite, and a specific example is information indicating the configuration and characteristics of the target satellite. The target satellite is an artificial satellite that captures the target satellite image DIN3. The target satellite image DIN3 is a satellite image used in inference processing using a trained model. The high-precision land use map D101 is a land use map that shows land use conditions estimated as land use conditions in the observation range of the reference high-resolution image DIN1. The simulated satellite image D102 is a simulated satellite image.

[0018] As a specific example, the degradation process is a process that simulates image degradation caused by at least one of the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics, the GSD (Ground Sampling Distance), the sensitivity performance, the noise characteristics, the data compression process when generating the target satellite image DIN3, and the quantization process when generating the target satellite image DIN3.

[0019] An example of degradation processing using sensor location information for a target satellite will be described. Optical satellites often carry multiple sensors, each with a different wavelength band, that receive light. Multiband images obtained from the multiple sensors are combined to generate a composite satellite image. In this case, color shifts may occur in the composite image due to the influence of the physical misalignment between the sensors. Color shifts may also occur in subpixels. When downsampling from the reference high-resolution image DIN1, the degradation processing unit 102 shifts the downsampling offset (binning start position) for each band, thereby generating a simulated satellite image D102 that reproduces subpixel color shifts. In other words, the degradation processing may involve downsampling the reference high-resolution image DIN1 using offsets that differ for each wavelength band.

[0020] An example of degradation processing using the PSF of a target satellite will be described. For optical satellite images, a PSF is determined as an index of resolution that combines multiple degradation models, such as the sensor, optical performance, aperture shape, turbulence, and the effects of image streaming. Similarly, the reference high-resolution image DIN1 also has a PSF as an index of resolution that takes into account the sensor, optical performance, and the effects of turbulence. By applying a target PSF to the reference high-resolution image DIN1 using the PSF of the target satellite and the PSF of the reference high-resolution image DIN1, degradation processing can be performed to convert the resolution of the reference high-resolution image DIN1 to a resolution equivalent to that of the target satellite image, thereby generating a simulated satellite image D102. Specific examples of degradation processing using a PSF include a method of multiplying the PSF in frequency space and a method of convolving the PSF with the reference high-resolution image DIN1.

[0021] Furthermore, when the PSF is checked for each band, the PSF may differ for each band. For example, the MTF (Modulated Transfer Function) of the multispectral band may be higher than the PSF of the panchromatic band, causing aliasing in the multispectral image. The degradation processor 102 can also reproduce aliasing in the simulated satellite image D102 by incorporating such differences in PSF into the degradation process.

[0022] An example of degradation processing using the GSD of a target satellite will be described. The degradation processing unit 102 compares the GSD of the reference high-resolution image DIN1 with the GSD of the target satellite, and simulates the GSD using a downsampling method such as bilinear interpolation or bicubic interpolation.

[0023] When performing degradation processing using GSD, it is desirable that the ratio between the GSD of the reference high-resolution image DIN1 and the GSD of the target satellite be equal to or greater than the high-resolution magnification of the land use classification system 1.

[0024] An example of degradation processing using the sensitivity performance of a target satellite will be described. The signal-to-noise ratio (SNR) of the target satellite image can be estimated from the sensitivity performance of the target satellite and the amount of light from the assumed subject. If the noise is assumed to be additive white Gaussian noise, the SNR is related to the standard deviation of the Gaussian noise. Therefore, the degradation processing unit 102 can also perform degradation processing in which noise equivalent to the noise of the target satellite image is applied to the reference high-resolution image DIN1 based on the SNR.

[0025] Furthermore, when noise characteristics other than additive white Gaussian noise are obtained from the target satellite information DIN2, for example, when information indicating striped noise or FPN (Fixed Pattern Noise) is obtained, the degradation processing unit 102 can also perform degradation processing in which the obtained noise characteristics are added to the reference high-resolution image DIN1.

[0026] In addition, if the target satellite image DIN3 is a thermal infrared image, the degradation processing unit 102 can also perform degradation processing in which shot noise due to internal radiation, 1 / f noise, shading, etc. are treated as noise characteristics and added to the reference high-resolution image DIN1.

[0027] Another possible noise factor is the influence of the atmosphere. An example of degradation processing that simulates the atmosphere will be described. When the high-resolution remote sensing method is an imaging means from within the atmosphere, the reference high-resolution image DIN1 does not contain noise caused by the atmosphere. On the other hand, the satellite image contains noise such as a decrease in image contrast caused by the atmosphere, occlusion, and turbulence. The degradation processing unit 102 generates image degradation caused by the atmosphere based on a publicly available atmospheric model, and applies the generated image degradation to the reference high-resolution image DIN1, thereby generating a simulated satellite image D102.

[0028] An example of degradation processing based on data compression of a target satellite will be described. Data compression is performed when downlinking a target satellite image to a ground station. If data compression is performed using lossy compression, image quality degradation due to compression occurs. The degradation processing unit 102 can reproduce the degradation by applying the same data compression to the reference high-resolution image DIN1.

[0029] 3 is a block diagram showing an example of the configuration of the degradation processing unit 102. The degradation processing unit 102 according to this example performs any one or a combination of color shift degradation processing, resolution degradation processing, noise degradation processing, and compression degradation processing. The compression degradation processing is processing that simulates image degradation due to data compression.

[0030] 4 is a block diagram showing an example of the configuration of the learning device 20. The learning device 20 includes a data acquisition unit 201 and a model generation unit 202.

[0031] The data acquisition unit 201 acquires training data. Here, the training data is teacher data that indicates a combination of a simulated satellite image D102 and a reference high-resolution image DIN1 that corresponds to a correct answer to be inferred from a target satellite image DIN3. The acquired training data is provided to the model generation unit 202.

[0032] The model generation unit 202 learns a high-precision land-use map D101 corresponding to the simulated satellite image D102 based on the training data provided by the data acquisition unit 201. In other words, the model generation unit 202 learns a combination of the simulated satellite image D102 and the high-precision land-use map D101 indicated by the training data, thereby generating a trained model for inferring an optimal high-precision land-use map D101 corresponding to the target satellite image DIN3. At this time, the model generation unit 202 may generate the trained model by modifying a pre-trained model. Thereafter, the model generation unit 202 stores the generated trained model in the trained model storage unit 90. As a specific example, the model generation unit 202 uses training data consisting of a combination of the simulated satellite image D102 and a land-use map indicating the land use status in the observation range of the first reference image to learn the correspondence between the simulated satellite image D102 and the land-use map based on machine learning, thereby generating a trained model for inferring a land-use map from the simulated satellite image D102. In this example, the simulated satellite image D102 is an image generated by applying degradation processing to the first reference image, which is the reference high-resolution image DIN1, and has a resolution equivalent to that of the first satellite image. The image having a resolution equivalent to that of the first satellite image may be an image having the same resolution as that of the first satellite image, or may be an image having a resolution close to that of the first satellite image. The first satellite image is a target satellite image DIN3 captured by a target satellite. The land use map in this example corresponds to the high-precision land use map D101. The reference high-resolution image DIN1 in this example has a resolution higher than that of the first satellite image.

[0033] The model generation unit 202 can use, as a learning algorithm, a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning. As an example, a case where a neural network is applied to a trained model will be described here.

[0034] When the model generation unit 202 uses supervised learning, the simulated satellite image D102 and the high-accuracy land-use map D101 indicated by the learning data must be a pair of data containing the same subject. When the model generation unit 202 uses unsupervised learning, the simulated satellite image D102 and the high-accuracy land-use map D101 do not need to contain the same subject.

[0035] As a specific example, the model generation unit 202 learns a high-precision land-use map D101 corresponding to the simulated satellite image D102 by so-called supervised learning in accordance with a neural network model. Here, supervised learning refers to a technique in which a learning device is provided with a set of data indicating an input and data indicating a result corresponding to the input as learning data, thereby learning the features present in the learning data and inferring a result from the input based on the learning result.

[0036] A neural network is composed of an input layer consisting of multiple neurons, an intermediate layer (hidden layer) consisting of multiple neurons, and an output layer consisting of multiple neurons. The number of intermediate layers may be one or more.

[0037] FIG. 5 is a schematic diagram showing an example of a three-layer neural network. As shown in FIG. 5, when a three-layer neural network is used, multiple input values ​​are input to input layer X1 through input layer X3. Each input value is appropriately multiplied by a first weight w1_1 through a first weight w1_6 (hereinafter also referred to as a first weight W1). Calculated values, which are values ​​obtained by appropriately multiplying each input value by the first weight w1_1 through the first weight w1_6, are input to intermediate layer Y1 and intermediate layer Y2. Each calculated value is appropriately multiplied by a second weight w2_1 through a second weight w2_6 (hereinafter also referred to as a second weight W2). Each output value, which is values ​​obtained by appropriately multiplying each calculated value by the second weight w2_1 through the second weight w2_6, is output from output layer Z1 through output layer Z3. Each output value varies depending on the value of the first weight W1 and the value of the second weight W2.

[0038] In this embodiment, the model generation unit 202 learns a trained model for inferring an optimal high-precision land-use map D101 corresponding to the simulated satellite image D102 by so-called supervised learning, in accordance with a combination of the simulated satellite image D102 and the high-precision land-use map D101 indicated by the training data acquired by the data acquisition unit 201. That is, as a specific example, in the training process of the trained model, the model generation unit 202 adjusts the first weight W1 and the second weight W2 of the neural network related to the trained model so that a result output from the output layer in response to input of the simulated satellite image D102 to the input layer approaches the high-precision land-use map D101 as a correct answer.

[0039] 6 is a block diagram showing an example of the configuration of the inference device 30. The inference device 30 includes a data acquisition unit 301 and an inference unit 302. The inference device 30 uses the trained model provided by the learning device 20 to infer an estimated land use map DOUT from a target satellite image DIN3. The target satellite image DIN3 is an image acquired from a target satellite.

[0040] The data acquisition unit 301 has a function of acquiring a target satellite image DIN3. The acquired target satellite image DIN3 is provided to the inference unit 302. It is assumed that the target satellite image DIN3 includes a target subject.

[0041] The inference unit 302 uses the learned model stored in the learned model storage unit 90 to infer an estimated land use map DOUT corresponding to the target subject from the target satellite image DIN3. That is, by inputting the target target satellite image DIN3 to the learned model, the inference unit 302 obtains an estimated land use map DOUT inferred from the input target satellite image DIN3, the estimated land use map DOUT corresponding to the target subject. The estimated land use map DOUT corresponding to a certain target satellite image DIN3 is a land use map that indicates the land use status estimated as the land use status in the observation range of the certain target satellite image DIN3. As a specific example, the inference unit 302 inputs a first satellite image to the learned model to infer a land use map that indicates the land use status in the observation range of the first satellite image.

[0042] 7 shows an example of the hardware configuration of the input image generation device 10 according to this embodiment. The input image generation device 10 is made up of a computer. The input image generation device 10 may also be made up of multiple computers.

[0043] As shown in the figure, the input image generation device 10 is a computer that includes hardware such as a processor 51, a memory 52, an auxiliary storage device 53, an input / output IF (Interface) 54, and a communication device 55. These pieces of hardware are appropriately connected via signal lines 59.

[0044] The processor 51 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of a computer. Specific examples of the processor 51 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The input image generation device 10 may include multiple processors that replace the processor 51. The multiple processors share the role of the processor 51.

[0045] The memory 52 is typically a volatile storage device, and a specific example is RAM (Random Access Memory). The memory 52 is also called a primary storage device or a main memory. Data stored in the memory 52 is saved in the secondary storage device 53 as needed.

[0046] The auxiliary storage device 53 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 53 is loaded into the memory 52 as needed. The memory 52 and the auxiliary storage device 53 may be configured integrally.

[0047] The input / output IF 54 is a port to which an input device and an output device are connected. Specific examples of the input / output IF 54 include a USB (Universal Serial Bus) terminal. Specific examples of the input device include a keyboard and a mouse. Specific examples of the output device include a display.

[0048] The communication device 55 is a receiver and a transmitter, and is specifically a communication chip or a NIC (Network Interface Card).

[0049] Each unit of the input image generating device 10 may use the input / output IF 54 and the communication device 55 as appropriate when communicating with other devices.

[0050] The auxiliary storage device 53 stores a land use classification program. The land use classification program is a program that causes a computer to realize the functions of each unit of the input image generation device 10. The land use classification program is loaded into the memory 52 and executed by the processor 51. The functions of each unit of the input image generation device 10 are realized by software.

[0051] Data used when executing the land use classification program and data obtained by executing the land use classification program are stored in a storage device as appropriate. Each part of the input image generation device 10 uses a storage device as appropriate. Specific examples of the storage device include at least one of a memory 52, an auxiliary storage device 53, a register in the processor 51, and a cache memory in the processor 51. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer. The functions of the memory 52 and the auxiliary storage device 53 may be realized by other storage devices.

[0052] The land use classification program may be recorded on a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The land use classification program may be provided as a program product. The hardware configurations of other devices included in the land use classification system 1 may also be similar to the hardware configuration of the input image generation device 10.

[0053] ***Explanation of Operation*** The operating procedures of each device provided in the land use classification system 1 are collectively referred to as a land use classification method. Also, the programs that realize the operation of each device provided in the land use classification system 1 are collectively referred to as a land use classification program.

[0054] 8 is a flowchart showing an example of a learning process performed by the learning device 20. The process performed by the learning device 20 will be described with reference to FIG.

[0055] (Step S11) The data acquisition unit 201 acquires learning data via the input image generation device 10. The acquired learning data is provided to the model generation unit 202.

[0056] (Step S12) The model generation unit 202 generates a trained model by learning the high-precision land-use map D101, which is an output corresponding to the simulated satellite image D102, through so-called supervised learning based on the combination of the simulated satellite image D102 and the high-precision land-use map D101 indicated by the training data.

[0057] (Step S13) The trained model storage unit 90 stores the generated trained model.

[0058] 9 is a flowchart showing an example of processing by the inference device 30 to infer an estimated land use map DOUT corresponding to the target satellite image DIN3. The processing by the inference device 30 will be described with reference to FIG.

[0059] (Step S21) The data acquisition unit 301 acquires the target satellite image DIN3. The acquired target satellite image DIN3 is provided to the inference unit 302.

[0060] (Step S22) The inference unit 302 inputs the target satellite image DIN3 into the learned model stored in the learned model memory unit 90, and infers an estimated land use map DOUT corresponding to the input target satellite image DIN3.

[0061] (Step S23) The inference device 30 outputs the estimated land use map DOUT generated by the inference.

[0062] ***Explanation of the effects of embodiment 1*** As described above, according to the land use classification system 1 of embodiment 1, with regard to the method for training a land use classification model for optical satellite images, when an actual satellite image is input into the trained model, the land use situation in the observation range of the actual satellite image can be classified with relatively high accuracy.

[0063] ***Other Configurations*** <Modification 1> In the first embodiment, an example has been described in which supervised learning is applied as the learning algorithm used by the model generation unit 202, but the first embodiment is not limited to this example. As a specific example, reinforcement learning, unsupervised learning, semi-supervised learning, or the like may be used as the learning algorithm in addition to supervised learning.

[0064] <Variation 2> The model generation unit 202 may learn an estimated land use map corresponding to a simulated satellite image in accordance with learning data created for a plurality of land use classification systems including the land use classification system 1. Note that the model generation unit 202 may acquire learning data from a plurality of land use classification systems used in the same area, or may learn an estimated land use map corresponding to a simulated satellite image using learning data collected from a plurality of land use classification systems that operate independently in different areas.

[0065] <Variation 3> The model generation unit 202 may add or remove a land use classification system for collecting training data from the targets during the process. Furthermore, the model generation unit 202 may apply a trained model that has trained an estimated land use map corresponding to a simulated satellite image for a certain land use classification system 1 to another land use classification system, and re-train the estimated land use map corresponding to the simulated satellite image for the other land use classification system to update the trained model.

[0066] <Modification 4> Deep learning, which is a learning algorithm that learns to extract feature quantities themselves, may be used as the learning algorithm used in the model generation unit 202. Furthermore, the model generation unit 202 may perform machine learning according to other known methods, such as genetic programming, functional logic programming, or support vector machines.

[0067] <Variation 5> When a target satellite image DIN3 that observes the same area as the observation range of the reference high-resolution image DIN1 is available, the learning device 20 may train a trained model using training data consisting of a pair of a post-registration satellite image D103 and a high-accuracy land-use map D101 corresponding to the reference high-resolution image DIN1. The post-registration satellite image D103 is a satellite image generated by performing registration on the target satellite image DIN3 based on the reference high-resolution image DIN1.

[0068] FIG. 10 shows an example of the configuration of an input image generating device 10b when using such learning data. The input image generating device 10b includes a registration unit 103 instead of the degradation processing unit 102. A target satellite image DIN3 is input to the input image generating device 10b instead of the target satellite information DIN2. The registration unit 103 performs registration between the reference high-resolution image DIN1 and the target satellite image DIN3 to generate a registered satellite image D103. As a specific example, the learning data includes a pair of a second satellite image and a land use map showing the land use status in the observation range of the second reference image. In this case, the registration unit 103 performs registration processing between the second reference image and the second satellite image to generate a registered satellite image D103, which is an image corresponding to the second satellite image. The second reference image is the reference high-resolution image DIN1. The second satellite image is the target satellite image DIN3, which includes observation results for the same range as the observation range of the second reference image. In this example, the trained model is a model generated by learning the correspondence between the second satellite image and the land use map corresponding to the second reference image based on machine learning, and is a model that infers the land use map corresponding to the second reference image from the second satellite image.

[0069] It is desirable that the observation period of the target satellite image DIN3 be close to that of the reference high-resolution image DIN1, but this is not always the case in areas with little change in land use. Furthermore, when generating learning data, it may be necessary to align the reference high-resolution image DIN1 with the target satellite image DIN3. The alignment unit 103 performs the alignment process in such cases.

[0070] <Variation 6> The target satellite image DIN3 may be an image captured by exposure to light rays having wavelengths in the visible light band, or may be an image captured by exposure to light rays having wavelengths in the infrared light band, such as the near-infrared light band or the far-infrared light (thermal infrared) band.

[0071] 11 shows an example of the hardware configuration of the input image generation device 10 according to this modification. The input image generation device 10 includes a processing circuit 58 instead of the processor 51, the processor 51 and memory 52, the processor 51 and auxiliary storage device 53, or the processor 51, memory 52, and auxiliary storage device 53. The processing circuit 58 is hardware that realizes at least a part of the components included in the input image generation device 10. The processing circuit 58 may be dedicated hardware, or may be a processor that executes a program stored in the memory 52.

[0072] When processing circuitry 58 is dedicated hardware, processing circuitry 58 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Input image generation device 10 may include multiple processing circuits that replace processing circuitry 58. The multiple processing circuits share the role of processing circuitry 58.

[0073] In the input image generation device 10, some of the functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0074] The processing circuitry 58 is realized by, for example, hardware, software, firmware, or a combination of these. The processor 51, memory 52, auxiliary storage device 53, and processing circuitry 58 are collectively referred to as the "processing circuitry." In other words, the functions of the functional components of the input image generation device 10 are realized by the processing circuitry. The hardware configurations of other devices included in the land use classification system 1 may also be similar to that of this modified example.

[0075] ***Other Embodiments*** Although the first embodiment has been described, it is also possible to combine multiple parts of this embodiment and implement it. Alternatively, it is also possible to implement this embodiment in part. In addition, this embodiment may be modified in various ways as needed, and may be implemented in any combination, either as a whole or in part. Note that the above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, its applications, and uses. The procedures described using flowcharts, etc. may be modified as appropriate.

[0076] 1 Land use classification system, 10, 10b Input image generation device, 101 Inference unit, 102 Degradation processing unit, 103 Alignment unit, 20 Learning device, 201 Data acquisition unit, 202 Model generation unit, 30 Inference device, 301 Data acquisition unit, 302 Inference unit, 51 Processor, 52 Memory, 53 Auxiliary storage device, 54 Input / output IF, 55 Communication device, 58 Processing circuit, 59 Signal line, 90 Learned model storage unit, DIN1 Reference high-resolution image, DIN2 Target satellite information, DIN3 Target satellite image, D101 High-precision land use map, D102 Simulated satellite image, D103 Aligned satellite image, DOUT Estimated land use map.

Claims

1. A model generation unit generates a trained model that infers the land use map from the simulated satellite image by learning the correspondence between the simulated satellite image and the land use map based on machine learning, using training data consisting of a simulated satellite image, which is an image generated by applying a degradation process to a first reference image, which is a reference high-resolution image having a higher resolution than the resolution of the first satellite image, and an image having a resolution equivalent to the resolution of the first satellite image, and a land use map showing the land use situation in the observation range of the first reference image. A learning device equipped with A land use classification system comprising, The first satellite image is used in the inference process using the trained model. The aforementioned degradation process is a land use classification system that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.

2. A model generated by learning the correspondence between the simulated satellite image and the land use map based on machine learning using training data consisting of a simulated satellite image, which is an image generated by applying a degradation process to a first reference image, which is a reference high-resolution image having a higher resolution than the resolution of the first satellite image, and having a resolution equivalent to the resolution of the first satellite image, and a land use map showing the land use situation in the observation range of the first reference image, and a trained model which is a model that infers the land use map from the simulated satellite image, by inputting the first satellite image into the trained model, which infers the land use map showing the land use situation in the observation range of the first satellite image. An inference device equipped with A land use classification system comprising, The aforementioned degradation process is a land use classification system that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.

3. The land use classification system according to claim 1 or 2, wherein the reference high-resolution image is an image captured using either an aerial photography means or a satellite capable of capturing an image having a resolution higher than the resolution of the target satellite image.

4. The land use classification system according to claim 1 or 2, wherein the degradation process is a process in which downsampling is performed with different offsets for each wavelength band when downsampling the reference high-resolution image.

5. The training data includes a pair of a second satellite image, which is a target satellite image, containing observation results in the same range as the observation range of the second reference image, which is a reference high-resolution image, and a land use map showing the land use situation in the observation range of the second reference image. The aforementioned land use classification system further, A degradation processing unit that performs the aforementioned degradation process, A positioning unit generates a post-aligned satellite image, which is an image corresponding to the second satellite image, by performing a positioning process between the second reference image and the second satellite image. Input image generation device equipped with Equipped with, The land use classification system according to claim 1 or 2, wherein the trained model is a model generated by learning the correspondence between the second satellite image and the land use map corresponding to the second reference image based on machine learning, and is a model that infers the land use map corresponding to the second reference image from the second satellite image.

6. The land use classification system according to claim 1 or 2, wherein the target satellite image is an image captured by exposing it to light rays having wavelengths in the visible light band.

7. The land use classification system according to claim 1 or 2, wherein the target satellite image is an image captured by exposing it to light having wavelengths in the infrared light band.

8. A land use classification method that generates a trained model for inferring the land use map from the simulated satellite image by using training data consisting of a simulated satellite image, which is an image generated by applying a degradation process to a first reference image, which is a high-resolution reference image having a resolution higher than that of a first satellite image, which is a target satellite image captured by a target satellite, and which has a resolution equivalent to that of the first satellite image, and a land use map showing the land use situation in the observation range of the first reference image, and learning the correspondence between the simulated satellite image and the land use map based on machine learning, the method comprising: a computer, which is a learning device, applying a degradation process to a first reference image, which is a high-resolution reference image having a resolution higher than that of a first satellite image, which is a target satellite image captured by a target satellite, and a simulated satellite image having a resolution equivalent to that of the first satellite image; and a land use map showing the land use situation in the observation range of the first reference image. The first satellite image is used in the inference process using the trained model. The aforementioned degradation process is a land use classification method that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.

9. Model generation process: A model that generates a trained model that infers the land use map from the simulated satellite image by learning the correspondence between the simulated satellite image and the land use map based on machine learning, using training data consisting of a simulated satellite image, which is an image generated by applying a degradation process to a first reference image, which is a reference high-resolution image having a higher resolution than the resolution of the first satellite image, and having a resolution equivalent to the resolution of the first satellite image, and a land use map showing the land use situation in the observation range of the first reference image. A land use classification program that causes a computer, which is a learning device, to execute, The first satellite image is used in the inference process using the trained model. The aforementioned degradation process is a land use classification program that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.

10. A model is generated by a computer, which is an inference device, by applying a degradation process to a first reference image, which is a reference high-resolution image having a resolution higher than the resolution of a first satellite image, which is a target satellite image captured by a target satellite, and having a resolution equivalent to the resolution of the first satellite image, and using training data consisting of a combination of the first satellite image and a land use map showing the land use situation in the observation range of the first reference image, the model is generated by learning the correspondence between the simulated satellite image and the land use map based on machine learning, and the first satellite image is input to a trained model, which is a model that infers the land use map from the simulated satellite image, thereby inferring a land use map showing the land use situation in the observation range of the first satellite image, The aforementioned degradation process is a land use classification method that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.

11. An inference process is performed by inputting the first satellite image into a trained model, which is a model that infers the land use map from the simulated satellite image. This model is generated by applying a degradation process to a first reference image, which is a reference high-resolution image having a higher resolution than the resolution of the first satellite image, and has a resolution equivalent to the resolution of the first satellite image. This trained model is generated by learning the correspondence between the simulated satellite image and the land use map based on machine learning. A land use classification program that causes a computer, which is an inference device, to execute, The aforementioned degradation process is a land use classification program that simulates image degradation caused by at least one of the following: the sensor arrangement of the target satellite, the PSF (Point Spread Function) characteristics of the target satellite, the GSD (Ground Sampling Distance) of the target satellite, the sensitivity performance of the target satellite, the noise characteristics of the target satellite, the data compression process during the generation of the target satellite image, and the quantization process during the generation of the target satellite image.