Image providing system, image providing method, program, and information processing device

Abstract

To provide an image providing system capable of responding to further needs of a user regarding images.SOLUTION: An image providing system acquires images obtained by photographing the ground surface from a flying object flying in the atmosphere or out of the atmosphere and stores them. The image providing system determines a coverage factor of a covering object covering the ground surface included in each image of a plurality of stored images. The image providing system searches for images in which the coverage factor satisfies a prescribed condition and extracts them, rearranges them according to the coverage factors, and outputs them.SELECTED DRAWING: Figure 12

Description

Technical Field

[0004] , , , ,

[0005] , , ,

[0003] , , , , <00​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​However, the method described in Patent Document 1 only colors the satellite images of each region in two colors, and does not show the ranking of coverage rates. Therefore, it is not suitable for surveys where ranking of coverage rates is necessary. For example, when estimating the yield of crops in a field, the proportion of cover (e.g., fields) in the satellite image would be useful. If it were possible to search for satellite images of areas suitable for agriculture (e.g., areas with low cloud coverage) based on the coverage rate of cover, it would be useful in determining the ranking of agricultural investments, such as selecting areas suitable for specific crops. Although agriculture has been introduced here, knowing the ranking of the coverage rate of specific cover would often be useful in other fields such as fishing and insurance. In these applications, it is necessary to sort the search results of satellite images according to the coverage rate of cover and present them to the user. Therefore, the present invention aims to provide an image provision system that can meet the further needs of users regarding images. [Means for solving the problem]

[0006] The present invention includes, for example, an acquisition unit that acquires an image obtained by imaging the Earth's surface from a flying object flying within or outside the atmosphere, A storage unit that stores multiple images acquired by the acquisition unit, each from a different region or at a different time of shooting, A determination unit determines the coverage rate of the covering material covering the ground surface, which is visible in each of the multiple images stored in the aforementioned memory unit. A search unit searches for and extracts images from among the plurality of images stored in the storage unit in which the coverage rate satisfies specific conditions, A sorting unit that rearranges a plurality of images extracted by the search unit according to the coverage rate, The present invention provides an image provisioning system having an output unit that outputs a plurality of images rearranged according to the coverage ratio. [Effects of the Invention]

[0007] According to the present invention, an image provisioning system is provided that can meet the further needs of users regarding images. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram illustrating an example of an image archiving system configuration. [Figure 2] This block shows an example of a server configuration. [Figure 3] This is an explanatory diagram showing an example of a table design for an image database. [Figure 4] This is an example of a table design for a geographics database. [Figure 5] This flowchart shows an example of the process of registering satellite images into a database. [Figure 6] This flowchart shows an example of latitude and longitude detection processing in satellite images using a latitude and longitude detector. [Figure 7] This is a flowchart showing an example of the domain learning process for a domain classifier. [Figure 8] This is a flowchart illustrating an example of the processing procedure for domain classification. [Figure 9] This is a flowchart illustrating an example of the learning process for an object detector. [Figure 10] This flowchart shows an example of the processing steps for object detection. [Figure 11] This flowchart shows an example of how to provide images. [Figure 12] This is an example of a user interface displayed on a terminal for users to search for desired satellite imagery. [Figure 13] This is an example of a search result displayed after a user specifies their desired satellite image search criteria. [Figure 14] This is an example of a user interface for searching satellite imagery by allowing users to enter their desired keywords. [Figure 15] This is a diagram illustrating the functions of the control unit. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of the features described in the embodiments are essential for the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Also, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] [Satellite Image Archive System (Image Providing System)] Hereinafter, some embodiments of the present invention will be described in detail. FIG. 1 shows a configuration example of a satellite image archive system. In the form of this embodiment, a satellite image archive system that can acquire, store, and search satellite images desired by a user will be described. The satellite image archive system includes an information processing device 1, an antenna 2, a satellite 3, and a terminal 4. The information processing device 1 and the terminal 4 are communicably connected via a network N such as the Internet. In the form of this embodiment, for the sake of simplicity of explanation, the satellite image archive system will be described as including the information processing device 1, the antenna 2, the satellite 3, and the terminal 4. However, the configuration of the satellite image archive system is not limited to such a configuration. For example, the antenna 2, the satellite 3, and the terminal 4 may be provided by another system.

[0011] Information processing device 1 is an information processing device capable of various information processing and information transmission and reception. For example, information processing device 1 may be a server device, a personal computer, a virtual machine, or a computer that provides cloud services on the Internet. In this embodiment, dozens to hundreds of artificial satellites 3 may be in orbit around the Earth in a constellation. The weight of the artificial satellites 3 may be 100 kg or less. The artificial satellites 3 transmit satellite images generated by photographing the Earth's surface to antenna 2. Information processing by information processing device 1 includes, for example, latitude and longitude detection processing of satellite images acquired by artificial satellites 3, area classification processing, object detection processing, and preview thumbnail generation processing. Latitude and longitude detection processing is the process of detecting the latitude and longitude of the Earth's surface captured in the satellite image. Area classification processing is the process of classifying areas of the Earth's surface captured in the satellite image. For example, it is the process of classifying cover materials that cover the Earth's surface (e.g., clouds, seas, lakes, salt lakes, fields, mountainous areas, forests, deserts, rice paddies, rivers, snow-capped mountains, etc.). Object detection processing is the process of detecting objects (e.g., aircraft, vehicles, ships, etc.) that appear in satellite images. Preview thumbnail generation processing is the process of generating thumbnail images of satellite images. Information processing device 1 registers the information processing results in a database. When information processing device 1 receives the user's desired satellite image search criteria from the web browser of terminal 4, it extracts satellite images that match the search criteria from the database and sends the extraction results to terminal 4. Search criteria include, for example, geographical area, date and time of capture, and others. In this way, information processing device 1 plays a central role in the image archiving system (image provision system).

[0012] Antenna 2 is a receiving device that receives satellite images transmitted from satellite 3 and housekeeping data (HK data), which is data representing the operating status of satellite 3 itself. Antenna 2 transfers the satellite images and HK data received from satellite 3 to the information processing device 1. Antenna 2 may also have a function to transmit instructions to satellite 3 to photograph the Earth's surface.

[0013] The artificial satellite 3 has at least one of an optical sensor (e.g., a CCD image sensor, a CMOS image sensor) and a microwave sensor (e.g., a synthetic aperture radar) as an imaging device capable of capturing images. CCD is an abbreviation for Charge-Coupled Device. CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor. Here, the satellite image may be a still image or a moving image (video).

[0014] The artificial satellite 3 is a circumferential satellite that observes the earth's surface from outside the atmosphere such as in space or within the atmosphere. However, the artificial satellite 3 may be any flying object that flies in space or in the atmosphere, such as a drone, an aircraft, or a helicopter. Therefore, the flying object is not limited to an artificial satellite. In this embodiment, for the sake of convenience in explanation, the satellite image refers to an image acquired by such a flying object.

[0015] When the information processing device 1 provides a satellite image to a user, it may also provide the user with the classification result of regions and the detection result of objects. In this case, the information processing device 1 may use a region classifier and an object detector generated by machine learning. The region classifier may classify regions of covering objects that cover the earth's surface (e.g., clouds, water, sand, trees, asphalt, etc.). The object detector detects objects existing on the earth's surface (e.g., aircraft, vehicles, ships, etc.). The information processing device 1 may provide a satellite image with metadata indicating the area (area or coverage rate) of the region of the covering object classified by the region classifier added thereto. Further, the information processing device 1 may provide a satellite image with metadata indicating the name and the number of the objects detected by the object detector added thereto.

[0016] The terminal 4 is a terminal device (e.g., a PC, a smartphone, a tablet terminal, etc.) operated by a user. PC is an abbreviation for Personal Computer. However, the terminal 4 may be any computer equipped with a user interface such as a web browser.

[0017] The user may use terminal 4 to specify the target period as a search criterion for satellite imagery. The information processing device 1 searches for satellite images from among multiple satellite images whose acquisition date and time fall within the target period. The search criteria may be metadata such as geographical area (latitude and longitude), acquisition date and time (acquisition period), type of cover, coverage rate, type of object, and number of objects. The search criteria may also be any keywords.

[0018] [Information Processing Device] Figure 2 is a block diagram showing an example configuration of the information processing device 1. The information processing device 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 has one or more arithmetic processing units such as CPUs, MPUs, GPUs, and FPGAs, and reads and executes programs P stored in the auxiliary memory unit 14. This enables the execution of various information processing tasks. CPU is an abbreviation for Central Processing Unit. MPU is an abbreviation for Microprocessing Unit. GPU is an abbreviation for Graphical Processing Unit. FPGA is an abbreviation for Field Programmable Gate Array. The main memory unit 12 is a temporary storage area such as SRAM or DRAM, which temporarily stores programs and data necessary for the control unit 11 to perform arithmetic processing. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The communication unit 13 has at least one of a wireless communication circuit or a wired communication circuit, and is used by the information processing device 1 to send and receive information with external devices.

[0019] The auxiliary storage unit 14 is a non-volatile storage device such as a hard disk, SSD (solid-state drive), or flash memory. The auxiliary storage unit 14 permanently records the program P required by the control unit 11 for calculation processing, as well as other data. The auxiliary storage unit 14 stores the image DB 141, image pool 142, region classifier 143, object detector 144, latitude and longitude detector 145, thumbnail generator 146, and feature DB 147. The region classifier 143, object detector 144, latitude and longitude detector 145, and thumbnail generator 146 are collections of computer programs and data. The region classifier 143, object detector 144, latitude and longitude detector 145, and thumbnail generator 146 may also be functions implemented by the control unit 11 according to program P. In the latter case, the region classifier 143, object detector 144, latitude and longitude detector 145, and thumbnail generator 146 become part of the control unit 11.

[0020] Image DB141 is a database that stores metadata associated with satellite images acquired from artificial satellite 3. Image pool 142 is a storage area that stores satellite images associated with each record in Image DB141.

[0021] The region classifier 143 is a classifier that classifies the cover of the ground surface. The region classifier 143 may be, for example, a trained model generated using training data by machine learning.

[0022] The object detector 144 is an object detector that detects objects present on the Earth's surface. The object detector 144 may be, for example, a trained model generated using training data through machine learning, similar to the region classifier 143.

[0023] The latitude and longitude detector 145 determines the latitude and longitude of the center and the four corners of the satellite image. For example, the latitude and longitude detector 145 may determine these latitudes and longitudes by comparing the features of the satellite image observed by artificial satellite 3 with the features of the satellite image that serve as training data for which the latitude and longitude are known.

[0024] The thumbnail generator 146 compresses the data size of satellite images observed by artificial satellite 3 and generates satellite images (thumbnail images) for display on the preview screen. The compression ratio may be, for example, about 1 / 4 to 1 / 16 of the original satellite image. The size of the thumbnail image should be such that it can sufficiently reduce the amount of data transmitted when displayed on the preview screen of a web browser on terminal 4, etc., while still allowing the user to recognize the characteristics and appearance of the image.

[0025] [Database] Figure 3 shows an example of the table design for the image DB141. The image DB141 may have, for example, an image master table T1, a coverage amount table T2, a coverage position table T3, an object count table T4, and an object position table T5.

[0026] The image master table T1 may include columns for image ID, image file name, shooting date and time, image center latitude, image center longitude, top-left latitude, top-left longitude, top-right latitude, top-right longitude, bottom-left latitude, bottom-left longitude, bottom-right latitude, bottom-right longitude, target latitude, target longitude, address, and tags.

[0027] The Image ID column stores a unique identifier (Image ID) used to identify satellite images. Image IDs are also stored in the Image ID columns of the Coverage Table T2, Coverage Location Table T3, Object Count Table T4, and Object Location Table T5. Thus, multiple tables are linked by the Image ID.

[0028] The image file name column stores the file names of satellite images stored in the image pool 142. The date and time of capture column stores the date and time when satellite 3 captured images of the Earth's surface. The date and time may be expressed in Coordinated Universal Time (UTC). The image center latitude column and image center longitude column indicate the latitude and longitude of the center of the satellite image, respectively. The top-left latitude column, top-left longitude column, top-right latitude column, top-right longitude column, bottom-left latitude column, bottom-left longitude column, bottom-right latitude column, and bottom-right longitude column indicate the latitude or longitude of the four corners of the satellite image, respectively. These latitudes and longitudes are detected by the latitude and longitude detector 145. The target latitude column and target longitude column indicate the latitude and longitude of the target location or object that satellite 3 targets when capturing images of the Earth's surface. The address column indicates the address obtained by applying reverse geocoding to the latitude and longitude of the center of the satellite image. Reverse geocoding is a method of obtaining an address from latitude and longitude. The tag column indicates the names of geographical features (e.g., facility names, mountain names, river names, etc.) contained within a rectangular geographical area determined by the latitude and longitude of the four corners of the satellite image. The feature names and their corresponding latitude and longitude are stored in Feature DB147. Here, "feature" is a concept that represents all things on the ground, whether natural or artificial, such as buildings, roads, and rivers, and "feature name" is the name of that feature.

[0029] In the image master table T1, columns representing latitude and longitude may be defined together as spatial data types, similar to geometry or geography types in geographic information systems. Geometry types are data types that model the Earth as a plane, while geography types model the Earth as a sphere. These data types include points, linestrings, and polygons. For example, the latitude and longitude of the image center and the latitude and longitude of the target image may be defined as points, while the latitude and longitude of the four corners of the image may be defined as polygons. By defining positional data such as latitude and longitude as spatial data types in the table design in this way, it becomes possible to query the database using spatial operations.

[0030] Spatial operations on spatial data allow, for example, to search for the image center latitude and longitude (point) of a satellite image contained within a selected rectangular area (polygon) on a map, or to search for features such as rivers (linestrings) contained within the areas (polygons) at the four corners of a satellite image. By using spatial operations in this way, more sophisticated queries to databases can be achieved. Furthermore, by creating spatial indexes on columns of spatial data types in tables, queries to spatial data can be made faster.

[0031] The cover amount table T2 includes, for example, columns for image ID, cloud cover, field cover, water cover, sand cover, forest cover, and road cover. The image ID column shows the image identifier associated with the image ID column of the image master table T1. The cloud cover, field cover, water cover, sand cover, forest cover, and road cover columns each show the coverage percentage of clouds, fields, water, sand, forests, and roads in the satellite image classified by the region classifier 143. These coverage percentages can be converted to the area of ​​the ground surface covered by the cover, based on the resolution of the satellite image.

[0032] The cover location table T3 includes columns for image ID, cloud cover location, field cover location, water cover location, sand cover location, forest cover location, and road cover location. The image ID column indicates the image identifier associated with the image ID column of the image master table T1. The cloud cover location column, field cover location column, water cover location column, sand cover location column, forest cover location column, and road cover location column each indicate the pixel location of the cover in the satellite image classified by the region classifier 143. The control unit 11 may overlay the regions (covers) identified by these cover locations onto the corresponding locations in the satellite image.

[0033] The object count table T4 includes an image ID column, an aircraft count column, a vehicle count column, and a ship count column. The image ID column shows the image identifier associated with the image ID column of the image master table T1. The aircraft count column, vehicle count column, and ship count column each show the number of objects detected by the object detector 144.

[0034] The object position table T5 includes an image ID column, an aircraft position column, a vehicle position column, and a ship position column. The image ID column indicates the image identifier associated with the image ID column of the image master table T1. The aircraft position column, vehicle position column, and ship position column each indicate the pixel position of an object detected by the object detector 144. The control unit 11 can highlight and display the detected objects in the satellite image by overlaying rectangles around the positions of these objects onto the satellite image.

[0035] Figure 4 shows an example of the table design for the feature database T147. The feature database T6 has a feature table T6. The feature table T6 includes feature name columns, latitude columns, and longitude columns. The feature name column stores various feature names, such as facility names, mountain names, and river names. The latitude and longitude columns store the latitude and longitude indicating the location of the facility, mountain, or river identified by the feature name. Feature names contained within the rectangular areas of the four corners of the image in the image master table T1 can be searched using the latitude and longitude columns of the feature table T6, and the feature names found in the search can be associated with the satellite image as tags.

[0036] In the feature table T6, the latitude and longitude columns may also be defined together as a spatial data type. By defining latitude and longitude together as a spatial data type, it becomes possible to search for the location (point) of a feature contained within a rectangular area (polygon). Defining locations such as latitude and longitude as spatial data types in this way allows for more advanced queries to the database using spatial calculations.

[0037] In this embodiment, only one information processing device 1 is shown, but it is not limited to this, and it may be configured to distribute the load across multiple devices. For example, the image DB 141, image pool 142, and feature DB 147 may be provided on separate devices.

[0038] [flowchart] ● Registration of satellite images Figure 5 is a flowchart illustrating an example of the process of registering satellite images into a database. The process of registering satellite images taken by artificial satellite 3 into image DB141 will be explained based on the procedure in Figure 5.

[0039] In step S501, the control unit 11 of the information processing device 1 acquires satellite images from the artificial satellite 3. If multiple satellite images have already been acquired and stored in the image pool 142, the acquisition process in step S501 becomes a read process from the image pool 142 to the work area of ​​the main memory unit 12.

[0040] In step S502, the control unit 11 determines whether the satellite image is a valid image. For example, if the satellite image is overexposed or underexposed, the control unit 11 determines that the satellite image is not a valid image. The control unit 11 may also determine the validity of the satellite image based on whether the image format of the satellite image is normal. For example, the control unit 11 may determine the validity of the satellite image based on whether the image format of the satellite image is corrupted due to a communication error or the like. This may also be determined based on error detection results based on CRC (Cyclic Redundancy Code). If the satellite image is not valid, the control unit 11 proceeds to step S510 and isolates the satellite image in the isolation area of ​​the image pool 142. On the other hand, if the satellite image is a valid image, the control unit 11 proceeds to step S503.

[0041] In step S503, the control unit 11 performs the process of acquiring the latitude and longitude of the satellite image. This acquires the latitude and longitude of the center and the four corners of the satellite image. Details of the latitude and longitude acquisition process will be described later with reference to Figure 6.

[0042] In step S504, the control unit 11 performs region classification processing on the satellite image. This obtains classification results for multiple regions that make up the satellite image (such as the type of cover present in each region and its coverage rate or area). Details of the region classification processing will be described later with reference to Figure 8. The region classification processing may be performed, for example, by the region classifier 143.

[0043] In step S505, the control unit 11 performs object detection processing on the satellite image. This obtains the type and number of objects contained in the satellite image. Details of the object detection processing will be described later with reference to Figure 10. Note that the object detection processing may also be performed using the object detector 144.

[0044] In step S506, the control unit 11 obtains the addresses of geographic locations captured in the satellite image based on the latitude and longitude of the center and four corners of the satellite image. For example, the control unit 11 may obtain the addresses from the latitude and longitude by reverse geocoding. This obtains the address of the center of the satellite image and the addresses of each of the four corners. These addresses become part of the metadata.

[0045] In step S507, the control unit 11 tags the satellite image with the names of features contained within a rectangular area identified by the latitude and longitude of the four corners of the satellite image. For example, the control unit 11 identifies a rectangular latitude and longitude range determined by the four corners of the satellite image based on the latitude and longitude of the four corners of the satellite image. The control unit 11 searches the feature DB 147 for the names of features contained within the identified latitude and longitude range. The feature names of the records that hit in the search are obtained as tags. The feature names become part of the metadata. The feature names may also be names of landmarks (famous facilities or buildings).

[0046] In step S508, the control unit 11 generates a thumbnail image of the satellite image. For example, the thumbnail image may be generated by a thumbnail generator 146.

[0047] In step S509, the control unit 11 registers metadata for the satellite image in the image database 141. The metadata includes the latitude and longitude of the center and four corners of the satellite image, area classification results, object detection results, address, and tags (feature names). Off-nadia angle and satellite image azimuth angle may also be registered in the image database 141 as metadata. The off-nadia angle and satellite image azimuth angle may be calculated by the control unit 11 from HK data, for example.

[0048] ● Latitude and longitude detection process Figure 6 is a flowchart showing an example of the latitude and longitude detection process (S503) of satellite images using the latitude and longitude detector 145. Based on Figure 6, the process of obtaining the positional information (latitude and longitude) of satellite images observed by satellite 3 using the latitude and longitude detector 145 will be explained.

[0049] In step S601, the control unit 11 obtains the latitude and longitude of the satellite image target being processed for latitude and longitude detection from HK data or the like.

[0050] In step S602, the control unit 11 acquires training images of the area surrounding the target to be photographed. Training images are, for example, pre-photographed satellite images in which the latitude and longitude of all pixels are known. In this case, a large number of satellite images with latitudes and longitudes close to those of the target to be photographed may be acquired from the image pool 142.

[0051] In step S603, the control unit 11 performs matching between the feature quantities of the satellite image to be processed and the feature quantities of multiple training images. This determines the latitude and longitude of the center and the four corners of the satellite image to be processed. At this time, the control unit 11 may calculate an evaluation value indicating the degree of matching between the feature quantities of the satellite image to be processed and the feature quantities of multiple training images, and adopt the latitude and longitude of the center and the four corners of the training image with the highest evaluation value as the latitude and longitude of the center and the four corners of the image to be processed. Alternatively, the control unit 11 may identify multiple training images with high evaluation values, weight the latitude and longitude of the center and the four corners of the multiple training images according to the evaluation value of those training images, and calculate the weighted average value of the latitude and longitude of the center and the weighted average value of the latitude and longitude of the four corners. If the evaluation value is below a threshold, the control unit 11 may determine that the matching has failed and skip the calculation of latitude and longitude.

[0052] In step S604, the control unit 11 determines whether the matching was successful. This determination corresponds to determining whether the latitude and longitude of the center and the four corners of the satellite image to be processed were successfully detected. If the matching fails because the evaluation value is lower than the threshold, the control unit 11 proceeds to step S610 and isolates the satellite image to be processed. On the other hand, if the matching is successful, the control unit 11 proceeds to step S605 and registers the latitude and longitude of the center and the four corners of the satellite image in the image DB 141. This registration process may also be performed in step S509 as described above.

[0053] If the detection of latitude and longitude in the satellite image fails, the control unit 11 may switch to a slower but more precise algorithm than the one currently in use and perform the latitude and longitude detection process again.

[0054] ● Domain learning processing (learning processing for coverings) Figure 7 is a flowchart showing an example of the domain learning process for the domain classifier 143. Based on Figure 7, the process of generating the domain classifier 143 using machine learning will be explained. The domain learning process is assumed to be executed before the process shown in Figure 5 is executed.

[0055] In step S701, the control unit 11 acquires training data by associating the correct values ​​of the classification results obtained by classifying each covering area contained within the satellite image with the satellite image. Here, the covering areas (coverings) to be classified include, for example, clouds, water, sand, fields, roads, etc.

[0056] In step S702, the control unit 11 generates a region classifier 143 by training it with satellite images included in the training data. The machine learning algorithm used to train the training data may be deep learning, support vector machines, or other methods.

[0057] ● Domain classification processing Figure 8 is a flowchart showing an example of the processing procedure for region classification (S504). Based on Figure 8, the process of classifying various regions included in satellite images using the region classifier 143 will be explained.

[0058] In step S801, the control unit 11 acquires satellite images taken by the artificial satellite 3 and divides the satellite images into multiple regions. The control unit 11 may also store the latitude and longitude of the center or four corners of each region, associated with each region, in order to manage the position of each region. The position information of each region is used as the coverage position in the coverage position table T3.

[0059] In step S802, the control unit 11 inputs each of the multiple regions divided from the satellite image into the region classifier 143 and classifies each region. For example, the type of covering material covering the Earth's surface in each region is identified.

[0060] In step S803, the control unit 11 determines the proportion of each cover in the satellite image (e.g., coverage rate, area of ​​the Earth's surface covered by the cover) based on the classification results obtained by the region classifier 143. This proportion is used as the coverage amount in the coverage amount table T2.

[0061] In step S804, the control unit 11 registers the type of cover and its percentage (e.g., coverage rate, area of ​​the ground surface covered by the cover) in the image database 141. As shown in Figure 3, based on the type of cover, the percentage of the cover is registered in the coverage amount column of the coverage amount table T2, and the location of the cover (latitude and longitude of the region) is registered as the cover location in the cover location table T3. For example, if the satellite image with image ID=1 is the target of processing, and the cover is cloud, the percentage of clouds in the satellite image is registered in the cloud coverage amount column of the cover amount table T2 for the record with image ID=1. Similarly, the latitude and longitude indicating the location of the clouds in the satellite image are registered in the cloud cover location in the cover location table T3 for the record with image ID=1.

[0062] ● Generation of object detectors (learning process) Figure 9 is a flowchart showing an example of the learning process for the object detector 144. Based on Figure 9, the process of generating the object detector 144 using machine learning will be explained.

[0063] In step S901, the control unit 11 acquires training data by associating the correct values ​​of detection results for each object contained in the satellite image with the satellite image. Examples of objects to be detected include automobiles, aircraft, and ships. In this way, training data is obtained in which the correct values ​​of each object are assigned to the satellite image.

[0064] In step S902, the control unit 11 generates the object detector 144 by inputting a sufficient amount of training data into the object detector 144. As a result, when satellite images taken by the artificial satellite 3 are input to the object detector 144, the object detector 144 can output the type and number of objects present in the satellite images. The machine learning algorithm used with the training data may be deep learning, support vector machines, or any other type.

[0065] ● Object detection processing Figure 10 is a flowchart showing an example of the processing procedure for object detection (S505). Based on Figure 10, the process of detecting various objects included in satellite images using the object detector 144 will be explained.

[0066] In step S1001, the control unit 11 acquires satellite images generated by the artificial satellite 3 taking pictures of the Earth's surface from the image pool 142, and divides the acquired satellite images into multiple regions.

[0067] In step S1002, the control unit 11 inputs multiple regions extracted from the satellite image one by one into the object detector 144 and detects objects present in each region. This allows the type and location of the objects captured in the satellite image to be determined.

[0068] In step S1003, the control unit 11 counts the number of objects included in the satellite image based on the detection results obtained by the object detector 144. The object count may also be performed by the object detector 144. In this way, the number of objects of each type is determined.

[0069] In step S1004, the control unit 11 registers the type and number of objects obtained from the satellite image to be processed in the image DB 141. As shown in Figure 3, the column in the object count table T4 is specified according to the type of object. For example, if the image ID of the satellite image is 1 and the type of object is an aircraft, the number of aircraft is registered in the aircraft count column of the record for image ID=1 in the object count table T4. Similarly, the aircraft's position (latitude and longitude) is registered in the aircraft position column of the record for image ID=1 in the object position table T5.

[0070] ● Satellite image provision processing Figure 11 is a flowchart illustrating the process of providing satellite imagery. Here, the process of providing satellite imagery is exemplified as a Web service, but it may also be implemented as a communication service that provides satellite imagery according to other communication protocols.

[0071] In step S1101, the control unit 11 receives input for the search criteria and sorting rules for satellite images transmitted from the terminal 4. The sorting rules are rules for rearranging the order of multiple satellite images included in the search results.

[0072] Figure 12 shows an example of a user interface 1200 displayed on terminal 4 for searching for a desired satellite image from satellite images stored in image pool 142 using image DB 141. The pointer 1202 moves in response to the operation of the pointing device on terminal 4. The pointing device is, for example, a mouse or a touch panel sensor (a panel-type sensor that detects touch). In this example, a rectangular search area 1203 is set on a portion of the wide-area satellite image 1201 displayed on the user interface 1200 by dragging the pointer 1202. This selects or narrows down the satellite image to be searched, thus reducing search time. It will also improve the hit rate of the satellite image desired by the user. Note that a keyboard may be used as an input device along with the pointing device, or in place of the pointing device. Furthermore, the wide-area satellite image 1201 may be moved east, west, south, or north by operating the pointer 1202. The control unit 11 reads satellite images corresponding to the latitude and longitude in response to the operation of the pointer 1202 from the image pool 142 and seamlessly displays wide-area satellite images 1201 on the user interface 1200 of the terminal 4.

[0073] The display tool 1230 is a control object that instructs the control unit 11 to zoom in (+) or zoom out (-) of the wide-area satellite image 1201. The wide-area satellite image 1201 may be a map image instead of a satellite image. Alternatively, an image of a globe may be used as the wide-area satellite image 1201. Furthermore, the wide-area satellite image 1201 may be a two-dimensional map or a three-dimensional map.

[0074] Figure 12 illustrates a rectangular search area 1203, but the shape of the search area 1203 may be other shapes. For example, the shape of the search area 1203 may be a polygon, a circle, an ellipse, or any shape drawn or selected by the user. Furthermore, an upper limit may be placed on the size (area) of the search area 1203. This would help improve search speed. In addition, multiple search areas 1203 may be selected simultaneously.

[0075] The search box 1210 is a user interface for receiving desired search criteria entered by the user. The date specification unit 1211 receives the date and time the satellite image to be searched was taken. As shown in Figure 12, the date specification unit 1211 may also receive a specification of a shooting period defined by a start date and an end date. The sorting rule specification unit 1213 receives a specification of whether to sort the satellite images taken within the shooting period specified by the date specification unit 1211 in ascending order or descending order by shooting date.

[0076] The area / object designation unit 1214 accepts the designation of the type of covering or the type of object. The area / object designation unit 1214 may also display candidate coverings classified by the area classifier 143 and candidate objects detected by the object detector 144, and display the covering or object selected by the user from these candidates. The sorting rule designation unit 1215 accepts the designation of the sorting rule for the detection results of the covering or object selected by the user. In this example, it is possible to select whether to sort the detection results in ascending order or descending order. The threshold setting unit 1216 accepts the designation of a threshold for the covering percentage or the number of objects. The search button 1217 is a button for commanding the terminal 4 to send search conditions, including the latitude and longitude of the four corners of the search area 1203, and an instruction to execute the search to the information processing device 1.

[0077] Here, two lines are displayed as the region / object designation section 1214, but the same number of lines as the number of objects and coverings detected in advance when the search region 1203 is specified may be displayed. Also, the order of each line may be determined based on the coverage rate or number. For example, if it is known in advance that the cloud coverage rate in the search region 1203 is greater than the field coverage rate, the cloud line may be displayed above the field line.

[0078] In S1102, the control unit 11 searches for satellite images based on the search conditions received from the terminal 4. Satellite images are stored in the image pool 142, associated with metadata such as the image ID stored in the image DB 141, the date and time of capture, the operational history data (HK data) of the artificial satellite 3, the size (coverage rate, area) of each region (coverage) classified by the region classifier 143, the number of various objects detected by the object detector 144, and addresses obtained by reverse geocoding and tags associated with the feature DB 147. The control unit 11 extracts the image ID of the satellite image that matches the search conditions from the image DB 141, and further extracts the image file of the satellite image corresponding to the image ID from the image pool 142. The control unit 11 states that the satellite image corresponding to the extracted image ID may be the original satellite image or a thumbnail image created from the satellite image.

[0079] In S1103, the control unit 11 sorts the multiple search results extracted by the search (e.g., satellite images (thumbnail images) and their metadata) according to the sorting rules included in the search conditions. In other words, the display order of the search results, such as satellite images, displayed in the web browser of terminal 4 is rearranged.

[0080] In S1104, the control unit 11 provides the sorted search results to the terminal 4. For example, information about the web page displaying the search results is sent to the terminal 4. The terminal 4 displays the search results received from the information processing device 1 on its screen.

[0081] Figure 13 illustrates a popup 1300 that displays search results. The popup 1300 is a display element such as a popup window. The hit count display unit 1302 shows the number of satellite images extracted by the search. The result display unit 1301 displays each search result. For example, the result display unit 1301 includes a thumbnail image, the date of capture, the latitude and longitude of the center, and the coverage rate of the cover selected by the user. This popup 1300 consists of three pages, and the page to be displayed is switched via the page switching unit 1303. Also, five search results are displayed per page, but this is just an example. Furthermore, the 12 search results are sorted according to the sorting rules specified by the user.

[0082] Incidentally, it is likely that users are most interested in satellite images that are closer to the center of the selected rectangular search area 1203. Therefore, when the control unit 11 generates search results in response to the user's search request, it may rank the search results higher if the latitude and longitude of the center of the satellite image is close to the center of the selected rectangular search area 1203 on the map.

[0083] In the popup 1300, search results sorted by clouds may be displayed first, followed by search results sorted by fields. Whether the search results for clouds or fields are displayed first may be linked to which is displayed higher in the area / object specification section 1214.

[0084] In the threshold setting unit 1216 illustrated in Figure 12, the threshold set for a region (covering) may, for example, specify a lower limit and an upper limit for the coverage rate. In Figure 12, the lower limit for the coverage rate is set to 0%, and the upper limit is set to 10%. Therefore, in this example, satellite images with cloud coverage rates of 0% or more and 10% or less will be extracted based on the image DB 141.

[0085] In the example in Figure 12, the shooting period is set to 2021 / 01 / 02 to 2021 / 01 / 05. The display order for the shooting date is set to descending. The display order for areas where the cover is cloud is set to ascending. The threshold for cloud cover is set to 0% or more and 10% or less. The display order for areas where the cover is field is set to descending. The threshold for field cover is set to 70% or more and 100% or less. Therefore, in the example of the search results in Figure 13, thumbnail images and their metadata are displayed for satellite images where the cloud cover is 0% or more and 10% or less, and the field cover is 70% or more. In addition, the shooting date is sorted in descending order. The cloud cover is sorted in ascending order. The field cover is sorted in descending order. In other words, within the specified shooting period, satellite images that are clear and contain a lot of fields are displayed in order from the most recent shooting date. This would, for example, allow for the precise extraction of satellite imagery necessary to predict crop yields from fields.

[0086] When a thumbnail image is selected in the popup 1300, the image ID associated with the thumbnail image is sent from terminal 4 to the information processing device 1. The control unit 11 sends the download page for the satellite image associated with the terminal ID to terminal 4. When authentication information is entered on the download page displayed in terminal 4, the control unit 11 may perform user login authentication. If login authentication is successful, the control unit 11 reads the satellite image corresponding to the image ID from the image pool 142 and sends it to terminal 4. Terminal 4 displays the received satellite image. This allows the user to check the details of the satellite image. When downloading satellite images, the control unit 11 may charge the user on a pay-per-download basis or on a subscription basis.

[0087] The control unit 11 may automatically save the search conditions associated with the user each time a search is performed, and display the same search conditions as the previous search conditions as the default search conditions on the user interface 1200 for subsequent searches.

[0088] Figure 14 shows a keyword input unit 1400 on terminal 4 that accepts the user's desired keyword input. When the search button 1217 is pressed, terminal 4 sends a search request to the information processing device 1 that includes the keyword entered in the keyword input unit 1400 and the latitude and longitude of the search area 1203.

[0089] The keyword input section 1400 may contain any keywords related to local features, such as addresses, place names, facility names, or river names.

[0090] The control unit 11 performs a full-text search of the address column and tag column of the image master table T1 in accordance with the search request received from the terminal 4, and transmits the search results, which include satellite images corresponding to the hit keywords and their metadata, to the terminal 4.

[0091] To speed up full-text search, an inverted index using N-grams or similar methods may be adopted for the image master table T1. In addition to full-text search by keywords (e.g., address, tags), other search conditions such as shooting period and latitude / longitude may be added. Search conditions such as ascending or descending sort by shooting date may also be added.

[0092] Furthermore, satellite imagery presents a unique problem: depending on the satellite's attitude and orbit, the center of the target satellite image may differ. For example, when targeting Tokyo Tower, depending on the date and time, Tokyo Tower may be displayed in the center or shifted to the right. By rearranging the satellite images according to their coverage and outputting them to terminal 4, users can visually grasp the desired satellite image, improving user convenience.

[0093] <Technical concepts derived from examples> [Perspective 1] Figure 15 shows the functions realized by the control unit 11 executing program P, etc. The acquisition unit 1501 acquires images obtained by imaging the Earth's surface from a flying object flying within or outside the atmosphere. For example, the acquisition unit 1501 may receive satellite images transmitted from the artificial satellite 3 and transferred via the antenna 2 via the communication unit 13. The satellite images are passed to the region classifier 143, object detector 144, latitude and longitude detector 145, and thumbnail generator 146 described above. The satellite images may also be stored in the image pool 142. In other words, the auxiliary storage unit 14 and the image pool 142 function as storage units that store multiple images acquired by the acquisition unit, each from a different region and / or at a different time of acquisition. The region classifier 143 and the coverage determination unit 1502 determine the coverage rate of the covering material covering the Earth's surface that is captured in each of the multiple images stored in the storage units. The counter 1503 counts the number of objects detected from the satellite images by the object detector 144. The registration unit 1504 assigns an image ID to the satellite image and registers information such as the coverage rate of the cover, the number of objects, latitude and longitude, and thumbnail images in the image database 141.

[0094] The control unit 11 has a web server 1511 that provides a web page displaying search results to the terminal 4. The reception unit 1512 receives various requests, such as search requests, page transition requests, and download requests, from the web browser of the terminal 4 via the web server 1511. The search unit 1513 searches for and extracts images from among multiple images stored in the memory unit that satisfy specific coverage conditions. These specific conditions may be part of the search request. The page creation unit 1514, the web server, and the display device of the terminal 4 function as output units that output the images extracted by the search unit 1513 that satisfy the specific coverage conditions. This provides an image provision system that can meet the further needs of the user.

[0095] The sorting unit 1515 functions as a sorting unit that rearranges multiple images extracted by the search unit according to the coverage rate. The page creation unit 1514, the web server 1511, and the display device of the terminal 4 may function as output units that output multiple images rearranged according to the coverage rate. As a result, multiple images are rearranged according to the coverage rate of a predetermined covering material.

[0096] [Perspective 2] The sorting unit 1515 may sort the multiple images in ascending or descending order based on the coverage rate obtained for each of the multiple images.

[0097] [Perspective 3] The reception unit 1512 or user interface 1200 functions as a condition reception unit that accepts sorting conditions (e.g., sorting rules). The search unit 1513 may sort multiple images whose coverage rate meets specific conditions according to the conditions accepted by the condition reception unit. This makes it possible to sort the search results in the order desired by the user.

[0098] The sorting unit 1515 may sort the multiple images in ascending or descending order based on the coverage rate of a first cover and the coverage rate of a second cover, which are obtained for each of the multiple images. In sorting, the coverage rate of the first cover may take precedence over the coverage rate of the second cover. In Figure 13, the cloud coverage rate (ascending order) takes precedence over the field coverage rate (descending order). Therefore, when two satellite images with the same cloud coverage rate are found, these two satellite images are sorted in descending order according to the field coverage rate.

[0099] [Perspective 4] The reception unit 1512 or user interface 1200 may function as a type reception unit that accepts the type of covering. The search unit 1513 may search for and extract images in which the coverage rate of a specific type of covering accepted by the type reception unit meets specific conditions. This will allow the user to obtain search results for a specific type of covering they are interested in.

[0100] [Perspective 5] The reception unit 1512 or the user interface 1200 may display two or more types of cover extracted from multiple satellite images stored in the memory unit as selection candidates. The reception unit 1512 or the user interface 1200 may accept the type of cover from the displayed selection candidates. As explained in relation to Figure 13, when clouds and fields are extracted as types of cover in the search area 1203, clouds and fields are displayed as selection candidates. The user may select either one of clouds or fields, or both, through the reception unit 1512 or the user interface 1200. The reception unit 1512 may also display a list of selection candidates and display the type of cover selected by the user from that list on the user interface 1200.

[0101] [Perspective 6] As illustrated in Figure 3, the memory unit may store multiple images, each associated with a date of capture. The search unit 1513 may search for and extract images that were taken of the ground surface on a date or period specified as a search criterion, and whose coverage rate satisfies specific conditions. This would make it possible to acquire images focusing on a specific time period. For example, it would be possible to estimate the yield of grain from the coverage rate of grain during the grain harvest season.

[0102] [perspective 7] Specific conditions (e.g., search criteria) may include the coverage rate being above or below a predetermined threshold. This would make it possible, for example, to obtain images of the ground surface on a clear day with low cloud cover.

[0103] [Perspective 8] Coverings may include, for example, clouds, seas, lakes, salt lakes, fields, mountainous areas, forests, deserts, rice paddies, rivers, and snow-capped mountains. However, these are only examples of coverings, and other coverings may be designated as objects of classification and search.

[0104] [Perspective 9] As illustrated in Figure 3, the memory unit may store location information (e.g., latitude and longitude) representing the location of the ground surface captured in each image. The search unit 1513 may search for and extract images where the coverage rate of the ground surface at a specific location specified by the user (e.g., latitude and longitude included in the search area 1203) satisfies specific conditions. This will allow the user to extract images of the specific location they desire.

[0105] [Perspective 10] As illustrated in Figure 4, the feature database 147 is an example of a storage unit that holds the names of predetermined shooting targets and the location information of those shooting targets. A specific location specified by the user may be the location information of a shooting target held in the storage unit. For example, a specific target name such as "Tokyo Tower" or "Senso-ji Temple" may be entered into the keyword input unit 1400. The search unit 1513 may search for images based on the location indicated by the location information associated with the name of a specific shooting target specified by the user. For example, the search unit 1513 may determine the latitude and longitude of a specific target name by referring to the feature database 147 based on the specific target name. This will make it possible to easily extract images near a specific target.

[0106] [Perspectives 11, 12] The search unit 1513 may search and extract images from among multiple images in which the coverage rate of the covering material satisfies specific conditions, and in which the number of objects specified by the user satisfies predetermined conditions. The objects may be aircraft, vehicles, or ships. This would make it possible to acquire images focusing on the number of specific objects. For example, it would be possible to acquire images focusing on the number of ships in a particular canal. It would also be possible to easily count the number of vehicles operating in rural areas where traffic volume measuring equipment is not installed.

[0107] [Perspective 13] The flying object may be a satellite, an aircraft, or a drone. In other words, the image may be not only satellite imagery, but also aerial photographs taken by an aircraft or drone.

[0108] [Perspective 14] According to this embodiment, the acquisition step involves acquiring an image obtained by imaging the Earth's surface from a flying object flying within or outside the atmosphere, A storage step involves storing multiple images acquired by the acquisition step, each from a different region or at a different time of capture, in a storage unit. A determination step is made for determining the coverage rate of the covering material covering the ground surface that is visible in each of the multiple images stored in the memory unit. A search step of searching for and extracting images from among the plurality of images stored in the memory unit in which the coverage rate satisfies a specific condition, A sorting step in which a plurality of images extracted in the search step are sorted according to the coverage rate, An image providing method is provided, comprising an output step of outputting the plurality of images rearranged according to the coverage ratio.

[0109] [Perspective 15] Program P includes an acquisition process to obtain images obtained by imaging the Earth's surface from a flying object in or outside the atmosphere, A storage step involves storing multiple images acquired by the acquisition step, each from a different region or at a different time of capture, in a storage unit. A determination step is made for determining the coverage rate of the covering material covering the ground surface that is visible in each of the multiple images stored in the memory unit. A search step of searching for and extracting images from among the plurality of images stored in the memory unit in which the coverage rate satisfies a specific condition, A sorting step in which a plurality of images extracted in the search step are sorted according to the coverage rate, This is an example of a program that causes a computer to perform an output step of outputting the plurality of images that have been rearranged according to the coverage ratio.

[0110] [Perspective 16] According to this embodiment, an acquisition unit acquires an image obtained by imaging the Earth's surface from a flying object flying within or outside the atmosphere, A determination unit determines the coverage rate of covering materials on the ground surface captured in each image, for multiple images acquired by the acquisition unit, each from a different region or at a different time of capture. A sorting unit that sorts multiple images from the aforementioned multiple images in order of the coverage rate which satisfies a specific condition, An information processing device is provided, which includes an output unit that outputs the plurality of images rearranged according to the coverage ratio.

[0111] [Perspective 17] The object detector 144 and counter 1503 function as determination units that determine the number of objects in each of the multiple images stored in the memory unit. The search unit 1513 may search for and extract images from the multiple images stored in the memory unit whose number satisfies a specific condition. The page creation unit 1514, web server 1511, and terminal 4 may function as output units that output images extracted by the search unit whose number satisfies a specific condition.

[0112] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0113] 1: Information processing unit, 11: Control unit, 12: Main memory unit, 14: Auxiliary memory unit

Claims

1. An acquisition unit that acquires a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a determination unit that determines, for each image acquired by the acquisition unit, the type of object that covers the surface of the Earth shown in the image and its coverage rate; a storage unit that stores the type of the covering object and its covering rate determined by the determination unit together with the image in association with the image; a search unit that searches for and extracts images, among the plurality of images stored in the storage unit, in which the type of the covering object and its coverage rate satisfy predetermined search conditions; a sorting unit that sorts the images extracted by the search unit according to the coverage rate; an output unit that outputs the plurality of images sorted according to the coverage rate; An image providing system having the above configuration.

2. The image providing system according to claim 1 , wherein the sorting unit sorts the images in ascending or descending order based on the coverage rate calculated for each of the images.

3. 3. The image providing system according to claim 1, wherein the sorting unit sorts the images in ascending or descending order based on the coverage rate of a first coating and the coverage rate of a second coating calculated for each of the images, and in the sorting, the coverage rate of the first coating is given priority over the coverage rate of the second coating.

4. the storage unit stores the plurality of images in association with their respective shooting dates; 4. The image providing system according to claim 1, wherein the search unit searches for and extracts images that were captured on a specified shooting date or shooting period and whose type and coverage rate of the covering object satisfy the search conditions.

5. The image providing system according to claim 1 , wherein the covered object includes any one of a cloud, an ocean, a lake, a salt lake, a field, a mountainous region, a forest, a desert, a rice paddy, a river, and a snow-capped mountain.

6. the storage unit stores location information representing a geographical position on the surface of the Earth captured in each image in association with the image; The image providing system according to claim 1 , wherein the search unit searches for and extracts images in which the type of the covering object and its coverage rate satisfy the search conditions in a specified geographical location.

7. The determination unit further determines the number of objects appearing in the image, The image providing system according to claim 1 , wherein the search unit searches for and extracts images in which the number of the objects satisfies a predetermined condition from among a plurality of images in which the type of the covering object and its coverage rate satisfy the search condition.

8. The image providing system of claim 7 , wherein the object is one of an aircraft, a vehicle, or a watercraft.

9. The image providing system according to claim 1 , wherein the flying object is any one of an artificial satellite, an aircraft, and a drone.

10. An acquisition unit that acquires a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a detection unit that detects the type and number of objects appearing in each image acquired by the acquisition unit; a storage unit that stores the type and the number of the objects detected by the detection unit together with the image in association with the image; a search unit that searches for and extracts images whose detection results satisfy predetermined search conditions from among the plurality of images stored in the storage unit; a sorting unit that sorts the images extracted by the search unit according to a result of the detection; an output unit that outputs the plurality of images sorted according to a result of the detection; An image providing system having the above configuration.

11. The image providing system of claim 10, wherein the sorting unit sorts the multiple images in ascending or descending order based on the object detection results obtained for each of the multiple images.

12. An acquisition step of acquiring a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a determination step of determining, for each image acquired by the acquisition step, the type of object covering the surface of the Earth shown in the image and its coverage rate; a storage step of storing the type of the coating material and its coating rate determined in the determination step in a storage unit together with the image in association with the image; a search step of searching for and extracting images, among the plurality of images stored in the storage unit, whose types of the covering object and their coverage rates satisfy predetermined search conditions; a sorting step of sorting the plurality of images extracted in the search step according to the coverage rate; an output step of outputting the plurality of images sorted according to the coverage rate; An image providing method for an image providing system comprising the steps of:

13. An acquisition step of acquiring a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a determination step of determining, for each image acquired by the acquisition step, the type of object covering the surface of the Earth shown in the image and its coverage rate; a storage step of storing the type of the coating material and its coating rate determined in the determination step in a storage unit together with the image in association with the image; a search step of searching for and extracting images, among the plurality of images stored in the storage unit, whose types of the covering object and their coverage rates satisfy predetermined search conditions; a sorting step of sorting the plurality of images extracted in the search step according to the coverage rate; an output step of outputting the plurality of images sorted according to the coverage rate; A program that causes a computer to execute the following.

14. An acquisition unit that acquires a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a determination unit that determines, for each image acquired by the acquisition unit, the type of object that covers the surface of the Earth shown in the image and its coverage rate; a storage unit that stores the type of the covering object and its covering rate determined by the determination unit together with the image in association with the image; a sorting unit that sorts images, among the plurality of images stored in the storage unit, in which the type of the covering object and the covering rate thereof satisfy a predetermined search condition according to the covering rate; an output unit that outputs the plurality of images sorted according to the coverage rate; An information processing device having the above configuration.

15. An acquisition step of acquiring a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a detection step of detecting the type and number of objects appearing in each image acquired by the acquisition step; a storage step of storing the type and the number of the objects detected by the detection step in a storage unit together with the image in association with the image; a search step of searching and extracting images, the detection results of which satisfy a predetermined search condition, from among the plurality of images stored in the storage unit; a sorting step of sorting the plurality of images extracted in the search step according to a result of the detection; an output step of outputting the plurality of images sorted according to a result of the detection; An image providing method for an image providing system comprising the steps of:

16. An acquisition step of acquiring a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a detection step of detecting the type and number of objects appearing in each image acquired by the acquisition step; a storage step of storing the type and the number of the objects detected by the detection step in a storage unit together with the image in association with the image; a search step of searching and extracting images, the detection results of which satisfy a predetermined search condition, from among the plurality of images stored in the storage unit; a sorting step of sorting the plurality of images extracted by the search step in accordance with a result of the detection; and an output step of outputting the plurality of images rearranged in accordance with a result of the detection.

17. An acquisition unit that acquires a plurality of images obtained by imaging the surface of the Earth from an airborne vehicle; a detection unit that detects the type and number of objects appearing in each image acquired by the acquisition unit; a storage unit that stores the type and the number of the objects determined by the detection unit together with the image in association with the image; a sorting unit that sorts images, among the plurality of images stored in the storage unit, whose results of the detection satisfy a predetermined search condition in accordance with the results of the detection; an output unit that outputs the plurality of images sorted according to a result of the detection; An information processing device having the above configuration.

Images